Cost Anomaly Detection

ZopNight detects unusual cost spikes across seven dimensions using a dual-method algorithm (percentage deviation + z-score). Each anomaly includes a severity level, root cause analysis, and affected resource breakdown. Detection runs as a scheduled cron job.

Detection dimensions

Anomalies are detected independently across seven dimensions. Each dimension has its own detector. You subscribe to alerts via Notifications.

Detection algorithm

ZopNight uses a dual-method approach on a 7-day rolling average baseline:

1. Primary — percentage deviation: compares today's cost against the 7-day rolling average. Requires at least 4 days of data.
2. Secondary — z-score: measures how many standard deviations today's cost is above the mean (positive spikes only — cost drops are not flagged). Requires at least 14 days of data and only upgrades severity (never downgrades). Only applied when the standard deviation exceeds 10% of the mean to avoid false escalation on stable costs.

Dimensions averaging less than $1/day are skipped to reduce noise.

Severity levels

Root cause analysis

Each anomaly is enriched with a root cause derived from a 14-day lookback over cost records and activity events. Root cause checks run against an in-memory index (no per-resource database queries) and include:

For org-level anomalies, additional checks detect new account onboarding (60%+ of the spike from new resources) and resource type spikes (e.g. "50 EC2 instances each +$5/day").

Resource group and team dimensions use a cascading root cause strategy: membership diff first, then group-level drill-down, then per-resource fallback — with early exit when a step explains the spike.

Notification cooldown & dedup

• 7-day notification cooldown — once an anomaly key (org / type / key) is notified, re-notification is suppressed for 7 days. The rolling-average baseline absorbs the new cost level during that window so a single resize event doesn't page you for a week straight.
• Duplicate-record dedup — when the same key produces another anomaly within 7 days at similar cost (<20% delta), the new anomaly is dropped before storage. The first occurrence is kept; subsequent rows are the same event repeating. A cost increase of 20% or more is treated as a genuinely new event.
• New-resource suppression — anomalies triggered primarily by resources first seen within 3 days are marked suppressed and not notified. They still appear in the UI as info severity dots.
• Team redistribution filter — when team costs move but the org's net change across all teams is under 20% of the largest single-team move, the team anomalies are dropped. This is cost moving between teams (for example, a retag), not new spend.
• Resource cap — at most 10 resource-level anomalies are kept per org per run, the ones with the highest deviation.

Notification event types

Anomaly notifications use 15 event types (3 severities × 5 dimensions), each independently subscribable via Notifications. The Azure dimensions have no event types of their own — they reuse the cloud account and resource group ones:

cost.anomaly.warning              # org-level
cost.anomaly.critical
cost.anomaly.emergency

cost.anomaly.cloud_account.warning # per cloud account
cost.anomaly.cloud_account.critical
cost.anomaly.cloud_account.emergency

cost.anomaly.resource_group.warning # per resource group
cost.anomaly.resource_group.critical
cost.anomaly.resource_group.emergency

cost.anomaly.resource.warning      # per resource
cost.anomaly.resource.critical
cost.anomaly.resource.emergency

cost.anomaly.team.warning          # per team
cost.anomaly.team.critical
cost.anomaly.team.emergency

Subscriptions support scope filtering — subscribe org-wide or to specific entities (a particular cloud account, resource group, or team).

Two-level anomaly drawer

Anomalies appear on cost trend charts as pulsing severity-coloured dots (daily view only). Clicking a dot or the dismissible banner opens a two-level drawer:

• Level 1 — list: anomaly cards with severity badge, date, summary text, expected → actual cost change, and filter tabs (all / emergency / critical / warning / info).
• Level 2 — detail: stat cards (Actual / Expected / Deviation %), parsed insight bullets, a resource-type breakdown table, and an affected-resources table with expandable rows showing per-resource root cause. Resource UIDs are copiable.

Detail opens stacked over the list — closing detail reveals the list, so you can walk a triage workflow without losing context. For a single anomaly, the detail panel opens directly.

API

{
  "data": {
    "items": [
      {
        "id": "a1b2c3d4-...",
        "orgId": "org_001",
        "date": "2026-04-25",
        "anomalyType": "resource",
        "anomalyKey": "i-0abc123def456",
        "severity": "warning",
        "method": "percent",
        "actualCostUsd": 28.90,
        "expectedCostUsd": 12.40,
        "deviationPercent": 133.06,
        "rootCause": "Insights:\n- Hourly rate increased from $0.42 to $0.97\n...",
        "suppressed": false,
        "notified": true,
        "notifiedAt": "2026-04-25T21:00:00Z",
        "createdAt": "2026-04-25T20:55:12Z"
      }
    ],
    "total": 12,
    "hasMore": true,
    "page": 1,
    "limit": 20
  }
}

Anomalies also appear inline on trend data points via GET /reports/trends — each trend item may include an anomaly object with severity, actualCostUsd, expectedCostUsd, deviationPercent, topResources (array of contributing resources), and rootCause (enriched insight text).

Detection schedule

Anomaly detection runs once a day as a cron job at 20:55 UTC. It analyses the most recent date that has cost records.

When the deployment runs the durable jobs mechanism, the anomaly_detection job replaces this cron. It screens each org one day at a time, an hour after the day ends. It waits for that day's cost rollup and billing sync to finish first.

See Reports for the surrounding cost views and Notifications for how to subscribe to anomaly alerts.