AI Systems

Threat analytics is time series work too

4 min read

Threat analytics is time series work more often than teams admit.

Security data is full of categories: user, host, process, IP, domain, rule, severity, tactic, technique. Those labels matter. But a lot of the signal lives in time: how often something happens, when it happens, how fast it changes, what usually comes before it, and whether the current rhythm belongs in the environment.

the timestamp finishes the sentence

A single login failure is boring. A burst of failures against dormant accounts at 2:13 AM is different. A PowerShell command is boring in some places and strange in others. A new cloud region may be normal during a deployment window and suspicious on a quiet weekend.

The event is only half the sentence.

local baselines beat one big baseline

The first mistake in time-based security analytics is treating the whole environment as one population.

Different users have different rhythms. So do service accounts, build agents, laptops, domain controllers, Kubernetes nodes, and cloud workloads. A backup job that moves gigabytes every night is normal. The same transfer pattern from an executive laptop is not.

I would rather start with local baselines:

entity: user:jlondon
feature: failed_login_count
window: 15 minutes
baseline: median by hour-of-week over 30 days
current: 18
expected: 0-2

The hour-of-week part matters. Monday morning is not Saturday night.

the window is the product decision

Time windows are product decisions disguised as math.

Count failed logins over one minute and you catch bursts. Count them over one day and you catch slow guessing. Count distinct countries over seven days and you may find travel or credential sharing. Count process launches over five seconds and you may find script behavior that disappears in hourly aggregation.

window=1m   failed_logins=4    likely noise
window=15m  failed_logins=39   spray pattern
window=24h  failed_logins=41   one burst, not sustained

That is why I dislike analytics that hide the window. An alert should say what shape it saw.

rhythms change and drift matters

Security systems inherit business rhythms.

People log in during work hours. CI systems run on merge patterns. Finance systems spike near close. Retail systems move differently during holidays. Developers do strange things late at night before a launch.

Seasonality does not mean an event is safe. It means the model needs context before it calls the event weird.

Patterns also change. Teams adopt a new VPN. Companies migrate identity providers. Deployments move regions. Help desk scripts change process names. Attackers also create drift by moving slowly enough that the new baseline absorbs them.

That makes drift review important.

feature: outbound_bytes_to_new_domains
entity_group: engineering_laptops
baseline_period: previous_30_days
candidate_period: last_7_days
change: +340%
annotation: new telemetry exporter rollout began 2025-05-01

The annotation is doing real work.

sequences matter more than single alerts

Some security stories are sequences.

A suspicious login followed by a password reset followed by mailbox rule creation is different from any one of those events alone. A container image pull followed by a new outbound connection followed by a secret read deserves a different score than the same events spread across unrelated hosts over several weeks.

login_success(new_device)
  -> privilege_check
  -> mailbox_rule_created
  within 20 minutes

That is not forecasting. It is temporal pattern matching.

late data changes the design

Threat analytics systems live with late data.

Logs arrive out of order. Endpoint agents buffer events. Cloud APIs lag. Network sensors drop packets. A batch job lands after the alert already fired.

If the system assumes perfect ordering, the analytic will be brittle. A better design records event time and ingest time separately.

{
  "event_time": "2025-05-07T03:14:22Z",
  "ingest_time": "2025-05-07T03:18:41Z",
  "source": "endpoint",
  "entity": "host:build-17",
  "event": "process_start"
}

That distinction matters for investigation and for training data.

what the team should measure

Accuracy is not enough for security analytics.

A detection that finds credential abuse six hours late may be technically correct and operationally weak. A detection that fires in one minute with no context may be fast and useless.

I would track:

  • detection latency from event time
  • ingest latency by source
  • alert volume by hour
  • analyst reopen rate
  • false positives by entity group
  • missed incidents promoted into regression tests
  • drift annotations added per week

Those metrics tell you whether the analytic fits the way the security team works.

The useful question is rarely “did this event happen?” It is closer to “does this event belong here, at this time, after these other events, for this entity, given what normally happens?”

That is a time-series question with security consequences.

Jeremy London

About Jeremy London

Engineering leader and builder in Denver. I write about AI platforms, agents, security, reliability, homelab infrastructure, and the parts of engineering work that have to survive production.