Queues make AI workflows calmer

Queues give AI workflows an honest place to put slow work.

A request that performs retrieval, model calls, tool execution, retries, file processing, verifier passes, or human review has already outgrown a single patience window. Short classification or routing decisions can remain inline. The longer path needs a durable boundary.

That boundary starts with identity.

{
  "jobId": "summarize-ticket-1842",
  "idempotencyKey": "ticket-1842:v3",
  "attempt": 2,
  "visibleAfter": "2026-07-13T12:10:00Z",
  "deadLetterAfter": 5
}

The idempotency key is the important field here. A model call can succeed just before a worker crashes. A tool can finish while the network connection disappears. A user can press the button twice. A visibility timeout can cause the queue to deliver the same job again. Without a stable identity, each event looks like new work.

Large inputs also deserve a durable home outside the queue message. Store the document, image, prompt bundle, or repository snapshot, then let the job reference a specific version. Retries should read the same input rather than whatever happened to remain in memory.

retries should explain themselves

“Retry on failure” collapses several different operational decisions into one.

A provider timeout or network error may be transient. A malformed schema, missing permission, deleted source, or deterministic verifier rejection will not be fixed by repeating the same attempt. The queue should record that distinction.

type JobFailure =
  | { kind: "transient"; retryAfterMs: number; reason: string }
  | { kind: "input"; reason: string }
  | { kind: "permission"; reason: string }
  | { kind: "policy"; reason: string }
  | { kind: "verifier"; reason: string }
  | { kind: "bug"; reason: string }

dead letters are a review queue

A dead-letter queue is evidence that the workflow reached a case it could not classify or recover from. Treating it as a failure closet throws away information. Treating it as a review queue creates a place to inspect the input, the attempts, and the missing decision.

backpressure is a feature

“Not yet” is a useful system response. A queue can keep agent workflows from competing all at once for model capacity, database connections, reviewer attention, and budget.

Queues do not make AI work simple. They make the slow, expensive, failure-prone parts legible: a job has an identity, an input version, an attempt history, a failure reason, and a place to wait. That is a calmer shape for software to operate.