Agent observability needs user-level events

The HTTP request returned 200. The queue job completed. The tool schema validated. The user is still looking at the wrong draft.

That is the observability problem with agents. The system can be healthy at the infrastructure layer while the product behavior is wrong.

I want a run record that lets somebody answer the question a support person would actually ask: what did the agent believe it was doing, and which event made that belief wrong?

The run is the useful unit. It should connect the request to the selected context, route, tool calls, approvals, artifacts, checks, and final state. The exact storage model can vary. The relationship cannot.

{
  "runId": "run_74",
  "request": "update the active draft",
  "context": ["drafts/current.mdx"],
  "tools": ["read_file", "write_file", "markdown_check"],
  "approvals": [],
  "artifacts": ["src/content/blog/example.mdx"],
  "verification": { "markdown_check": "passed" },
  "state": "completed"
}

The interesting fields are the ones normal traces tend to omit. Which file did the agent treat as active? What context snapshot did it use? Did an approval cover the write that actually happened? Did the check run against the changed file or merely exit successfully somewhere nearby?

I would keep the technical trace beside product events rather than trying to replace one with the other. Latency, tokens, cache hits, queue time, and tool errors explain how the machine moved. Events such as wrong_artifact_selected, stale_context_used, approval_mismatch, and rollback_unavailable explain why the user experienced the run as a failure.

That vocabulary changes incident review. “Tool call failed” is a useful machine event. “Agent could not update the selected draft” is the event the product owner can act on.

Approval needs the same treatment. Store what the user saw, not only that a button was clicked. If the approval said “edit one file” and the agent changed five, the log should make the mismatch obvious.

The final test is replay. An engineer should be able to start from “the agent edited the wrong draft” and find the context selection, model call, tool arguments, approval state, and verification result that led there.

Keep the spans. Add the story.