Evals should have come before agent hype

The agent hype cycle got ahead of the measurement habit.

That is what bothered me then, and it is still the thing I would fix first. Demos had browser clicks, file edits, plans, retries, and a final answer that looked like work happened. The missing piece was whether the system actually got better at the task or just looked busier on the screen.

motion is cheap

An agent can do a lot and still fail the job.

It can open the right file and edit the wrong section. It can call the right API with a stale ID. It can run tests and ignore the failure. It can browse three pages and cite the least relevant one. It can ask for approval at the exact moment when approval is least useful.

That is why evals need to inspect the run, not just the final answer. For code work, the expected end state might be a passing test, a specific diff, no unrelated edits, and no generated junk left behind. For browser work, it might be a submitted form with the right fields and no wandering into disallowed domains. For research, it might be primary sources and no unsupported claims. For workflow agents, it might be the right ticket state plus a useful audit comment.

If the eval only grades the final text, it misses the part that actually matters.

keep a run log that people can read

Before fancy benchmarks, I want a boring run log.

Not a pasted transcript. A structured record of what the agent attempted:

type AgentRunEvent =
  | { type: "observe"; at: string; source: string; summary: string }
  | { type: "plan"; at: string; steps: string[] }
  | { type: "tool_call"; at: string; tool: string; inputHash: string }
  | { type: "tool_result"; at: string; tool: string; ok: boolean; outputHash: string }
  | { type: "approval"; at: string; scope: string; decision: "approved" | "denied" }
  | { type: "final"; at: string; answer: string }

The hashes are there because logs should not casually store secrets or whole file contents. The point is to reconstruct the path without trusting the agent’s self-report. Once you have that, you can ask better questions: did it observe the right thing, choose the right tool, handle failure, and stop with the system in the state you expected?

For agents, eval data is operational data.

the ugly cases teach faster

Early eval sets should be small and mean.

I do not mean impossible. I mean full of the awkward cases that break demos:

• the obvious file name is wrong
• the first search result is stale
• a command succeeds with a warning that matters
• two tools can satisfy the request but one violates scope
• the user asks for a broad change and one subtask is unsafe
• the browser page loads slowly
• the answer is present but buried in a collapsed section
• the test suite passes only because the changed path is uncovered

Those cases teach faster than a pile of happy paths. Happy paths prove the loop can move. Ugly cases prove whether it can recover.

The eval should include the reason the case exists. Otherwise future maintainers will delete the weird examples because they look random. I would rather have a slightly embarrassing suite that remembers the real failures than a clean one that forgot why it was built.

{
  "id": "repo-edit-stale-test-output",
  "task": "Fix the failing parser test",
  "trap": "The first test command prints cached output unless run with --no-cache.",
  "expected": "Agent reruns the test with cache disabled before claiming success."
}

That trap field is not for the agent. It is for the humans maintaining the suite. It explains what the case is protecting.

autonomy needs a scorecard

People talk about autonomy like it is a single dial. It is not.

An agent might be free to navigate but need approval before mutation. It might choose tools freely inside a narrow sandbox. It might draft a plan but wait before execution. It might retry failures automatically but escalate permission changes. Each of those choices has a different risk shape.

The scorecard should reflect that. For each task type, I would track:

• completion rate
• unsafe action attempts
• unnecessary approval requests
• missed approval opportunities
• tool-call error rate
• recovery after failed tool calls
• unrelated file or state changes
• time and cost per successful task

Completion rate by itself rewards reckless agents. A system that completes more tasks by touching unrelated files is not better. A system that avoids every risky action by asking for approval constantly may be safer, but it is not useful. The eval has to show those tradeoffs clearly.

This is where agent evals differ from ordinary model evals. The output matters, but the path matters too because the path can damage the environment.

approval belongs in the eval

Approval flows are part of agent quality. They are not only a safety feature.

An agent can fail by asking too late, asking too often, asking without enough context, or asking for the wrong scope. A useful eval should inspect the approval moment:

• what action was the agent about to take?
• did the request describe the effect clearly?
• was the scope narrow enough?
• did the agent continue correctly after approval?
• did denial stop the action?

That matters because a bad approval request trains the user to click through. “Approve tool call” is not enough. The user needs to know whether the agent is about to read a file, edit a file, submit a form, install a dependency, delete state, or contact an external service. If the approval text is too vague to evaluate, it is too vague to show.

the harness should feel boring

The right time to build the eval harness is when the agent can barely do the task.

At that stage, the cases are small. The event log is simple. The allowed tools are few. The team still remembers why each failure matters. Adding evals then feels premature, but waiting makes the job worse. Once the agent has ten tools, long context, browser state, memory, retries, approvals, and background jobs, every eval decision becomes political.

Start with five tasks. Record the run. Grade the end state. Add the first ugly failure. Keep going.

That would have made the agent hype cycle healthier. The demos still would have been exciting. They should be. Watching a system complete real work is strange and fun. But the next question should have been waiting: did it get better, or did it only move more?