Evals should have come before agent hype

Agent demos got impressive before agent measurement got serious.

That mismatch explains a lot of the weirdness around early agent products. The demos had motion: browser clicks, file edits, tool calls, plans, retries, summaries, and a final answer that looked like work happened. The measurement usually lagged behind. Did the agent complete the task? Did it use the right source? Did it skip a required step? Did it leave a mess? Did it succeed because the task was easy or because the system was actually improving?

Those questions should have been first-class from the beginning.

I understand why they were not. Demos are fast feedback. Evals feel slow when the basic loop barely works. When an agent finally navigates a page, edits a file, or calls a tool without falling over, the temptation is to add more capability. The annoying thing is that every new capability makes the system harder to evaluate later.

The harness should come early, before the agent becomes interesting enough to fool you.

motion is not task completion

An agent can do a lot and still fail.

It can open the right file and edit the wrong section. It can call the right API with a stale ID. It can run tests and ignore the failing output. It can write a confident final answer while leaving the working tree dirty. It can browse three pages and cite the least relevant one. It can ask for approval at the moment where approval is least useful.

This is why agent evals need to look at the run, not only the final answer.

A task-completion eval should know the expected end state. For code work, that might be a passing test, a specific file diff, no unrelated edits, and no generated artifacts left behind. For browser work, it might be a submitted form with expected fields and no navigation to disallowed domains. For research, it might be citations from primary sources and no unsupported claims. For a workflow tool, it might be the right ticket state plus an audit comment.

If the eval only grades the final message, it misses the work.

the first artifact should be a run log

Before fancy agent benchmarks, I want a boring run log.

Not a transcript pasted into a debug panel. A structured record of what the agent attempted:

type AgentRunEvent =
  | { type: "observe"; at: string; source: string; summary: string }
  | { type: "plan"; at: string; steps: string[] }
  | { type: "tool_call"; at: string; tool: string; inputHash: string }
  | { type: "tool_result"; at: string; tool: string; ok: boolean; outputHash: string }
  | { type: "approval"; at: string; scope: string; decision: "approved" | "denied" }
  | { type: "final"; at: string; answer: string }

The hashes are there because logs should not casually store secrets or full file contents. The event shape gives the evaluator a way to reconstruct the path without trusting the agent’s self-report.

That run log changes the conversation. Instead of asking whether the final answer sounded good, you can ask whether the agent observed the right thing, selected the right tool, handled failure, and stopped with the system in the expected state.

For agents, eval data is operational data.

simple tasks need ugly cases

Early agent eval sets should be small and mean.

I do not mean impossible. I mean full of the kind of awkward cases that break demos:

• the obvious file name is wrong
• the first search result is stale
• a command succeeds with a warning that matters
• two tools can satisfy the request but one violates scope
• the user asks for a broad change and one subtask is unsafe
• the browser page loads slowly
• the answer is present but inside a collapsed section
• the test suite passes only because the changed code path is not covered

Those cases teach the team faster than twenty happy paths. Happy paths prove the loop can move. Ugly cases prove whether the loop can recover.

The eval should include the reason the case exists. Otherwise future maintainers will delete the weird examples because they look arbitrary.

{
  "id": "repo-edit-stale-test-output",
  "task": "Fix the failing parser test",
  "trap": "The first test command prints cached output unless run with --no-cache.",
  "expected": "Agent reruns the test with cache disabled before claiming success."
}

That trap field is not for the agent. It is for the evaluator and the humans maintaining the suite. It explains which behavior the case protects.

autonomy should have a scorecard

People talk about agent autonomy as if it is a single dial. It is not.

An agent might be autonomous in navigation but require approval before mutation. It might choose tools freely inside a narrow sandbox. It might draft a plan but wait before execution. It might retry failures automatically but escalate permission changes. Each of those choices has a different risk shape.

The scorecard should reflect that. For each task type, I would track:

• completion rate
• unsafe action attempts
• unnecessary approval requests
• missed approval opportunities
• tool-call error rate
• recovery after failed tool calls
• unrelated file or state changes
• time and cost per successful task

Completion rate alone rewards reckless agents. A system that completes more tasks by touching unrelated files is not better. A system that avoids every risky action by asking for approval constantly may be safer but useless. The eval has to make those tradeoffs visible.

This is where agent evals differ from ordinary model evals. The model output is only one part of the result. The path matters because the path can damage the environment.

regression means behavior changed

Agent regressions are often behavioral, not textual.

A model upgrade might produce final answers that read better while planning worse. A new tool description might reduce tool-call errors but increase unnecessary calls. A stricter approval policy might stop risky edits and also prevent harmless cleanup. A longer context window might help research tasks and hurt file-edit tasks because the agent carries too much stale context forward.

The eval suite should compare runs at the behavior level:

case: repo-edit-stale-test-output
baseline:
  completed: true
  reran_tests: true
  unrelated_files_changed: 0
candidate:
  completed: true
  reran_tests: false
  unrelated_files_changed: 0
decision: regression

Both runs may have a nice final answer. Only one respected the task.

This is why I do not trust demo-based agent development for long. Demos compress the run into a story. Evals preserve the parts of the run that contradict the story.

approval is an eval dimension

Approval flows are part of agent quality. They are not only a safety feature.

An agent can fail by asking too late, asking too often, asking without enough context, or asking for the wrong scope. A useful eval should inspect the approval moment:

• What action was the agent about to take?
• Did the request describe the effect clearly?
• Was the scope narrow enough?
• Did the agent continue correctly after approval?
• Did denial stop the action?

That matters because a bad approval request trains the user to click through. “Approve tool call” is not enough. The user needs to know whether the agent is about to read a file, edit a file, submit a form, install a dependency, delete state, or contact an external service.

If the approval text is too vague to evaluate, it is too vague to show.

the harness should be boring before the demo is amazing

The right time to build the eval harness is when the agent can barely do the task.

At that stage, the cases are small. The event log is simple. The allowed tools are few. The team still remembers why each failure matters. Adding evals then feels premature, but waiting makes the job worse. Once the agent has ten tools, long context, browser state, memory, retries, approvals, and background jobs, every eval decision becomes political.

Start with five tasks. Record the run. Grade the end state. Add the first ugly failure. Keep going.

That would have made the agent hype cycle healthier. The demos still would have been exciting. They should be exciting. Watching a system complete real work is strange and fun. But the next question should have been waiting:

Did it get better, or did it only move more?