Synthetic data is a practice room

Synthetic data is useful for evals when it acts like a practice room.

I use it when I need to rehearse cases that are rare, expensive, sensitive, or just annoying to collect. Fraud systems need weird account behavior before the weird behavior shows up in production. Tool-calling systems need malformed tool results and ambiguous intent before those failures become common. Extraction systems need invoices with missing totals or inconsistent currency before the model sees them in the wild.

The mistake is treating synthetic data like proof. It is closer to a sparring partner.

write the behavior first

Before I generate anything, I write the behavior down:

behavior: refund_policy_exception_handling
system_under_test: support_response_agent
expected_capability:
  - identify the relevant policy clause
  - detect whether an exception applies
  - ask for missing order information when needed
must_not:
  - promise a refund without evidence
  - cite a policy that was not retrieved
  - ignore region-specific rules

That contract keeps the dataset from becoming a bucket of plausible text.

provenance matters more than volume

Every synthetic record should say where it came from, which generator created it, which prompt produced it, who reviewed it, and whether it belongs in training, evaluation, or only exploratory analysis.

{
  "id": "refund-exception-syn-0042",
  "behavior": "refund_policy_exception_handling",
  "source_basis": ["policy/refunds-v3.md", "support/failure-17.md"],
  "generator_model": "teacher-model-2025-11",
  "generator_prompt": "synthetic-refund-cases:v6",
  "label_source": "human_review",
  "split": "eval_synthetic",
  "allowed_uses": ["regression_eval", "debugging"],
  "review_notes": "Customer is in a region with a manual override path."
}

Without provenance, synthetic data becomes hard to retire. Six months later the team sees a passing eval and forgets that half the examples came from an old policy, an old prompt, and an old failure taxonomy.

use it to fill gaps

I would start by looking for gaps, not by asking a model for ten thousand examples.

Maybe the real dataset has plenty of ordinary refunds but almost no policy exceptions. Maybe the retrieval system has English examples but almost none from mixed-language snippets. Maybe the tool caller has clean successes but few partial failures. Maybe the classifier has plenty of positive and negative labels but very few ambiguous examples where reviewers disagree.

That is what synthetic data is good at:

• rare policy combinations
• hard negatives that look almost correct
• old bugs that need regression coverage
• adversarial phrasing around a known boundary
• long-tail entity formats
• corrupted or incomplete inputs
• tool outputs that are valid JSON but operationally useless

keep training and eval apart

The next mistake is mixing training and eval.

If a teacher model generates both, the student may just get better at the teacher’s style. The score rises. The product may not.

I separate the uses:

• training augmentation
• regression eval
• adversarial eval
• exploratory review
• production holdout

The production holdout is the anchor. It should include real failures, real user language, and examples that came from outside the synthetic generator’s imagination.

hard negatives do a lot of work

The synthetic examples I trust most are hard negatives.

A hard negative looks tempting but should fail. For retrieval, it may be a chunk that uses the same words as the query but answers a different question. For classification, it may be an input that contains a risky phrase without actually being unsafe. For entity extraction, it may be a company name that looks like a person.

{
  "query": "Can I refund a subscription after 30 days?",
  "candidate_chunk": "Subscriptions may be refunded within 30 days of purchase.",
  "label": "insufficient",
  "reason": "The chunk answers the ordinary policy but not the exception path."
}

That kind of record teaches the eval to reject shallow lexical matches.

review the labels, not the prose

Clean prose is a trap.

What matters is whether the expected answer follows from the evidence, whether the negative case is actually negative, whether two reviewers would agree, and whether the example is testing the intended behavior or something else.

For a small eval set, I would rather have 200 reviewed synthetic examples than 20,000 unreviewed ones. Reviewer disagreement matters too. If humans disagree on the label, the example may belong in a separate ambiguity slice instead of the main pass-fail score.

keep the real world in the loop

Synthetic examples age. Policies change. Products change. Model families change. The bug that inspired the dataset disappears. The generator prompt stops matching the current system.

A maintenance pass should remove examples tied to deleted behavior, refresh examples when source documents change, keep old regressions when the failure still matters, and sample current production failures back into review.

The synthetic room is only useful if the team keeps leaving it to check what happened outside.