Synthetic data is a practice room

Synthetic data is most useful when it gives an eval a safe place to rehearse failure.

Fraud systems need unusual account behavior before unusual behavior appears in production. Tool-calling systems need malformed results and ambiguous requests. Extraction systems need invoices with missing totals and inconsistent currencies.

That makes synthetic data a practice room. It does not make it proof.

Start with the behavior

Before generating examples, write down the behavior the dataset is meant to exercise:

behavior: refund_policy_exception_handling
system_under_test: support_response_agent
expected_capability:
  - identify the relevant policy clause
  - detect whether an exception applies
  - ask for missing order information when needed
must_not:
  - promise a refund without evidence
  - cite a policy that was not retrieved
  - ignore region-specific rules

That contract prevents a pile of plausible text from masquerading as an evaluation.

Provenance beats volume

Each record should carry enough history to be retired or questioned later: its source material, generator, prompt, reviewer, and intended use.

{
  "id": "refund-exception-syn-0042",
  "behavior": "refund_policy_exception_handling",
  "source_basis": ["policy/refunds-v3.md", "support/failure-17.md"],
  "generator_model": "teacher-model-2025-11",
  "generator_prompt": "synthetic-refund-cases:v6",
  "label_source": "human_review",
  "split": "eval_synthetic",
  "allowed_uses": ["regression_eval", "debugging"],
  "review_notes": "Customer is in a region with a manual override path."
}

Without that metadata, a passing eval can outlive the policy, prompt, and failure taxonomy that produced it.

Fill gaps, do not inflate the set

Look at what the real data does not contain.

Perhaps ordinary refunds are common but policy exceptions are rare. Perhaps the retrieval set is in English while users regularly send mixed-language snippets. Perhaps the tool caller has many clean successes and almost no partial failures. Perhaps reviewers disagree most often on ambiguous cases.

Synthetic examples can target:

• rare combinations of policy conditions
• hard negatives that look nearly correct
• old bugs that need regression coverage
• adversarial wording around a known boundary
• long-tail entity formats
• corrupted or incomplete inputs
• valid JSON that is useless operationally

This is more valuable than asking a generator for ten thousand generic examples.

Keep the splits honest

Training augmentation, regression evals, adversarial evals, exploratory review, and production holdouts should be separate uses. If one teacher model generates both training and evaluation data, the student may become better at the teacher’s style while the score rises for the wrong reason.

The production holdout is the anchor. It should contain real language, real failures, and examples that did not originate in the synthetic generator’s imagination.

Hard negatives are particularly useful:

{
  "query": "Can I refund a subscription after 30 days?",
  "candidate_chunk": "Subscriptions may be refunded within 30 days of purchase.",
  "label": "insufficient",
  "reason": "The chunk answers the ordinary policy but not the exception path."
}

The record looks relevant lexically, but it should fail because it does not answer the actual question. That is the kind of boundary a shallow score can miss.

Review labels, not polish

Clean prose is easy to overvalue. The harder questions are whether the expected answer follows from the evidence, whether the negative case is genuinely negative, and whether two reviewers agree on the label.

For a small eval, 200 reviewed examples can be more useful than 20,000 unreviewed ones. Reviewer disagreement may deserve its own ambiguity slice rather than being forced into a pass-fail score.

Synthetic examples also age. Policies change, products change, models change, and the bug that justified an example may disappear. Keep regressions that still matter, refresh records when source documents change, retire stale behavior, and feed current production failures back into review.

The practice room works only when the team keeps leaving it to see what happened outside.