Data contracts are where AI quality starts

When a model gives a bad answer, the prompt is an obvious suspect. Add an instruction. Ask for citations. Tell it to prefer current data.

Sometimes that fixes the problem. But prompts often end up carrying assumptions that belong in the data pipeline: what a field means, whether a value is current, who can see the source, and whether a missing value is actually missing.

A data contract makes those assumptions explicit before they reach the model.

Define the safe assumptions

For an AI feature, a contract can specify field names, types, null semantics, freshness, ownership, deletion behavior, label definitions, allowed enum values, and permission metadata.

ticket.id: stable unique identifier
ticket.body: latest customer-visible message body
ticket.priority: one of low, normal, high, urgent
ticket.account_tier: nullable only when account is deleted
ticket.updated_at: source system update time
ticket.permissions: users and groups allowed to view source

This is not administrative detail. It is the boundary around what a downstream system may safely infer. Without it, every ambiguity arrives at the model as a reasoning task.

Give nulls and labels a meaning

Null handling is one of the quickest ways to expose a weak contract.

Unknown, unavailable, redacted, not applicable, not collected, and deleted are different conditions. A prompt can describe those differences, but the data should carry them in a form the pipeline can validate.

owner_state:
  assigned
  unassigned
  deleted
  redacted
  unknown

Labels need the same discipline. severity, priority, risk, and status can look like interchangeable strings while encoding different policies and workflows. If an evaluator or model depends on one, the field needs an owner, a definition, and a way to represent change.

Freshness is part of correctness

AI features commonly pass through several derived layers: source table, export job, parser, chunks, embeddings, index, retrieval results, and prompt context.

Every layer can succeed technically while serving stale information. A contract should describe the freshness the consumer is allowed to expect:

source: customer_docs
freshness_slo: indexed within 15 minutes of source update
delete_slo: removed from index within 5 minutes of source deletion
permission_slo: permission changes reflected before retrieval

If a policy changed this morning and retrieval still returns yesterdays chunk, a fluent answer is still a wrong answer.

Carry permissions with the object

Permission checks at the source application do not automatically protect a derived index.

Any object that may be retrieved, summarized, embedded, or added to context needs enough metadata for downstream systems to preserve its access boundary:

source_object_id: required
tenant_id: required
visibility: public | tenant | group | user
allowed_group_ids: required when visibility = group
allowed_user_ids: required when visibility = user
permission_version: required

This does not solve authorization by itself. It prevents the permission boundary from disappearing silently as data moves through the system.

Turn the contract into checks

A contract earns its keep when a pipeline can enforce it.

Useful checks are often plain and unglamorous:

required field missing
enum value unknown
timestamp older than freshness limit
permission metadata absent
duplicate source object ids
chunk without source hash
label distribution moved beyond threshold

Those failures should stop or quarantine the affected data. Otherwise the team will discover the regression through a bad answer and respond by adding more instructions to the prompt.

Evals need stable inputs

An evaluation set is only as meaningful as the assumptions behind it.

If its fields are undocumented, its permissions are stale, or its labels changed meaning, the model is being graded against a moving target. A contract gives the evaluation something concrete to name:

eval_set: support_escalation_holdout_v4
source_contract: support_ticket_v7
label_contract: escalation_risk_v3
created_from_snapshot: 2025-03-01
excluded_after: 2025-03-01
permission_model: tenant_group_v2

When the contract changes, the evaluation may need a new version too. That is not process for its own sake; it is how a score remains interpretable.

The model is one part of the answer-producing system. The contract defines what the inputs mean, what may be trusted, and what must fail before the model sees it.