Structured Outputs made prompts less magical

Structured Outputs made prompts less magical in the best possible way.

Before schema-constrained output became a first-class API feature, a lot of teams were doing the same awkward dance. Tell the model to return JSON. Add “valid JSON only.” Add “do not include Markdown.” Add examples. Parse it. Watch it fail on a weird input. Add a retry. Add a repair prompt. Add a regex you will regret.

That worked often enough to be tempting and failed often enough to be infrastructure.

OpenAI Structured Outputs moved a common integration problem from prompt superstition into schema work. With strict: true, the model output can be constrained to match a developer-supplied JSON Schema through response formats or tool definitions.

That changes the engineering conversation.

schemas are product contracts

A schema is a parser convenience and a product contract.

If the product needs an extraction result, define the extraction result. If the UI needs sections, define the sections. If a tool needs arguments, define the arguments. If a workflow needs a decision, define the decision.

{
  "type": "object",
  "properties": {
    "priority": {
      "type": "string",
      "enum": ["low", "medium", "high"]
    },
    "summary": {
      "type": "string"
    },
    "requiresHumanReview": {
      "type": "boolean"
    }
  },
  "required": ["priority", "summary", "requiresHumanReview"],
  "additionalProperties": false
}

That shape says what the rest of the system can rely on.

It also forces product decisions. Is priority an enum? Is review required a boolean? Can the model invent new fields? What happens when the answer is unknown? Where does uncertainty live?

Those are better questions than “how strongly should we beg the model to output JSON?”

validity is not correctness

Structured output solves one class of problem. It does not solve all of them.

A valid object can still be wrong.

The model can choose the wrong enum. It can extract the wrong date. It can set requiresHumanReview to false when the case is ambiguous. It can summarize a document badly while still returning a perfect schema.

That distinction matters.

The schema makes the output machine-readable. The application still needs semantic checks, evals, and sometimes human review.

For a production workflow, I would split checks:

• schema validity
• domain validity
• policy validity
• evidence support
• confidence or review routing
• downstream side effects

Structured Outputs can make the first check much more reliable. The other checks remain real work.

optionality needs design

Schemas force uncomfortable decisions about optional fields.

Sometimes a field is genuinely optional. Sometimes the product is avoiding a decision. Sometimes the model should return null. Sometimes it should return an empty array. Sometimes it should include an unknown enum value. Sometimes the request should fail validation because the missing field means the workflow cannot continue.

Those choices should be explicit.

For example:

{
  "type": "object",
  "properties": {
    "dueDate": {
      "type": ["string", "null"],
      "description": "ISO date when explicitly present in the source."
    },
    "dueDateEvidence": {
      "type": "string"
    }
  },
  "required": ["dueDate", "dueDateEvidence"],
  "additionalProperties": false
}

That schema says the field is always present, but the value may be null. The product can distinguish “the model forgot the key” from “the source did not contain a due date.”

That is a small thing that removes a lot of integration weirdness.

tool calls get cleaner

Structured Outputs also fit naturally with tool calls.

Tool arguments should be boring. The model should not invent an argument name, omit required fields, or pass a string where the tool expects a number. If the tool schema is strict, the model has a narrower space to wander.

That does not make tool use safe by itself.

The application still has to validate identity, permissions, business rules, rate limits, and side effects. But the tool boundary gets cleaner because schema shape is less negotiable.

The model/tool contract becomes:

model proposes structured arguments
  -> schema constraint shapes arguments
  -> application validates authority and policy
  -> tool executes or refuses
  -> result returns to model or user

The schema is one layer. It is not the whole guardrail.

schemas need versions

Once a structured output feeds a product workflow, the schema needs a version.

Changing an enum value, adding a required field, renaming a property, or changing null behavior can break consumers. The model call may still succeed while the application, database, dashboard, or downstream job starts interpreting the result differently.

I would record schema version with each run:

{
  "schema": "ticket_triage_result@3",
  "model": "gpt-4o-2024-08-06",
  "prompt": "ticket_triage@12"
}

That record makes migrations and incident reviews much easier. If a field changed last week, the team should not have to guess which outputs used the old contract.

evals should include schema and meaning

Structured output evals should test both shape and meaning.

Shape:

• valid JSON
• required fields present
• no extra properties
• enums valid
• arrays within expected bounds
• nullable fields represented consistently

Meaning:

• extracted values match source
• uncertainty is routed correctly
• review flags appear when needed
• citations support fields
• tool arguments are allowed
• downstream workflow behaves correctly

If shape passes and meaning fails, the schema did its job and the task logic needs work.

That is useful clarity.

less magic is the win

Structured Outputs did not make language models deterministic databases.

They made a common integration boundary less squishy.

Instead of treating formatting as a persuasion problem, teams can treat it as schema work. That moves effort into a place software engineers already understand: types, validators, contracts, fixtures, and failure cases.

The prompt still matters. The model still matters. The data still matters. But the output shape stops being a superstition and starts being an interface.

Interfaces deserve names, versions, tests, and owners. Structured output finally makes that obvious in production code that other systems trust.