All concepts
Structured Output
Constrain model responses to JSON, schemas, or tool-call shapes.
Transformers & LLMs · Intermediate · ~8 min
In plain English
Instead of asking nicely for JSON and hoping, constrain the model's choices at each step so only valid JSON can come out.
Why it's worth your time
It's the difference between a demo and something you can put a parser behind, and it's now a first-class API feature everywhere.
If you remember three things
- Constrained decoding masks invalid tokens — it can't produce malformed output
- A schema is a contract; prose asking for JSON is a hope
- Field names and descriptions act as prompt text
Overview
Structured output constrains a model to emit machine-readable responses — valid JSON, a specific schema, or a tool-call shape — rather than free prose. Techniques like function calling, JSON mode, or grammar-constrained decoding enforce validity so the result can safely feed downstream code, tools, or databases.
How it works
- Start: Task The application needs a machine-readable answer, not free-form prose.
- Task -> Schema Define required fields, types, enums, and nesting.
- Schema -> Constrained Decode JSON mode, function calling, grammar constraints, or repair loops keep output valid.
- Constrained Decode -> Validator Parse and validate the response before trusting it.
- Validator -> App Action Structured output safely feeds downstream code, tools, or databases.
In an interview
It's making the model return parseable, schema-conforming output instead of free text. You define required fields, types, and enums, then enforce them via function calling, JSON mode, or constrained/grammar decoding that masks invalid tokens, and validate before trusting. This lets LLM output reliably drive APIs and code.
Production defaults
- Use native
- the provider's schema/tool-use mode over prompt-and-pray. It's a different mechanism, not a better prompt
- Schema design
- shallow, few required fields, descriptive names. Deep nesting degrades quality noticeably
- Enums
- for anything categorical. It removes a whole class of parsing failure
- Always
- validate after parsing. Valid JSON is not the same as correct content
What breaks
- Valid JSON, wrong values — Structure was enforced, semantics weren't. Add field descriptions and validate business rules in code.
- Quality dropped when you added the schema — Over-constrained or deeply nested. Flatten it, and let reasoning happen in a free-text field before the structured ones.