LLM Structured Outputs Keep Failing: How to Actually Write a Strict-Mode Schema
Bottom line: Turning strict on for a normal JSON Schema usually returns a 400: every field required, additionalProperties false on every object, $ref and allOf rejected outright. This guide shows how to batch-rewrite a loose schema into a strict-mode-compliant one, with nullable unions, hard limits, and a refusal triage checklist.

The takeaway: strict: true is not a "be more careful" toggle — it is a contract that accepts only a subset of JSON Schema. Every field required, additionalProperties: false on every object, $ref and allOf rejected outright. Ship an unrewritten schema under strict mode and you get a 400, not better output.
The same schema that passes your local validator gets bounced back the moment you attach strict: true and send it to the model API. The model did not get dumber — the provider drew a hard boundary around the schema subset that constrained decoding needs, and your schema crossed it.
What strict mode actually changes: locking the sampler into a grammar
In ordinary mode, model-generated JSON only probably matches your schema, backstopped by the "please output JSON" line in your prompt. On OpenAI's own evals, gpt-4-0613 following complex schemas from prompting alone scored under 40 percent; with strict on, gpt-4o-2024-08-06 hit 100 percent on the same suite.
The mechanism is constrained decoding: on the first request the API compiles your schema into a finite state machine, and during decoding every token must be admitted by that state machine. The model cannot emit tokens that violate the schema. This is not a reminder — it is a physical lock. The cost comes in two installments: the first compile of a new schema takes extra time (typically under 10 seconds, up to a minute for complex schemas) and adds roughly a 5-8 percent first-token latency increase; compiled artifacts are cached, so later requests pay nothing.
Be clear about what it guarantees and what it does not: syntax and shape are locked, semantics are not. The model can still confidently fill estimate_hours with 400 or extract the wrong email address. Shape is a floor, not a ceiling — semantic quality still needs an eval suite. Frontier models under load in 2026 still show roughly a 0.4 percent residual malformation rate, so a retry loop remains mandatory.
Three hard rules: all required, no extra keys, nullable instead of optional
Strict mode imposes three rewrite requirements on your schema, and each one runs counter to ordinary JSON Schema intuition:
- Every field must be listed in required. Strict mode has no concept of an optional field. To express "optional," keep the field in required but declare the type as a nullable union —
"type": ["string", "null"]— and let the model return null when absent. This is the exact opposite of standard practice, where required lists only the truly necessary fields. - Every object must explicitly declare additionalProperties: false. In ordinary schemas you often leave this unset (see our article on how additionalProperties behaves across runtimes), but in strict mode it is an admission requirement: omit it and the request fails. The reason is blunt — the state machine can only generate declared keys, so any undeclared key is unreachable by construction.
- The Python trap: Pydantic drops defaulted fields from required. A
priority: str = "low"removes one entry from the generated schema's required list, and strict mode rejects it. Either drop the default or batch-fill with the script below.
All three rules must recurse into every nested object. Fixing the top level while leaving child objects unrewritten still produces a 400. Here is a minimal compliant schema:
{
"type": "object",
"properties": {
"intent": { "type": "string", "enum": ["book", "cancel", "question", "other"] },
"note": { "type": ["string", "null"] }
},
"required": ["intent", "note"],
"additionalProperties": false
}
These constructs fail outright: banned and unenforced keywords
Strict mode supports only a subset of JSON Schema. Split the keywords into two buckets and handle them differently:
| Bucket | Keywords | Behavior under strict mode |
|---|---|---|
| Rejected outright | $ref / $defs references, allOf, not, dependentRequired, dependentSchemas, if/then/else |
The API returns an error; the request fails |
| Silently unenforced | pattern, minLength, maxLength, format, minimum, maximum, multipleOf (fine-tuned models) |
No error, no enforcement — regex constraints must be re-checked in your application layer |
The handling differs: the first bucket must be rewritten — inline your $ref definitions and merge allOf into properties. The second bucket can stay in the schema (some configurations accept them), but never treat them as a defense. Relying on "pattern": "^v\\d+\\.\\d+\\.\\d+$" to guarantee version format is running naked under strict mode; run an explicit validation pass on every response. One more rule at the top: the schema root cannot be an array or a bare union type — wrap it in an object.
Hard limits: 5,000 properties, 10 nesting levels, 120,000 characters
Even with every keyword compliant, exceeding the size limits still fails. The official OpenAI numbers:
- At most 5,000 object properties across the whole schema, with at most 10 levels of nesting (Anthropic's nesting cap is about 6 levels);
- Total string length of property names, definition names, enum values, and const values capped at 120,000 characters;
- At most 1,000 enum values in total; a single string enum with more than 250 values may not exceed 15,000 characters of string total.
You rarely hit these, but "using the entire company dictionary as an enum" and "compressing a three-level object graph into one level" are both real production failures. When you hit one, split: break a giant schema into multiple calls or flatten the structure. One useful side effect: output keys follow the key order of the schema, so downstream parsers that depend on a fixed order can treat that as a feature.
A script that batch-rewrites a loose schema into strict form
Hand-editing layer by layer invites misses. The recursive script below (the pattern published in DevToolLab's 2026 guide) fills in "every object closed, every key required" in one pass:
def make_strict(schema: dict) -> dict:
"""additionalProperties:false + every key required, at every level."""
if isinstance(schema, dict):
if schema.get("type") == "object" and "properties" in schema:
schema["additionalProperties"] = False
schema["required"] = list(schema["properties"].keys())
for value in schema["properties"].values():
make_strict(value)
elif isinstance(schema, dict):
for value in schema.values():
make_strict(value)
elif isinstance(schema, list):
for item in schema:
make_strict(item)
return schema
The companion workflow: first infer a schema skeleton from sample JSON (the /convert/schema tool page on this site does one-way JSON-to-Schema inference), treat the skeleton as the starting point, tighten it with the three strict rules, and run make_strict as a final backstop. Note that the script only fixes shape — nullable unions, enums, and semantic fields still need human decisions — and any schema containing $ref must be inlined before it can be fed through.
The Anthropic equivalent: tool-as-schema
Anthropic has no strict: true flag. The idiomatic 2026 pattern is to attach the response schema as a tool's input_schema and force the model to call it via tool_choice:
await anthropic.messages.create({
model: "claude-sonnet-4-6",
tools: [{ name: "emit", description: "Emit final summary", input_schema: schema }],
tool_choice: { type: "tool", name: "emit" },
messages,
});
The tool-call channel itself validates JSON, so the effect is equivalent to strict mode. The differences live in the details: Anthropic's nesting cap is about 6 levels, it supports .nullable() rather than .optional(), and all three providers (OpenAI, Anthropic, Gemini) accept slightly different schema subsets. Reusing one schema unchanged across providers is unreliable — re-verify whenever you switch models.
Triage refusal, truncation, and timeouts before you retry
When strict mode hands you something invalid, there are only three causes, and each needs a different response:
- Refusal: the model declined the request (unsafe content and similar). The response's
refusalfield is set — reading JSON at that point is meaningless; route to a human or reword the request. - Truncation:
finish_reasonislength, meaning max_tokens was insufficient and the output is physically incomplete. Schema constraints cannot save a truncated payload; raise the budget or compress the output structure. For identification and partial recovery, see our article on diagnosing and repairing truncated JSON. - Malformation: when the provider does not fully support the subset, bad JSON can still slip through. Do not retry with just "try again" — attach the exact validation error back into the prompt: "severity must be low/medium/high/critical; you output urgent; please correct and re-emit." Retries with error context cut the average retry count from 3 to about 1.5.
Set a hard retry ceiling (two attempts, say); beyond it, degrade to a smaller correction model or a human queue. Do not let one bad output stall the whole pipeline.
Pre-ship checklist
| Check | Pass criterion |
|---|---|
| required | Every field of every object appears in required |
| additionalProperties | Explicit false on every object, down to the deepest level |
| Optional semantics | Modeled with ["string", "null"] unions; no bare optional |
| Banned keywords | No $ref / allOf / if-then-else / dependentRequired |
| Unenforced keywords | pattern and format constraints re-validated in the app layer |
| Scale | ≤5,000 properties, ≤10 nesting levels, ≤120,000 string characters |
| Triage | refusal / length / parse failure handled on separate paths |
| Retry | Retries carry the original validation error, capped at 2, then degrade |
| Semantics | Evals or field-level assertions outside the schema |
Once this checklist passes, shape reliability is effectively locked in. For semantics and contract evolution — versioning schemas and making a broken schema turn CI red first — see welding schema contracts into CI.
FAQ
Does strict mode guarantee correct field values?
No. It only locks syntax and shape at decoding time: every key present, types correct, enums legal. It says nothing about semantic correctness — the model can still emit well-formed but wrong data. The semantic layer needs an eval suite or field-level assertions.
Why must optional fields go into required?
The strict-mode state machine requires every declared property to be reachable; omitting a required entry fails validation. Optional semantics are simulated with a ["string", "null"] union: the field always appears, the value may be null, and downstream code treats null as absent.
Are pattern and minLength completely useless?
Under fine-tuned models and most strict configurations they are not enforced. They do not error, which makes them dangerous — you think you have a guard and you do not. Validate responses that rely on those constraints explicitly with ajv or jsonschema after receipt.
Does strict mode slow responses down?
Only the first request with a new schema pays the compile cost (typically under 10 seconds, up to a minute for complex schemas). After the compiled artifact is cached, subsequent requests see roughly a 5-8 percent first-token latency bump. In steady state it is negligible; only frequently changing schemas incur real cost.
What should I re-test after a model version change?
Three things: whether the version's supported schema subset changed, whether the refusal and finish_reason fields are still present, and whether nesting or scale limits tightened. Providers accept different subsets, so before reusing a schema across models, verify it with one minimal request.