JSON Toolkit

LLM Structured Outputs Keep Failing: How to Actually Write a Strict-Mode Schema

Bottom line: Turning strict on for a normal JSON Schema usually returns a 400: every field required, additionalProperties false on every object, $ref and allOf rejected outright. This guide shows how to batch-rewrite a loose schema into a strict-mode-compliant one, with nullable unions, hard limits, and a refusal triage checklist.

Advanced~8 min readPublished 2026-11-19Updated 2026-11-19
Diagram of strict-mode JSON Schema rewriting rules for LLM structured outputs

The takeaway: strict: true is not a "be more careful" toggle — it is a contract that accepts only a subset of JSON Schema. Every field required, additionalProperties: false on every object, $ref and allOf rejected outright. Ship an unrewritten schema under strict mode and you get a 400, not better output.

The same schema that passes your local validator gets bounced back the moment you attach strict: true and send it to the model API. The model did not get dumber — the provider drew a hard boundary around the schema subset that constrained decoding needs, and your schema crossed it.

What strict mode actually changes: locking the sampler into a grammar

In ordinary mode, model-generated JSON only probably matches your schema, backstopped by the "please output JSON" line in your prompt. On OpenAI's own evals, gpt-4-0613 following complex schemas from prompting alone scored under 40 percent; with strict on, gpt-4o-2024-08-06 hit 100 percent on the same suite.

The mechanism is constrained decoding: on the first request the API compiles your schema into a finite state machine, and during decoding every token must be admitted by that state machine. The model cannot emit tokens that violate the schema. This is not a reminder — it is a physical lock. The cost comes in two installments: the first compile of a new schema takes extra time (typically under 10 seconds, up to a minute for complex schemas) and adds roughly a 5-8 percent first-token latency increase; compiled artifacts are cached, so later requests pay nothing.

Be clear about what it guarantees and what it does not: syntax and shape are locked, semantics are not. The model can still confidently fill estimate_hours with 400 or extract the wrong email address. Shape is a floor, not a ceiling — semantic quality still needs an eval suite. Frontier models under load in 2026 still show roughly a 0.4 percent residual malformation rate, so a retry loop remains mandatory.

Three hard rules: all required, no extra keys, nullable instead of optional

Strict mode imposes three rewrite requirements on your schema, and each one runs counter to ordinary JSON Schema intuition:

  1. Every field must be listed in required. Strict mode has no concept of an optional field. To express "optional," keep the field in required but declare the type as a nullable union — "type": ["string", "null"] — and let the model return null when absent. This is the exact opposite of standard practice, where required lists only the truly necessary fields.
  2. Every object must explicitly declare additionalProperties: false. In ordinary schemas you often leave this unset (see our article on how additionalProperties behaves across runtimes), but in strict mode it is an admission requirement: omit it and the request fails. The reason is blunt — the state machine can only generate declared keys, so any undeclared key is unreachable by construction.
  3. The Python trap: Pydantic drops defaulted fields from required. A priority: str = "low" removes one entry from the generated schema's required list, and strict mode rejects it. Either drop the default or batch-fill with the script below.

All three rules must recurse into every nested object. Fixing the top level while leaving child objects unrewritten still produces a 400. Here is a minimal compliant schema:

{
  "type": "object",
  "properties": {
    "intent": { "type": "string", "enum": ["book", "cancel", "question", "other"] },
    "note": { "type": ["string", "null"] }
  },
  "required": ["intent", "note"],
  "additionalProperties": false
}

These constructs fail outright: banned and unenforced keywords

Strict mode supports only a subset of JSON Schema. Split the keywords into two buckets and handle them differently:

Bucket Keywords Behavior under strict mode
Rejected outright $ref / $defs references, allOf, not, dependentRequired, dependentSchemas, if/then/else The API returns an error; the request fails
Silently unenforced pattern, minLength, maxLength, format, minimum, maximum, multipleOf (fine-tuned models) No error, no enforcement — regex constraints must be re-checked in your application layer

The handling differs: the first bucket must be rewritten — inline your $ref definitions and merge allOf into properties. The second bucket can stay in the schema (some configurations accept them), but never treat them as a defense. Relying on "pattern": "^v\\d+\\.\\d+\\.\\d+$" to guarantee version format is running naked under strict mode; run an explicit validation pass on every response. One more rule at the top: the schema root cannot be an array or a bare union type — wrap it in an object.

Hard limits: 5,000 properties, 10 nesting levels, 120,000 characters

Even with every keyword compliant, exceeding the size limits still fails. The official OpenAI numbers:

  • At most 5,000 object properties across the whole schema, with at most 10 levels of nesting (Anthropic's nesting cap is about 6 levels);
  • Total string length of property names, definition names, enum values, and const values capped at 120,000 characters;
  • At most 1,000 enum values in total; a single string enum with more than 250 values may not exceed 15,000 characters of string total.

You rarely hit these, but "using the entire company dictionary as an enum" and "compressing a three-level object graph into one level" are both real production failures. When you hit one, split: break a giant schema into multiple calls or flatten the structure. One useful side effect: output keys follow the key order of the schema, so downstream parsers that depend on a fixed order can treat that as a feature.

A script that batch-rewrites a loose schema into strict form

Hand-editing layer by layer invites misses. The recursive script below (the pattern published in DevToolLab's 2026 guide) fills in "every object closed, every key required" in one pass:

def make_strict(schema: dict) -> dict:
    """additionalProperties:false + every key required, at every level."""
    if isinstance(schema, dict):
        if schema.get("type") == "object" and "properties" in schema:
            schema["additionalProperties"] = False
            schema["required"] = list(schema["properties"].keys())
            for value in schema["properties"].values():
                make_strict(value)
        elif isinstance(schema, dict):
            for value in schema.values():
                make_strict(value)
    elif isinstance(schema, list):
        for item in schema:
            make_strict(item)
    return schema

The companion workflow: first infer a schema skeleton from sample JSON (the /convert/schema tool page on this site does one-way JSON-to-Schema inference), treat the skeleton as the starting point, tighten it with the three strict rules, and run make_strict as a final backstop. Note that the script only fixes shape — nullable unions, enums, and semantic fields still need human decisions — and any schema containing $ref must be inlined before it can be fed through.

The Anthropic equivalent: tool-as-schema

Anthropic has no strict: true flag. The idiomatic 2026 pattern is to attach the response schema as a tool's input_schema and force the model to call it via tool_choice:

await anthropic.messages.create({
  model: "claude-sonnet-4-6",
  tools: [{ name: "emit", description: "Emit final summary", input_schema: schema }],
  tool_choice: { type: "tool", name: "emit" },
  messages,
});

The tool-call channel itself validates JSON, so the effect is equivalent to strict mode. The differences live in the details: Anthropic's nesting cap is about 6 levels, it supports .nullable() rather than .optional(), and all three providers (OpenAI, Anthropic, Gemini) accept slightly different schema subsets. Reusing one schema unchanged across providers is unreliable — re-verify whenever you switch models.

Triage refusal, truncation, and timeouts before you retry

When strict mode hands you something invalid, there are only three causes, and each needs a different response:

  • Refusal: the model declined the request (unsafe content and similar). The response's refusal field is set — reading JSON at that point is meaningless; route to a human or reword the request.
  • Truncation: finish_reason is length, meaning max_tokens was insufficient and the output is physically incomplete. Schema constraints cannot save a truncated payload; raise the budget or compress the output structure. For identification and partial recovery, see our article on diagnosing and repairing truncated JSON.
  • Malformation: when the provider does not fully support the subset, bad JSON can still slip through. Do not retry with just "try again" — attach the exact validation error back into the prompt: "severity must be low/medium/high/critical; you output urgent; please correct and re-emit." Retries with error context cut the average retry count from 3 to about 1.5.

Set a hard retry ceiling (two attempts, say); beyond it, degrade to a smaller correction model or a human queue. Do not let one bad output stall the whole pipeline.

Pre-ship checklist

Check Pass criterion
required Every field of every object appears in required
additionalProperties Explicit false on every object, down to the deepest level
Optional semantics Modeled with ["string", "null"] unions; no bare optional
Banned keywords No $ref / allOf / if-then-else / dependentRequired
Unenforced keywords pattern and format constraints re-validated in the app layer
Scale ≤5,000 properties, ≤10 nesting levels, ≤120,000 string characters
Triage refusal / length / parse failure handled on separate paths
Retry Retries carry the original validation error, capped at 2, then degrade
Semantics Evals or field-level assertions outside the schema

Once this checklist passes, shape reliability is effectively locked in. For semantics and contract evolution — versioning schemas and making a broken schema turn CI red first — see welding schema contracts into CI.

FAQ

Does strict mode guarantee correct field values?

No. It only locks syntax and shape at decoding time: every key present, types correct, enums legal. It says nothing about semantic correctness — the model can still emit well-formed but wrong data. The semantic layer needs an eval suite or field-level assertions.

Why must optional fields go into required?

The strict-mode state machine requires every declared property to be reachable; omitting a required entry fails validation. Optional semantics are simulated with a ["string", "null"] union: the field always appears, the value may be null, and downstream code treats null as absent.

Are pattern and minLength completely useless?

Under fine-tuned models and most strict configurations they are not enforced. They do not error, which makes them dangerous — you think you have a guard and you do not. Validate responses that rely on those constraints explicitly with ajv or jsonschema after receipt.

Does strict mode slow responses down?

Only the first request with a new schema pays the compile cost (typically under 10 seconds, up to a minute for complex schemas). After the compiled artifact is cached, subsequent requests see roughly a 5-8 percent first-token latency bump. In steady state it is negligible; only frequently changing schemas incur real cost.

What should I re-test after a model version change?

Three things: whether the version's supported schema subset changed, whether the refusal and finish_reason fields are still present, and whether nesting or scale limits tightened. Providers accept different subsets, so before reusing a schema across models, verify it with one minimal request.

References

Related articles

JSON Schema Contract Testing: Wire Your Validation Rules into CI

A finished schema is where decay begins, not where work ends: one typo silently widens validation, one refactor breaks every fixture. This guide gives a workable contract testing setup — positive, negative, and snapshot fixtures, schema-first linting, compile-once reuse, and four CI gates that turn contract breakage into a red build.

Advanced~8 min readUpdated 2026-11-05

When an LLM JSON Output Is Truncated: Read the Stop Reason First

The model returns JSON that stops mid-value, and the parser only gives you a column number. Closing the brackets makes it parse again, but every missing field is now hidden inside the data. This article draws the line between what truncation repair can honestly recover and what must be retried, from stop_reason gating to schema field ordering.

Advanced~5 min readUpdated 2026-11-14

Without additionalProperties, Schema Validation Validates Nothing

A passing validation does not mean clean data. Pydantic ignores unknown fields by default, Zod strips them, and Ajv lets them through — hallucinated parameters and typo'd keys all pass silently. This article shows the three forms of silent acceptance, then gives complete recipes for closing objects, controlling coercion, and tightening every nesting level.

Intermediate~8 min readUpdated 2026-11-02