When an LLM JSON Output Is Truncated: Read the Stop Reason First
Bottom line: The model returns JSON that stops mid-value, and the parser only gives you a column number. Closing the brackets makes it parse again, but every missing field is now hidden inside the data. This article draws the line between what truncation repair can honestly recover and what must be retried, from stop_reason gating to schema field ordering.

Truncation is not a syntax error, it is missing data. Syntax can be repaired; truncation can only be partly recovered. The moment a repair helper closes the gap, the remaining gap stops being a parse error and starts being something nobody knows about.
What truncation looks like: a column number will not save you
The model writes JSON against your schema, hits the output cap halfway through, and the text just stops. Feed this into Python 3.11's json module:
{"scores": {"clarity": 8, "depth": 7}, "items": [{"name": "a", "qty": 2}, {"name": "b", "qty"
json.loads raises Expecting ':' delimiter, col 94. The column points at the "qty" key with no value behind it, but the actual cause is that the document has no ending at all. Closing it with a greedy regex plus "} fails outright; a string-aware stack close (section six below) parses, but items holds 2 entries and the third partial record is gone. The column number tells you where the text stopped. It never tells you how much you lost.
That is the first real difference between truncation and an ordinary JSON error. A trailing comma has a well-defined end state: remove it and the document is whole. Truncation has no such end state. Repair it and you still have a fragment.
Truncation is not syntax damage: the blind spot in every repair helper
Repair helpers like jsonrepair exist to turn syntactically broken JSON into legal JSON: strip markdown fences, drop trailing commas, convert single quotes. In those cases the intent is intact and the repair is just cleaning. A truncated input looks like broken JSON to the same helper, so it closes the open string, the open array and the open object, and hands back something that parses. Of 83 truncated Claude reports, the helper "fixed" 79. Only 4 failed hard. The error monitor saw a failure rate twenty times lower than reality.
What did the repaired output look like? Question 6's feedback string ended mid-sentence, questions 7 through 9 did not exist, and the next_steps array was gone. Pydantic validated it anyway, because next_steps had a default of [] and the feedback list had no length constraint. Every layer did exactly its job. The product produced a confidently wrong report.
Put syntax repair into a truncation path and it quietly changes from cleaning to fabricating. So the boundary has to come first: a repair helper should only handle formatting slop on complete answers; a cut-off response should never reach it. For the formatting-slop side, see the JSON repair guide.
Step one is always the stop reason
Every major API tells you why generation stopped, in the response metadata, for free. It is also the single most ignored line of code in LLM integrations. Anthropic exposes stop_reason, OpenAI's chat API exposes finish_reason: max_tokens means truncated, end_turn or stop means the model finished on its own.
resp = client.messages.create(model=MODEL, max_tokens=4096, ...)
if resp.stop_reason != "end_turn":
raise TruncatedOutput(
f"stop_reason={resp.stop_reason}, "
f"output_tokens={resp.usage.output_tokens}"
)
data = json.loads(resp.content[0].text)
Once truncation becomes an explicit exception, the retry path has a trigger and monitoring has a signal. Those 83 reports were found afterwards with a single query over the stop_reason field in logs that had been stored all along and never read. Gemini reports length/incomplete in the same spirit; when you add a new vendor, check the mapping table before assuming anything.
max_tokens is a safety rail, not a budget
The direct cause is usually an output cap sized for an earlier version of the product: a 1500-token limit that starts cutting 39% of long reports the moment long sessions ship. Raising the cap is free. You are billed for tokens actually generated, so a 4096 cap on a report that needs 1100 tokens costs exactly the same as a 1500 cap. The cap exists to stop runaway output, not to save money.
Size it from data: watch the real output-token distribution for a week, take p99, leave headroom. The team behind the incident chose 4096 because their p99 was 2350 after the fix. The other direction is to shrink the output itself. Repeated keys in homogeneous arrays are where tokens go, and swapping in a tabular encoding before the call (see the TOON format guide) cuts the volume that has to fit under the cap.
Schema field order decides what gets cut
Models write JSON top to bottom. When the budget runs out, whatever sits at the end of your property list disappears first. A report schema ordered summary, per_question_feedback, scores, next_steps loses the scores first, and scores are the part users screenshot and compare week to week. The fix is a reorder, not a redesign: short critical fields first, long regenerable detail later, free-text notes last.
Vendor behaviour also changes the consequence of the same truncation:
| Vendor | Structured output mechanism | Behaviour when truncated |
|---|---|---|
| OpenAI | response_format with strict: true |
Sampling-level guarantee but the narrowest dialect; usually refuses to generate |
| Anthropic | tool use input_schema |
No decode-time guarantee; may hand back unclosed braces |
| Gemini | responseSchema with JSON MIME type |
Decode enforces enums and types only; truncation still possible |
So "structured output is on, no validation needed" is a myth. The looser the vendor guarantee, the more you need your own schema validation and retry loop. How to build that contract test is covered in JSON Schema contract testing.
When you genuinely need partial recovery: string-aware stack closing
Some cases must squeeze something out of the fragment: streaming previews, degraded responses, human review queues. There the closing operation is the last rung of a repair ladder, and it has two requirements. First, the bracket scan must be string-aware: track quote state and escapes character by character, otherwise a { inside a value corrupts the stack. Second, delete partial key-value pairs before closing. A "qty" key with no value cannot be closed into {"qty"}; the whole pair has to go.
def close_truncated(s: str) -> str:
# 1) String-aware scan; close an open string first
out, in_str, esc = [], False, False
for ch in s:
out.append(ch)
if esc: esc = False; continue
if ch == '\\' and in_str: esc = True; continue
if ch == '"': in_str = not in_str
text = ''.join(out) + ('"' if in_str else '')
# 2) Remove partial pairs and dangling colon or comma
text = re.sub(r',\s*"[^"]*"\s*$', '', text)
text = re.sub(r'[{,]\s*"[^"]*"\s*:\s*$', '}', text)
text = re.sub(r':\s*$', '', text)
text = re.sub(r',\s*$', '', text)
# 3) Rescan the stack and close LIFO
stack, in_str, esc = [], False, False
for ch in text:
if esc: esc = False; continue
if ch == '\\' and in_str: esc = True; continue
if ch == '"': in_str = not in_str; continue
if not in_str:
if ch in '{[': stack.append(ch)
elif ch in '}]' and stack: stack.pop()
return text + ''.join('}' if c == '{' else ']' for c in reversed(stack))
Run on the input from the opening section, this returns {"scores": {...}, "items": [{"name": "a", "qty": 2}, {"name": "b"}]} on Python 3.11 with the standard library only. It parses, and the third record is dropped. Local-model tooling uses dedicated libraries such as partial-json-parser for the same job: close by stack, trim by completeness.
The critical part is disclosure. The closing function must report which rung produced the result; anything produced by the truncation rung is a partial result and must be labelled as such: re-ask for the missing scope, retry with a larger cap, or tell the user their data is incomplete. Silently pretending nothing happened is manufacturing hallucinations with deterministic code.
Repair or retry: a boundary table
| Input shape | Correct action | Why |
|---|---|---|
| Fences, trailing commas, single quotes | Repair helper | Intent is complete, only syntax is dirty |
| Cut mid-string | Retry, or partial recovery plus disclosure | The value itself is missing |
| Cut mid-structure | Partial recovery plus disclosure | Completed fields survive, partial pairs must go |
| Parses but looks wrong | Business validation | Not truncation, but frequently concurrent |
A finance team put a number on the cost. Long invoices were truncated, the bracket-closing step produced valid JSON, the missing total took a default of 0, and a system with 98.2% format compliance shipped roughly 200 wrong invoices a month. After they treated truncation as a hard downgrade and added an identity check (total == sum of line items), the monetary error rate fell from 0.8% to 0.02% while format compliance moved only to 99.1%. They had optimized "format compliance" for three months against the wrong metric. Passing the parser is not the same as passing the business.
The four-item checklist: gate on the stop reason before parsing; size the cap from p99 rather than as a budget; put critical schema fields first; let the repair helper see only complete answers' formatting slop. Do all four and truncation turns from silent data corruption back into a loud, retryable failure.
Can I just close the brackets and use the result
Not directly. Closing only makes it syntactically legal; the data is still a fragment with a half sentence at the end, dropped trailing records, and default values standing in for real ones. Mark the output as partial, trigger a retry or degradation path, and log the recovery rate.
How do I tell truncation apart from a genuine syntax error
Read the stop reason in the response metadata: max_tokens, length or incomplete means truncated, while end_turn or stop with a parse failure means real syntax damage. Without metadata, look at the tail: cut inside a quote or bracket with the outermost structure unclosed is almost always truncation.
Why is a repaired truncation more dangerous than a loud failure
A repair helper optimizes for "parses", and a closed truncation parses, so monitoring stays quiet and downstream validation passes on defaults. That is how 83 truncated reports produced only 4 errors. The helper has no notion of intent and cannot know what is missing, so truncation must be intercepted upstream.
Does raising max_tokens cost more money
No. Billing follows tokens actually generated; the cap is only a rail. A 4096 cap on a 1100-token output bills exactly like a 1500 cap. Set it from the p99 of real output plus headroom and spend your effort on preventing runaway generation instead.