One JSON, Two Verdicts: Parser Differences and the Trust Boundary of Validation
Bottom line: When a validator accepts a document and the downstream service crashes, two parsers have usually disagreed about the same text. This article tests duplicate keys, NUL bytes, invalid UTF-8, NaN and leading zeros against real runtimes, explains how CVE-2026-33948 turned parser disagreement into an attack surface, and shows how to assign a single arbiter to every boundary.

Passing validation proves exactly one thing: the text was accepted by that parser, under that policy. Swap the parser and the verdict can flip.
The symptom: tool A passes, tool B rejects
Here is the scene everyone has met. An API response is pasted into an online formatter and gets a green checkmark. The same payload reaches a downstream service and throws. When you put both parsers side by side and compare character by character, neither side is usually "wrong" — the data landed precisely on a point where the two implementations disagree.
There are not many such points, but each one is common. The table below records actual behavior for high-frequency inputs (tested against Node 22 JSON.parse, Python 3.11 stdlib json, jq 1.8 and mainstream online validators):
| Input | Node JSON.parse | Python json.loads | jq | Notes |
|---|---|---|---|---|
{"a":1,"a":2} duplicate key |
accepts, keeps last | accepts, keeps last | accepts, keeps last | RFC 8259 says only SHOULD; all four let it through |
{"n":NaN} |
error | accepts | error | Python stdlib allows it by default — the biggest silent trap |
{"n":01} leading zero |
error | error | error | uniformly rejected, but error positions differ |
"中文" stored as GBK bytes |
error or mojibake | depends on encoding | error | RFC requires UTF-8, but decoding belongs to the IO layer |
{"a":1} with a BOM |
error | accepts (utf-8-sig) | error | whoever strips the BOM decides |
| comments, single quotes, trailing commas | error | error | error | only JSON5/JSONC parsers accept them |
Look at row two: json.loads('{"cpu":NaN}') is legal in Python. The standard library deliberately accepts NaN/Infinity for backwards compatibility, and many pipelines have counted these junk lines as normal records without noticing the contamination. Blocking it requires an explicit parse_constant argument — code below.
Where the differences come from: spec gaps and implementation choices
RFC 8259 is not vague; it deliberately leaves three areas "up to the implementation", and every implementation fills them differently.
First, uniqueness of object names. Section 4 says names SHOULD be unique — a recommendation, not a rule. So a duplicate key is not a syntax error, and parsers everywhere choose "last one wins". A business validator then sees only the surviving value; the value the sender actually sent first has vanished without an error or a log line.
Second, numbers. The grammar only fixes the shape int frac exp; magnitude is left to the implementation. JavaScript forces everything through IEEE 754 doubles, so 9007199254740993 becomes ...992. jq keeps decimal literals faithfully since 1.8, but any arithmetic drops back to doubles. The same document can present three different values across comparison, storage and transport.
Third, UTF-8 and invisible characters. The spec requires exchanged JSON to be UTF-8, but how a file is opened, whether a BOM is stripped and how invalid bytes are treated are IO-layer decisions. The same byte stream gives different verdicts depending on whether Python opens it as utf-8 or utf-8-sig.
A fourth source is permissive dialects. JSON5 allows comments, single quotes and trailing commas; JSONC relaxes the same rules in the VS Code ecosystem. Config files parsed with a lenient parser and APIs parsed with a strict one is the most common divergence inside a single team.
CVE-2026-33948: when disagreement becomes an attack surface
Parser differences are usually a UX annoyance. Once "validation" and "parsing" run on two different implementations, they become an attack surface. CVE-2026-33948, disclosed in 2026, is a textbook case: jq versions before 1.8.2 allowed embedded NUL bytes (\0) when reading JSON from files, so the same input produced two different verdicts — the validation stage treated the content as ending there, while the actual processing stage kept reading past it. The bypass is complete. The GitHub security advisory confirms the fix: upgrade to 1.8.2.
Abstract the lesson: in any "validate first, parse second" chain, if the two steps use different implementations or configurations, an attacker can craft an input the validator rejects and the parser accepts, smuggling a payload past the check. The same structure holds at the gateway — API7's argument for writing strict parsing into the API contract is that parse semantics are part of the contract, not private behavior of an implementation. When a gateway rejects a request under one rule set and the backend service accepts the same body under another parser, the contract has already broken.
A self-check takes one command — confirm every stage runs the same jq version, including this fix:
jq --version
# jq-1.7.1 ← vulnerable, upgrade
# jq-1.8.2 ← fixed
Duplicate keys: the last value silently swallowed
Duplicate keys deserve their own section because they are the one boundary every mainstream parser accepts while nearly every developer assumes it errors.
import json
text = '{"role": "guest", "role": "admin"}'
data = json.loads(text)
print(data) # {'role': 'admin'}
print("guest" in data.values()) # False: the first value is gone
If that text feeds a signature check or a permission decision, an attacker can append one same-name field to overwrite the privilege, and your validation code only ever sees the final value. Node's JSON.parse behaves identically. To detect rather than silently overwrite, Python accepts an object_pairs_hook:
from collections import Counter
def reject_dup(pairs):
counts = Counter(k for k, _ in pairs)
dup = [k for k, c in counts.items() if c > 1]
if dup:
raise ValueError(f"duplicate keys: {dup}")
return dict(pairs)
data = json.loads('{"a":1,"a":2}', object_pairs_hook=reject_dup)
# ValueError: duplicate keys: ['a']
RFC 8259 section 4 puts it plainly: for interoperability, names SHOULD be unique. Promoting that SHOULD into a validation rule costs one traversal and removes the most common source of "two tools, two verdicts".
The fix: assign one arbiter to every boundary
The repair is not "switch to a stricter parser". It is giving every boundary point an explicit, written policy — and making validation and execution use the same one. The table below works as a review checklist:
| Boundary | Lenient default | Recommended verdict | Representative configuration |
|---|---|---|---|
| duplicate keys | last value wins | reject | Python object_pairs_hook, Go json/v2 rejects by default |
| NaN / Infinity | Python accepts | reject | Python parse_constant raising |
| unknown members | silently ignored | decide per contract | Go json/v2 RejectUnknownMembers(true) |
| invalid UTF-8 | varies by implementation | reject | explicit byte-stream validation in the decode layer |
| BOM | whoever strips it decides | strip once at the entry | fixed utf-8-sig or strip-before-parse |
| validation vs execution split | each side its own | one implementation, one config | validators reuse the production parser |
On the Python side, switching off the two most dangerous defaults takes one call:
import json
def _reject(value):
raise ValueError(f"non-standard constant: {value}")
data = json.loads(
raw,
parse_constant=_reject, # block NaN / Infinity
object_pairs_hook=reject_dup, # block duplicate keys
)
encoding/json/v2 in Go (aliased as jsonv2) makes the whole policy explicit: RejectUnknownMembers(true) rejects unknown fields, and duplicate names plus invalid UTF-8 are rejected by default. The GopherForge practice page is precise about the order: bound the input bytes, decode under an explicit policy, validate the domain and authorize, and only then allow side effects — and every failure path must be exercised with fixtures rather than judged by a changed error string.
Put the parse policy into contracts and corpora
The policy needs protection against drift. Two actions are enough.
First, write "what we accept, what we reject" into the interface documentation next to the field definitions. API7 recommends a stable error code so clients can distinguish a permanent contract error from a transient infrastructure failure — a rejected duplicate key must not look like a gateway timeout, or you invite a retry storm. Do not echo the full request body into logs: raw payloads may carry credentials, and some invalid byte sequences are unsafe to render.
Second, build an ambiguity corpus. Turn every row of the table above into a test fixture: duplicate keys, differently-cased keys, unknown members, invalid UTF-8, BOM, a trailing second value, empty input. Assert two things in CI: invalid inputs fail without producing a usable partial value, and accepted inputs survive an encode/decode round trip unchanged. When a parser is upgraded, replay production-shaped samples in a protected environment in comparison mode, and review the change in the acceptance set as a real contract change instead of assuming old fixtures still cover everything.
These actions complement JSON validation in CI: jsonlint, ajv and jq pipelines, which covers how syntax and schema validation slot into the pipeline; this article covers what comes before the first node — the parser itself. Injection-class risks such as prototype pollution and XSS are covered in the JSON security risk checklist; parser-level trust-chain issues like duplicate keys and NUL bytes are exactly layer zero of that list. For locating plain syntax errors, see JSON grammar boundaries from RFC 8259.
A pre-launch checklist
- Align parser versions across the chain: jq, language runtimes, gateway, offline validators — CVE-2026-33948-class fixes must land everywhere;
- Explicitly reject duplicate keys and NaN: put
object_pairs_hookandparse_constantinto shared wrappers, never calljson.loadsbare in business code; - Decide unknown members per route: forward-compatible consumers may ignore them, command endpoints must set
RejectUnknownMembers; - Ambiguity corpus in CI: at least 8 fixtures, plus an old-vs-new comparison run before any parser upgrade;
- Classifiable errors: distinct codes for contract errors and infrastructure errors, retry only the latter.
Back to the opening line: passing validation means "accepted by some parser under some policy". Make that policy explicit — into the contract, into the corpus, into version pinning — and "one JSON, two verdicts" finally disappears.
FAQ
Are duplicate keys actually an error in the JSON spec?
Not a syntax error. RFC 8259 section 4 phrases it as names SHOULD be unique — a recommendation, not a requirement — so mainstream parsers accept duplicates and keep the last value by default. The spec does recommend uniqueness for interoperability. Whether to promote it into a validation error is a contract decision; detection costs a single traversal.
Why does Python's json library accept NaN while JavaScript throws?
Python's standard library deliberately accepts NaN and Infinity so it can read back data it historically emitted lossily. ECMAScript's JSON.parse implements the grammar strictly and throws a SyntaxError. Passing parse_constant in Python rejects them, and one line of configuration aligns the two runtimes.
Which versions does CVE-2026-33948 affect, and how is it fixed?
jq before 1.8.2, when reading JSON from files, accepts embedded NUL bytes that make the validation stage and the actual parser disagree about the same input, enabling a validation bypass. The GitHub advisory's fix is to upgrade to 1.8.2 or a build containing commit 6374ae0. Pin versions in scripts and CI, and run the ambiguity corpus before upgrading.
I validate once at the entry and reuse the result — am I still exposed?
Yes, but the risk concentrates on that single entry point. If the entry parser, its configuration and its version are pinned, and the ambiguity corpus covers duplicate keys, NUL bytes and UTF-8, there is no second arbiter inside the chain. The dangerous topology is a gateway on one implementation and services on another, or validation on one parser and execution on another — that hands the verdict to the disagreement itself.