Debugging Minified JSON: From line 1 column 23611 to the Exact Broken Byte

Bottom line: line and column numbers only mean something in multi-line documents. Once JSON is minified into a single line, every error lives on line 1 — the number you need is not the line, it is the character index.

A minified JSON document has exactly one line, so the line/column pair in an error message collapses a four-million-byte position into one useless phrase. The workaround is to skip line/column entirely and read the character offset instead — the humble (char 23610) at the end of the message is the number that actually saves you.

Every number below was produced locally. The sample is a single-line document holding 13,000 billing events, 2,011,084 characters in total, with one comma deliberately deleted after the event_id of event #154. The environment is Python 3.11.1, Node 22.13.1 and jq 1.7.

1. First, the facts: three parsers, three numbers

The same broken file, fed to three tools:

复制Python 3.11.1  Expecting ',' delimiter: line 1 column 23611 (char 23610)
Node  22.13.1  Expected ',' or '}' after property value in JSON at position 23610 (line 1 column 23611)
jq    1.7      Expected separator between values at line 1, column 23619     exit code 5

Side by side:

The takeaway is blunt: Python's char N and Node's position N are the same number and do line up. jq uses a scheme of its own — do not waste time reconciling it.

The three numbers differ not because somebody miscalculated, but because they count different things. Python and Node report where the parser stopped; jq reports where the token it failed to parse begins. A handful of already-consumed characters sit between the two — 9 in this sample.

2. Why line 1 column N is noise on this kind of file

Line numbers are a derived quantity that assumes "it has been a while since the last newline". A minified document has one \n, usually at the end, so:

• lineno is always 1;
• colno degrades into "how many characters past the start of that one line";
• and that line is four million characters wide.

Jump to column 23611 in an editor and you see nothing useful. Worse, tools such as less and vim crawl on extremely long lines; nudging the cursor along with the arrow keys burns twenty minutes.

Flip it around: the offset is the native quantity, and line/column are computed from it. Use the native quantity.

3. Approach one: in Python, use e.pos and ignore lineno

JSONDecodeError carries three numbers, and only pos is worth reading. lineno and colno are derivative values that collapse to nothing on a single-line document.

复制# locate.py - Python 3.11.1
import json
import sys

path = sys.argv[1]
raw = open(path, encoding='utf-8').read()

try:
    json.loads(raw)
except json.JSONDecodeError as error:
    print(f'{type(error).__module__}.{type(error).__name__}: {error}')
    start = max(0, error.pos - 60)
    end = min(len(raw), error.pos + 60)
    print(f'\nchar {start}..{end} of {len(raw)}:')
    print(raw[start:end])
    print(' ' * (error.pos - start) + '^')
    sys.exit(1)

print('valid JSON')

Run it against the broken file:

复制$ python3 locate.py bench/minified-json-error-position/broken.json
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 23611 (char 23610)

char 23550..23670 of 2011083:
ua":"Mozilla/5.0 Chrome/141.0.0.0"},{"event_id":"evt_000154""user_id":100154,"email":"user154@example.com","plan":"pro",
                                                            ^

The caret marks the spot where a comma is missing. Twenty minutes becomes one command.

How wide should the window be: ±60 characters almost always spans a complete key/value pair. Too narrow and you lose context; too wide and the output fills the screen. Deeply nested documents can go to ±120.

4. Approach two: no Python in the container — use jq plus cut

Production containers often ship without Python, but they almost always have jq, or can get it with one apt-get install jq. What jq gives you is a loud failure:

复制$ jq . bench/minified-json-error-position/broken.json
jq: parse error: Expected separator between values at line 1, column 23619
$ echo $?
5

Exit code 5 means invalid input, so under set -e it aborts the script the same way grep returning 1 does. That is a feature.

jq has no flag for printing the offending bytes, so hand the position to cut:

复制# Note -c (characters), not -b (bytes)
$ cut -c23550-23670 bench/minified-json-error-position/broken.json
"ua":"Mozilla/5.0 Chrome/141.0.0.0"},{"event_id":"evt_000154""user_id":100154,"email":"user154@example.com","plan":"pro",

Once the comma is back, jq can do what it is actually good at:

复制$ jq -r '.events | length' bench/minified-json-error-position/payload.json
13000
$ jq -r '.events[154] | "\(.event_id) \(.plan) \(.amount_cents)"' bench/minified-json-error-position/payload.json
evt_000154 pro 74479

5. Approach three: no runtime, or data that must not touch disk — paste it into a browser

The first two approaches require something installed and permission to write files. If you are on a bare laptop, or the payload is full of customer emails and IP addresses and "I pasted it into some website" is a sentence you do not want in the incident report, the third option fits.

Paste the whole string into an online JSON parser and it puts the caret on the offending character and labels the line and column; the advantage of a local tool is that those 4 MB never leave the machine and never land in somebody else's logs.

Two caveats:

1. Pick a tool that says explicitly it processes everything in the browser, never one that POSTs your content to a backend;
2. Pasting tolerates very large files, but past a few tens of megabytes the tab itself eats memory — go back to approach one at that point.

6. Character offset is not byte offset: the second trap with CJK data

This one bites both approaches on mixed Chinese/English payloads.

JSONDecodeError.pos is a character offset (a count of Unicode code points), because it results from parsing a str. cut -c slices by character and lines up with it. But head -c, tail -c and dd slice by byte, and under UTF-8 one slip puts everything off.

A Chinese sample makes it concrete (5 orders, 148 characters, 222 bytes):

复制>>> raw = open('bench/minified-json-error-position/cn.json', encoding='utf-8').read()
>>> raw[25:35]                                    # characters 25-35
'城市":"杭州"},'
>>> open('bench/minified-json-error-position/cn.json','rb').read()[25:35]      # bytes 25-35
b'DD00000","'

Same offset: the character view lands on a field name, the byte view lands on the tail of an order id. A single CJK character occupies 3 bytes under UTF-8, and the drift accumulates with position.

Rule: after reading e.pos, slice with cut -c or a Python slice, never with head -c / dd bs=1 skip=N. If byte tools are all you have, convert first: len(raw[:error.pos].encode('utf-8')).

7. Syntax fixed, data still wrong: three silent traps

This is exactly why debugging cannot stop at "it parsed". Each of the following either never raises, or raises long after the value is used.

Trap 1: duplicate keys — last one wins, nobody complains

RFC 8259 section 4 is clear: names within an object SHOULD be unique, and when they are not, behaviour is unpredictable — most implementations keep the last occurrence, others raise. All three tools measured here keep the last:

复制Input   {"order_id":"A-1001","total":4200,"order_id":"A-1002"}
Node    {"order_id":"A-1002","total":4200}
Python  {'order_id': 'A-1002', 'total': 4200}
jq      {"order_id":"A-1002","total":4200}

Not a single warning. If the overwritten field was amount_cents, you find out when reconciliation is off by three cents.

Trap 2: NaN and Infinity — Python accepts them, Node does not

RFC 8259 section 6 states plainly that number literals contain no NaN and no Infinity. Python's json.loads accepts both by default, because its own float has those values:

复制>>> json.loads('{"discount":NaN}')
{'discount': nan}
>>> json.dumps(json.loads('{"discount":NaN}'))
'{"discount": NaN}'    # written straight back out

Hand that output to Node and you get:

复制SyntaxError: Unexpected token 'N', "{"discount":NaN}" is not valid JSON

In other words: bad data clears Python and detonates at the downstream consumer.

Trap 3: integers beyond 2^53 lose precision quietly

One document, two different values:

复制JSON source   {"order_no":12345678901234567890}
Python        12345678901234567890          <- kept as-is
Node          12345678901234567000          <- last three digits flattened

The reason in one line: JavaScript's Number.MAX_SAFE_INTEGER is 9007199254740991 (2^53-1). MDN points out specifically that the reviver runs after the value has already been parsed into a JavaScript number, so precision is lost before the reviver — it cannot be recovered on the deserialising side. The only fix is for the producer to serialise large IDs as strings.

The three traps compared:

Catch the first two in one pass

复制# check.py - Python 3.11.1
import json
import sys
from collections import Counter


def no_dupes(pairs):
    counts = Counter(key for key, _ in pairs)
    dupes = sorted(key for key, n in counts.items() if n > 1)
    if dupes:
        raise ValueError(f'duplicate key(s): {dupes}')
    return dict(pairs)


def reject_constant(name):
    raise ValueError(f'non-standard JSON constant: {name}')


raw = open(sys.argv[1], encoding='utf-8').read()
try:
    data = json.loads(raw, object_pairs_hook=no_dupes, parse_constant=reject_constant)
except json.JSONDecodeError as error:
    start, end = max(0, error.pos - 60), min(len(raw), error.pos + 60)
    print(f'{error}\n\nchar {start}..{end} of {len(raw)}:')
    print(raw[start:end])
    print(' ' * (error.pos - start) + '^')
    sys.exit(1)
except ValueError as error:
    print(f'rejected: {error}')
    sys.exit(1)

print(f'valid JSON, {len(raw)} chars')

The cost, stated honestly: on the same 2,011,084-character sample, this strict version lands at 35.2-36.2 ms against 9.4-9.9 ms for the lenient one, roughly 3.7x slower (3.6x to 3.8x across five rounds of five runs each, median per round) — object_pairs_hook runs a Counter for every object. Keep it out of the CI happy path and switch to it only when a delivery looks suspicious.

8. Checklist

After fixing the file it is tempting to pretty-print it for readability, but the same sample measures 2,011,084 characters minified (separators=(',',':')) and 2,869,100 characters at indent=2 — 43% more. That weight travels into Git history, CI caches and network transfer, so keep the pretty version local and squeeze it back to one line before shipping.

• [ ] Look for (char N) / position N first; ignore line 1 column N;
• [ ] Python uses e.pos, jq pairs with cut -c — same character offsets, so never reconcile jq's column against them;
• [ ] For CJK documents slice by character (cut -c, Python slices), never head -c / dd;
• [ ] Container scripts should treat jq exit code 5 as "input invalid";
• [ ] Re-check suspicious data with object_pairs_hook + parse_constant and accept the roughly 3.7x cost;
• [ ] Ship any ID above 2^53 as a string; do not expect a reviver to rescue it;
• [ ] Pretty-printed output (measured +43%) stays local: use /minify to compress back to one line before shipping or committing; above a few MB switch to streaming instead, covered in Parsing, rendering and streaming large JSON.

FAQ

Why do all three tools report a different position?

Because they count different things. Python's char N and Node's position N both mean "where the parser stopped" — on this sample both are 23610, so they match. jq reports where the offending token starts, 23619 here, nine further along. Nobody miscalculated and no correction is needed; only compare numbers within the same tool.

Should I use cut -c or head -c?

Use cut -c. JSONDecodeError.pos is an offset counted in Unicode characters, and cut -c slices by character, so the two agree. head -c, tail -c and dd slice by byte, which drifts on UTF-8 documents containing CJK text and can even cut a character in half. On the Chinese sample, the same offset yields 城市":"杭州"}, from the character view and DD00000"," from the byte view.

Is duplicate-key handling really unguarded everywhere?

It is. RFC 8259 section 4 only says names SHOULD be unique and states that behaviour is unpredictable when they are not. Python, Node and jq all measured here apply last-one-wins without emitting any warning. Catching it requires your own object_pairs_hook, at the cost of about 3.7x parse time.

Why does Python read NaN fine while Node throws?

RFC 8259 section 6 disallows NaN and Infinity as number literals. Python's json.loads accepts them by default because Python's own float has those values, so bad data clears Python and gets written straight back out. Node's JSON.parse follows the spec strictly and throws a SyntaxError on that output. Adding parse_constant at the entry point rejects it immediately.

Do 20-digit order numbers always lose precision over JSON?

Only when the consumer parses them into an IEEE 754 double. Python's int is arbitrary precision and keeps 12345678901234567890; JavaScript's Number.MAX_SAFE_INTEGER is 9007199254740991, so the same field becomes 12345678901234567000. MDN notes explicitly that the reviver runs after parsing, so nothing can be recovered downstream — the only reliable fix is for the producer to emit strings.