Reading JSON grammar boundaries from RFC 8259

Bottom line: when a JSON syntax error appears, find the first illegal character and classify it as a single quote, a trailing comma, a comment, or a control character instead of re-reading the whole document; on the encoding side, confirm UTF-8 without a BOM and carry integers beyond 2^53 - 1 as strings.

The body of the specification is only a dozen pages, yet behind every "MUST", "SHOULD", and "MAY" hides an incident that once kept a team up all night.

1. How the three texts relate

Learning JSON often gets confusing because of three numbers: RFC 4627, RFC 7159, RFC 8259, plus ECMA-404. The relationship is actually quite clear:

Which one should you read? RFC 8259 alone is enough. ECMA-404 is equivalent in its grammar portion; the difference is that RFC 8259 also specifies "transport as UTF-8" and "the interoperability issues implementers should watch for".

2. Six value types

The grammar productions of the specification can be condensed into a single diagram:

复制JSON-text  = ws value ws
value      = object / array / number / string / "true" / "false" / "null"
object     = "{" [ member *( "," member ) ] "}"
member     = string ws ":" ws value
array      = "[" [ value *( "," value ) ] "]"

There are only six key points:

1. Objects use curly braces, members are separated by commas, and a comma after the last member is not allowed;
2. Arrays use square brackets, elements are separated by commas, and trailing commas are likewise not allowed;
3. Numbers allow no leading +, no leading zeros (01 is invalid), no bare decimal point (1. is invalid), and no NaN or Infinity;
4. Strings must use double quotes; single quotes are invalid;
5. Booleans and null are lowercase literals; True and NULL are both invalid;
6. Only whitespace characters 0x20 (space), 0x09 (tab), 0x0A (line feed), and 0x0D (carriage return) may appear between structural elements.

The snippet below is valid JSON, though many people assume it is wrong the first time they see it — because it is a bare top-level string:

复制"just a string"

Since RFC 7159, the top level can indeed be any value. In APIs, though, it is still recommended to use only objects or arrays; the reason is in section 6.

3. What can actually appear directly inside a string

This is the part most easily fooled by "it runs fine on my machine". The specification's rules for strings are:

• Characters allowed to appear directly: %x20-21, %x23-5B, %x5D-10FFFF;
• In other words, the double quote " (0x22) and the backslash \ (0x5C) must be escaped;
• All control characters below 0x20 must be escaped, including line feed, tab, and carriage return;
• The only available escape sequences are \", \\, \/, \b, \f, \n, \r, \t, and \uXXXX.

复制{
  "ok": "Chinese text, emoji 🎉 and full-width symbols can be written directly",
  "escaped": "A line break goes here\nand a tab\tfinishes it",
  "control": "\u0007 is the bell character and must be escaped",
  "quote": "He said: \"don't use single quotes\""
}

\/ is a legacy escape (it prevents </script> from closing early). Modern scenarios rarely need it, but parsers must support it.

4. Encoding: UTF-8, no BOM

Section 8.1 of RFC 8259 says three things:

1. JSON text used for interchange must be encoded in UTF-8;
2. Implementations must not add a BOM (EF BB BF) at the beginning of the text;
3. Parsers may ignore the BOM in order to be more robust.

Two typical failures in the real world:

• Windows Notepad saves as "UTF-8 with BOM", the file is rejected by JSON.parse with Unexpected token;
• A Chinese JSON file saved as GBK turns into garbled text on a pipeline that decodes only as UTF-8, yet the syntax still happens to pass, so the error is deferred to the business layer.

The handling is plain: strip the BOM when reading the file, then decode explicitly as UTF-8; when writing, make sure no BOM is added.

复制/**
 * Removes the UTF-8 BOM at the start of a string.
 * @param text The original text
 * @returns The text without a BOM
 */
function stripBom(text: string): string {
  return text.charCodeAt(0) === 0xfeff ? text.slice(1) : text;
}

A note on media types while we are here: application/json no longer accepts a charset parameter; writing one does not raise an error, but it is an outdated practice.

5. Three places where the specification deliberately leaves room

The places where the specification declines to pin things down are exactly where implementation differences come from.

1. Range and precision of numbers

RFC 8259 only says "this specification does not limit the range or precision of numbers", while noting that most implementations use IEEE 754 double precision. This means the same 9223372036854775807 becomes 9223372036854776000 in JavaScript, yet stays exact in Java's long.

Conclusion: integers beyond 2^53 - 1, and monetary amounts that need many decimal places of precision, should always be carried as strings, with the pattern written explicitly in the schema.

2. Duplicate keys

The specification's wording is "the names within an object should (SHOULD) be unique"; it does not say what to do when they are not.

• JSON.parse keeps the last one;
• Some parsers keep the first one;
• Some libraries simply raise an error.

Conclusion: do not write duplicate keys, and do not rely on any particular behavior in your validators. Adding a "duplicate key detection" step at build time is far cheaper than investigating afterwards.

3. Limits on depth and length

The specification sets no limit on nesting depth, but every implementation has one. V8 throws RangeError: Maximum call stack size exceeded at tens of thousands of levels of nesting. Servers must impose their own limits; this is a security requirement, not a specification requirement.

6. Testing your understanding with counterexamples

Every line below is invalid JSON. Feed them to a parser and see whether the error messages are helpful:

复制{ 'name': 'json' }        // single quotes
{ name: "json" }          // key name not quoted
{"a": 1,}                 // trailing comma
[1, 2, 3,]                // trailing comma in an array
{"a": 01}                 // leading zero
{"a": +1}                 // leading plus sign
{"a": .5}                 // missing integer part
{"a": NaN}                // non-standard literal
{"a": undefined}          // not a JSON value
{"a": 1} // comment       // comment

An interesting boundary case: {"a": 1} followed by a comment is valid in JSON5, valid in JSONC, and invalid in standard JSON. So "the configuration file runs fine but the API rejects it" is almost always a dialect problem.

7. Production checklist

• [ ] All externally transmitted JSON uses application/json without charset;
• [ ] File read/write pipelines strip the BOM and use explicit UTF-8;
• [ ] Integers beyond 2^53 are carried as strings and annotated in the schema;
• [ ] Duplicate keys are detected at build time;
• [ ] The server limits request body size and nesting depth;
• [ ] Top-level values of public APIs are objects or arrays, never bare scalars.

FAQ

How do I quickly tell whether a piece of text is valid JSON?

Hand the text to a parser or an online parsing tool, read the position of the first illegal character, then check it against RFC 8259's six value types and whitespace rules. The top level may be any JSON value, but between structural elements only space, tab, line feed, and carriage return are allowed. A single quote, an unquoted key, a trailing comma, or a comment means it is not standard JSON.

Why must JSON use UTF-8 and carry no BOM?

Section 8.1 of RFC 8259 requires JSON text used for interchange to be encoded in UTF-8, says implementations must not add a BOM at the start, and lets parsers ignore the BOM for robustness. A file with a BOM is rejected by JSON.parse with Unexpected token, even though nothing looks wrong to the eye. So strip the BOM when reading and decode explicitly as UTF-8.

Should I use JSON5 or JSONC to get around these grammar limits?

Yes, but only for local reading such as configuration files; data exchanged between systems should still be standard JSON. The counterexamples show that {"a": 1} followed by a comment is valid in JSON5 and JSONC but invalid in standard JSON, which is why "the config file runs but the API rejects it" is almost always a dialect problem. When converting, strip the comments, single quotes, and trailing commas first and hand the result to a standard parser.