JSON Text Sequences (RFC 7464): Streaming JSON Delimited by RS

Bottom line: use JSON-seq when you control both ends, need a registered media type, and want the reader to resynchronise after truncation; use NDJSON when the data feeds log systems, must be human-readable, or has to work with existing tools — never force JSON-seq on a peer that only understands line-delimited input.

NDJSON splits records on newlines, but a newline is a human convention. RFC 7464 writes the boundary into the byte stream itself: an invisible 0x1E before every record.

1. The problem solved by a single byte

RFC 7464 was published in February 2015 under the title JSON Text Sequences, and it registers the media type application/json-seq. It adds no new data types. It only defines how a sequence of JSON texts is arranged inside one byte stream. The whole rule fits in a sentence: put one RS before each JSON text and one LF after it.

• RS is the ASCII record separator, code point 0x1E, known in Unicode as INFORMATION SEPARATOR TWO (U+001E);
• LF is the line feed, 0x0A.

Two records look like this at the byte level:

复制1E 7B 22 69 64 22 3A 31 7D 0A      ← 0x1E {"id":1} + LF
1E 7B 22 69 64 22 3A 32 7D 0A      ← 0x1E {"id":2} + LF

The RS sits before the record, which is the opposite of NDJSON's "write the content, then write the delimiter" layout. That direction changes the recovery behaviour: a reader that sees 0x1E knows the previous record has ended, so even a record truncated mid-write cannot blur the boundary of the next one.

This is not a theoretical design. The RFC explicitly mentions records appended to log files that get truncated by the filesystem after a crash or an administrative kill, and it says a parser should not abort the whole read because of it.

2. Two rule sets: encoders and parsers

RFC 7464 unusually provides two ABNF grammars, one that constrains writers and one that relaxes readers.

In engineering terms, there are five hard rules:

1. UTF-8 is mandatory; UTF-16 and UTF-32 must not be used;
2. The parser treats "any byte other than RS" as a possible JSON text and tries to parse it; on failure it skips to the next RS and keeps going;
3. Consecutive RS bytes do not represent empty elements and are ignored;
4. There is no end-of-sequence marker, so the stream simply ends;
5. When the top-level value is a number, true, false, or null, the record must end with a whitespace byte, otherwise it is dropped as possibly truncated.

The fifth rule is the one people forget. Objects, arrays, and strings are self-delimiting in JSON; numbers are not. <RS>123<RS> cannot tell you whether it holds a complete 123 or a truncated 1234. The RFC therefore requires writers to append an LF, using it as a canary byte for truncation detection. Likewise <RS>true<RS> might originally have been trueish.

3. Choosing between JSON-seq and NDJSON

Both formats stream, but they target different problems:

The decision is short: use JSON-seq when you control both ends, want a formal media type, and need the reader to resynchronise after truncation; use JSON Lines and NDJSON when the data feeds log systems, needs to be readable by humans, or must work with existing tools.

4. A parser: split on 0x1E, then JSON.parse

The most tangible benefit of RFC 7464 is that you do not need a streaming JSON parser. Split by byte, then call JSON.parse on each segment:

复制/** ASCII record separator, defined by RFC 7464 as 0x1E */
const RECORD_SEPARATOR = 0x1e;

/**
 * Parses a chunk of JSON Text Sequence bytes.
 * @param buffer Bytes that have been read so far
 * @returns Successfully parsed records and the number of failed records
 */
function parseJsonSequence(buffer: Buffer): { records: unknown[]; badCount: number } {
  const records: unknown[] = [];
  let badCount = 0;
  let recordStart = buffer.indexOf(RECORD_SEPARATOR);

  while (recordStart !== -1) {
    const nextRecordStart = buffer.indexOf(RECORD_SEPARATOR, recordStart + 1);
    const recordEnd = nextRecordStart === -1 ? buffer.length : nextRecordStart;
    const rawText = buffer.subarray(recordStart + 1, recordEnd).toString('utf8').trim();

    if (rawText !== '') {
      try {
        records.push(JSON.parse(rawText));
      } catch {
        badCount += 1;
      }
    }

    recordStart = nextRecordStart;
  }

  return { records, badCount };
}

Three implementation notes:

• Byte-level splitting is safe. UTF-8 continuation bytes live in the 0x80–0xBF range, so they can never equal 0x1E. Splitting on 0x1E cannot cut a multi-byte character in half.
• Buffer the tail when streaming. While reading in chunks, the final segment may not have reached the next RS yet. Keep it in the buffer until the next chunk arrives; otherwise you will parse the same half-record twice.
• Count failures instead of aborting. The RFC tells parsers to skip to the next RS and continue, so badCount is a warning metric, not an exception. A truncated final record also belongs in that bucket.

Memory use depends on the largest single record plus one buffer, not on the total file size. That is the same idea discussed in parsing and streaming large JSON.

5. Control characters inside strings

JSON requires every control character from U+0000 to U+001F to be escaped inside a string, and RS falls in that range. A valid JSON string can therefore contain only the five characters \u001e, never a raw 0x1E byte. That is why splitting on 0x1E is always safe for valid data.

The danger lives on the writer side:

1. Hand-building JSON text with '\x1e', inserting a raw control character into a string;
2. Building JSON with template literals instead of JSON.stringify, leaving newlines, quotes, and control characters unescaped;
3. Copying text that contains RS from a terminal or editor, where the control character gets swallowed or replaced.

The correct pattern is always to generate the text first, then attach the delimiter:

复制const recordText = JSON.stringify(record);
const line = `${String.fromCharCode(RECORD_SEPARATOR)}${recordText}\n`;

JSON.stringify emits control characters as \u00XX, so a raw RS can never appear inside the record. Once this step is right, the splitting logic only has to deal with bytes and never needs to understand JSON syntax.

Most failures in this category surface as JSON.parse throwing Unexpected token, especially the confusing case where "the first character is an invisible control byte". The JSON error message lookup table walks through that class of error line by line.

6. HTTP transport and the media type

application/json-seq is a registered media type with no required or optional parameters, and the character set is fixed to UTF-8. On HTTP, three practices matter:

• Send Accept: application/json-seq and respond with Content-Type: application/json-seq;
• To stream for real, the server must use chunked transfer and disable response buffering in intermediaries such as reverse proxies, CDNs, and load-testing tools;
• A client reading chunk by chunk may receive a chunk that ends in the middle of a record, so the parser must buffer until the next RS before emitting a record.

If the other side only understands line-delimited input, do not force JSON-seq on it. It is a standard, but its ecosystem penetration is far below NDJSON, so compatibility should be decided during protocol negotiation rather than after a parse failure.

7. Command line, logs, and tool support

jq is one of the few tools with native support; its --seq flag exists precisely for RFC 7464:

复制printf '\x1e{"id":1}\n\x1e{"id":2}\n' | jq --seq -c '.id'

That prints 1 and 2 on separate lines. For a streaming endpoint, remember to disable curl's buffering:

复制curl -sN https://api.example.com/events | jq --seq -c '.type'

jq --seq reads RFC 7464 input, ignores bad records, and keeps parsing. That is particularly useful when tailing a log file that is still being written: the last record may be half-written, and the output will not show a fake trailing object.

7.1 Native support by environment

7.2 Common pitfalls

The mistakes cluster on the reading side:

1. Reading with readline, which leaves the invisible RS prefix on the first line so JSON.parse always fails;
2. Opening a .json-seq file in an editor where control characters are invisible, so it looks like one bizarre JSON document;
3. Splitting with split(/\n/), which conflates escaped newlines inside a record with the LF between records;
4. Treating RS as an ordinary character in documentation, so it disappears after copy and paste.

8. When it is worth using

Good fits:

• You own both the producer and the consumer, with no third-party tools in between;
• The data is a continuously appended event stream and the process may crash, so you need recovery from truncation;
• The protocol deserves a formal media type rather than a convention like .jsonl;
• Individual records may be top-level numbers or booleans, where an explicit truncation rule matters.

Poor fits:

• Logs must go into ELK, Loki, or another system that only understands line-delimited input;
• People need to cat, grep, and awk the data directly;
• The other side can only parse application/json.

9. Checklist

• [ ] The writer generates records with JSON.stringify, then prepends 0x1E and appends \n;
• [ ] Records holding top-level numbers, booleans, or null end with LF;
• [ ] The reader splits on bytes rather than reading line by line;
• [ ] Streaming reads keep an unclosed tail in the buffer until the next RS arrives;
• [ ] Parse failures are counted and skipped, never fatal to the sequence;
• [ ] Public interfaces declare application/json-seq explicitly and keep an NDJSON fallback.

FAQ

Should I adopt JSON-seq for a public API endpoint?

Only when both ends can be changed or the format is negotiated up front. JSON-seq is a registered standard with the application/json-seq media type, but its ecosystem penetration is far below NDJSON, so forcing it on a third-party client usually fails. Decide compatibility during protocol negotiation and keep an NDJSON fallback.

Should I convert an existing NDJSON pipeline to JSON-seq?

Usually not. NDJSON already streams, appends, and skips bad lines, and nearly every log and big-data tool accepts it. Convert only when you need the IANA-registered media type or the resynchronisation that a leading delimiter provides, and only when the producer and the consumer can be changed together.

Which is more common, JSON Text Sequences or NDJSON?

NDJSON. Its rules are simpler, its tool coverage is far wider, and log systems and big-data pipelines generally accept it. JSON-seq wins on the IANA-registered media type and on the resynchronisation that comes from putting the delimiter first, which suits controlled streaming protocols rather than general data exchange.

Why is the delimiter placed before the record rather than after it?

Because records can be truncated. With a leading delimiter, a reader that sees RS knows a new record has begun, so the previous record cannot corrupt the next one whether it was complete or not. With a trailing delimiter, an interrupted write leaves a final segment that has neither an ending marker nor a following start marker, so the boundary can only be guessed.

What happens if 0x1E appears inside a record?

A valid JSON string must escape RS as \u001e, so a raw 0x1E should never appear. If it does, the text is no longer valid JSON: splitting on 0x1E will break at the wrong place and JSON.parse will fail. The fix belongs on the writer side: use JSON.stringify instead of hand-built text.

Can a browser parse application/json-seq directly?

No. Browsers ship no JSON-seq parser; fetch only delivers the bytes. You need to use response.body.getReader(), accumulate chunks until the next 0x1E, and call JSON.parse on each segment. For small payloads, reading the whole body as a string first and splitting afterwards also works.

Why does a top-level number need a trailing LF?

Objects, arrays, and strings are self-delimiting in JSON, while numbers are not: <RS>123<RS> could be a complete 123 or a truncated 1234. RFC 7464 therefore requires a whitespace byte after each record, and the LF acts as a canary for truncation detection. The same rule applies to top-level true, false, and null.