Versioning JSON Contracts: Compatibility, Deprecation, and Data Migration

Bottom line: run version evolution as expand, backfill, then contract, and persist the version inside the document; confirm consumers tolerate unknown fields and enum values before shipping anything, and take deprecation through freeze, announce, observe, and remove in that order.

The hard part of versioning is not changing code; it is that you cannot know who still reads the old field. Compatibility is never a one-way promise — it is an ongoing alignment between the publisher and unknown consumers.

1. Compatibility boundaries: additive is not automatically safe

Textbooks call a new optional field "backward compatible", but that only holds when consumers tolerate unknown fields. Adding a field still breaks clients in at least three situations:

• the consumer validates with a schema that sets additionalProperties: false;
• the consumer walks the object in a fixed order, or reads values by position instead of by key;
• the new field reuses an old name but changes its meaning, for example widening status from "order status" to "fulfillment status".

A safe addition needs three preconditions: the field is optional, its meaning is independent, and consumers ignore unknown keys. Confirm all three before you even discuss a version number.

复制{
  "schemaVersion": 2,
  "orderId": "ord-8842",
  "status": "paid",
  "refundDeadline": "2026-11-30T00:00:00+08:00",
  "channel": "wechat"
}

Here refundDeadline and channel are optional v2 additions, while orderId and status keep their original semantics. schemaVersion is persisted with the document rather than living only in an HTTP header.

2. Breaking changes you cannot ship

Treat the following changes as red lines that require a new version or a dual-write transition. Updating the docs will not save you:

Changing meaning is the most dangerous row: schema validation stays green, monitoring reports no errors, and only business metrics drift. Any change in meaning should be shipped as a new field.

3. Where the version lives: document, URL, media type

The three common placements serve different purposes:

A practical division of labor: use media-type negotiation on the wire, and always persist a version field in the document. Old messages in a queue will not upgrade because you changed a URL, and neither will backup files. Only an in-document schemaVersion gives replay, backfill, and audits a foundation.

复制GET /orders/ord-8842 HTTP/1.1
Host: api.bbdrug.com
Accept: application/vnd.acme.order.v2+json

If you do version the URL, at least keep the old path returning the old shape for the entire deprecation window instead of quietly pointing it at the new implementation. For broader API contract decisions, see Key points for designing JSON API data contracts.

4. Opening a deprecation window

Deprecation is not a single announcement; it is a window with a start, an observation period, and an end. A workable rhythm:

1. Freeze: stop writing the old field, but keep reading it.
2. Announce: mark the removal date in the docs, the changelog, and a response header at the same time.
3. Observe: count reads of the old field per caller until it stays at zero for two consecutive periods.
4. Remove: only after the window closes, reads are zero, and regression tests pass, delete the field and its compatibility code.

The Sunset response header (RFC 8594) gives clients a machine-readable removal date. Deprecation is still an IETF draft: usable, but do not assume every client understands it. Pair it with a Link header pointing at the migration guide:

复制Deprecation: true
Sunset: Wed, 31 Mar 2027 23:59:59 GMT
Link: <https://bbdrug.com/docs/deprecations/order-status>; rel="deprecation"; type="text/html"

Window length is a team convention. A common floor is one release cycle or 90 days, whichever is longer. Public APIs should get more, internal services can get less, but never less than the time every caller needs to upgrade.

5. Finding consumers before you deprecate

Before deprecating, you must be able to answer "who still reads this?" Otherwise the window is self-deception. Rank the evidence by reliability:

• Gateway logs: aggregate old-field access by appId, User-Agent, and client version.
• Distributed traces: tag the old field in the serialization layer so a trace shows the calling service name directly.
• Queue consumer registry: the list of consumer groups is more reliable than a code search.
• Repository search: rg only covers your own code; external callers still require logs.

复制rg --type ts --type js 'legacyStatus|statusCode' src/ --stats

A search hit proves "someone in this repository reads it", not "nobody else does". Export the gateway logs day by day and confirm that old-field reads have stayed at zero before removing anything.

6. Data migration: expand, backfill, contract

Martin Fowler's Parallel Change describes three steps: expand (old and new fields coexist and both versions can be read), backfill (move historical records to the new shape), and contract (remove the old field and compatibility branches). Do not reverse the order, or you will have no rollback path.

复制// Migration skeleton: cursor by primary key, conditional writes for idempotency
async function migrateBatch(collection, cursor, batchSize = 500) {
  const records = await collection
    .find({ _id: { $gt: cursor } })
    .limit(batchSize)
    .toArray();

  if (records.length === 0) {
    return { nextCursor: cursor, done: true };
  }

  const operations = records
    .filter((record) => record.schemaVersion < 2)
    .map((record) => ({
      updateOne: {
        filter: { _id: record._id, schemaVersion: record.schemaVersion },
        update: {
          $set: {
            schemaVersion: 2,
            status: record.legacyStatus ?? record.status,
            refundDeadline: record.refundDeadline ?? null,
          },
        },
      },
    }));

  if (operations.length > 0) {
    await collection.bulkWrite(operations);
  }

  return { nextCursor: records.at(-1)._id, done: false };
}

Details that are easy to miss:

• Idempotency: include the old version number in the filter, so re-running never rewrites an already migrated record.
• Batching and checkpoints: advance by _id cursor, store a checkpoint, and resume from it after a failure.
• Rehearsal: run the script against a production snapshot and compare record counts and checksums before and after.
• Rollback: keep the old field until the new field passes validation; switch read paths with a flag and never roll back by editing schemaVersion alone.
• Validation: run business constraints after migration (non-negative amounts, legal state transitions), not just a JSON parse check.

Configuration data follows the same process; see Managing JSON for configuration, fixtures, and local data for how to organize those files.

7. Contract tests: put compatibility in CI

Compatibility cannot depend on code review memory; it needs tests. Two layers are enough:

1. Shape tests: validate responses with JSON Schema, but do not set additionalProperties: false on schemas used by consumers.
2. Consumer-subset tests: extract the fields the old consumer truly relies on into a sub-schema and assert that new responses still satisfy it.

复制// Contract test: new responses must still satisfy the old consumer subset
import Ajv from 'ajv';
import assert from 'node:assert/strict';
import { test } from 'node:test';

const ajv = new Ajv({ strict: false });
const validateLegacySubset = ajv.compile(legacyConsumerSubsetSchema);

test('a v2 response still satisfies the v1 consumer subset', () => {
  const response = buildOrderResponse({ schemaVersion: 2 });
  const { schemaVersion, ...legacyFields } = response;
  const isValid = validateLegacySubset(legacyFields);

  assert.equal(isValid, true, JSON.stringify(validateLegacySubset.errors));
});

The sub-schema declares only the fields and types the old consumer reads and lets every new field through. That catches removed fields and type changes without false alarms on additions. Keep case data in a fixtures directory, ordered and diffable; for merge conflicts in those files, see JSON file Git conflicts: union writes broken JSON, merge by key instead.

8. Canary rollout: scale by dimension, stop on metrics

Compatibility problems rarely surface in unit tests; they surface in a canary. Widen exposure from narrow to broad:

Watch at least three metrics: API error rate, contract validation failure rate, and the read ratio between old and new fields. Define automatic stop conditions, such as 5xx above 1% or a sudden rebound in old-field reads, and roll back the flag. Keep dual writes on during the canary so rollback only requires switching the read path, not repairing data.

9. A compatibility checklist before shipping

• [ ] New fields are optional, semantically independent, and do not reuse old names.
• [ ] Consumers handle unknown fields and unknown enum values with a fallback.
• [ ] The version field is persisted with the document and is negotiable on the wire.
• [ ] The deprecation window has an announcement, a Sunset header, and observation.
• [ ] Old-field reads can be traced to specific callers through logs or traces.
• [ ] The migration script is idempotent, batched, resumable, and has a rollback path.
• [ ] Contract tests cover the legacy consumer subset and run in CI.
• [ ] The canary has explicit exposure dimensions and stop metrics.

FAQ

Is adding an optional field always backward compatible?

No. It is compatible only when consumers ignore unknown fields and avoid strict schema validation. A consumer using additionalProperties: false or reading by position will still fail. Verify their tolerance with a contract test before shipping.

Should schemaVersion live in the URL or the response body?

The wire can negotiate versions through media types or URL paths, but stored records must carry schemaVersion inside the document. Queues, offline backups, and audits never pass through a URL, so only an in-document version can support replay and backfill.

Can a deprecated field be deleted immediately?

No. Stop writing it, announce the removal date, and count reads per caller. Delete only after the window closes and reads stay at zero. Before deletion, run a full regression to confirm no hidden cron job or report still reads it.

What if a migration script fails halfway through?

Design it as an idempotent job: conditional updates, a primary-key cursor, and checkpoints. Resume from the checkpoint after a failure, and repeated runs must not write twice. For rollback, keep the old field and a snapshot; do not just revert schemaVersion.

How should I sequence a data migration?

Follow Parallel Change: expand first so old and new fields coexist and both versions can be read, backfill historical records into the new shape, then contract by removing the old field and compatibility branches. Do not reverse the order, or you lose the rollback path, and make the migration script idempotent, batched, and resumable.