The underwriters asked for one change. Today a recommendation cites rules, R-ROOF-03 for example, but it does not say which evidence each reason rests on: the broker's email, the roof assessment, a passage of guidance. They want that on screen, so they can open the evidence instead of searching for it. The design is simple. Version 2 of the recommendation gets a new field, evidence_ids, a list such as ["EV-EMAIL-1004", "EV-ROOF-0412"], and it is required: a recommendation without its evidence is what the change is meant to stop.
The team makes the obvious move. They change the agent's schema and instructions on Monday, and the underwriting agent starts writing version 2. Within the hour the underwriter cockpit stops showing new recommendations: it still reads with version 1, which is a closed object, and every new recommendation now carries a field it has never heard of. The team quickly ships a version 2 reader to the cockpit. Now the new ones show, and every recommendation stored before Monday fails to open, because it has no evidence_ids. Nothing about the data was wrong. The order of the change was.
Plain English. Schema evolution is changing a contract while data written under the old version still exists, and while producers and consumers that know only the old version are still running. For example: adding evidence_ids while last month's recommendation records sit in the database and the cockpit has not been redeployed yet.
Who still lives on the old version#
A contract has three kinds of party, and they never all change at the same moment:
| Party | In our example | Changes when |
|---|---|---|
| Producers | the underwriting agent (its prompt, model setting, schema) | the agent is redeployed |
| Consumers | the underwriter cockpit, the audit export, a reporting job | each is redeployed on its own schedule |
| Stored records | every recommendation record written so far | never, unless someone migrates them |
The third row is the one teams forget. A message on a queue lives for minutes. A stored recommendation record lives for years: it may be reopened when the underwriter reviews a renewal, or read back by an auditor. The contract has to keep working for all of them.
To talk about this precisely, two words with a direction:
Backward compatible means a new consumer can read old data. For example: a version 2 cockpit opens a version 1 recommendation stored last month.
Forward compatible means an old consumer can read new data. For example: a version 1 cockpit, not yet redeployed, opens a version 2 recommendation written this morning.
"Compatible" alone means nothing until you say which direction.
Before AI. Kafka teams meet this every week. A schema registry for Kafka topics names these directions as compatibility types. Its documentation (read on 2026-11-17) says BACKWARD means a "consumer using schema X can process data produced with schema X or X-1", FORWARD means "data produced using schema X can be read by consumers with schema X or X-1", that BACKWARD is its default, and that under BACKWARD you "upgrade all consumers before you start producing new events". The ideas on this page are the same ones. What is different with a model as the producer is that the producer can drift without anyone redeploying it, and that every change of the schema is also a change of the instructions the model reads.
What one change breaks#
The example classifies the change from version 1 to version 2 with schema_changes() in examples/uw-contracts/evolve.py. Here is the real output, run in a scratch copy:
added property evidence_ids breaks forward
evidence_ids is required breaks backward
One small change, two breaks, in opposite directions. Adding a property breaks forward compatibility only because version 1 is a closed object (additionalProperties: false): an old strict reader refuses any field it does not know. Making the property required breaks backward compatibility: a new reader refuses old data that lacks it.
A reader is not only a schema version. It is a version plus a style. Four readers in the example:
| Reader | What it does |
|---|---|
v1_strict | version 1, closed object: rejects any field it does not know |
v1_tolerant | version 1, ignores unknown fields |
v2_strict | version 2: evidence_ids is required |
v2_upcast | version 2, but turns a version 1 message into version 2 on read, marking the evidence as unknown |
Plain English. A tolerant reader is a consumer that ignores what it does not know and needs only what it uses. For example: the reporting job counts recommendations by word; it never looks at evidence_ids, so it should not fail because the field appeared.
Plain English. An upcast turns an old message into the new shape at the moment it is read, and says what is unknown. For example: a version 1 recommendation read by a version 2 reader gets evidence_ids: [] and a note that the evidence was never recorded. It does not get evidence ids guessed from the reasons.
Each producer version against each reader, from the real run:
producer reader outcome
v1 v1_strict ok reads it
v1 v1_tolerant ok reads it
v1 v2_strict break missing required property "evidence_ids"
v1 v2_upcast upcast evidence_ids unknown: upcast from version 1; the evidence was never recorded
v2 v1_strict break property "evidence_ids" is not allowed
v2 v1_tolerant ok reads it
v2 v2_strict ok reads it
v2 v2_upcast ok reads it
Two cells break, and they are the two stories of Monday. v2 against v1_strict is the cockpit going blank. v1 against v2_strict is the stored records failing to open. This is case UW-S19, one of this site's own cases: a valid version 1 recommendation read by a consumer that expects version 2.
Diagram: Two producer versions, four readers · two cells break, in opposite directions
Each row is what the agent writes; each column is how a consumer reads. The two breaks are the two failures of a careless rollout: an old strict reader meets a new field, and a new strict reader meets an old record.
- Agent writes version 1
- ok: Version 1 read by its own strict reader.
- ok: Nothing unknown to ignore.
- break: missing required property evidence_ids: a new reader cannot read old data.
- upcast: evidence_ids becomes empty with a note that the evidence was never recorded.
- Agent writes version 2
- break: property evidence_ids is not allowed: an old closed reader cannot read new data.
- ok: The unknown field is ignored; safe only because nothing it uses changes meaning.
- ok: Version 2 read by its own strict reader.
- ok: A version 2 message needs no upcast.
Common confusion. "Be strict on what you accept" (the closed object of the schema design page) and "ignore what you do not know" (the tolerant reader) look like opposite advice. They apply to different readers. The validator that checks what the model wrote, right after it wrote it, should be strict: an unknown field there is the model inventing something, likepremium_usdinUW-S07. A consumer reading records that already passed validation, possibly written under a newer version, can be tolerant about fields it does not use. Strict at the producer's boundary; tolerant, where safe, downstream.
Where tolerance is unsafe#
A tolerant reader is safe only when the field it ignores cannot change the meaning of the fields it uses. Three cases where that fails:
- A new field that qualifies an old one. If version 3 added
recommendation_scope: "partial"to say "this approve covers the building only", a tolerant version 2 reader would show a plain approve. Ignoring the field changes what the reader thinks the record says. - A new enum value. If the provider of the roof assessment starts sending
status: "partial"(caseUW-S20), a reader that maps unknown statuses tookturns "only part of the roof was visible" into a confirmed age. The example rejects the value at the tool boundary, counts it as drift and degrades torequest_info. - A required field for this consumer. The cockpit's evidence panel needs
evidence_ids. A tolerant reader that shows an empty panel for a version 1 record must say "not recorded", not "none".
Adding a required field without breaking anyone#
The fix for Monday is not a better schema. It is an order of steps, usually called expand and contract. First expand: make every reader accept both shapes. Then move the producer. Then deal with the stored data. Only then contract: make the new field required.
Here are two rollout orders from the example, step by step:
producer_first
step 1: agent writes v1, cockpit reads with v1_strict live ok stored ok
step 2: agent writes v2, cockpit reads with v1_strict live break stored ok
step 3: agent writes v2, cockpit reads with v2_strict live ok stored break
expand_contract
step 1: agent writes v1, cockpit reads with v1_strict live ok stored ok
step 2: agent writes v1, cockpit reads with v2_upcast live upcast stored upcast
step 3: agent writes v2, cockpit reads with v2_upcast live ok stored upcast
step 4: agent writes v2, cockpit reads with v2_upcast live ok stored ok
step 5: agent writes v2, cockpit reads with v2_strict live ok stored ok
In expand_contract, no step breaks. Step 2 is the expand: the cockpit learns version 2 but upcasts version 1, so it reads both. Step 3 moves the agent. Step 4 is the migration: stored version 1 records are rewritten as version 2, with the upcast's marker that their evidence is unknown. Step 5 is the contract: only now does the cockpit require evidence_ids, because no live or stored record lacks it.
Diagram: Expand, move, migrate, contract · adding a required field with no step that breaks
Read left to right. Readers learn both shapes first, the producer moves second, stored records are migrated third, and the field becomes required last, when nothing that exists lacks it.
- 1 · Expand
- Readers accept both: The cockpit reads version 2 and upcasts version 1 on read.
- 2 · Move the producer
- Agent writes v2: The schema and instructions change; watch the failure rate of the new field.
- 3 · Migrate
- Rewrite stored records: Old records get evidence_ids empty and a marker that they were migrated.
- 4 · Contract
- Readers require it: Only now is evidence_ids required; nothing live or stored lacks it.
Plain English. A migration rewrites stored data from the old shape to the new one. For example: every stored version 1 recommendation record gets evidence_ids: [] and a marker that says it was migrated. A migration is a one-way change to data people rely on, so it is versioned, tested on a copy, and reversible where possible.
A breaking change is any change that makes some live reader or some stored record fail. Expand and contract does not make a breaking change safe by itself; it splits it into steps that are each non-breaking, in an order where every reader can read everything that exists at that moment.
In our insurance agent. The model is a producer that changes when its instructions change. Addingevidence_idsto the schema is also an instruction to the model to provide evidence, and the domain stage of the validation page checks those ids against what the run really gathered (caseUW-S14). So step 3 of the rollout is a change of behaviour as well as of shape. Run it behind a flag, watch the validation failure rate for the new field, and only then go to step 5.
Try it: three rollout orders#
The lab shows the grid and lets you step through three orders: producer first, strict consumer first, and expand-migrate-contract. Watch the stored old record as well as the live message: the strict-consumer-first order breaks the live message at step 2, and the producer-first order breaks stored records at step 3.
| Order | Step that breaks | What breaks |
|---|---|---|
| Producer first | step 2, then step 3 | the cockpit cannot read new recommendations; then it cannot read stored old ones |
| Strict consumer first | step 2 | the cockpit requires evidence_ids before the agent sends it |
| Expand, migrate, contract | none | every reader reads every live message and every stored record at every step |
Version numbers and where they live#
A version must be written somewhere a reader can see it before it parses the rest. Three common places:
| Where | Example | Fits |
|---|---|---|
| A field in the payload | "schema_version": 2 | stored records; the reader picks the schema from the field |
| The envelope around it | the payload_type recommendation.v2 of the handoff envelope | messages between agents (the boundaries page) |
| The schema's own id | $id ending in recommendation.v2.json | registries and documentation |
In this example the recommendation the model writes carries no version field: the model should not be trusted to say which version it followed. The workflow knows which schema it validated against and stamps that into the record and the envelope. The record has its own schema_version, which evolves separately from the recommendation inside it.
What this is not. Durable storage of records and how an agent's memory or checkpoints are versioned belong to memory.sammuthu.com. Messages between agents as a protocol, with its own versioning rules, belong to a2a.sammuthu.com. This page is about the shape of one contract over time.
For a coding agent. A coding agent'schange_reportgains a requiredcoverage_deltafield. CI is the consumer and old CI runs are the stored records. Deploy CI that accepts reports with or without the field first, then update the agent's instructions, then make the field required in CI. The CI side of that is described on diff-to-deploy.sammuthu.com.
Checklist#
- For every change, write down which direction it breaks: forward, backward, both or neither.
- Validate strictly where the model's output enters; read tolerantly downstream only where an ignored field cannot change meaning.
- Add a required field in this order: readers accept both, producer moves, stored data migrates, readers require.
- Upcast old data with an explicit "unknown" marker; never infer missing values from other fields.
- Stamp the schema version in code (record, envelope), not in what the model writes.
- Keep every version of every schema, and the code that reads it, as long as records of that version exist.
- Treat a schema change as an instruction change for the model: roll it out behind a flag and watch its failure rate.
Adding and removing enum valuesThe most common breaking change that does not look like one
Adding a value to an enum breaks forward compatibility: an old reader with the old list rejects the new word. Removing a value breaks backward compatibility: stored records still contain it. For the recommendation's four words, a fifth (say refer) needs the same expand and contract: readers learn it first, then the model may use it. For enums that come from outside, like the roof-data provider's status, you cannot sequence the producer, so the reader must have an explicit rule for unknown values. In the example that rule is: reject, count as drift, degrade to request_info (UW-S20).
Closed objects and forward compatibilityThe price of additionalProperties false
A closed object makes every added field a forward-breaking change for every strict reader. That is a real cost, and some teams leave objects open for that reason. On this site the model's output is closed anyway, because an unknown field from a model is far more often an invention than a newer version: there is no older producer that could send it. Payloads between services written by code are different: there an open object with a tolerant reader is often the better trade. Decide per boundary, and write the decision down next to the schema.
Replaying old records through new validatorsA migration test that already exists in your logs
If the workflow keeps replay artifacts (the raw output, the schema and the facts it was validated against, see observability), a new schema version can be tested against months of real outputs before release: how many old outputs would the new validator reject, and why? That is the most honest compatibility test available, because it uses what the producer really wrote, not what the team imagines it wrote.
In one breath#
A contract changes while old producers, old consumers and old stored records all still exist, so every change has a direction: backward means new readers can read old data, forward means old readers can read new data. Adding a required field breaks both unless you sequence it: readers accept both shapes, the producer moves, stored records migrate with an explicit unknown marker, and only then do readers require the field. Validate strictly where the model's output enters, read tolerantly downstream only where an ignored field cannot change meaning, and stamp versions in code, never in what the model writes.
You can now change a contract without breaking the cockpit or the archive. But a contract that never breaks can still carry an attack: a reason string that passes every stage can hold markup or an instruction aimed at whatever reads it next. That is the next page, security.