Case UW-S17, one of this site's own scripted cases, starts from the email of UW-003. The broker writes about a frame retail unit built in 1988, 12,000 square feet, not sprinklered, wind zone B, one small loss in 2023, and then: "The owner does not know the roof age." The intake agent fills the submission and writes "roof_age_years": 0.
Every check on the shape passes. roof_age_years is an integer between 0 and 150. Zero is in range. And zero means something: a roof replaced this year, the best possible roof, the opposite of "nobody knows". If that value travels on, the rule check sees a new roof, nothing triggers R-ROOF-02 ("roof age unknown: request it"), and the broker is never asked.
The model did not do this out of malice. The schema let it. The field was a plain integer, nothing in its name or description said what to write when the email is silent, and nothing asked where the value came from. This page is about designing contracts so that a model is led to the right value and a check can catch the wrong one.
A schema has two readers#
Plain English. A field description is the sentence in a schema that says what a field means, in words. For example: "Roof age stated on the application. Required but nullable: null means nobody said; 0 would mean a new roof."
The validator reads the types, enums and limits. The model reads the names and descriptions too: when a provider constrains output to a schema, or when the schema is in the instructions, every name and description is text the model sees while writing. The JSON Schema specification describes title and description as annotations that "decorate a user interface with information about the data", where "a description will provide explanation about the purpose of the instance". Here the "user interface" is the model.
So a schema has two readers, and good design serves both:
| Choice | For the validator | For the model |
|---|---|---|
roof_age_years instead of roof | one meaning per field | the unit and the thing are in the name |
area_sq_ft instead of area | (none: an integer is an integer) | says which unit to write, and invites a conversion when the email says square metres |
building_value_usd, integer | rejects "2.4M" | says whole dollars, not a phrase |
an enum for occupancy | rejects anything outside six words | shows the six words, so the model picks rather than invents |
| a description on every field | (ignored) | says what to write when the source is silent |
Plain English. A unit is the scale a number is measured in. For example: square feet versus square metres, years versus months. A number without its unit is half a fact: 1,858 square metres and 1,858 square feet differ by a factor of almost eleven.
Put the unit in the name of every number. Descriptions can be lost (a tool that strips annotations, a model that skims); names cannot.
Enums over free text#
A field that has a small set of meaningful values should be an enum, not a string. For example: recommendation has four words. As free text the model would write "approved", "Escalate to senior", "refer", and every consumer would need its own guess at what each means. As an enum, the schema stage rejects anything else and the re-ask names the four words.
The cost of an enum is evolution: adding a fifth word is a change for every reader (see evolution). The cost of free text is paid on every single output. For any field a program will branch on, the enum wins. Keep free text for what only a person will read, like reasons, and treat it as untrusted (see security).
Required, optional, nullable: three different things#
These three words are used loosely in most codebases. In a contract with a model as producer they must be exact.
Plain English. A required field is a key that must be present. For example: roof_age_years must always be in the submission.
Plain English. An optional field is a key that may be absent. For example: basement may be left out when the email does not mention it.
Plain English. A nullable field may hold null. For example: roof_age_years may be null, which means "nobody said".
They combine, and each combination says something different:
| Combination | Means | In the catalog |
|---|---|---|
| required, not nullable | always known; a missing value is an error | submission_id, wind_zone, year_built, sprinklered |
| required, nullable | the producer must state whether it knows; null is an answer | roof_age_years (the only one) |
| optional, not nullable | absent when not mentioned; never null | basement, stories, roof_material |
| optional, nullable | absent when not considered; null when looked for and not found | provider_roof_age_years, roof_evidence_id |
roof_age_years is the one field that is both required and nullable, and that is deliberate. A rule depends on it (R-ROOF-02 fires when it is unknown). Making it optional would let the intake agent leave it out without anyone noticing whether it looked. Making it required but not nullable would force a number where there is none, which is how 0 happened. Required and nullable says: you must answer, and "unknown" is an allowed answer. The description says the rest: null means nobody said; 0 would mean a new roof.
Common confusion. "Optional" is often used for "nullable". In Python,Optional[int]means "int or None", which is nullable, and says nothing about whether the key may be missing. In Java, a field of a record is always present after reading, and an absent JSON key often arrives asnull. The typed models page shows where Pydantic and the Java stack tell absent andnullapart, and where they do not.
The 63 fields, by who fills them#
The submission has 63 fields in the example's catalog, examples/uw-contracts/field_catalog.json. Every field has a source, and the source decides who may write it:
fields: 63
by source: {'code': 5, 'email': 44, 'provider': 11, 'derived': 3}
required: 16 | nullable: 8 | required and nullable: ['roof_age_years']
| Source | Count | Who writes it | Examples |
|---|---|---|---|
email | 44 | the intake agent (a model), quoting the broker's email | occupancy, year_built, area_sq_ft, roof_age_years |
provider | 11 | a tool result from an outside provider | provider_roof_age_years, flood_zone, roof_evidence_id |
code | 5 | the workflow, stamped | submission_id, received_at, intake_run_id, source_email_id |
derived | 3 | the workflow, computed from other fields | building_age_years, roof_age_gap_years |
The design rule behind the table: the model fills what only the model can know; code stamps what code knows. Only a model can read "The owner says the roof was replaced 9 years ago" and turn it into 9. Only code knows which run this is, when the email arrived and which evidence id the email has. A model asked to write submission_id will usually copy it correctly and will sometimes invent one; there is no reason to take that risk. In the example the domain stage checks every code-stamped field against the run's own values (D-STAMP), and the recommendation record's identifiers are stamped by build_record(), never taken from the model.
Derived fields follow the same logic. roof_age_gap_years (application versus provider) is arithmetic. The model must not compute it, because a model's arithmetic is a guess and the rule R-ROOF-03 depends on it.
| Group | Fields | Required | From the email | From a provider | Stamped or derived by code |
|---|---|---|---|---|---|
| identity | 5 | 5 | 0 | 0 | 5 |
| broker | 3 | 1 | 3 | 0 | 0 |
| insured | 4 | 1 | 4 | 0 | 0 |
| location | 10 | 1 | 5 | 5 | 0 |
| building | 9 | 4 | 9 | 0 | 0 |
| roof | 7 | 1 | 4 | 3 | 0 |
| protection | 7 | 1 | 4 | 3 | 0 |
| values | 6 | 1 | 6 | 0 | 0 |
| history | 4 | 1 | 4 | 0 | 0 |
| coverage | 5 | 0 | 5 | 0 | 0 |
| derived | 3 | 0 | 0 | 0 | 3 |
The table above counts the catalog by group; the lab lets you search and filter all 63 rows.
Diagram: Who fills the 63 fields · the model fills what only it can know, code stamps the rest
Read left to right. Three sources feed one submission: the model reads the email, tools return provider data, and code stamps identifiers and computes derived values. Each source has its own check.
- Sources
- Broker's email: The intake agent reads it and quotes the words each value came from.
- Provider results: Tool results from outside providers, each with an evidence id.
- The workflow: Identifiers stamped and arithmetic computed by code.
- Checks
- Provenance check: Each email value must quote words that are in the email and agree with them.
- Stamp check: A code-stamped field must equal the run's own value.
- Result
- Submission: 63 fields, 16 required, each with a known source.
Provenance: where each value came from#
Plain English. Provenance is the record of where a value came from: the source, the evidence id, and the exact words it was read from. For example: area_sq_ft came from the email EV-EMAIL-1004, from the words "1,858 square metres", converted from square metres.
Plain English. An evidence id is a short identifier of one piece of evidence the run gathered. For example: EV-EMAIL-1004 is the broker's email; EV-ROOF-0412 is the provider's roof assessment.
The submission carries a provenance map: for every field filled from the email, an entry with source, evidence_id and quote. That turns three invisible mistakes into checkable ones. The domain stage of the intake contract checks that each email value has an entry, that the quote really appears in the email, that a number in the value appears in the quote, and that a quote in square metres is declared as a conversion. Here are the two intake cases of the example, attempt by attempt (real run in a scratch copy):
UW-S17 attempt 1: roof_age_years=0 first failure domain | /roof_age_years D-PROVENANCE-MISSING
UW-S17 attempt 2: roof_age_years=None first failure - |
UW-S18 attempt 1: area_sq_ft=1858 building_value_usd=2.4M first failure schema | /building_value_usd type
UW-S18 attempt 2: area_sq_ft=1858 building_value_usd=2400000 first failure domain | /area_sq_ft D-UNIT
UW-S18 attempt 3: area_sq_ft=20000 building_value_usd=2400000 first failure - |
UW-S17: the 0 has no provenance entry, because there are no words in the email it could quote. The re-ask says "a value with no source: say where it came from, or use null", and attempt 2 writes null. Now the rule check sees an unknown roof and recommends asking for it.
UW-S18: the email of SUB-1004 gives the floor area as "1,858 square metres" and the value as "about 2.4 million dollars". Attempt 1 writes "2.4M", and the schema catches the type. Attempt 2 writes 2400000 and 1858 for area_sq_ft: both integers, both in range, both schema-valid. Only provenance catches the unit: the quote says square metres, the field holds square feet, and no conversion was declared. Attempt 3 declares the conversion, and the check recomputes it (1,858 square metres is about 20,000 square feet, within one percent):
{"source": "email", "evidence_id": "EV-EMAIL-1004", "quote": "1,858 square metres", "unit_conversion": "sq_m_to_sq_ft"}
A schema checks shape, not truth. Provenance is what lets a check compare the shape with the source.
In our insurance agent. The underwriter cockpit shows each submission field with its quote on hover: "1,858 square metres (converted)". An underwriter who doubts a value reads the broker's words in one second, instead of opening the email and searching.
Diagram: UW-S18 attempt by attempt · the schema catches the string, only provenance catches the unit
Read left to right. Attempt 1 fails the schema on a phrase. Attempt 2 is schema-valid and still wrong by a factor of almost eleven; the quote reveals square metres. Attempt 3 declares the conversion, and the check recomputes it.
- Attempt 1
- "2.4M": building_value_usd must be an integer.
- Attempt 2
- area_sq_ft 1858: An integer in range, so the schema passes; the quote says square metres.
- Attempt 3
- area_sq_ft 20000: The provenance entry says sq_m_to_sq_ft; the check recomputes it within one percent.
Closed objects#
Plain English. A closed object is an object that allows only the fields its schema names; any other key is an error. In JSON Schema that is additionalProperties: false. For example: the recommendation rejects premium_usd (case UW-S07).
For a model's output, closed is the right default. An extra key from a model is almost never a newer version of the contract; it is the model adding something nobody asked for, and sometimes something it must not do, like quoting a price. The cost is on the other side: every added field becomes a forward-breaking change for strict readers, which the evolution page shows how to sequence.
Before AI. A relational table is a closed object: anINSERTwith a column that does not exist fails. ANOT NULLcolumn is "not nullable". A column with aDEFAULTis close to "optional". Most of this page is what a careful database designer already does; what is new is a producer that reads your column comments and is tempted to fill aNOT NULLcolumn with a plausible zero.
What this is not. How the instructions around the schema are written (the prompt that asks for the output) belongs to prompt.sammuthu.com. Which evidence the model can see when it fills a field belongs to context.sammuthu.com. This page designs the contract itself.
For a coding agent. A coding agent'schange_reporthasfiles_changed,tests_run,tests_passed,riskandneeds_review. Same rules:riskis an enum, not a sentence;needs_reviewis not the agent's to decide (in the example schema it isconst: true); and the commit hash and pipeline id are stamped by CI, never written by the agent. How CI consumes it is on diff-to-deploy.sammuthu.com.
Checklist#
- Put the unit in the name of every number (
_years,_sq_ft,_usd,_pct) and write a description for every field. - Use an enum for every field a program will branch on; keep free text for people, and mark it untrusted.
- Decide required, optional and nullable separately for every field; use required and nullable when a rule depends on "unknown".
- Say in the description what to write when the source is silent.
- Let the model fill only what it alone can know; stamp identifiers and compute derived fields in code.
- Require provenance (source, evidence id, verbatim quote, declared conversion) for every value read from a document.
- Close every object the model writes.
Flat or nested63 fields in one object, or groups
The catalog groups fields (identity, location, roof, values...) but the submission is one flat object. Nesting by group reads better and lets a group be optional as a whole, at the price of longer paths in every error and more places to put provenance. Providers' constrained modes also tend to support only a subset of JSON Schema, and deep nesting with unions is where support thins out. A practical rule: nest when a group has its own lifecycle or owner (a roof_assessment from a tool), stay flat when the fields are filled together from one source.
Per-field confidenceTempting, and usually the wrong field to add
A common idea is to let the model attach a confidence to every extracted field. The number is what the model says about itself, not a measured probability, and it tends to be high exactly where it should not be. Provenance is a better use of the same space: a quote can be checked; a confidence cannot. The recommendation keeps one stated_confidence, named so that nobody mistakes it for a measurement.
Large schemas and the model63 fields in one call, or several smaller extractions
A schema with 63 fields and long descriptions is a lot of text for the model to keep in mind while writing. Teams split such extractions into several calls by group, each with its own smaller schema, then merge in code. The merged object is validated against the full schema anyway. Whether one call or several gives better values for your documents is an evaluation question (eval.sammuthu.com), and it depends on the provider and model.
In one breath#
A schema is read by the validator and by the model, so names carry units, descriptions say what to write when the source is silent, and every field a program branches on is an enum. Decide required, optional and nullable separately, and use required-and-nullable where a rule depends on "unknown", so that unknown can never look like zero. Let the model fill only what it alone can know, stamp everything else in code, and make every value point at the words it came from, because provenance is what lets a check compare shape with truth.
You can now design a contract that leads the model to the right value and lets a check catch the wrong one. The next page puts the checks themselves in order: validation stages.