All chapters
  1. Start hereParseable is not valid. Valid is not correct
  2. Foundations · 1From free text to fields and records: why contracts matter
  3. Foundations · 2Syntax, schema, semantics, business validity: four different kinds of right
  4. Foundations · 3JSON and JSON Schema: types, enums, nullable fields, unions and nested objects
  5. Building · 4Typed models: Pydantic in Python, records and Bean Validation in Java
  6. Building · 5Provider output modes: prompting, JSON mode, constrained decoding and tool arguments
  7. Building · 6Tool input, tool result, agent handoff, final response: four contracts on one path
  8. Building · 7Schema design: names, required or optional, and where each value came from
  9. Operating · 8Validation stages: parse, schema, cross-field, domain rules
  10. Operating · 9Errors and repair: retry, clarify, correct within bounds, fall back
  11. Operating · 10Schema evolution: versions, compatibility and migrations
  12. Operating · 11Security: a valid string can still be an attack
  13. Operating · 12Observing contracts: validation failures, schema drift and replay artifacts
  14. Operating · 13Testing contracts: golden examples, property-based tests, fuzzing and contract suites
  15. Reference · 14Frameworks and standards: what each one checks for you
  16. Reference · 15Advanced: partial objects in a stream, polymorphism, multilingual data and typed events
  17. Reference · 16The insurance lab: one submission, four contracts, end to end
  18. ReferenceGlossary
  19. AboutAbout this site and its sources
Operating · 8

Validation stages: parse, schema, cross-field, domain rules

Four stages in a fixed order, each with its own error and its own owner. What each stage reads, what it returns, and why the order is not a matter of taste.

What you will understand
Four stages
Parse, schema, cross-field, domain: each answers one different question about the same output.
A fixed order
Why a later check run on a value of the wrong shape does not crash, it lies.
The error record
Stage, path, code, message: an error written so that code and people can both act on it.
Owners
Who writes each check, where it runs, and what it needs to know that the schema does not.

Case UW-S10 is one of this site's own cases (the UW-S cases are scripted outputs, built on the facts of the series' shared cases). The facts come from UW-002: a masonry warehouse, a roof of 8 years that the roof-data provider confirms, wind zone A, sprinklers. The underwriting agent writes this:

{"recommendation": "decline", "reasons": ["The roof is too old for wind zone rules under R-ROOF-01."], "citations": ["R-ROOF-01"], "missing": [], "stated_confidence": 0.66, "requires_human": true, "policy_version": "2026.2"}

It parses. Every field has the right type. decline is one of the four allowed words. R-ROOF-01 matches the pattern of a rule id. A decline cites a rule, as a decline must. Each check that looks at the text alone says yes. And the recommendation is false: R-ROOF-01 is about roofs older than 25 years in wind zone C, and this roof is 8 years old in zone A. If the workflow stops checking after the schema, this output reaches the underwriter cockpit as a well-formed reason to turn away a clean risk.

The model recommends; a person decides. The underwriter would probably catch it. But the point of a contract is that the underwriter should not have to re-check every field by hand. This page is about the checks between "the model returned something" and "the underwriter sees it": what each one asks, in which order they run, and what an error looks like when one says no.

Four questions about one output#

Plain English. Validation means checking a value and getting back a list of what is wrong with it. An empty list means "no problem found by this check". It never means "correct". For example: checking that stated_confidence is a number between 0 and 1 returns an empty list for 0.66, and that says nothing about whether 0.66 means anything.

The four layers of four kinds of right become four pieces of code here. Each piece is called a stage: one step of validation, with its own input, its own question and its own errors. The code names are parse, schema, cross_field and domain.

StageQuestionReadsNeeds to knowFails on, in our cases
parseIs this text JSON at all?raw textthe JSON grammarUW-S01 prose, UW-S03 cut off
schemaDoes the value have the agreed shape?parsed valuethe JSON Schema of the contractUW-S04 "approved", UW-S06 "high"
cross_fieldDoes the record agree with itself?a schema-valid valuerules that tie fields togetherUW-S08 request_info with nothing missing
domainIs it true for this run?a self-consistent valuethe policy fixture, the facts, the evidence of this runUW-S10 a rule that does not apply

Read the "Needs to know" column from top to bottom. Each stage needs more knowledge than the one before. The parser needs only the grammar of JSON. The schema stage needs the contract. The cross-field stage needs rules that the contract's authors wrote about combinations of fields. The domain stage needs the world: which version of the policy fixture this run read, what the roof-data provider returned, which evidence was gathered. That is why the domain stage is the only one that can catch UW-S10. The text is fine; the world disagrees with it.

Common confusion. Layer 3, semantics, is wider than the cross-field stage. Semantics also covers units (square metres in a square-foot field) and the difference between null and 0. Some of that code can check from the record alone. Some of it needs the source the value came from, and in this example that check lives in the domain stage of the intake contract (UW-S17, UW-S18). The stage names say where code runs; the layer names say what kind of right is being checked.

Diagram: Four stages, one output · each needs more knowledge than the one before

Read left to right. The raw text of the model enters on the left. Each stage either hands the value on or stops with errors. Only an output that passes all four is wrapped in a record by code; the underwriter still decides.

  1. In
    1. Raw output: Whatever the model returned: JSON, prose, a fence, half an object.
  2. 1 · parse
    1. Parse: Turns text into a value or stops with the position of the first bad character.
  3. 2 · schema
    1. Schema: Checks the shape: required fields, types, enums, patterns, a closed object.
  4. 3 · cross_field
    1. Cross-field: Checks that fields agree with each other.
  5. 4 · domain
    1. Domain: Checks the output against the policy fixture, the facts and the evidence of this run.
  6. Out
    1. Recommendation record: Code wraps the accepted output with stamped identifiers. A person decides.
    2. Error records: Any stage that says no stops the line and hands its errors to the repair step.

The diagram shows the normal path. One raw output enters on the left. Each stage either passes the value on or stops with errors. Only an output that passes all four becomes a recommendation record, and even then the underwriter decides.

The smallest version: four functions in a row#

The smallest primitive. A stage is a function from a value to a list of errors. Four of them in a row, stopping at the first that returns errors, is the whole mechanism. In examples/uw-contracts/validate.py the function is run_stages(text, ctx). The ctx (the context) carries what the later stages need to know: the schema, the policy version, the rules, the facts and the evidence ids of this run.

The demonstration below runs UW-S10 through the four stages. What a successful run proves: the first three stages pass, and the domain stage alone rejects the output, with two errors.

import json
from model_sim import load_case
from validate import recommendation_ctx, run_stages

case = load_case("UW-S10")                     # decline citing R-ROOF-01 for a roof of 8 years
ctx = recommendation_ctx(case["facts"], case["evidence"])
res = run_stages(case["attempts"][0], ctx)     # stop at the first stage that fails
print("first failure:", res["first_failure"])
for s in res["stages"]:
    print(f"  {s['stage']:<12} {s['status']}")
for e in res["stages"][3]["errors"]:
    print(json.dumps(e))

Output of a real run, in a scratch copy of the example (the model here is a scripted stand-in, so these results show how the mechanism works, not how good any model is):

first failure: domain
  parse        pass
  schema       pass
  cross_field  pass
  domain       fail
{"stage": "domain", "path": "/recommendation", "code": "D-RULE-RESULT", "message": "the rule check gives approve (no rule triggered); the output says decline"}
{"stage": "domain", "path": "/citations/0", "code": "D-CITATION-UNSUPPORTED", "message": "R-ROOF-01 does not apply to these facts"}

The domain stage does not ask the model whether it is right. It runs the same deterministic rule check the workflow runs anyway, eligibility() in validate.py, and compares. The rule check says approve, because no rule fires for these facts. The output says decline. One more design choice is visible in the code: a mismatch is allowed when the model recommends escalate, because sending a case to a senior underwriter is always a safe thing to recommend. That is a decision of this example, written down where the check is. Your domain may decide differently.

In our insurance agent. The rule check exists before the model does: it is ordinary code over the policy fixture. The model adds reasons a person can read and a judgement where the rules leave room. The domain stage keeps the two from disagreeing silently.

The error record#

Plain English. A validation error is a small record that says which stage failed, where in the value, which check, and why in words. For example: {"stage": "schema", "path": "/recommendation", "code": "enum", "message": "\"approved\" is not one of ..."}.

The path is a JSON Pointer. A JSON Pointer is a short string that names one value inside a JSON document: /citations/0 is the first item of the citations array, and the empty string "" is the whole document. It is defined by RFC 6901, which says a pointer "defines a string syntax for identifying a specific value within a JavaScript Object Notation (JSON) document". Two characters need escaping inside a key: ~ is written ~0 and / is written ~1.

Each part of the record has one reader in mind:

FieldFor whomWhy it is there
stagethe repair loop, the dashboardsDecides the next action: a parse error and a domain error call for different repairs (next page)
pathcode and peoplePoints at the value without quoting it; a dashboard can count failures per field
codecodeA stable id (enum, X-MISSING-EMPTY, D-RULE-RESULT, R-HUMAN-01) to count, alert on and test against; messages may be reworded, codes may not
messagethe model on a re-ask, the underwriter on a fallbackSays what is wrong in words that can be acted on
Before AI. A Spring service that validates a request body with Bean Validation returns a 400 with a list of field errors: the property path, the constraint, the message. The idea is the same. What changes with a model as the producer is the reader of the message: it is fed back to the model as the next instruction, so it must be precise, short and free of anything you would not want repeated back.

Look at the codes once more. Schema errors use the JSON Schema keyword (enum, required, type), because the validator produces them. Cross-field errors start with X-, domain errors with D-, and a check that enforces a rule of the policy fixture uses the rule id itself, R-HUMAN-01. A team that reads a failure dashboard in production can tell at a glance whether the producer broke the shape or the truth.

Why the order is not a matter of taste#

The obvious design is to run every check and report everything at once. It sounds more helpful: one round trip, all the problems. It is how a form on a web page often behaves.

Here is what happens when the checks run on a value the schema has rejected. The output says request_info, and missing is the string "roof_age_years" instead of a list. The demonstration runs the same text twice, once stopping at the first failing stage and once running every stage:

import json
from validate import recommendation_ctx, run_stages
from model_sim import load_case
case = load_case("UW-S01")
ctx = recommendation_ctx(case["facts"], case["evidence"])
good = json.loads(case["attempts"][-1])
bad = json.dumps(dict(good, missing="roof_age_years", recommendation="request_info"))
for stop in (True, False):
    res = run_stages(bad, ctx, stop_at_first=stop)
    print("stop_at_first =", stop, [(s["stage"], s["status"]) for s in res["stages"]])
    for s in res["stages"]:
        for e in s["errors"]:
            print("   ", e["stage"], e["path"], e["code"], "-", e["message"])
stop_at_first = True [('parse', 'pass'), ('schema', 'fail'), ('cross_field', 'not_run'), ('domain', 'not_run')]
    schema /missing type - expected array, got string
stop_at_first = False [('parse', 'pass'), ('schema', 'fail'), ('cross_field', 'pass'), ('domain', 'fail')]
    schema /missing type - expected array, got string
    domain /recommendation D-RULE-RESULT - the rule check gives escalate (R-ROOF-03); the output says request_info

Look at the second run. The cross-field stage says pass. Its check is "request_info must name at least one missing field", written as "the list missing is not empty". A non-empty string is not empty either, so the check passes on a value it was never written for. Nothing crashed. The check lied.

That is the reason for the order. Each stage is written assuming the stage before it passed. The cross-field check assumes missing is a list because the schema said so. The domain check assumes citations holds strings that look like rule ids. Run a later stage on a value the earlier stage rejected, and it either crashes (the explorer lab below marks that as crashed) or, worse, returns an answer about a value it does not understand.

Inside one stage the rule is the opposite: collect every error. The schema stage reports a wrong enum and a missing field together, because both are true at once and one re-ask can fix both.

Diagram: Why stop at the first failing stage · a later check on a wrong type does not crash, it lies

Top lane: the schema rejects missing as a string and the line stops; one true error reaches the repair step. Bottom lane: the same value runs through every stage; the cross-field check passes a value it was never written for, and the domain stage reports a misleading error.

  1. Stop at first
    1. missing is a string: The output says request_info and missing is "roof_age_years", not a list.
    2. schema: fail: expected array, got string.
    3. Stop: The re-ask carries only the type error, which is what must be fixed.
  2. Run everything
    1. Same output: The same value is checked by every stage regardless.
    2. cross_field: pass: "missing is not empty" is true of a non-empty string, so the check passes on a value it was never written for.
    3. domain: fail: The rule check says escalate; the error points at the recommendation, not at the type.
Production pitfall. A validator that "helpfully" reports domain errors on a schema-invalid value produces messages like "the rule check gives escalate; the output says request_info" when the real problem is a type. A re-ask built from that message sends the model after the wrong fix. Report the earliest failing stage only.

Try it: switch stages off#

The lab below takes the outputs of UW-S04 to UW-S15. All of them parse; that is why they were chosen. Switch a stage off and see which stage catches the output next, or whether it reaches the underwriter. Two cases are worth trying first. UW-S10 with the domain stage off reaches the cockpit as a confident decline. UW-S15 passes every stage with everything on, and still carries markup and an injected instruction in a reason: a schema checks shape, not truth, and not what a string does where it lands.

CaseThe output, in a few wordsCaught atError code
UW-S04"recommendation": "approved"schemaenum
UW-S05policy_version absentschemarequired
UW-S06"stated_confidence": "high"schematype
UW-S07extra field premium_usdschemaadditionalProperties
UW-S08request_info, nothing missingcross_fieldX-MISSING-EMPTY
UW-S09decline, no citationcross_fieldX-CITATIONS-EMPTY
UW-S10decline under R-ROOF-01, roof 8 yearsdomainD-RULE-RESULT, D-CITATION-UNSUPPORTED
UW-S11cites R-ROOF-09domainD-RULE-UNKNOWN
UW-S12policy_version 2025.4domainD-POLICY-VERSION
UW-S13requires_human falsedomainR-HUMAN-01
UW-S14evidence id never gathereddomainD-EVIDENCE-UNKNOWN
UW-S15markup and an instruction in a reasonnone(flagged untrusted by code)

Where each stage runs, and who owns it#

A stage is code, and code has an owner. In this example all four stages run in the workflow, the code around the agents, right after the model returns and before anything is stored. Each stage's knowledge has a different owner, though:

StageOwner of the checkChanges when
parsenobody: the JSON grammarnever
schemathe owners of the contract (producer and consumer agree)the contract gets a new version
cross_fieldthe owners of the contractsomeone finds a contradiction the schema cannot express
domainthe owners of the rules: here, whoever maintains the policy fixturethe policy fixture changes (2025.4 to 2026.2), or new evidence types appear

This split is useful when something fails in production. A rise in schema failures is a conversation with whoever changed the producer (the prompt, the model setting, the provider). A rise in domain failures after a policy release is a conversation with the rule owners: the model may be right about the old rules.

For a coding agent. A coding agent that reports its work as a change_report (files_changed, tests_run, tests_passed, risk, needs_review) can be checked the same way. Schema: risk is one of low, medium, high. Cross-field: tests_passed is not greater than tests_run. Domain: every path in files_changed appears in the actual diff of the branch, which only CI can see. The review and CI side of that story lives on diff-to-deploy.sammuthu.com.
What this is not. These stages check one output against its contract. Checks around the whole agent (what it may call, what it may see, what a person must approve) are guardrails, owned by guardrails.sammuthu.com. Measuring how often outputs are right over many runs is evaluation, owned by eval.sammuthu.com.

How libraries and standards do it#

The pattern is the same everywhere; the split between stages differs.

JSON Schema (a standard). The 2020-12 core specification defines output formats for validation results. Each result carries a keywordLocation and an instanceLocation, both JSON Pointers: "the location of the JSON value within the instance being validated". It names four formats, flag (a single boolean), basic (a flat list), detailed and verbose (hierarchies following the schema), and says an implementation "SHOULD provide at least one of the "flag", "basic", or "detailed" format". The error record on this page is close to the basic shape with the stage added. Read the details in the specification itself; this site's validator does not claim to implement those formats.

Pydantic (a library) and the Java pair, Jackson with Bean Validation (a library and a standard). The recorded run in examples/uw-contracts/twins/ (Pydantic 2.14.3; Jackson 2.23.0 and Hibernate Validator 9.1.6.Final, run on 2026-11-17) shows the two shapes of the same idea. Pydantic parses, converts and checks constraints in one call and returns one list of errors. The Java path has two steps: Jackson stops at the first problem it meets while reading, and Bean Validation runs only if reading succeeded. So a range error is never reported together with a type error. Neither library knows your policy fixture: the domain stage is always your code. The typed models page compares the two in detail.

Checklist#

Short-circuit by field, not only by stageRunning later checks on the fields that passed

Stopping at the first failing stage is the simple rule. A finer rule runs later stages only on the fields the schema accepted, so that one bad field does not hide a domain error on another. It is more work to get right: every cross-field and domain check must declare which fields it reads, and must be skipped when any of them failed. The gain is one fewer re-ask in cases with two independent problems. Measure how often that happens before you build it; in most contracts with seven to ten fields, the simple rule costs little.

What each stage costsMicroseconds, then lookups, then sometimes a network call

Parse and schema checks are local and cheap. Cross-field checks are local too. Domain checks may need data: the policy fixture of the right version, the facts of the submission, the list of evidence ids gathered in this run. Keep that data in the run's context instead of fetching it again inside the validator, both for speed and so that a replay sees exactly what the original validation saw. A domain check that calls a remote service at validation time makes validation itself a source of timeouts, which then need their own handling.

Validate on both sides of a boundary?Producer-side checks are a courtesy; consumer-side checks are the contract

When a model produces the output, the producer cannot validate itself in any binding way: the workflow does it. When code produces a payload for another service, it is common to validate before sending (to fail early with a clear message) and again on receipt (because the receiver cannot trust the sender's version of the schema). The receiving check is the one that protects the system. The boundaries page applies this to all four contracts on the path.

In one breath#

Validation is four questions in a fixed order: is it JSON, does it have the agreed shape, does it agree with itself, is it true for this run. Each stage needs more knowledge than the one before, and each is written assuming the one before passed, so you stop at the first failure and collect everything inside it. The error record says stage, path, code and message, so the repair loop, the dashboards and the person reading it all know what to do next. Only the domain stage knows the world, and only your code can write it.

You can now tell, for any failing output, which stage should catch it and what the error should say. What is still open is what to do with that error: fix it in code, send it back to the model, ask a person, or stop. That is the next page, repair.