All chapters
  1. Start hereParseable is not valid. Valid is not correct
  2. Foundations · 1From free text to fields and records: why contracts matter
  3. Foundations · 2Syntax, schema, semantics, business validity: four different kinds of right
  4. Foundations · 3JSON and JSON Schema: types, enums, nullable fields, unions and nested objects
  5. Building · 4Typed models: Pydantic in Python, records and Bean Validation in Java
  6. Building · 5Provider output modes: prompting, JSON mode, constrained decoding and tool arguments
  7. Building · 6Tool input, tool result, agent handoff, final response: four contracts on one path
  8. Building · 7Schema design: names, required or optional, and where each value came from
  9. Operating · 8Validation stages: parse, schema, cross-field, domain rules
  10. Operating · 9Errors and repair: retry, clarify, correct within bounds, fall back
  11. Operating · 10Schema evolution: versions, compatibility and migrations
  12. Operating · 11Security: a valid string can still be an attack
  13. Operating · 12Observing contracts: validation failures, schema drift and replay artifacts
  14. Operating · 13Testing contracts: golden examples, property-based tests, fuzzing and contract suites
  15. Reference · 14Frameworks and standards: what each one checks for you
  16. Reference · 15Advanced: partial objects in a stream, polymorphism, multilingual data and typed events
  17. Reference · 16The insurance lab: one submission, four contracts, end to end
  18. ReferenceGlossary
  19. AboutAbout this site and its sources
Operating · 9

Errors and repair: retry, clarify, correct within bounds, fall back

A validation error is a message for someone. Which errors code may fix, which go back to the model, which need a question to a person, and when to stop and hand over.

What you will understand
Four repairs
Fix by code, re-ask the model, ask a person, or stop: each fits a different kind of error.
What code may fix
Only what cannot change the meaning: unwrap a fence, yes; close a bracket, never.
Bounds
A limit on attempts, carried as attempt_number, and what happens at the limit.
Fallback
A record with no recommendation is a valid end, and better than a guessed one.

The validation stages of the previous page have done their job on case UW-S16, one of this site's own scripted cases. On the facts of UW-004 (application says the roof is 9 years old, the roof-data provider says 22), the underwriting agent wrote "recommendation": "approve or escalate". The schema stage said no: that is not one of the four allowed words. Now what?

Three answers come to mind at once, and all three are tempting. Pick the first word, approve, and move on. Pick the safer word, escalate. Or ask the model again and keep asking until it gives a valid word. The first two are code deciding what the model meant, on a field that decides where a quote request goes. The third has no end: in this case the model answers approve/escalate the second time and escalate (or approve) the third.

Plain English. Repair is any action that turns a failed output into either an accepted output or a controlled hand-over to a person. For example: removing a markdown fence around correct JSON is a repair; so is giving up after three attempts and telling the underwriter there is no valid recommendation. This page sorts the possible actions by who acts, what they may change, and when to stop.

An error is a message for someone#

Every validation error has a reader who can do something about it. The question that picks the repair is: who can fix this, and can they fix it without changing what was meant?

RepairWho actsMay changeFits errors likeIn our cases
Deterministic repaircodeonly the wrapping of the texta code fence, a sentence before the JSONUW-S02
Re-askthe model, given the errorsanything; the result is validated againa wrong word, a missing field, a rule that does not applyUW-S04 to UW-S14
Clarificationa person outside the run (the broker, through the underwriter)the factsa fact nobody has: the roof age in UW-003a request_info recommendation
Fallbackcode stops; a person takes overnothing: it records what failedanything still failing at the limitUW-S13, UW-S16

Deterministic repair is a fix by code that cannot change meaning. In UW-S02 the model wrote a correct recommendation inside a markdown code fence, after the sentence "Here is the recommendation as requested:". Removing the fence and the sentence changes no value. Code can do that with certainty, so no re-ask is needed.

Re-ask means sending the errors back to the model and asking again. It raises the attempt number: attempt 1 failed, attempt 2 is the re-ask. In UW-S04 the model wrote "approved"; the re-ask says which value at which path is not allowed, and attempt 2 says "escalate".

Clarification means asking a person for a fact that nobody in the run has. It is not a reaction to a bad output; it is the right output. When the roof age is unknown (UW-003), the correct recommendation is request_info with missing: ["roof_age_years"], and the underwriter asks the broker. Code never "repairs" a missing fact.

Fallback is the bounded end: stop asking, store the errors, hand the case to a person.

Diagram: Who can fix this error · four repairs, from cheapest to most expensive

Read left to right. An error goes to the cheapest actor who can fix it without changing what was meant. Code may only unwrap. The model gets the errors. A missing fact goes to a person. Whatever still fails at the limit falls back to the underwriter.

  1. Code
    1. Deterministic repair: Removes a fence or prose around one complete JSON value; adds nothing.
  2. The model
    1. Re-ask: Sends the stage and each error with its path; the new output is validated from the start.
  3. A person, for a fact
    1. Clarification: A fact nobody in the run has is requested from the broker through the underwriter.
  4. A person, for the case
    1. Fallback: Code stops, stores no recommendation and the errors; the underwriter decides from the evidence.

What code must never fix#

Deterministic repair is the cheapest action, and that is its danger. A team that sees many failures of one kind is tempted to fix them in code. Some of those fixes are safe. Most are not.

FailureTempting code fixWhy it is wrong
Output cut off by the output limit (UW-S03)close the open string and bracketsCode cannot know what was cut. A cut reasons list may have lost the sentence that carried the decision
policy_version absent (UW-S05)fill in the current versionThe point of the field is that the model says which policy it read. Filling it in makes the check say nothing
requires_human: false (UW-S13)set it to trueThe value is a symptom: the model behaved as if no person would decide. Flipping it hides that the output should not be trusted
"approve or escalate" (UW-S16)take the first word, or the safer wordCode picks the meaning. On the routing field of a quote request
Extra field premium_usd (UW-S07)drop the field and accept the restThe model tried to quote a price, which it must not do. Dropping the field accepts the output that tried
"stated_confidence": "high" (UW-S06)map high to 0.8The number is invented by code and then looks like the model's

The test is one question: could two careful engineers disagree about what the fixed value should be? If yes, code must not fix it. Unwrapping a fence passes the test. Every row of the table fails it.

unwrap() in examples/uw-contracts/repair.py is the only deterministic repair in the example. It takes the text inside a code fence, or from the first { to the last }, and accepts the result only if it parses on its own. It removes characters; it never adds one. On the cut-off output of UW-S03 it returns None: there is no closing brace to find, and it will not invent one.

Common confusion. "JSON repair" libraries that close brackets and fix quotes are useful when a person pastes broken JSON and checks the result by eye. Between a model and a program, nobody checks the result, so a repaired value enters the system as if the model had written it. Use them, if at all, to build a better error message, never to produce the accepted value.

The re-ask, and what it carries#

A re-ask is a new model call. What it sends matters as much as that it is sent. The example builds the message with reask_message() from the error records of the failed attempt. Here is the real message for UW-S04, produced by running the example in a scratch copy:

Your last output failed validation at the schema stage:
- /recommendation: "approved" is not one of "approve", "decline", "request_info", "escalate"
Return only the corrected JSON object. Do not add fields. Do not change the values that were accepted.

Three choices are visible in it. It names the stage, so the model knows whether the problem is shape or truth. It gives each error with its JSON Pointer path, so the model knows exactly which value to change. And it says what must not change: values that passed must stay as they were. Without that line a re-ask often returns a different recommendation with the original problem fixed and a new one introduced.

What the message leaves out is a choice too. It does not paste back the whole failed output, and it never quotes untrusted text from it. If a reason string carried an injected instruction, as in UW-S15, a re-ask that echoes the reason would put the instruction into the next prompt with the authority of the workflow. The security page explains why.

Before AI. A message consumer reading from a Kafka topic retries a failed message a fixed number of times, then sends it to a dead-letter topic for a person to inspect. A re-ask loop has the same shape with one difference that changes the design: each retry asks the producer to produce something new. The retry is not idempotent. The result of attempt 2 is a different output, so it goes through all four stages again from the start.

Bounds: attempt_number and the limit#

Plain English. A bound is a fixed limit on how many times the workflow tries. For example: at most 3 attempts per run, then stop. The example uses 3; the number is illustrative.

attempt_number is the counter. It starts at 1 for the first output of a run and rises by one with each re-ask. It is not the run: run_id (run-7f3a style) names one execution of the workflow for one submission, and all attempts of that run share it. On this site there is one trace per run, and each attempt is a span that carries its attempt_number, so a single trace shows the whole repair story. Tracing itself is explained on observe.sammuthu.com.

The loop in repair_loop() is short enough to read whole. For each attempt from 1 to the limit: get the output, run the stages; if the parse failed and deterministic repair is allowed, try unwrap() and run the stages again; if everything passed, accept; if attempts remain and re-ask is allowed, re-ask; otherwise, fall back. The real run over four cases:

UW-S02: accepted_after_repair  [(1, 'parse', 'accept')]
UW-S03: accepted_after_reask   [(1, 'parse', 're_ask'), (2, '-', 'accept')]
UW-S04: accepted_after_reask   [(1, 'schema', 're_ask'), (2, '-', 'accept')]
UW-S16: fallback               [(1, 'schema', 're_ask'), (2, 'schema', 're_ask'), (3, 'schema', 'fallback')]

Each tuple is one attempt: its number, the stage that failed first, and the action taken. The model here is a scripted stand-in, so these results show how the mechanism works, not how good any model is.

Why a limit at all? Three reasons, each sufficient. Cost: each re-ask is a full model call with the original context plus the errors. Time: the underwriter, or the broker waiting for a quote, is waiting. And evidence: when the same error comes back twice, the third call rarely differs. UW-S13 returns requires_human: false three times in a row. A model that keeps behaving as if no person will decide is telling you something a fourth attempt will not change.

Diagram: The bounded loop on UW-S16 · two re-asks, then a record with no recommendation

Read top to bottom. The workflow validates each attempt from the parse stage. Two re-asks carry the errors. At the limit of 3 the workflow stops and writes a record with no recommendation, which the underwriter sees.

  1. Model (stand-in) to Workflow (code): attempt 1: approve or escalate. The schema stage rejects the word: not one of the four.
  2. Workflow (code) to Model (stand-in): re-ask with the enum error. attempt_number becomes 2.
  3. Model (stand-in) to Workflow (code): attempt 2: approve/escalate. Still not one of the four words.
  4. Workflow (code) to Model (stand-in): re-ask with the enum error. attempt_number becomes 3, the limit.
  5. Model (stand-in) to Workflow (code): attempt 3: escalate (or approve). Still ambiguous; the limit is reached.
  6. Workflow (code): fallback record. recommendation null, status no_valid_recommendation, the errors, requires_human true.
  7. Workflow (code) to Underwriter: no valid recommendation. The underwriter decides from the submission and the evidence.

Try it: step through the loop#

The lab runs the real loop in your browser (the same code as the Python, checked row by row). Pick a failing case, set the limit and the allowed actions, and step through. Things to try: UW-S02 with deterministic repair switched off becomes a re-ask, one more model call for nothing. UW-S04 with the limit at 1 falls back even though attempt 2 would have been valid. UW-S13 falls back at any limit.

CaseAttempt 1 fails atAction per attempt (limit 3, both actions allowed)Outcome
UW-S01parse (prose)re-ask, acceptaccepted after re-ask
UW-S02parse (fence)unwrap, acceptaccepted after deterministic repair
UW-S03parse (cut off)re-ask, acceptaccepted after re-ask
UW-S04schema (enum)re-ask, acceptaccepted after re-ask
UW-S10domain (rule does not apply)re-ask, acceptaccepted after re-ask
UW-S13domain (R-HUMAN-01)re-ask, re-ask, fallbackfallback
UW-S16schema (ambiguous word)re-ask, re-ask, fallbackfallback

The fallback record#

A fallback is not a failure of the workflow. It is the workflow working: a bounded attempt, an honest record, a person in charge. What makes it honest is what the record contains. Here is the real record code writes at the end of UW-S13 (the error list shortened to one entry):

{
  "schema_version": 1,
  "submission_id": "SUB-1004",
  "run_id": "run-7f0d",
  "attempt_number": 3,
  "trace_id": "6863b07d0489a3e9d670f28baaa6fcb4",
  "policy_version": "2026.2",
  "evidence_ids": [],
  "status": "no_valid_recommendation",
  "recommendation": null,
  "validation": [],
  "errors": [
    {
      "stage": "domain",
      "path": "/requires_human",
      "code": "R-HUMAN-01",
      "message": "every recommendation needs a decision by an underwriter"
    }
  ],
  "requires_human": true,
  "human_decision": null
}

recommendation is null. Not the last attempt, not a "best effort". The last attempt failed validation, so it is not a recommendation. status says so in words the underwriter cockpit can show: no valid recommendation. The errors travel with the record, so the underwriter knows why and an engineer can count it later. requires_human is true, as for every record. human_decision is null until the underwriter acts. Every identifier in it was stamped by code, never taken from the model.

In our insurance agent. The underwriter sees the submission, the roof assessment and the message "no valid recommendation (validation failed 3 times at the domain stage)". They decide from the evidence, as they would have before any agent existed. The workflow cost them nothing but the time to read one line.
What this is not. Retries of a timed-out tool, backoff under load, idempotency keys and circuit breakers are reliability, owned by reliable.sammuthu.com. This page is about repairing an output that arrived and failed its contract. What runs next after a fallback (a queue, a notification, a different agent) is orchestration, owned by orchestrate.sammuthu.com.
For a coding agent. A coding agent's change_report fails validation because tests_passed is "all" instead of a number. A re-ask with the error is right. Filling in tests_passed = tests_run in code is the coding-agent version of flipping requires_human: CI would then report tests as passed that nobody counted. The harness that runs such loops is described on harness.sammuthu.com.

Checklist#

Choosing the limitMeasure the second-attempt success rate before picking a number

The right limit is an empirical question. Record, per error code, how often attempt 2 succeeds and how often attempt 3 succeeds after attempt 2 failed. If attempt 3 almost never succeeds for domain errors, a limit of 2 for domain errors and 3 for parse errors (where a larger output limit often fixes truncation) spends less for the same result. A limit per stage is a small change to repair_loop(). Measure on recorded runs, not on a guess; eval.sammuthu.com covers how.

Re-ask in the same conversation or a fresh oneThe failed output in context helps and hurts

A re-ask can append the error message to the conversation that produced the failed output, or start a fresh call with the original input and the error list. Appending is cheaper to write and lets the model see its own mistake, which often helps with schema errors. It also keeps the failed output, and anything injected into it, in the context of the next attempt. For outputs that carried untrusted text, a fresh call with only the original, trusted input plus the error codes is the safer design. What goes into the model's context is the subject of context.sammuthu.com.

Repair when the provider constrains the outputFewer parse and schema failures; the same domain failures

When the provider restricts decoding to your schema, most parse and schema failures disappear, and with them most re-asks. Truncation and refusals still happen, and no constrained decoding knows your policy fixture. The repair loop stays, with a different mix: fewer cheap fixes, the same expensive ones. The provider modes page describes what each mode guarantees.

In one breath#

Every validation error goes to whoever can fix it without changing what was meant: code may only unwrap, the model gets the errors and a re-ask, a missing fact becomes a request to a person, and anything still failing at the limit becomes a record with no recommendation and the errors attached. The limit is small and explicit, attempt_number counts against it, and every new attempt goes through all four stages. Code that fills in, flips or picks values on the model's behalf turns a visible failure into an invisible one.

You can now decide, for any error, the repair and its bound. What happens when the contract itself has to change, while old outputs, old readers and stored records still exist, is the next page: schema evolution.