T-04Structured Output & Validation

Validate + repair

1. Downstream code needs typed data, so the schema goes into the prompt.
2. The model generates freely — and usually gets it right.
3. Every output is parsed and checked against the schema before anything uses it.
4. Invalid: a string where a number belongs.
5. The exact error goes back with the bad output — fix this field, not “try again”.
6. The retry validates, and only then reaches your code — at the cost of a loop, latency, and a failure path you own.

Constrained decoding

1. Same schema — but this time it is compiled into a grammar before generation starts.
2. At every token, anything that would break the schema is masked out.
3. So the output cannot be malformed: the right keys, the right types, every required field.
4. Not a proof, though. A response cut off at max_tokens, or a refusal, still won’t match the schema.
5. And some rules — ranges, lengths — can’t be expressed in the grammar, so your code still checks them.
6. Check the stop reason, keep the range checks, and shape errors are gone by construction.