Generative AI / Basic: generation and evidence foundations / Lesson 4

Structured outputs and validation

Separate parseable syntax, schema conformance, domain validity, factual support, and authority to use a generated result.

About 5 min readContent reviewed 2026-10-26

On this page

Lesson goals

Explain the assumptions and trust boundaries of structured outputs and validation.
Interpret both examples and distinguish offline fixture results from model or production evidence.
Diagnose the listed failures and justify the worked decisions while preserving task, data, permission, and uncertainty contracts.

A valid format is only one guarantee

JSON can be parsed while containing the wrong fields, unsupported claims, an unauthorized action, or an impossible business value. Treat generated content as untrusted input at every downstream boundary. The first offline validator distinguishes invalid JSON, a wrong type, an invalid domain count, and an accepted synthetic command. It is a deliberately small validator for one contract, not a full JSON Schema engine. A schema-constrained generation feature, when supported by a selected runtime, can narrow output form but cannot establish truth, ownership, or permission by itself. No provider-specific structured-output API is taught or invoked here.

python Offline layered validation

import json
def reject_constant(value):
    raise ValueError("nonstandard JSON constant")
def validate(text):
    try:
        value = json.loads(text, parse_constant=reject_constant)
    except ValueError:
        return "invalid_json"
    if not isinstance(value, dict) or set(value) != {"count"} or type(value["count"]) is not int:
        return "schema_error"
    if not 0 <= value["count"] <= 3:
        return "domain_error"
    return "accepted"
for candidate in ['{"count": 2}', '{"count": true}', '{"count": 5}', "count=2"]:
    print(validate(candidate))

Expected output

accepted
schema_error
domain_error
invalid_json

Only the exact allowed integer count within the synthetic bound is accepted. The function is a narrow application validator and not a real model-output quality measurement.

Declare the schema and its dialect

A schema should state required fields, accepted types, bounds, enums, nesting, and extra-field behavior. The second example declares a JSON Schema 2020-12 object for an answer and source IDs. It is syntax-checked but not executed by a model or a schema engine in this lesson. Real runtimes can support only a subset or a different dialect, so verify compatibility before relying on enforcement. A field called citations is not automatically a verified bibliography. Limit field and array sizes to protect consumers, and decide how null, missing, empty, duplicate, or unexpected values are handled.

json Illustrative answer schema

{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","required":["answer","source_ids"],"properties":{"answer":{"type":"string","maxLength":200},"source_ids":{"type":"array","maxItems":5,"uniqueItems":true,"items":{"type":"string"}}},"additionalProperties":false}

The document constrains shape and size only. It neither proves source existence or entailment nor defines a provider API, and no schema engine or model executes it here.

Validate domain and evidence after shape

A schema can require an integer but not necessarily prove that the integer is a truthful count from current evidence. Domain checks can enforce an available-stock bound or a relationship between fields, while evidence checks compare claims with authorized sources. The first validator uses exact integer type so Python booleans are not accidentally accepted as counts. Keep resource authorization separate from field validation. Do not construct SQL, shell commands, HTML, or tool requests from model text without a reviewed consumer contract. A perfectly shaped object can still be malicious, unsupported, stale, or addressed to the wrong tenant.

Bound correction and failure policy

A malformed candidate can be rejected, routed for review, or retried under a bounded repair policy. Retrying indefinitely increases cost and latency and can repeat the same failure. Do not silently fill missing facts or change a requested operation to make the object validate. If repair uses another generation step, preserve the original task, trusted schema, evidence, and authorization constraints. Record whether the failure was parsing, schema, domain, evidence, or policy so the correct component can be investigated. A parser exception should not become an apparently successful empty answer that downstream code treats as verified.

Test adversarial and ordinary fixtures offline

Include correct examples, omitted fields, unknown keys, wrong types, huge strings, nulls, duplicate references, unsupported claims, and unauthorized actions. A local parser test verifies the application’s response to a fixture, not the frequency of real model failures. Keep fixture labels explicit and use no real customer content or credentials. No language model, external validator service, API charge, or tool effect is involved. A robust system preserves a clear boundary between proposed content and accepted state, and its release record explains which validators ran and which properties still need human or external evidence.

Common errors and fixes

Parseable JSON is released as verified data

Why it happens: Syntax replaced schema, evidence, and policy checks.
How to fix it: Apply each required validation layer explicitly.
Prevent it next time: Test correct syntax with incorrect meaning.

Boolean true is accepted as a count

Why it happens: A language-level subtype shortcut weakened the intended type rule.
How to fix it: Enforce the exact contract type and bounds.
Prevent it next time: Include booleans and nulls in numeric fixtures.

Repair invents a missing fact

Why it happens: Validation success was prioritized over evidence.
How to fix it: Reject or abstain when the required fact is unsupported.
Prevent it next time: Bound repairs and preserve original evidence and constraints.

Check your understanding

1. A generated object has an integer refund window but the source never states one. Is schema success enough?

Show worked solution

No. The type check establishes shape, not factual support. Require evidence for the number or return the declared insufficient-evidence outcome. Do not repair the missing fact with a plausible default unless the application contract explicitly defines that default as policy.

2. Design the failure labels for invalid JSON, extra fields, and a valid but unauthorized operation.