Structured candidate fails: separate parsing, schema, and support

Understand the likely causes

• Likely causes include an unclear output contract, unsupported schema dialect, truncated generation, weak parsing assumptions, missing domain constraints, or consumer code that trusts model text directly. A prompt asking for JSON is not equivalent to a verified schema engine, and provider-specific constrained generation still requires its documented compatibility and limits.

Preserve safe input and contract identity

Record prompt, model bundle, schema dialect and version, evidence scope, truncation status, validator revision, and the safe failure class. Do not log real sensitive candidates or pass them to an interpreter for inspection. Use a public minimal fixture. The first complete program rejects duplicate keys and nonstandard numeric constants under its chosen strict parsing contract before checking an exact integer field. It is a narrow local application validator, not a full JSON Schema implementation or a model-quality measurement.

import json
def unique_object(pairs):
    result = {}
    for key, value in pairs:
        if key in result: raise ValueError("duplicate key")
        result[key] = value
    return result
def reject_constant(value):
    raise ValueError("nonstandard constant")
def classify(text):
    try:
        value = json.loads(text, object_pairs_hook=unique_object, parse_constant=reject_constant)
    except ValueError:
        return "parse_contract_error"
    if not isinstance(value, dict) or set(value) != {"count"} or type(value["count"]) is not int:
        return "schema_error"
    return "accepted_fixture"
for text in ['{"count":2}', '{"count":1,"count":2}', '{"count":NaN}', '{"count":true}']:
    print(classify(text))

accepted_fixture
parse_contract_error
parse_contract_error
schema_error

Classify the failing validation layer

Parsing establishes a representation under the chosen parser rules. Schema checks required fields, types, bounds, and extras; domain checks application invariants; support checks evidence; policy checks authority. Python booleans need deliberate handling when the contract requires an exact integer. A valid source ID still needs a supporting passage and applicable version. Keep the failure label specific so a schema repair is not incorrectly used to solve a missing fact or an unauthorized operation.

Inspect the consumer before broadening acceptance

Check whether downstream code expects JSON, a typed command, a query parameter, a tool proposal, or display text. Escaping for display differs from authorization to execute. Do not strip arbitrary surrounding text, coerce every value, or evaluate code just to make a candidate fit. A relaxed parser can create disagreement with another component and hide duplicated or ambiguous fields. If the declared schema is wrong, change it deliberately with compatibility tests and review of old clients or stored records.

Use a bounded correction or abstention policy

The second record separates maximum attempts, total budget, preserved evidence, and final failure behavior. It is editorial metadata, not a real runtime configuration. A repair step must not invent a missing source, alter the user’s operation, or grant itself approval. Revalidate all required layers after correction. If evidence or permission is absent, use the declared abstention, denial, or review path rather than repeatedly asking for a more confident answer. Count repair work in latency and cost, and stop before it exhausts resources needed by other requests.

repair_policy: synthetic-review-record
maximum_attempts: 2
total_budget: independently-bounded
preserve: [original_task, trusted_schema, allowed_evidence, authorization]
repeat_checks: all-required-layers
terminal_result: explicit-failure-abstention-or-review
claim: no-model-or-runtime-executed

Verify the original defect and adjacent cases

Test accepted data, malformed text, duplicate keys, nonfinite constants, unknown fields, nulls, wrong types, domain violations, unsupported citations, and unauthorized proposals as applicable. Assert stable classifications and no side effect before release. These offline cases prove the validator’s response only, not how often an actual model fails. Preserve a held-out comparison and retest relevant cases when schema, parser, model, or repair policy changes.

Keep it from recurring

• Version schemas and consumers together, use exact bounded contracts, and separate shape from evidence and authority. Keep repair attempts observable and limited, with safe diagnostics and held-out regressions. Never use generated text as code merely because a model was asked to produce a structured result.