Generative AI / Basic: generation and evidence foundations / Lesson 6

Factuality, uncertainty, and abstention

Distinguish factual error, unsupported context claims, calibrated confidence, and an honest insufficient-evidence response.

About 5 min readContent reviewed 2026-10-26

On this page

Lesson goals

Explain the assumptions and trust boundaries of factuality, uncertainty, and abstention.
Interpret both examples and distinguish offline fixture results from model or production evidence.
Diagnose the listed failures and justify the worked decisions while preserving task, data, permission, and uncertainty contracts.

Define the failure for the task

The term hallucination is used for several failures, including incorrect world facts, invented citations, or claims unsupported by the supplied context. State which property an evaluation checks. A creative fiction task and a policy-answering task have different success criteria. A claim can be factually true but unsupported under an evidence-only contract, or faithfully repeat a source that is itself outdated or wrong. The first offline example compares a structured candidate with a tiny authoritative synthetic fact store. It does not perform natural-language entailment or measure a language model. Source authority and freshness still need separate review.

python Offline exact-fact support fixture

facts = {"policy-lab-v1": {"refund_days": 14}}
def supported(candidate):
    source = facts.get(candidate["source_id"])
    return source is not None and source["refund_days"] == candidate["refund_days"]
print(supported({"source_id": "policy-lab-v1", "refund_days": 14}))
print(supported({"source_id": "policy-lab-v1", "refund_days": 30}))
print(supported({"source_id": "invented-source", "refund_days": 14}))

Expected output

True
False
False

Only the matching fact with an existing synthetic source is supported. This exact-field comparison is not natural-language factuality verification.

Treat confidence language as output, not proof

A model can sound certain while wrong or cautious while correct. Self-reported confidence may be useful only after validation for the exact task and setup; it is not automatically a calibrated probability. The second synthetic fixture has two predictions at confidence 0.9 but only half correct. Its Brier score calculation illustrates one probabilistic evaluation idea, not a real model result or a sufficient sample for deployment decisions. Calibration, accuracy, abstention coverage, and the cost of different errors should be evaluated together. Never translate a fluent explanation or a similarity score directly into a trustworthy percentage.

python Offline confidence and outcome comparison

predictions = [(0.9, 1), (0.9, 0)]
mean_confidence = sum(p for p, label in predictions) / len(predictions)
accuracy = sum(label for p, label in predictions) / len(predictions)
brier = sum((p - label) ** 2 for p, label in predictions) / len(predictions)
print(round(mean_confidence, 2), round(accuracy, 2), round(brier, 2))
print("synthetic two-case illustration only")

Expected output

0.9 0.5 0.41
synthetic two-case illustration only

Mean confidence is 0.9, accuracy 0.5, and Brier score 0.41 for two invented cases. This tiny sample is not calibration evidence for any real system.

Make abstention a useful contract

An insufficient-evidence outcome should explain the missing support and, where appropriate, ask for a specific document or route the task for review. It is not the same as a technical outage or a policy denial. Define when the system must refuse to guess and what information would change that decision. Over-abstention can make a system useless, while under-abstention can create unsupported authority. Test both supported and deliberately unanswerable cases. A generated citation ID that exists in the context proves only a reference relationship; the cited passage must actually support the answer’s relevant claim and qualifiers.

Use evidence and verification at the right layer

Retrieval, a calculator, a database, or a human reviewer can provide different kinds of evidence, but each has scope, trust, and failure limitations. A tool result should be tied to a verified operation and current data rather than accepted because it appears in a model transcript. For arithmetic, deterministic computation may be more appropriate than free-form generation. For a policy claim, compare authorized source text and version. Keep the application’s final release decision explicit, especially when the output can affect rights, money, safety, or access. The model is a candidate producer, not the source of its own certification.

Evaluate uncertainty without fabricating measurements

Use labeled representative fixtures, explicit scoring rules, held-out cases, and error review. Record whether answers were unsupported, contradicted, incomplete, or appropriately abstained rather than hiding all failures in one accuracy number. Sample sizes and uncertain labels limit interpretation. Both examples here are public deterministic exercises with no trained model, API, external verifier, or real user data. Their outputs establish code behavior only. A trustworthy reporting process distinguishes measured evidence from assumptions and states the remaining uncertainty, instead of presenting an attractive confidence number or a selected anecdote as production reliability.

Common errors and fixes

A confident answer is labeled calibrated

Why it happens: Wording was substituted for observed probability quality.
How to fix it: Compare confidence with labeled outcomes for the actual task.
Prevent it next time: Report sample and label uncertainty.

An existing citation is assumed to entail the claim

Why it happens: Reference presence replaced support checking.
How to fix it: Check the cited passage and relevant qualifiers.
Prevent it next time: Include valid-ID but wrong-support fixtures.

The system guesses when evidence is absent

Why it happens: No explicit abstention or review contract existed.
How to fix it: Return a useful insufficient-evidence outcome.
Prevent it next time: Test deliberately unanswerable questions.

Check your understanding

1. A candidate cites a real passage but changes fourteen days to thirty. Which checks pass and fail?

Show worked solution

The source-ID existence check can pass, while factual support fails. Keep those checks separate and reject or correct the unsupported number using the authorized evidence. A bibliography-shaped answer is not automatically grounded.

2. Why should an unanswerable fixture be included in evaluation?