Generative AI / Advanced: evaluation, reliability, and accountability / Lesson 16

Observability, drift, and feedback

Track system versions, stage failures, and changing task outcomes while protecting sensitive prompts and avoiding causal claims from one dashboard.

About 5 min readContent reviewed 2026-10-26

On this page

Lesson goals

Explain the assumptions and trust boundaries of observability, drift, and feedback.
Interpret both examples and distinguish offline fixture results from model or production evidence.
Diagnose the listed failures and justify the worked decisions while preserving task, data, permission, and uncertainty contracts.

Observe the application contract

Operational telemetry should explain whether the authorized task completed correctly and where a failure occurred. Request count, latency, token or unit usage, retrieval coverage, schema errors, unsupported claims, abstention, and tool denials describe different things. A provider success status is not a supported answer, and a valid schema is not an authorized effect. The first synthetic JSON event contains only safe stage and version references. It records no real prompt or user and is never exported. Use enough context to investigate without copying every document, credential, or model-visible byte into a permanent log.

json Synthetic privacy-conscious diagnostic event

{"event":"candidate_rejected","stage":"claim_support","bundle":"lab-bundle-4","corpus":"lab-corpus-7","prompt":"lab-prompt-3","validator":"lab-validator-2","operation_reference":"public-lab-a","raw_prompt_logged":false}

The record names the stage and versions without containing an actual prompt, answer, token, or user. It is syntax-checked only and is not exported to a service.

Record the full candidate bundle

Model identity alone is insufficient when prompts, validators, retrieval, source versions, chunking, adapters, decoding, caches, and policy can change behavior. Keep a release association for these inputs and the evaluation protocol. Managed service aliases or runtime defaults may change, so verify available version controls and document reproducibility limits under the actual contract. Identical settings do not justify a universal claim of bit-for-bit generation determinism. Use repeated or paired evaluation when necessary and interpret differences with task and sampling uncertainty rather than blaming the latest component named in a release note.

Separate input, source, and outcome shifts

A changed user mix, new language, updated policy corpus, retrieval bug, altered model, or evaluator drift can all change scores. The second offline example compares two authored error-rate windows and preserves their denominators. It does not prove a real regression, statistical significance, or cause. Check whether labels and eligibility changed before comparing percentages. A rising abstention rate can mean missing evidence, a stricter policy, or a new task distribution; it is not automatically worse generation. Keep representative slices and reviewed failure cases alongside aggregates, including cases where telemetry itself is missing.

python Offline denominator-preserving comparison

windows = [("baseline", 2, 100), ("candidate", 4, 100)]
for name, errors, eligible in windows:
    print(name, f"{100 * errors / eligible:.1f}%", eligible)
print("authored counts; no causal or significance claim")

Expected output

baseline 2.0% 100
candidate 4.0% 100
authored counts; no causal or significance claim

The fixture rates are two and four percent over one hundred cases each. These invented counts do not establish a production shift or identify its cause.

Design feedback without contaminating evaluation

User feedback is evidence to investigate, not automatically a trustworthy gold label. A positive rating can reward style while missing factual or access failure; an abusive report can be misleading. Review consent, data rights, sensitivity, and representativeness before retaining feedback or using it for training. Keep incident examples and development corrections separate from the protected evaluation set, recording any exposure. Human review and automated scoring also need rubric and version ownership. Do not silently treat the same generated answer as both a prediction and its reference, which creates a circular evaluation that can never reveal the relevant mistake.

Use monitoring to trigger bounded decisions

Define owners, thresholds, uncertainty review, rollback or fallback, and privacy-safe evidence for meaningful incidents. An alert should identify an actionable user or policy impact rather than every isolated candidate failure. Retention, access, redaction, and telemetry failure behavior are part of the design. No real monitoring backend, model API, trace exporter, user account, or feedback system is contacted here. Local fixtures establish calculations and record shapes only. Observability is successful when it supports a justified correction and preserves the limits of the evidence, not when a dashboard displays many numbers or a guessed root cause.

Common errors and fixes

A rate change is immediately called model drift

Why it happens: Input mix, labels, source, and evaluator changes were ignored.
How to fix it: Compare versioned stages and denominators before attribution.
Prevent it next time: Track representative slices and data lineage.

Raw prompts are logged by default

Why it happens: Diagnostic convenience overrode privacy and retention design.
How to fix it: Use safe metadata and reviewed selective evidence handling.
Prevent it next time: Test redaction, access, and deletion workflows.

Feedback becomes its own evaluation answer key

Why it happens: Predictions and reference labels were not independent.
How to fix it: Use reviewed independent evidence and protected evaluation cases.
Prevent it next time: Record label origin and review uncertainty.

Check your understanding

1. Abstention rises after the corpus loses several documents. Is the generator necessarily worse?

Show worked solution

No. The system may be correctly reflecting missing evidence. Inspect corpus coverage, retrieval, context packing, policy, and task mix before assigning blame. Measure usefulness and supported-answer behavior with the changed source state recorded.

2. Why should an error-rate chart include eligibility counts and version metadata?