MLOps / Prediction and evidence foundations / Lesson 2

Data schemas and quality gates

Validate meaning and eligibility at the data boundary without disguising policy changes as cleaning.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Define type, units, identity, ranges, and missingness.
Report rejected records and reason counts.
Distinguish schema compatibility from data-distribution evidence.

A schema includes meaning beyond field types

A numeric field can represent minutes, seconds, an identifier, or a coded category. Specify names, types, units, allowed values, missingness, observation identity, and when the value is available. Feature order matters for positional model inputs even when every name is present. A schema version is a useful identifier only if it actually refers to a preserved contract. Do not accept a request merely because it carries a familiar version number. Check the relevant structure and semantic metadata, and define what happens when the producer sends unsupported or ambiguous data.

Quality rules encode eligibility decisions

A nonnegative finite duration rule is appropriate only when that follows from the measurement definition. Real negative values may instead represent a legitimate signed change, and zero may mean a real observation rather than missingness. The first fixture applies a declared duration rule and preserves rejected ids with reasons. It does not infer the rule from the observed values. Choose whole-batch rejection, row quarantine, or another justified behavior before interpreting the model result. A cleanup that silently removes difficult observations can improve a metric while changing the population the model is supposed to serve.

python Deterministic offline standard-library fixture

from math import isfinite
rows = [{"id":"a", "minutes":3.0}, {"id":"b", "minutes":float("nan")},
        {"id":"c", "minutes":-1.0}]
accepted, rejected = [], []
for row in rows:
    value = row["minutes"]
    reasons = []
    if not isfinite(value): reasons.append("nonfinite")
    elif value < 0: reasons.append("negative")
    (rejected if reasons else accepted).append((row["id"], reasons))
print("accepted", [key for key,_ in accepted])
print("rejected", rejected)

Expected output

accepted ['a']
rejected [('b', ['nonfinite']), ('c', ['negative'])]

The declared nonnegative finite-duration rule produces explicit row identities and reasons; it does not silently replace rejected measurements.

Count coverage and defects separately

Report total inputs, accepted records, rejected records, duplicates, missingness, and rule-specific defects. One record can violate several rules, so reason counts may sum to more than rejected-row counts. A denominator must identify whether it refers to rows, entities, valid targets, or events. Investigate spikes against comparable populations and collection periods. A small batch can produce unstable rates, while a stable average can conceal one source with a severe defect. Preserve provenance sufficient to locate the producing boundary without placing private raw records or credentials into public logs or teaching examples.

Compatibility is an ordered relationship

The second fixture changes feature order and units while preserving the version and the set of feature names. A positional prediction function could therefore consume valid-looking numbers with the wrong meaning. A repair might reorder by names under a validated mapping, or reject a request that violates the declared interface. It should not guess from magnitudes alone. Compatibility also includes category vocabularies, normalization parameters, time interpretation, and optional-field defaults. Keep producer and consumer contracts testable with a small known input that would fail if two coordinates were swapped.

python Deterministic offline standard-library fixture

schema = {"features":["booked_seats", "days_remaining"],
          "units":["count", "days"], "version":1}
request = {"features":["days_remaining", "booked_seats"],
           "units":["days", "count"], "version":1}
print("same version", request["version"] == schema["version"])
print("same ordered contract", request == schema)
print("same feature set", set(request["features"]) == set(schema["features"]))

Expected output

same version True
same ordered contract False
same feature set True

A matching version and feature set do not establish ordered feature compatibility; names, positions, units, and interpretation belong together.

Quality gates are evidence with explicit limits

A gate can prove that supplied data passed its implemented checks. It cannot prove the labels are correct, the population is representative, or every hidden measurement problem is absent. Schema checks and distribution checks complement each other: a unit switch can violate a semantic rule, while a legitimate seasonal change can alter a distribution without being defective. Record the rule version, result counts, and blocked or accepted artifact identity. These examples validate synthetic in-memory records only. They do not inspect a production dataset, run a validation platform, or claim that a universal threshold makes a model trustworthy.

Common errors and fixes

Invalid records disappear without an audit

Why it happens: A generic cleaning step silently changed eligibility.
How to fix it: Preserve counts, ids where permitted, and reasons under the chosen policy.
Prevent it next time: Review rejection rules before evaluating model outcomes.

Matching column names are treated as positional compatibility

Why it happens: Order, units, or transformations differed.
How to fix it: Validate the complete ordered contract or apply an explicit mapping.
Prevent it next time: Keep a sentinel input that exposes coordinate swaps.

Passing a schema gate is reported as data correctness

Why it happens: Implemented structural checks were confused with all semantic properties.
How to fix it: State what was checked and assess labels and provenance separately.
Prevent it next time: Keep limitations and rule versions alongside gate results.

Check your understanding

1. Three records are rejected, but nonfinite and range reason counts total four. Is this necessarily inconsistent?

Show worked solution

No. A record can fail more than one rule, so reason counts can overlap. Report unique rejected-row count separately from per-rule counts, preserve a deterministic reason policy, and verify that accepted plus rejected rows cover the supplied input without duplication.

2. A duration source changes seconds to milliseconds but keeps numeric type. What should happen?