Resolve inference schema and preprocessing mismatches

Understand the likely causes

• Likely causes include refitting a scaler at inference, omitting an imputer, rebuilding a category vocabulary, reversed feature columns, inconsistent target inversion, nonfinite values, or incompatible artifacts.
• NumPy broadcasting can turn a shape mistake into a plausible wrong result. A matching feature count alone does not establish matching meaning.

1. Confirm the artifact and execution context

Record the immutable estimator, preprocessing, class/target mapping, threshold where relevant, and runtime versions. Confirm that both paths actually load the intended reviewed artifacts rather than similarly named stale files. Use synthetic values that expose different columns and units, not an all-zero record that can hide swaps. Do not load an untrusted pickle-like file to investigate the issue. Serialization can execute code, so artifact trust remains a separate requirement. Keep credentials and private feature records out of the report; a reproducible local fixture is usually more useful than a sensitive production payload.

2. Check semantic schema before numeric shape

Compare feature names and order, units, types, allowed categories, missingness, and finite-value policy. Then inspect array dimensions, batch axis, and output shape. The local example rejects a reversed name list even though both matrices have two columns. This is a deliberate boundary check, not a universal schema framework. If the production API uses named fields, map them through the frozen schema rather than relying on dictionary insertion order from an unrelated client. Reject or route invalid records under the documented fallback; replacing every invalid input with zero can conceal the defect and create confident-looking wrong answers.

import numpy as np
expected = ["bookings", "days"]
def validate(values, names):
    if list(names) != expected:
        raise ValueError("Feature order mismatch")
    X = np.asarray(values, dtype=float)
    if X.ndim != 2 or X.shape[1] != 2 or not np.isfinite(X).all():
        raise ValueError("Invalid feature matrix")
    return X
print(validate([[3.0, 5.0]], expected).shape)
try:
    validate([[3.0, 5.0]], ["days", "bookings"])
except ValueError as error:
    print(error)

(1, 2)
Feature order mismatch

3. Trace each fitted transformation in order

Inspect the stored imputation values, means, scales, selected indices, component basis, and category vocabulary. Compare intermediate values on one fixture through both implementations. Future batches should apply fitted values rather than recompute their own statistics. Preserve the declared policy for constant or all-missing training columns and unknown categories. If the target was transformed, invert predictions consistently before evaluating in original units. A different library version or numeric backend can add variation, but do not blame floating-point noise before ruling out an obvious column swap, missing transform, or stale calibration mapping.

4. Verify records, batches, and controlled failures

Test the same record alone and in differently composed batches, valid boundary values, missing fields, extra columns, unknown categories, and nonfinite inputs. The approved output convention should remain stable, and invalid cases should produce the documented rejection or fallback. Compare predictions with an appropriate tolerance only after semantic equivalence is established. Add the exposing fixture to regression tests and release preprocessing and estimator versions together. A fixed local prediction match is evidence for that fixture, not proof of general predictive quality or every deployment platform. The repair should identify the exact contract that diverged and preserve a known-good rollback artifact.

Keep it from recurring

• Version the whole inference contract, preserve feature names and units, and validate input before numeric operations. Keep training/inference parity fixtures and an artifact trust policy; a seed, filename, or successful matrix product is not a substitute.