Machine Learning / Evidence and operational responsibility / Lesson 15

Fairness, slices, and potential harms

Assess context-dependent harms and subgroup evidence without reducing fairness to one aggregate number or a proxy-removal rule.

About 4 min readContent reviewed 2026-10-26

On this page

Lesson goals

Fairness starts with people and the decision
Inspect equal accuracy with different error patterns
Evaluate mitigation and the full workflow

Fairness starts with people and the decision

A fairness assessment asks who can be harmed, how the system is used, and which outcomes matter. It cannot be completed by computing one statistic after training. Labels, sampling, access to the service, and the decision policy can all reflect historical or operational inequities. Identify affected groups and relevant intersections with appropriate privacy and governance controls. Removing a sensitive column does not guarantee fairness because other features can encode related information and the target itself can be biased. Include domain stakeholders and an appeal or review process where the use case requires it. Technical evidence supports a contextual judgment rather than replacing that judgment.

Different criteria ask different questions

Selection-rate parity concerns how often groups receive a positive decision. Equalized-odds-style criteria compare true-positive and false-positive rates conditional on outcomes, while equality of opportunity focuses on an appropriate true-positive-rate condition. Calibration describes probability agreement and is another distinct property. Which criterion matters depends on the meaning of the decision, the quality of labels, and the harm being considered. Objectives can conflict, and a metric threshold is not a universal ethical or legal certificate. State the chosen definitions, reference population, comparison groups, and acceptable uncertainty. Do not silently swap definitions until one makes the system look favorable.

Inspect equal accuracy with different error patterns

The first example contains two invented groups with four observations each. Both have the same accuracy in this fixture, yet their selection, true-positive, and false-positive rates differ. The printed table exposes those denominators instead of declaring one overall score fair. These tiny groups cannot support a real population conclusion, and the example contains no real demographic data. A practical assessment needs support counts, uncertainty, label-quality review, and consequences of the particular mistakes. If a denominator is zero, report that undefined case rather than replacing it with a convenient number that suggests evidence where none exists.

python Fixed synthetic offline example

import numpy as np
group = np.array(["A"] * 4 + ["B"] * 4)
y = np.array([1, 1, 0, 0, 1, 1, 0, 0])
pred = np.array([1, 0, 0, 0, 1, 1, 1, 0])
for name in ["A", "B"]:
    mask = group == name
    actual, chosen = y[mask], pred[mask]
    tpr = np.sum((actual == 1) & (chosen == 1)) / np.sum(actual == 1)
    fpr = np.sum((actual == 0) & (chosen == 1)) / np.sum(actual == 0)
    print(f"{name} n={int(mask.sum())} selection={chosen.mean():.2f} tpr={tpr:.2f} fpr={fpr:.2f}")

Expected output

A n=4 selection=0.25 tpr=0.50 fpr=0.00
B n=4 selection=0.75 tpr=1.00 fpr=0.50

All denominators are positive in this invented table. Equal overall accuracy would not reveal these different conditional errors.

Aggregation can conceal a small group

The second example compares a row-weighted error with an equal-group mean. The larger group has no supplied errors and the smaller group has only errors, so the two summaries differ. Neither weighting is automatically the correct normative objective. Report the choice and examine each group rather than hiding a weak slice behind volume. Intersectional groups can reveal failures absent from broad averages, but their small sample sizes also increase uncertainty and privacy concerns. Repeatedly searching many slices can find unstable extremes, so keep exploratory findings distinct from a prespecified confirmatory assessment.

python Fixed synthetic offline example

import numpy as np
error = np.array([0.0, 0.0, 0.0, 0.0, 1.0, 1.0])
group = np.array(["large"] * 4 + ["small"] * 2)
means = [error[group == name].mean() for name in ["large", "small"]]
print(f"row-weighted error={error.mean():.3f}")
print(f"macro-group error={np.mean(means):.3f}")

Expected output

row-weighted error=0.333
macro-group error=0.500

The different summaries reflect different weighting units. The code does not declare either number a universal fairness score.

Evaluate mitigation and the full workflow

A mitigation can change thresholds, training weights, representation, data collection, or the human process. Validate its impact on the chosen harm and important secondary outcomes rather than optimizing a metric in isolation. Group-specific decision policies can raise governance and deployment questions that require context-specific review, not an automatic code recipe. Test accessibility, recourse, monitoring, and the behavior of reviewers as well as model scores. A seemingly fair predictor can be embedded in an unfair allocation process. Preserve the assumptions and limitations in the model record and revisit them when the population, labels, or operating policy changes.

Common errors and fixes

One accuracy value is labeled fair

Why it happens: Subgroup harms and error types were not assessed.
How to fix it: Choose contextual criteria and inspect supported slices.
Prevent it next time: Report counts and limitations beside every group metric.

Sensitive-column removal is treated as a guarantee

Why it happens: Proxies, labels, and workflow effects remain.
How to fix it: Assess the complete data and decision process.
Prevent it next time: Review representations and outcomes, not only column names.

A tiny slice rate is treated as a stable population fact

Why it happens: Support and uncertainty were ignored.
How to fix it: Report limited evidence and obtain appropriate additional data.
Prevent it next time: Distinguish exploratory slicing from confirmatory checks.

Check your understanding

1. Can equal accuracy across two groups establish equalized odds?

Show worked solution

No. Accuracy combines correct positives and negatives under each group’s prevalence. Equalized-odds-style comparisons examine conditional true-positive and false-positive rates separately. Compute their support counts and uncertainty, and decide whether that criterion addresses the actual harm rather than treating equality alone as sufficient.

2. Why should a mitigation be tested beyond the metric it optimizes?