Data Analysis / Intermediate: Relationships and Uncertainty / Lesson 10

Exploratory Relationships and Stratification

Inspect paired patterns, subgroup mixtures, and unusual observations without confusing a descriptive association with a causal explanation.

About 5 min readContent reviewed 2026-10-25

On this page

Lesson goals

Explain what Pearson correlation measures and misses.
Reconcile pooled and stratified rates.
Investigate outliers and preserve the exploratory nature of findings.

Match observations before comparing variables

A relationship analysis needs correctly paired observations: each x and y must belong to the same unit and relevant period. Sorting the two columns independently destroys pairing and can manufacture an impressive correlation. State missing-pair exclusions, measurement units, and repeated observations per person or device before calculating a coefficient. A large row count does not guarantee many independent units.

Inspect a scatter plot and subgroup context alongside a numeric summary. Pearson correlation describes linear association on the selected scale, not every form of relationship. It can be unstable with influential observations and undefined when a variable has no variation. Neither a high correlation nor a confidence interval around it proves that changing one variable causes the other to change.

A zero linear coefficient can hide a clear pattern

The first program pairs symmetric x values with their squares. Pearson correlation is zero although y is determined exactly by x squared. The opposing linear directions cancel, so the coefficient does not describe the curved structure. The printed pairs make that structure inspectable even without an image. This is a counterexample to “uncorrelated means unrelated,” not an estimate from a random study.

Use transformations only with a stated reason and retain the original scale's interpretation. Searching many transformations and reporting only the strongest pattern creates selection effects. For categories, nonlinear patterns, or repeated measures, another summary or model may be needed. The right next step is to formulate and check a question, not to force every relationship into one universal correlation score.

python Inspect a nonlinear relationship with zero Pearson correlation

from statistics import correlation

x = [-2, -1, 0, 1, 2]
y = [value * value for value in x]
print("pairs", list(zip(x, y)))
print(f"pearson {correlation(x, y):.3f}")

Expected output

pairs [(-2, 4), (-1, 1), (0, 0), (1, 1), (2, 4)]
pearson 0.000

Symmetric x and squared y produce Pearson0.000, despite the exact nonlinear relationship. The paired rows remain visible.

Explain how subgroup mixtures reverse pooled comparisons

A pooled rate weights subgroup rates by each group's observed denominators. If groups encounter different mixtures of easy and hard cases, the pooled ranking can reverse the within-stratum ranking, often called Simpson's paradox. The second program makes A lower than B in both strata yet much higher when pooled because A has a larger easy-case share. No arithmetic contradiction is involved.

Equal-mixture standardization produces another target: the rate each group would have under the same half-easy, half-hard weighting of its observed stratum rates. It is a descriptive comparison, not automatically a causal adjustment or prediction. Choose weights from the question and intended population. Do not select a stratifier merely because it creates a preferred ranking; inappropriate conditioning can introduce other biases.

python Pooled and common-mixture rates answer different questions

data = {
    "A": {"easy": (90, 100), "hard": (1, 10)},
    "B": {"easy": (19, 20), "hard": (20, 100)},
}
for stratum in ("easy", "hard"):
    rates = {g: data[g][stratum][0] / data[g][stratum][1] for g in data}
    print(f"{stratum} A{rates['A']:.1%} B{rates['B']:.1%}")
pooled = {g: sum(s for s, n in data[g].values()) /
             sum(n for s, n in data[g].values()) for g in data}
equal_mix = {g: sum(s / n for s, n in data[g].values()) / 2 for g in data}
print(f"pooled A{pooled['A']:.1%} B{pooled['B']:.1%}")
print(f"equal_mix A{equal_mix['A']:.1%} B{equal_mix['B']:.1%}")

Expected output

easy A90.0% B95.0%
hard A10.0% B20.0%
pooled A82.7% B32.5%
equal_mix A50.0% B57.5%

A has lower rates in both strata but a much larger easy-case share. Applying common weights changes the comparison target and restores B's higher descriptive rate.

Investigate influence without deleting inconvenient data

An unusual point can be a recording error, a unit mismatch, a genuine rare event, or a distinct subgroup. Its influence depends on the summary: an extreme value can move a mean or fitted line far more than a median. Flagging a point is a prompt for investigation, not evidence that it is false. Trace provenance and measurement context before changing it.

A sensitivity table can show results with all observations and under a justified alternative rule. For values10,11,12,13,100, the mean is29.2 and median12; excluding100 yields mean11.5. Report that exclusion and its reason rather than presenting the smaller mean as repaired truth. Robust methods reduce sensitivity but still need a clearly defined target and valid data.

Keep discovery separate from confirmation

Exploration generates hypotheses and exposes data problems. Record the filters, transformations, subgroup choices, and number of comparisons tried so the final pattern is not mistaken for a prespecified test. Holdout data or a later planned study can evaluate a discovered relationship under an appropriate design. A persuasive story added after the plot does not establish the mechanism.

Before communicating, state the population represented, measurement limits, missingness, possible confounding, and uncertainty. Ask whether the claimed action requires causal evidence that descriptive data cannot supply. A careful exploratory conclusion says what was observed, which alternative explanations remain, and what additional evidence would distinguish them. It neither dismisses every pattern nor promotes every pattern into a causal finding.

Common errors and fixes

A near-zero correlation was reported as no relationship.

Why it happens: A linear summary was used as a universal dependence test.
How to fix it: Inspect paired plots and plausible nonlinear structure.
Prevent it next time: Include nonlinear and constant-variable cases in analysis checks.

Pooled rankings were interpreted as like-for-like performance.

Why it happens: Groups had different subgroup mixtures and denominators.
How to fix it: Show stratified rates and a justified common-mixture comparison.
Prevent it next time: Record the comparison population and weighting rule.

An influential point disappeared without explanation.

Why it happens: An inconvenient observation was treated as invalid automatically.
How to fix it: Investigate provenance and report sensitivity under explicit rules.
Prevent it next time: Keep raw data and exclusion reasons.

Check your understanding

1. Why does a zero Pearson value for y=x² on symmetric x not imply no relationship?

Show worked solution

Pearson summarizes linear co-variation. Positive and negative x directions cancel here while the curve is exact. Inspect the paired values or scatter plot, and describe the nonlinear relationship without claiming that a synthetic mathematical pattern establishes real-world causality.

2. A has rates90% and10%; B has95% and20%. Compare them under equal stratum weights.