R Programming / Analysis interfaces and table integrity / Lesson 12

Tests, intervals, and dependence

Use R’s inferential interfaces with an explicit estimand, sampling unit, and comparison family.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

State the null, alternative, and interval target.
Choose paired versus independent structures from the design.
Interpret multiplicity adjustment without overstating evidence.

The design comes before the function call

A hypothesis test evaluates a statistic under a specified null model and sampling assumptions. A confidence interval targets a parameter through a procedure with a repeated-sampling coverage interpretation. Before calling a function, define the observational unit, target quantity, dependence structure, and selection process. Repeated measurements from one person or device are not automatically independent samples. A technically successful t.test call cannot validate the design. This lesson uses invented numbers to teach R interfaces and interpretation, and offers no medical, financial, legal, or other consequential recommendation based on those numbers.

Read a returned test as a structured object

The first fixture calls a two-sided one-sample t test with a declared reference mean and confidence level. The returned htest object includes an estimate, statistic, degrees of freedom, p-value, and interval. Extract named components instead of parsing the printed report. The synthetic mean is three and the Student interval is displayed to three decimals. Its interpretation depends on the relevant sampling and distribution assumptions, not on the fact that the endpoints were computed. A 95% confidence procedure does not assign a 95% frequentist probability to the fixed parameter lying in this particular realized interval.

r Illustrative base-R script; R runtime not available during review

x <- c(1, 2, 3, 4, 5)
result <- t.test(x, mu = 0, alternative = "two.sided", conf.level = 0.95)
cat(sprintf("mean=%.1f; df=%.0f\n", unname(result$estimate), unname(result$parameter)))
cat(sprintf("interval=%.3f,%.3f\n", result$conf.int[1], result$conf.int[2]))

Expected output

mean=3.0; df=4
interval=1.037,4.963

The interval is the one-sample Student interval under its assumptions for these five synthetic values; no substantive population inference is intended.

Pairing is a relationship between observations

For before-and-after measurements on the same units, a paired analysis works with aligned within-unit differences under suitable assumptions. Equal vector lengths alone do not establish correct pairing. Join by the stable unit id and inspect missing pairs. An independent two-sample analysis answers a different question and has a different uncertainty calculation; Welch and equal-variance variants also encode different assumptions. Do not choose the variant that yields the smallest p-value after trying several. Record the planned comparison and any exclusions. A small p-value does not measure effect size, practical importance, or the probability that the null hypothesis is true.

Multiple comparisons require a declared family

The second fixture applies Holm adjustment to three supplied p-values and counts decisions at 0.05. The family size and method are explicit. Multiplicity procedures control a stated error criterion under their conditions; they do not repair biased sampling, invalid tests, or selective reporting. Effect estimates and uncertainty remain necessary, and the adjusted p-value is not a corrected effect size. If comparisons were selected after inspecting many others, adjusting only the displayed subset can misrepresent the actual selection process. Keep the full comparison plan and report relevant negative results.

r Illustrative base-R script; R runtime not available during review

raw <- c(0.01, 0.04, 0.03)
adjusted <- p.adjust(raw, method = "holm")
cat("raw=", paste(raw, collapse = ","), "\n", sep = "")
cat("adjusted=", paste(adjusted, collapse = ","), "\n", sep = "")
cat("rejections at 0.05=", sum(adjusted <= 0.05), "\n", sep = "")

Expected output

raw=0.01,0.04,0.03
adjusted=0.03,0.06,0.06
rejections at 0.05=1

The declared family has three supplied p-values; adjustment changes the decision rule without changing effect sizes or repairing invalid tests.

Check assumptions and report limits visibly

Inspect distributions, outliers, missingness, sample sizes, and dependence at the unit that supports the analysis. Alternative procedures such as permutation or rank methods have their own assumptions and targets; they are not automatic repairs for every failed normality check. Report the estimated effect with units, interval definition, eligible counts, test choice, and important limitations. Distinguish a failure to reject from proof of equivalence; equivalence needs its own margin and procedure. All R results in this lesson are derived expectations. No t.test, quantile routine, adjustment routine, or inferential engine was executed during review.

Common errors and fixes

A paired test compares different units by row position

Why it happens: Vectors were sorted or filtered independently.
How to fix it: Align complete pairs by stable identity before constructing differences.
Prevent it next time: Test known pairs and report missing-pair exclusions.

A p-value is described as the probability the null is true

Why it happens: The conditional sampling calculation was inverted.
How to fix it: State the null-model meaning and report effect and interval separately.
Prevent it next time: Use a fixed interpretation checklist for test outputs.

Only favorable comparisons receive multiplicity adjustment

Why it happens: The reported family omitted earlier inspected comparisons.
How to fix it: Define and disclose the relevant comparison and selection family.
Prevent it next time: Record planned tests and analysis revisions.

Check your understanding

1. Why can five repeated measurements from one device not automatically replace five independent devices?

Show worked solution

Within-device dependence changes the information and sampling uncertainty. The unit supporting the claim may be the device, with repeated measurements requiring an appropriate hierarchical or repeated-measures design. A function receiving five numbers cannot infer that structure from their vector length.

2. What changes in the Holm fixture at a 0.05 decision threshold?