R Programming / Analysis interfaces and table integrity / Lesson 11
Linear models and design matrices
Read a fitted model through its data, formula, numerical rank, residuals, and prediction boundary.
On this page
Lesson goals
- Trace a formula into a model matrix.
- Extract coefficients and predictions deliberately.
- Distinguish exact fit, identifiability, and valid inference.
A formula declares a model structure
The formula y ~ x describes a response and predictor terms, with an intercept included by default in an ordinary linear model. It is a language object evaluated using data and an associated environment. Supply a data frame explicitly so input columns are visible. Formula operators have modeling meanings; adding I around an arithmetic expression asks for that expression as a term rather than another formula operation. A design matrix translates terms and category coding into numeric columns. Inspect that matrix when the coefficient interpretation is unclear instead of inferring meaning from a printed formula alone.
Separate fitting from prediction inputs
The first fixture supplies four points exactly on a line and fits lm with na.fail so missing observations cannot be silently omitted under the example contract. The coefficients are one and two, and predictions at four and five are nine and eleven. Extract the result through coef and predict rather than depending on the layout of a printed model summary. Newdata must preserve feature meaning, factor-level policy, and transformation definitions. The displayed predictions extrapolate beyond the supplied x range; exact agreement with a chosen synthetic rule does not justify real-world extrapolation.
d <- data.frame(x = 0:3, y = c(1, 3, 5, 7))
fit <- lm(y ~ x, data = d, na.action = na.fail)
cat(sprintf("intercept=%.1f; slope=%.1f\n", coef(fit)[1], coef(fit)[2]))
predicted <- predict(fit, newdata = data.frame(x = c(4, 5)))
cat("predictions=", paste(round(predicted, 6), collapse = ","), "\n", sep = "")
intercept=1.0; slope=2.0
predictions=9,11The synthetic observations lie exactly on y=1+2x; this checks formula and prediction semantics without establishing inference assumptions or external predictive quality.
Rank determines which coefficients can be identified
The second fixture includes both x and twice x alongside an intercept. These columns contain only two independent directions, so the three-column matrix has rank two. Many combinations of the two slope coefficients produce the same fitted values. A software routine may mark an aliased coefficient missing or apply a numerical rank rule, but the underlying identifiability problem remains. Adding observations on the same proportional relationship does not create independent information. Review feature definitions, contrasts, and measurement dependencies before treating a missing coefficient as an ordinary missing-data issue.
d <- data.frame(x = 0:3, y = c(1, 3, 5, 7))
X <- model.matrix(~ x + I(2*x), data = d)
cat("columns=", ncol(X), "; rank=", qr(X)$rank, "\n", sep = "")
cat("duplicated slope information=", isTRUE(all.equal(X[, 3], 2*X[, 2])), "\n", sep = "")
columns=3; rank=2
duplicated slope information=TRUEThe intercept and two proportional feature columns cannot identify three independent coefficients; the rank diagnosis does not require reporting an unstable fit.
Residuals describe a fitted discrepancy
A residual is observed response minus fitted value for a training observation. Patterns against fitted values, predictors, time, or groups can reveal a mismatch with assumptions, unusual observations, or an inappropriate functional form. A small residual sum alone does not establish independent errors, constant variance, a causal effect, or reliable predictions on new data. Distinguish a confidence interval for a conditional mean from a prediction interval for a new response; their uncertainty targets differ. Model diagnostics require enough appropriate observations, while a perfect four-point synthetic line is mainly useful for checking the programming interface.
Keep data selection and transformation honest
Document which rows entered the fit, how missingness was handled, what units and category levels were used, and which choices were informed by validation outcomes. Formula evaluation can look outside the supplied data when a name is missing, creating an accidental environment dependency. Validate required names before fitting and preserve the observation ids used by the model. For predictive selection, learn preprocessing under the permitted training boundary. For inference, justify the sampling and error assumptions separately. These examples provide derived base-R behavior only; no lm fit, QR decomposition, model parser, or inferential runtime was executed during review.
Common errors and fixes
A coefficient is missing despite complete measurements
- Why it happens
- The design matrix is rank deficient or terms are aliased.
- How to fix it
- Inspect model.matrix and independent column relationships.
- Prevent it next time
- Check rank and feature definitions before interpreting coefficients.
A formula uses an unexpected external vector
- Why it happens
- A required name was absent from data and resolved in the formula environment.
- How to fix it
- Validate names and supply the intended data explicitly.
- Prevent it next time
- Run self-contained fixtures without hidden workspace variables.
An exact training fit is presented as reliable forecasting
- Why it happens
- Interpolation of supplied points was confused with external validity.
- How to fix it
- Evaluate new-data behavior and assumptions under an honest protocol.
- Prevent it next time
- Label synthetic interface checks separately from predictive evidence.
Check your understanding
1. Why can x and 2*x not identify two separate effects in this model?
Show worked solution
Their contributions combine as beta1*x+beta2*2*x=(beta1+2*beta2)*x. The data identify the combined slope, not both coefficients independently. Remove or redefine redundant terms according to the modeling question rather than assigning meaning to an arbitrary numerical solution.
2. What should be checked before predict receives a new factor level?
Show worked solution
Compare the new labels with the fitted level and contrast contract, define a supported unknown-category policy, and verify preprocessing. A numeric code of the same type is not sufficient. Do not silently map a new meaning onto an old level merely to make prediction complete.
Key takeaways
- Formula terms become numeric design columns.
- Rank and coefficient interpretation require explicit inspection.
- Training fit, prediction evidence, and inference assumptions differ.