Machine Learning / Prediction and data foundations / Lesson 1

Problem framing and useful baselines

Turn an operational question into a prediction contract, then compare learning with a simple defensible alternative.

About 5 min readContent reviewed 2026-10-26

On this page

Lesson goals

Define the decision before choosing an algorithm
Write an explicit feature and target contract
Define success and a reason to stop

Define the decision before choosing an algorithm

Machine learning estimates patterns from examples to support a defined task. Begin with the decision someone will make, the available information, and the consequences of error. Predicting tomorrow’s workshop attendance differs from explaining why attendance changed or deciding which intervention causes improvement. A predictive association does not automatically establish causation. State the prediction unit, prediction time, target horizon, and intended user. Also ask whether a rule or ordinary calculation already solves the problem. A more complex model is useful only if its verified benefit outweighs the operational, data, and maintenance costs under that actual contract.

Match the learning setting to the question

Regression predicts a numeric target under a chosen loss, while classification estimates categories or their scores. Clustering explores similarity without supplied target labels, and dimensionality reduction constructs a smaller representation. These are different tasks, not interchangeable names for “AI.” A class label can hide an uncertain or delayed measurement, and a continuous target can have units or bounds that matter to evaluation. Define what one row represents and whether repeated rows come from the same entity. That decision later determines valid splits and uncertainty estimates. This course uses small invented records solely to expose such reasoning, never as evidence of real-world accuracy.

Write an explicit feature and target contract

The first example separates a two-feature array from a numeric target vector and checks their alignment. Columns mean booked seats and days until the event; the target means eventual attendance. These meanings must be documented outside the bare array because NumPy cannot infer them. In particular, a feature must exist at the declared prediction time. A final attendance count accidentally included among inputs would make a seemingly strong prediction meaningless. Record units, allowed values, missingness, and entity/time identifiers alongside the schema. Array shape checks are useful executable evidence, but shape alone cannot prove that a feature is legitimate or correctly measured.

python Fixed synthetic offline example

import numpy as np
X = np.array([[10.0, 7.0], [20.0, 3.0], [15.0, 1.0]])
y = np.array([8.0, 18.0, 12.0])
assert X.ndim == 2 and y.ndim == 1 and X.shape[0] == y.shape[0]
print(X.shape, y.shape)

Expected output

(3, 2) (3,)

Three rows align with three targets. The two feature meanings remain part of the documented schema, not something shape can validate.

Establish a baseline under the same evaluation rule

The second example uses the training-target mean as a constant predictor and computes absolute error on separate fixed outcomes. A regression mean baseline is natural for squared-error training, while a median can suit absolute-error objectives; choose the comparator according to the loss and operational task. Do not recompute the baseline from evaluation targets, which would leak their information. Classification may use a majority class or a prior-probability baseline, and time-ordered work may use a last-observation rule. The tiny calculation verifies arithmetic only. It does not establish that this constant model is suitable for a real forecasting system.

python Fixed synthetic offline example

import numpy as np
y_train = np.array([2.0, 4.0, 6.0])
y_test = np.array([3.0, 5.0])
prediction = np.full(y_test.shape, y_train.mean())
print(prediction.tolist())
print(f"fixture MAE={np.abs(y_test - prediction).mean():.2f}")

Expected output

[4.0, 4.0]
fixture MAE=1.00

The fixed predictor learns four from training only. Both held-out errors have magnitude one; this is a fixture result, not a benchmark.

Define success and a reason to stop

Choose a primary metric, important slices, resource limits, and a fallback before tuning many models. Compare alternatives on the same held-out protocol rather than selecting whichever metric makes each look best. A model can improve average loss while harming a critical subgroup or becoming too slow to use. Label availability, human review, privacy, and the ability to act on a prediction can determine whether the project is worthwhile. Keep a record of the baseline, assumptions, and known limitations. A decision not to deploy can be the correct outcome when evidence is weak or no useful improvement appears.

Common errors and fixes

The algorithm is chosen before the task is defined

Why it happens: No prediction time, unit, or loss was specified.
How to fix it: Write the operational contract and a baseline first.
Prevent it next time: Review data availability and decision consequences before modeling.

A future measurement is used as an input

Why it happens: The feature was unavailable at prediction time.
How to fix it: Remove it or redefine the legitimate task explicitly.
Prevent it next time: Audit each feature’s availability timestamp.

The baseline uses the test-set mean

Why it happens: Evaluation outcomes influenced the comparator.
How to fix it: Fit the baseline using training data only.
Prevent it next time: Apply the same separation rules to simple and complex models.

Check your understanding

1. Frame a prediction for an event organizer who acts three days before an event.

Show worked solution

Define one event as the unit, attendance as the future target, and the three-day cutoff as the feature-availability boundary. Bookings known then may be valid; final check-in counts are not. Choose the decision and loss, then compare with a training-only historical baseline on a time-respecting evaluation.

2. Why can a simpler constant predictor remain the right product choice?