Machine Learning / Models and controlled selection / Lesson 10

Clustering and distance

Explore similarity under a declared representation and evaluate clusters without pretending that unsupervised groups are true categories.

About 4 min readContent reviewed 2026-10-26

On this page

Lesson goals

Similarity is a modeling choice
Perform one complete assignment-update step
Evaluate stability, usefulness, and limitations

Similarity is a modeling choice

Clustering groups observations according to a chosen representation, distance, and algorithm. It does not discover a unique natural taxonomy from data alone. Changing feature scale, encoding, or included dimensions can change which records are near each other. Define the exploratory or operational purpose, such as summarizing usage patterns, before interpreting a cluster label. The numeric label zero or one has no inherent meaning and can be permuted without changing the partition. A cluster can reflect measurement artifacts or collection sites rather than the phenomenon of interest. Use domain review and independent evidence before assigning substantive names to the groups.

K-means has a specific geometry

K-means minimizes a within-cluster squared-distance objective under its formulation, alternating assignment and centroid updates. It is suited to that centroid geometry rather than every cluster shape or mixed data type. Initialization matters and the iterative solution can reach a local optimum. Outliers can strongly influence means, and a feature with a large numeric scale can dominate Euclidean distance. Fit any learned scaling under the intended evaluation boundary. The chosen number of clusters is a model decision, not a count supplied by the algorithm as universal truth. Empty-cluster handling and stopping behavior must be defined by the implementation.

Perform one complete assignment-update step

The first example assigns four one-dimensional observations to two supplied centroids, recomputes their means, and calculates the resulting within-cluster squared error. The two means become one and nine, with total inertia four. This is one transparent iteration on a fixed fixture, not a production convergence routine or proof that two clusters are correct. The helper asserts that each chosen group has members before computing its mean. A real implementation must handle an empty cluster under a documented policy and consider multiple initializations. Test cluster comparisons through partitions or aligned labels rather than demanding one arbitrary label numbering.

python Fixed synthetic offline example

import numpy as np
x = np.array([0.0, 2.0, 8.0, 10.0])
centers = np.array([0.0, 10.0])
labels = np.argmin((x[:, None] - centers[None, :]) ** 2, axis=1)
assert all(np.any(labels == k) for k in range(2))
updated = np.array([x[labels == k].mean() for k in range(2)])
inertia = np.sum((x - updated[labels]) ** 2)
print(labels.tolist())
print(updated.tolist())
print(f"fixture inertia={inertia:.1f}")

Expected output

[0, 0, 1, 1]
[1.0, 9.0]
fixture inertia=4.0

The fixed assignments yield centroids one and nine. Each of four residuals has magnitude one after the update.

Inspect scale effects in distance

The second example computes distances to a query before and after an explicitly declared rescaling. It shows how the nearest observation can change when units dominate one axis. The rescaling is a supplied pedagogical policy, not a statistic fitted from evaluation data. In a real pipeline, learned transformation parameters must come from the permitted fitting set. Do not standardize every feature blindly: binary indicators, counts, circular quantities, and domain distances can need different representations. Missing data and invalid categories also need a policy before distance calculations; replacing them arbitrarily can create artificial neighborhoods.

python Fixed synthetic offline example

import numpy as np
points = np.array([[0.0, 100.0], [2.0, 0.0]])
query = np.array([0.0, 0.0])
raw = np.linalg.norm(points - query, axis=1)
scale = np.array([1.0, 100.0])
scaled = np.linalg.norm((points - query) / scale, axis=1)
print(int(np.argmin(raw)))
print(int(np.argmin(scaled)))

Expected output

1
0

The nearest index changes from one to zero under the supplied scale. This illustrates representation dependence, not a preferred universal scaling.

Evaluate stability, usefulness, and limitations

Internal measures such as silhouette or inertia describe aspects of the chosen geometry and partition. They do not prove that clusters are meaningful, fair, or useful for a downstream action. Inertia generally falls as more centroids are available, so minimizing it alone can favor excessive complexity. Examine stability across perturbations and seeds, group sizes, outliers, and domain interpretation. If external labels are used for evaluation, keep them separate from unsupervised fitting unless the task explicitly changes. Deployment also needs a rule for assigning new observations or detecting that they do not fit existing groups. Record limitations instead of turning an exploratory chart into an unsupported categorical claim.

Common errors and fixes

Cluster ids are interpreted as an ordered risk score

Why it happens: Arbitrary labels were given unsupported meaning.
How to fix it: Interpret partitions with independent domain evidence.
Prevent it next time: Treat label numbering as permutation-invariant.

A large-unit feature determines all clusters

Why it happens: The distance representation ignored units.
How to fix it: Review and validate the feature-distance contract.
Prevent it next time: Inspect sensitivity to justified scaling choices.

More clusters are chosen solely because inertia falls

Why it happens: The fitting objective was used as a complete quality criterion.
How to fix it: Evaluate complexity, stability, and usefulness separately.
Prevent it next time: Document the selection purpose and evidence.

Check your understanding

1. Why can two clustering runs describe the same partition with different label numbers?

Show worked solution

Labels are identifiers assigned to groups, not fixed semantic values. Compare membership relationships or align labels before comparing arrays. A raw equality check can report disagreement when one run simply swaps zero and one without changing which observations are grouped together.

2. What evidence is needed before naming a cluster “high-value customers”?