Python / Maintainable systems / Lesson 17

Profiling, algorithmic cost, and memory

Measure representative work, distinguish time from retained memory, and improve the dominant cost without changing the behavioral contract.

About 5 min readContent reviewed 2026-10-23

On this page

Lesson goals

Begin with a question and baseline
Bound caches and validate gains

Begin with a question and baseline

Performance work begins with a measurable question: latency per request, throughput for a batch, peak memory, or retained memory after repeated operations. Define realistic input sizes and preserve a correctness baseline before changing implementation. A tiny synthetic example can demonstrate an API but cannot prove an application-wide speedup. Record interpreter, build, operating system, and relevant dependency versions when comparing results. Warm-up effects, disk caches, background load, and different data distributions can dominate small timing differences. Prefer a repeated measurement and a clearly stated workload over a single impressive number copied from a different machine. An optimization must improve the metric users actually need.

Separate timing and attribution

timeit helps compare isolated snippets with repeated execution; cProfile attributes calls and time to functions in a broader execution. Profiling adds overhead and can distort tiny functions, so use it to find likely dominant work, then measure the candidate change in context. Read cumulative time to locate expensive call paths and internal time to locate work performed in a function itself. A function called frequently may matter even if each call is cheap. Avoid optimizing the line that merely looks complicated before measuring. The first example uses cProfile programmatically but prints deterministic call facts rather than unstable timing values.

Algorithmic changes beat decoration

An algorithm that repeatedly scans a growing list can do quadratic work; a dictionary or set can often replace membership scans with expected constant-time lookups, subject to hashing behavior and memory cost. Complexity describes growth, not a guaranteed stopwatch result. For small input a simpler loop may remain preferable. Preserve ordering, duplicates, and error policy when changing representations; replacing a list with a set can make a function faster by accidentally discarding required information. The profiling example intentionally has a small obvious workload so a reader can independently verify the result and inspect that the intended function was actually called.

python Algorithmic changes beat decoration — worked example

import cProfile
import pstats

def sum_squares(values):
    return sum(value * value for value in values)

profiler = cProfile.Profile()
result = profiler.runcall(sum_squares, range(5))
stats = pstats.Stats(profiler)
called = any(key[2] == "sum_squares" for key in stats.stats)
print(result)
print(called)

Expected output

30
True

The independently checkable result is 0 + 1 + 4 + 9 + 16. The profile contains the target function; actual timing inspection is deliberately left environment-specific.

Trace allocation, not every kind of memory

tracemalloc records traced Python memory allocations after tracing starts. Snapshots can be compared to find lines associated with growth. It does not equal a complete process resident-memory measurement and cannot account for every native extension allocation in the same way. sys.getsizeof similarly reports a shallow object size rather than recursively counting everything reachable. The second example allocates a known retained byte buffer, checks broad allocation relationships, and prints Booleans instead of machine-dependent byte totals. A peak can be high even when retained memory later falls, so distinguish short-lived working space from references that remain alive after an operation completes.

python Trace allocation, not every kind of memory — worked example

import tracemalloc

tracemalloc.start()
payload = bytearray(100_000)
current, peak = tracemalloc.get_traced_memory()
print(current >= len(payload))
print(peak >= current)
tracemalloc.stop()

Expected output

True
True

The live buffer contributes at least its payload length to traced memory, and the recorded peak cannot be below the current measurement. No exact platform-specific overhead is claimed.

Bound caches and validate gains

Caching trades repeated computation for retained memory and invalidation complexity. Choose a bounded policy, define when data becomes stale, and consider whether cached mutable results expose shared state. A growing dictionary is not automatically a leak if growth is an intended bounded workload, but an unbounded cache in a long-running process deserves investigation. After optimizing, rerun contract tests and compare representative time and memory measurements. Change one dominant factor at a time so the cause of improvement remains clear. Document tradeoffs in ordinary terms: lower latency may require more memory, while smaller batches may reduce memory at the expense of per-item overhead.

Common errors and fixes

An optimization changes duplicate handling

Why it happens: A set replaced a list without preserving the original contract.
How to fix it: Choose a structure that preserves required multiplicities and order.
Prevent it next time: Run contract tests before accepting performance changes.

getsizeof appears much smaller than process memory

Why it happens: It measures a shallow object rather than the full object graph and runtime.
How to fix it: Use an appropriate allocation or process-memory tool for the question.
Prevent it next time: Name the measured memory metric explicitly.

A cache grows forever in a service

Why it happens: There is no eviction or input-key bound.
How to fix it: Set a bounded cache policy and monitor retained growth.
Prevent it next time: Design invalidation and capacity together with caching.

Check your understanding

1. A profiler shows a cheap helper called a million times. What should you investigate?

Show worked solution

Check whether repeated calls reflect necessary work or repeated parsing, lookup, or conversion that can move outside a loop. Measure aggregate cost, preserve semantics, and compare a representative workload after the change. A tiny per-call saving can matter at that frequency, but the profile must justify the priority.

2. How would you distinguish a temporary peak from retained growth?