Python / Reliable programs / Lesson 9

Testing contracts with the standard library

Build small deterministic tests around observable behavior, including boundaries and expected failures, instead of testing implementation accidents.

About 4 min readContent reviewed 2026-10-23

On this page

Lesson goals

A test makes a claim
Control dependencies and interpretation

A test makes a claim

A test supplies controlled inputs, performs an operation, and checks an observable result against a justified expectation. The expected result should come from the requirement or an independent argument, not from copying the same implementation into the test. For a minimum-of-two function, equal values and negative numbers are meaningful cases; random examples alone may miss them. Tests document confidence within their chosen scope and do not prove that every possible execution is correct. Start by stating the behavior in one sentence, then choose the smallest case that would fail if that behavior were broken.

Partition the input space

Partition inputs into groups expected to behave similarly, and test boundaries between groups. A quantity parser has valid positive input, zero, negative input, malformed text, and perhaps whitespace depending on its contract. For a collection operation, include empty, single-item, duplicate, and ordinary multi-item cases. Use an expected exception as part of the behavior rather than treating any failure as success. Keep each test focused enough that its name and failed assertion identify the broken promise. A large scenario with many unrelated assertions often hides which setup assumption first stopped being valid.

Use unittest deliberately

The unittest module provides TestCase assertions and a runner. Tests are methods whose names start with test, and assertions express what result is required. setUp can create fresh fixtures per test; temporary directories prevent writing to real project data. The first example defines a tiny parser and tests a normal value plus a rejection case, then prints only stable summary information. assertRaises verifies the exception type, and its regex variant can also check meaningful message text. Avoid asserting an entire traceback because filenames and line numbers are incidental and change as the code evolves.

python Use unittest deliberately — worked example

import io
import unittest

def positive(text):
    value = int(text)
    if value <= 0:
        raise ValueError("must be positive")
    return value

class ParserTests(unittest.TestCase):
    def test_valid(self):
        self.assertEqual(positive("4"), 4)
    def test_zero(self):
        with self.assertRaisesRegex(ValueError, "positive"):
            positive("0")

result = unittest.TextTestRunner(stream=io.StringIO()).run(
    unittest.defaultTestLoader.loadTestsFromTestCase(ParserTests)
)
print(result.testsRun, result.wasSuccessful())

Expected output

2 True

Two independent behaviors are tested. Capturing the runner stream avoids time-dependent runner output while retaining the actual test result.

Check relationships as well as answers

Sometimes a property connects inputs and outputs more generally than one expected number. Sorting should preserve all items and produce a nondecreasing order; splitting minutes should reconstruct the original amount. The second example checks reconstruction and the remainder’s range across several boundary values using subTest. This is a finite table of meaningful cases, not a claim of exhaustive proof. A property can still be weak: checking only that an output is sorted would accept an empty output that lost all data. Combine independent properties with a few explicit expected results to avoid testing only half of the contract.

python Check relationships as well as answers — worked example

import io
import unittest

class SplitTests(unittest.TestCase):
    def test_reconstruction(self):
        for total in [0, 59, 60, 137]:
            with self.subTest(total=total):
                hours, minutes = divmod(total, 60)
                self.assertEqual(hours * 60 + minutes, total)
                self.assertTrue(0 <= minutes < 60)

result = unittest.TextTestRunner(stream=io.StringIO()).run(
    unittest.defaultTestLoader.loadTestsFromTestCase(SplitTests)
)
print(result.wasSuccessful())

Expected output

True

Every table entry must satisfy both reconstruction and remainder range. subTest associates a failed assertion with the input that triggered it.

Control dependencies and interpretation

A deterministic unit test should not depend on live network services, wall-clock timing, arbitrary file order, or state left by a previous test. Pass a clock or data source as a dependency when the behavior needs one. unittest.mock can substitute a collaborator at its lookup location, but excessive mocking may test a fictional interaction instead of real behavior. Keep some integration tests for important boundaries such as actual JSON serialization in a temporary directory. When a bug is fixed, first confirm that the regression case fails on the faulty behavior and passes on the repair. Coverage shows executed paths, not the quality of assertions.

Common errors and fixes

A test passes after the implementation becomes wrong

Why it happens: The expected result repeats the same flawed algorithm.
How to fix it: Derive expected answers independently from a small known example.
Prevent it next time: Ask which plausible defect would make each assertion fail.

Tests fail only when run together

Why it happens: They share mutable fixtures or leaked environment state.
How to fix it: Create fresh fixtures and clean resources within each test.
Prevent it next time: Run affected tests alone and together when investigating isolation.

A timed assertion fails on a slower machine

Why it happens: The test confuses functional behavior with an unstable timing budget.
How to fix it: Test results deterministically and measure performance separately.
Prevent it next time: Avoid fixed elapsed-time thresholds in ordinary unit tests.

Check your understanding

1. Choose four tests for a function that accepts integer scores from zero through one hundred.

Show worked solution

Test zero and one hundred as accepted boundaries, plus minus one and one hundred one as rejected neighbors. Add a representative internal value and a noninteger type if the contract includes type validation. The boundaries directly distinguish inclusive comparisons from off-by-one mistakes.

2. A sorting test checks only output == sorted(output). What important defect can escape?