API Testing / Basic: trustworthy API observations / Lesson 1

Turn an API promise into testable evidence

Build explicit oracles for a fictional library API and state what an offline test actually establishes.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Separate a business requirement, a protocol contract, and a test oracle.
Design observable positive and negative cases without contacting an API.

Start with an observable promise

A test needs a reason to judge an outcome. For a fictional reading-list API, the promise might be that adding an available book creates exactly one entry owned by the authenticated reader. That promise contains several independently observable properties: the response identifies an entry, ownership comes from trusted context, the chosen book is preserved, and the stored count increases once. Writing only “returns success” discards most of the requirement. Ask the product owner which outcomes matter, including what must remain unchanged after refusal. Keep invented classroom identifiers separate from real accounts and data.

Keep the oracle independent

An oracle is the rule used to decide whether the observed result is acceptable. If the expected owner is calculated by calling the same faulty owner-selection function as the implementation, agreement provides weak evidence. Derive expected values from a small reviewed fixture instead. The first example deliberately compares both response and resulting state, because a correct response can accompany an incorrect mutation. It models a boundary in memory; it does not demonstrate routing, authentication middleware, serialization, persistence, or an HTTP connection. Those need different evidence at their own boundaries.

python Check the response and mutation together

before = []
response = {"status": 201, "body": {"id": "entry-7", "book": "book-4", "owner": "reader-a"}}
after = [{"id": "entry-7", "book": "book-4", "owner": "reader-a"}]
assert response["status"] == 201
assert response["body"] == after[0]
assert len(after) == len(before) + 1
assert after[0]["owner"] == "reader-a"
print("creation evidence accepted")

Expected output

creation evidence accepted

This complete offline program checks a synthetic observation against explicit expected values. Its message means these assertions passed. It supplies no evidence that a deployed service creates or protects entries.

Choose cases by differences that matter

Create a small matrix before multiplying examples. Include an available book, an unknown book, a duplicate addition, and an unauthenticated request. Give each case a distinct purpose and an expected state effect. The matrix is a planning artifact, not an executable test suite. Avoid copying implementation branches directly into case names; that can hide missing business behavior. A useful negative case confirms both the documented refusal and absence of an entry, event, or notification. Record uncertain requirements as questions rather than silently choosing whatever the current service does.

json A reviewed fictional decision table

{"operation":"add-book","cases":[{"case":"available","status":201,"entryDelta":1},{"case":"unknown-book","status":404,"entryDelta":0},{"case":"duplicate","status":409,"entryDelta":0}]}

The status choices are this fictional contract, not mandatory choices for every API. The table is non-runnable data. An approved real contract must define its own identity, duplicate, and disclosure behavior.

Match the test boundary to the question

Use a pure function test for a business decision, a component test for serialization and application wiring, and an authorized integration test for actual protocol and storage behavior. A large end-to-end test can reveal a user-visible failure while giving little information about its cause. Smaller checks help locate the broken boundary. Conversely, thousands of small mocks cannot prove that the assembled system works. Describe each test by the dependencies it includes and replaces. Tool labels differ across teams, so a clear boundary description is more reliable than the word integration alone.

Report evidence with its limits

A useful result includes the requirement identifier, fixture version, observation, comparison rule, and environment boundary. “All green” omits whether the intended cases even ran. If an example is illustrative, say so. If a local script passes, state the exact property it checked without expanding that claim into server reliability or security. Preserve failure details that explain the mismatch while excluding private payloads. This discipline makes later changes reviewable: another reader can determine which promise was exercised and which deployment assumptions still require verification.

Common errors and fixes

A success status hides a wrong owner

Why it happens: The oracle checks transport outcome only.
How to fix it: Compare the trusted fixture owner and resulting entry.
Prevent it next time: Require response and state assertions for ownership-sensitive operations.

Expected and actual values share a faulty helper

Why it happens: The test duplicates the implementation decision.
How to fix it: Write independently reviewed expected values for a minimal case.
Prevent it next time: Review the source of every important expected value.

A local pass is reported as a deployed guarantee

Why it happens: The evidence boundary was not documented.
How to fix it: State which dependencies were simulated and which were absent.
Prevent it next time: Attach environment and fixture scope to the result.

Check your understanding

1. A response returns 201 and entry-7, but two entries appear. Which assertion is missing?

Show worked solution

The test must compare the state delta, not just the response identifier. With an empty fixture, require one resulting entry and its exact fields. In an event-producing implementation, also check that the contractually relevant side effect occurs once. A response alone cannot establish that property.

2. Design evidence for an unknown book without deciding real production policy.