API Testing / Basic: trustworthy API observations / Lesson 6

Verify error contracts and unchanged state

Test useful, stable failure information without leaking internal details or treating every error as equivalent.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Differentiate malformed input, domain refusal, permission denial, and dependency failure.
Check problem-detail structure together with the protected state and disclosure policy.

Classify the failed promise

Errors are part of an API contract, not incidental text produced when the happy path stops. A malformed representation, a valid but unavailable book, an unauthenticated caller, and an unavailable dependency have different meanings. Choose stable categories that clients can handle without parsing a human sentence. A service may intentionally conceal resource existence from an unauthorized caller, so a test must follow the documented disclosure policy rather than demand maximal detail. The classroom contract uses synthetic cases only and never probes a real identity or resource.

Use machine-readable meaning deliberately

Problem Details provides a standard representation for API errors; adopting it does not decide the entire application policy. The type identifies the problem category, while human-facing fields should not become brittle parsing targets. An HTTP status in the body must not contradict the actual response status. Extensions can carry documented validation details, but they must avoid private values and internal stack traces. The first program checks one fictional problem observation. The reserved example URL is an identifier and is never retrieved; no problem registry or web service is contacted.

python Check a fictional problem-detail observation

actual_status = 422
problem = {"type": "https://catalog.example.invalid/problems/invalid-page", "title": "Invalid page size", "status": 422, "field": "pageSize"}
assert problem["status"] == actual_status
assert problem["type"].endswith("/invalid-page")
assert problem["field"] == "pageSize"
assert "trace" not in problem
print("problem observation accepted")

Expected output

problem observation accepted

The complete program checks this selected application shape and status consistency. It is not a complete RFC validator. It does not prove that an actual HTTP response uses application/problem+json or excludes every sensitive detail.

Treat refusal as a state transition

A rejected request should leave the protected business state unchanged unless a documented ancillary effect is expected. Logging a denial may be legitimate; creating a reading-list entry before validation and then returning an error is not. Capture a minimal before/after observation and include related outbox or notification state when it belongs to the contract. The second program uses immutable snapshots of local dictionaries to demonstrate the check. It cannot prove transaction rollback, asynchronous side-effect cancellation, or real database isolation. Those are separate integration questions with their own approved fixtures.

python Check the rejection and protected state together

from copy import deepcopy
before = {"entries": [{"id": "entry-1", "book": "book-2"}], "notifications": []}
after = deepcopy(before)
response = {"status": 409, "code": "BOOK_ALREADY_LISTED"}
assert response["status"] == 409
assert response["code"] == "BOOK_ALREADY_LISTED"
assert after == before
print("refusal preserves protected state")

Expected output

refusal preserves protected state

This complete local observation has no mutation and no external notification. It verifies the stated fixture comparison only. The 409 choice belongs to the fictional duplicate-addition contract and is not prescribed for all duplicate operations.

Separate stable categories from readable text

A translated title or improved explanation should not break a client that uses the documented machine category. Test exact error identifiers and field paths where stability is promised, and test human text for appropriate presence and safe content rather than arbitrary punctuation. Conversely, do not weaken an error-code assertion to accept any string. Validate that the client can distinguish retryable dependency refusal from a permanent input correction case. A test plan should include malformed, boundary, and cross-field cases without inventing an error precedence rule that the contract never promised.

Preserve useful failure evidence safely

An error report should show the case, expected category, observed category, and relevant structural difference. Replace real request bodies with synthetic reproductions and avoid logging authorization headers, cookies, or private record contents. Internal diagnostics can correlate a request using a nonsecret identifier, but public problem details should not expose stack frames or infrastructure names. If a failure is intermittent, investigate the dependency and timing boundary rather than broadening the accepted errors. A deterministic local reproduction of the category mapping is useful, yet it remains distinct from observing a real deployed failure.

Common errors and fixes

A translated title breaks a client test

Why it happens: The client parses human text instead of the stable category.
How to fix it: Compare the documented type or code and validate readable text separately.
Prevent it next time: Include locale changes in error-contract review.

A refused request still creates an entry

Why it happens: Validation happens after protected mutation or rollback is incomplete.
How to fix it: Correct the transaction boundary and verify all relevant side effects.
Prevent it next time: Require negative cases to compare before/after state.

Body status differs from HTTP status

Why it happens: Error mapping uses inconsistent sources.
How to fix it: Derive both from one reviewed mapping and test the actual boundary.
Prevent it next time: Cover each documented category and public disclosure rule.

Check your understanding

1. A duplicate-addition case returns the correct code but sends a notification. Does it pass?

Show worked solution

Not under this contract. The protected entry and notification state must remain unchanged. Add the notification observation to the negative case and investigate whether the side effect occurs before refusal. A public error response cannot compensate for an unintended business action.

2. Two invalid fields are supplied together. Must their error order be asserted?