API Testing / Intermediate: interacting contracts / Lesson 7

Test schemas and directional compatibility

Use explicit schema assumptions and consumer expectations to assess a contract change.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Explain why request and response compatibility have different directions.
Distinguish schema acceptance from semantic and client compatibility.

Use a schema for the claim it can support

A schema describes allowed structures and constraints under a particular dialect. It can reject a missing identifier or an incorrect value type, but it cannot by itself prove ownership, correct pricing, or a successful state transition. An OpenAPI document also describes operations, parameters, responses, and security requirements; merely validating one body leaves those other dimensions unchecked. Version the document with the implementation under review. Treat examples as useful illustrations rather than the complete accepted domain, because a single example cannot specify every optional combination or boundary.

State the dialect and unknown-field policy

Keyword interpretation depends on the chosen schema dialect and validator configuration. In a request object, required controls presence while a type including null controls which present values are accepted. These are different decisions. Unknown fields may be accepted, rejected, or ignored under an explicit policy; clients must not infer the choice from one sample. The JSON example declares a small draft 2020-12 request shape. It remains non-runnable because no schema engine is installed or exercised. Reading a schema is not equivalent to executing its full vocabulary.

json A pinned request schema with explicit presence rules

{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","required":["bookId"],"properties":{"bookId":{"type":"string","minLength":1},"note":{"type":["string","null"]}},"additionalProperties":false}

The schema requires bookId, permits an omitted or explicitly null note, and rejects undeclared properties. It has not been run through a schema validator. A real tool must support the declared dialect and configured vocabularies.

Compare compatibility in the correct direction

For requests, a new server should continue accepting the old valid requests when backward compatibility is promised. Requiring a previously optional field narrows acceptance and can break existing callers. For responses, the new server must keep producing values that supported old consumers can interpret. An added enum value can break a consumer with an exhaustive switch, even when adding an ordinary optional property would be harmless to that consumer. The program demonstrates one finite request-domain comparison. It is not a general schema-diff algorithm and cannot evaluate all possible payloads.

python Check a finite acceptance-set model

old_accepted = {"bookId-only", "bookId-and-note"}
new_requiring_note = {"bookId-and-note"}
new_optional_note = {"bookId-only", "bookId-and-note", "bookId-and-null-note"}
assert not old_accepted <= new_requiring_note
assert old_accepted <= new_optional_note
print("request compatibility direction checked")

Expected output

request compatibility direction checked

The complete local program shows that removing a formerly valid request breaks the declared compatibility promise. The named cases represent reviewed fixtures. No claim is made about arbitrary schemas, client generators, or a running server.

Look for changes that preserve shape

A change from seconds to milliseconds can keep the same integer schema and still break every timeout calculation. Similar silent changes include currency units, sort order, timezone interpretation, and the meaning of an empty collection. Add semantic examples that assert units and relationships, not only types. Consumer evidence should identify the operations and fields actually used by each supported consumer. Avoid universal claims that any additive change is safe; acceptance of unknown properties, exhaustive enums, generated clients, and validation strictness vary across consumers.

Build a reviewable change decision

Compare the old and proposed contract, list affected supported consumers, and run targeted examples at the relevant boundaries. Record whether a result came from document inspection, a local model, a schema tool, or an actual approved component test. A breaking change may require a new version, migration period, or consumer update; a schema diff alone does not choose the product policy. Keep deprecated behavior tests until the agreed support window closes, and remove them through a reviewed decision rather than letting a failing test silently redefine compatibility.

Common errors and fixes

An optional field becomes required without a migration

Why it happens: The review considers only the new happy path.
How to fix it: Test old valid payloads against the proposed acceptance policy.
Prevent it next time: Track compatibility direction and supported consumer versions.

A unit change passes all type assertions

Why it happens: The schema shape stays unchanged while meaning changes.
How to fix it: Assert documented units and cross-field relationships.
Prevent it next time: Review semantic changes alongside document differences.

A schema is called validated after visual inspection

Why it happens: No compatible engine executed the declared vocabulary.
How to fix it: Label inspection accurately and run the selected validator separately when authorized.
Prevent it next time: Record tool, dialect, configuration, and evidence boundary.

Check your understanding

1. An API adds status=archived to responses. Why might an old client fail?

Show worked solution

An old consumer may accept only active and paused or use an exhaustive switch. Test the actual supported consumer contract and its fallback behavior. The JSON type is still string, so type-only validation cannot establish compatibility. Decide whether to coordinate consumer support or version the change.

2. Does required:[note] permit a request with no note when note allows null?