Software Architecture / Foundations: promises, structure, and ownership / Lesson 2

Make quality goals measurable

Convert broad quality words into workload-scoped scenarios and reason about competing outcomes.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Write quality scenarios with stimulus, environment, response, and measure.
Separate a business target from a tested result and an operational commitment.

Replace adjectives with observable behavior

Fast, secure, and maintainable express intentions, but they do not tell a reviewer what to examine. For the workshop service, a performance scenario specifies a particular request, a workload, a response boundary, and a percentile over a defined interval. A changeability scenario instead describes a developer task, such as adding a new cancellation rule without editing notification delivery. Different scenarios demand different evidence. A system may respond quickly in a quiet lab yet be difficult to change or recover after a fault.

State the conditions alongside the number

The following JSON is an invented target rather than an observed benchmark. It defines a read-only listing operation with a fixed synthetic dataset and an explicit client-side measurement boundary. The percentile means that at least ninety-five percent of sampled requests should complete within the threshold under the stated conditions. It says nothing about a different dataset, a reservation write, or every possible network. Recording exclusions avoids turning a limited test into an unjustified promise about the entire service.

json A scoped latency target

{"operation":"list workshops","environment":"isolated lab","dataset_rows":1000,"concurrent_clients":5,"response":"complete response body","target_p95_ms":300,"status":"target, not measured"}

The example is valid JSON. Its numeric values are teaching assumptions; it neither runs requests nor claims this target suits a real product.

Define the denominator before discussing availability

A success ratio needs a definition of eligible attempts and successful outcomes. Counting only HTTP responses can conceal rejected valid reservations, while counting every malformed request as a service outage distorts a different way. Specify whether an availability objective is request-based or time-based, the observation window, and where failures are measured. Maintenance exclusions and missing telemetry require explicit treatment. A contractual service agreement is a separate commitment; architecture notes must not invent one by presenting a hypothetical objective as a promise to customers.

Expose conflicts instead of adding unrelated targets

Suppose administrators require immediate cancellation visibility, while a cached listing improves latency. A longer cache lifetime can help one measure and weaken the other. Document the freshness requirement and choose an invalidation or versioning approach that fits it. Likewise, extra replicas can improve some availability scenarios while introducing consistency and operating costs. Prioritize a few important quality scenarios and compare options against all of them. An unranked list of desirable properties cannot explain why a decision accepts one cost to protect another outcome.

yaml A correctness-oriented quality scenario

scenario: cancelled workshop stops accepting reservations
stimulus: administrator confirms cancellation
environment: normal operation
response: new reservation requests are rejected
measure: rejection begins before cancellation confirmation returns

The measure names an ordering guarantee instead of an arbitrary speed threshold. A stale read model can still display the workshop, but the authoritative reservation path must enforce the declared rule.

Design evidence that could disprove the claim

For each scenario, decide what would count as failure before measuring. A latency test needs representative data and a warm-up policy; a changeability review needs a concrete change and a record of touched boundaries. A recovery exercise needs an explicit fault and correctness checks after restoration. Preserve the test method, raw observations, and environment so another person can assess scope. When evidence is incomplete, record an open risk with an owner. Repeating a favorable average does not answer a scenario about tail latency or lost reservations.

Common errors and fixes

The requirement only says highly available.

Why it happens: No eligible operation, window, or measurement point was defined.
How to fix it: Write a failure scenario with a measurable response.
Prevent it next time: Review quality goals with both business and operational owners.

An average hides a slow minority of requests.

Why it happens: The chosen statistic does not match the user concern.
How to fix it: Inspect an appropriate percentile and distribution under the same workload.
Prevent it next time: Specify the statistic and sample boundary in advance.

A lab target is presented as a contractual guarantee.

Why it happens: Desired, measured, and promised values were merged.
How to fix it: Label target and evidence separately and involve the agreement owner.
Prevent it next time: Keep operational commitments in their authorized process.

Check your understanding

1. A team reports 100 ms average response time. Can it claim p95 below 300 ms?

Show worked solution

No. Many distributions share that average while having different tails. Obtain individual observations or a correctly configured histogram, define the percentile calculation and window, and compare the same operation and workload with the target. Report the sample size and conditions. An average is useful evidence for a different question, but it does not establish the stated percentile.

2. Rewrite maintainable for a new discount rule in the booking service.