SRE / Basic: promises and measurements / Lesson 3
SLO windows and latency distributions
Choose a measurable target and time window, and explain why latency objectives need event thresholds rather than averages alone.
On this page
Lesson goals
- Separate SLI, SLO, and SLA.
- Compare rolling and calendar windows.
- Interpret latency thresholds and percentiles.
State the target in one sentence
An SLO combines a defined indicator, a target, and an evaluation window. A fictional availability objective says that at least 99.9% of eligible reservation attempts succeed over a rolling twenty-eight-day window. That statement does not define success by itself; it depends on the event contract from the previous lesson. An SLA is an agreement that may include consequences when terms are missed, so legal or commercial commitments need separate review. Keep the internal engineering objective explicit rather than assuming that a supplier agreement automatically represents the complete user journey.
Understand what enters and leaves the window
A rolling window drops old observations continuously while admitting new ones. A calendar window resets at a known boundary and can simplify some reporting processes. Neither is universally correct. A monthly report can conceal a bad day inside otherwise healthy traffic, while a rolling ratio can improve when an old incident ages out without any repair. Document timezone, event timestamp policy, late-arrival handling, and evaluation delay. The first record fixes these choices for a synthetic design review; its values illustrate precision rather than a recommended monitoring-product format.
objective: reservation-availability
target_good_fraction: 0.999
window: rolling-28-days
time_basis: event-completion-utc
late_data_policy: revise-with-auditable-correction
report_status: provisional-until-collection-delay-passes
A report is provisional while expected observations may still arrive. The record cannot configure a monitoring platform.
Count timely outcomes instead of trusting a mean
An average combines fast and slow requests into one number and can hide a painful tail. For a latency SLO, classify eligible attempts against a threshold such as completing within three hundred milliseconds. Define whether errors count as bad even if they return quickly; otherwise fast failures may improve a latency graph. In the offline example four requests meet the threshold and one does not, so timely success is eighty percent. The synthetic mean happens to be two hundred eighty milliseconds, which is below the threshold despite one very slow outcome.
latency_ms = [100, 100, 100, 100, 1000]
good = sum(value <= 300 for value in latency_ms)
print(sum(latency_ms) / len(latency_ms))
print(f"{100 * good / len(latency_ms):.1f}%")
280.0
80.0%The mean is below 300 ms but only 80% of requests meet that deadline. This list is not a load test or a measured service distribution.
Treat percentile estimates carefully
A percentile answers a distribution question, but implementations and interpolation conventions differ. Histogram-based estimates also depend on bucket boundaries. Do not average instance p99 values and call the result a service p99: different instances can have different traffic and distributions. Aggregate compatible underlying observations or histogram counts before estimating the population percentile. A threshold-based SLI can often be computed more directly when the histogram has the required boundary. Preserve units, scope, and sample count alongside the result so a sparse interval is not mistaken for strong statistical evidence.
Review the target with people affected by it
Start with observed behavior, user tolerance, and available engineering capacity. A target just below current performance may institutionalize poor experience; an unattainable target may generate permanent emergency work. Separate important journeys when their needs differ, while avoiding a catalogue so large that nobody can explain it. Record the reason for each target and revisit it after meaningful product changes. When changing an objective, retain historical definitions and explain the transition rather than rewriting past reports. An SLO is useful when it guides a decision about reliability investment and release risk.
Common errors and fixes
The monthly ratio rises without a repair
- Why it happens
- An old incident left the rolling window.
- How to fix it
- Inspect window membership and recent burn separately.
- Prevent it next time
- Annotate incident timing and window semantics.
Fast errors improve the latency objective
- Why it happens
- The numerator counted quick failures as good.
- How to fix it
- Apply success and deadline requirements together.
- Prevent it next time
- Test timeout and fast-error classifications.
The mean of instance p99 values is reported as p99
- Why it happens
- Percentiles were treated as additive statistics.
- How to fix it
- Aggregate compatible distributions before estimation.
- Prevent it next time
- Document histogram boundaries and aggregation rules.
Check your understanding
1. Why can a 28-day SLO pass while users report an outage this morning?
Show worked solution
The long-window total can remain above target despite a concentrated recent failure. Inspect short-window impact and budget consumption before concluding that the complaint is inconsistent. The long window supports planning; a separate actionable alert should detect a current incident.
2. Two instances report p99 values of 100 ms and 900 ms. Can you report 500 ms?
Show worked solution
No. You need their underlying compatible distributions and traffic counts; neither equal weighting nor traffic weighting of percentiles reconstructs the combined percentile. Obtain mergeable histogram data or raw representative observations, then state any estimation limitations.
Key takeaways
- An objective includes its window and event meaning.
- Averages cannot establish tail-latency compliance.
- Preserve distribution and historical definition boundaries.