DevOps / Advanced: reliability, trust, and governance / Lesson 14

SLOs and error-budget decisions

Define user-centered reliability indicators and use their budgets to guide release and operational decisions.

About 5 min readContent reviewed 2026-10-25

On this page

Lesson goals

Define eligible and good user events for an SLI.
Compute a request-based budget with explicit units.
Use burn rate and accountable budget policy.

Measure a useful user event

A service-level indicator, SLI, measures a defined aspect of service behavior; an SLO sets a target over a window. Define eligible events, good events, measurement location, and exclusions before choosing a percentage. Successful HTTP transport may still be a bad booking if the operation lost data or exceeded useful latency. The first record describes a request-based availability SLO for a synthetic service. Its target is illustrative rather than an industry requirement. Include important low-volume operations and explain where the measurement can miss client-visible failures. A target without a trustworthy denominator is not a reliable agreement.

yaml Synthetic request-based SLO

journey: create-valid-booking
window: rolling-30-days
eligible: authenticated-valid-create-attempts
good: correct-durable-result-within-800ms
target_percent: 99.9
measurement: approved-user-path-instrumentation

The record defines a good outcome and measurement scope. The latency and percentage are example policy choices and do not claim measured production performance.

Translate the target into a budget

The error budget is the allowed amount of bad service under the chosen SLO definition. For a request-based 99.9% target over one million eligible requests, one thousand may be bad before that target is missed. This is not automatically 43.2 minutes of downtime: that time calculation applies to a time-based 30-day interpretation with its own assumptions. The second simulation uses integer arithmetic for the request budget. It counts 250 bad requests as 25% of the allowed thousand, leaving 750 under the simplified fixed-denominator example. Real rolling windows change as events enter and leave.

python Harmless fixed-window budget arithmetic

eligible = 1_000_000
allowed_bad = eligible * (1000 - 999) // 1000
observed_bad = 250
print(allowed_bad)
print(allowed_bad - observed_bad)
print(f"{100 * observed_bad / allowed_bad:.1f}%")

Expected output

1000
750
25.0%

The fixed sample permits 1,000 bad events, has 750 remaining, and has used 25.0%. It is a simplified request budget, not a live alert or time-based downtime calculation.

Use burn rate with context

Burn rate compares observed bad-event fraction with the fraction allowed by the objective. A rate of five means budget is being consumed five times as fast as the SLO allowance under comparable measurement. Short windows detect severe incidents quickly but can be noisy; longer windows confirm sustained impact. Combine windows and minimum evidence according to the service rather than paging on a single failure. Separate missing telemetry from actual good service. An unavailable measurement pipeline must not silently turn every unobserved request into a success or remove the denominator without explanation.

Write an actionable budget policy

An error-budget policy explains what the team does when reliability degrades: slow risky releases, focus on remediation, or require additional review, with explicit emergency exceptions. It is a shared prioritization agreement rather than a punishment for an operator. Do not automatically freeze every change because some changes repair the incident. Name decision owners and keep the policy connected to user impact and business risk. Review repeated budget consumption for systemic causes such as fragile dependencies or release design, and reserve capacity for improvement instead of treating all reliability work as an interruption.

Review the objective as the service changes

Choose a target that reflects user needs and feasible investment, not a borrowed number or an impossible promise of perfect reliability. A service-level agreement may include contractual consequences and is distinct from an internal SLO. Revisit objectives when user journeys, architecture, traffic mix, or measurement changes materially. Keep historical definitions so apparent improvement is not merely a changed exclusion rule. Use SLOs alongside security, data integrity, and accessibility obligations that may not fit a simple availability ratio. An unused percentage is paperwork; the value comes from better decisions about risk and work.

Common errors and fixes

The denominator excludes real failures

Why it happens: Only successful server logs were counted as eligible traffic.
How to fix it: Define and instrument eligible attempts at the appropriate boundary.
Prevent it next time: Review missing data and exclusion rules with users.

An SLO freeze blocks an urgent repair

Why it happens: The policy had no risk-based emergency exception.
How to fix it: Use a documented accountable exception for remediation.
Prevent it next time: Write decision rights and review outcomes after incidents.

Request availability is converted directly to downtime

Why it happens: Request and time denominators were mixed.
How to fix it: Use the actual SLI definition and window.
Prevent it next time: Include units and denominator in every budget report.

Check your understanding

1. At 99.9% over two million requests, 600 are bad. Compute the fixed-sample budget use.

Show worked solution

The allowance is 2,000 bad requests. Six hundred consumes 30%, leaving 1,400 in the fixed example. State the window and eligibility rules alongside the number; a rolling window may change the denominator and remaining budget at the next observation.

2. A dashboard improves after excluding a failing client version. What must be reviewed?