Azure / Advanced: secure and recoverable architecture / Lesson 16

Capacity, latency, and performance evidence

Connect demand, bottlenecks, quotas, scaling delay, and cost accountability to a complete workload. Use synthetic queue arithmetic while keeping measured Azure performance explicitly unclaimed.

About 4 min readContent reviewed 2026-10-25

On this page

Lesson goals

Define a representative workload and success condition.
Distinguish average rates from burst and tail behavior.
Explain quotas, service capacity, and end-to-end bottlenecks.

A performance claim needs conditions and correctness

Define operation mix, payload size, data volume, arrival pattern, concurrency, geographic clients, and a successful business outcome. A service returning errors quickly is not a successful optimization. Separate client latency, network waiting, application work, database time, queue delay, and background completion. Resource metrics are clues to a bottleneck, not substitutes for user outcomes. A realistic Azure capacity review includes selected service limits, regional availability, deployment shape, dependency budgets, and failure-mode headroom. This chapter runs no load test and provides no Azure benchmark; its examples expose the assumptions a later authorized experiment must measure.

An average can conceal the demanding interval

The synthetic profile has average arrival five units per second and peak arrival twelve, while modeled processing is eight. The average suggests spare processing, but a burst can create backlog and violate latency even without exhausting the period’s total capacity. Real measurements need distributions and duration, not only one mean. Percentiles require adequate samples and a defined population; never average unrelated percentile labels into a new percentile. The JSON model below is not service telemetry or an autoscale rule. It records questions about burst duration, scaling delay, downstream limits, and failure behavior that must be resolved before accepting the design.

json A hypothetical workload profile

{
  "model": "synthetic-workload-profile",
  "average_arrival_units_per_second": 5,
  "peak_arrival_units_per_second": 12,
  "processing_units_per_second": 8,
  "unresolved": [
    "burst-duration",
    "scaling-delay",
    "downstream-budget",
    "degraded-capacity"
  ]
}

Expected output

average spare capacity does not prove acceptable burst latency

No throughput value comes from Azure. This is not a sizing recommendation or a service-limit statement.

Backlog drains only when service exceeds arrivals in the model

For a backlog of one hundred units, continuous arrival five, and processing eight per second, the net modeled drain is three per second. Rounding up yields thirty-four seconds under the simplified constant-rate assumption. If arrivals meet or exceed processing, this model cannot drain the backlog. It excludes scheduling, retry, batching, lock contention, partition imbalance, and external limits, so it is not a prediction of Service Bus or Functions recovery time. Use it to challenge a proposed design, then obtain representative authorized evidence for the actual processing path and user deadline.

python Offline constant-rate backlog calculation

from math import ceil
backlog = 100
arrival_per_second = 5
processing_per_second = 8
net = processing_per_second - arrival_per_second
assert net > 0
print(ceil(backlog / net))

Expected output

Thirty-four seconds belongs to this arithmetic model only. No Azure queue, worker, scale policy, or network is exercised.

More compute can move the bottleneck rather than remove it

Scaling applications can multiply database connections, storage transactions, identity requests, telemetry, and outbound traffic. A quota is an administrative limit and is not the same as physical capacity availability or workload throughput. A larger resource can remain slow because of query shape, hot partitions, serialization, or a dependency rate limit. Measure achieved workload and errors together, and review cache correctness before counting hit-rate improvements. Capacity reservations and commercial commitments, where considered, require current scoped assessment rather than generic cost-saving promises. Prefer a small hypothesis-driven experiment that can distinguish causes over changing several resource sizes at once.

Accept performance with a documented tradeoff

Compare equivalent builds and workloads, preserve correctness assertions, and report variability, failures, and measurement endpoints. Include the cost and operational effects of extra capacity without inventing a currency estimate. Verify degraded behavior, scale-in safety, and recovery after a burst as well as the steady state. This course performs no Azure profiling, capacity request, quota change, resource resize, or load generation. Its evidence consists of local models only. A future authorized acceptance test should tie observed latency and completion to user objectives, dependency constraints, security, and a repeatable recovery path.

Common errors and fixes

Average throughput is called a tail-latency guarantee

Why it happens: Burst shape and response distribution are ignored.
How to fix it: Measure representative bursts and the defined latency population.
Prevent it next time: Record duration, errors, and sample limitations beside percentiles.

Adding app instances overwhelms a downstream service

Why it happens: Dependency budgets do not scale with local compute.
How to fix it: Bound concurrency and review aggregate connection/transaction demand.
Prevent it next time: Include downstream behavior in capacity experiments.

A quota increase is assumed to guarantee usable capacity

Why it happens: Administrative allowance and service capacity are conflated.
How to fix it: Verify the selected resource availability and workload behavior separately.
Prevent it next time: Keep limits, availability, and measured throughput distinct.

Check your understanding

1. Why does the model drain one hundred units in thirty-four seconds rather than thirteen?

Show worked solution

New work continues to arrive at five units per second, so only three of the eight processed units reduce existing backlog. The rounded-up constant-rate calculation is ceil(100/3). It excludes real service effects and must not be presented as a measured Azure drain time.

2. What should accompany a claim that a larger compute option improved an API?