ASP.NET Core / Advanced: Verification and Reliable Operations / Lesson 15

Performance, Caching, and Request Limits

Optimize a verified workload while preserving authorization, tenant isolation, freshness, and bounded resource use.

About 5 min readContent reviewed 2026-10-26

On this page

Lesson goals

Choose measurements before tuning implementation details.
Define cache identity, expiry, invalidation, and privacy.
Distinguish rate/concurrency limits from unbounded waiting.

A performance claim needs an equivalent workload

Begin with the resource question: latency distribution, throughput, CPU, allocation, retained memory, or dependency pressure. Record build/runtime versions, warmup, input distribution, and concurrency. A faster endpoint that skips authorization, returns stale data outside contract, or loads fewer required records is not an equivalent improvement. Use profiling before guessing which source line is expensive.

Avoid synchronous blocking and unbounded buffering on request paths, but do not add pooling or parallelism indiscriminately. A lower allocation count can increase retention and lifetime complexity. Keep correctness, cancellation, and ownership tests beside the benchmark. No load or production performance experiment is run by this course.

A cache key is part of the security and data contract

Choose cache scope and key dimensions from what makes a response reusable: tenant, principal/permissions where relevant, resource/version, locale, query, and representation. A key omitting tenant can leak one tenant’s result to another. A read-only interface over mutable cached data does not prevent another alias from changing it. Define immutable values or copying/ownership explicitly.

The first program uses a real local MemoryCache with two deliberately different tenant keys. It checks only that those keys do not collide in this fixture. It is not an output-cache middleware or tenant-authorization test. MemoryCache size units are application-defined, and eviction/expiration are not a universal process-memory ceiling.

csharp Check isolated cache-key separation

using System;
using Microsoft.Extensions.Caching.Memory;

using var cache = new MemoryCache(new MemoryCacheOptions { SizeLimit = 2 });
cache.Set("tenant-a:report:7", "synthetic-report", new MemoryCacheEntryOptions().SetSize(1));
bool same = cache.TryGetValue("tenant-a:report:7", out string? _);
bool other = cache.TryGetValue("tenant-b:report:7", out string? _);
Console.WriteLine($"sameTenant={same} otherTenant={other}");

Expected output

sameTenant=True otherTenant=False

The local keys differ; no HTTP output cache or production tenant boundary is validated.

Response and output caching have distinct contracts

HTTP cache headers describe client/intermediary behavior, while ASP.NET Core output caching applies configured server policies. Adding services/middleware alone does not mean every response is cached. Default protections and custom policy choices must be reviewed for authentication, cookies, status, vary dimensions, and invalidation. Do not override conservative behavior and assume private responses remain safe automatically.

Plan freshness, cache stampede handling, dependency changes, and invalidation ownership. A fast stale result can be incorrect even when its status is 200. Test user/tenant changes, query variants, invalidation, and unauthorized responses through the actual configured middleware path before making an HTTP caching claim.

A concurrency limiter bounds simultaneous permits

The second program acquires two permits from a local ConcurrencyLimiter and verifies that a third immediate acquisition is rejected. This tests the limiter component with zero queue capacity, not HTTP partitioning or distributed rate enforcement. Concurrency limiting differs from a request-per-time-window rate. Both need a deliberate rejection/status policy.

Choose partition keys from trusted bounded identity or route dimensions rather than unbounded arbitrary client strings. Process-local limits can multiply across replicas, so a fleet-wide guarantee needs a suitable design. Queueing, deadlines, cancellation, and downstream capacity must be reviewed together.

csharp Exercise a local concurrency limiter

using System;
using System.Threading.RateLimiting;

using var limiter = new ConcurrencyLimiter(new ConcurrencyLimiterOptions { PermitLimit = 2, QueueLimit = 0 });
using var first = limiter.AttemptAcquire();
using var second = limiter.AttemptAcquire();
using var third = limiter.AttemptAcquire();
Console.WriteLine($"first={first.IsAcquired} second={second.IsAcquired} third={third.IsAcquired}");

Expected output

first=True second=True third=False

Two permits are acquired and the third is rejected without queueing or any request traffic.

Preserve bounded semantics across the full path

Review request/body size, pagination, response buffering/streaming, compression, serialization, query projection, connection pools, and retries under the actual workload. A small local timing is not evidence of tail latency or overload safety. Compression and caches can interact with secrets and representation; validate privacy and correctness rather than chasing one smaller number.

Retain representative correctness cases, measurement conditions, and the reason a tradeoff was accepted. Recheck when runtime, workload, or dependency behavior changes materially. All examples here are bounded component checks; no benchmark framework, load generator, configured cache backend, or application server is executed.

Common errors and fixes

A cached result crosses tenants.

Why it happens: The key or policy omits an authorization-relevant dimension.
How to fix it: Redesign cache identity and validate the protected path.
Prevent it next time: Test different users/tenants and permission changes.

An in-process limit is reported as a fleet-wide cap.

Why it happens: Replica multiplication was omitted.
How to fix it: Define the enforcement scope and a suitable global design if required.
Prevent it next time: Record partition and deployment assumptions.

A faster implementation changes freshness or validation.

Why it happens: Performance comparison does not preserve the contract.
How to fix it: Restore equivalent behavior before measuring.
Prevent it next time: Keep correctness/privacy checks alongside performance evidence.

Check your understanding

1. Why does a cache hit for tenant-a and a miss for tenant-b not prove that a public endpoint is tenant-safe?

Show worked solution

The test controls the keys directly and does not include trusted tenant derivation, authorization, output-cache policy, serialization, invalidation, or concurrent requests. It is useful evidence about this local fixture only. Test the actual protected endpoint with distinct validated identities and resource ownership, and verify no private data is reused outside its permitted scope.

2. A limiter queues every excess request indefinitely. What resource and behavior problems remain?