ASP.NET Core / Advanced: Verification and Reliable Operations / Lesson 14

Observability and Operational Contracts

Connect structured logs, traces, metrics, health, and ownership without logging secrets or treating one green signal as complete correctness.

About 5 min readContent reviewed 2026-10-26

On this page

Lesson goals

Choose signals from an operational question.
Define correlation and low-cardinality dimensions.
Separate liveness, readiness, and user-facing indicators.

Different signals establish different facts

Logs describe events, traces connect work across an operation, and metrics summarize populations over time. Health endpoints describe declared checks, not every user transaction. A 200 health response does not prove a database migration, authorization policy, cache, or recovery path is correct. Start with the question and define the signal’s resource, time, and eligibility scope.

Telemetry can be sampled, filtered, delayed, or missing. An absence of logs does not prove no request occurred. Preserve application/revision identity, units, outcome categories, and safe correlation so an incident can be traced without dumping every request body. User-facing SLIs need a defined numerator and denominator.

Use a stable operation identity without exposing payloads

The first complete program assigns a synthetic trace identifier to an HttpContext and records only safe method/path/status fields. It tests constructing a bounded diagnostic record, not a logging provider or distributed trace export. In a real system, incoming correlation headers must be treated under their trust and validation policy rather than blindly accepted as authoritative identity.

Prefer structured log templates with named properties over unbounded interpolated data when using ILogger. Choose levels and event categories intentionally. Logging connection strings, bearer tokens, cookies, or full sensitive DTOs in an error path can turn an incident into a data leak.

csharp Construct a safe synthetic diagnostic record

using System;
using System.Collections.Generic;
using Microsoft.AspNetCore.Http;

var context = new DefaultHttpContext { TraceIdentifier = "synthetic-op-7" };
context.Request.Method = "GET";
context.Request.Path = "/reports";
context.Response.StatusCode = 200;
var record = new Dictionary<string, string>
{
    ["operation"] = context.TraceIdentifier,
    ["method"] = context.Request.Method,
    ["route"] = "/reports",
    ["outcome"] = "success"
};
Console.WriteLine($"operation={record["operation"]} fields={record.Count} outcome={record["outcome"]}");

Expected output

operation=synthetic-op-7 fields=4 outcome=success

The record has four controlled fields and no token/body; no telemetry backend is contacted.

Units and cardinality are correctness decisions

A request-error rate, active-request count, latency distribution, and queue age describe different populations. Define exclusions, time windows, and aggregation. Averaging percentages without denominators or averaging percentiles does not automatically create the overall metric. A route-template dimension is often bounded; raw URLs or user IDs can create high cardinality and expose sensitive data.

The second program computes a synthetic success ratio from explicit counts. It demonstrates arithmetic only, not instrumented ASP.NET metrics, a dashboard, or alert delivery. Match actual instrument types and labels to the question and test reset/missing-data behavior in the intended observability environment.

csharp Calculate a synthetic request indicator

using System;

int eligibleRequests = 1000;
int goodRequests = 985;
if (eligibleRequests <= 0 || goodRequests < 0 || goodRequests > eligibleRequests)
    throw new InvalidOperationException("invalid synthetic counters");
decimal ratio = (decimal)goodRequests / eligibleRequests;
Console.WriteLine($"good={goodRequests} eligible={eligibleRequests} percent={(ratio * 100).ToString("0.0", System.Globalization.CultureInfo.InvariantCulture)}");

Expected output

good=985 eligible=1000 percent=98.5

The declared counters give 98.5 percent; no production SLI or alert is validated.

Readiness and liveness need distinct failure meaning

Liveness asks whether restarting the process is an appropriate response to its state; readiness asks whether it should receive work under a defined service criterion. A dependency outage should not automatically trigger destructive restart loops if restart cannot fix it. Health-check details can be sensitive and should not be exposed publicly without a deliberate policy.

Separate endpoint reachability, check result, orchestrator interpretation, and human response. Registration alone does not prove the route, authorization, or deployment probe is correct. Keep startup/shutdown draining and migration readiness in the operating contract rather than returning healthy unconditionally to avoid restarts.

A signal without ownership is not an incident process

Define who receives an alert, what it means, which runbook applies, and how to verify recovery. Review telemetry cost, retention, access, and sampling as application requirements. A tracing/logging backend is an external dependency with its own availability and data-handling contract; application correctness should not depend on unsafe synchronous diagnostic calls.

Tests should inspect safe fields, error context, correlation, no-data cases, and health response policy. Isolated integration should cover actual instrumentation and probe wiring where authorized. These examples use only local records and arithmetic; no logger destination, trace collector, health server, or external monitoring service is started.

Common errors and fixes

A raw user ID is used as an unbounded metric label.

Why it happens: Diagnostic detail is confused with aggregate dimension design.
How to fix it: Use bounded reviewed dimensions and separate controlled logs where needed.
Prevent it next time: Review cardinality and sensitive fields.

A dependency outage triggers endless liveness restarts.

Why it happens: Health meaning is not tied to a useful recovery action.
How to fix it: Separate readiness/dependency failure from restart-worthy process failure.
Prevent it next time: Test orchestrator interpretation conceptually and in approved integration.

A green dashboard is called proof of recovery.

Why it happens: A summary signal replaces end-to-end service evidence.
How to fix it: Verify the actual user operation and dependencies.
Prevent it next time: Record measurement scope and gaps.

Check your understanding

1. Why should a latency metric use a stable route template rather than every raw request URL as a dimension?

Show worked solution

Raw URLs can contain arbitrary IDs, query values, and sensitive information, producing high cardinality and difficult aggregation. A route template provides a bounded operation category while detailed diagnostics can remain in controlled logs. Define what is excluded and preserve units/measurement points. A technically accepted label set is not automatically a useful or safe observability design.

2. A health endpoint returns 200 but all order submissions fail. What should be reviewed?