Django / Correctness and production evidence / Lesson 18

Observability and operational readiness

Connect safe logs, health signals, release evidence, and recovery procedures without claiming that one successful check proves the whole service.

About 4 min readContent reviewed 2026-10-25

On this page

Lesson goals

Observe a request without exposing its secrets
Record a small structured event safely
Release and recovery need independent evidence

Observe a request without exposing its secrets

Useful observability connects a request or job to its outcome, timing, dependency behavior, and failure class. A correlation identifier helps join events across layers without logging complete submitted records. Choose log levels according to operational meaning and avoid duplicate reports at every call depth. Passwords, session tokens, reset links, and sensitive query values do not belong in ordinary logs. Django logging configuration can route structured evidence to the reviewed collector, but storage access, retention, and redaction remain part of the design. A detailed traceback is valuable in restricted diagnostics and may be inappropriate in a public response.

Separate liveness, readiness, and business health

A process responding to a simple endpoint proves a narrow liveness condition. Readiness asks whether this instance should receive its intended workload, while business health can require deeper invariants and dependency evidence. Do not make a lightweight probe perform expensive writes or unlimited dependency checks. A transient dependency failure should not automatically trigger a restart storm that worsens the incident. Define who consumes each signal and what action it permits. An HTTP 200 health response with no checks must not be described as proof that authentication, database integrity, external payments, backups, or every route is working.

Record a small structured event safely

The first complete program logs a synthetic completion event into an in-memory buffer with an explicit request id field. It disables propagation for that local logger and removes its handler afterward, so the fixture neither writes a project log nor leaks a handler into another test. The printed line is deterministic. A production logger would add reviewed timestamp, severity, operation, and trace conventions without indiscriminately copying context. Logging itself can fail or become expensive, so operational design should consider backpressure and bounded payloads. A log statement is evidence that code reached a point, not independent proof a remote effect completed successfully.

python Isolated executable example

from django.conf import settings
settings.configure(
    SECRET_KEY="synthetic-course-key",
    USE_I18N=False, USE_TZ=True,
    INSTALLED_APPS=[],
    DATABASES={"default": {"ENGINE": "django.db.backends.sqlite3", "NAME": ":memory:"}},
    DEFAULT_AUTO_FIELD="django.db.models.AutoField",
    ALLOWED_HOSTS=["testserver"],
)
import django
django.setup()

import io
import logging

stream = io.StringIO()
logger = logging.getLogger("course.local.fixture")
logger.setLevel(logging.INFO)
logger.propagate = False
handler = logging.StreamHandler(stream)
handler.setFormatter(logging.Formatter("%(levelname)s %(message)s request=%(request_id)s"))
logger.addHandler(handler)
try:
    logger.info("request complete", extra={"request_id": "lab-7"})
    print(stream.getvalue().strip())
finally:
    logger.removeHandler(handler)
    handler.close()

Expected output

INFO request complete request=lab-7

The fixture logs only a synthetic event into memory and removes its own handler. No external collector or application log is used.

Keep a liveness fixture honest about its scope

The second program returns a tiny local JsonResponse and inspects it with RequestFactory. It confirms only that the isolated view can report the declared liveness result. It does not create a listening server, query a database, verify deployment routing, or prove readiness. A real readiness endpoint should use bounded, intentionally selected evidence and a response contract suitable for the orchestrator. Protect any detailed diagnostic endpoint with the appropriate access policy. Avoid placing secret versions, full settings, infrastructure addresses, or raw error payloads into a publicly available health response merely because an operator finds them convenient.

python Isolated executable example

from django.conf import settings
settings.configure(
    SECRET_KEY="synthetic-course-key",
    USE_I18N=False, USE_TZ=True,
    INSTALLED_APPS=[],
    DATABASES={"default": {"ENGINE": "django.db.backends.sqlite3", "NAME": ":memory:"}},
    DEFAULT_AUTO_FIELD="django.db.models.AutoField",
    ALLOWED_HOSTS=["testserver"],
)
import django
django.setup()

import json
from django.http import JsonResponse
from django.test import RequestFactory

def live(request):
    return JsonResponse({"status": "ok"})

response = live(RequestFactory().get("/live/"))
print(response.status_code, json.loads(response.content)["status"])

Expected output

200 ok

This local liveness view checks no dependencies. Its successful response is deliberately not described as database or deployment readiness.

Release and recovery need independent evidence

A deployment review should connect the tested artifact, supported Django/Python versions, configuration source, schema compatibility, asset manifest, and rollback limits. Framework deployment checks can identify concerns but do not replace a restore drill or application smoke tests under the selected environment. Plan how workers stop accepting new work, finish or cancel owned tasks, and release resources under the platform’s grace period. Backups must be restorable under the actual engine and key/storage arrangement, not merely recent filenames. Use an incident record that states what is known, what remains uncertain, and which approved reversible change will test the next hypothesis. This course does not perform a production rollout.

Common errors and fixes

A public error response reveals credentials

Why it happens: Internal diagnostic context was exposed without filtering.
How to fix it: Separate safe client feedback from restricted diagnostics.
Prevent it next time: Review log and response fields for sensitive data.

A health check is called proof of complete readiness

Why it happens: The probe’s narrow condition was overstated.
How to fix it: Define each signal and its permitted interpretation.
Prevent it next time: Test consumer actions as well as probe status.

A recent backup is assumed restorable

Why it happens: No compatible isolated restore was verified.
How to fix it: Maintain a reviewed restore drill and recovery evidence.
Prevent it next time: Measure recovery under the actual dependency arrangement.

Check your understanding

1. What should a correlation id accomplish without becoming a secret carrier?

Show worked solution

It should connect relevant events for one request or job using a safe identifier, while actual credentials and private payloads remain outside ordinary logs. Validate any client-supplied form of the id and limit its length and formatting before including it in structured evidence.

2. Why can restarting every instance on one dependency timeout worsen an outage?