Docker / Compose and reliable runtime contracts / Lesson 9

Health checks, readiness, and dependency startup

Define what a probe proves and avoid confusing startup ordering with continuing application availability.

About 4 min readContent reviewed 2026-10-25

On this page

Lesson goals

Separate running, healthy, ready, and externally reachable states.
Design bounded probes and dependency recovery rather than relying on sleep.

Health is a declared test, not a universal truth

A running process can be unable to serve useful work. A health check gives the engine a success or failure signal from a configured command; the meaning depends entirely on that command. Readiness asks whether the service should receive the intended traffic now. A probe that returns success when Python exists proves less than one that performs an HTTP request, and neither automatically proves database permissions, business correctness, or external routing.

Make the probe executable and bounded

The fixture already contains Python and urllib, so its probe need not assume curl is installed in a slim image. An internal loopback request checks the local listener and response path. The fixture answers every GET with success, so this is intentionally only a process-local HTTP check. Production probes need a documented contract for required dependencies, timeout behavior, and safe low-cost responses without changing persistent state.

yaml A fixture-specific HTTP health check

services:
  api:
    image: parcel-api:lab
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=2).read()"]
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 5s

The command succeeds only if the local request completes without an exception within its bounds. It does not test the host publication at 18080. start_period changes how early failures count during initialization; it is not a promise that startup always completes after exactly five seconds. No healthy status is asserted without execution.

Dependency ordering applies at specific transitions

Compose can wait for a dependency marked service_healthy before starting a dependent service. A simple dependency relation normally establishes startup order rather than full readiness. These conditions do not remove the need for application reconnect logic if a dependency later fails or is replaced. A healthy database at one instant does not keep every future query successful, and a dependent process may need to recover without being recreated.

yaml A one-shot client gated by API health

services:
  api:
    image: parcel-api:lab
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=2).read()"]
      interval: 10s
      timeout: 3s
      retries: 3
  client:
    image: python:3.13-slim
    depends_on:
      api:
        condition: service_healthy
    command: ["python", "-c", "import urllib.request; print(urllib.request.urlopen('http://api:8080/', timeout=2).read().decode().strip())"]

Expected output

Conditional client output: parcel-api

If the declared API becomes healthy and remains available for the request, the client prints parcel-api and exits successfully. That conditional result is derived from the fixture, not an observed deployment. A later network failure can still invalidate the request; dependency gating is not a transaction or permanent availability guarantee.

Avoid probes that create their own incident

Expensive queries, full dependency scans, or synchronized probes can overload an already struggling service. Use a small request with a timeout shorter than the outer health timeout and select intervals from recovery goals. Be deliberate about whether dependency loss should make the service unready or merely degraded. Overly strict liveness-like checks can cause restart storms under supervisors that restart unhealthy instances; ordinary Docker health status alone is not such a universal restart action.

Observe transitions and external behavior

Read the recorded health history, probe exit status, and application logs, then compare with a request from the actual consumer network. A green internal probe can coexist with a wrong host port, certificate failure, or missing route at the public edge. Track startup duration and readiness flapping under realistic load. Tests should include dependency interruption and recovery, not merely the first successful request after a clean start.

Common errors and fixes

Health check fails because curl is not found.

Why it happens: The probe assumed a diagnostic utility that the runtime image never installed.
How to fix it: Use an available suitable client or intentionally include a reviewed probe tool.
Prevent it next time: Validate probe dependencies against the final runtime stage.

A healthy container is unreachable from the host.

Why it happens: The local probe did not test host publication or the external request path.
How to fix it: Inspect the consumer path separately from internal health evidence.
Prevent it next time: Define exactly which boundary each probe covers.

The dependent service fails after a dependency restart.

Why it happens: Startup gating was mistaken for permanent connection recovery.
How to fix it: Add bounded reconnect and retry behavior appropriate to the operation.
Prevent it next time: Rehearse mid-operation dependency failures and recovery.

Check your understanding

1. Does service_healthy prove a later purchase can commit?

Show worked solution

No. It proves the configured health condition was met at the relevant startup gate. A purchase has its own authentication, data, transaction, and dependency requirements. The application must validate and recover those operations independently and avoid turning a historical probe success into a business guarantee.

2. Why is a fixed sleep weaker than a bounded readiness condition?