Networking / Advanced: policy, evidence, capacity, and resilient design / Lesson 18

Design resilient dependencies and reversible change

Turn availability goals into explicit dependency, failure-domain, verification, and rollback requirements.

About 4 min readContent reviewed 2026-10-25

On this page

Lesson goals

Map addressing, naming, identity, and forwarding dependencies across failures.
Evaluate synthetic resilience and acceptance records without administering a network.

Begin with the operation that must remain usable

A resilient design starts with a required user operation, tolerated failures, recovery objective, and acceptable degraded behavior. Counting devices or links is insufficient. A request may depend on address configuration, neighbor discovery, routing, DNS, time, certificates, identity, and application state. Name the owners and failure boundaries of those dependencies. Distinguish losing one component from losing an entire site, provider, power domain, or control system. Simpler explicit dependencies are often easier to verify than complex redundancy whose interactions and recovery rules are undocumented.

Redundant names can conceal one shared dependency

Two paths can terminate on different devices while sharing power, upstream transit, management, or naming. The following offline model requires every listed component of a path to be available. It compares shared and separated power dependencies under one declared failure. Real designs contain more conditions, capacities, and partial failures than these sets represent. The result is a counterexample to simplistic redundancy claims, not proof that a particular physical layout or provider combination satisfies an availability target.

python Test a declared common-mode failure offline

failed = {'power-1'}
shared = {'A': {'edge-A', 'power-1'}, 'B': {'edge-B', 'power-1'}}
diverse = {'A': {'edge-A', 'power-1'}, 'B': {'edge-B', 'power-2'}}
def survivors(paths):
    return sorted(name for name, dependencies in paths.items() if dependencies.isdisjoint(failed))
print(f'shared_power_survivors={survivors(shared)}')
print(f'diverse_power_survivors={survivors(diverse)}')

Expected output

shared_power_survivors=[]
diverse_power_survivors=['B']

Failure of power-1 removes both shared-power paths but leaves path B in the separated fixture. The labels are synthetic components, not actual devices. Unmodeled dependencies and insufficient surviving capacity can still invalidate a real service design.

Address and DNS recovery have independent clocks

A second authoritative DNS server helps only if its zone data, reachability, and operational dependencies are appropriate. Recursive caches, negative caching, client refresh behavior, and existing connections can delay adoption of a changed answer. A low TTL does not force every application to reconnect instantly. Address allocation and neighbor state have different lifecycles again; duplicate ownership after failover can break otherwise correct routes. Include IPv4 and IPv6 behavior, resolver access, certificate identity, and dependency bootstrapping in the design rather than treating DNS or DHCP redundancy as a complete service recovery mechanism.

A change needs success and rollback conditions

Record the intended addressing, routes, policy, names, dependencies, and configuration revision before a change. Define required positive and negative outcomes and a bounded observation window. The synthetic acceptance record below evaluates three independent requirements; passing reachability cannot excuse unexpected access or a violated delay budget. Its thresholds are declared teaching choices, not general network recommendations. The Boolean decision neither performs rollback nor proves that rollback would be safe; execution authority, state compatibility, and the approved recovery procedure remain separate responsibilities.

python Evaluate a synthetic acceptance report

report = {'required_paths_ok': True, 'latency_ms': 120, 'unexpected_allows': 1}
checks = {
    'paths': report['required_paths_ok'],
    'latency': report['latency_ms'] <= 100,
    'policy': report['unexpected_allows'] == 0,
}
accepted = all(checks.values())
print(f'accepted={accepted}')
print('failed_checks=' + ','.join(name for name, passed in checks.items() if not passed))

Expected output

accepted=False
failed_checks=latency,policy

The report passes required paths but fails its declared latency and policy checks, so accepted is False. Values are fixed teaching data, not measured traffic or a scan. The program produces a decision record only and never changes configuration or activates a rollback.

Rehearse degraded service and controlled failback

A recovery plan includes configuration and zone backups, required credentials handled securely, ownership, dependency order, and evidence that the service is usable again. Failback is another change and can move traffic or state unexpectedly. Preserve observations and identify whether a safe rollback, forward correction, or isolated restoration is appropriate. Offline dependency reviews and synthetic acceptance cases prepare those decisions without exercising live infrastructure. Any later failure injection, traffic generation, route change, or device administration needs its own scope and authorization. No such operational action occurs here.

Common errors and fixes

Two devices are treated as two independent failure domains

Why it happens: Power, upstream, control, or naming dependencies remain shared.
How to fix it: Map dependency intersections and check surviving service capacity.
Prevent it next time: State the exact failure each redundancy claim tolerates.

A DNS update is assumed to move all clients immediately

Why it happens: Caching, refresh behavior, and existing connections were omitted.
How to fix it: Include realistic naming and connection adoption behavior in recovery objectives.
Prevent it next time: Treat DNS state and application sessions as separate dependencies.

Rollback begins without checking state compatibility

Why it happens: Configuration reversal was assumed to undo every external or persistent effect.
How to fix it: Choose the approved rollback or forward-recovery path for the observed state.
Prevent it next time: Rehearse restoration and document limits before deployment.

Check your understanding

1. Why does the shared-power fixture have no surviving path?

Show worked solution

Both alternatives require power-1, and that component belongs to the declared failed set. Different edge labels do not remove the common dependency. Moving only pathB to power-2 leavesB available in this simplified model, but other shared dependencies and remaining capacity still require review.

2. Would reducing the report’s latency to 80 make the change acceptable?