AWS / Advanced: reliability, security, and governance / Lesson 15
Backup, disaster recovery, and reconciliation
Define recoverable data, authorized access, dependency order, and verified user outcomes across account and Regional boundaries.
On this page
Lesson goals
- Explain the assumptions, responsibilities, and trade-offs of backup, disaster recovery, and reconciliation.
- Interpret both examples and distinguish declared intent from verified evidence.
- Diagnose the listed failures and justify the worked decisions without using a real AWS account, credential, or service.
Define objectives before selecting a strategy
Recovery point objective describes tolerated data-loss interval, while recovery time objective describes the allowed recovery duration under stated conditions. They are requirements to test, not properties automatically supplied by Multi-AZ or backup scheduling. The first synthetic record names a consistency scope and an isolated recovery target. Include detection, decision, access, provisioning, restoration, validation, and reconciliation in the timeline. A scheduled backup every fifteen minutes does not guarantee that the latest restorable point is fifteen minutes old. Failed copies, missing logs, key access, or inconsistent datasets can make actual recoverability worse than the schedule suggests.
workload: synthetic-booking
rpo_minutes: 15
rto_minutes: 60
consistency_scope: booking-and-payment-reference-ledger
recovery_target: isolated-reviewed-location
external_side_effects: disabled-in-rehearsal
key_and_identity_access: independently-verified
The objectives are fictional requirements, not measured AWS performance. The scope includes related records and key access rather than just storage bytes.
Separate availability, copies, and independence
Replication can improve selected availability properties but may also preserve unwanted application changes. A historical copy needs defined consistency, retention, deletion protection, ownership, and restoration. AWS Backup covers supported resources and configurations rather than every possible application or external system. Cross-account or cross-Region copying has service, encryption, policy, and destination prerequisites. A retained archive that cannot be decrypted or accessed during the incident is not a usable recovery capability. Review which failure and credential domains the copy survives, and avoid assuming that another AZ supplies the same independence as another account or Region.
Compare recovery patterns and data contracts
Backup-and-restore, pilot-light, warm-standby, and multi-site designs trade ongoing cost, preparedness, complexity, and recovery time. An active-active label does not solve concurrent data conflicts or duplicated side effects. Choose from the user requirement and the exact service support rather than assuming the most complex design is safest. DNS, network, certificates, quotas, identity, artifacts, and configuration may need preparation in the recovery location. Define which system is authoritative during transition and how split-brain or delayed replication is detected. A failback to the original location is another migration requiring data and traffic planning.
Verify the actual recoverable point
The second offline model compares a synthetic incident time with the latest verified point, independently of the planned schedule. A real restore exercise must also validate schema, relationships, access, application version, and selected user journeys. Disable real external side effects in the recovery lab so restored jobs do not charge accounts or send customer messages. Reconcile payments, messages, and idempotency records that may be newer outside the restored database. Use approved evidence and never download sensitive backup content merely for a tutorial. A successful service restore API response is only one intermediate observation.
incident_minute = 120
latest_verified_point_minute = 90
planned_interval_minutes = 15
print(incident_minute - latest_verified_point_minute)
print("actual verified gap differs from planned schedule")
30
actual verified gap differs from planned scheduleThe synthetic verified gap is thirty minutes despite a fifteen-minute plan. It says nothing about a real backup system, service guarantee, or acceptable loss.
Maintain a rehearsed recovery capability
A runbook needs decision authority, safe copy selection, dependency order, key access, scoped permissions, stop conditions, communication ownership, and closure evidence. Measure the full sequence in an authorized isolated environment with synthetic work and update it after architecture, scale, or credential changes. No backup, copy, vault policy, restore, failover, DNS change, account access, or AWS request is performed here. The arithmetic cannot certify RPO or RTO. Recovery is established when authorized people restore an appropriate point and verify correct useful service, including uncertain external effects, within the declared constraints.
Common errors and fixes
Backup schedule is reported as actual RPO
- Why it happens
- Latest verified restoration point was not checked.
- How to fix it
- Measure the recoverable point and communicate the real gap.
- Prevent it next time
- Monitor backup and restore validity, not only job scheduling.
Recovery restores data but duplicates external effects
- Why it happens
- Other systems retained newer operations.
- How to fix it
- Reconcile ledgers and idempotency before reopening affected work.
- Prevent it next time
- Include cross-system consistency in rehearsals.
A cross-Region copy cannot be used
- Why it happens
- Keys, permissions, service support, or destination setup were missing.
- How to fix it
- Follow the approved recovery access and service contract.
- Prevent it next time
- Test independent restore prerequisites in advance.
Check your understanding
1. A database is restored to an earlier point but a payment provider kept later charges. What remains?
Show worked solution
Reconcile the payment references and business ledger using authorized evidence before replaying operations. Do not charge again just because the restored database lacks a row. Restoration and external transaction recovery are different obligations.
2. Why is a warm standby not automatically a successful disaster-recovery plan?
Show worked solution
It may lack current data, keys, identity, quotas, routing, compatible artifacts, or sufficient useful capacity. Verify the full transition and user outcome, including failback and reconciliation. Prepared compute reduces only part of the recovery problem.
Key takeaways
- Recovery point objective describes tolerated data-loss interval, while recovery time objective describes the allowed recovery duration under stated conditions.
- Backup-and-restore, pilot-light, warm-standby, and multi-site designs trade ongoing cost, preparedness, complexity, and recovery time.
- A runbook needs decision authority, safe copy selection, dependency order, key access, scoped permissions, stop conditions, communication ownership, and closure evidence.