AWS / Advanced: reliability, security, and governance / Lesson 18
Architecture decisions, release, and recovery
Make service trade-offs explicit and connect a reviewed architecture to safe release evidence, incident ownership, and user outcomes.
On this page
Lesson goals
- Explain the assumptions, responsibilities, and trade-offs of architecture decisions, release, and recovery.
- Interpret both examples and distinguish declared intent from verified evidence.
- Diagnose the listed failures and justify the worked decisions without using a real AWS account, credential, or service.
Write a decision instead of a product list
An architecture decision should name the user requirement, constraints, alternatives, assumptions, evidence, and consequences. Choosing EC2, Lambda, RDS, S3, or messaging is meaningful only in relation to workload behavior and team capability. A larger service diagram can increase operational and security complexity without improving the user outcome. The first synthetic decision record compares synchronous booking confirmation with asynchronous follow-up work and states the consistency boundary. It is not a universal recommendation for microservices or serverless. Use the Well-Architected pillars to surface trade-offs and open questions rather than declaring one service stack best for every application.
decision: synthetic-booking-confirmation-boundary
requirement: durable-correct-response-before-confirmation
follow_up: asynchronous-notification-with-idempotency
alternatives: reviewed-for-latency-operations-cost-and-recovery
assumptions: explicit-service-mode-and-Region-support
status: teaching-record-not-a-deployment-plan
The record distinguishes the authoritative transaction from optional follow-up. It selects no real AWS resources and makes no universal service recommendation.
Model one concrete end-to-end operation
Trace identity, network, compute, state, events, telemetry, and recovery for a valid request and for partial failure. A database write followed by event publication can leave a gap unless the application coordinates them through a suitable protocol such as an outbox. A durable queue does not make the original database transaction and an external payment atomic. The second synthetic sequence labels the commit boundary and the separate publish-retry obligation. Decide what clients see while follow-up work is pending, how duplicates are recognized, and who reconciles an uncertain external result. Architecture quality includes those edge cases, not only the happy-path arrows.
1. validate and authorize the synthetic request
2. atomically record booking and pending-notification intent
3. return the documented committed booking result
4. publish pending notification through reviewed retry logic
5. mark progress and reconcile uncertain duplicate outcomes
claim: protocol sketch, not executable transaction code
The sketch exposes the separate publication obligation. A real implementation needs a durable atomic store, consumer idempotency, retention, and failure tests; no code or message is executed here.
Release immutable identity with compatible state
Build reviewed inputs into a verifiable artifact and record its configuration, schema, identity, and credential references separately. Gradual exposure, blue-green, or another strategy must match data compatibility and available capacity. An old application version may not read new data or use a rotated credential, so a rollback is not automatically safe. CloudFormation status, Lambda publication, EC2 readiness, or load-balancer health each prove a limited boundary. Define user-path, integrity, latency, and error evidence before expansion. Keep approval tied to the exact artifact and target context rather than a mutable tag or an unverified latest build.
Respond to incidents through supported evidence
State impact and uncertainty, assign coordination and technical ownership, preserve the first useful safe evidence, and choose a bounded authorized action. Distinguish AccessDenied, transport failure, throttling, capacity shortage, and application validation rather than applying one broad administrator or restart fix. Compare account, Region, principal, resource, deployment, and time before drawing conclusions. Traffic recovery can leave messages, data, or security tasks unresolved, so keep residual owners and closure criteria visible. A post-incident review should improve missing controls and contracts instead of blaming the last person to change a template. No incident message or recovery action is sent or performed here.
Maintain an architecture that can be operated
Keep service support, quotas, cost dimensions, permissions, recovery evidence, and team skills current as demand and AWS features change. A dependency introduced for convenience can become a shared failure domain or an unowned bill. Retire unnecessary complexity only after reviewing data and trust consequences. Use authorized isolated exercises and safe synthetic work for validation; this course creates no real resources and runs no cloud, network, administrative, deployment, or load actions. Its examples support reasoning, not a certification that an account or architecture is secure, compliant, inexpensive, or highly available. Good decisions remain reviewable and can explain both their value and their limits.
Common errors and fixes
More services are called better architecture
- Why it happens
- Product count replaced requirements and operating evidence.
- How to fix it
- Compare alternatives against user, trust, cost, and recovery needs.
- Prevent it next time
- Keep decision records and revisit assumptions.
A successful infrastructure release hides business failure
- Why it happens
- Service control-plane status replaced end-to-end verification.
- How to fix it
- Check the bounded user and data contract before completion.
- Prevent it next time
- Define release evidence and abort ownership in advance.
Rollback is used after incompatible state change
- Why it happens
- Artifact history was confused with reversible data.
- How to fix it
- Use the reviewed forward-repair or reconciliation plan.
- Prevent it next time
- Test mixed-version and credential compatibility before exposure.
Check your understanding
1. A booking commits but publishing its notification fails. What must an architecture preserve?
Show worked solution
A durable pending-publication record or equivalent protocol must make follow-up discoverable and safely retryable without undoing a valid booking or duplicating effects. Define client-visible state and reconciliation ownership. A queue alone cannot recover intent that was never durably recorded.
2. When should a simple architecture remain instead of adopting a more elaborate managed-service pattern?
Show worked solution
When evidence shows it meets user, security, reliability, scale, and cost constraints with a clearer operating model. Added services need justified value and ownership. Prefer an explicit trade-off record over novelty, and revisit it when workload or requirements materially change.
Key takeaways
- An architecture decision should name the user requirement, constraints, alternatives, assumptions, evidence, and consequences.
- Build reviewed inputs into a verifiable artifact and record its configuration, schema, identity, and credential references separately.
- Keep service support, quotas, cost dimensions, permissions, recovery evidence, and team skills current as demand and AWS features change.