Proxmox VE / Advanced: coordination and recovery evidence / Lesson 16

Replication, snapshots and storage failures

Compare scheduled local-storage replication, shared storage and distributed storage while keeping recovery-point, capacity and backup claims distinct.

About 4 min readContent reviewed 2026-10-28

On this page

Lesson goals

Explain incremental snapshot-based replication under supported backends.
Measure last successful synchronization rather than schedule alone.
Assess storage recovery load and independent backups.

Compare three data-placement models

Local storage keeps guest bytes on a node unless another mechanism copies them. Shared storage exposes the same data to multiple hosts under its access model. Distributed storage such as Ceph spreads storage responsibility across services and failure domains. Proxmox storage replication is another distinct mechanism: supported local volumes are synchronized at scheduled intervals using snapshots and incremental changes. None of these descriptions alone guarantees an application recovery objective. The first diagram distinguishes the models so a reviewer can ask where the latest writable copy lives and which failures make it inaccessible.

text Different placement and recovery contracts

local + scheduled replication: source volume -> periodic target copy
shared storage: host A + host B -> same storage service
distributed storage: client -> distributed storage services and placement
independent backup: separately retained recoverable history

The diagram configures no storage service and does not imply every backend supports replication.

Reason from last success

A configured replication interval is an intention. The usable target reflects the last successful synchronization, and failed or delayed jobs can make it older. Writes after that point may be lost if the source fails and recovery uses the target. The second program calculates a synthetic age of seventeen minutes from integer timestamps. It does not query replication jobs. RPO assessment must use actual completion and application consistency evidence, not the nominal interval printed in a settings page. Also distinguish the copy’s data point from the later time at which a transfer finished.

python Synthetic replicated-data age

incident_minute = 120
last_usable_data_minute = 103
print(incident_minute - last_usable_data_minute)

Expected output

Seventeen minutes is the modeled gap. It is not a measured replication lag or a guaranteed data-loss amount.

Check supported backends and scope

The official replication framework documents supported storage types and required matching storage definitions. In the reviewed model, local ZFS is the documented supported replication backend; recheck the release-specific support table before design. Do not generalize this to arbitrary directory, LVM or external storage just because those backends can hold a VM disk. Confirm every included volume, target availability and capacity. External mounts and devices need their own treatment. A replication task that covers one disk cannot justify a claim about an application whose state spans several independently updated data sources.

Budget for recovery and rebalancing

Replication and distributed-storage recovery consume network, CPU, memory and disk activity in addition to guest demand. Ceph introduces monitors, managers and storage daemons with their own health and quorum concerns; the Proxmox cluster quorum is a separate coordination layer. A degraded storage system can remain available yet have less redundancy and higher latency. Do not fill all capacity based on a healthy-state dashboard. Reserve and validate the transition requirements for the intended failure scenario, and keep storage recovery traffic from undermining latency-sensitive cluster communication through an unreviewed shared bottleneck.

Keep backup and ownership separate

Replication can copy unwanted changes and does not automatically provide independently retained history or protection from shared administrative deletion. Keep a backup strategy and restore evidence alongside the replication design. During failure recovery, establish the latest usable data and exclusive guest ownership before making a target writable. A locally visible replica is not permission to start a second instance while the source outcome is uncertain. This course executes no snapshot, replication, Ceph or storage command. Its age calculation is a narrow offline model, and actual durability and recovery claims remain explicitly untested.

Common errors and fixes

A schedule is reported as guaranteed replication freshness

Why it happens: Failed jobs and actual data timestamps were ignored.
How to fix it: Assess the last usable successful data point and its application consistency.
Prevent it next time: Track freshness separately from configured frequency.

All storage backends are assumed eligible for the same replication

Why it happens: Disk-hosting capability replaced the replication support table.
How to fix it: Check backend-specific support and included volume semantics.
Prevent it next time: Version the storage capability inventory.

A replica replaces an independently retained backup

Why it happens: Availability copies were confused with recoverable history.
How to fix it: Design separate retention, deletion protection and restoration evidence.
Prevent it next time: Review failure and corruption scenarios individually.

Check your understanding

1. A job is scheduled every five minutes but last usable data is seventeen minutes old. Which supports RPO review?

Show worked solution

The seventeen-minute observed data gap is relevant, subject to consistency and timestamp confidence. The five-minute interval is only intent. Investigate delayed or failed synchronization and evaluate the usable target against the application’s recovery objective rather than assuming the configured schedule was achieved.

2. Why should Ceph health and Proxmox quorum not be collapsed into one green indicator?