Course outline

SQLite / Advanced: concurrency, recovery, and application boundaries / Lesson 14

Backups, restoration, and integrity evidence

Design SQLite recovery as a tested capability using consistent copies, isolated restoration, integrity checks, and explicit application dependencies.

About 5 min readContent reviewed
On this page

Lesson goals

  • Distinguish a live file from a consistent backup.
  • Exercise the online backup API safely.
  • Verify a restored copy and its business meaning.

Start with acceptable loss and recovery time

Recovery point objective describes acceptable data-loss exposure, while recovery time objective describes how long service restoration may take. A daily copy can be recent-looking yet expose many hours of work. A quickly restored database may still fail because an application version, encryption extension, external file, or configuration dependency is missing. Inventory the complete supported recovery unit.

Decide who owns backup scheduling, retention, access, verification, and restoration. A copy on the same failing storage does not protect against that failure. SQLite's embedded nature moves these responsibilities into the application or deployment design; it does not remove them. A backup timestamp and a successful copy command are useful observations, not proof that a consistent business service can be recovered.

Use an engine-coordinated copy boundary

The SQLite backup API copies database content through connections, supporting memory and file databases without assuming an ordinary live-file copy is consistent. Python exposes it as Connection.backup. Its source, destination, completion, and error behavior need deliberate ownership. Do not start a backup of an unintended configured path because the API itself is convenient.

The first complete program copies a synthetic memory source, then changes the source. The destination still contains the earlier copied value. This demonstrates an independent completed copy, not crash recovery or a concurrent performance guarantee. Autocommit is explicit so no unfinished source write transaction accidentally makes the demonstration wait indefinitely.

python Create and inspect an independent memory copy
import sqlite3
from contextlib import closing

with closing(sqlite3.connect(":memory:", isolation_level=None)) as source, \
     closing(sqlite3.connect(":memory:", isolation_level=None)) as copy:
    source.executescript("CREATE TABLE item(value INTEGER); INSERT INTO item VALUES(2);")
    source.backup(copy)
    source.execute("UPDATE item SET value=99")
    print("source", source.execute("SELECT value FROM item").fetchone()[0])
    print("copy", copy.execute("SELECT value FROM item").fetchone()[0])
Expected output
source 99
copy 2

The completed backup retains2 even after the source changes to99. Both databases are private and closed.

Restore into a separate target and check it

The second program uses a new private temporary directory for both source and backup files, then restores into another memory connection. It checks integrity and a known business value. All handles close before temporary storage is removed. No real application file is copied, replaced, renamed, or opened. The actual check output is part of the tested example.

A production drill should also verify schema version, foreign-key consistency, representative queries, application compatibility, and the recovered point in time. Measure retrieval, restoration, validation, and service readiness together. An isolated successful toy restore proves the example's mechanics, not the organization's recovery time or the health of a different backup stored elsewhere.

python A private-file backup and separate restoration drill
import sqlite3
from pathlib import Path
from tempfile import TemporaryDirectory
from contextlib import closing

with TemporaryDirectory(prefix="sqlite-backup-training-") as temporary:
    folder = Path(temporary)
    with closing(sqlite3.connect(folder / "source.db", isolation_level=None)) as source, \
         closing(sqlite3.connect(folder / "backup.db", isolation_level=None)) as backup, \
         closing(sqlite3.connect(":memory:", isolation_level=None)) as restored:
        source.executescript("CREATE TABLE account(id INTEGER PRIMARY KEY,amount INTEGER); INSERT INTO account VALUES(1,42);")
        source.backup(backup)
        print("backup_check", backup.execute("PRAGMA integrity_check").fetchone()[0])
        backup.backup(restored)
        print("restored", restored.execute("SELECT amount FROM account WHERE id=1").fetchone()[0])
        print("foreign_key_issues", len(restored.execute("PRAGMA foreign_key_check").fetchall()))
Expected output
backup_check ok
restored 42
foreign_key_issues 0

The backup passes its synthetic integrity check; the restored database contains42 and reports no foreign-key violations.

Do not discard live state while copying files

In WAL mode, committed data can reside in the WAL rather than only in the main database file. Copying the main file alone during active use can therefore create an incomplete or inconsistent recovery asset. Deleting the WAL to β€œclean up” is not a backup procedure. Choose the documented backup API, a properly coordinated snapshot, or another approved consistent-copy method.

VACUUM INTO can create a compact copy on supported releases, with its own destination and operational restrictions. It is not identical to an ordinary backup schedule or an arbitrary overwrite command. Storage capacity, filesystem guarantees, destination ownership, and application quiescence requirements must be reviewed. Never add destructive cleanup just to make an example path appear reusable.

Treat corruption evidence as a recovery incident

integrity_check and quick_check inspect structural consistency with different cost and coverage, while foreign_key_check addresses relational violations separately. A result of ok is not proof of every business invariant or of correct application data. If a check reports corruption, preserve evidence, restrict further writes through the incident process, and evaluate known-good recovery before attempting best-effort extraction.

Investigate storage, runtime defects, application misuse, and the exact file/sidecar history. SQLite recovery tools can be useful but do not guarantee complete original semantics or privacy of recovered deleted content. Do not label a salvaged file healthy merely because it opens. A recovery record should include source identity, method, losses or uncertainties, integrity evidence, and application-level reconciliation.

Common errors and fixes

A copied main file misses recent committed records.

Why it happens
Live WAL content was omitted from the copy boundary.
How to fix it
Use an engine-coordinated consistent backup and test restoration.
Prevent it next time
Document journal mode and backup ownership.

A backup API call waits unexpectedly.

Why it happens
Source/destination transaction state or contention was not considered.
How to fix it
Review explicit transaction completion and documented backup behavior.
Prevent it next time
Test the exact host API with isolated data.

An opened recovered file was called fully correct.

Why it happens
File readability was mistaken for structural and business validity.
How to fix it
Run appropriate checks and reconcile application facts.
Prevent it next time
Keep measured isolated restoration drills.

Check your understanding

1. A backup restores quickly but contains yesterday's data. Which objective can still fail?

Show worked solution

The recovery point/data-loss objective can fail despite a good restore duration. Determine how much committed work is protected and test the full capture-to-restoration process. Speed alone does not measure acceptable loss.

2. Does integrity_check returning ok prove there are no orphaned references or incorrect business balances?

Show worked solution

No. Foreign-key checks have their own scope, and business invariants require application-level reconciliation. Combine structural, relational, schema-version, and representative business checks rather than assigning one command every guarantee.

Key takeaways

  • A backup is a consistent recovery asset, not just a recent filename.
  • Restore drills verify both mechanism and application readiness.
  • Structural, relational, and business integrity are complementary checks.

Official references