Course outline

SQLite / Advanced: concurrency, recovery, and application boundaries / Lesson 13

Concurrency, WAL, and busy snapshots

Explain SQLite writer serialization and reader snapshots, reproduce bounded private-file contention, and apply version-aware WAL and retry policies.

About 5 min readContent reviewed
On this page

Lesson goals

  • Trace a reader snapshot across another commit.
  • Distinguish BUSY, LOCKED, and stale-snapshot contexts.
  • Review WAL files, checkpoints, and patch requirements.

WAL changes reader interaction, not the one-writer rule

SQLite supports multiple readers but one active writer per database at a time. WAL can let readers retain an earlier snapshot while a writer commits new content. It does not create unlimited simultaneous writers or remove all busy conditions. Transaction duration and the number of connections matter, but adding a connection pool cannot manufacture write capacity beyond the locking model.

Rollback-journal and WAL modes have different coordination details. Keep the actual journal mode, transaction sequence, and host API settings recorded. :memory: databases do not turn into normal WAL-file databases merely by requesting the pragma. These demonstrations therefore use newly created private temporary files and close every connection before removing their directory; no project database or shared service is opened.

Observe a stable reader and a later view

The first Python program selects1 in an open reader transaction, commits a writer's change to2, then reads again through the same reader. It still sees1. After the reader transaction ends, its next query sees2. This is a controlled sequential schedule with two connections, not a threaded load test or a measurement of throughput.

The example's WAL activation is checked rather than assumed. Host filesystem and VFS support are part of the environment. WAL normally requires same-host shared-memory coordination and is not a general network-filesystem solution. A deployment's permissions must cover relevant database, journal, and shared-memory behavior, not just the main file's apparent readability.

python A sequential private-file WAL snapshot demonstration
import sqlite3
from pathlib import Path
from tempfile import TemporaryDirectory
from contextlib import closing

with TemporaryDirectory(prefix="sqlite-wal-training-") as temporary:
    path = Path(temporary) / "training.db"
    with closing(sqlite3.connect(path, isolation_level=None)) as writer, \
         closing(sqlite3.connect(path, isolation_level=None)) as reader:
        assert writer.execute("PRAGMA journal_mode=WAL").fetchone()[0] == "wal"
        writer.executescript("CREATE TABLE item(value INTEGER); INSERT INTO item VALUES(1);")
        reader.execute("BEGIN")
        first = reader.execute("SELECT value FROM item").fetchone()[0]
        writer.execute("UPDATE item SET value=2")
        same = reader.execute("SELECT value FROM item").fetchone()[0]
        reader.execute("COMMIT")
        later = reader.execute("SELECT value FROM item").fetchone()[0]
        print("snapshot", first, same, later)
Expected output
snapshot 1 1 2

The open reader keeps1 across the writer commit; a new reader transaction observes2. No simultaneous checkpoint/write race is attempted.

Handle contention with an outcome-aware boundary

The second program holds a write transaction and gives a second connection zero wait time, producing a known SQLITE_BUSY outcome. After rollback releases the first writer, the second can begin and commit. This distinguishes availability of the writer role from whether a particular business request is safe to retry. The program creates no threads and repeats no load.

A busy timeout can wait for some transient contention but does not repair a stale read snapshot. SQLITE_BUSY_SNAPSHOT requires abandoning the old snapshot and retrying the complete eligible operation with fresh reads. SQLITE_LOCKED often points to a different connection/shared-cache or statement interaction. Preserve extended error information and context instead of treating every β€œlocked” message as one identical condition.

python A bounded second-writer conflict and release
import sqlite3
from pathlib import Path
from tempfile import TemporaryDirectory
from contextlib import closing

with TemporaryDirectory(prefix="sqlite-busy-training-") as temporary:
    path = Path(temporary) / "training.db"
    with closing(sqlite3.connect(path, isolation_level=None, timeout=0)) as first, \
         closing(sqlite3.connect(path, isolation_level=None, timeout=0)) as second:
        first.executescript("CREATE TABLE item(value INTEGER); INSERT INTO item VALUES(1);")
        first.execute("BEGIN IMMEDIATE")
        try:
            second.execute("BEGIN IMMEDIATE")
        except sqlite3.OperationalError as error:
            assert error.sqlite_errorcode == sqlite3.SQLITE_BUSY
            print("contender", error.sqlite_errorname)
        else:
            raise AssertionError("expected bounded writer contention")
        finally:
            first.execute("ROLLBACK")
        second.execute("BEGIN IMMEDIATE")
        second.execute("UPDATE item SET value=3")
        second.execute("COMMIT")
        print("after_release", second.execute("SELECT value FROM item").fetchone()[0])
Expected output
contender SQLITE_BUSY
after_release 3

The second writer is rejected while the first owns the transaction, then succeeds after explicit rollback.

Keep checkpoint and sidecar ownership explicit

WAL content is periodically checkpointed into the database. Long-lived readers can prevent complete checkpoint progress or WAL reuse, so a growing WAL can be a transaction-lifecycle symptom rather than permission to delete it. The -wal and -shm sidecars are part of the live database state and coordination model. Copying only the main file during active use can omit committed content.

Choose backup and checkpoint procedures from the documented runtime and workload. Do not delete sidecars, force arbitrary checkpoints, or change synchronous settings as a generic fix for busy errors. Review long readers, open statements, write duration, storage behavior, and application cancellation. A bounded retry should avoid synchronized storms and must not repeat irreversible external effects blindly.

Separate tested behavior from production patch guidance

Official WAL documentation reviewed on2026-10-25 describes the WAL-reset race, fixed in3.51.3 and selected backports3.44.6 and3.50.7. The documented scenario involves concurrent write/checkpoint activity across connections and can corrupt a database. The3.45.1 validation runtime is therefore a historical teaching target, not a recommendation for production WAL deployment. No attempt is made to reproduce that race.

Check the actual vendor build and applicable fix rather than comparing version strings lexically or assuming a framework bundles a patched engine. Keep a recoverable backup and test application concurrency with supported patched software. When sustained multiwriter, remote shared-file, or operational requirements exceed the embedded model, evaluate a different storage architecture instead of promising that WAL removes every limitation.

Common errors and fixes

database is locked

Why it happens
Another connection owns an incompatible resource or a snapshot upgrade cannot proceed.
How to fix it
Classify extended error and transaction state before a bounded retry.
Prevent it next time
Keep transactions short and test overlap schedules.

WAL grows while a reader remains open.

Why it happens
The reader retains an old snapshot needed for checkpoint/reuse behavior.
How to fix it
Finish the owned read transaction and inspect documented checkpoint state.
Prevent it next time
Track reader lifetime and cancellation cleanup.

A3.45.1 demonstration was treated as patch advice.

Why it happens
Validation identity was confused with current production suitability.
How to fix it
Consult official WAL fix notes and vendor patch status.
Prevent it next time
Record feature support and patch policy separately.

Check your understanding

1. Why can a WAL reader still see1 after another connection commits2?

Show worked solution

Its active read transaction retains the earlier snapshot. End that transaction and begin fresh work to observe the later commit. This is not evidence that the commit failed, and a stale snapshot should not be promoted blindly into a write attempt.

2. Does setting a long busy timeout solve every write conflict?

Show worked solution

No. It can wait for some contention but does not create multiple writers or repair a stale snapshot. Classify the operation and use a bounded policy with complete transaction restart where required, preserving request identity and external-side-effect safety.

Key takeaways

  • WAL retains one-writer serialization and version-specific caveats.
  • Busy outcomes require transaction-aware classification.
  • Sidecars, checkpoints, backups, and patch status form one operational contract.

Official references