Most databases rely on write-ahead logging (WAL) to ensure durability and atomicity. Before modifying a page on disk, the database writes a log entry describing the change. If the system crashes mid-transaction, the WAL serves as a recovery journal. It works, but it adds complexity: you’re writing data twice, managing log segments, and coordinating checkpoint operations.

Shadow paging takes a fundamentally different approach. Instead of logging changes and modifying pages in place, the database writes modified pages to new locations on disk. The old pages remain untouched until the transaction commits. At commit time, the database atomically updates a single root pointer to reference the new page tree. If the system crashes before the pointer update, the old tree is still intact. If it crashes after, the new tree is complete. Either way, you get atomicity without a separate log.

How It Works

Every page in the database lives in a tree structure rooted at a master pointer. When a transaction modifies a page, the database allocates a new physical location, writes the updated content there, and leaves the original page unchanged. Any parent pages that reference the modified page also need updating, so they get copied to new locations as well. This cascades up the tree until you reach the root.

The root pointer itself is the commit point. It’s a single atomic write, often just a few bytes stored in a known location. Once that write completes, the transaction is durable. Before the write, the database still points to the old tree. After, it points to the new one. No log replay, no redo or undo records.

This makes every committed transaction a complete snapshot of the database. The old pages form a consistent point-in-time view. If you want snapshot isolation or time-travel queries, you can keep the old root pointer around and read from the previous tree. This is exactly what LMDB does: readers hold references to old snapshots while writers build new ones.

The Copy-On-Write Cascade

The performance story depends heavily on how much of the tree you need to copy. If you update a single leaf page, you also need to copy every ancestor page from the leaf to the root. For a deep tree, that’s a lot of writes. A single logical update can trigger a cascade of physical copies.

This makes shadow paging write-intensive for small, scattered updates. Updating 100 random rows might mean copying hundreds of pages across the entire tree. WAL-based systems, by contrast, write a compact log entry for each change and modify the original pages in place. For random write workloads, shadow paging can amplify write volume significantly.

Bulk updates fare better. If you’re modifying many rows in adjacent pages, the copy cascade is amortized. You copy each ancestor page once, regardless of how many leaf pages beneath it changed. Workloads that modify large contiguous regions or append new data benefit from this structure.

Space Management and Garbage Collection

Old pages don’t disappear automatically. After committing a transaction, the database has both the old tree and the new one. The old pages become garbage unless they’re still referenced by an open snapshot. Eventually, you need to reclaim that space.

Some systems handle this with reference counting or epoch-based reclamation. Once no readers hold a reference to a snapshot, the pages in that snapshot can be freed. Others use background compaction to rewrite the tree and consolidate live data. SQLite’s rollback journal mode uses shadow paging implicitly during a transaction, but it commits by replacing the original file, avoiding long-term garbage.

The space overhead can be substantial for long-running snapshots. A reader holding an old snapshot prevents the database from reclaiming any pages modified since that snapshot was created. In a high-churn environment, this can bloat disk usage quickly.

Where It Appears

LMDB is the canonical example. It uses shadow paging exclusively, with no WAL. The entire database is a memory-mapped B+tree, and every write transaction produces a new root. Readers never block writers, and writers never block readers, because they’re working with different snapshots. The tradeoff is write amplification and space reclamation overhead.

SQLite supports shadow paging implicitly during transactions, though it typically uses rollback journals or WAL mode for durability. Some research databases and versioned data structures use shadow paging as a core design principle because it naturally supports immutability and snapshot isolation.

When It Makes Sense

Shadow paging fits read-heavy workloads with occasional bulk writes and a need for snapshot isolation. If you want lock-free readers and can tolerate write amplification, it’s a clean design. It also suits environments where you value simplicity over raw write throughput: no log format to parse, no checkpoint logic, no complex crash recovery.

But if you have high write concurrency, small random updates, or tight space constraints, WAL-based designs usually win. The cascading copy overhead and space reclamation complexity make shadow paging a niche choice, powerful in the right context but not a universal replacement for traditional logging.