Every database makes a promise: once it tells you a write succeeded, that write should survive a crash, a power loss, or a kernel panic seconds later. Keeping that promise without grinding every transaction to a halt is harder than it sounds, and the technique that makes it possible is one of the oldest and most quietly important ideas in database design: write-ahead logging, or WAL.

The Problem WAL Solves

A database keeps most of its working data in memory, because memory is fast and disk is slow. But memory is volatile. If the process dies before those in-memory changes reach durable storage, the changes are gone, even though the client was already told the transaction committed.

The naive fix is to flush every modified page to disk before acknowledging a write. That works, but it’s brutally slow. Database pages are often much larger than the actual change being made, so rewriting a whole page for a one-row update wastes disk bandwidth. Worse, a crash in the middle of writing a page can leave it half-written and corrupt, which is arguably worse than losing the update entirely.

WAL sidesteps both problems by changing what gets written first.

The Core Idea

Instead of modifying data pages directly and hoping the write completes cleanly, the database first appends a compact record describing the change to a sequential log file. Only after that log record is safely on disk does the transaction get acknowledged as committed. The actual data pages can be updated in memory and flushed to disk later, in batches, whenever it’s convenient.

This ordering is the whole trick. Sequential appends to a log are fast, because there’s no seeking involved and the write pattern is predictable. And because the log entry is small and self-contained, writing it is far less likely to be interrupted mid-write than a full page rewrite.

If the system crashes before the underlying data pages are updated, that’s fine. On restart, the database replays the log from the last known-good checkpoint, reapplying any committed changes that never made it to the data files. This recovery process, often called redo, reconstructs the exact state the database was in at the moment of the crash. Some WAL implementations also support undo, rolling back changes from transactions that were in progress but never committed.

Why It’s Everywhere

WAL isn’t a niche optimization. It’s the durability mechanism underneath most relational databases, and the same pattern shows up in filesystems (journaling), key-value stores, and distributed consensus systems, where a replicated log is the source of truth that state machines replay to reach agreement.

The reason it generalizes so well is that “append sequentially, then apply asynchronously” is a good answer to a problem that shows up constantly in systems design: how do you make an operation durable without making it slow. Sequential writes are cheap on spinning disks and still meaningfully cheaper than random writes on SSDs, since they play nicer with wear leveling and write amplification.

The Tradeoffs

WAL isn’t free. The log itself needs to be truncated or archived periodically, or it grows without bound. Checkpointing, the process of flushing enough data pages to disk that old log entries can be discarded, has to be tuned carefully. Checkpoint too aggressively and you lose the batching benefits that made WAL worthwhile; checkpoint too rarely and recovery after a crash takes longer, since there’s more log to replay.

There’s also a durability knob most databases expose: whether to fsync the log on every commit, which guarantees durability but costs latency, or to buffer commits and sync periodically, which trades a small window of potential data loss for significantly higher throughput. Understanding which mode a database defaults to, and why, is often the difference between a system that behaves as expected during an outage and one that has an unpleasant surprise waiting.

WAL is the kind of infrastructure that works so reliably it becomes invisible. But it’s worth understanding, because the same append-first, apply-later pattern keeps reappearing anywhere a system needs to be both fast and honest about what it’s promised.