Most discussions of write-ahead logging focus on durability guarantees and crash recovery. But production databases face a less glamorous problem: what happens to old log files after their data has been checkpointed? The naive answer is to delete them. The performance-conscious answer is to recycle them.

WAL recycling is a technique where databases reuse log file segments instead of constantly creating and deleting files. It sounds like a minor optimization, but at high throughput it significantly reduces filesystem overhead and prevents performance cliffs that can appear when file creation becomes a bottleneck.

The Cost of File Creation

Creating a file isn’t free. The filesystem must allocate an inode, update directory entries, potentially allocate disk blocks, and perform multiple synchronous metadata updates to maintain consistency. On many filesystems, these operations require lock acquisition that serializes across the entire filesystem or at least within a directory.

For a database writing hundreds of megabytes per second to its WAL, this means creating new log segments every few seconds or minutes. If each creation requires acquiring a global lock and performing synchronous I/O for metadata updates, it becomes a throughput bottleneck. Worse, it introduces latency spikes. A transaction that happens to commit while the database is creating a new log segment may stall waiting for filesystem operations to complete.

The problem compounds on cloud block storage where metadata operations often have higher baseline latency than local SSDs. A 5ms file creation latency barely matters for occasional operations but becomes painful when repeated hundreds of times per hour.

How Recycling Works

Instead of deleting log segments after they’re no longer needed for recovery, the database renames them and adds them to a pool of recyclable segments. When a new segment is needed, the database pulls from the pool rather than creating a fresh file.

The implementation varies but typically looks like this: log segments are named sequentially (000000010000000A, 000000010000000B, etc.). When segment 000000010000000A is no longer needed because all its data has been checkpointed to permanent tables, the database renames it to something like 000000010000001F—the next needed segment number—rather than deleting it.

From the filesystem’s perspective, this is just a rename operation within the same directory. The inode stays the same, disk blocks remain allocated, and no metadata synchronization is required beyond updating directory entries. The database then seeks to the beginning of the file and starts writing new log records.

The Space-Performance Trade-off

Recycling means keeping more disk space allocated than strictly necessary. If your peak write rate creates 50 log segments per hour but your average is 10, you’ll have roughly 50 segments allocated even during quiet periods. For 16MB segments, that’s 800MB of disk space that stays allocated.

This trade-off made sense even when disk was expensive, but it’s essentially free on modern systems. The performance benefit—eliminating creation latency spikes and reducing lock contention—far outweighs the cost of a few hundred megabytes of disk space.

Most databases expose configuration parameters to control how many segments to keep in the recycling pool. PostgreSQL’s wal_recycle parameter, for example, can disable recycling entirely if you prefer to reclaim space aggressively, though this is rarely desirable in production.

Copy-on-Write Filesystem Interactions

WAL recycling has interesting interactions with copy-on-write filesystems like Btrfs and ZFS. These filesystems avoid in-place overwrites, so when the database seeks to the beginning of a recycled segment and starts writing, the filesystem allocates new blocks rather than reusing the old ones. The old blocks are freed later during garbage collection.

This partially defeats the purpose of recycling—you still allocate new blocks—but it eliminates the file creation overhead. The database doesn’t wait for metadata operations, and the filesystem can batch block allocation more efficiently than it can batch file creation. It’s a smaller win than on traditional filesystems, but still measurable at high throughput.

Observing Recycling in Production

You can watch recycling happen in real time. On PostgreSQL, the pg_wal directory shows log segments being renamed rather than created and deleted. Monitoring the directory size shows it stabilizes at some multiple of the segment size rather than constantly growing and shrinking.

Performance monitoring during recycling transitions can reveal whether file creation was actually a bottleneck. If P99 commit latency drops noticeably after enabling recycling, filesystem operations were likely stalling transactions. If there’s no difference, your workload may not generate WAL quickly enough for creation overhead to matter.

WAL recycling is one of those optimizations that seems obvious in retrospect but took years to become standard practice. It’s a reminder that filesystem operations have real costs at scale, and that sometimes the best way to avoid overhead is to avoid the operation entirely.