Log-structured merge trees power some of the most widely deployed storage engines today, from RocksDB and LevelDB to Cassandra and ScyllaDB. While the basic LSM architecture—buffering writes in memory and flushing them to immutable sorted files on disk—solves the write throughput problem elegantly, it introduces a new challenge: compaction. The process of merging and rewriting these immutable files is where LSM trees make their most consequential performance tradeoffs.

Why Compaction Exists

An LSM tree accumulates sorted runs of key-value pairs over time. Without compaction, a single read would need to check every run to find the latest version of a key. Compaction merges overlapping runs into fewer, larger files, reducing read amplification and reclaiming space from deleted or overwritten keys. But this comes at a cost: compaction rewrites data that was already written to disk, a phenomenon called write amplification.

The ratio of actual bytes written to disk versus bytes written by the application defines write amplification. In LSM systems, this ratio can range from 10x to 50x depending on the compaction strategy and workload. High write amplification consumes disk bandwidth, increases wear on SSDs, and can bottleneck overall throughput.

Size-Tiered Compaction

Size-tiered compaction waits until several files of similar size accumulate, then merges them into a single larger file. This strategy minimizes write amplification because each byte is rewritten only once per tier as it moves up the size hierarchy. It performs well for write-heavy workloads and time-series data where old data is rarely updated.

The downside is space amplification. Because files can contain many versions of the same key across different tiers, the storage footprint can be much larger than the logical data size. Read performance also suffers when multiple overlapping files exist at the same tier, forcing reads to check each one.

Leveled Compaction

Leveled compaction organizes files into discrete levels, where each level is roughly ten times the size of the previous one. Files within a level do not overlap in key range. When a level exceeds its size threshold, files are merged into the next level, maintaining the non-overlapping invariant.

This strategy optimizes for read performance. A read touches at most one file per level, keeping read amplification low. It also keeps space amplification minimal since only one version of each key exists per level. The tradeoff is higher write amplification—data is rewritten multiple times as it moves through levels. For read-heavy workloads or deployments where storage cost matters, leveled compaction is often the right choice.

Tiered-Plus-Leveled Hybrids

Some systems blend the two approaches. Upper levels use size-tiered compaction to minimize write amplification for recent data, while lower levels use leveled compaction to control space and improve read performance on older, stable data. This hybrid adapts to workloads with mixed access patterns, though it adds complexity to tuning and capacity planning.

Compaction and Latency

Compaction is I/O-intensive and can interfere with foreground operations. Systems mitigate this through throttling, scheduling compaction during low-traffic periods, or using separate I/O bandwidth allocations. Some engines offer predictable compaction—incrementally merging small chunks to avoid large, disruptive rewrites. Others prioritize user-facing operations and allow compaction to lag when load is high, accepting temporary read slowdowns.

Choosing a Strategy

The right compaction strategy depends on workload characteristics. Write-heavy systems with cold historical data favor size-tiered compaction. Read-heavy systems or those with frequent updates benefit from leveled compaction. Monitoring write amplification, space amplification, and P99 read latencies provides the signals needed to tune compaction parameters or switch strategies as requirements evolve.

Compaction is the engine that makes LSM trees viable at scale, but it is also their most expensive operation. Understanding how different strategies navigate the tradeoff space helps operators deploy LSM-based systems with predictable performance and cost.