Memory fragmentation is a silent tax on long-running JVM applications. Even after garbage collection reclaims unused objects, the heap can become a swiss cheese of small free regions too scattered to satisfy large allocation requests. The JVM must compact the heap to defragment memory, but moving live objects while application threads are running creates a complex coordination problem.

The Fragmentation Problem

When objects are allocated and freed over time, the heap develops gaps. An application might have 2GB of free memory but fail to allocate a 10MB array because no contiguous block exists. Without compaction, the only solution is expanding the heap size, wasting memory and increasing GC overhead.

Traditional stop-the-world collectors solve this by pausing all application threads, then sequentially copying live objects to pack them together. This works but creates pause times proportional to live set size. For heaps measured in tens or hundreds of gigabytes, pause times become unacceptable for latency-sensitive applications.

Concurrent Compaction Mechanics

Modern collectors like ZGC and Shenandoah perform compaction concurrently with application threads. The core challenge is that application code holds pointers to objects the collector is actively moving. If an application thread dereferences a stale pointer, it accesses invalid memory.

The solution relies on memory barriers and pointer remapping. When the collector selects heap regions to compact, it copies live objects to new locations but leaves forwarding pointers at the old addresses. The collector uses read barriers, small pieces of injected code that intercept object access. When a thread reads an object reference, the barrier checks if it points to an evacuated region and follows the forwarding pointer if needed.

This indirection has a cost. Every object load potentially executes barrier logic, though modern implementations optimize the common path to a single conditional branch that CPUs predict well. The barrier updates the reference to point directly to the new location, so subsequent accesses skip the forwarding step.

Selecting Regions to Compact

Not all heap regions are equally worth compacting. The collector profiles regions by liveness, the ratio of live bytes to total capacity. Regions with 5% liveness yield more free space per byte copied than regions with 80% liveness.

Collectors typically compact the sparsest regions first, maximizing fragmentation reduction per unit of work. This heuristic reduces overall compaction overhead because the collector processes fewer live objects to reclaim the same amount of contiguous space.

Some collectors use separate young and old generation regions. Young regions experience high mortality rates, making them efficient compaction targets. Old regions compact less frequently but often accumulate the most severe fragmentation over time.

Write Barrier Coordination

Compaction creates another problem during the copying phase. If an application thread writes a reference to an object being moved, the collector must track that write to avoid losing the reference. Write barriers intercept reference stores and record them in thread-local buffers.

The collector periodically drains these buffers, ensuring all references to evacuated objects are updated. This maintains the invariant that no stale pointers exist once the old region is reclaimed.

Trade-offs and Overheads

Concurrent compaction trades pause time for throughput. Memory barriers add CPU overhead to every object access, typically 2-10% depending on workload characteristics and barrier implementation. Applications with tight inner loops dereferencing many objects see higher overhead than those dominated by I/O or computation on primitive types.

The approach also requires more memory. The collector needs space for both old and new object copies during evacuation. Most implementations reserve a percentage of heap capacity as evacuation headroom, unavailable for application data.

Modern JVMs make concurrent compaction the default for low-latency configurations because the pause time benefits outweigh the throughput costs for most interactive services. The ability to compact a 100GB heap in under 10ms fundamentally changes what’s possible for JVM-based systems at scale.