Photo by Brian Kostiuk on Unsplash
CPU Cache Line Bouncing: Multi-Core Contention and Coherence Traffic
When multiple CPU cores repeatedly read and write the same cache line, performance can collapse. The cache line bounces between cores, forcing each to wait while the cache coherence protocol transfers ownership. What looks like a simple shared variable becomes a serialization bottleneck that limits scalability across cores.
How Cache Line Bouncing Happens
Modern CPUs maintain cache coherence through protocols like MESI. When one core writes to a cache line, the protocol ensures that all other cores invalidate their copies. If another core then writes to the same line—even a different byte within it—it must first obtain exclusive ownership. The line transfers from one core’s cache to another, generating coherence traffic on the interconnect.
This becomes pathological when multiple cores take turns writing. Each write forces the line to bounce to a new owner. The core requesting ownership must wait for the transfer, often paying a latency penalty of hundreds of cycles. At high contention, the cache line spends more time in transit than being used, and throughput degrades to the speed of the coherence protocol rather than the speed of the cache.
False Sharing and Spatial Locality
Cache line bouncing often occurs through false sharing: when independent variables happen to share the same cache line. Two threads might update logically separate counters, but if those counters are adjacent in memory and fall within the same 64-byte line, every update triggers a coherence exchange. The sharing is false because the variables are independent, but the CPU’s granularity of coherence tracking treats them as one unit.
Languages with compact data structures amplify this. A struct with multiple fields, or an array of small objects, packs data tightly. That density is excellent for sequential access but disastrous when different cores write to nearby fields concurrently. The cache line becomes a false sharing hazard that nullifies the performance gains of parallelism.
Performance Impact at Scale
The cost of cache line bouncing scales with the number of cores. Two cores competing for a line cause slowdowns; eight cores competing cause collapse. Each core’s write invalidates the others, creating a storm of coherence messages. Latency increases, throughput drops, and the system spends more cycles managing coherence than doing work.
Profiling tools can expose this. Linux perf can count cache coherence events like cache-misses and node-load-misses. Intel VTune identifies contested cache lines directly. When a hot variable shows up with high invalidation rates across multiple cores, bouncing is the likely culprit.
Mitigation Strategies
The primary defense is to avoid sharing. Redesign data structures so that each core owns its cache lines exclusively. Use per-core counters instead of a single shared counter, aggregating results only when needed. Pad structures so that frequently written fields occupy separate cache lines—sacrificing memory for performance.
When sharing is unavoidable, batching writes reduces bouncing. Instead of updating a shared variable on every operation, accumulate changes locally and flush them periodically. This trades freshness for throughput, amortizing the coherence cost across many updates.
Read-copy-update (RCU) patterns can eliminate bouncing for read-heavy workloads. Readers access data without taking locks or triggering coherence traffic, while writers create new versions. Reads vastly outnumber writes in many systems, so keeping reads coherence-free yields major gains.
For contended data that must be updated frequently, consider using atomic operations with relaxed memory ordering. While still contended, relaxed atomics allow the CPU to avoid some of the coherence overhead that sequentially consistent operations require.
Architecture and Scalability
Cache line bouncing is a fundamental constraint of shared-memory multiprocessing. As core counts increase, the coherence fabric becomes the bottleneck. Systems with many cores and NUMA topologies see even worse effects when bouncing crosses NUMA nodes, adding remote memory latency to the coherence delay.
This is why modern high-performance systems increasingly adopt shared-nothing architectures. Thread-per-core designs, lock-free data structures, and message-passing models all aim to minimize shared mutable state. When cores don’t share writable cache lines, they don’t bounce them, and the system scales linearly with core count.
Understanding cache line bouncing is essential for building software that performs well on modern multi-core hardware. The abstraction of shared memory is convenient, but the underlying coherence mechanism imposes real costs. Designing for cache exclusivity, not just correctness, is what separates scalable code from code that stalls under parallelism.