Victim Caches: The Small Buffer That Reduces Conflict Misses


Direct-mapped caches are fast and simple, but they have a problem: two memory addresses that map to the same cache line create conflict misses, even when the cache is mostly empty. Victim caches solve this with a small, fully-associative buffer that catches evicted cache lines before they disappear, giving them a second chance.

The Direct-Mapped Cache Conflict Problem

In a direct-mapped cache, each memory address maps to exactly one cache line based on its address bits. When two frequently accessed addresses happen to map to the same line, they thrash: each access evicts the other’s data, causing a miss even though the rest of the cache has free space. This is a conflict miss, distinct from capacity misses (cache is full) and compulsory misses (first access to data).

Set-associative caches reduce conflict misses by allowing each address to map to multiple possible locations, but they’re more complex and slower to access. Every additional way in the set requires another tag comparison in parallel, consuming more silicon and power. Direct-mapped caches remain attractive for L1 caches where access latency is critical, but the conflict miss problem persists.

How Victim Caches Work

A victim cache sits between the L1 cache and the next memory level. It’s a small, fully-associative buffer—typically 4 to 16 entries—that stores cache lines recently evicted from L1. When the L1 cache needs to evict a line due to a conflict, that line goes into the victim cache instead of being discarded.

On an L1 miss, the processor checks the victim cache in parallel with (or just before) accessing L2. If the missing line is found in the victim cache, it swaps places with the conflicting line currently in L1. The previously evicted line gets restored to L1, and the line that caused the conflict moves into the victim cache. This swap operation preserves both working set items instead of discarding one.

The fully-associative organization means any evicted line can go into any victim cache slot, so the buffer doesn’t introduce new conflict misses of its own. The small size keeps the associative lookup fast despite the parallel tag comparisons.

Why It Works

Victim caches exploit temporal and spatial locality in a specific way. When two addresses conflict in L1, there’s often a pattern: the program alternates between them or accesses them in short bursts. The victim cache captures this thrashing behavior by holding the most recently evicted lines—exactly the ones likely to be needed again soon.

The performance gain comes from converting what would be L2 or memory accesses into much faster victim cache hits. Even though the victim cache adds one more place to check, its small size and fully-associative design make lookups fast enough that the net effect is positive. The typical working set for many programs includes small clusters of conflicting addresses, and a 4-to-8-entry victim cache can capture most of these cases.

Implementation Tradeoffs

The victim cache’s small size is both its strength and its limitation. A larger victim cache would catch more conflicts but would slow down the associative lookup. The sweet spot is usually 4 to 16 entries, where the lookup remains a single cycle and the hit rate on thrashing patterns stays high.

The swap operation requires additional datapath complexity. When a victim cache hit occurs, the hardware must move data in both directions: the hit line goes to L1, and the evicted L1 line goes into the victim cache. This bidirectional movement adds control logic and potentially an extra write port, which is why victim caches are typically only used with L1, where conflict miss rates are highest.

Modern processors often use more sophisticated techniques—higher associativity in L1 itself, skewed associativity, or hash-rehash schemes—but the victim cache remains a clean, focused solution to conflict misses. It doesn’t try to eliminate all misses; it just gives the most recently evicted lines one more chance, and that’s often enough.