Modern CPUs organize cache memory in a hierarchy—typically L1, L2, and L3—with each level trading capacity for speed. What’s less obvious is the policy governing whether data present in a faster, smaller cache must also exist in slower, larger ones. This relationship, called cache inclusivity, has significant implications for effective cache capacity and performance.

Inclusive Caches: Redundant but Simple

An inclusive cache requires that any data in L1 or L2 must also be present in L3. If a cache line exists in L1, a copy lives in L3 as well. This creates redundancy—the same 64-byte cache line occupies space at multiple levels.

The advantage is coherence simplicity. In multi-core systems, when one core modifies data, other cores’ copies must be invalidated. With inclusive caches, the L3 can act as a directory: checking L3 tells you whether any core might have that line cached. You don’t need to snoop every core’s private L1 and L2. This makes cache coherence protocols faster and less power-hungry.

The cost is reduced effective capacity. If L3 is 12 MB and L1+L2 across all cores total 2 MB, inclusive organization means only 10 MB of L3 holds unique data. You’re paying for SRAM that duplicates what’s already in faster caches.

Intel used inclusive L3 caches extensively through their Core architecture generations. The coherence benefits outweighed the capacity loss for most workloads.

Exclusive Caches: Maximum Capacity

An exclusive cache takes the opposite approach: data in L1 cannot simultaneously exist in L3. When a line is fetched into L1, it’s evicted from L3 (or never placed there). When L1 evicts a line, it gets written to L3, which may then evict something else to main memory.

This maximizes total cache capacity. That 12 MB L3 plus 2 MB of L1+L2 now provides a full 14 MB of effective storage. No redundancy.

The downside is complexity. Coherence snooping must check all cache levels, increasing latency and power. Evictions also become more involved—data migrates between levels rather than simply being invalidated.

AMD used exclusive caches in some Bulldozer-era designs. The capacity gains were meaningful for workloads with large datasets, but the coherence overhead and implementation complexity proved challenging.

NINE: Non-Inclusive Non-Exclusive

Most modern processors use a hybrid policy called NINE (Non-Inclusive Non-Exclusive). Data in L1 may also be in L3, but doesn’t have to be. L3 acts as both a backing store and a victim cache, holding data evicted from L2 as well as lines that were never promoted to L1.

NINE preserves most of the capacity benefits of exclusivity—L3 isn’t forced to duplicate L1/L2 contents—while keeping coherence manageable. The L3 can still serve as an approximate filter: if a line isn’t in L3, it’s probably not in any private cache either. This isn’t guaranteed, so you need additional mechanisms (like snoop filters or small directories), but the common case remains fast.

Intel shifted to NINE starting with Skylake server processors. AMD’s Zen architectures also use NINE policies. The flexibility allows the hardware to adapt: frequently accessed data naturally migrates to L1 while L3 holds evicted working set, maximizing utilization of both levels without strict rules.

Why It Matters

Cache inclusivity affects real performance in subtle ways. Inclusive caches can show worse behavior with large working sets that spill beyond L1/L2 capacity, since L3 is partially wasted on duplicates. Exclusive and NINE caches extract more value from the same silicon area, but require more sophisticated coherence logic.

For software, this is mostly transparent—you don’t control inclusivity policy. But understanding it clarifies why cache sizing isn’t straightforward arithmetic. A CPU with 16 MB of L3 doesn’t necessarily provide 16 MB of effective cache. The inclusivity policy, combined with associativity limits and workload access patterns, determines actual available capacity.

As core counts rise and coherence traffic increases, expect continued evolution in cache organization. Some designs are exploring victim caches, probe filters, and distributed directories to decouple coherence from the L3 data array entirely. The simple inclusive model worked well for quad-core chips, but sixty-core server processors demand more sophisticated approaches.

Cache hierarchies appear simple on paper: L1 is fast, L3 is big. The inclusivity policy reveals that even this seemingly straightforward design space involves nuanced tradeoffs between capacity, coherence cost, and implementation complexity.