Modern CPUs are fast, but memory access is slow. This fundamental asymmetry drives a programming paradigm shift that inverts traditional object-oriented wisdom: data-oriented design prioritizes how data flows through the CPU cache hierarchy over how code is organized into logical abstractions.

The Memory Wall Problem

A CPU can execute hundreds of instructions in the time it takes to fetch data from main RAM. L1 cache access takes around 4 cycles, L2 maybe 12 cycles, L3 around 40 cycles, but RAM access costs 200+ cycles. This gap has widened for decades as processor speeds increased faster than memory speeds.

Traditional object-oriented design encourages grouping related data and behavior together. A Game Entity might contain position, velocity, health, AI state, and rendering data in one structure. When you iterate over thousands of entities to update positions, you load entire objects into cache even though you only need position and velocity. The rest is cache pollution: wasted bandwidth evicting data you’ll actually use soon.

Structure of Arrays vs Array of Structures

Data-oriented design reorganizes memory around access patterns. Instead of an array of Entity structs, you maintain separate arrays: one for positions, one for velocities, one for health values. When updating movement, you iterate through tightly packed position and velocity arrays. Each cache line holds data you actually need, not a mix of relevant and irrelevant fields.

This isn’t just theoretical. Games that switched from array-of-structures to structure-of-arrays patterns report 2-10x speedups on core loops. The CPU’s hardware prefetcher can predict sequential access patterns and load cache lines ahead of time, but only when data is actually sequential in memory.

Rethinking Polymorphism

Object-oriented polymorphism creates data layout problems. Virtual function calls require dereferencing a vtable pointer, and storing different derived types in the same container scatters objects across memory. Iterating a vector<Animal*> where each animal might be a different subclass means chasing pointers to different heap allocations—each a potential cache miss.

Data-oriented alternatives replace runtime polymorphism with data. Instead of a hierarchy of enemy types with virtual update() methods, store enemy data in type-specific contiguous arrays and dispatch to appropriate update functions based on type tags. This eliminates pointer indirection and enables better cache utilization.

Cache-Friendly Algorithms

Data-oriented thinking changes algorithm selection. A binary search on a sorted array sounds efficient: O(log n) comparisons. But each comparison might miss cache, jumping to a random memory location. For moderately sized datasets, linear search through a packed array can outperform binary search because sequential access keeps data in cache and enables prefetching.

Hash tables similarly suffer from pointer chasing and scattered allocations. Open addressing schemes that store keys and values inline in a contiguous array often outperform chained hash tables for cache-resident datasets, despite worse theoretical collision handling.

Hot-Cold Data Splitting

Not all data within a logical entity is accessed with the same frequency. Player health gets checked constantly during gameplay, but the player’s achievement list only matters during menu screens. Grouping hot data separately from cold data improves cache density.

Database systems have used column-oriented storage for exactly this reason: analytical queries touch few columns but many rows. Storing each column contiguously means reading only relevant data, not entire row structures.

Tradeoffs and Context

Data-oriented design trades code abstraction for performance. Splitting a logical entity across multiple arrays makes code less intuitive. Adding a new field requires touching multiple data structures. Maintenance complexity increases.

This tradeoff makes sense when performance matters: game engines, high-frequency trading systems, scientific simulations, database internals. It makes less sense for typical application logic where developer velocity and maintainability outweigh nanosecond-level performance gains.

The key insight remains valuable regardless: modern performance is dominated by memory access patterns, not instruction counts. Understanding cache behavior and organizing data accordingly unlocks performance improvements that algorithmic optimization alone cannot achieve. Whether you fully embrace data-oriented design or simply apply its principles selectively, thinking about memory layout is essential for performance-critical code.