Thread Migration Cost: When Moving Work Between Cores Becomes Expensive


When an operating system moves a thread from one CPU core to another, it’s not just changing which execution unit runs the code. That migration triggers a cascade of hidden costs that can degrade performance in ways that aren’t obvious from looking at CPU utilization alone.

What Gets Left Behind

Modern CPUs maintain per-core state that’s optimized for the work currently running. When a thread migrates, it leaves behind warm caches, TLB entries, and branch predictor state that took time to build up. The destination core starts cold, needing to rebuild all of that context from scratch.

L1 and L2 caches are core-private in most architectures. When a thread moves, every cache line it was using must be fetched again, even if the data still exists in the last level cache. For memory-intensive workloads, this can mean hundreds of nanoseconds or even microseconds of additional latency per migration event. The thread essentially pays the cost of cache warming twice: once on the original core, and again after migration.

Translation lookaside buffers suffer the same fate. Virtual-to-physical address mappings that were cached in the old core’s TLB need to be reestablished on the new core through expensive page table walks. High TLB miss rates after migration can stall memory accesses while the hardware performs multi-level lookups through page tables.

Cache Coherence Traffic

Migration doesn’t just invalidate caches—it generates coherence protocol traffic. If the migrated thread writes to memory that was cached on its previous core, the cache coherence protocol must invalidate those lines and transfer ownership. In MESI-based systems, this means bus traffic and potential serialization as cores coordinate exclusive access.

For threads that frequently modify shared data structures, migration amplifies coherence overhead. Instead of maintaining a stable pattern of which cores “own” which cache lines, migration shuffles ownership unpredictably, increasing the rate of expensive cache-to-cache transfers and invalidation storms.

NUMA Penalty

On NUMA systems, migration can move a thread farther from its memory. A thread that was running on a core with local memory access might migrate to a different socket, turning fast local accesses into slow remote ones. The latency gap between local and remote NUMA memory can be 2-3x or more, and this penalty persists for every memory access until either the thread migrates back or the OS migrates the memory pages themselves.

Automatic NUMA balancing in Linux attempts to track access patterns and migrate pages closer to the cores that touch them most, but this is reactive and comes with its own overhead. Frequent thread migration fights against page migration heuristics, creating churn as the system tries to optimize for an unstable access pattern.

When Migration Makes Sense

Despite these costs, migration isn’t always wrong. Schedulers migrate threads to balance load, respond to priority changes, or avoid contention. An idle core can often absorb migration costs better than letting work pile up on a busy one. The key is avoiding unnecessary migrations where the cost exceeds the benefit.

Some workloads are migration-tolerant by nature. Batch jobs that stream through large datasets and don’t fit in cache anyway pay little penalty. Latency-sensitive services with small working sets, on the other hand, suffer disproportionately when their carefully warmed state evaporates.

Controlling Migration

CPU affinity masks let applications restrict which cores a thread can run on, reducing migration at the cost of flexibility. Pinning latency-critical threads to specific cores eliminates migration overhead entirely but can lead to load imbalance if done carelessly. Thread-per-core architectures take this further, eliminating both migration and cross-core synchronization by partitioning work completely.

Scheduler tunables like sched_migration_cost_ns in Linux control how aggressively the kernel migrates threads for load balancing. Increasing this value makes the scheduler more reluctant to migrate, preferring cache warmth over perfect load distribution. Modern schedulers also track per-thread execution history and penalize migrations that would cross NUMA domains or cache hierarchies unnecessarily.

The performance cost of thread migration is a reminder that CPU cores aren’t perfectly interchangeable execution slots. Each core carries state, and moving work between them is a disruptive event with measurable latency consequences. Understanding when and why threads migrate helps explain performance cliffs that raw CPU metrics don’t capture.