Software Prefaulting: Warming Memory Before It's Accessed
Modern operating systems use demand paging: memory isn’t actually allocated until you touch it. This lazy approach saves resources, but it introduces latency spikes when a page fault occurs during a critical operation. Software prefaulting is the technique of deliberately triggering these faults early, when the latency doesn’t matter.
The Demand Paging Problem
When you allocate memory with malloc or mmap, the operating system typically just reserves virtual address space. No physical pages are assigned yet. The first time your code reads or writes to that memory, the CPU triggers a page fault. The kernel handles it, allocates a physical page, updates the page table, and resumes execution.
For most workloads, this is fine. But for latency-sensitive systems—real-time applications, high-frequency trading, game engines, audio processing—a page fault at the wrong moment creates an unacceptable stall. The fault handling might take microseconds or even milliseconds if the system is under memory pressure.
How Prefaulting Works
Software prefaulting walks through allocated memory regions ahead of time, touching each page to force the kernel to back it with physical RAM. The simplest implementation just writes a byte to the first address of each page:
void prefault(void *addr, size_t len) {
char *p = (char *)addr;
size_t page_size = sysconf(_SC_PAGESIZE);
for (size_t i = 0; i < len; i += page_size) {
p[i] = 0; // Touch each page
}
}
This ensures all page faults happen during initialization, not during the hot path. Some systems read instead of write, but writing is safer—it forces the kernel to allocate a writable page immediately, avoiding copy-on-write overhead later.
Stack Prefaulting
Stacks are particularly tricky. Thread stacks are usually allocated with a guard page at the bottom for overflow detection. But the rest of the stack is demand-paged. A deeply nested function call that suddenly needs many stack frames can trigger multiple page faults in sequence.
Real-time systems sometimes prefault the entire stack at thread creation. Linux’s pthread library supports this indirectly through the PTHREAD_STACK_MIN constant and careful use of alloca or variable-length arrays to touch stack pages during initialization.
High-performance servers sometimes spawn threads with large stacks (several megabytes) but only touch a fraction during normal operation. Prefaulting the expected working set eliminates surprise faults without wasting physical memory on the full allocation.
Transparent Huge Pages and Prefaulting
When transparent huge pages are enabled, prefaulting gets more complex. The kernel might initially allocate 4KB base pages, then later promote them to 2MB huge pages if access patterns justify it. Or it might allocate huge pages immediately.
Some applications explicitly disable transparent huge pages and use explicit huge page allocations (via hugetlbfs or MAP_HUGETLB) combined with prefaulting. This gives deterministic behavior: you know exactly when allocation happens and what page size you’re getting.
When Not to Prefault
Prefaulting has costs. It increases startup time and memory footprint. If your application allocates gigabytes but only uses a small working set, prefaulting everything wastes RAM and creates memory pressure for other processes.
It’s also unnecessary for workloads that can tolerate occasional latency spikes. Batch processing, background jobs, and most user-facing web applications don’t need it.
The value proposition is clearest when latency predictability matters more than average throughput, when you have a known memory access pattern, and when you’re willing to trade increased baseline memory usage for eliminated tail latency.
Prefaulting in Practice
Databases often prefault buffer pool memory at startup. Memory-mapped database engines might prefault critical index regions. Language runtimes with ahead-of-time compiled code sometimes prefault the code pages themselves.
Some kernel-bypass networking stacks prefault packet buffer rings. Audio and video processing pipelines prefault frame buffers. Embedded real-time systems routinely prefault everything they’ll touch.
The pattern appears whenever the cost of a page fault—measured in latency variance—exceeds the cost of eager allocation. It’s one of many techniques for converting unpredictable lazy behavior into predictable eager behavior, trading memory and initialization time for runtime consistency.