Serverless computing promised to make infrastructure disappear. You write a function, upload it, and the platform handles provisioning, scaling, and teardown. But anyone who has run a latency-sensitive workload on a serverless platform knows the catch: cold starts. Years into the serverless era, the problem hasn’t gone away, it’s just been managed down. Understanding why reveals a lot about the tradeoffs baked into on-demand compute.

What Actually Happens During a Cold Start

When a request arrives for a function that isn’t currently running anywhere, the platform has to do real work before your code executes. It needs to find a host with available capacity, allocate a sandbox or micro-VM, load the runtime, initialize the language environment, pull in your code and dependencies, and run any top-level initialization logic before the handler even fires. For a Node.js function this might be tens of milliseconds. For a JVM-based function with a large dependency tree, it can stretch into seconds.

Once that instance is warm, subsequent requests reuse it and skip most of this work, which is why cold starts are inconsistent. A function can feel instant most of the time and then spike in latency the moment traffic patterns shift, a new version deploys, or the platform scales down idle instances to save resources.

Why It’s a Structural Tradeoff, Not a Bug

The entire value proposition of serverless is that you don’t pay for idle capacity. Providers aggressively reclaim unused instances because keeping every function warm indefinitely would defeat the economics of the model for both the provider and the customer. Cold starts are the visible cost of that elasticity.

This is fundamentally different from a traditional server, which stays warm because you’re paying for it whether or not it’s handling traffic. Serverless shifts that cost from “always paying for idle capacity” to “occasionally paying in latency.” Any fix has to work within that constraint rather than against it, which is why the industry has converged on mitigation rather than elimination.

How Platforms Mask the Problem

A few strategies show up repeatedly across serverless providers:

• Provisioned or reserved concurrency: customers pay to keep a set number of instances warm at all times, essentially buying back some of the always-on model for the workloads that need it.
• Lightweight virtualization: technologies like microVMs and hardware-assisted sandboxing reduce the overhead of spinning up an isolated execution environment compared to a full VM boot.
• Snapshotting and checkpointing: some platforms take a snapshot of a fully initialized process and restore from that snapshot instead of running initialization from scratch, which can cut cold start time significantly for heavier runtimes.
• Smaller, faster runtimes: the popularity of Go and Rust for serverless functions, and the push toward lighter interpreters and ahead-of-time compilation for other languages, is partly a direct response to cold start sensitivity. Less to initialize means less latency when a new instance spins up.
• Predictive pre-warming: providers use traffic patterns to keep capacity ready ahead of expected demand, though this only helps for predictable load and does nothing for the first request of a genuinely new spike.

The Road Ahead

None of these approaches eliminate cold starts, they narrow the window in which they matter and reduce their severity. That’s likely to remain the case, because the underlying tension between cost efficiency and instant readiness is structural, not something a clever runtime tweak can fully dissolve.

What’s changed is who has to think about it. A few years ago, cold starts were something every serverless developer needed to actively design around. Increasingly, they’re an implementation detail the platform manages on your behalf, with knobs available for the workloads where a few hundred milliseconds actually matters. That’s a reasonable place for the industry to land: not a solved problem, but one that’s been pushed far enough into the background that most applications never notice it.