Kubernetes workloads face a fundamental tradeoff: provision too many replicas and you waste resources; provision too few and users experience degraded performance or outages. Horizontal Pod Autoscaling (HPA) resolves this tension by dynamically adjusting the number of pod replicas based on observed metrics, letting clusters respond to load in real time.

How HPA Works

The HPA controller runs as a control loop in the Kubernetes control plane, periodically querying metrics and comparing them against target values. Every 15 seconds by default, it examines resource utilization—CPU, memory, or custom metrics exposed via the Metrics Server or an adapter—and calculates the desired replica count using a simple proportional formula:

desiredReplicas = ceil(currentReplicas × (currentMetricValue / targetMetricValue))

If average CPU utilization across all pods is 80% and the target is 50%, HPA scales out. If utilization drops to 30%, it scales in. The controller respects minimum and maximum replica bounds set in the HPA resource definition, preventing runaway scaling or complete scale-down.

Metric Sources and Custom Scaling

Out of the box, HPA supports CPU and memory utilization via the Metrics Server, a lightweight aggregator of kubelet resource metrics. For production workloads, teams often need more nuanced signals: request rate, queue depth, database connection pool saturation, or business metrics like active sessions.

Custom metrics are exposed through the Custom Metrics API or External Metrics API, typically backed by Prometheus, Datadog, or other observability platforms. An HPA resource can target any metric that reflects load, not just raw resource consumption. For example, a web service might scale on HTTP requests per second, while a batch processor scales on the number of unprocessed jobs in a queue.

Scaling Behavior and Stability

HPA includes built-in dampening to prevent thrashing. Scale-up decisions execute quickly to handle traffic spikes, but scale-down is more conservative—by default, the controller waits five minutes before removing replicas. This asymmetry prevents rapid oscillations when load fluctuates near the threshold.

The behavior field in HPA v2 allows fine-grained control over scaling velocity. You can specify policies like “scale up by at most 100% every 60 seconds” or “scale down by at most 10 pods every 5 minutes,” tuning responsiveness to match workload characteristics. Fast-scaling policies suit bursty traffic; slower ramp-down protects against premature capacity reduction during temporary lulls.

Limitations and Tradeoffs

HPA reacts to load but cannot predict it. For workloads with known traffic patterns—daily peaks, scheduled batch jobs—combining HPA with scheduled scaling or predictive autoscaling tools yields better results. Cold-start latency also matters: if pods take 30 seconds to become ready, HPA cannot save an already-overwhelmed service.

Resource requests must be set accurately. HPA calculates CPU percentage relative to the pod’s requested CPU, not actual capacity. If requests are too low, HPA scales prematurely; if too high, it underscales. Right-sizing requests is foundational work that HPA builds upon, not replaces.

HPA operates at the pod level and does not coordinate with cluster autoscaling. If HPA needs more replicas but the cluster lacks available nodes, pods remain pending until the cluster autoscaler provisions additional capacity—a process that can take minutes. Tight coordination between HPA and cluster autoscaling, along with buffer capacity or overprovisioning, mitigates this gap.

When to Use HPA

Horizontal Pod Autoscaling shines for stateless services and parallel workloads where adding replicas directly increases capacity: web frontends, API servers, worker pools, and stream processors. It fits poorly for stateful systems—databases, caches, consensus groups—where adding replicas requires coordination, state redistribution, or rebalancing.

Used correctly, HPA turns fixed-capacity clusters into elastic infrastructure that adapts to actual demand, reducing waste during off-peak hours and maintaining headroom during spikes. It is not a silver bullet, but it is a fundamental building block of cost-effective, resilient Kubernetes operations.