Deploying new code or scaling down infrastructure shouldn’t drop active requests. Yet without careful orchestration, that’s exactly what happens when a server shuts down while still handling traffic. Connection draining solves this problem by giving in-flight requests time to complete before a server exits.

The Problem: Abrupt Termination

When a load balancer routes traffic to a pool of backend servers, each server maintains active connections handling requests. If you kill a server immediately—whether for a deployment, autoscale-down event, or instance replacement—those connections terminate mid-request. Users see timeout errors, partial responses, or cryptic network failures.

The naive approach is to simply stop accepting new connections and hope everything finishes quickly. But HTTP requests can take seconds or minutes to complete, especially for large uploads, streaming responses, or long-running API calls. A hard shutdown creates a race condition between request completion and process termination.

How Connection Draining Works

Connection draining coordinates three components: the load balancer, the application server, and the orchestration layer (like Kubernetes or an autoscaler).

When a server needs to shut down, it first deregisters from the load balancer’s healthy pool. The load balancer stops sending new connections to that server but preserves existing ones. The server continues processing active requests while rejecting any stragglers that arrive during the deregistration propagation window.

After a configured timeout—typically 30 to 300 seconds depending on workload characteristics—the server force-closes any remaining connections and exits. This timeout balances graceful completion against the need to actually finish the shutdown process.

Implementation Patterns

Most production systems implement connection draining at multiple layers. Cloud load balancers like AWS ALB and Google Cloud Load Balancing offer built-in draining with configurable timeouts. When you deregister a target, the load balancer marks it as “draining” and gives it time to finish.

At the application layer, servers need to handle shutdown signals properly. Instead of exiting immediately on SIGTERM, the process should stop accepting new connections, wait for active requests to complete (up to a deadline), then exit cleanly. In Node.js, this means closing the HTTP server and waiting for the close event. In Go, it’s server.Shutdown(ctx) with a timeout context.

Container orchestrators add another layer. Kubernetes sends SIGTERM, waits for a grace period (default 30 seconds), then sends SIGKILL. The pod’s preStop hook can coordinate with external systems—deregistering from a service mesh control plane or external load balancer before the application shutdown even begins.

Edge Cases and Gotchas

WebSocket and long-polling connections complicate draining. A WebSocket might stay open for hours, far exceeding any reasonable drain timeout. The solution is usually application-level coordination: the server sends a “closing” message over the WebSocket, the client reconnects to a different instance, then the old connection closes.

Propagation delay is another subtle issue. When a server deregisters from a load balancer, that change takes time to propagate—DNS updates, health check intervals, distributed cache invalidation. During this window, new requests can still arrive at a draining server. The server must handle them gracefully, either by processing them (if the drain just started) or returning a retriable error like 503 Service Unavailable.

Badly behaved clients that don’t respect connection timeouts can hold a server hostage. That’s why a hard deadline is essential—after the drain period expires, connections must be severed even if requests haven’t completed.

Why It Matters

Connection draining is invisible when it works and catastrophic when it doesn’t. A well-implemented drain process means deployments happen continuously without user impact. A missing or misconfigured drain means every deployment drops a percentage of requests, creating error spikes correlated with releases.

As systems move toward more frequent deployments and aggressive autoscaling, graceful shutdown becomes a reliability requirement, not an optimization. The complexity is manageable, but it requires coordination across the stack—from the load balancer down to signal handlers in application code.