Connection Pool Sizing: Why More Isn't Always Better
Database connection pools are everywhere in backend systems, but most applications get the sizing wrong. The default configuration is often far too large, leading to worse performance than a smaller pool would deliver. The reason comes down to queueing theory and how databases handle concurrent work.
The Conventional Wisdom Is Wrong
Many applications use connection pool sizes of 50, 100, or even 200 connections. The logic seems sound: more connections mean more parallelism, which should mean better throughput. But databases don’t work that way.
A database server has finite resources: CPU cores, disk I/O bandwidth, and memory. When too many concurrent queries compete for these resources, context switching overhead increases, disk seeks become randomized instead of sequential, and cache hit rates drop. The database spends more time coordinating work than actually executing it.
The empirical evidence is consistent across PostgreSQL, MySQL, and other relational databases: performance peaks at a relatively small number of active connections, typically somewhere between 5 and 20, depending on the workload and hardware. Beyond that point, adding more connections increases latency and decreases throughput.
Queueing Theory Provides the Framework
The mathematical model that explains this behavior is queueing theory, specifically the relationship between utilization, queue depth, and latency. As a system approaches full utilization, queue depth and wait times increase non-linearly.
A connection pool acts as a throttle. When sized correctly, it creates a queue at the application tier where requests wait for an available connection. This prevents the database from becoming overloaded. When sized too large, the queue moves to the database itself, where requests wait for CPU, disk I/O, or locks—but with all the overhead of an established connection consuming memory and appearing in process lists.
The key insight is that you want the queue in your application, not in your database. Application-side queueing is cheaper and more observable. You can monitor queue depth, set timeouts, and shed load gracefully. Database-side queueing manifests as degraded performance that’s harder to diagnose.
Calculating an Appropriate Size
A widely cited formula comes from the PostgreSQL community: connections = ((core_count * 2) + effective_spindle_count). For a modern SSD-backed database server with 8 cores, this suggests around 17 connections.
This formula provides a reasonable starting point, but the optimal size depends on your specific workload characteristics. OLTP workloads with short queries may benefit from slightly higher connection counts. Analytical workloads with long-running queries need fewer connections to avoid resource contention.
The important principle is that the pool size should reflect the database’s capacity for concurrent work, not the application’s demand. If your application has 1000 concurrent users, you don’t need 1000 database connections. You need enough connections to keep the database busy without overwhelming it, and you need queueing to manage the gap.
Connection Lifetime and Warm-up Costs
Another consideration is the cost of connection establishment and teardown. Opening a database connection involves TCP handshakes, TLS negotiation, authentication, and session initialization. These operations can take tens of milliseconds.
Connection pools amortize this cost by keeping connections open and reusing them across requests. But oversized pools waste memory on both the application and database side. Each idle connection consumes kernel resources, database process memory, and entry space in connection tracking structures.
Some connection pool implementations support dynamic sizing, growing and shrinking based on demand. This sounds attractive but introduces complexity. The pool needs to distinguish between temporary spikes and sustained load increases, and it needs hysteresis to prevent oscillation. Static sizing based on database capacity is often simpler and more predictable.
Observability Matters More Than Tuning
The most important aspect of connection pool configuration isn’t getting the exact number right—it’s having visibility into pool behavior. You need metrics for:
- Active connections (currently executing queries)
- Idle connections (available in the pool)
- Waiting requests (queued at the application tier)
- Connection acquisition time (how long requests wait for a connection)
- Connection usage distribution (whether connections are evenly utilized)
These metrics tell you whether your pool is sized appropriately. If connections are always maxed out and queue depth is growing, you may need a larger pool—or you may have a slow query problem. If most connections sit idle, you’ve oversized the pool.
The correct pool size depends on your database’s hardware, your query patterns, and your performance requirements. But the upper bound is much lower than most developers assume, and understanding the queueing dynamics helps you tune for reality instead of intuition.