Every operation carries overhead. A database write incurs network round-trip costs, fsync penalties, and lock acquisition. A disk I/O pays for seek time and command dispatch. An API call absorbs connection setup, TLS handshake, and serialization tax. When these fixed costs dominate, throughput collapses and tail latency climbs.

Batching and coalescing solve this by spreading fixed overhead across many operations. Instead of processing one item at a time, systems group multiple requests together and handle them in a single pass. The result: higher throughput, lower per-operation latency, and more efficient use of underlying resources.

Batching: Grouping Requests at the Application Layer

Batching explicitly collects multiple operations and submits them as a group. Database drivers batch inserts into multi-row statements. Message queues group acknowledgments. Network stacks combine small packets into larger frames.

The fixed cost is paid once per batch rather than once per item. A database that commits 100 individual transactions pays 100 fsync calls; batching those into a single transaction reduces it to one. The same logic applies to RPC calls, disk writes, and GPU kernel launches.

Batching introduces latency for early arrivals. The first request in a batch must wait for the batch window to close before processing begins. This delay is the core tradeoff: higher throughput at the cost of increased per-item latency. Adaptive batch sizes and timeout-based flush policies mitigate this by balancing wait time against batching efficiency.

Coalescing: Merging Redundant Operations

Coalescing identifies duplicate or overlapping operations and merges them. If three goroutines request the same cache key simultaneously, coalescing executes one fetch and shares the result. If multiple threads dirty the same memory page, the OS coalesces them into a single writeback.

Unlike batching, which groups distinct operations, coalescing eliminates redundancy. The result is both reduced work and lower resource contention. HTTP caches use request coalescing to avoid stampedes when many clients request the same cold resource. File systems coalesce adjacent writes to reduce fragmentation and I/O overhead.

Coalescing requires detecting equivalence. Cache keys and memory addresses are straightforward, but more complex operations need careful deduplication logic. Some systems hash request parameters; others maintain in-flight operation registries to detect overlaps.

Where Batching and Coalescing Appear

Database commit logs batch writes to amortize fsync overhead. Group commit in MySQL and PostgreSQL collects concurrent transactions and flushes them together, dramatically improving write throughput under high concurrency.

Network interfaces use interrupt coalescing to reduce CPU wakeups. Instead of triggering an interrupt for every received packet, NICs accumulate packets and fire a single interrupt per batch. This trades microseconds of latency for significant CPU efficiency gains.

Graphics APIs batch draw calls. Submitting thousands of individual draw commands creates severe CPU-GPU synchronization overhead. Modern APIs like Vulkan and Metal batch commands into command buffers, allowing the GPU to execute many operations without round-tripping to the CPU.

Event-driven systems batch and coalesce UI updates. React batches state updates within a single event loop tick, avoiding redundant re-renders. Browser engines coalesce layout recalculations and repaints, processing accumulated DOM changes in a single pass.

Distributed systems use Nagle’s algorithm to batch small TCP segments, trading latency for reduced packet overhead. Cloud providers batch API calls to reduce rate-limit pressure and improve cost efficiency.

Implementation Patterns

Time-based batching flushes when a timeout expires, guaranteeing bounded latency. Size-based batching flushes when the batch reaches a threshold, prioritizing throughput. Hybrid strategies combine both: flush on timeout or size, whichever comes first.

Coalescing typically maintains a registry of in-flight operations keyed by request signature. When a duplicate request arrives, the system attaches the new caller to the existing operation rather than launching a duplicate.

Backpressure management becomes critical in batching systems. If batches fill faster than they can be processed, queues grow unbounded. Systems must either apply backpressure to producers or implement adaptive flush policies that favor latency when load increases.

The Tradeoffs

Batching increases latency for the first operation in a batch but improves average throughput. Systems with strict latency SLOs may prefer smaller batches or disable batching entirely for latency-sensitive paths while using it for bulk operations.

Coalescing reduces redundant work but adds complexity. Maintaining deduplication state and correctly sharing results across callers introduces coordination overhead. For low-duplication workloads, the cost may exceed the benefit.

Both techniques amortize fixed costs, making them most valuable when overhead dominates payload processing. As operation costs drop or payloads grow, batching and coalescing deliver diminishing returns.

Modern infrastructure relies on batching and coalescing to achieve efficiency at scale. Understanding when and how to apply them is essential for building systems that handle high-throughput workloads without sacrificing observability or correctness.