Compute-Storage Disaggregation: How Cloud Databases Are Rethinking Their Foundations

Traditional databases co-locate compute and storage on the same machine. The query engine, transaction manager, and the actual data pages all live together. This is efficient — reads and writes hit local disk rather than a network — but it creates rigid coupling that doesn’t fit cloud infrastructure well. To scale storage, you scale compute, and vice versa.

Disaggregated storage breaks that coupling. Compute nodes don’t own data; they read from and write to a shared storage layer. Compute becomes stateless or nearly stateless, which changes what failure, scaling, and cost look like in fundamental ways.

How the Pattern Plays Out by Workload

Disaggregation isn’t a single design — it manifests differently depending on access patterns.

Analytics. Snowflake’s architecture is the canonical example here. Query compute is entirely decoupled from data stored in object storage. You can run multiple independent compute clusters against the same dataset simultaneously, or shut down compute entirely between queries and pay nothing. For workloads that are bursty or scheduled rather than continuous, this maps directly to cost savings.

Transactional databases. Aurora takes a more surgical approach. Rather than putting everything on object storage, it separates the MySQL or PostgreSQL compute layer from a purpose-built distributed storage system. Critically, Aurora replicates just write-ahead log records across the network to storage nodes, rather than full dirty pages. The storage layer itself reconstructs pages from log records. This reduces network I/O significantly and lets the storage system handle replication and durability without coordinating extensively with the compute layer above it.

Streaming. Kafka’s tiered storage offloads older log segments to object storage while keeping recent data on local broker disks. Consumers catching up on historical data read from object storage; real-time consumers read from the local tier. This lets operators retain months of event history without provisioning broker capacity proportionally.

Why Object Storage Made This Viable

The whole pattern depends on a property that became true in the cloud era: object storage is cheap, durable, and scales to arbitrary capacity without operational work. S3-compatible stores offer high aggregate throughput for large sequential reads, which maps well onto analytics and log-oriented workloads.

The architectural bet is that you can absorb object store latency — milliseconds rather than microseconds — through caching, batching, and careful I/O layout. For analytics scanning gigabytes of columnar data, this works cleanly. For transactional workloads, you need a buffer: either a local NVMe hot tier that absorbs recent writes before they land in the shared layer, or storage nodes placed physically close to compute to keep round-trip times low.

What You Give Up

The tradeoffs are real and worth being direct about.

Latency. A cache miss on object storage costs milliseconds. On local NVMe it costs microseconds. Warm-path performance depends heavily on cache hit rates, which means cold starts after idle periods are genuinely painful. Serverless databases typically hedge this with pre-warmed compute pools, but the problem doesn’t disappear.

Storage layer complexity. In a traditional single-node database, “the storage layer” is a disk and a filesystem. In a disaggregated system, it’s a distributed system with its own replication protocol, consistency guarantees, and failure modes. Aurora’s storage service is a substantial engineering artifact. As a managed service you inherit it for free; building and operating an equivalent yourself is a significant undertaking.

Write durability latency. Writes typically need to reach the shared storage layer before they’re acknowledged as durable. How quickly that round trip completes — and how many nodes must confirm — directly determines your commit latency. Tuning this is a first-order concern in any disaggregated OLTP design.

Why It’s Becoming the Default

Despite those costs, the economics are compelling. Compute and storage have different scaling curves, different pricing, and different failure modes. Overprovisioning compute to co-locate with large datasets wastes money. The ability to scale them independently — scale out compute for a heavy reporting job, then release it — better matches how cloud workloads actually behave.

There’s also an operational argument: pushing durability responsibility into a dedicated storage layer lets the database compute layer treat crashes as simple, fast recovery rather than a distributed coordination problem. Failover that previously took minutes can happen in seconds when compute is stateless and storage is always available.

Disaggregation won’t replace every database architecture. Workloads demanding microsecond write latency still benefit from tight co-location with fast local storage. But for a growing share of cloud database workloads, the question is no longer whether to separate compute and storage — it’s how far to push the separation and where to place the caches.