A user clicks “like” on a post. The application reads the current count, increments it, and writes it back. Simple enough. But when two users click at exactly the same time, one of those likes disappears. This is the read-modify-write problem, and it’s one of the most common concurrency bugs in distributed systems.

The Lost Update Pattern

A read-modify-write cycle breaks down into three distinct steps: fetch the current value, compute a new value based on it, and store the result. The problem emerges when multiple processes execute these steps concurrently against the same data.

Thread A reads a counter with value 10. Thread B also reads the counter, still seeing 10. Thread A increments to 11 and writes it back. Thread B increments its copy to 11 and writes it back. The final value is 11, not 12. One increment vanished.

This pattern appears everywhere: inventory systems decrementing stock, bank accounts tracking balances, analytics dashboards aggregating metrics, distributed counters in Redis or Memcached, and even filesystem operations that check-then-act. Any time the new value depends on the old value, you have a read-modify-write dependency.

Why Locks Aren’t Enough

The intuitive fix is locking: acquire a mutex, read the value, modify it, write it back, release the mutex. This works within a single process, but breaks down in distributed systems where state lives in a remote database or cache.

Consider an API that increments a counter in Postgres. Even if application code uses locks, two separate API instances can both read the same row, increment their local copies, and write back conflicting values. The database sees two independent transactions, each internally consistent, but the final result is still wrong.

Database-level row locks help, but they require careful use of SELECT FOR UPDATE and introduce contention. In high-throughput systems, pessimistic locking becomes a bottleneck. In distributed caches like Redis or Memcached, traditional locks don’t exist at all.

Atomic Operations as the Solution

The fundamental fix is to collapse the read-modify-write cycle into a single atomic operation that the storage layer executes indivisibly. Instead of fetching a value, computing locally, and writing back, you send an instruction: “increment this counter by 1.”

Redis provides INCR and HINCRBY. DynamoDB offers UpdateItem with atomic increments. Postgres supports UPDATE counters SET value = value + 1. MongoDB has $inc. These operations guarantee that the read, modify, and write happen as one indivisible step, eliminating the race condition entirely.

The key property is that atomicity happens at the storage layer, not in application code. The system itself ensures no interleaving. Two concurrent INCR commands will serialize correctly, and both increments will apply.

Compare-and-Swap and Optimistic Locking

When atomic primitives aren’t available, compare-and-swap (CAS) provides an alternative. The client reads a value along with a version number or timestamp. It computes the new value, then attempts to write it back with a condition: “only update if the version is still X.” If another client modified the value in the meantime, the version changed, and the write fails. The client must retry the entire cycle.

This is optimistic concurrency control. Instead of locking upfront, you assume no conflict and detect collisions after the fact. Databases implement this with version columns or WHERE clauses that match the original value. HTTP supports it via ETag and If-Match headers. CAS works well when conflicts are rare, but high contention forces many retries.

DynamoDB’s conditional writes, Cassandra’s lightweight transactions, and etcd’s transaction API all expose CAS semantics. Application code must handle retries and backoff, adding complexity compared to native atomic operations.

Implications for System Design

Recognizing read-modify-write cycles changes how you build systems. Instead of fetching entities, mutating them in memory, and saving them back, you send transformation commands to the storage layer. Instead of GET /counter, increment locally, then PUT /counter, you expose POST /counter/increment.

Event-sourcing architectures avoid the problem entirely by never updating in place. Instead of modifying a balance, you append a transaction event. The balance becomes a derived view, computed by replaying events. There’s no read-modify-write cycle because there’s no update.

Stateless services pair well with atomic operations. Rather than holding locks or coordinating across instances, each request independently issues atomic commands to the data layer. The storage system serializes conflicting operations, and the application stays simple.

When atomic operations aren’t available, you must choose between pessimistic locking (safe but slow) and optimistic concurrency (fast but complex). The choice depends on contention: low contention favors optimism, high contention favors locking or redesigning around atomic primitives.

Read-modify-write cycles are subtle, common, and dangerous. The fix isn’t more sophisticated application logic. It’s pushing the operation to the layer that owns the data and ensuring it executes atomically.