Every distributed cache, database, and load balancer eventually runs into the same problem: given a piece of data and a set of servers, which server should own it. The obvious answer is to hash the key and take the result modulo the number of servers. It works fine right up until you add or remove a server, at which point almost every key maps to a different machine. For a cache, that means a wave of misses. For a database, it means a massive, disruptive data migration. Consistent hashing exists to avoid exactly that.

The Modulo Problem

Say you have four cache servers and you distribute keys with hash(key) % 4. This spreads load evenly, but the moment you go from four servers to five, the modulo changes for nearly every key. A key that used to land on server 2 might now land on server 4. Since none of the servers already have that data in the new spot, you get a near-total cache invalidation. In a database context, this is even worse: it means physically moving most of your data around the cluster just because you added one node.

Consistent hashing was designed specifically to minimize this churn. The core promise is that when you add or remove one node out of N, only about 1/N of the keys need to move. Everything else stays exactly where it was.

How the Ring Works

The classic implementation maps both servers and keys onto a circular hash space, usually visualized as a ring running from 0 to some large maximum value (like the output range of a hash function such as MD5 or SHA-1). Each server is hashed to one or more points on this ring. Each key is also hashed onto the ring, and it belongs to the first server found by walking clockwise from the key’s position.

Adding a server means placing a new point on the ring. Only the keys that fall between the new server’s position and the previous server going counterclockwise get reassigned; everyone else is untouched. Removing a server has the same limited blast radius: its keys simply flow to the next server clockwise, and nothing else moves.

Virtual Nodes Solve the Uneven Distribution Problem

A naive version of this idea has a flaw: with only one point per server on the ring, load distribution can be lumpy. Some servers end up owning much larger arcs of the ring than others, especially with a small number of servers. The standard fix is virtual nodes, where each physical server is hashed to many points on the ring (dozens or hundreds) instead of one. This smooths out the distribution and also makes rebalancing more graceful when a server is added or removed, since its share of keys is spread across many small ranges rather than one large one.

Where This Shows Up in Practice

Consistent hashing (or close variants of it) underlies distributed caches like memcached client libraries, partitioning schemes in distributed databases and key-value stores, and request routing in load balancers and CDNs. It’s also foundational to peer-to-peer systems that need to locate data without a central directory. Some systems use a related but distinct approach called rendezvous hashing, which achieves similar minimal-disruption properties through a different mechanism: each server computes a hash combined with the key, and the key goes to whichever server produces the highest score. Both approaches solve the same underlying problem with different tradeoffs around lookup complexity and implementation simplicity.

Why It Still Matters

None of this is new technology. The core idea dates back decades and has been well understood since the early days of large-scale web caching. But it remains one of those foundational techniques that shows up quietly inside almost every horizontally scaled system, whether or not the engineers using it think about it explicitly. Anytime a system needs to add capacity without a full data reshuffle, some form of consistent hashing is probably doing the work behind the scenes. Understanding it makes it much easier to reason about why certain distributed systems handle scaling events gracefully while others require painful, all-hands migrations.