Photo by Marvin Meyer on Unsplash
String Interning: Deduplicating Immutable Objects in Memory
When your application creates thousands of string objects containing the same text, you’re burning memory on redundant copies. String interning solves this by maintaining a canonical pool of unique values, replacing duplicates with references to a single shared instance.
The technique applies broadly to any immutable object, but strings are the most common case. Every time you parse JSON, read configuration files, or process structured data, you encounter repeated string values: status codes, field names, enum-like constants, common user inputs. Without interning, each occurrence allocates separate memory.
How Interning Works
An intern pool is essentially a hash table mapping values to their canonical instances. When you intern a string, the runtime checks if an equivalent value already exists in the pool. If so, it returns the existing reference. If not, it adds the string to the pool and returns it.
# Conceptual implementation
intern_pool = {}
def intern(s):
if s not in intern_pool:
intern_pool[s] = s
return intern_pool[s]
The pool transforms value equality into reference equality. Two interned strings with identical content are literally the same object in memory, so you can compare them with a pointer check instead of character-by-character comparison.
Language Runtime Strategies
Many languages intern string literals automatically at compile time. When you write "error" in multiple places in your source code, the compiler typically allocates a single string object that all references share. This is safe because literals are immutable by definition.
Java interns all string literals and provides String.intern() for runtime interning. The JVM maintains a global pool, which historically caused memory pressure because interned strings lived in a fixed-size PermGen space. Modern JVMs moved the pool to the regular heap with better garbage collection integration.
Python interns short strings and identifiers automatically. Strings that look like valid identifiers get interned at compile time, while others require explicit sys.intern() calls. Python’s approach is opportunistic: intern common cases automatically, let developers opt in for the rest.
.NET’s String.Intern() provides similar functionality, with the intern pool surviving garbage collection by design. Once a string enters the pool, it typically stays for the lifetime of the AppDomain.
Memory vs Speed Tradeoffs
Interning trades memory for CPU during the insertion check. Each intern operation requires a hash computation and lookup. For strings used only once, you pay this cost without benefit. The win comes when you have many duplicates.
The intern pool itself consumes memory for its hash table structure. A pool with millions of entries needs significant overhead for buckets, collision chains, or open addressing gaps. You’re betting that deduplicating the strings saves more memory than the pool costs.
Garbage collection behavior changes with interning. Pooled objects stay reachable as long as the pool exists, preventing collection even if application code no longer needs them. Some implementations use weak references in the pool to allow collection when only the pool holds a reference, but this adds complexity.
When to Intern
Intern when you have many copies of relatively few distinct values. Parsing formats with repeated keys (JSON, XML, CSV headers) is ideal. Database column names, HTTP header names, and configuration keys all benefit.
Don’t intern user-generated content with high cardinality. Interning unique strings from a million users just fills the pool with entries you’ll never reuse, wasting memory on pool overhead while preventing garbage collection.
Watch for memory leaks in long-running processes. If your pool grows unbounded with occasional unique values, you’re creating a memory sink. Some applications use LRU or size-bounded pools to limit growth, though this reduces the deduplication benefit.
Implementation Considerations
Thread safety matters for shared pools. Concurrent intern operations need synchronization, which adds contention. Some runtimes use concurrent hash maps or partition the pool into segments to reduce lock contention.
Hashing performance affects intern cost. Cryptographic hashes are overkill; you need fast, collision-resistant hashing for short strings. Many implementations cache hash codes in string objects to avoid recomputation.
Interning shows up beyond strings. Flyweight patterns for configuration objects, parsed URLs, or small immutable data structures use the same principle. The key requirement is immutability: shared objects must never change, or you violate expectations about independent instances.
The technique is simple but powerful when applied correctly. Profile your memory usage to identify duplication, measure the cardinality of your string values, then decide if interning fits your specific workload.