PERFORMANCE Signal 494
Array-backed LRU hash table eliminates runtime allocations and global lock contention
The table uses pre-allocated arrays, sharded TTAS spinlocks and NUMA-aware layout to provide a concurrent LRU cache that avoids heap allocations and lock contention.
By removing global locks and per-operation allocations, the design reduces tail latency and improves throughput on many-core systems. This makes it attractive for latency-sensitive layers such as caching, network routing, storage subsystems and kernel components where standard containers become a bottleneck.
Written by elseif from the cluster below · every claim links back to a sourceThe three things worth knowing
Zero runtime allocations are achieved by storing hash buckets and LRU nodes in pre-allocated flat arrays, eliminating heap fragmentation and OS lock stalls.
Each shard owns an exclusive TTAS spinlock, allowing independent operation and linear scaling with the number of physical CPU cores.
NUMA-aware placement of shards and lock-free destruction of payloads preserve cache locality and keep eviction latency predictable.
THE READ
What the cluster adds up to.
The implementation replaces the typical std::unordered_map plus std::list guarded by a global std::shared_mutex with a design that partitions data into independent shards. Each shard holds its own TTAS spinlock, a contiguous mega-block of arrays for buckets and nodes, and uses 32-bit indices instead of pointers. This layout removes global lock contention and pointer chasing, improving cache locality. The table also adds NUMA-aware distribution of shards and lock-free destruction of payloads.
Adopting the table requires pre-allocating the mega-block size based on the expected maximum entries and choosing a shard count that matches the hardware core count. Tuning the lazy LRU promotion safe zone and providing a custom allocator (if needed) adds configuration overhead. The code must be compiled with support for TTAS spinlocks in user-mode or rely on the Windows EX_PUSH_LOCK when used in kernel mode. Alignment to cache-line boundaries is necessary to avoid false sharing.
The approach is less suitable when the workload cannot predict a fixed upper bound on the number of cached items, because the table does not grow beyond its pre-allocated capacity. Environments without NUMA topology or with a single socket see limited benefit from the NUMA-aware placement. In low-core-count scenarios the extra sharding and synchronization logic may introduce overhead that outweighs gains. Finally, if the application already meets its performance goals with standard library containers, the added complexity may not be justified.
Written by elseif from the cluster below · checked for specifics the sources never containedTHE CLUSTER