Hash-Table on Opensource internals

ClickHouse Splits GROUP BY into 256 Buckets for Parallel Merges

Jay Hubbert — Wed, 21 Oct 2026 09:00:00 +0000

TL;DR: A GROUP BY needs a serial merge at the end — and that’s where the memory high-water mark lives. ClickHouse splits the key space into 256 buckets by the top bits of the hash, so the merge becomes 256 independent, lock-free, parallel jobs.

A GROUP BY in ClickHouse is usually computed by several threads at once, each with its own hash table, and the tables are then merged.

Merging N hash tables of M entries each is not something you want to do on one thread. It’s a serial phase in an otherwise parallel query, and it’s the phase where the memory high-water mark is reached.

The way out is to make the merge divisible. If the keys are partitioned into a fixed number of buckets by a deterministic function of the key, then two tables can be merged bucket by bucket, and different buckets can be merged by different threads with no coordination at all: bucket 17 of table A can only collide with bucket 17 of table B.

That’s what TwoLevelHashTable in src/Common/HashTable/TwoLevelHashTable.h does, and it’s what the group_by_two_level_threshold setting controls.

The structure

The two-level table is literally an array of ordinary hash tables:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:9
/** Two-level hash table.
 * Represents 256 (or 1ULL << BITS_FOR_BUCKET) small hash tables (buckets of the first level).
 * To determine which one to use, one of the bytes of the hash function is taken.
 *
 * Usually works a little slower than a simple hash table.
 * However, it has advantages in some cases:
 * - if you need to merge two hash tables together, then you can easily parallelize it by buckets;
 * - delay during resizes is amortized, since the small hash tables will be resized separately;
 * - in theory, resizes are cache-local in a larger range of sizes.
 *
 * With `BITS_FOR_BUCKET = 0` there is a single bucket: routing folds to a constant.
 * Lookups and inserts compile down to what the single-level table does.
 *
 * Buckets share no state.
 * Threads may fill different buckets at the same time when every bucket is written under its own lock.
 */

The bucket index comes out of the hash:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:58
 static constexpr UInt32 NUM_BUCKETS = 1ULL << BITS_FOR_BUCKET;
 static constexpr UInt32 MAX_BUCKET = NUM_BUCKETS - 1;

 size_t hash(const Key & x) const { return Hash::operator()(x); }

 /// NOTE Bad for hash tables with more than 2^32 cells.
 static size_t getBucketFromHash(size_t hash_value) { return (hash_value >> (32 - BITS_FOR_BUCKET)) & MAX_BUCKET; }

BITS_FOR_BUCKET defaults to 8, so 256 buckets, and the bucket is taken from the high bytes of the 64-bit hash:

 text
 
 
 64-bit hash
 |<-------------------------->|
 | bucket | sub-table index |
 | 8 bits | (mask & (size-1)) |
 +-----------+---------------------+
 | 0..255 | depends on the bucket's own size |
 +-----------+---------------------+

 hash >> 24 & 255 -> which of the 256 sub-tables
 hash & (2^k - 1) -> which cell inside that sub-table

The choice of the high bits is not arbitrary, and it’s the subtle part of the whole design. The single-level table uses the low bits of the hash to index its cells:

 cpp
 
 
// src/Common/HashTable/HashTable.h:238
 /// From the hash value, get the cell number in the hash table.
 size_t place(size_t x) const { return x & mask(); }

If the bucket were also selected from the low bits, the two uses would collide. Every bucket would receive keys whose low bits are a fixed value, and the sub-table’s own probe would be confined to a single cell. The sub-tables would degenerate into 256 linked lists.

Taking the bucket from the top of the hash and the cell from the bottom keeps the two independent — which is exactly what a good hash function provides.

The default bucket count is 8 bits, and the comment “Bad for hash tables with more than 2^32 cells” is about the other direction: the shift hash_value >> (32 - BITS_FOR_BUCKET) throws away everything above bit 32, so the bucket selection only sees 32 bits of the hash. For a table with more than 2^32 cells, the top bits would carry real information and throwing them away would lose it.

The sub-tables are ordinary tables

Each bucket is a plain HashTable, so all the machinery from the previous article — zeroed cells, linear probing, Robin Hood ordering — applies inside a bucket. The two-level table only adds the routing:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:316
 template <typename KeyHolder>
 void ALWAYS_INLINE emplace(KeyHolder && key_holder, LookupResult & it, bool & inserted, size_t hash_value)
 {
 size_t buck = getBucketFromHash(hash_value);
 impls[buck].emplace(key_holder, it, inserted, hash_value);
 }

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:333
 LookupResult ALWAYS_INLINE find(Key x, size_t hash_value)
 {
 size_t buck = getBucketFromHash(hash_value);
 return impls[buck].find(x, hash_value);
 }

and the storage is a plain array of sub-tables:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:100
 Impl impls[NUM_BUCKETS];

 /// `bucket_cells_prefix[b]` is the number of cells in the buckets before `b`. See `computeBucketPrefix`.
 std::vector<size_t> bucket_cells_prefix;

Impl impls[NUM_BUCKETS] is 256 sub-tables by value, not by pointer. Each sub-table owns its buffer, so the whole structure is 256 small allocations plus the bookkeeping of each.

How a bucket grows

The buckets do grow, but they grow in a different shape from the single-level table. The grower used by the two-level table caps the quadratic-growth phase at 2^15 cells and switches to the Robin Hood mode above that:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:26
/// The buckets grow by 4 times while they are small, and switch to the Robin Hood mode when they become large,
/// see `HashTableGrowerWithPrecalculationAndRobinHood`.
template <size_t initial_size_degree = 8, size_t robin_hood_size_degree = 16>
using TwoLevelHashTableGrower = HashTableGrowerWithPrecalculationAndRobinHood<initial_size_degree, 15, robin_hood_size_degree>;

The template parameters are initial_size_degree = 8, max_size_degree_quadratic = 15, robin_hood_size_degree = 16. So:

up to 2^15 cells per bucket, the sub-table quadruples its size on overflow (a degree increment of two);
from 2^15 to 2^16 it doubles;
above 2^16 cells per bucket it switches to the Robin Hood mode with a 90% load factor, and from then on it keeps doubling like any other table — but at 90% load instead of 50%.

The effect of the switch is memory, not capacity. A bucket that grows large stops paying the factor-of-two penalty of the 0.5 load factor, because the same number of keys now needs half as many cells.

That’s also what makes the external-memory path possible: when the aggregation spills to disk, it spills bucket by bucket, and each bucket’s data is self-contained.

Resizing is amortised

A single-level table resizes by re-inserting every element. At the moment of the resize, the whole table has to be walked and every cell re-placed.

In a two-level table, each bucket resizes on its own, and since the buckets are filled roughly uniformly, the individual resizes happen at different times. The total work is the same, but it’s spread out, and — crucially — the pause is per-bucket rather than table-wide.

The comment in the header says it exactly:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:14
 * - delay during resizes is amortized, since the small hash tables will be resized separately;

When is the two-level table used?

The decision is a setting, and it’s a pair of settings now — one counting keys, one counting bytes:

 cpp
 
 
// src/Core/Settings.cpp:1500
 DECLARE(UInt64, group_by_two_level_threshold, 100000, R"(
From what number of keys, a two-level aggregation starts. 0 - the threshold is not set.
)", 0) \
 DECLARE(UInt64, group_by_two_level_threshold_bytes, 50000000, R"(
From what size of the aggregation state in bytes, a two-level aggregation begins to be used. 0 - the threshold is not set. Two-level aggregation is used when at least one of the thresholds is triggered.
)", 0) \

So by default a GROUP BY switches to the two-level structure once it holds 100 000 keys, or once its state grows to 50 MB — whichever happens first. Both thresholds can be forced from SQL:

 sql
 
 
SELECT user_id, count()
FROM events
GROUP BY user_id
SETTINGS group_by_two_level_threshold = 1000000,
 group_by_two_level_threshold_bytes = 50000000;

Setting either threshold to 0 disables that trigger — which is exactly what the pipeline does when a two-level aggregation would be useless.

The predicate that implements this lives in src/Interpreters/Aggregator.cpp:

 cpp
 
 
// src/Interpreters/Aggregator.cpp:118
bool worthConvertToTwoLevel(
 size_t group_by_two_level_threshold, size_t result_size, size_t group_by_two_level_threshold_bytes, auto result_size_bytes)
{
 // params.group_by_two_level_threshold will be equal to 0 if we have only one thread to execute aggregation (refer to AggregatingStep::transformPipeline).
 return (group_by_two_level_threshold && result_size >= group_by_two_level_threshold)
 || (group_by_two_level_threshold_bytes && result_size_bytes >= static_cast<Int64>(group_by_two_level_threshold_bytes));
}

Note the && in each disjunct: a threshold of 0 disables that particular trigger, which is why the settings can be turned off individually.

The conversion itself happens during the aggregation, when a block has been added and the table is re-examined:

 cpp
 
 
// src/Interpreters/Aggregator.cpp:2668
 bool worth_convert_to_two_level = worthConvertToTwoLevel(
 params.group_by_two_level_threshold, result_size, params.group_by_two_level_threshold_bytes, result_size_bytes);

 /** Converting to a two-level data structure.
 * It allows you to make, in the subsequent, an effective merge - either economical from memory or parallel.
 */
 if (result.isConvertibleToTwoLevel() && worth_convert_to_two_level)
 result.convertToTwoLevel();

and the conversion itself rebuilds the variant with the two-level type:

 cpp
 
 
// src/Interpreters/AggregatedDataVariants.cpp:263
void AggregatedDataVariants::convertToTwoLevel()
{
 ProfileEvents::increment(ProfileEvents::AggregationConvertedToTwoLevel);

 if (aggregator)
 LOG_TRACE(aggregator->log, "Converting aggregation data to two-level.");

 switch (type)
 {
#define M(NAME) \
 case Type::NAME: \
 NAME ## _two_level = std::make_unique<decltype(NAME ## _two_level)::element_type>(*(NAME)); \
 (NAME ## _two_level)->top_k_heap = std::move((NAME)->top_k_heap); \
 (NAME).reset(); \
 type = Type::NAME ## _two_level; \
 break;

 APPLY_FOR_VARIANTS_CONVERTIBLE_TO_TWO_LEVEL(M)

#undef M

 default:
 throw Exception(ErrorCodes::LOGICAL_ERROR, "Wrong data variant passed.");
 }
}

The TwoLevelHashTable has a constructor that takes a single-level table and distributes it, which is what makes the conversion a one-liner:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:116
 template <typename Source>
 requires(!std::is_arithmetic_v<Source>)
 explicit TwoLevelHashTable(const Source & src)
 {
 typename Source::const_iterator it = src.begin();

 /// It is assumed that the zero key (stored separately) is first in iteration order.
 if (it != src.end() && it.getPtr()->isZero(src))
 {
 insert(it->getValue());
 ++it;
 }

 for (; it != src.end(); ++it)
 {
 const Cell * cell = it.getPtr();
 size_t hash_value = cell->getHash(src);
 size_t buck = getBucketFromHash(hash_value);
 impls[buck].insertUniqueNonZero(cell, hash_value);
 }
 }

The zero key is handled explicitly, because it lives outside the buffer and doesn’t follow the bucket rule — it goes to bucket 0.

The merge is the point

Now the reason the whole thing exists. The aggregation pipeline decides whether a two-level aggregation is even allowed:

 cpp
 
 
// src/Processors/QueryPlan/AggregatingStep.cpp:503
 bool allow_to_use_two_level_group_by = pipeline.getNumStreams() > 1 || params.max_bytes_before_external_group_by != 0;

Two conditions: either there’s more than one stream (so the merge is worth parallelising), or the query is allowed to spill to disk (so the bucket structure is needed to spill incrementally). If neither holds, the thresholds are zeroed out and the aggregation stays single-level:

 cpp
 
 
// src/Processors/QueryPlan/AggregatingStep.cpp:520
 if (!allow_to_use_two_level_group_by)
 {
 params.group_by_two_level_threshold = 0;
 params.group_by_two_level_threshold_bytes = 0;
 }

When the two-level table is used, the merge stage is a source that hands out one bucket at a time, and the pipeline runs several of them in parallel:

 cpp
 
 
// src/Processors/Transforms/AggregatingTransform.cpp:208
/// Worker which merges buckets for two-level aggregation.
/// Atomically increments bucket counter and returns merged result.
class ConvertingAggregatedToChunksWithMergingSource final : public ISource
{
public:
 static constexpr UInt32 NUM_BUCKETS = 256;

The worker takes the next bucket with an atomic counter and merges it across all the per-thread tables:

 cpp
 
 
// src/Processors/Transforms/AggregatingTransform.cpp:272
 Chunk generate() override
 {
 UInt32 bucket_num = shared_data->next_bucket_to_merge.fetch_add(1);

 if (bucket_num >= NUM_BUCKETS)
 {
 data.reset();
 return {};
 }

The atomic counter is the entire coordination mechanism. No lock, no work queue, no barrier. A worker that finishes bucket 3 immediately grabs bucket 4, and the workers end up balanced by the cost of the buckets they happen to pick up. Since the buckets partition the key space, no two workers ever touch the same key.

The actual merge of one bucket, in Aggregator::mergeBucketImpl, is “for each thread’s table, merge its bucket into the first thread’s bucket”:

 cpp
 
 
// src/Interpreters/Aggregator.cpp:5200
void NO_INLINE Aggregator::mergeBucketImpl(
 ManyAggregatedDataVariants & data, Int32 bucket, Arena * arena, std::atomic<bool> & is_cancelled) const
{
 /// We merge all aggregation results to the first.
 AggregatedDataVariantsPtr & res = data[0];

 cpp
 
 
// src/Interpreters/Aggregator.cpp:5213
 auto & dst = getDataVariant<Method>(*res).data.impls[bucket];

Note that the bucket is an index into the impls[] array of the two-level table — the same array the routing used. That’s the whole contract: the bucket is a function of the key, therefore the bucket is a partition of the key space.

The same trick appears in the uniqExact aggregate function, which keeps a single-level set and converts it to a two-level one when it grows:

 cpp
 
 
// src/AggregateFunctions/UniqExactSet.h:386
 static bool worthConvertingToTwoLevel(size_t size) { return size > 100'000; }

and merges it bucket-wise in parallel:

 cpp
 
 
// src/AggregateFunctions/UniqExactSet.h:225
 auto thread_func = [&two_level_ptrs, &first_two_level, next_bucket_to_merge, &is_cancelled]()
 {
 while (true)
 {
 if (is_cancelled.load(std::memory_order_seq_cst))
 return;

 const auto bucket = next_bucket_to_merge->fetch_add(1);
 if (bucket >= NUM_BUCKETS)
 return;

 for (size_t j = 1; j < two_level_ptrs.size(); ++j)
 {
 if (is_cancelled.load(std::memory_order_seq_cst))
 return;

 first_two_level.impls[bucket].merge(two_level_ptrs[j]->impls[bucket]);
 }
 }
 };

with the same bounded-threads trick:

 cpp
 
 
// src/AggregateFunctions/UniqExactSet.h:246
 const size_t max_threads_to_enqueue = std::min<size_t>(thread_pool.getMaxThreads(), NUM_BUCKETS);

There’s no point in queueing more workers than there are buckets.

The string flavour

Strings get their own two-level table, because the string table isn’t a HashTable but a composite of five sub-tables (one per key size class). The routing therefore has to pick both the bucket and the size class:

 cpp
 
 
// src/Common/HashTable/TwoLevelStringHashTable.h:77
 /// The bucket-aware size-class dispatch: the shared implementation in `StringHashTable`
 /// classifies and packs the key and computes its hash; the resolver here picks the bucket
 /// from that hash and hands out the bucket's submap of the right class.
 template <typename Self, typename Func, typename KeyHolder>
 static auto ALWAYS_INLINE dispatch(Self & self, KeyHolder && key_holder, Func && func)
 {
 return Impl::dispatchOnKeyClass(
 [&](auto key_class, size_t hash_value) -> auto &
 { return self.impls[getBucketFromHash(hash_value)].template submapForClass<decltype(key_class)::value>(); },
 std::forward<KeyHolder>(key_holder),
 StringHashTableHash{},
 std::forward<Func>(func));
 }

256 buckets times 5 size classes is 1280 sub-tables. That’s a lot of bookkeeping for a table that usually isn’t huge, which is why the string tables are only converted to two-level when the aggregation really needs it.

Offsets across buckets

One thing the two-level table has to solve is numbering the cells globally. The aggregation code refers to cells by an offset into the whole table, and with 256 separate buffers there’s no single array to index into.

The solution is a prefix sum over the buckets:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:435
 /// Prefix sums that `offsetInternal` uses to number cells across all buckets.
 /// Call it after the last insert and before reading offsets, never while another thread reads them.
 void computeBucketPrefix()
 {
 bucket_cells_prefix.assign(NUM_BUCKETS, 0);
 size_t run = 0;
 for (UInt32 i = 0; i < NUM_BUCKETS; ++i)
 {
 bucket_cells_prefix[i] = run;
 run += impls[i].getBufferSizeInCells();
 }
 }

and the offset itself:

 cpp
 
 
// src/Common/HashTable/TwoLevelHashTable.h:448
 /// Number of the cell over all buckets. The zero cell gets 0. Any other cell gets its position in the
 /// concatenated bucket buffers plus one. So the number fits an array of `getBufferSizeInCells() + 1`.
 size_t offsetInternal(ConstLookupResult ptr) const
 {
 if constexpr (NUM_BUCKETS == 1)
 return impls[0].offsetInternal(ptr);

 const size_t buck = getBucketFromHash(ptr->getHash(*this));
 if (ptr->isZero(impls[buck]))
 return 0;

 chassert(!bucket_cells_prefix.empty(), "computeBucketPrefix must run before an offset is read");
 return bucket_cells_prefix[buck] + (ptr - impls[buck].buf) + 1;
 }

The “+1” is the zero key’s slot, which is numbered 0.

Trade-offs

What the two-level table buys:

A parallelisable merge. The merge of N per-thread tables becomes 256 independent merges with an atomic counter and no locks. The serial tail of a parallel GROUP BY mostly disappears.
Incremental spilling. Because a bucket is self-contained, the aggregation can write one bucket to disk and continue, and the merge can read buckets back one at a time.
Amortised resize pauses. The expensive rehash is per-bucket, so no single pause is as long.
A memory-efficient large bucket. The Robin Hood switch at 2^16 cells means a bucket that grows large halves its memory overhead per key, at the cost of a little CPU for the chain ordering.

What it costs:

One extra indirection on every probe. The single-level table computes hash & mask; the two-level table computes a shift, an array index, and then the mask. The header admits it: “Usually works a little slower than a simple hash table.”
256 allocations instead of one. Each sub-table has its own buffer, its own grower, its own size counter.
A weaker guarantee for large tables. The bucket is selected from 32 bits of the hash, so a table with more than 2^32 cells is explicitly out of scope.
Complexity at the boundaries. The zero key doesn’t belong to a bucket; the prefix sums must be recomputed after growth; the string variant has to route on two axes at once.

This is a good example of a structural optimisation. Rather than making the hash table faster, ClickHouse changes the shape of the data so that the expensive part of the query — the merge — can be decomposed into 256 embarrassingly parallel pieces.

Wrapping up

Here’s the thing: the whole design rests on one sentence. The bucket is a function of the key, so the bucket is a partition of the key space. Every other property — parallel merge, incremental spill, amortised resize — falls out of that.

What to go do about it: if you have a GROUP BY over a high-cardinality column that takes a long time to finish but not a long time to start, you’re probably hitting the serial merge. Check AggregationConvertedToTwoLevel in system.events — if it’s zero and your key count is above 100 000, the conversion isn’t happening, and that’s your smoking gun.

And if you want to tune it, group_by_two_level_threshold is the knob. Lower it and the merge gets parallel sooner; the cost is one extra indirection per probe on the smaller table.

ClickHouse Packs Short Strings Into Hash Table Keys: No Allocation Per Key

Jay Hubbert — Wed, 21 Oct 2026 09:00:00 +0000

TL;DR: ClickHouse never stores a std::string in a hash table cell. It splits keys into size classes and packs strings of up to 24 bytes into plain integers, so your GROUP BY allocates nothing per key.

Strings are the most common GROUP BY key you’ll ever write. They’re also the worst case for a hash table.

Put a std::string in a cell and you get one heap allocation per distinct key. Put a std::string_view in a cell and the bytes have to outlive the table — which for an aggregation means copying them into an arena first.

ClickHouse’s answer is a hash table that is not a hash table. Instead of one table with a variable-size key, StringHashTable keeps five separate tables, one per key size class, and packs short keys into fixed-width integers.

The payoff: every cell is the same size, no key ever allocates, and the compiler can dispatch on key length at compile time.

The whole thing lives in src/Common/HashTable/StringHashTable.h, with the map specialisation in StringHashMap.h.

Five tables, one per size class

The composite table is five sub-tables:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:234
template <typename SubMaps>
class StringHashTable : private boost::noncopyable
{
protected:
 static constexpr size_t NUM_MAPS = 5;
 // Map for storing empty string
 using T0 = typename SubMaps::T0;

 // Short strings are stored as numbers
 using T1 = typename SubMaps::T1;
 using T2 = typename SubMaps::T2;
 using T3 = typename SubMaps::T3;

 // Long strings are stored as std::string_view along with saved hash
 using Ts = typename SubMaps::Ts;
 using Self = StringHashTable;

and the storage:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:254
 T0 m0;
 T1 m1;
 T2 m2;
 T3 m3;
 Ts ms;

Here’s what lives where:

Table	Key type	Holds
`m0`	—	the empty string, in a dedicated slot
`m1`	`StringKey8` (`UInt64`)	1..8 bytes
`m2`	`StringKey16` (`UInt128`)	9..16 bytes
`m3`	`StringKey24`	17..24 bytes
`ms`	`std::string_view` + saved hash	25+ bytes, or short strings with a trailing zero

And the key types are just integers:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:10
using StringKey8 = UInt64;
using StringKey16 = UInt128;
struct StringKey24
{
 UInt64 a;
 UInt64 b;
 UInt64 c;

 bool operator==(const StringKey24 rhs) const { return a == rhs.a && b == rhs.b && c == rhs.c; }
};

So a 20-byte string becomes three UInt64s and nothing else. No length field. No pointer. No allocation.

The length is implicit — it’s recovered from the position of the first zero byte.

The packing: how a string becomes a number

The heart of the file is dispatchOnKeyClass. It classifies a key, packs it, hashes it, and calls a callback with the right sub-table. It’s written for speed, and the comment says so out loud:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:357
 // Dispatch is written in a way that maximizes the performance:
 // 1. Always memcpy 8 times bytes
 // 2. Use switch case extension to generate fast dispatching table
 // 3. Funcs are named callables that can be force_inlined
 //
 /// Note: the key bytes must be readable in 8-byte chunks (a `std::string` cannot be passed
 /// here, while e.g. `ColumnString::getDataAt` can), because the packing copies 8 bytes at a
 /// time.
 template <typename Resolver, typename KeyHolder, typename HashProvider, typename Func>
 static auto ALWAYS_INLINE dispatchOnKeyClass(Resolver && resolver, KeyHolder && key_holder, HashProvider && hash_of, Func && func)
 {
 const auto & x = keyHolderGetKey(key_holder);
 const size_t sz = x.size();
 if (sz == 0)
 {
 keyHolderDiscardKey(key_holder);
 return func(resolver(std::integral_constant<size_t, 0>{}, 0), VoidKey{}, 0);
 }

 if (x[sz - 1] == 0)
 {
 // Strings with trailing zeros are not representable as fixed-size
 // string keys. Put them to the generic table.
 const size_t res = hash_of(x);
 return func(resolver(std::integral_constant<size_t, 4>{}, res), std::forward<KeyHolder>(key_holder), res);
 }

 const char * p = x.data();
 // pending bits that needs to be shifted out
 const char s = (-sz & 7) * 8;

The switch ((sz - 1) >> 3) that follows turns the length into one of four cases, and the compiler turns that into a jump table or a chain of comparisons:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:394
 switch ((sz - 1) >> 3)
 {
 case 0: // 1..8 bytes
 {
 // first half page
 if ((reinterpret_cast<uintptr_t>(p) & 2048) == 0)
 {
 memcpy(&n[0], p, 8);
 if constexpr (std::endian::native == std::endian::little)
 n[0] &= -1ULL >> s;
 else
 n[0] &= -1ULL << s;
 }
 else
 {
 const char * lp = x.data() + x.size() - 8;
 memcpy(&n[0], lp, 8);
 if constexpr (std::endian::native == std::endian::little)
 n[0] >>= s;
 else
 n[0] <<= s;
 }
 const size_t res = hash_of(k8);
 keyHolderDiscardKey(key_holder);
 return func(resolver(std::integral_constant<size_t, 1>{}, res), k8, res);
 }

Three tricks are hiding in those twenty lines. This is the part people get wrong the first time they read it.

Trick one: the trailing-zero test. A fixed-size string key encodes its length implicitly, by the position of the first zero byte. That means a string whose last byte is zero cannot be represented at all: "ab\0" and "ab" would pack to the same 8 bytes. So such strings get pushed to the generic table before the switch:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:376
 if (x[sz - 1] == 0)
 {
 // Strings with trailing zeros are not representable as fixed-size
 // string keys. Put them to the generic table.
 const size_t res = hash_of(x);
 return func(resolver(std::integral_constant<size_t, 4>{}, res), std::forward<KeyHolder>(key_holder), res);
 }

Trick two: the mask/shift pair. s = (-sz & 7) * 8 is the number of padding bits you need to round the length up to a multiple of 8. In the first branch, the 8 bytes are read forwards from the start of the string and the padding bits are cleared with a mask. In the second branch, the 8 bytes are read backwards from the end of the string and the padding bits are shifted out.

Trick three: the “first half page” rule. Reading 8 bytes from a 3-byte string reads 5 bytes past its end. That’s only safe if those bytes are mapped. The condition

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:399
 if ((reinterpret_cast<uintptr_t>(p) & 2048) == 0)

tests whether the address is 2048-byte aligned. If it is, at least 2048 bytes of the page are still ahead of it, so reading 8 bytes forward cannot fault. Otherwise the code reads backwards instead, ending exactly at the last byte of the string.

The bytes before the string that this reads are then shifted out — the >>= s discards exactly the (-sz & 7) bytes that belong to the previous string or to the column’s offset array. So the garbage is read but never used.

The 9..16 and 17..24 byte cases use the same trick with two and three words:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:420
 case 1: // 9..16 bytes
 {
 memcpy(&n[0], p, 8);
 const char * lp = x.data() + x.size() - 8;
 memcpy(&n[1], lp, 8);
 if constexpr (std::endian::native == std::endian::little)
 n[1] >>= s;
 else
 n[1] <<= s;
 const size_t res = hash_of(k16);
 keyHolderDiscardKey(key_holder);
 return func(resolver(std::integral_constant<size_t, 2>{}, res), k16, res);
 }

Notice that the first word is always read forwards from p — for a 9-byte string that’s inside the string — while the last word is read backwards from the end. The overlap between them is harmless: the bytes in the middle get read twice with the same values.

Unpacking: getting the string back

Going the other way, the length is recovered from the number of leading zero bytes:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:21
inline std::string_view ALWAYS_INLINE toStringView(const StringKey8 & n)
{
 chassert(n != 0);
 if constexpr (std::endian::native == std::endian::big)
 return {reinterpret_cast<const char *>(&n), 8ul - (std::countr_zero(n) >> 3)};
 else
 return {reinterpret_cast<const char *>(&n), 8ul - (std::countl_zero(n) >> 3)};
}

countl_zero(n) >> 3 is the number of whole zero bytes at the end of the word, so 8 - that is the string length. On a big-endian machine the zero bytes sit at the other end, hence the countr_zero variant.

The empty-cell rule, specialised per size class

The “empty cell is a zeroed cell” rule from the generic table needs a sensible notion of “zero” for a packed string key. For StringKey16 the rule is “the last word is zero”:

 cpp
 
 
// src/Common/HashTable/StringHashMap.h:20
template <typename TMapped>
struct StringHashMapCell<StringKey16, TMapped> : public HashMapCell<StringKey16, TMapped, StringHashTableHash, HashTableNoState>
{
 using Base = HashMapCell<StringKey16, TMapped, StringHashTableHash, HashTableNoState>;
 using value_type = typename Base::value_type;
 using Base::Base;
 static constexpr bool need_zero_value_storage = false;
 bool isZero(const HashTableNoState & state) const { return isZero(this->value.first, state); }

 // Zero means unoccupied cells in hash table. Use key with last word = 0 as
 // zero keys, because such keys are unrepresentable (no way to encode length).
 static bool isZero(const StringKey16 & key, const HashTableNoState &) { return key.items[1] == 0; }
 void setZero() { this->value.first.items[1] = 0; }

The reasoning is the same as in the generic table. A packed key with a zero last word would have to encode the length “16” with no room for a terminating zero byte, so such a key cannot exist. The encoding is injective on all valid keys, and the all-zero pattern is unreachable. That makes it safe to use as the empty marker.

Same for StringKey24, where the marker is the third word:

 cpp
 
 
// src/Common/HashTable/StringHashMap.h:49
 // Zero means unoccupied cells in hash table. Use key with last word = 0 as
 // zero keys, because such keys are unrepresentable (no way to encode length).
 static bool isZero(const StringKey24 & key, const HashTableNoState &)
 { return key.c == 0; }
 void setZero() { this->value.first.c = 0; }

The generic std::string_view cell can’t do this. A string_view is a pointer and a length, and a zeroed one is a null pointer with length 0 — which is a legal key, namely the empty string. So the generic cell declares that it needs no zero storage, and the empty string gets its own slot instead:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:134
template <typename Cell>
struct StringHashTableEmpty // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init) - `zero_value_storage` is raw storage for placement new, only written when `has_zero` becomes true
{
 using Self = StringHashTableEmpty;

 bool has_zero = false;
 alignas(Cell) std::byte zero_value_storage[sizeof(Cell)];

public:
 bool hasZero() const { return has_zero; }

And that empty slot isn’t a hash table at all — it’s a single cell with an emplace that ignores the key entirely:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:172
 template <typename KeyHolder>
 void ALWAYS_INLINE emplace(KeyHolder &&, LookupResult & it, bool & inserted, size_t = 0)
 {
 if (!hasZero())
 {
 setHasZero();
 inserted = true;
 }
 else
 inserted = false;
 it = zeroValue();
 }

Hashing: CRC32C where it can, CityHash where it can’t

The hash function is chosen per class, and for the fixed-size classes it’s the hardware CRC32C instruction:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:46
struct StringHashTableHash
{
#if defined(__SSE4_2__)
 size_t ALWAYS_INLINE operator()(StringKey8 key) const
 {
 size_t res = -1ULL;
 res = _mm_crc32_u64(res, key);
 return res;
 }
 size_t ALWAYS_INLINE operator()(StringKey16 key) const
 {
 size_t res = -1ULL;
 res = _mm_crc32_u64(res, key.items[0]);
 res = _mm_crc32_u64(res, key.items[1]);
 return res;
 }

with an AArch64 CRC32 path, an s390x path, and CityHash as the portable fallback:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:114
#else
 size_t ALWAYS_INLINE operator()(StringKey8 key) const
 {
 return CityHash_v1_0_2::CityHash64(reinterpret_cast<const char *>(&key), 8);
 }

CRC32 is a poor hash function in the abstract — the header of the generic hash file says so — but it’s one instruction per 8 bytes with a three-cycle latency, and the table is small enough to fit in cache most of the time.

There’s another requirement, and it’s the one that actually constrains the design: the hash has to be consistent with the packing. The hash is computed over the packed word, not over the original bytes, so "abc" and "abc\0..." can’t collide by accident. They hash different words.

The lookup result: why you never see the key

The string table doesn’t return a pointer to a cell. It returns a pointer to the mapped value, wrapped in a small type:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:214
template <typename Mapped>
struct StringHashTableLookupResult
{
 Mapped * mapped_ptr;
 StringHashTableLookupResult() {} /// NOLINT
 StringHashTableLookupResult(Mapped * mapped_ptr_) : mapped_ptr(mapped_ptr_) {} /// NOLINT
 StringHashTableLookupResult(std::nullptr_t) {} /// NOLINT
 const VoidKey getKey() const { return {}; } /// NOLINT
 auto & getMapped() { return *mapped_ptr; }
 auto & operator*() { return *this; }
 auto & operator*() const { return *this; }
 auto * operator->() { return this; }
 auto * operator->() const { return this; }
 explicit operator bool() const { return mapped_ptr; }

The reason is that the key lives in the cell, and the caller usually only wants the aggregate state. But the key is still there — just in a different sub-table with a different representation. Returning a pointer to it would mean returning a pointer to a StringKey8 or to a string_view, which have different types depending on the size class.

So the key is erased from the interface: getKey() returns a VoidKey.

Reserving and growing: where the heuristic leaks

The string table grows differently from the generic one:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:207
template <size_t initial_size_degree = 8>
struct StringHashTableGrower : public HashTableGrowerWithPrecalculation<initial_size_degree>
{
 // Smooth growing for string maps
 void increaseSize() { this->increaseSizeDegree(1); }
};

The generic grower doubles the buffer on overflow. The string grower grows by a factor of two in the exponent, i.e. by one power of two — the same thing, but expressed as a single degree increment so it composes with the precalculated mask.

Reserving is divided evenly over the four real sub-maps, because the caller only knows the total number of keys:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:317
 explicit StringHashTable(size_t reserve_for_num_elements)
 : m1{reserve_for_num_elements / 4}
 , m2{reserve_for_num_elements / 4}
 , m3{reserve_for_num_elements / 4}
 , ms{reserve_for_num_elements / 4}
 {
 }

That guess is often wrong. A table of 20-byte strings uses m3 only, and m1, m2, ms reserve space they will never touch.

The aggregator knows this, and refuses to pre-reserve the string tables:

 cpp
 
 
// src/Interpreters/Aggregator.cpp:5214
 /// `StringHashTable::reserve` splits the hint evenly over its four size-class sub-maps,
 /// while a real key set concentrates in one of them, so it is not reserved.
 constexpr bool can_reserve = requires { dst.reserve(size_t{}); } && !requires { dst.emptyStringSlot(); };

And the emptyStringSlot() method exists precisely so that this requires expression can tell a StringHashTable from an ordinary hash table:

 cpp
 
 
// src/Common/HashTable/StringHashTable.h:312
 /// The dedicated slot of the empty-string key, which the sub-table dispatch hashes to 0.
 T0 & emptyStringSlot() { return m0; }

Where it’s used: three string-key aggregation variants

The string map is one of the standard aggregation data variants:

 cpp
 
 
// src/Interpreters/AggregatedData.h:53
using AggregatedDataWithPackedStringKey = HashMap<PackedStringRef, AggregateDataPtr>;
using AggregatedDataWithShortStringKey = StringHashMap<AggregateDataPtr>;
using AggregatedDataWithStringKey = HashMapWithSavedHash<std::string_view, AggregateDataPtr>;

Three flavours of string key, three tables:

AggregatedDataWithPackedStringKey — a PackedStringRef, a pointer plus a length, with the payload inline for short strings;
AggregatedDataWithShortStringKey — the composite StringHashMap of this article;
AggregatedDataWithStringKey — a plain std::string_view map with a saved hash, where the key bytes live in an arena.

The StringHashMap variant is the interesting one, because it’s the only one where the key doesn’t live anywhere else. The string is in the table, as three machine words.

Trade-offs: what you win and what you pay

What the packed string table buys you:

No allocation per key. A 20-byte key is three UInt64s inside the cell. Nothing is copied into an arena, nothing is freed.
A uniform cell size. Every cell of m1 is 8 bytes of key plus the mapped value; every cell of m3 is 24 bytes plus the mapped value. The table can be scanned, and the compiler can generate good code for it.
A compile-time dispatch on length. dispatchOnKeyClass is a template over the size class, so each branch is specialised: the m1 path hashes one word, the m3 path hashes three.
A hash over the packed form. The hash is computed over exactly the bytes that are stored, so the stored key and the hashed key can never disagree.

What it costs you:

The bytes of the key are duplicated. The cell holds the packed key, and if the caller needs the original string it has to be reconstructed with toStringView.
The trailing-zero case is a wart. A string ending in \0 — which happens with binary data, and with FixedString columns padded with zeros — can’t use the packed classes and falls back to the generic table, where the key is a string_view into the arena.
The reservation heuristic is bad. Reserving n/4 for each of the four sub-maps wastes memory whenever the keys concentrate in one class, which is the normal case.
The empty string needs its own slot, and the key type of the generic sub-table isn’t the same as the key type of the packed ones, so the composite has to be written as a template over a SubMaps bundle rather than as a plain HashTable.

This is a good example of a general principle: when the key type has a size distribution, splitting the table by size class turns a variable-size-cell problem into a set of fixed-size-cell problems, and lets the compiler specialise each one.

Wrapping up

Worth internalizing: a GROUP BY over strings is a memory-bound benchmark before it’s a CPU-bound one. ClickHouse’s whole trick here is to make the key be the data — no arena, no pointer chasing, no allocator traffic.

So if you’re tuning an aggregation and it feels allocation-heavy, check what your key type maps to. Short, non-zero-terminated strings hit the fast path. FixedString padded with zeros and raw binary blobs do not — and that’s a silent cliff, not an error message.

Next time you’re profiling, watch for memcpy traffic coming out of the aggregator rather than out of the storage layer. That’s the signature of this code path not doing what you expect.

ClickHouse Marks Empty Hash Table Cells With Zeroed Memory

Jay Hubbert — Wed, 21 Oct 2026 09:00:00 +0000

TL;DR: ClickHouse’s hash table has no occupancy bit, no tombstone, and no sentinel pointer. A cell is empty when its key is all zero bytes, and the buffer starts zeroed because the allocator uses calloc. That one decision ripples through the cell layout, the zero-key special case, deletion, and even why the hash function has to be good.

Every GROUP BY in ClickHouse is a hash table. The table has to hold millions of keys, be probed once per input row, and be resized while it is being filled — all while the query is competing for memory with everything else on the machine.

std::unordered_map is simply not competitive here: it allocates a node per element, keeps a pointer chain, and pays a pointer chase for every collision.

So ClickHouse ships its own hash table in src/Common/HashTable/HashTable.h: a flat array of cells, open addressing, linear probing. The probing is textbook. The interesting part is the bookkeeping around emptiness.

There is no “occupied” bit. No tombstone. No sentinel pointer. No version counter. A cell is empty when its key is all zero bytes, and the whole buffer starts out zeroed by the allocator.

That single decision shapes the entire file: the allocator, the cell layout, the way the zero key is handled, the way deletion works, and the reason the hash function has to be good.

The shape of the table

 cpp
 
 
// src/Common/HashTable/HashTable.h:518
template <typename Key, typename Cell, typename Hash, typename Grower, typename Allocator>
class HashTable : private boost::noncopyable,
 protected Hash,
 protected Allocator,
 protected Cell::State,
 public ZeroValueStorage<Cell::need_zero_value_storage, Cell> /// empty base optimization
{

The table is one contiguous buffer of Cells plus a Grower that knows the buffer size:

 cpp
 
 
// src/Common/HashTable/HashTable.h:550
 size_t m_size = 0; /// Amount of elements
 Cell * buf; /// A piece of memory for all elements except the element with zero key.
 Grower grower;

Cell is a policy class. The simplest one stores nothing but a key:

 cpp
 
 
// src/Common/HashTable/HashTable.h:154
template <typename Key, typename Hash, typename TState = HashTableNoState>
struct HashTableCell
{
 using State = TState;

 using key_type = Key;
 using value_type = Key;
 using mapped_type = VoidMapped;

 Key key;

The map flavour stores a pair, and the variant that ClickHouse uses for aggregation additionally caches the hash so that a rehash does not have to recompute it:

 cpp
 
 
// src/Common/HashTable/HashMap.h:165
template <typename Key, typename TMapped, typename Hash, typename TState = HashTableNoState>
struct HashMapCellWithSavedHash : public HashMapCell<Key, TMapped, Hash, TState> // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init) - `saved_hash` is set by `setHash` immediately after placement construction on insert; on the hot aggregation/join path we must avoid the redundant store
{
 using Base = HashMapCell<Key, TMapped, Hash, TState>;

 size_t saved_hash;

Empty means zero

Here is the whole emptiness protocol:

 cpp
 
 
// src/Common/HashTable/HashTable.h:191
 /// Whether the key is zero. In the main buffer, cells with a zero key are considered empty.
 /// If zero keys can be inserted into the table, then the cell for the zero key is stored separately, not in the main buffer.
 /// Zero keys must be such that the zeroed-down piece of memory is a zero key.
 bool isZero(const State & state) const { return isZero(key, state); }
 static bool isZero(const Key & key, const State & /*state*/) { return ZeroTraits::check(key); }

 /// Set the key value to zero.
 void setZero() { ZeroTraits::set(key); }

and the generic part:

 cpp
 
 
// src/Common/HashTable/HashTable.h:89
namespace ZeroTraits
{

template <typename T>
bool check(const T x)
{
 return bitEquals(x, T{});
}

template <typename T>
void set(T & x) { x = T{}; }

}

bitEquals is bitwise, not ==, and that matters for floating point:

 cpp
 
 
// src/Common/HashTable/HashTable.h:77
template <typename T>
inline ALWAYS_INLINE bool bitEquals(T a, T b)
{
 if constexpr (is_floating_point<T>)
 /// Note that memcmp with constant size is a compiler builtin.
 return 0 == memcmp(&a, &b, sizeof(T)); /// NOLINT
 else
 return a == b;
}

The comment above it explains why bit equality is mandatory rather than merely nice:

 cpp
 
 
// src/Common/HashTable/HashTable.h:71
/** Numbers are compared bitwise.
 * Complex types are compared by operator== as usual (this is important if there are gaps).
 *
 * This is needed if you use floats as keys. They are compared by bit equality.
 * Otherwise the invariants in hash table probing do not met when NaNs are present.
 */

NaN never equals itself, so a ==-based emptiness test could never recognise an empty float cell — and the probe loop would run right off the end of the buffer.

The zeroed buffer comes from the allocator:

 cpp
 
 
// src/Common/HashTable/HashTableAllocator.h:8
/**
 * We are going to use the entire memory we allocated when resizing a hash
 * table, so it makes sense to pre-fault the pages so that page faults don't
 * interrupt the resize loop. Set the allocator parameter accordingly.
 */
using HashTableAllocator = Allocator<true /* clear_memory */, true /* populate */>;

clear_memory becomes calloc:

 cpp
 
 
// src/Common/Allocator.cpp:79
 if (alignment <= MALLOC_MIN_ALIGNMENT) [[likely]]
 {
 if constexpr (clear_memory)
 buf = __real_calloc(size, 1);
 else
 buf = __real_malloc(size);

So the invariant is: the bytes of a fresh buffer are zero, therefore a fresh cell is empty. Nothing more is needed to track occupancy. The diagram of the buffer is therefore very simple:

 text
 
 
 buf buf + bufSize()
 | |
 v v
 +--------+--------+--------+--------+--------+--------+--------+--------+
 | cell 0 | cell 1 | cell 2 | cell 3 | cell 4 | cell 5 | cell 6 | cell 7 |
 +--------+--------+--------+--------+--------+--------+--------+--------+
 key=A (zero) key=C (zero) key=B (zero) (zero)
 [used] [empty] [used] [empty] [used] [empty] [empty]

Note what is not there: no occupancy byte, no pointer, no version. A HashMapCell<UInt64, AggregateState*> is 16 bytes of pure payload. The empty/used distinction is free.

The zero key gets its own slot

There is one hole in the “zero means empty” scheme: 0 is a perfectly legal key. If it were stored in the buffer, it would be indistinguishable from an empty cell. So it is stored outside the buffer, in a ZeroValueStorage that the table inherits from:

 cpp
 
 
// src/Common/HashTable/HashTable.h:474
template <typename Cell>
struct ZeroValueStorage<true, Cell> // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init) - `zero_value_storage` is raw storage for placement new, only written when `has_zero` becomes true
{
private:
 bool has_zero = false;
 alignas(Cell) std::byte zero_value_storage[sizeof(Cell)];

public:
 bool hasZero() const { return has_zero; }

 void setHasZero()
 {
 has_zero = true;
 new (zeroValue()) Cell();
 }

and the insert path routes the zero key to it:

 cpp
 
 
// src/Common/HashTable/HashTable.h:1354
 bool ALWAYS_INLINE emplaceIfZero(const Key & x, LookupResult & it, bool & inserted, size_t hash_value)
 {
 /// If it is claimed that the zero key can not be inserted into the table.
 if constexpr (!Cell::need_zero_value_storage)
 return false;

 if (unlikely(Cell::isZero(x, *this)))
 {
 it = this->zeroValue();

 if (!this->hasZero())
 {
 ++m_size;
 this->setHasZero();
 this->zeroValue()->setHash(hash_value);
 inserted = true;
 }
 else
 inserted = false;

 return true;
 }

 return false;
 }

The zero key lives at a fixed address, so a lookup for it is a branch and a pointer, never a probe. The cost of the scheme is one bool plus one Cell of storage per table, and a slightly awkward iteration order (the zero cell is visited first, then the buffer).

And this is not an exotic corner. 0 is a perfectly ordinary grouping key:

 sql
 
SELECT k, count() FROM t GROUP BY k;

with a k column full of zeros produces a group whose key is the zero key, and that group is the one stored in the separate slot.

Note also that the design is conditional: a cell type can declare that zero keys are impossible (need_zero_value_storage = false), and then ZeroValueStorage<false, Cell> degenerates to nothing at all. The string tables do exactly that — see the article on the string-specialised map.

Probing

The probe sequence is the simplest possible one: linear probing with a mask.

 cpp
 
 
// src/Common/HashTable/HashTable.h:232
 /// The size of the hash table in the cells.
 size_t bufSize() const { return 1ULL << size_degree; }

 size_t maxFill() const { return 1ULL << (size_degree - 1); }
 size_t mask() const { return bufSize() - 1; }

 /// From the hash value, get the cell number in the hash table.
 size_t place(size_t x) const { return x & mask(); }

 /// The next cell in the collision resolution chain.
 size_t next(size_t pos) const { ++pos; return pos & mask(); }

 /// Whether the hash table is sufficiently full. You need to increase the size of the hash table, or remove something unnecessary from it.
 bool overflow(size_t elems) const { return elems > maxFill(); }

The load factor is 0.5: maxFill() is half the buffer. And the probe loop itself:

 cpp
 
 
// src/Common/HashTable/HashTable.h:559
 size_t ALWAYS_INLINE findCell(const Key & x, size_t hash_value, size_t place_value) const
 {
 while (!buf[place_value].isZero(*this) && !buf[place_value].keyEquals(x, hash_value, *this))
 {
 place_value = grower.next(place_value);
#ifdef DBMS_HASH_MAP_COUNT_COLLISIONS
 ++collisions;
#endif
 }

 return place_value;
 }

Two things to note. First, isZero is checked before keyEquals: the loop stops at the first empty cell, because with linear probing an empty cell terminates the chain. Second, the comparison is a bitwise equality against the key, and for the HashMapCellWithSavedHash variant it is short-circuited by the cached hash.

The zero key is handled before the probe:

 cpp
 
 
// src/Common/HashTable/HashTable.h:1617
 LookupResult ALWAYS_INLINE find(const Key & x)
 {
 if (Cell::isZero(x, *this))
 return this->hasZero() ? this->zeroValue() : nullptr;

 return findNonZeroCell(x, hash(x));
 }

Insertion

emplace computes the hash, routes the zero key, and otherwise finds a place:

 cpp
 
 
// src/Common/HashTable/HashTable.h:1380
 template <typename KeyHolder>
 void ALWAYS_INLINE emplaceNonZeroImpl(size_t place_value, KeyHolder && key_holder,
 LookupResult & it, bool & inserted, size_t hash_value)
 {
 it = &buf[place_value];

 if (!buf[place_value].isZero(*this))
 {
 keyHolderDiscardKey(key_holder);
 inserted = false;
 return;
 }

 keyHolderPersistKey(key_holder);
 const auto & key = keyHolderGetKey(key_holder);

 new (&buf[place_value]) Cell(key, *this);
 buf[place_value].setHash(hash_value);
 inserted = true;
 ++m_size;

 if (unlikely(grower.overflow(m_size)))
 {
 try
 {
 resize();
 }
 catch (...)
 {
 /** If we have not resized successfully, then there will be problems.
 * There remains a key, but uninitialized mapped-value,
 * which, perhaps, can not even be called a destructor.
 */
 --m_size;
 buf[place_value].setZero();
 inserted = false;
 keyHolderDiscardKey(key_holder);
 throw;
 }

 // The hash table was rehashed, so we have to re-find the key.
 it = findNonZeroCell(key, hash_value);
 chassert(it);
 }
 }

Two details worth pausing on. The cell is placement-newed; the buffer is raw memory, and the destructor of a non-trivial cell is only run for cells that were actually constructed (destroyElements walks the table and calls the destructor only for non-zero cells).

And the resize happens after the insert, not before. That means the table can exceed its nominal capacity by one element, and it means the iterator has to be re-found afterwards, because the rehash moved the cell. The exception handler is careful to undo the insert (decrement the size, zero the cell) so that a failed allocation does not leave a half-built cell behind.

Growing: the wrap-around special case

The resize path is where the linear-probing invariants get hairy. When the buffer grows, every element has to be re-placed, and an element that was at the end of the old buffer may belong at the beginning of the new one because its collision chain wrapped around:

 cpp
 
 
// src/Common/HashTable/HashTable.h:837
 void resize(size_t for_num_elems = 0, size_t for_buf_size = 0)
 {
#ifdef DBMS_HASH_MAP_DEBUG_RESIZES
 Stopwatch watch;
#endif

 size_t old_size = grower.bufSize();

 /** In case of exception for the object to remain in the correct state,
 * changing the variable `grower` (which determines the buffer size of the hash table)
 * is postponed for a moment after a real buffer change.
 * The temporary variable `new_grower` is used to determine the new size.
 */
 Grower new_grower = grower;

and the re-placement loop:

 cpp
 
 
// src/Common/HashTable/HashTable.h:884
 size_t i = 0;
 for (; i < old_size; ++i)
 if (!buf[i].isZero(*this))
 reinsert(buf[i], buf[i].getHash(*this));

 /** There is also a special case:
 * if the element was to be at the end of the old buffer, [ x]
 * but is at the beginning because of the collision resolution chain, [o x]
 * then after resizing, it will first be out of place again, [ xo ]
 * and in order to transfer it where necessary,
 * after transferring all the elements from the old halves you need to [ o x ]
 * process tail from the collision resolution chain immediately after it [ o x ]
 */
 size_t new_size = grower.bufSize();
 for (; i < new_size && !buf[i].isZero(*this); ++i)
 reinsert(buf[i], buf[i].getHash(*this));

The picture in that comment is the whole problem: after the buffer is doubled, the element at the very end of the old buffer (x) is now in the middle of the new one, and the element that was pushed behind it (o) belongs before it. The second loop therefore keeps going past the end of the old buffer, as long as the cells it finds are non-empty, re-inserting them.

The re-insertion itself is a memcpy of a trivially-copyable cell into the new position:

 cpp
 
 
// src/Common/HashTable/HashTable.h:918
 void reinsert(Cell & x, size_t hash_value)
 {
 size_t place_value = grower.place(hash_value);

 /// If the element is in its place.
 if (&x == &buf[place_value])
 return;

 /// Compute a new location, taking into account the collision resolution chain.
 place_value = findCell(Cell::getKey(x.getValue()), hash_value, place_value);

 /// If the item remains in its place in the old collision resolution chain.
 if (!buf[place_value].isZero(*this))
 return;

 /// Copy to a new location and zero the old one.
 x.setHash(hash_value);
 memcpy(static_cast<void*>(&buf[place_value]), &x, sizeof(x)); /// NOLINT(bugprone-undefined-memory-manipulation)
 x.setZero();
 }

The grower doubles the size until max_size_degree (23) and then grows by one step at a time:

 cpp
 
 
// src/Common/HashTable/HashTable.h:247
 /// Increase the size of the hash table.
 void increaseSize()
 {
 size_degree += size_degree >= max_size_degree ? 1 : 2;
 }

Robin Hood: what changed recently

The most interesting recent addition to the file is a second probing discipline. The classic HashTableGrower keeps the load factor at 0.5; the new HashTableGrowerWithPrecalculationAndRobinHood keeps 0.5 until the table reaches 2^16 cells, and then switches to Robin Hood hashing with a 90% load factor — in place, without reallocating:

 cpp
 
 
// src/Common/HashTable/HashTable.h:336
/** Like `HashTableGrowerWithPrecalculation`, but adapts to large hash tables.
 *
 * For large hash tables, random memory access is the main bottleneck, not the computations.
 * When the buffer reaches 2^robin_hood_size_degree cells, the grower switches the hash table to the Robin Hood mode:
 * the max load factor is increased from 0.5 to robin_hood_max_load_factor_percent / 100 (the hash table becomes up to two times smaller),
 * and the elements inside collision resolution chains are ordered by their optimal positions,
 * so a lookup of a missing key can stop early instead of walking to the end of the chain.
 *
 * The memory layout is the same in both modes, and switching happens in place, without growing the buffer.
 * Hash tables check the mode only if the grower has the `isRobinHood` method (see `GrowerWithRobinHood`),
 * so there is no cost for other growers.
 *
 * Requirements to the cell: obtaining the hash value with `getHash` must be cheap (the hash is saved or cheap to compute),
 * and the hash table must not keep pointers to cells while inserting other elements,
 * because an insertion may move other elements along the collision resolution chain.
 */

The payoff is twofold. The table becomes up to twice as small (0.9 vs 0.5 load factor), which halves the number of cache misses in a random probe; and a failed lookup can stop early, because the chain is sorted by “distance from the ideal position”:

 cpp
 
 
// src/Common/HashTable/HashTable.h:607
 size_t ALWAYS_INLINE findCellRobinHood(const Key & x, size_t hash_value, size_t place_value, bool & found) const
 {
 for (size_t distance = 0;; ++distance)
 {
 const Cell & cell = buf[place_value];
 if (cell.isZero(*this))
 {
 found = false;
 return place_value;
 }
 if (cell.keyEquals(x, hash_value, *this))
 {
 found = true;
 return place_value;
 }
 /// The element at the optimal position of the key cannot be closer to its own optimal position, so do not compute its hash.
 if (distance && distanceFromOptimalPosition(cell, place_value) < distance)
 {
 found = false;
 return place_value;
 }
 place_value = grower.next(place_value);
#ifdef DBMS_HASH_MAP_COUNT_COLLISIONS
 ++collisions;
#endif
 }
 }

“Distance from the optimal position” is just the difference between where the key would go and where it is, measured modulo the buffer size:

 cpp
 
 
// src/Common/HashTable/HashTable.h:597
 size_t ALWAYS_INLINE distanceFromOptimalPosition(const Cell & cell, size_t place_value) const
 {
 return grower.place(place_value - grower.place(cell.getHash(*this)));
 }

Robin Hood ordering also changes insertion: the new key may have to be placed before an existing one, which means shifting the rest of the chain to the right:

 cpp
 
 
// src/Common/HashTable/HashTable.h:664
 void NO_INLINE shiftCellsRight(size_t place_value)
 {
 size_t empty_place = findEmptyCell(place_value);

 if (empty_place < place_value)
 {
 /// The chain wraps around the end of the buffer.
 size_t last_place = grower.bufSize() - 1;
 memmove(static_cast<void *>(&buf[1]), static_cast<const void *>(&buf[0]), empty_place * sizeof(Cell));
 memcpy(static_cast<void *>(&buf[0]), static_cast<const void *>(&buf[last_place]), sizeof(Cell));
 empty_place = last_place;
 }

 memmove(static_cast<void *>(&buf[place_value + 1]), static_cast<const void *>(&buf[place_value]), (empty_place - place_value) * sizeof(Cell));
 buf[place_value].setZero();
 }

and the chains have to be re-sorted when the mode is switched on — by an insertion sort, since the chains are “already almost ordered” in practice:

 cpp
 
 
// src/Common/HashTable/HashTable.h:681
 /** Reorder the elements in every collision resolution chain by their optimal positions, as required in the Robin Hood mode.
 * Used after the elements were placed by the ordinary linear probing: on resize, or when switching to the Robin Hood mode.
 * It is an insertion sort, which is fast for the chains that are already almost ordered.
 */

Deletion without tombstones

Deleting from an open-addressing table usually means writing a tombstone, so that the probe chain is not cut short. Tombstones accumulate, they need to be cleaned up, and they slow down every subsequent lookup.

ClickHouse does not use tombstones. Instead it moves the elements of the chain:

 cpp
 
 
// src/Common/HashTable/HashTable.h:731
 /// Remove the element at place_value without calling its destructor, moving the following elements of the collision resolution chain to keep them reachable.
 /// Not inlined to keep the code of inserts small.
 void eraseCellAt(size_t erased_key_position)
 {
 /// We need to guarantee loop termination because there will be empty position
 chassert(m_size < grower.bufSize());

 size_t next_position = erased_key_position;

 /**
 * During element deletion there is a possibility that the search will be broken for one
 * of the following elements, because this place erased_key_position is empty. We will check
 * next_element. Consider a sequence from (erased_key_position, next_element], if the
 * optimal_position of next_element falls into it, then removing erased_key_position
 * will not break search for next_element.
 * If optimal_position of the element does not fall into the sequence (erased_key_position, next_element]
 * then deleting a erased_key_position will break search for it, so we need to move next_element
 * to erased_key_position. Now we have empty place at next_element, so we apply the identical
 * procedure for it.
 * If an empty element is encountered then means that there is no more next elements for which we can
 * break the search so we can exit.
 *
 * In the Robin Hood mode, the elements of a chain are ordered by their optimal positions,
 * and this procedure moves them one cell to the left, keeping the order.
 */

The rule the comment spells out is the classic one: an element may stay where it is if its ideal position is not before the hole; otherwise it has to slide left into the hole, which moves the hole one step further right, and the procedure repeats:

 cpp
 
 
// src/Common/HashTable/HashTable.h:798
 /// Move the element to the freed place
 memcpy(static_cast<void *>(&buf[erased_key_position]), static_cast<void *>(&buf[next_position]), sizeof(Cell));

 /// Now we have another freed place
 erased_key_position = next_position;
 }

 buf[erased_key_position].setZero();
 --m_size;
 }

This costs a memcpy per displaced element but keeps the table free of garbage: the buffer is always exactly as full as m_size says, and a probe chain never walks over a tombstone.

The allocator: zeroed and pre-faulted

HashTableAllocator is Allocator<true /* clear_memory */, true /* populate */>. The second flag is the interesting one: it pre-faults the pages of a large allocation with MADV_POPULATE_WRITE, so that the resize loop does not stop for page faults:

 cpp
 
 
// src/Common/Allocator.cpp:59
void prefaultPages([[maybe_unused]] void * buf_, [[maybe_unused]] size_t len_)
{
#if defined(MADV_POPULATE_WRITE)
 if (len_ < POPULATE_THRESHOLD)
 return;

 if (!is_supported_by_kernel) [[unlikely]]
 return;

 auto [buf, len] = adjustToPageSize(buf_, len_, ::getPageSize());
 ::madvise(buf, len, MADV_POPULATE_WRITE);
#endif
}

 cpp
 
 
// src/Common/Allocator.cpp:132
/// Constant is chosen almost arbitrarily, what I observed is 128KB is too small, 1MB is almost indistinguishable from 64MB and 1GB is too large.
extern const size_t POPULATE_THRESHOLD = std::max(Int64{16 * 1024 * 1024}, ::getPageSize());

The rationale is spelled out in HashTableAllocator.h itself: the whole allocation will be used, so pre-faulting it costs nothing extra and removes a source of latency spikes from the middle of a rehash. The kernel support check is a version test (Linux 5.14+), not an EINVAL probe, because madvise returns EINVAL for many unrelated reasons.

And note how calloc and the “zero means empty” invariant fit together perfectly: calloc of a large block is free on Linux for big allocations (the kernel hands out zero pages), so the “clear the buffer” requirement that the emptiness protocol imposes costs nothing at all.

Prefetching

A hash table probe is a random memory access; at 90% load factor with Robin Hood ordering, a lookup of a missing key can touch several cache lines. ClickHouse tries to hide that latency by prefetching the cell a few iterations ahead. The helper measures its own iteration latency to decide how far ahead to look:

 cpp
 
 
// src/Common/HashTable/Prefetching.h:33
class PrefetchingHelper
{
public:
 size_t calcPrefetchLookAhead()
 {
 static constexpr auto assumed_load_latency_ns = 100;
 static constexpr auto just_coefficient = 4;
 static constexpr auto numerator = just_coefficient * assumed_load_latency_ns * iterations_to_measure;
 size_t elapsed_ns = watch.elapsedNanoseconds();
 if (unlikely(elapsed_ns == 0))
 return max_look_ahead_value;
 size_t look_ahead = (numerator + elapsed_ns - 1) / elapsed_ns;
 return std::clamp<size_t>(look_ahead, min_look_ahead_value, max_look_ahead_value);
 }

 static constexpr size_t getInitialLookAheadValue() { return min_look_ahead_value; }

 static constexpr size_t iterationsToMeasure() { return iterations_to_measure; }

private:
 static constexpr size_t iterations_to_measure = 100;
 static constexpr size_t min_look_ahead_value = 4;
 static constexpr size_t max_look_ahead_value = 32;

The look-ahead is thus between 4 and 32 cells, tuned to the machine: the formula is 4 * 100ns * 100 / elapsed, i.e. four times the assumed memory latency, expressed in iterations.

The prefetching iterator of the table uses it, and also prefetches the key bytes of the cell it is about to look at, not just the cell:

 cpp
 
 
// src/Common/HashTable/HashTable.h:1021
 void prefetch()
 {
 static_assert(CouldPrefetchKey<cell_type>);
 auto * buf_end = container->buf + container->grower.bufSize();
 iter_count++;
 if (prefetch_ptr < buf_end && prefetched_count < prefetch_ahead) [[likely]]
 {
 if (iter_count == DB::PrefetchingHelper::iterationsToMeasure()) [[unlikely]]
 prefetch_ahead = prefetching.calcPrefetchLookAhead();
 auto n = prefetch_ahead - prefetched_count;
 cell_type * last_ptr = nullptr;
 for (size_t i = 0; i < n; ++i)
 {
 while (prefetch_ptr < buf_end && prefetch_ptr->isZero(*container))
 ++prefetch_ptr;

 if (prefetch_ptr < buf_end) [[likely]]
 {
 last_ptr = prefetch_ptr;
 prefetch_ptr++;
 prefetched_count++;
 }
 else
 break;
 }

 if (last_ptr) [[likely]]
 keyPrefetch(last_ptr->getKey());
 }
 }

keyPrefetch touches up to four cache lines of the key data:

 cpp
 
 
// src/Common/HashTable/HashTableKeyHolder.h:182
inline void keyPrefetch(const std::string_view key)
{
 const size_t cache_line_size = 64;
 // Prefetch up to 4 cache lines.
 const size_t max_prefetch = cache_line_size * 4;
 const size_t n = std::min(key.size(), max_prefetch);
 const char * ptr = key.data();
 const char * end = ptr + n;
 for (; ptr < end; ptr += cache_line_size)
 {
 __builtin_prefetch(ptr);
 }
}

Keys that live outside the table

Not every key can live inside the cell. A std::string_view key points at data owned by somebody else — often a column that will be gone by the time the aggregation is finalised. ClickHouse solves this with a key holder: a small policy object that knows how to make a temporary key persistent, and when to throw it away.

 cpp
 
 
// src/Common/HashTable/HashTableKeyHolder.h:53
/**
 * Returns the key. Can return the temporary key initially.
 * After the call to keyHolderPersistKey(), must return the persistent key.
 */
template <typename Key>
inline Key & ALWAYS_INLINE keyHolderGetKey(Key && key) { return key; } /// NOLINT(bugprone-return-const-ref-from-parameter)

/**
 * Make the key persistent. keyHolderGetKey() must return the persistent key
 * after this call.
 */
template <typename Key>
inline void ALWAYS_INLINE keyHolderPersistKey(Key &&) {}

/**
 * Discard the key. Calling keyHolderGetKey() is ill-defined after this.
 */
template <typename Key>
inline void ALWAYS_INLINE keyHolderDiscardKey(Key &&) {}

The default is a no-op: for a key stored by value, there is nothing to do, and the branch is optimised away. The ArenaKeyHolder version copies the bytes into an Arena when the key turns out to be new:

 cpp
 
 
// src/Common/HashTable/HashTableKeyHolder.h:98
inline void ALWAYS_INLINE keyHolderPersistKey(DB::ArenaKeyHolder & holder)
{
 // Normally, our hash table shouldn't ask to persist a zero key,
 // but it can happened in the case of clearable hash table (ClearableHashSet, for example).
 // The clearable hash table doesn't use zero storage and
 // distinguishes empty keys by using cell version, not the value itself.
 // So, when an empty std::string_view is inserted in ClearableHashSet we'll get here key of zero size.
 holder.key = std::string_view{holder.pool.insert(holder.key.data(), holder.key.size()), holder.key.size()};
}

The elegance is in where the decision is made. The hash table knows whether the key it just found was new; the caller does not. So the table calls keyHolderDiscardKey when the key already existed (nothing to persist, and the arena allocation is rolled back) and keyHolderPersistKey when it did not:

 cpp
 
 
// src/Common/HashTable/HashTable.h:1386
 if (!buf[place_value].isZero(*this))
 {
 keyHolderDiscardKey(key_holder);
 inserted = false;
 return;
 }

 keyHolderPersistKey(key_holder);

The hash function has to be good

Because the table uses linear probing and the low bits of the hash to pick the bucket, a poor hash function is catastrophic: the low bits of a timestamp-like key are nearly constant, and every key lands in the same few cells. The header says so bluntly:

 cpp
 
 
// src/Common/HashTable/Hash.h:15
/** Hash functions that are better than the trivial function std::hash.
 *
 * Example: when we do aggregation by the visitor ID, the performance increase is more than 5 times.
 * This is because of following reasons:
 * - in Metrica web analytics system, visitor identifier is an integer that has timestamp with seconds resolution in lower bits;
 * - in typical implementation of standard library, hash function for integers is trivial and just use lower bits;
 * - traffic is non-uniformly distributed across a day;
 * - we are using open-addressing linear probing hash tables that are most critical to hash function quality,
 * and trivial hash function gives disastrous results.
 */

The default integer hash is the MurmurHash finalizer:

 cpp
 
 
// src/Common/HashTable/Hash.h:26
/** Taken from MurmurHash. This is Murmur finalizer.
 * Faster than intHash32 when inserting into the hash table UInt64 -> UInt64, where the key is the visitor ID.
 */
inline UInt64 intHash64(UInt64 x)
{
 x ^= x >> 33;
 x *= 0xff51afd7ed558ccdULL;
 x ^= x >> 33;
 x *= 0xc4ceb9fe1a85ec53ULL;
 x ^= x >> 33;

 return x;
}

Five instructions and two multiplies turn “the low bits are constant” into “the low bits are random”. For string keys and general keys, the CRC32 hardware instruction is used where it exists — three cycles of latency and one per cycle of throughput, which is why the string tables use it even though CRC32 is a poor hash in the abstract:

 cpp
 
 
// src/Common/HashTable/Hash.h:40
/** CRC32C is not very high-quality as a hash function,
 * according to avalanche and bit independence tests (see SMHasher software), as well as a small number of bits,
 * but can behave well when used in hash tables,
 * due to high speed (latency 3 + 1 clock cycle, throughput 1 clock cycle).
 * Works only with SSE 4.2 support.
 */

Trade-offs

What the “zero means empty” scheme buys:

Zero bytes of metadata per cell. A HashMapCell<UInt64, T*> is 16 bytes, all of it payload. A scheme with an occupancy bit would need to steal a bit from the key or add a byte.
A free calloc. The kernel can hand out pre-zeroed pages; the emptiness protocol costs nothing at allocation time.
No tombstones, ever. Deletion moves cells instead of marking them, so the table never degrades.
A cheap scan. for (auto & cell : buf) if (!cell.isZero()) ... is the iteration order; no indirection, no linked list.

What it costs:

The key must be zero-comparable, and bit-comparable. Floating point keys are compared with memcmp, and NaN semantics have to be papered over.
The zero key is special. It lives in a separate slot, it is visited first during iteration, and its handling adds a branch to every insert and lookup.
The hash must be excellent, because linear probing amplifies clustering, and the low bits of the hash are used directly.
Rehashing is a memcpy storm. Growing the table means moving every cell, and the wrap-around case needs a second pass over the head of the new buffer.

And the recent Robin Hood addition changes the balance again: it trades a little CPU (reordering chains, occasionally shifting cells on insert) for a halving of the memory footprint at large sizes — which, for a table that is probed randomly, is exactly the right trade, because the cost is cache misses, not instructions.

Wrapping up

The one thing to remember: “empty means all-zero bytes” buys you a cell layout that is 100% payload, with no occupancy bit to steal from your key.

The part people get wrong: this is not free. The zero key becomes a special case, float keys need bitwise comparison, and the hash function has to be genuinely good — a five-line Murmur finalizer stands between you and a 5× slower aggregation.

What to go do about it: if you ever profile a GROUP BY and see the cost dominated by cache misses on a huge hash table, remember that the Robin Hood grower exists precisely for that case: it halves the footprint at large sizes.

ClickHouse Evaluates IN with Hash Sets

Jay Hubbert — Wed, 21 Oct 2026 09:00:00 +0000

TL;DR: IN is not a comparison, it is a hash set. ClickHouse picks one of thirteen specialisations based on the shape of the right-hand side, packs integer keys into the hash table’s own key, and — if the set grows past half your memory — spills it to disk behind per-block Bloom filters.

x IN (1, 2, 3) looks like a comparison. x IN (SELECT user_id FROM logins WHERE …) looks like a join. Under the hood, both become the same thing: a hash set, built once, probed once per row of the left-hand side.

The interesting part isn’t “it is a hash table” — it’s which hash table. ClickHouse picks a different data structure for every shape of the right-hand side, and the set can get big enough that it no longer fits in memory and has to spill to disk behind Bloom filters.

Let’s walk it.

The set is a plan artifact, not a function argument

The IN operator is a normal SQL function whose second argument is not a value but a set object:

 cpp
 
 
// src/Functions/in.cpp:145
 ColumnPtr column_set_ptr = arguments[1].column;
 const ColumnSet * column_set = tryGetColumnSet(column_set_ptr);
 if (!column_set)
 throw Exception(ErrorCodes::ILLEGAL_COLUMN, "Second argument for function '{}' must be Set; found {}",
 getName(), column_set_ptr->getName());

 auto future_set = column_set->getData();
 if (!future_set)
 {
 if (dry_run)
 return ColumnUInt8::create(input_rows_count, static_cast<UInt8>(0));

 throw Exception(ErrorCodes::LOGICAL_ERROR, "No Set is passed as the second argument for function '{}'", getName());
 }

The set arrives as a ColumnSet holding a FutureSet — a promise, because the set may not be built yet when the function’s type is resolved. A separate step in the pipeline builds it, and the function then asks for it:

 cpp
 
 
// src/Functions/in.cpp:160
 auto set = future_set->get();
 if (!set)
 {
 if (dry_run)
 return ColumnUInt8::create(input_rows_count, static_cast<UInt8>(0));

 throw Exception(ErrorCodes::LOGICAL_ERROR, "Not-ready Set is passed as the second argument for function '{}'", getName());
 }

There are three kinds of FutureSet:

 cpp
 
 
// src/Interpreters/PreparedSets.h:66
class FutureSet
{
public:
 virtual ~FutureSet() = default;

 /// Returns set if set is ready (created and filled) or nullptr if not.
 virtual SetPtr get() const = 0;
 /// Returns set->getElementsTypes(), even if set is not created yet.
 virtual DataTypes getTypes() const = 0;
 /// If possible, return set with stored elements useful for PK analysis.
 virtual SetPtr buildOrderedSetInplace(const ContextPtr & context) = 0;

FutureSetFromTuple — the right-hand side is a literal list, IN (1, 2, 3). It is built during analysis, because the values are already known.
FutureSetFromSubquery — the right-hand side is a SELECT. It is built during execution, by a CreatingSetStep that runs before the steps that use it.
FutureSetFromStorage — the right-hand side is an ENGINE = Set table. The set already exists; the query only reads it.

The literal case is where the “prepare once, use many” idea shows up, because the same literal list may appear many times in a query:

 cpp
 
 
// src/Interpreters/ActionsVisitor.cpp:557
 auto set_key = right_arg->getTreeHash(/*ignore_aliases=*/ true);
 if (auto set = prepared_sets.findTuple(set_key, set_element_keys))
 return set; /// Already prepared.

 return prepared_sets.addFromTuple(set_key, right_arg_func, std::move(block), context->getSettingsRef());

The key is the tree hash of the AST of the right-hand side, so x IN (1,2,3) and y IN (1,2,3) share one set. The prepared sets live in a per-query container:

 cpp
 
 
// src/Interpreters/PreparedSets.h:301
 using SetsFromTuple = std::unordered_map<Hash, std::vector<FutureSetFromTuplePtr>, Hashing>;
 using SetsFromStorage = std::unordered_map<Hash, FutureSetFromStoragePtr, Hashing>;
 using SetsFromSubqueries = std::unordered_map<Hash, FutureSetFromSubqueryPtr, Hashing>;

The hash is 128 bits of CityHash, and the container hashes it down to 64 bits:

 cpp
 
 
// src/Interpreters/PreparedSets.h:278
struct FutureSetHashing
{
 UInt64 operator()(const FutureSet::Hash & key) const { return key.low64 ^ key.high64; }
};

The shape of the thing

 text
 
 
 SELECT ... WHERE x IN (SELECT id FROM t)
 |
 v
 CreatingSetStep ---> FutureSetFromSubquery
 | (built during execution, not analysis)
 v
 ActionsDAG: FunctionIn(x, ColumnSet) ---> Set::execute(key_columns)
 |
 chooseMethod(key_columns) -+--> key8 / key16 / key32 / key64
 | (one integer key: the key IS the slot)
 +--> keys32 / keys64 / keys128 / keys256
 | (fixed-width tuple: packed into one word)
 +--> key_string / key_fixed_string
 | (one string: chars arena + saved hash)
 +--> nullable_keys128 / nullable_keys256
 | (value + null flag packed together)
 +--> hashed
 (anything else: 128-bit hash only)

Choosing the data structure

This is the part that makes ClickHouse’s IN fast. The set’s key type is not “some value” — it is a tuple of columns with concrete types, and each shape deserves a different hash table.

SetVariants::chooseMethod inspects the key columns and picks one of thirteen implementations:

 cpp
 
 
// src/Interpreters/SetVariants.h:417
 #define APPLY_FOR_SET_VARIANTS(M) \
 M(key8) \
 M(key16) \
 M(key32) \
 M(key64) \
 M(key_string) \
 M(key_fixed_string) \
 M(keys32) \
 M(keys64) \
 M(keys128) \
 M(keys256) \
 M(nullable_keys128) \
 M(nullable_keys256) \
 M(hashed)

The decision tree, in order:

One nullable numeric key. The value and the NULL flag are packed into one 16-byte key, so the hash table needs no separate null handling:

 cpp
 
 
// src/Interpreters/SetVariants.cpp:149
 if (has_nullable_key)
 {
 /// At least one key is nullable. Therefore we choose a method
 /// that takes into account this fact.
 if ((keys_size == 1) && (nested_key_columns[0]->isNumeric()))
 {
 /// We have exactly one key and it is nullable. We shall add it a tag
 /// which specifies whether its value is null or not.
 size_t size_of_field = nested_key_columns[0]->sizeOfValueIfFixed();
 if ((size_of_field == 1) || (size_of_field == 2) || (size_of_field == 4) || (size_of_field == 8))
 return Type::nullable_keys128;

One numeric key. A single UInt32 column gets a HashSet<UInt32> — the key is the hash-table key, no packing, no hashing beyond the CRC:

 cpp
 
 
// src/Interpreters/SetVariants.cpp:180
 /// If there is one numeric key that fits into 64 bits
 if (keys_size == 1 && nested_key_columns[0]->isNumeric() && !nested_key_columns[0]->lowCardinality())
 {
 size_t size_of_field = nested_key_columns[0]->sizeOfValueIfFixed();
 if (size_of_field == 1)
 return Type::key8;
 if (size_of_field == 2)
 return Type::key16;
 if (size_of_field == 4)
 return Type::key32;
 if (size_of_field == 8)
 return Type::key64;

Several fixed-width keys. They are packed into 32/64/128/256 bits and the whole packed word is the key:

 cpp
 
 
// src/Interpreters/SetVariants.cpp:199
 /// If the keys fit in N bits, we will use a hash table for N-bit-packed keys
 if (all_fixed && keys_bytes <= 4)
 return Type::keys32;
 if (all_fixed && keys_bytes <= 8)
 return Type::keys64;
 if (all_fixed && keys_bytes <= 16)
 return Type::keys128;
 if (all_fixed && keys_bytes <= 32)
 return Type::keys256;

One string key. A dedicated method with the string’s own arena:

 cpp
 
 
// src/Interpreters/SetVariants.cpp:209
 /// If there is single string key, use hash table of it's values.
 if (keys_size == 1
 && (typeid_cast<const ColumnString *>(nested_key_columns[0])
 || (isColumnConst(*nested_key_columns[0]) && typeid_cast<const ColumnString *>(&assert_cast<const ColumnConst *>(nested_key_columns[0])->getDataColumn())))
 return Type::key_string;

Anything else. The fallback hashes the whole tuple to 128 bits and stores only the hash — the original values are not kept:

 cpp
 
 
// src/Interpreters/SetVariants.cpp:218
 /// Otherwise, will use set of cryptographic hashes of unambiguously serialized values.
 return Type::hashed;

The comment in the header explains the memory trade-off of that last choice:

 cpp
 
 
// src/Interpreters/SetVariants.h:357
 /** Unlike Aggregator, `concat` method is not used here.
 * This is done because `hashed` method, although slower, but in this case, uses less RAM.
 * since when you use it, the key values themselves are not stored.
 */

Presence-only vs counting

There are two flavours of the same set family. A plain IN only needs to know whether a key is present, so it uses a HashSet whose cell maps to VoidMapped. INTERSECT ALL / EXCEPT ALL need how many times a key occurs, so they use a HashMap with a UInt64 counter. The mapping is derived from the table rather than passed in, so a method stays “a set” when given a HashSet and becomes a multiset when given a HashMap:

 cpp
 
 
// src/Interpreters/SetVariants.h:30
/// The method structs below are shared between presence-only sets (a `HashSet`, whose cell reports
/// `VoidMapped`) and counting multisets (a `HashMap`, with a real mapped type, used by INTERSECT
/// ALL / EXCEPT ALL). The mapped type is derived from the table rather than passed explicitly, so a
/// method stays "a set" when given a `HashSet` and becomes a counting multiset when given a `HashMap`.
template <typename TData>
using SetMethodMapped = std::conditional_t<std::is_same_v<typename TData::mapped_type, VoidMapped>, void, typename TData::mapped_type>;

The choice of table type per key shape is visible in the variant declarations:

 cpp
 
 
// src/Interpreters/SetVariants.h:344
 std::unique_ptr<SetMethodOneNumber<UInt32, HashSet<UInt32, HashCRC32<UInt32>>>> key32;
 std::unique_ptr<SetMethodOneNumber<UInt64, HashSet<UInt64, HashCRC32<UInt64>>>> key64;
 std::unique_ptr<SetMethodString<HashSetWithSavedHash<std::string_view>>> key_string;
 std::unique_ptr<SetMethodFixedString<HashSetWithSavedHash<std::string_view>>> key_fixed_string;
 std::unique_ptr<SetMethodKeysFixed<HashSet<UInt32, HashCRC32<UInt32>>>> keys32;
 std::unique_ptr<SetMethodKeysFixed<HashSet<UInt64, HashCRC32<UInt64>>>> keys64;
 std::unique_ptr<SetMethodKeysFixed<HashSet<UInt128, UInt128HashCRC32>>> keys128;
 std::unique_ptr<SetMethodKeysFixed<HashSet<UInt256, UInt256HashCRC32>>> keys256;
 std::unique_ptr<SetMethodHashed<HashSet<UInt128, UInt128TrivialHash>>> hashed;

HashSetWithSavedHash is worth a note: for strings it stores the hash next to the string, so a probe that hits the same bucket doesn’t have to re-hash the candidate key.

Building the set

Insertion goes through emplaceKey on the method’s state, one row at a time, with a twist: the same loop can build a deduplication filter, which records which rows were new. That filter is how the set keeps the original values around for primary-key analysis:

 cpp
 
 
// src/Interpreters/Set.cpp:370
 typename Method::State state(key_columns, key_sizes, nullptr);

 /// For all rows
 for (size_t i = 0; i < rows; ++i)
 {
 if constexpr (has_null_map)
 {
 if ((*null_map)[i])
 {
 if constexpr (build_filter)
 {
 (*out_filter)[i] = false;
 }
 continue;
 }
 }

 [[maybe_unused]] auto emplace_result = state.emplaceKey(method.data, i, variants.string_pool);

 if constexpr (build_filter)
 (*out_filter)[i] = emplace_result.isInserted();
 }

emplace_result.isInserted() is the deduplication: a row whose key was already in the set is not new, so it gets filtered out of the “explicit set elements” — the copy of the values kept for index analysis.

Keeping the values is optional and bounded. When the set grows past max_elements_to_fill, the copy is dropped and the set keeps only the hashes:

 cpp
 
 
// src/Interpreters/Set.cpp:495
 bool inserted = insertFromColumnsImpl(columns, holder);
 if (inserted && fill_set_elements)
 {
 if (max_elements_to_fill && max_elements_to_fill < data->getTotalRowCount())
 {
 /// Drop filled elementes
 fill_set_elements = false;
 set_elements.clear();
 }
 else
 appendSetElements(holder);
 }

That’s a graceful degradation: the set still answers IN correctly, it just can no longer be used to prune granules by primary key. The threshold is the setting use_index_for_in_with_subqueries_max_values.

Probing the set

The probe is the hot path: it runs once per row of the left-hand side.

 cpp
 
 
// src/Interpreters/Set.cpp:1002
 Arena pool;
 typename Method::State state(key_columns, key_sizes, nullptr);

 /// Clustered key columns (e.g. a primary key prefix) arrive in runs of equal consecutive
 /// rows. The consecutive-keys optimization in ColumnsHashing handles them inside `findKey`:
 /// the last-element cache compares the key with the previous row's before probing the hash
 /// table, and `HashMethodHashed` additionally compares the raw key bytes before even
 /// calculating the hash.
 for (size_t i = 0; i < rows; ++i)
 {
 if (has_null_map && (*null_map)[i])
 {
 vec_res[i] = negative;
 }
 else
 {
 auto find_result = state.findKey(method.data, i, pool);
 vec_res[i] = negative ^ find_result.isFound();
 }
 }

Two things worth noticing.

The “last element cache”. When the input is sorted (or clustered by a primary key), consecutive rows often have the same key. The hashing state remembers the last key and its answer, so a repeat costs one comparison instead of a hash computation plus a probe. This optimisation is on for numeric keys:

 cpp
 
 
// src/Interpreters/SetVariants.h:44
template <typename FieldType, typename TData, bool use_cache = true> /// UInt8/16/32/64 for any types with corresponding bit width.
struct SetMethodOneNumber
{
 using Data = TData;
 using Key = typename Data::key_type;

 Data data;

 using State = ColumnsHashing::HashMethodOneNumber<typename Data::value_type,
 SetMethodMapped<Data>, FieldType, set_method_use_cache<Data, use_cache>>;

and off for strings, with a measurement in the comment:

 cpp
 
 
// src/Interpreters/SetVariants.h:62
/// For the case where there is one string key.
/// Unlike aggregation, the consecutive-keys cache stays disabled for string keys: a set probe's
/// per-row work is just hash+lookup, which for strings is cheaper than the extra bytewise
/// comparison the cache check performs on every miss. Measured 2026-08 on 20M-row `IN` probes:
/// enabling the cache made unclustered inputs 40% slower for short `String` keys and 24% slower
/// for `FixedString(36)` keys, while clustered inputs (runs of equal consecutive keys) improved
/// only 8-16%. In aggregation the same check is relatively cheap because the per-row work there
/// (aggregate-state updates) is much heavier.
template <typename TData>
struct SetMethodString
{
 using Data = TData;
 using Key = typename Data::key_type;

 Data data;

 using State = ColumnsHashing::HashMethodString<typename Data::value_type, SetMethodMapped<Data>, true, false>;

That is a rare and welcome thing to find in a codebase: an optimisation that was measured, found harmful for one type family, and turned off — with the numbers kept in the source. Most codebases just leave the “optimisation” in and wonder why strings are slower than integers.

The NULL rule. A row whose key contains a NULL is never in the set, so the answer is negative (0 for IN, 1 for NOT IN) without a lookup. The rationale is in the header:

 cpp
 
 
// src/Interpreters/Set.h:202
 /** How IN works with Nullable types.
 *
 * For simplicity reasons, all NULL values and any tuples with at least one NULL element are ignored in the Set.
 * And for left hand side values, that are NULLs or contain any NULLs, we return 0 (means that element is not in Set).
 *
 * If we want more standard compliant behaviour, we must return NULL
 * if lhs is NULL and set is not empty or if lhs is not in set, but set contains at least one NULL.
 * It is more complicated with tuples.
 * For example,
 * (1, NULL, 2) IN ((1, NULL, 3)) must return 0,
 * but (1, NULL, 2) IN ((1, 1111, 2)) must return NULL.
 *
 * We have not implemented such sophisticated behaviour.
 */

So NULL IN (1, 2, 3) is 0, not NULL. A deliberate deviation from the SQL standard, documented in the source. Gotcha if you are porting queries from a stricter engine.

Type coercion on the probe

The left-hand side of the IN may have a different type from the set’s key — the set’s type wins, and the probe column is cast. The cast is cached, so it isn’t rebuilt for every block:

 cpp
 
 
// src/Interpreters/Set.h:249
 /// A cache for cast functions (if any) to avoid rebuilding cast functions
 /// for every call to `execute`
 mutable std::unique_ptr<InternalCastFunctionCache> cast_cache;

and when the value does not fit the key type, it is not an error — it is simply “not a member”:

 cpp
 
 
// src/Interpreters/Set.cpp:833
 if (can_convert_leniently)
 {
 /// `transform_null_in` makes NULL an ordinary matchable value, so the accurate-or-null
 /// cast above is not taken. A value the key type cannot represent is still simply not a
 /// member - which is what the literal `IN` list concludes, because it compares in a
 /// common supertype - but converting it strictly raised `CANNOT_CONVERT_TYPE` on rows
 /// the read path happened to deliver, so `v IN set_table` failed where `v IN (1)`
 /// returned 0.

Size limits and overflow

The set is subject to max_rows_in_set and max_bytes_in_set, checked after every inserted chunk:

 cpp
 
 
// src/Interpreters/Set.cpp:556
 bool within_limits = limits.check(data->getTotalRowCount(), data->getTotalByteCount(), "IN-set", ErrorCodes::SET_SIZE_LIMIT_EXCEEDED);
 if (!within_limits)
 is_truncated = true;

The overflow mode is set_overflow_mode: throw (the default) aborts the query, break truncates the set and returns the partial result — which is wrong for IN, but is the only sane option for a GLOBAL IN over a huge table, where a partial set is still useful.

Spilling to disk

A set built from a subquery can be enormous: x IN (SELECT id FROM big_table) over a billion rows. ClickHouse 26.10 added spilling: the set moves to a temporary file, and the in-memory part shrinks to a directory plus a set of Bloom filters.

The disk representation is not the original values. Each key is converted into a compact disk key: numbers and FixedStrings of at most 32 bytes are packed into integers, and everything else becomes a 128-bit hash:

 cpp
 
 
// src/Interpreters/Set.h:85
 /// the keys that come first in the order of their disk keys rather than the first ones to arrive. Disk
 /// keys keep the order of the keys that the set packs into integers in memory, numbers and `FixedString`
 /// values of at most 32 bytes in total, with `NULL` after the values, so these are the smallest keys.
 /// Other keys, such as `String`, are stored as 128-bit hashes, whose order is arbitrary.

The keys are then sorted, so a lookup can binary-search the directory and find the block that may contain the key.

The Bloom filters

Each block of the set gets its own Bloom filter, and all the filters share one array of bits:

 cpp
 
 
// src/Interpreters/DiskSetBlockFilters.h:15
/// Bloom filters of the blocks of a set on disk, which the set keeps in memory. A lookup checks the filter of
/// the only block that can hold its key, and reads the block only when the filter lets the key pass.
///
/// The filters share one array of bits. Each block takes a range of it in proportion to its keys, and each
/// key sets a few bits in the range of its block. A single filter for all keys would take as much memory, but
/// a key would set bits anywhere in it, leading to writes at random places all over the array. For a large
/// set, the array is far larger than the cache, so each such write would miss the cache and the TLB, and
/// these misses would slow down writing the set, because the set fills the filters on the single thread that
/// writes its blocks. The range of a block is about as small as the block, so it stays in the cache while the
/// set writes the block. Lookups check the ranges in the order of the blocks, as they read the blocks.

That’s a lovely piece of engineering: the filters are per block so a write touches only a cache-resident range, not the whole (possibly huge) filter array. Think of it as giving each block its own little padlock instead of one giant lock everyone fights over.

The filter is sized by the number of distinct keys, which is estimated with a HyperLogLog counter while the set is being filled:

 cpp
 
 
// src/Interpreters/DiskSetBlockFilters.h:29
 /// Counts the distinct keys of a set as they arrive, so that the bloom filters can be sized before the set
 /// writes its blocks. With 2^20 buckets, the estimate is within about 0.1% and the counter takes only 768 KiB.
 class DistinctKeyCounter
 {
 public:
 void add(std::span<const Key> keys)
 {
 for (const Key & key : keys)
 counter.insert(hashKey(key));
 }

and the bits-per-key is chosen to fit the memory budget, trading false positives for memory:

 cpp
 
 
// src/Interpreters/DiskSetBlockFilters.h:46
 /// Creates the filters for about `expected_keys` keys. They take 8 bits per key, which lets about 3% of
 /// the keys that the set does not have pass. When that does not fit in `max_bytes` bytes with the ranges
 /// of the blocks, they take fewer bits per key, so that they fit: with 2 bits per key, about 40% of these
 /// keys pass, and with 1 bit per key, about 63%. Every block but the last holds at least
 /// `min_keys_per_block` keys, which bounds the number of ranges. Returns nothing when fewer than 1 bit
 /// per key fits, because such filters let so many keys pass that they do not pay for filling them.

The number of hash positions per key is capped at 3, with the reasoning spelled out:

 cpp
 
 
// src/Interpreters/DiskSetBlockFilters.h:82
 /// A key sets the bits at most at this many positions in the range of its block. Each position costs a
 /// memory access when a lookup checks a key that the set has, while more positions let only slightly
 /// fewer other keys pass: with 8 bits per key, 3 positions let about 3.1% of them pass, and 6 positions,
 /// the most effective number, about 2.2%.
 static constexpr size_t max_positions_per_key = 3;

The lookup

A lookup against the disk set is a five-step algorithm: collect the probes, sort them, find the blocks, consult the filters, read the blocks.

 cpp
 
 
// src/Interpreters/DiskSet.cpp:121
 /// Step 1: Collect the probes: the rows whose keys are in the range of the set, except a row that repeats
 /// the key of the previous row, which takes that row's answer in step 5. So a run of equal keys, as in a
 /// sorted column, takes one probe.
 struct Probe
 {
 Key key;
 size_t row;
 };

 VectorWithMemoryTracking<Probe> probes;
 probes.reserve(keys.size());
 const Key & first_key = directory.getFirstKey(0);
 for (size_t row = 0; row < keys.size(); ++row)
 {
 const auto & key = keys[row];
 if ((row == 0 || key != keys[row - 1]) && first_key <= key && key <= last_key)
 probes.push_back({key, row});
 }

Note the range check first_key <= key && key <= last_key: keys outside the set’s key range cannot be in the set, and are answered without any I/O.

Then the probes are sorted, so that each block is read once and in order:

 cpp
 
 
// src/Interpreters/DiskSet.cpp:143
 /// Step 2: Sort the probes by key. Probes of equal keys then come together, so each distinct key is
 /// looked up once, and the probes of each block come together in block order, so they share a read.
 /// Results go back to the original rows, so evaluating `IN` does not reorder the input.
 ::sort(probes.begin(), probes.end(), [](const Probe & lhs, const Probe & rhs) { return lhs.key < rhs.key; });

and the filters decide which blocks are actually read:

 cpp
 
 
// src/Interpreters/DiskSet.cpp:174
 for (size_t begin = 0; begin < probes.size();)
 {
 size_t end = begin + 1;
 while (end < probes.size() && probes[end].key == probes[begin].key)
 ++end;

 block = directory.findBlock(probes[begin].key, block);
 if (!filters || filters->mayContain(block, probes[begin].key))
 {
 if (lookups.empty() || block != lookups.back().block)
 needed_blocks.push_back(block); /// Each block once, in ascending order
 lookups.push_back({block, begin, end});
 }
 else
 {
 for (size_t i = begin; i < end; ++i)
 skipped_rows += count_rows_of_probe(probes[i]);
 }
 begin = end;
 }

The number of rows the filters answered is a profile event, so the effectiveness of the filters is directly observable:

 cpp
 
 
// src/Interpreters/DiskSet.cpp:195
 /// A skipped key counts once for each of its rows, as `ExternalSetLookupKeys` does, so that the two
 /// events give the share of the rows that the filters answer.
 ProfileEvents::increment(ProfileEvents::ExternalSetBloomFilterSkippedKeys, skipped_rows);

When does it spill?

The threshold is a memory budget, and the default is “half of the available memory”:

 cpp
 
 
// src/Core/Settings.cpp:3905
 DECLARE(Double, max_bytes_ratio_before_external_set, 0.5, R"(
Fraction of available server or user memory used to calculate the threshold for spilling the set of `IN`
with a subquery to disk, at the start of execution. For example, `0.5` uses half of the available memory.

The set spills while it is built and while it is used, and the two cases are handled differently. While it is built, the keys are handed to a DiskSetBuilder that sorts them into runs; while it is used, the keys are written out while the lookups keep reading the in-memory table, and the disk set replaces it at the end.

The bloom filter has its own budget:

 cpp
 
 
// src/Core/Settings.cpp:3917
 DECLARE(UInt64, max_bytes_for_external_set_bloom_filter, 128_MiB, R"(
Memory budget, in bytes, for the Bloom filter that each set of `IN` written to disk keeps in memory. The
filter lets lookups of values that are not in the set skip reading the disk. It does not speed up lookups
of values that are in the set.

Note the asymmetry in that sentence: the filter helps for misses, not for hits. A key that is in the set still costs a block read. That’s the fundamental trade of a Bloom filter, and it’s why the filter is only worth it when the selectivity is low.

Trade-offs

What it buys: IN over a subquery is a hash-table probe per row, with a data structure chosen to fit the key type exactly. No row-at-a-time interpretation, no generic comparison, and for the common cases (one integer key, one string key) the key is the hash-table key.

What it costs:

Memory. The set is proportional to the number of distinct values on the right-hand side. x IN (SELECT id FROM orders) with 100 M distinct ids is a 100 M-entry hash table. This is why the spilling machinery exists, and why max_rows_in_set exists.
Spilling is much slower than RAM. A probe against a disk set is a sort, a binary search, a Bloom filter check and possibly a block read. ClickHouse sorts the probe keys to make the reads sequential, which is a lot of work per block. The alternative — failing the query with SET_SIZE_LIMIT_EXCEEDED — is often better for the user.
The hash fallback loses information. For a heterogeneous key, the set stores a 128-bit hash of the tuple, not the tuple. Two different tuples with the same hash would be “equal”; the probability is negligible, but it is not zero.
NULL semantics deviate from the standard, as documented above.
The hashed method is slower per probe than the packed methods, because the key must be serialised and hashed on every probe rather than once at build time. For a set of strings, the string method stores the string and its hash; the hashed method stores only the hash.

That last point is the general lesson of the whole design: ClickHouse’s IN is fast not because hashing is fast, but because the shape of the data is known at analysis time and the implementation specialises on it — thirteen variants, each with its own hash function, its own packing, its own arena, and its own micro-optimisations, some of which are enabled and some of which are deliberately disabled after measurement.

Wrapping up

The one thing to remember: IN is a hash set, and the set’s data structure is chosen from the type of the right-hand side at analysis time. That is why IN (1,2,3) on an integer column is essentially free.

What to go do about it: if you have a query with IN (SELECT ...) over a big table, check whether the right-hand side has an index or a small enough cardinality. If neither, the hash set is going to be the bottleneck — and spilling it to disk makes it worse, not better.

The part people get wrong: a Bloom filter in front of a disk set only helps with misses. If your IN mostly hits, the filter is pure overhead.

ClickHouse Estimates Distinct Counts by Thinning a Hash Table

Jay Hubbert — Wed, 21 Oct 2026 09:00:00 +0000

TL;DR: uniq() is a hash table that deliberately throws data away. When it fills up, it halves its sample rate, keeps the survivors, and multiplies the count back up by the thinning factor. The state stays at a few hundred KiB no matter how many distinct values you feed it.

uniq() is one of the most-used aggregate functions in ClickHouse, and one of the most surprising. It is not exact. It is not a HyperLogLog. It is a hash table that throws data away on purpose, thinning itself as it fills up, and then multiplies back up by the thinning factor to estimate the cardinality.

The algorithm behind it is BJKST (Bretag, Karp, Kutz, Schlossmann — “near-optimal algorithms for estimating the number of distinct elements in a stream of data”). The implementation lives in src/AggregateFunctions/UniquesHashSet.h, and it is one of the more elegant pieces of code in the aggregation framework: a fixed-size table, a single integer of state, and a mathematically exact estimator of what the sample means.

The scheme

The header states the general recipe for approximate aggregation:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:27
/** Approximate calculation of anything, as usual, is constructed according to the following scheme:
 * - some data structure is used to calculate the value of X;
 * - Not all values are added to the data structure, but only selected ones (according to some selectivity criteria);
 * - after processing all elements, the data structure is in some state S;
 * - as an approximate value of X, the value calculated according to the maximum likelihood principle is returned:
 * at what real value X, the probability of finding the data structure in the obtained state S is maximal.
 */

And then the specific recipe:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:38
/** Very simple hash-set for approximate number of unique values.
 * Works like this:
 * - you can insert UInt64;
 * - before insertion, first the hash function UInt64 -> UInt32 is calculated;
 * - the original value is not saved (lost);
 * - further all operations are made with these hashes;
 * - hash table is constructed according to the scheme:
 * - open addressing (one buffer, position in buffer is calculated by taking remainder of division by its size);
 * - linear probing (if the cell already has a value, then the cell following it is taken, etc.);
 * - the missing value is zero-encoded; to remember presence of zero in set, separate variable of type bool is used;
 * - buffer growth by 2 times when filling more than 50%;
 * - if the set has more than UNIQUES_HASH_MAX_SIZE elements, then all the elements are removed from the set,
 * not divisible by 2, and then all elements that do not divide by 2 are not inserted into the set;
 * - if the situation repeats, then only elements dividing by 4, etc., are taken.
 * - the size() method returns an approximate number of elements that have been inserted into the set;
 * - there are methods for quick reading and writing in binary and text form.
 *
 * A 32-bit hash cannot distinguish more than a few billion elements: the estimate degrades quickly
 * beyond a billion, and above ~30 billion the correction formula even resulted in undefined behavior
 * (issue #6078). To keep the estimate accurate up to trillions, in addition to the main set a small
 * "wide" set of full 64-bit hashes (XOR-ed with a constant, see `wideHash`) is maintained (state version 1).
 * It is thinned out aggressively from the very beginning - only values with UNIQUES_HASH_SET_WIDE_INITIAL_SKIP_DEGREE
 * zero low-order bits enter it - so it stays empty or tiny for the states where the main set is accurate
 * (about 4 values per million distinct elements), and it is bounded by UNIQUES_HASH_SET_WIDE_MAX_SIZE
 * values thinned out further. Once the wide set has enough values for its own estimate to be reliable
 * (which happens at about two billion elements), size() switches to it.
 *
 * The wide set has to be maintained from the first insertion: the high bits of the hashes
 * of the already consumed elements cannot be recovered later. In particular, a state written
 * in the legacy format (state version 0) carries no wide set, and its elements were never
 * offered to one. Such a state is marked as having an incomplete wide sample: it drops its own
 * wide set, never starts a new one, and propagates the mark through merges, so that size()
 * of anything it is merged into falls back to the legacy 32-bit estimator. Estimating from
 * a wide set that covers only a part of the elements would undercount the rest entirely,
 * which is much worse than the degrading precision of the 32-bit estimate.
 */

That comment is the whole algorithm. Read it twice: the key insight is that the set does not grow without bound — it changes what it accepts.

The constants

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:75
/// The maximum degree of buffer size before the values are discarded
#define UNIQUES_HASH_MAX_SIZE_DEGREE 17

/// The maximum number of elements before the values are discarded
#define UNIQUES_HASH_MAX_SIZE (1ULL << (UNIQUES_HASH_MAX_SIZE_DEGREE - 1))

/** The number of least significant bits used for thinning. The remaining high-order bits are used to determine the position in the hash table.
 * (high-order bits are taken because the younger bits will be constant after dropping some of the values)
 */
#define UNIQUES_HASH_BITS_FOR_SKIP (32 - UNIQUES_HASH_MAX_SIZE_DEGREE)

/// Initial buffer size degree
#define UNIQUES_HASH_SET_INITIAL_SIZE_DEGREE 4

So the main set holds at most 2^16 = 65 536 hashes in a buffer of 2^17 slots, and 15 low bits of the hash are reserved for the thinning mask. The remaining 17 high bits select the cell:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:195
 size_t place(HashValue x) const { return (x >> UNIQUES_HASH_BITS_FOR_SKIP) & mask(); }

 /// The value is divided by 2 ^ skip_degree
 bool good(HashValue hash) const { return hash == ((hash >> skip_degree) << skip_degree); }

good() is the thinning test: the low skip_degree bits of the hash must be zero. The high bits pick the position, because the low bits are constant across all surviving elements — using them for the position would pile every element into the same cell.

Thinning

The state is three integers plus a flag:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:155
 UInt32 m_size; /// Number of elements
 UInt8 size_degree{}; /// The size of the table as a power of 2
 UInt8 skip_degree; /// Skip elements not divisible by 2 ^ skip_degree
 bool has_zero; /// The hash table contains an element with a hash value of 0.

And the moment of truth is shrinkIfNeed:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:346
 /** If the hash table is full enough, then do resize.
 * If there are too many items, then throw half the pieces until they are small enough.
 */
 void shrinkIfNeed()
 {
 if (unlikely(m_size > max_fill()))
 {
 if (m_size > UNIQUES_HASH_MAX_SIZE)
 {
 while (m_size > UNIQUES_HASH_MAX_SIZE)
 {
 if (unlikely(skip_degree >= max_skip_degree))
 throw DB::Exception(DB::ErrorCodes::INCORRECT_DATA,
 "Cannot thin out UniquesHashSet: {} elements remain at the maximum skip degree of {}",
 static_cast<size_t>(m_size), static_cast<size_t>(max_skip_degree));

 ++skip_degree;
 rehash();
 }
 }
 else
 resize();
 }
 }

So: the set grows normally until it holds more than 65 536 elements. Then it does not grow — it thins. skip_degree goes up by one, and every element whose hash is not divisible by 2^skip_degree is deleted:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:200
 /// Delete all values whose hashes do not divide by 2 ^ skip_degree
 void rehash()
 {
 for (size_t i = 0; i < buf_size(); ++i)
 {
 if (buf[i])
 {
 if (!good(buf[i]))
 {
 buf[i] = 0;
 --m_size;
 }
 /** After removing the elements, there may have been room for items,
 * which were placed further than necessary, due to a collision.
 * You need to move them.
 */
 else if (i != place(buf[i]))
 {
 HashValue x = buf[i];
 buf[i] = 0;
 reinsertImpl(x);
 }
 }
 }

The picture is this:

 text
 
 
 distinct values arriving: A B C D E F G H ... (millions)

 skip_degree = 0 -> accept everything, table grows
 table full at 65536 elements, distinct count so far maybe 200 000
 skip_degree = 1 -> accept only hashes with bit0 == 0 (half of them)
 table full again at 65536, distinct count so far maybe 400 000
 skip_degree = 2 -> accept only hashes with bits1..0 == 0 (a quarter)
 ...
 skip_degree = k -> accept 1 in 2^k elements

Each thinning step halves the sample rate, and the sample size stays roughly constant. The estimate multiplies the surviving count back up by 2^skip_degree.

Here’s the thing: the table does not need to know how many distinct values it has seen. It only needs to know how aggressively it is currently sampling. The count falls out of that at the end.

The estimator

The estimator is not simply m_size * 2^skip_degree — there is a correction for the systematic error of 32-bit hash collisions, and a pseudo-random remainder so that the result does not visibly sit on a power of two:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:842
 if (0 == skip_degree)
 return m_size;

 size_t res = m_size * (1ULL << skip_degree);

 /** Pseudo-random remainder - in order to be not visible,
 * that the number is divided by the power of two.
 */
 res += (intHashCRC32(m_size) & ((1ULL << skip_degree) - 1));

 /** Correction of a systematic error due to collisions during hashing in UInt32.
 * `fixed_res(res)` formula
 * - with how many different elements of fixed_res,
 * when randomly scattered across 2^32 buckets,
 * filled buckets with average of res is obtained.
 */
 return correctFor32BitCollisions(res);

The pseudo-random remainder is a lovely touch of honesty. Without it, every estimate would be a multiple of a power of two, and users would (correctly!) assume the result was rounded. It only costs one CRC and one mask.

The collision correction is an analytic formula for the birthday problem. If res distinct elements are scattered over 2^32 buckets, the number of occupied buckets is less than res because of collisions; to recover the number of distinct elements from the number of occupied buckets, you invert that relation:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:548
 /** With how many different elements, when randomly scattered across 2^32 buckets,
 * the given number of non-empty buckets is obtained on average.
 * This is the correction of the systematic error due to collisions of the 32-bit hashes.
 */
 static size_t correctFor32BitCollisions(size_t res)
 {
 size_t p32 = 1ULL << 32;

 /** When the number of distinct elements is much larger than 2^32, almost all buckets are filled
 * and `res` may reach (or, due to the pseudo-random remainder, exceed) 2^32,
 * where the formula below is not defined (previously this resulted in undefined behavior
 * when casting infinity or NaN to an integer). Saturate the estimate instead:
 * 32 bits of a hash contain no information about larger cardinalities anyway
 * (and the wide set takes over long before this point).
 */
 if (res >= p32)
 res = p32 - 1;

 return static_cast<size_t>(round(static_cast<double>(p32) * (log(static_cast<double>(p32)) - log(static_cast<double>(p32 - res)))));
 }

Read the formula as p32 * (ln p32 - ln(p32 - res)). For small res it is approximately res; for res near p32 it blows up, which is why the saturation guard exists — and that guard is the fix for issue #6078, where the cast of inf to an integer was undefined behaviour.

The wide set: fixing the 32-bit ceiling

The 32-bit hash has a hard ceiling. At a few billion distinct elements the correction formula is already straining, and above ~30 billion it was producing garbage. The fix, added in state version 1, is a second set that stores full 64-bit hashes — but only a tiny, heavily thinned sample of them.

The sample is defined by a skip degree of 18: only hashes with 18 zero low-order bits enter it, i.e. one in 262 144:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:89
/** Only hashes with this many zero low-order bits enter the wide set, so that it stays
 * negligibly small while the main set is accurate: one hash in 262144 passes.
 */
#define UNIQUES_HASH_SET_WIDE_INITIAL_SKIP_DEGREE 18

/// The maximum degree of the wide set buffer size before the values are thinned out further.
#define UNIQUES_HASH_SET_WIDE_MAX_SIZE_DEGREE 15

/// The maximum number of elements of the wide set: 16384 values take 256 KiB.
#define UNIQUES_HASH_SET_WIDE_MAX_SIZE (1ULL << (UNIQUES_HASH_SET_WIDE_MAX_SIZE_DEGREE - 1))

/** The estimate is taken from the wide set when it has at least this many values, i.e. starting
 * from about UNIQUES_HASH_SET_WIDE_MIN_SIZE_FOR_ESTIMATE * 2^UNIQUES_HASH_SET_WIDE_INITIAL_SKIP_DEGREE,
 * about two billion elements: at that point the statistical error of the wide estimate (about 1%)
 * meets the degrading precision of the 32-bit one.
 */
#define UNIQUES_HASH_SET_WIDE_MIN_SIZE_FOR_ESTIMATE 8192

So the wide set holds at most 16 384 hashes (256 KiB) and needs 8192 of them before it is trusted. Since only one in 262 144 elements enters it, 8192 elements means about 2.1 billion distinct values have been seen — exactly the point where the 32-bit estimator starts to degrade.

The wide set has one more twist: the hashes are XOR-ed with a constant before being stored.

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:518
 /** The wide set samples the full hashes XOR-ed with this constant rather than the hashes themselves.
 * The sample stays just as uniform, but the hash of a frequent value never becomes a candidate
 * systematically: the default hash of the integer 0 is 0, and zeros are frequent in real data -
 * without the offset every such row would be a candidate for the wide set and would cost
 * an extra pass over its batch (this doubled the time of `uniq` over such columns).
 * The low bits up to the maximum thinning of the main set are zero, so that every candidate
 * still passes good() of the main set (see `widePrefilterMask`), and the next ones are not,
 * so that the hash 0 is never a candidate.
 */
 static constexpr UInt64 wide_hash_offset = 0x9E3779B97F4A0000ULL;

The constant is the golden-ratio fraction used in many hash mixers (0x9E3779B97F4A0000 is 2^64 / φ). The problem it solves is delightfully concrete: the default hash of the integer 0 is 0, and the wide-set candidate test is “are the low 18 bits zero?”. The hash 0 passes that test — so every zero row in the data would enter the wide set, and the extra work doubled the runtime of uniq over columns full of zeros.

XOR-ing with an offset whose low bits are non-zero removes that pathology while preserving uniformity.

The offset has to satisfy two constraints, and they are checked at compile time:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:528
 static_assert((wide_hash_offset & ((1ULL << max_skip_degree) - 1)) == 0);
 static_assert((wide_hash_offset & ((1ULL << UNIQUES_HASH_SET_WIDE_INITIAL_SKIP_DEGREE) - 1)) != 0);

The first says: the offset’s low bits are zero up to the maximum thinning degree of the main set, so that a wide candidate still passes the main set’s good() test — the wide set is a subset of the main sample. The second says: the offset’s low 18 bits are not all zero, so the hash 0 is never a wide candidate.

Two constraints, two static_asserts. This is the good stuff: an invariant expressed as a compile-time proof instead of a comment.

The wide set is allocated lazily, in a single block holding both the header and the table:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:373
 static WideSet * allocWideSet(UInt8 new_size_degree)
 {
 /// The header and the table are placed in a single allocation.
 size_t bytes = sizeof(WideSet) + (1ULL << new_size_degree) * sizeof(UInt64);
 WideSet * res = new (HashTableAllocator().alloc(bytes)) WideSet;
 res->size_degree = new_size_degree;
 res->buf = reinterpret_cast<UInt64 *>(reinterpret_cast<char *>(res) + sizeof(WideSet));
 return res;
 }

and it is only allocated when the first candidate shows up — which, for most data, never happens:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:469
 if (!wide)
 wide = allocWideSet(UNIQUES_HASH_SET_INITIAL_SIZE_DEGREE);

Which estimator is used

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:820
 size_t size() const
 {
 /** The wide set gives a reliable estimate once it has enough values;
 * below that the 32-bit main set is more precise. If the wide sample does not cover
 * all the elements (see `wide_incomplete`), it is not there at all and the legacy
 * estimator is used, because a partial wide estimate would silently lose the rest.
 */
 if (wide && wide->m_size >= UNIQUES_HASH_SET_WIDE_MIN_SIZE_FOR_ESTIMATE)
 {
 size_t res = static_cast<size_t>(wide->m_size) * (1ULL << wide->skip_degree);

 /** Pseudo-random remainder - in order to be not visible,
 * that the number is divided by the power of two.
 */
 res += (intHashCRC32(wide->m_size) & ((1ULL << wide->skip_degree) - 1));

 /** Collisions of 64-bit hashes are negligible: their systematic error would reach
 * just 0.003% at 10^15 distinct elements, far below the statistical error of the estimate.
 */
 return res;
 }

Note the last comment: at 10^15 distinct elements, the systematic error of 64-bit hash collisions would be 0.003%, far below the statistical error of the estimate itself. So the wide set is not exact either — it is exact enough.

The hot loop: batching

The insertion path is where the performance lives, because uniq is usually applied to millions of rows. The implementation does not insert one hash at a time; it computes eight hashes and eight target cells first, then inserts them:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:749
 size_t i = 0;
 while (i < size)
 {
 /// Check that we have enough capacity and enough values to insert in batch. If not, we will insert values one by one which
 /// allows us to check the shrink condition after each insert and avoid inserting too many values before shrinking.
 if ((max_fill() - m_size) >= insert_many_batch_size && ((size - i) >= insert_many_batch_size))
 {
 /// We read and transform multiple values at once which allows both the compiler and the CPU to better optimize the code.
 /// We calculate place() even for !good() hashes to maximize data independence and enable better out-of-order execution.
 /// The extra work is negligible compared to the instruction level parallelization benefits.
 std::array<HashValue, insert_many_batch_size> hash_value; // NOLINT(cppcoreguidelines-pro-type-member-init,hicpp-member-init) - filled by the loop below before read
 for (size_t j = 0; j < insert_many_batch_size; ++j)
 {
 hash_value[j] = hash(Transform(data[i + j]));
 }

with the batch size fixed at eight:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:665
 /// This value is arbitrary. The optimal value might depend on the CPU pipeline depth, cache line size (64B = 8 UInt64s),
 /// hash function quality, collision rate, instruction parallelism, etc.
 /// We choose a value that is big enough to provide sufficient instruction level parallelism but not too big to bloat the code size.
 static constexpr size_t insert_many_batch_size = 8;

Two details worth noticing. place() is computed even for hashes that will be rejected by good(), on purpose: it keeps the loop free of data-dependent branches and lets the out-of-order engine overlap the work. And the wide-set candidates are collected into a small local buffer during the loop and inserted after it, because inserting them inside would force the compiler to reload the set’s fields from memory on every iteration:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:738
 /** The candidates for the wide set (see the comment at `widePrefilterMask`) are detected
 * behind the `good()` test, which every one of them passes and which already rejects all
 * but one value in 2^skip_degree, and they are only collected into a local buffer there.
 * They are inserted after the loop rather than inside it: a call inside would make
 * the compiler cautiously reload the fields of the set from memory all the time.
 * This loop is very sensitive to the code generation: the batch loops below must be
 * fully unrolled as they are without the wide set, otherwise `uniq` gets up to 50% slower.
 */

“otherwise uniq gets up to 50% slower” is a great reminder that in an aggregation hot loop, the code layout is part of the algorithm.

What is hashed

The value that goes into the set is not the value itself but a 64-bit hash of it, of which only the low 32 bits feed the main set:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:637
 /// The full 64-bit hash: its low 32 bits feed the main set, and the whole value the wide set (see `wideHash`).
 static UInt64 fullHash(Value key) { return static_cast<UInt64>(Hash()(key)); }
 static HashValue hash(Value key) { return static_cast<HashValue>(fullHash(key)); }

and the per-type hashing is chosen in the aggregate function:

 cpp
 
 
// src/AggregateFunctions/AggregateFunctionUniq.h:273
template <typename T, typename ColumnType>
struct AggregateFunctionUniqTraits
{
 static ALWAYS_INLINE auto hash(T x)
 {
 if constexpr (std::is_same_v<T, std::string_view>)
 {
 return CityHash_v1_0_2::CityHash64(x.data(), x.size());
 }
 else if constexpr (std::is_same_v<T, IPv6>)
 {
 /// For compatibility reasons IPv6 is hashed as FixedString(16).
 return CityHash_v1_0_2::CityHash64(reinterpret_cast<const char *>(&x), sizeof(x));
 }
 else if constexpr (is_floating_point<T>)
 {
 return bit_cast<UInt64>(x);
 }
 else if constexpr (sizeof(T) <= sizeof(UInt64))
 {
 return x;
 }
 else
 {
 return DefaultHash64<T>(x);
 }
 }

Note that for integers the “hash” is the identity: an integer’s low bits are used directly, which is fine because the thinning test only looks at the low 15..18 bits and the position comes from the high bits. For sequential integers the high bits are nearly constant too — which is precisely why the wide set XORs with an offset, and why the main set’s place() uses the high bits of the 32-bit hash, which is intHash32 and mixes.

Indeed the set’s own hash function is the 32-bit integer hash:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:108
/** This hash function is not the most optimal, but UniquesHashSet states counted with it,
 * stored in many places on disks (in many companies), so it continues to be used.
 */
struct UniquesHashSetDefaultHash
{
 size_t operator() (UInt64 x) const
 {
 return intHash32<0>(x);
 }
};

That comment is a nice piece of engineering history: the hash is frozen because the serialized states of uniq are stored on disk in many companies’ systems, and changing the hash would invalidate all of them.

The state format

Because the set is serialised into aggregate-function states that travel between servers, the format is versioned. Version 1 adds a flags byte:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:963
 /** Two serialization formats exist.
 * The legacy format (state version 0): skip_degree, then the number of values, then the 32-bit values
 * of the main set.
 * The new format (state version 1): a flags byte (bit 0 - a wide set follows the main set,
 * bit 1 - the wide sample is incomplete, see `wide_incomplete`; the two are mutually exclusive),
 * then the main set in the legacy layout, then, if present, the wide set: its skip_degree,
 * the number of values, and the 64-bit values.
 */

and a legacy state read back is marked as having an incomplete wide sample:

 cpp
 
 
// src/AggregateFunctions/UniquesHashSet.h:1048
 /** A non-empty state written in the legacy format carries elements that were never offered
 * to a wide set, so no wide estimate of this state (or of anything it is merged into)
 * can be trusted. An empty legacy state contributes nothing and does not spoil a merge.
 */
 if (use_legacy_format && m_size > 0)
 wide_incomplete = true;

That mark is sticky and propagates through merges, because the missing high bits of the already-consumed elements cannot be recovered. The alternative — estimating from a partially-covering wide sample — would silently undercount everything the sample missed, which is far worse than a merely imprecise estimate.

The whole family

uniq is one of several cardinality estimators in ClickHouse, and they make different trade-offs:

function	structure	exact?
`uniqExact`	a full hash set of the values	yes
`uniq`	the thinning set of this article	no, ~1-2% error
`uniqCombined`	small set → hash set → HyperLogLog, switching as it grows	no
`uniqTheta`	a datasketches theta sketch	no, mergeable

uniqCombined is the one that switches representation:

 cpp
 
 
// src/AggregateFunctions/AggregateFunctionUniqCombined.h:112
 // We want to migrate from |HashSet| to |HyperLogLogCounter| when the sizes in memory become almost equal.
 // The size per element in |HashSet| is sizeof(Key)*2 bytes, and the overall size of |HyperLogLogCounter| is 2^K * 6 bits.
 // For Key=UInt32 we can calculate: 2^X * 4 * 2 ≤ 2^(K-3) * 6 ⇒ X ≤ K-4.

and uniqExact keeps the two-level structure of the previous article in reserve, converting when it grows:

 cpp
 
 
// src/AggregateFunctions/AggregateFunctionUniq.h:154
 /// When creating, the hash table must be small.
 static constexpr size_t initial_size_degree = 4;
 using SingleLevelSet = HashSetWithStackMemory<Key, HashCRC32<Key>, initial_size_degree>;
 using TwoLevelSet = TwoLevelHashSet<Key, HashCRC32<Key>>;
 using Set = UniqExactSet<SingleLevelSet, TwoLevelSet>;

Note the “hash table must be small”: the initial buffer lives on the stack of the aggregate function state, so a uniqExact state that never grows past 2^4 slots allocates nothing on the heap at all.

A quick sanity check of the behaviour in SQL:

 sql
 
 
SELECT
 uniqExact(user_id) AS exact,
 uniq(user_id) AS approx,
 uniqCombined(user_id) AS combined
FROM events;

approx will differ from exact by a fraction of a percent on a few million distinct users, and by more on a few hundred million — that is the thinning, and it is the price of a bounded 64 KiB state.

Trade-offs

What the thinning set buys:

A bounded, tiny state. The main set is at most 2^17 × 4 bytes = 512 KiB, and the wide set at most 256 KiB. A uniq state never grows with the cardinality, which matters when the state is merged across a cluster or spilled to disk.
A mergeable state. Merging two uniq states is a union of hash sets plus a max(skip_degree), which is why uniq works with Distributed tables and AggregatingMergeTree.
A principled estimator. The thinning factor is known exactly, so the estimate is a maximum-likelihood estimate rather than a heuristic.
No allocation in the hot loop. The table is a flat array of UInt32, and the batch of eight hashes is computed with instruction-level parallelism.

What it costs:

Precision. The relative error grows with the cardinality, because the sample size stays constant while the multiplier grows. The wide set pushes the useful range from billions to trillions, but the error is still around 1% at the switch-over point.
A hash-quality dependency. The whole scheme assumes the hash is uniform; a bad hash breaks the thinning, because the surviving elements would not be a uniform sample.
Irreversible loss. Once thinned, the dropped hashes are gone. That is why the wide set has to be fed from the first insertion, and why a legacy state can never grow a wide sample.
A 32-bit ceiling without the wide set, which is exactly the bug that issue #6078 was, and why the state format has two versions.

This is a good example of an approximation that is statistically principled rather than ad hoc: the state is small because the algorithm throws away information on purpose, and the estimate is exact given the information that was kept.

Wrapping up

The one thing to remember: uniq() does not estimate by approximating — it estimates by sampling, and it knows the sample rate exactly. That is why the state never grows and why the estimate is a maximum-likelihood one rather than a guess.

The part people get wrong: the state is not just “a hash set”. It is a hash set plus a thinning factor, plus a second sparse 64-bit sample that takes over past ~2 billion distinct values. If you are estimating cardinalities in the trillions, you are living in the wide set’s world.

What to go do about it: compare uniq against uniqExact on your own data before you trust it. If the two agree within a percent, uniq is fine and cheaper. If they do not, and you need exactness, you know now what you are paying.

Hash-Table on Opensource internals

ClickHouse Splits GROUP BY into 256 Buckets for Parallel Merges

The structure

The sub-tables are ordinary tables

How a bucket grows

Resizing is amortised

When is the two-level table used?

The merge is the point

The string flavour

Offsets across buckets

Trade-offs

Further reading

Wrapping up

ClickHouse Packs Short Strings Into Hash Table Keys: No Allocation Per Key

Five tables, one per size class

The packing: how a string becomes a number

Unpacking: getting the string back

The empty-cell rule, specialised per size class

Hashing: CRC32C where it can, CityHash where it can’t

The lookup result: why you never see the key

Reserving and growing: where the heuristic leaks

Where it’s used: three string-key aggregation variants

Trade-offs: what you win and what you pay

Further reading

Wrapping up

ClickHouse Marks Empty Hash Table Cells With Zeroed Memory

The shape of the table

Empty means zero

The zero key gets its own slot

Probing

Insertion

Growing: the wrap-around special case

Robin Hood: what changed recently

Deletion without tombstones

The allocator: zeroed and pre-faulted

Prefetching

Keys that live outside the table

The hash function has to be good

Trade-offs

Further reading

Wrapping up

ClickHouse Evaluates IN with Hash Sets

The set is a plan artifact, not a function argument

The shape of the thing

Choosing the data structure

Presence-only vs counting

Building the set

Probing the set

Type coercion on the probe

Size limits and overflow

Spilling to disk

The Bloom filters

The lookup

When does it spill?

Trade-offs

Wrapping up

ClickHouse Estimates Distinct Counts by Thinning a Hash Table

The scheme

The constants

Thinning

The estimator

The wide set: fixing the 32-bit ceiling

Which estimator is used

The hot loop: batching

What is hashed

The state format

The whole family

Trade-offs

Further reading

Wrapping up