Group-By on Opensource internals

Polars Estimates Cardinality With HyperLogLog

Jay Hubbert — Wed, 21 Oct 2026 08:00:00 +0000

TL;DR: Polars has two cardinality sketches. The one in polars-compute has 16 384 one-byte registers, uses the low 14 bits of the hash as the register index and the number of leading zeros of the remaining bits as the register’s value, and estimates the cardinality with Ertl’s tau/sigma correction instead of the bias-correction table of HLL++. It is always dense — there is no sparse representation. The sketch the streaming engine actually uses is much smaller: 256 registers, one byte each, about 6.5% error. Both are used to make decisions: dictionary-encode a Parquet column, pick the build side of a hash join, publish a bloom filter, or size the hot table of a group-by.

Counting distinct values exactly means keeping a set of them, and the set is as big as the answer. A dataframe engine that wants to know “is this column worth dictionary-encoding?” or “which side of this join should I build the hash table on?” usually does not need the exact answer — it needs an answer that is right most of the time and costs a constant amount of memory no matter how many rows go past. That is what a cardinality sketch is, and Polars keeps several of them.

The register trick

The idea behind HyperLogLog is old and simple. Hash every value to 64 bits. Use p bits of the hash to pick one of m = 2^p registers. Use the position of the first 1 bit in the remaining bits as the value stored in that register, keeping the maximum ever seen. A register’s value is then “how far into the hash space did the most extreme value that landed here go”, and the distribution of those values across registers is what encodes the answer.

Polars’ implementation is in crates/polars-compute/src/hyperloglogplus.rs, and the constants that fix its accuracy and its memory are four lines:

 rust
 
 
/// The greater is P, the smaller the error.
const HLL_P: usize = 14_usize;
/// The number of bits of the hash value used determining the number of leading zeros
const HLL_Q: usize = 64_usize - HLL_P;
const NUM_REGISTERS: usize = 1_usize << HLL_P;
/// Mask to obtain index into the registers
const HLL_P_MASK: u64 = (NUM_REGISTERS as u64) - 1;

crates/polars-compute/src/hyperloglogplus.rs:31

And the whole update is three lines:

 rust
 
 
 /// Adds an element to the HyperLogLog.
 pub fn add(&mut self, obj: &T) {
 let hash = self.hash_value(obj);
 let index = (hash & HLL_P_MASK) as usize;
 let p = ((hash >> HLL_P) | (1_u64 << HLL_Q)).trailing_zeros() + 1;
 self.registers[index] = self.registers[index].max(p as u8);
 }

crates/polars-compute/src/hyperloglogplus.rs:88

Two details make this work. First, hash & HLL_P_MASK takes the low 14 bits as the index — the register choice is independent of the rank bits, which is what makes the estimator unbiased. Second, the | (1_u64 << HLL_Q) sets a sentinel bit at position 50, so trailing_zeros() is always defined (a hash of all zeros would otherwise have 64 trailing zeros and overflow the u8 register) and the rank is capped at 51, which fits in a byte.

The registers are one byte each, so the whole sketch is 16 KiB:

 rust
 
 
#[derive(Clone, Debug)]
pub struct HyperLogLog<T>
where
 T: Hash + ?Sized,
{
 registers: [u8; NUM_REGISTERS],
 phantom: PhantomData<T>,
}

crates/polars-compute/src/hyperloglogplus.rs:38

The hash is seeded with a fixed seed, not a random one:

 rust
 
 
/// Fixed seed for the hashing so that values are consistent across runs
///
/// Note that when we later move on to have serialized HLL register binaries
/// shared across cluster, this SEED will have to be consistent across all
/// parties otherwise we might have corruption. So ideally for later this seed
/// shall be part of the serialized form (or stay unchanged across versions).
const SEED: PlFixedStateQuality = PlFixedStateQuality::with_seed(0);

crates/polars-compute/src/hyperloglogplus.rs:57

That is a deliberate trade: a random seed would make two sketches of the same data disagree, which would break the merge operation and any attempt to compare estimates across runs. The comment shows the author thinking about shipping register arrays between machines.

Merging is the operation a sketch is really built for, and it is trivial because the registers are maxima:

 rust
 
 
 /// Merge the other [`HyperLogLog`] into this one
 pub fn merge(&mut self, other: &HyperLogLog<T>) {
 assert!(
 self.registers.len() == other.registers.len(),
 "unexpected got unequal register size, expect {}, got {}",
 self.registers.len(),
 other.registers.len()
 );
 for i in 0..self.registers.len() {
 self.registers[i] = self.registers[i].max(other.registers[i]);
 }
 }

crates/polars-compute/src/hyperloglogplus.rs:113

You can sketch a billion rows on four machines and combine the four sketches into one, and the answer is the answer for all of them.

The estimator, and what “HLL++” means here

The original HyperLogLog paper needed two corrections: a small-range correction (when the sketch is mostly empty registers) and a large-range correction (when the linear-count estimator breaks down and the two-largest harmonic mean takes over). HLL++ added a third thing: an empirically measured bias correction, a table of corrections per register value that removes the systematic error of the estimator in the small-to-medium range.

Polars implements none of that machinery. Its estimator is Ertl’s, from “New cardinality estimation algorithms for HyperLogLog sketches” (arXiv:1702.01284), and it folds the range corrections into the estimator itself:

 rust
 
 
 /// Guess the number of unique elements seen by the HyperLogLog.
 pub fn count(&self) -> usize {
 let histogram = self.get_histogram();
 let m = NUM_REGISTERS as f64;
 let mut z = m * hll_tau((m - histogram[HLL_Q + 1] as f64) / m);
 for i in histogram[1..=HLL_Q].iter().rev() {
 z += *i as f64;
 z *= 0.5;
 }
 z += m * hll_sigma(histogram[0] as f64 / m);
 (0.5 / 2_f64.ln() * m * m / z).round() as usize
 }

crates/polars-compute/src/hyperloglogplus.rs:121

The histogram counts how many registers hold each value. z is built from the register values in a harmonic-mean-like way; hll_tau is the correction for the case where the sketch has seen a lot of high values, and hll_sigma the correction for the case where most registers are still zero. Both are iterative series that stop when they converge:

 rust
 
 
/// Helper function sigma as defined in
/// "New cardinality estimation algorithms for HyperLogLog sketches"
/// Otmar Ertl, arXiv:1702.01284
#[inline]
fn hll_sigma(x: f64) -> f64 {
 if x == 1. {
 f64::INFINITY
 } else {
 let mut y = 1.0;
 let mut z = x;
 let mut x = x;
 loop {
 x *= x;
 let z_prime = z;
 z += x * y;
 y += y;
 if z_prime == z {
 break;
 }
 }
 z
 }
}

crates/polars-compute/src/hyperloglogplus.rs:135

The point of the tau/sigma estimator is that it is unbiased over the whole range, which makes the empirical bias table of HLL++ unnecessary. What the module does not have — and this is where the name is slightly misleading — is the other half of HLL++: the sparse representation. The registers are always a dense [u8; 16384], so every sketch costs 16 KiB, even if it has seen three values. There is no switch from a sparse list of (index, value) pairs to the dense array when the list gets too long. For a column with a handful of distinct values, Polars pays 16 KiB anyway.

The accuracy is the usual HyperLogLog accuracy, and the test suite states it in the only units that matter:

 rust
 
 
 fn compare_with_delta(got: usize, expected: usize) {
 let expected = expected as f64;
 let diff = (got as f64) - expected;
 let diff = diff.abs() / expected;
 // times 6 because we want the tests to be stable
 // so we allow a rather large margin of error
 // this is adopted from redis's unit test version as well
 let margin = 1.04 / ((NUM_REGISTERS as f64).sqrt()) * 6.0;

crates/polars-compute/src/hyperloglogplus.rs:221

1.04 / sqrt(16384) is 0.81% — the standard error of the estimator — and the test allows six times that, because a test that fails one run in a million is not a test.

The smaller sketch the streaming engine uses

The 16 KiB sketch is fine for a one-shot approx_n_unique. It is not fine for the streaming engine, where a sketch per partition per join side adds up, and where the answer only has to be good enough to make a decision. So polars-utils carries a second, much smaller implementation:

 rust
 
 
/// HyperLogLog in Practice: Algorithmic Engineering of
/// a State of The Art Cardinality Estimation Algorithm
/// Stefan Heule, Marc Nunkesser, Alexander Hall
///
/// We use m = 256 which gives a relative error of ~6.5% of the cardinality
/// estimate. We don't bother with stuffing the counts in 6 bits, byte access is
/// fast.
///
/// The bias correction described in the paper is not implemented, so this is
/// somewhere in between HyperLogLog and HyperLogLog++.
#[derive(Clone)]
pub struct CardinalitySketch {
 buckets: [u8; 256],
}

crates/polars-utils/src/cardinality_sketch.rs:20

The comment is unusually honest: this is a HyperLogLog without the bias correction, so “somewhere in between”. 256 bytes and 6.5% error is a completely different point on the accuracy/memory curve from 16 KiB and 0.8%, and for “should I publish this bloom filter?” the cheaper answer is the right one.

The update is the same register trick with different constants:

 rust
 
 
 /// Add a new hash to the sketch.
 #[inline]
 pub fn insert(&mut self, mut h: u64) {
 const ARBITRARY_ODD: u64 = 0x902813a5785dc787;
 // We multiply by this arbitrarily chosen odd number and then take the
 // top bits to ensure the sketch is influenced by all bits of the hash.
 h = h.wrapping_mul(ARBITRARY_ODD);
 let idx = (h >> 56) as usize;
 let p = 1 + (h << 8).leading_zeros() as u8;
 self.buckets[idx] = self.buckets[idx].max(p);
 }

crates/polars-utils/src/cardinality_sketch.rs:40

And the estimator is the classical one with the small-range correction:

 rust
 
 
 pub fn estimate(&self) -> usize {
 let m = 256.0;
 let alpha_m = 0.7123 / (1.0 + 1.079 / m);

 let mut sum = 0.0;
 let mut num_zero = 0;
 for x in self.buckets.iter() {
 sum = alg_add_f64(sum, inv_pow2(*x));
 num_zero += (*x == 0) as usize;
 }

 let est = (alpha_m * m * m) / sum;
 let corr_est = if est <= 5.0 / 2.0 * m && num_zero != 0 {
 // Small cardinality estimate, full 64-bit logarithm is overkill.
 m * (m as f32 / num_zero as f32).ln() as f64
 } else {
 est
 };

 if num_zero == self.buckets.len() {
 0
 } else {
 ((corr_est + 0.5) as usize).max(1)
 }
 }

crates/polars-utils/src/cardinality_sketch.rs:65

Two details are worth noticing. The hash is multiplied by an odd constant before the bits are used, which is a cheap way to mix the hash so that all of its bits influence the register choice. And the sum of 2^-bucket is accumulated with alg_add_f64, which on nightly builds uses Rust’s algebraic float addition so that the result does not depend on the order of the additions:

 rust
 
 
#[inline(always)]
pub fn alg_add_f64(a: f64, b: f64) -> f64 {
 #[cfg(feature = "nightly")]
 {
 a.algebraic_add(b)
 }
 #[cfg(not(feature = "nightly"))]
 {
 a + b
 }
}

crates/polars-utils/src/algebraic_ops.rs:2

With 256 terms of very different magnitudes, the order of the additions is not irrelevant to the last bits of the sum, and the last bits of the sum are the estimate.

The min of one is also there: an empty sketch estimates 0, and a sketch that has seen anything at all estimates at least 1.

Where the estimates are used

`approx_n_unique`

The obvious entry point. The chunked implementation is four lines:

 rust
 
 
 fn approx_n_unique(&self) -> IdxSize {
 let mut hllp = HyperLogLog::new();
 self.iter().for_each(|item| hllp.add(&item.to_total_ord()));
 hllp.count() as IdxSize
 }

crates/polars-core/src/chunked_array/ops/approx_n_unique.rs:17

to_total_ord is the float wrapper from the sorting story: for f64 it produces a TotalOrdWrap, whose Hash canonicalises -0.0 to 0.0 and all NaNs to one value. Without it, -0.0 and 0.0 would be counted as two distinct values and every NaN would be its own distinct value, which is rarely what the user meant.

Deciding whether to dictionary-encode a Parquet column

polars-compute exposes the sketch as an array-level cardinality estimate, and the Parquet writer uses it to decide whether dictionary encoding is worth it:

 rust
 
 
 let estimated_cardinality = polars_compute::cardinality::estimate_cardinality(array);

 if array.len() > 128 && (estimated_cardinality as f64) / (array.len() as f64) > 0.75 {
 return None;
 }

crates/polars-parquet/src/arrow/write/dictionary.rs:256

If more than 75% of the values are distinct, a dictionary is just an indirection with extra steps, so the column is written as plain values. The estimate does not have to be exact for that decision; it has to be right near the boundary, and 0.8% error is plenty.

The estimator itself is a dispatch on the physical type, and it handles strings by hashing the bytes:

 rust
 
 
fn binary_offset_array_estimate<O: Offset>(array: &BinaryArray<O>) -> usize {
 let mut hll = HyperLogLog::new();

 if array.has_nulls() {
 for v in array.iter() {
 let v = v.unwrap_or_default();
 hll.add(v);
 }
 } else {
 for v in array.values_iter() {
 hll.add(v);
 }
 }

 hll.count()
}

crates/polars-compute/src/cardinality.rs:127

Note that nulls are hashed as the default value — the empty string, or zero for a primitive — so they contribute at most one more distinct value, and none at all if the default value is in the data anyway. Booleans are handled exactly rather than sketched, because there are only two of them.

Choosing the build side of a hash join

This is where the small sketch earns its keep. The streaming engine samples both sides of an equi-join and estimates how many bytes each side’s hash table would need:

 rust
 
 
 let rows = rows as f64;
 Ok(Self {
 key_ratio: (sketch.estimate() as f64 / rows).min(1.0),
 min_hash_key_ratio: min_hash.and_then(|m| m.key_ratio()),
 key_width: key_bytes as f64 / rows,
 row_width: row_bytes as f64 / rows,
 })
 }

 /// Bytes retained when building a table of the distinct keys of `rows` rows.
 pub fn distinct_keys_build_bytes(&self, rows: usize) -> f64 {
 rows as f64 * self.key_ratio * (self.key_width + KEY_SLOT_OVERHEAD)
 }

crates/polars-stream/src/nodes/joins/utils.rs:356

Two ratios are computed. key_ratio is distinct keys over rows, straight from the cardinality sketch. min_hash_key_ratio is the same question asked differently: how many of the rows are distinct keys, counting each key once regardless of how often it appears. That is what the min-hash sketch is for:

 rust
 
 
 /// Add a new hash to the sketch.
 #[inline]
 pub fn insert(&mut self, h: u64) {
 const IDX_ODD: u64 = 0x85b0fc582ca15d65; // Randomly chosen.
 const TAG_ODD: u64 = 0x4a0da407caf4e9ad; // Randomly chosen.
 let idx = (h.wrapping_mul(IDX_ODD) >> (64 - NUM_ENTRIES.trailing_zeros())) as usize;
 let tag = (h.wrapping_mul(TAG_ODD) >> 32) as u32;
 let entry = &mut self.entries[idx];
 if tag < entry.tag {
 entry.tag = tag;
 entry.count = 0;
 }
 entry.count += (tag == entry.tag) as u32;
 }

crates/polars-utils/src/min_hash_sketch.rs:38

 rust
 
 
 /// Estimates the number of distinct keys as a fraction of the number of
 /// rows, weighting each distinct key equally.
 pub fn key_ratio(&self) -> Option<f64> {
 let mut nonempty = 0u64;
 let mut total = 0u64;
 for entry in self.entries.iter() {
 nonempty += (entry.count > 0) as u64;
 total += entry.count as u64;
 }
 (nonempty > 0).then(|| nonempty as f64 / total as f64)
 }

crates/polars-utils/src/min_hash_sketch.rs:60

Each entry keeps the smallest tag it has seen for that bucket and a count of how many rows hashed to that same tag. A key that appears many times wins its bucket’s tag competition and is counted once, so the ratio of non-empty entries to total count is the fraction of rows that are distinct keys.

Those two numbers feed the build-side decision:

 rust
 
 
 let left_unique =
 left.min_hash_key_ratio.unwrap_or(0.0) >= UNIQUE_KEY_RATIO;
 let right_unique =
 right.min_hash_key_ratio.unwrap_or(0.0) >= UNIQUE_KEY_RATIO;
 let mut left_score = left.all_rows_build_bytes(params.sample_limit);
 let mut right_score = right.all_rows_build_bytes(params.sample_limit);
 if left.key_ratio <= REF_KEY_RATIO && right_unique {
 left_score *= UNIQUE_BUILD_MARGIN / (left.key_ratio + 1e-6);
 }
 if right.key_ratio <= REF_KEY_RATIO && left_unique {
 right_score *= UNIQUE_BUILD_MARGIN / (right.key_ratio + 1e-6);
 }

crates/polars-stream/src/nodes/joins/equi_join.rs:521

If one side has unique keys and the other repeats them, the repeating side is assumed to reference the unique side, and it is penalised for being the build side. Getting this wrong costs memory and time, and the sample that produces these numbers is a sample — so the sketch is being used to make a decision that is right most of the time, with a bounded cost when it is wrong.

Publishing a bloom filter

The join runtime filter is a bloom filter built on the build side and probed by the other side. It is only worth publishing if it will reject enough probes, and the decision is made with a cardinality sketch of the build keys:

 rust
 
 
 let distinct = sketch.estimate();
 let weak = spec
 .probe_distinct
 .is_some_and(|probe| distinct as f64 > probe as f64 * MAX_BUILD_PROBE_DISTINCT_RATIO);
 let bloom = if weak {
 None
 } else if planned {
 // Every builder holding it was dropped above, so no thread inserts
 // anymore.
 shared
 .and_then(|bloom| {
 debug_assert_eq!(Arc::strong_count(&bloom), 1);
 Arc::into_inner(bloom).map(AtomicSplitBlockBloom::into_inner)
 })
 .filter(|bloom| distinct.saturating_mul(BLOOM_MIN_BITS_PER_KEY) <= bloom.num_bits())
 } else {
 KeyFilterSpec::bloom_for(distinct)
 };

crates/polars-stream/src/nodes/joins/runtime_filter.rs:328

If the build side has too many distinct keys relative to the probe side, the filter would not reject much, so it is dropped. The bloom filter itself is a split-block filter in the Parquet layout, 32-byte blocks and eight bits per key.

The F2 sketch: counting heavy keys

polars-utils also has a sketch for the second frequency moment:

 rust
 
 
const NUM_BUCKETS: usize = 64;

/// Estimates the second frequency moment F2 = sum_k f_k^2 of a stream of hashed
/// keys, where f_k is the number of times key k was inserted.
///
/// This is a single row of a Count-Sketch (the "fast AMS" sketch): every hash
/// adds +1 or -1 to one of 64 buckets, and the sum of squared buckets is an
/// unbiased estimate of F2 with a relative standard error of ~18%.
#[derive(Clone)]
pub struct F2Sketch {
 buckets: [i64; NUM_BUCKETS],
 num_inserts: u64,
}

crates/polars-utils/src/f2_sketch.rs:1

F2 is not a set-similarity measure; it is a measure of skew. If every key appears once, F2 equals the number of rows. If one key appears a thousand times, F2 is a million. So F2 tells you how heavy the tail of the frequency distribution is, and the streaming group-by uses exactly that to decide how big its hot table should be:

 rust
 
 
 fn maybe_grow_hot_tables(&mut self, max_size: usize) {
 let size = self.hot_table_size;
 if size >= max_size {
 return;
 }

 // Cheap early exit before estimating the number of distinct missed keys.
 let misses = self.miss_f2.num_inserts() as f64;
 if misses < HOT_TABLE_GROW_MIN_REPEAT * size as f64 {
 return;
 }

crates/polars-stream/src/nodes/group_by/sink.rs:209

Read the algebra. The model is: the missed rows consist of heavy keys that were each missed repeat times, plus the remaining distinct keys, each missed once. With D distinct keys (the cardinality sketch’s estimate, capped at the number of missed rows) and F2 the second moment over the missed rows only (the F2 sketch only sees the misses):

text

misses = D + heavy * (repeat - 1)
F2 = D + heavy * (repeat^2 - 1)

Subtracting gives F2 - misses = heavy * repeat * (repeat - 1), and misses - D = heavy * (repeat - 1), so their quotient is repeat; and F2 - 2 * misses + D = heavy * (repeat - 1)^2, so the square of misses - D divided by that is heavy. Both are in the code:

 rust
 
 
 let f2 = self.miss_f2.estimate();
 let distinct = (self.key_sketch.estimate() as f64).min(misses);
 let excess = misses - distinct;
 let denom = f2 - 2.0 * misses + distinct;
 if excess <= 0.0 || denom <= 0.0 {
 return;
 }
 let repeat = (f2 - misses) / excess;
 let heavy = excess * excess / denom;

crates/polars-stream/src/nodes/group_by/sink.rs:225

If a few keys are missing over and over, repeat is large and the hot table should grow to keep them; if the misses are spread over many distinct keys, repeat is close to one and growing the table would be wasted memory. That is the whole reason the F2 sketch exists. A cardinality sketch tells you how many distinct keys there are; F2 tells you how skewed they are, and skew is what decides whether a cache is worth growing.

The bottom line

The sketches in Polars are not one algorithm but a family, chosen per decision:

Sketch	Size	Error	Used for
`HyperLogLog` (polars-compute)	16 KiB	~0.8%	`approx_n_unique`, Parquet dictionary decision
`CardinalitySketch` (polars-utils)	256 B	~6.5%	join build side, bloom filter publishing, group-by distinct count
`MinHashSketch`	2 KiB	—	fraction of rows that are distinct keys
`F2Sketch`	512 B	~18%	heavy-key count in the streaming group-by

The trade-off is the usual one, and it is visible in the constants: the big sketch buys an order of magnitude of accuracy for 64 times the memory, and it is only used where the answer is shown to the user. The small sketch is used where the answer only has to tilt a decision — which side to build, whether to publish a filter, how big a table to allocate — and there the 6.5% error is free.

The register trick is what makes all of this possible: 64 bits of hash per value, a byte per register, and a maximum update, so that the whole cost of “how many distinct values are there” is one hash and one byte comparison per row, with a memory footprint that does not depend on the number of rows at all. The estimator on top is where the accuracy comes from, and Polars' choice of Ertl’s tau/sigma over HLL++’s empirical bias table is a nice example of a better algorithm replacing a lookup table.

Polars Encodes Group Keys as Rows to Make Hashing Cheap

Jay Hubbert — Wed, 21 Oct 2026 06:40:00 +0000

TL;DR: A composite group key is a tuple, and tuples are awkward to hash: you have to combine the hashes of the parts in a way that cannot collide across different arities or orderings. Polars sidesteps that by encoding each row of the key into a byte blob — a null sentinel, big-endian bytes for numbers, a terminator for strings — and then hashing the blob. One hash, one comparison, one allocation.

The encoder is not a black box: it is written so that the encoded bytes sort in the same order as the values, which is what lets the same representation be used for sorting, grouping and joining.

Grouping on one numeric column is easy. The physical value is the key, and the hash of that value is the hash of the key. Grouping on three columns is a different problem. Now the key is a tuple, and a tuple needs a hash function that is consistent across the whole query: two rows are in the same group if and only if their hashes agree and their keys are equal, and the “if and only if” has to survive the hashing.

The tempting shortcut — hash each column and XOR the hashes — is wrong in subtle ways. (a, b) and (b, a) can hash the same; (1, 2) and (1, 2, 0) can hash the same; the fix is to mix in the arity and the position of each column, and then you are maintaining a hand-rolled hash combiner with all the collision risk that implies.

Polars takes the other route: turn the tuple into a byte string, and let a well-tested byte hash do the work.

Encoding a row

The generic encoder lives in the polars-row crate and is used for the “row encoded” key representation. A key column’s contribution to a row is decided by its dtype, and the decision is made by a trait:

 rust
 
 
// crates/polars-row/src/fixed/numeric.rs:28
/// Encodes a value of a particular fixed width type into bytes
pub trait FixedLengthEncoding: Copy + Debug {
 // 1 is validity 0 or 1
 // bit repr of encoding
 const ENCODED_LEN: usize = 1 + size_of::<Self::Encoded>();

 type Encoded: Sized + Copy + AsRef<[u8]> + AsMut<[u8]>;

 fn encode(self) -> Self::Encoded;

 fn decode(encoded: Self::Encoded) -> Self;

 /// Invert all bits of an encoded value. Used for descending order.
 fn invert(encoded: Self::Encoded) -> Self::Encoded;

 /// Keep the encoded value if `keep`, otherwise all zeros.
 fn keep_or_zero(encoded: Self::Encoded, keep: bool) -> Self::Encoded;

 fn decode_reverse(encoded: Self::Encoded) -> Self {
 Self::decode(Self::invert(encoded))
 }
}

Every fixed-width value is written as one sentinel byte followed by the value’s bytes. The sentinel is 1 for a valid value and 0xFF (or 0x00, when sorting descending) for null:

 rust
 
 
// crates/polars-row/src/fixed/numeric.rs:230
pub unsafe fn encode<T: NativeType + FixedLengthEncoding>(
 buffer: &mut [MaybeUninit<u8>],
 arr: &PrimitiveArray<T>,
 opt: RowEncodingOptions,
 offsets: &mut [usize],
) {
 if arr.null_count() == 0 {
 encode_slice(buffer, arr.values().as_slice(), opt, offsets)
 } else {
 encode_slice_with_validity(
 buffer,
 arr.values().as_slice(),
 arr.validity().unwrap(),
 opt,
 offsets,
 )
 }
}

/// Write the sentinel byte and the encoded value.
#[inline(always)]
unsafe fn write_value<T: FixedLengthEncoding>(
 dst: *mut MaybeUninit<u8>,
 sentinel: u8,
 encoded: T::Encoded,
) {
 *dst = MaybeUninit::new(sentinel);
 std::ptr::write_unaligned(dst.add(1) as *mut T::Encoded, encoded);
}

/// Write the null sentinel followed by zeros.
#[inline(always)]
unsafe fn write_null<T: FixedLengthEncoding>(dst: *mut MaybeUninit<u8>, null_sentinel: u8) {
 *dst = MaybeUninit::new(null_sentinel);
 std::ptr::write_bytes(dst.add(1), 0, T::ENCODED_LEN - 1);
}

Note that a null is written as the sentinel plus zeros. That matters: it means a null key and a zero-valued key never produce the same bytes, and it means the encoding of a row is fixed width, so the row encoder can compute row offsets before it writes anything.

The value bytes themselves are chosen so that byte order equals value order. Unsigned integers are stored big-endian, which already sorts correctly. Signed integers need one extra step, because in two’s complement -1 sorts after 1 if you compare bytes:

 rust
 
 
// crates/polars-row/src/fixed/numeric.rs:78
// toggle the sign bit and then encode as big indian
macro_rules! encode_signed {
 ($n:expr, $t:ty) => {
 impl FixedLengthEncoding for $t {
 type Encoded = [u8; $n];

 #[inline(always)]
 fn encode(self) -> [u8; $n] {
 #[cfg(target_endian = "big")]
 {
 todo!()
 }

 let mut b = self.to_be_bytes();
 // Toggle top "sign" bit to ensure consistent sort order
 b[0] ^= 0x80;
 b
 }

Flipping the top bit maps the two’s complement range onto the unsigned range, which is exactly what big-endian byte comparison needs. Floats get the same treatment, plus a canonicalization step so that NaN has one representation and -0.0 sorts with 0.0:

 rust
 
 
// crates/polars-row/src/fixed/numeric.rs:176
impl FixedLengthEncoding for f64 {
 type Encoded = [u8; 8];

 #[inline]
 fn encode(self) -> [u8; 8] {
 // https://github.com/rust-lang/rust/blob/9c20b2a8cc7588decb6de25ac6a7912dcef24d65/library/core/src/num/f32.rs#L1176-L1260
 let s = canonical_f64(self).to_bits() as i64;
 let val = s ^ (((s >> 63) as u64) >> 1) as i64;
 val.encode()
 }

The ^ (((s >> 63) as u64) >> 1) is the standard “flip all bits if negative” transform, which maps IEEE-754 doubles onto a monotonically increasing integer ordering. Combined with the sign-bit flip in the signed-integer macro, the float encoder reuses the integer encoder.

Descending order is handled by inverting every bit, which turns “ascending” into “descending” without a second code path:

 rust
 
 
// crates/polars-row/src/fixed/numeric.rs:38
macro_rules! invert_and_keep_or_zero {
 ($t:ty) => {
 #[inline(always)]
 fn invert(encoded: Self::Encoded) -> Self::Encoded {
 (!Self::from_ne_bytes(encoded)).to_ne_bytes()
 }

Strings are the awkward case

Fixed-width values have a known size; strings do not. The string encoding solves this with a trick: since the bytes 0xFC..=0xFF never appear in valid UTF-8, every byte can be shifted by two and the result can never be confused with the terminator or the null sentinel:

 rust
 
 
// crates/polars-row/src/variable/utf8.rs:1
//! Row encoding for UTF-8 strings
//!
//! This encoding is based on the fact that in UTF-8 the bytes 0xFC - 0xFF are never valid bytes.
//! To make this work with the row encoding, we add 2 to each byte which gives us two bytes which
//! never occur in UTF-8 before and after the possible byte range. The values 0x00 and 0xFF are
//! reserved for the null sentinel. The values 0x01 and 0xFE are reserved as a sequence terminator
//! byte.
//!
//! This allows the string row encoding to have a constant 1 byte overhead.

The terminator is what lets a decoder find the end of a string without a length prefix — one byte of overhead per string instead of eight — and it is found with a vectorized byte search:

 rust
 
 
// crates/polars-row/src/variable/utf8.rs:107
/// Encode the bytes of a string followed by the terminator.
///
/// # Safety
/// `dst` must be writable for `s.len() + 1` bytes.
#[inline(always)]
unsafe fn encode_bytes(dst: *mut MaybeUninit<u8>, s: &[u8], xor_mask: u8) {
 if s.len() < BLOCK_SIZE {
 let mut block = [0u8; BLOCK_SIZE];
 std::ptr::copy_nonoverlapping(s.as_ptr(), block.as_mut_ptr(), s.len());
 encode_short(dst, block, s.len(), xor_mask);
 } else {
 for (i, &b) in s.iter().enumerate() {
 *dst.add(i) = MaybeUninit::new(xor_mask ^ (b + 2));
 }
 *dst.add(s.len()) = MaybeUninit::new(xor_mask ^ 0x01);
 }
}

Note that strings are not fixed width, so a row containing a string has a variable length. That is the main structural difference between the generic row encoding and the newer fixed-width key rows described at the end.

Hashing the bytes

Once the rows are encoded, hashing them is trivial — one hash per row:

 rust
 
 
// crates/polars-expr/src/hash_keys.rs:146
 if use_row_encoding {
 let keys = df.columns();
 let mut keys_encoded = _get_rows_encoded_unordered(keys).unwrap().into_array();

 if !null_is_valid {
 let validities = keys
 .iter()
 .map(|c| c.as_materialized_series().rechunk_validity())
 .collect_vec();
 let combined = combine_validities_and_many(&validities);
 keys_encoded.set_validity(combined);
 }

 // TODO: use vechash? Not supported yet for lists.
 // let mut hashes = Vec::with_capacity(df.height());
 // columns_to_hashes(df.columns(), Some(random_state), &mut hashes).unwrap();

 let hashes = keys_encoded
 .values_iter()
 .map(|k| random_state.hash_one(k))
 .collect();
 Self::RowEncoded(RowEncodedKeys {
 hashes: PrimitiveArray::from_vec(hashes),
 keys: keys_encoded,
 })

Two properties make this work. First, the hash is computed over the whole row, so the arity and the order of the columns are baked into the bytes — there is no way for (a, b) to collide with (b, a) unless the bytes themselves are equal. Second, the hash is computed once, up front, and stored alongside the keys:

 rust
 
 
// crates/polars-expr/src/hash_keys.rs:116
/// Represents a DataFrame plus a hash per row, intended for keys in grouping
/// or joining. The hashes may or may not actually be physically pre-computed,
/// this depends per type.
#[derive(Clone, Debug)]
pub enum HashKeys {
 RowEncoded(RowEncodedKeys),
 KeyRows(KeyRowKeys),
 Binview(BinviewKeys),
 Single(SingleKeys),
}

Pre-hashing is what makes partitioned joins possible: the hash decides which partition a key belongs to, so a probe only has to look at one partition’s table instead of all of them.

The map that stores the groups

The row-encoded grouper keeps a hash map from row bytes to group index. The map is specialised for exactly this shape — pre-hashed byte keys — and it stores the key bytes in an arena of buffers rather than allocating a Vec per key:

 rust
 
 
// crates/polars-utils/src/idx_map/bytes_idx_map.rs:25
/// An IndexMap where the keys are always [u8] slices which are pre-hashed.
pub struct BytesIndexMap<V> {
 table: HashTable<IdxSize>,
 tuples: Vec<(Key, V)>,
 key_data: Vec<Vec<u8>>,

 // Internal random seed used to keep hash iteration order decorrelated.
 // We simply store a random odd number and multiply the canonical hash by it.
 seed: u64,
}

The lookup is a hash compare followed by a byte compare:

 rust
 
 
// crates/polars-utils/src/idx_map/bytes_idx_map.rs:79
 /// Gets the index by insertion order of the given key.
 pub fn get_index_of(&self, hash: u64, key: &[u8]) -> Option<IdxSize> {
 self.table
 .find(hash.wrapping_mul(self.seed), |i| unsafe {
 let t = self.tuples.get_unchecked(*i as usize);
 hash == t.0.key_hash && key == t.0.get(&self.key_data)
 })
 .copied()
 }

That key == t.0.get(&self.key_data) is a memcmp over the row bytes. It is the whole equality check for a composite key — no per-column dispatch, no null handling at comparison time (nulls are in the bytes), no tuple comparison.

And the grouper itself is four lines:

 rust
 
 
// crates/polars-expr/src/groups/row_encoded.rs:23
 fn insert_key(&mut self, hash: u64, key: &[u8]) -> IdxSize {
 match self.idx_map.entry(hash, key) {
 Entry::Occupied(o) => o.index(),
 Entry::Vacant(v) => {
 let index = v.index();
 v.insert(());
 index
 },
 }
 }

The index handed out is the insertion order index, which is why the keys come back in group order for free.

When no encoding is needed

All of the above is machinery that exists only because there is more than one key column, or because the key is a string. For a single numeric key, Polars skips the encoding entirely:

 rust
 
 
// crates/polars-expr/src/groups/single_key.rs:13
pub struct SingleKeyHashGrouper<T: PolarsDataType> {
 /// Each key with its group.
 table: HashTable<(T::Physical<'static>, IdxSize)>,
 num_groups: IdxSize,
 /// The group of the null key, IdxSize::MAX if there is none.
 null_idx: IdxSize,
 random_state: PlRandomState,
}

The hash table is keyed by the physical value itself, and the null key gets its own group index instead of being hashed at all, so it can never be confused with a real value:

 rust
 
 
// crates/polars-expr/src/groups/single_key.rs:100
 for (hash, idx) in hashes.iter().zip(block) {
 let idx = *idx as usize;
 if validity.is_some_and(|v| !v.get_bit_unchecked(idx)) {
 if null_is_valid {
 if self.null_idx == IdxSize::MAX {
 self.null_idx = self.num_groups;
 self.num_groups += 1;
 }
 on_group(self.null_idx);
 }
 } else {
 let key = arr.value_unchecked(idx);
 match self.table.entry(*hash, |(k, _)| k.tot_eq(&key), hasher) {
 Entry::Occupied(o) => on_group(o.get().1),
 Entry::Vacant(v) => {
 v.insert((key, self.num_groups));
 on_group(self.num_groups);
 self.num_groups += 1;
 },
 }
 }
 }

The choice between the representations is made once, from the key schema:

 rust
 
 
// crates/polars-expr/src/groups/mod.rs:92
pub fn new_hash_grouper(key_schema: Arc<Schema>) -> Box<dyn Grouper> {
 if let Some(layout) = KeyRowLayout::new(key_schema.iter_values()) {
 Box::new(key_rows::KeyRowHashGrouper::new(Arc::new(layout)))
 } else if key_schema.len() > 1 {
 Box::new(row_encoded::RowEncodedHashGrouper::new())
 } else {
 let (_name, dt) = key_schema.get_at_index(0).unwrap();
 match dt {
 dt if dt.is_primitive_numeric() | dt.is_temporal() => {
 with_match_physical_numeric_polars_type!(dt.to_physical(), |$T| {
 Box::new(single_key::SingleKeyHashGrouper::<$T>::new())
 })
 },

Read it as a ladder: if the schema can be laid out as fixed-width key rows, use that; otherwise if there is more than one column, use generic row encoding; otherwise use the most specialized grouper the single column’s dtype allows.

Key rows: the fixed-width version

The generic row encoding has one structural weakness: string columns make rows variable length, which means row offsets are not known up front and the bytes are not word-aligned. The newer key row representation fixes both by computing a layout from the schema alone:

 rust
 
 
// crates/polars-expr/src/key_rows/layout.rs:72
/// The row format of keys of one schema. It depends only on the key dtypes: the null
/// bits are always present, also when nulls are not keys, so keys that disagree on
/// whether nulls match (the two sides of an anti join) share one layout.
///
/// Rows are written into zeroed words and null values are left zero, so equal keys
/// have equal words, including padding. The one exception is the second word of a
/// long view, which holds where its bytes are stored: equality skips that word and
/// compares the bytes instead.
#[derive(Debug, PartialEq)]
pub(crate) struct KeyRowLayout {
 pub(super) cols: Vec<ColLayout>,
 /// Byte offset of the null bits, one per column.
 pub(super) null_offset: usize,
 null_width: usize,
 pub(super) stride_words: usize,
 /// The first word of each view column, in column order.
 pub(super) view_words: Vec<usize>,
 /// Every word except the second half of each view.
 pub(super) plain_words: Vec<usize>,
}

Three decisions are worth pulling out.

Columns are ordered by width, descending. The layout sorts the fields so that the widest column comes first, which (a) makes the stride a power of two number of words and (b) means the comparison bails out early on the most discriminating column:

 rust
 
 
// crates/polars-expr/src/key_rows/layout.rs:90
 let null_width = cols.len().div_ceil(8).next_power_of_two();
 let mut fields: Vec<(usize, Option<usize>)> = widths
 .into_iter()
 .enumerate()
 .map(|(i, width)| (width, Some(i)))
 .chain([(null_width, None)])
 .collect();
 fields.sort_by_key(|(width, _)| Reverse(*width));

Nulls live inside the row. One bit per key column, always allocated even when no column has nulls. That is deliberate: it means the two sides of an anti join — where one side treats null as a key and the other does not — can share one layout and one hash table.

Equality is an XOR fold over words. Because rows are written into zeroed words, “equal” means “no differing bits”, which is one instruction per word for the fixed part:

 rust
 
 
// crates/polars-expr/src/key_rows/layout.rs:170
 pub(super) unsafe fn rows_eq(
 &self,
 a: &[u64],
 b: &[u64],
 mut long_eq: impl FnMut(View, View) -> bool,
 ) -> bool {
 if self.view_words.is_empty() {
 return a.iter().zip(b).fold(0, |acc, (x, y)| acc | (x ^ y)) == 0;
 }
 for w in &self.plain_words {
 if a.get_unchecked(*w) != b.get_unchecked(*w) {
 return false;
 }
 }

The strings are the awkward case again. An Arrow “view” is 16 bytes: short strings are inlined, long strings point into a buffer, and the second half of the view word holds a location, not data. So the word comparison skips that word and compares the bytes instead.

The hash is derived from the columns rather than from the row bytes, by folding each column’s contribution into a per-row hash:

 rust
 
 
// crates/polars-expr/src/key_rows/keys.rs:338
 let mut hashes = vec![random_state.hash_one(HASH_MULTIPLE); len];
 for col in &cols {
 col.hash_into(&mut hashes, random_state);
 }

The bottom line

The design in one sentence: encode the key once, then let bytes do all the work.

The trade-offs:

Encoding costs memory and a copy. Every distinct key is materialized as bytes and stored in the map’s arena. For a group-by on a wide string key that is a real cost, which is why single string keys get their own grouper that compares Arrow views directly instead of copying bytes.
The encoding is a format, so it has invariants. The sentinel byte, the terminator byte and the shifted UTF-8 bytes all have to be mutually exclusive, and the decoder has to know which convention a column was encoded with. RowEncodingOptions carries the descending and null-placement flags so that a row encoded one way is never decoded as another.
Fixed width buys alignment, at the cost of limits. The key-row layout needs a power-of-two word stride, one null-bit word, and at most two string columns; beyond that it gives up and the generic encoder takes over. The limits are explicit in the source (MAX_KEY_COLUMNS = 64, MAX_VIEW_COLUMNS = 2, crates/polars-expr/src/key_rows/layout.rs:13).
Sorting comes for free. Because the encoded bytes are order-preserving, the same representation can be sorted, compared and hashed. That is not an accident; it is why the sign bit is flipped and why floats are canonicalized before they are written.

Polars Aggregates Faster When the Keys Are Already Sorted

Jay Hubbert — Wed, 21 Oct 2026 06:30:00 +0000

TL;DR: Every ChunkedArray in Polars carries an IsSorted flag (Ascending, Descending, Not) in its metadata. The eager group-by reads that flag before it builds anything: if the key column is sorted, the groups are contiguous row ranges, so they are produced by a linear run-length scan instead of a hash table. The scan can also be split across threads, because the partitions are cut at group boundaries.

The output of that path is not just a different set of groups: it is a different type of group structure, GroupsType::Slice, which downstream aggregations consume with slice arithmetic instead of gathers.

Group-by is usually a hashing problem. You have N rows and a key column, and you want to know which rows share a key. The standard solution is a hash table: hash the key, look it up, append the row index to the bucket. That is O(N) but with a random-access pattern and a data structure that has to be allocated, grown and probed.

But there is a much cheaper structure available when the keys happen to be sorted, and sorted keys are not exotic: they are what a sort produces, what a parquet file with a sorted column contains, and what a range-partitioned stream delivers. When the keys are sorted, all the rows with the same key are adjacent, so the groups are just run boundaries. Finding them is a single pass over the array, with no allocation beyond the output vector and no hashing at all.

Polars exploits this, and the interesting part is how it decides.

text

 sort("k")
 |
 v
 [ 1 1 1 2 2 7 7 9 ] IsSorted::Ascending
 |
 v
 group_by("k")
 |
 +-- is_sorted_ascending_flag()? --- no ---> hash table
 | |
 | v
 | GroupsType::Idx
 | (row indices per group)
 |
 +-- yes ---> run-length scan ---> GroupsType::Slice
 (offset, len) per group
 [ (0,3) (3,2) (5,2) (7,1) ]

There is an obvious objection: sorting costs O(N log N), which is more than hashing costs. The flag only pays for itself when the sort was going to happen anyway, or when the data arrived sorted. In practice that is common: a sort before a group_by, a parquet file whose key column is physically ordered, a window function’s PARTITION BY output, a table that was already sorted by an upstream system. In those cases the sort is not a cost of the group-by, it is a fact about the input, and the group-by gets to use it.

The flag, not the check

Polars does not verify that the keys are sorted. It trusts a bit of metadata that is set when the column is sorted and cleared whenever a mutation could invalidate it:

 rust
 
 
// crates/polars-core/src/series/series_trait.rs:15
pub enum IsSorted {
 Ascending,
 Descending,
 Not,
}

 rust
 
 
// crates/polars-core/src/chunked_array/ops/sort/mod.rs:199
 let mut ca = ChunkedArray::from_vec(ca.name().clone(), vals);
 let s = if options.descending {
 IsSorted::Descending
 } else {
 IsSorted::Ascending
 };
 ca.set_sorted_flag(s);

The flag lives in the chunked array’s StatisticsFlags, next to things like “can fast explode list”, and it is propagated by operations that preserve order (slice, take with sorted indices, append of two compatible runs). Anything that writes a value in an arbitrary position resets it.

So the group-by’s job is not “is this sorted?” but “does the column claim to be sorted?”. That is a single bit test:

 rust
 
 
// crates/polars-core/src/frame/group_by/into_groups.rs:137
 fn group_tuples(&self, multithreaded: bool, sorted: bool) -> PolarsResult<GroupsType> {
 // sorted path
 if self.is_sorted_ascending_flag() || self.is_sorted_descending_flag() {
 // don't have to pass `sorted` arg, GroupSlice is always sorted.
 let groups = self.rechunk().create_groups_from_sorted(multithreaded);
 return Ok(GroupsType::new_slice(groups, false, true));
 }

Two things to notice. The sorted: bool argument of group_tuples is not about sorted keys — it is the “maintain the order of groups by first index” flag that group_by_stable passes, and it selects how the hash path orders its output. Key sortedness is detected from the column metadata.

Second, the sorted path returns GroupsType::new_slice(groups, false, true) and returns early. Nothing downstream is told “we took the fast path”; the type of the group structure is the message.

The run-length scan

Here is the whole sorted path:

 rust
 
 
// crates/polars-core/src/frame/group_by/into_groups.rs:96
 fn create_groups_from_sorted(&self, multithreaded: bool) -> GroupsSlice {
 if verbose() {
 eprintln!("group_by keys are sorted; running sorted key fast path");
 }
 let arr = self.downcast_iter().next().unwrap();
 if arr.is_empty() {
 return GroupsSlice::default();
 }
 let mut values = arr.values().as_slice();
 let null_count = arr.null_count();
 let length = values.len();

 // all nulls
 if null_count == length {
 return vec![[0, length as IdxSize]];
 }

 let mut nulls_first = false;
 if null_count > 0 {
 nulls_first = arr.get(0).is_none()
 }

 if nulls_first {
 values = &values[null_count..];
 } else {
 values = &values[..length - null_count];
 };

 let n_threads = RAYON.current_num_threads();
 if multithreaded && n_threads > 1 {
 let parts =
 create_clean_partitions(values, n_threads, self.is_sorted_descending_flag());
 let n_parts = parts.len();

 let first_ptr = &values[0] as *const T::Native as usize;
 let groups = parts.par_iter().enumerate().map(|(i, part)| {
 // we go via usize as *const is not send
 let first_ptr = first_ptr as *const T::Native;

 let part_first_ptr = &part[0] as *const T::Native;
 let mut offset = unsafe { part_first_ptr.offset_from(first_ptr) } as IdxSize;

 // nulls first: only add the nulls at the first partition
 if nulls_first && i == 0 {
 partition_to_groups(part, null_count as IdxSize, true, offset)
 }
 // nulls last: only compute at the last partition
 else if !nulls_first && i == n_parts - 1 {
 partition_to_groups(part, null_count as IdxSize, false, offset)
 }
 // other partitions
 else {
 if nulls_first {
 offset += null_count as IdxSize;
 };

 partition_to_groups(part, 0, false, offset)
 }
 });
 let groups = RAYON.install(|| groups.collect::<Vec<_>>());
 flatten_par(&groups)
 } else {
 partition_to_groups(values, null_count as IdxSize, nulls_first, 0)
 }
 }

The scan itself is a single loop over the values, comparing each element to the first element of the current run:

 rust
 
 
// crates/polars-arrow/src/legacy/kernels/sort_partition.rs:91
pub fn partition_to_groups_amortized<T>(
 values: &[T],
 first_group_offset: IdxSize,
 nulls_first: bool,
 offset: IdxSize,
 out: &mut Vec<[IdxSize; 2]>,
) where
 T: Debug + TotalEq + Sized,
{
 if let Some(mut first) = values.first() {
 out.clear();
 if nulls_first && first_group_offset > 0 {
 out.push([0, first_group_offset])
 }

 let mut first_idx = if nulls_first { first_group_offset } else { 0 } + offset;

 for val in values {
 // new group reached
 if val.tot_ne(first) {
 let val_ptr = val as *const T;
 let first_ptr = first as *const T;

 // SAFETY:
 // all pointers suffice the invariants
 let len = unsafe { val_ptr.offset_from(first_ptr) } as IdxSize;
 out.push([first_idx, len]);
 first_idx += len;
 first = val;
 }
 }

The trick in that loop is that the length of a run is computed by pointer arithmetic — val_ptr.offset_from(first_ptr) — rather than by counting. The loop never increments a counter; it just watches the pointer move. The comparison is tot_ne, the total-order equality, so floats behave (NaN is never equal to itself, but it is equal to itself under total order, and -0.0 and 0.0 compare equal).

Nulls get special treatment because Polars sorts them to one end of the array: the null run is emitted as its own group at the front or the back, and the values slice is trimmed to exclude it.

Making the scan parallel

A naive “split the array in half and scan each half” would be wrong: a group could straddle the boundary. Polars cuts the array at group boundaries instead, and the search for those boundaries is itself cheap because the array is sorted:

 rust
 
 
// crates/polars-arrow/src/legacy/kernels/sort_partition.rs:10
/// Find partition indexes such that every partition contains unique groups.
fn find_partition_points<T>(values: &[T], n: usize, descending: bool) -> Vec<usize>
where
 T: Debug + NativeType,
{
 let len = values.len();
 if n > len {
 return find_partition_points(values, len / 2, descending);
 }
 if n < 2 {
 return vec![];
 }
 let chunk_size = len / n;

 let mut partition_points = Vec::with_capacity(n + 1);

 let mut start_idx = 0;
 loop {
 let end_idx = start_idx + chunk_size;
 if end_idx >= len {
 break;
 }
 // first take that partition as a slice
 // and then find the location where the group of the latest value starts
 let part = &values[start_idx..end_idx];

 let latest_val = values[end_idx];
 let idx = if descending {
 part.partition_point(|v| v.tot_gt(&latest_val))
 } else {
 part.partition_point(|v| v.tot_lt(&latest_val))
 };

 if idx != 0 {
 partition_points.push(idx + start_idx)
 }

 start_idx += chunk_size;
 }
 partition_points
}

For each candidate boundary it looks at the value just past the boundary and binary-searches backwards within the chunk for the start of that value’s run. That gives “clean partitions”: every partition contains whole groups. create_clean_partitions then slices the array at those points, and each thread scans one slice independently. The offsets are fixed up by pointer arithmetic in create_groups_from_sorted so that the group offsets are global row indices, and the per-partition results are concatenated.

The null group is handled by only emitting it in one partition (the first if nulls come first, the last if nulls come last), with the other partitions compensating by an offset shift.

What the hash path looks like

For comparison, this is the same operation when the keys are not sorted:

 rust
 
 
// crates/polars-core/src/frame/group_by/hashing.rs:75
pub(crate) fn group_by<K>(keys: impl Iterator<Item = K>, sorted: bool) -> GroupsType
where
 K: TotalHash + TotalEq,
{
 let init_size = get_init_size();
 let (mut first, mut groups);
 if sorted {
 groups = Vec::with_capacity(get_init_size());
 first = Vec::with_capacity(get_init_size());
 let mut hash_tbl = PlHashMap::with_capacity(init_size);
 for (idx, k) in keys.enumerate_idx() {
 match hash_tbl.entry(TotalOrdWrap(k)) {
 Entry::Vacant(entry) => {
 let group_idx = groups.len() as IdxSize;
 entry.insert(group_idx);
 groups.push(unitvec![idx]);
 first.push(idx);
 },
 Entry::Occupied(entry) => unsafe {
 groups.get_unchecked_mut(*entry.get() as usize).push(idx)
 },
 }
 }
 } else {
 let mut hash_tbl = PlHashMap::with_capacity(init_size);
 for (idx, k) in keys.enumerate_idx() {
 match hash_tbl.entry(TotalOrdWrap(k)) {
 Entry::Vacant(entry) => {
 entry.insert((idx, unitvec![idx]));
 },
 Entry::Occupied(mut entry) => entry.get_mut().1.push(idx),
 }
 }
 (first, groups) = hash_tbl.into_values().unzip();
 }
 GroupsType::Idx(GroupsIdx::new(first, groups, sorted))
}

The hash path allocates a hash table, a unitvec per group, and produces GroupsType::Idx — a list of row indices per group. The sorted path produces GroupsType::Slice — a list of (offset, len) pairs, where the rows of a group are a contiguous range.

The multi-threaded variant of the hash path does not partition the rows; it partitions the hash space, so every thread has to look at every key:

 rust
 
 
// crates/polars-core/src/frame/group_by/hashing.rs:146
pub(crate) fn group_by_threaded_slice<T, IntoSlice>(
 keys: Vec<IntoSlice>,
 n_partitions: usize,
 sorted: bool,
) -> GroupsType
where
 T: ToTotalOrd,
 <T as ToTotalOrd>::TotalOrdItem: Send + Sync + Copy + DirtyHash,
 IntoSlice: AsRef<[T]> + Send + Sync,
{
 let init_size = get_init_size();

 // We will create a hashtable in every thread.
 // We use the hash to partition the keys to the matching hashtable.
 // Every thread traverses all keys/hashes and ignores the ones that doesn't fall in that partition.
 let out = RAYON.install(|| {
 (0..n_partitions)
 .into_par_iter()
 .map(|thread_no| {
 let mut hash_tbl = PlHashMap::with_capacity(init_size);

 let mut offset = 0;
 for keys in &keys {
 let keys = keys.as_ref();
 let len = keys.len() as IdxSize;

 for (key_idx, k) in keys.iter().enumerate_idx() {
 let k = k.to_total_ord();
 let idx = key_idx + offset;

 if thread_no == hash_to_partition(k.dirty_hash(), n_partitions) {
 match hash_tbl.entry(k) {
 Entry::Vacant(entry) => {
 entry.insert((idx, unitvec![idx]));
 },
 Entry::Occupied(mut entry) => {
 entry.get_mut().1.push(idx);
 },
 }
 }
 }
 offset += len;
 }
 hash_tbl
 .into_iter()
 .map(|(_k, v)| v)
 .collect_trusted::<Vec<_>>()
 })
 .collect::<Vec<_>>()
 });
 finish_group_order(out, sorted)
}

Note the shape of the work: every thread reads every key, computes its hash, and throws away the keys that are not its own. The hash table is partitioned rather than the data, so there is no work partitioning to do — but each thread touches the whole array. The sorted path does the opposite: the data is partitioned, so a thread only reads its own slice, and the only coordination is choosing where to cut.

That difference propagates all the way into the aggregations. Every aggregation has a match arm per group type:

 rust
 
 
// crates/polars-core/src/frame/group_by/aggregations/dispatch.rs:30
 pub unsafe fn agg_valid_count(&self, groups: &GroupsType) -> Series {
 // Prevent a rechunk for every individual group.
 let valid = self.rechunk_validity();

 match groups {
 GroupsType::Idx(groups) => agg_helper_idx_on_all::<IdxType, _>(groups, |idxs| {
 debug_assert!(idxs.len() <= self.len());
 if let Some(v) = &valid {
 let mut count = 0;
 for idx in idxs.iter() {
 count += unsafe { v.get_bit_unchecked(*idx as usize) as IdxSize };
 }
 Some(count)
 } else {
 Some(idxs.len() as IdxSize)
 }
 }),
 GroupsType::Slice { groups, .. } => {
 _agg_helper_slice::<IdxType, _>(groups, |[first, len]| {
 debug_assert!(len <= self.len() as IdxSize);
 if let Some(v) = &valid {
 let m = BitMask::from_bitmap(v).sliced(first as usize, len as usize);
 Some(m.set_bits() as IdxSize)
 } else {
 Some(len)
 }
 })
 },
 }
 }

The Idx arm walks a list of indices and tests each one’s validity bit. The Slice arm takes a bitmap slice — a contiguous run of bits — and counts the set bits in it, which is a popcount over a machine word rather than a per-row branch. Same answer, very different instruction mix.

The slice representation also enables a second class of optimization: an aggregation over a contiguous range can use rolling kernels, since the ranges are themselves monotone. agg_quantile_generic has a branch that checks whether the group slices are monotonic and, if so, hands the whole thing to a rolling-window kernel instead of computing a quantile per group.

The metadata that travels with the groups

The sorted fast path also leaves a trace in the output. When the group-by produces its key columns, it copies the sorted flag of the input key onto the output:

 rust
 
 
// crates/polars-core/src/frame/group_by/mod.rs:263
 GroupsType::Slice {
 groups,
 overlapping,
 monotonic: _,
 } => {
 if *overlapping && !groups.is_empty() {
 // Groups can be sliced.
 let offset = groups[0][0];
 let [upper_offset, upper_len] = groups[groups.len() - 1];
 return s.slice(
 offset as i64,
 ((upper_offset + upper_len) - offset) as usize,
 );
 }

 let indices = groups
 .iter()
 .map(|&[first, _len]| first)
 .collect_ca(PlSmallStr::EMPTY);
 // SAFETY: groups are always in bounds.
 let mut out = unsafe { s.take_unchecked(&indices) };
 // Sliced groups are always in order of discovery.
 out.set_sorted_flag(s.is_sorted_flag());
 out
 },

So df.sort("k").group_by("k").agg(...) produces a result whose k column is still flagged as sorted, and a subsequent group_by on that result takes the fast path again. The same reasoning is used for the Idx variant, but only when the groups are known to be ordered by first index.

The GroupsType::Slice variant carries two more pieces of metadata:

 rust
 
 
// crates/polars-core/src/frame/group_by/position.rs:260
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum GroupsType {
 Idx(GroupsIdx),
 Slice {
 // the groups slices
 groups: GroupsSlice,
 /// Indicates if the groups may overlap, i.e., at least one index MAY be
 /// included in more than one group slice.
 overlapping: bool,
 /// Indicates if the groups are rolling, i.e. for every consecutive group
 /// slice (offset, len), and given start = offset and end = offset + len,
 /// then both new_start >= start AND new_end >= end MUST be true.
 monotonic: bool,
 },
}

monotonic says the group ranges are themselves ordered and non-shrinking, which is exactly the precondition for a rolling kernel. overlapping says a row may appear in more than one group, which is false for a group-by but true for e.g. a window function. Both invariants are checked in debug builds:

 rust
 
 
// crates/polars-core/src/frame/group_by/position.rs:308
 pub fn new_slice(groups: GroupsSlice, overlapping: bool, monotonic: bool) -> Self {
 #[cfg(debug_assertions)]
 {
 fn groups_overlap(groups: &GroupsSlice) -> bool {
 if groups.len() < 2 {
 return false;
 }
 let mut groups = groups.clone();
 groups.sort();
 let mut prev_end = groups[0][1];

 for g in &groups[1..] {
 let start = g[0];
 let end = g[1];
 if start < prev_end {
 return true;
 }
 if end > prev_end {
 prev_end = end;
 }
 }
 false
 }

 assert!(overlapping || !groups_overlap(&groups));
 assert!(!monotonic || slice_groups_are_monotonic(&groups));
 }

The streaming version

The same idea exists in the streaming engine, where the input arrives in morsels and you cannot assume the whole column is in memory. There is a dedicated node for it, and it does the same run-length scan — but on a buffer that is flushed at group boundaries:

 rust
 
 
// crates/polars-stream/src/nodes/sorted_group_by.rs:86
 let mut offset = 0;
 let groups = idxs
 .iter()
 .map(|i| {
 let start = offset;
 offset += i;
 [start, *i]
 })
 .collect();
 let groups = GroupsType::new_slice(groups, false, true).into_sliceable();

idxs comes from rle_lengths(column, idxs), which computes the length of each run of equal values — the same run-length encoding, but as a list of lengths instead of a list of ranges. The node then buffers morsels until the buffer holds more than one distinct key, and flushes at a point where a group cannot straddle the boundary:

 rust
 
 
// crates/polars-stream/src/nodes/sorted_group_by.rs:247
 if fst == lst {
 continue;
 }

 let mut buf_key_column = buf_key_column.as_materialized_series().clone();
 buf_key_column.set_sorted_flag(IsSorted::Ascending);

 let descending = fst > lst;
 let num_flushable = search_sorted(
 &buf_key_column,
 &buf_key_column.tail(Some(1)),
 SearchSortedSide::Left,
 descending,
 )
 .unwrap();

The bottom line

The design in one sentence: sortedness is metadata, and metadata is exploited at every stage that sees it.

The trade-offs:

The fast path trusts the flag. If the flag is wrong, the group-by produces wrong groups — a silent correctness bug. That is why the flag is cleared on every mutation that could break the order, and why the sorted path rechunks first (it needs one contiguous chunk) but never re-validates.
It only pays off for a single key column. The multi-key path in group_by_with_series encodes all keys into row bytes first, and then hashes those bytes; the row bytes have no sortedness flag, so the sorted fast path is not available there.
The output type changes. GroupsType::Slice instead of GroupsType::Idx is not just a different representation: it selects slice-based aggregation code, enables rolling kernels through the monotonic flag, and lets the sorted flag survive into the output so the next group-by can take the same path.
Nulls are a special case everywhere. Nulls must be at one end of the sorted array, and the scan has to know which end, and the parallel version has to emit the null group exactly once.

Polars Groups Columns So One Pass Feeds Many Aggregations

Jay Hubbert — Wed, 21 Oct 2026 06:20:00 +0000

TL;DR: A group-by in Polars is two stages, not one. The first stage turns key columns into group ids: the keys are encoded (four different ways, depending on their dtype and count), hashed, and looked up in a hash table that hands out a group index per distinct key. The second stage is every aggregation expression, each of which consumes the same group structure. One grouping pass, many aggregations.

The streaming engine pushes this further: a fixed-size hot table keeps the frequent keys in memory, the rare ones are spilled, and the hot table is allowed to evict.

A naive group-by implementation computes the groups once per aggregation: df.group_by("k").agg(sum("a")) and df.group_by("k").agg(mean("b")) would each build their own hash table over the keys. That is a lot of duplicated work, and it scales badly with the number of aggregations.

Polars separates the two concerns. The grouping structure is computed once, and the aggregations are expressions that read it.

Two grouping engines, not one

Before going further: the type you may know as GroupsProxy is not in this tree. It has been split in two, and the split follows the execution engine:

the eager engine (polars-core) groups a materialized DataFrame into group tuples — GroupsType::Idx (a list of row indices per group) or GroupsType::Slice (a list of (offset, len) ranges, for sorted input);
the lazy/streaming engine (polars-expr) groups hash keys into group indices through the Grouper trait, because in a streaming query the keys arrive in morsels and the group table has to grow incrementally.

The eager one is the older design and is worth a look, because it is where the “sorted input” fast path lives:

 rust
 
 
// crates/polars-core/src/frame/group_by/into_groups.rs:20
/// Used to create the tuples for a group_by operation.
pub trait IntoGroupsType {
 /// Create the tuples need for a group_by operation.
 /// * The first value in the tuple is the first index of the group.
 /// * The second value in the tuple is the indexes of the groups including the first value.
 fn group_tuples(&self, _multithreaded: bool, _sorted: bool) -> PolarsResult<GroupsType> {
 unimplemented!()
 }
}

Note the sorted flag: it is a promise from the caller that the key column is already sorted, and it unlocks a path that never builds a hash table at all:

 rust
 
 
// crates/polars-core/src/frame/group_by/into_groups.rs:65
 fn create_groups_from_sorted(&self, multithreaded: bool) -> GroupsSlice {
 if verbose() {
 eprintln!("group_by keys are sorted; running sorted key fast path");
 }

Sorted keys mean the groups are contiguous ranges of rows, so the whole grouping reduces to a scan for run boundaries, which can be done in parallel over partitions of the sorted array.

For everything else, the eager path hashes:

 rust
 
 
// crates/polars-core/src/frame/group_by/into_groups.rs:30
fn num_groups_proxy<T>(ca: &ChunkedArray<T>, multithreaded: bool, sorted: bool) -> GroupsType
where
 T: PolarsNumericType,
 T::Native: TotalHash + TotalEq + DirtyHash + ToTotalOrd,

TotalHash/TotalEq are the important detail: they give floats a total order, so NaN == NaN and -0.0 == 0.0 behave sanely inside a hash table.

The `Grouper` trait

The lazy/streaming side has one trait, and it is small:

 rust
 
 
// crates/polars-expr/src/groups/mod.rs:45
/// A Grouper maps keys to groups, such that duplicate keys map to the same group.
pub trait Grouper: Any + Send + Sync {
 /// Creates a new empty Grouper similar to this one.
 fn new_empty(&self) -> Box<dyn Grouper>;

 /// Reserves space for the given number additional groups.
 fn reserve(&mut self, additional: usize);

 /// Returns the number of groups in this Grouper.
 fn num_groups(&self) -> IdxSize;

 /// Inserts the given subset of keys into this Grouper. If groups_idxs is
 /// passed it is extended such with the group index of keys[subset[i]].
 /// New groups get consecutive indices in the order they first occur.
 ///
 /// # Safety
 /// The subset indexes must be in-bounds.
 unsafe fn insert_keys_subset(
 &mut self,
 keys: &HashKeys,
 subset: &[IdxSize],
 group_idxs: Option<&mut Vec<IdxSize>>,
 );

 /// Returns the keys in this Grouper in group order, that is the key for
 /// group i is returned in row i.
 fn get_keys_in_group_order(&self, schema: &Schema) -> DataFrame;

That is the whole contract: insert keys, get group indices out, hand back the distinct keys in group order. The trait does not mention aggregations at all, which is exactly the point — the grouping structure is reusable by any number of consumers.

The other three methods on the trait (probe_partitioned_groupers, contains_key_partitioned_groupers, mark_groups_partitioned_groupers) exist for partitioned hash joins: they probe other groupers by hash partition, so a join can look up a key in the partition that owns it without searching all of them.

Four encodings, one interface

The interesting decision is how to represent a key so that it can be hashed and compared. Polars picks between four encodings based on the key schema:

 rust
 
 
// crates/polars-expr/src/groups/mod.rs:113
pub fn new_hash_grouper(key_schema: Arc<Schema>) -> Box<dyn Grouper> {
 if let Some(layout) = KeyRowLayout::new(key_schema.iter_values()) {
 Box::new(key_rows::KeyRowHashGrouper::new(Arc::new(layout)))
 } else if key_schema.len() > 1 {
 Box::new(row_encoded::RowEncodedHashGrouper::new())
 } else {
 let (_name, dt) = key_schema.get_at_index(0).unwrap();
 match dt {
 dt if dt.is_primitive_numeric() | dt.is_temporal() => {
 with_match_physical_numeric_polars_type!(dt.to_physical(), |$T| {
 Box::new(single_key::SingleKeyHashGrouper::<$T>::new())
 })
 },

 #[cfg(feature = "dtype-decimal")]
 DataType::Decimal(_, _) => {
 Box::new(single_key::SingleKeyHashGrouper::<Int128Type>::new())
 },
 #[cfg(feature = "dtype-categorical")]
 dt @ (DataType::Enum(_, _) | DataType::Categorical(_, _)) => {
 with_match_categorical_physical_type!(dt.cat_physical().unwrap(), |$C| {
 Box::new(single_key::SingleKeyHashGrouper::<<$C as PolarsCategoricalType>::PolarsPhysical>::new())
 })
 },

 DataType::String | DataType::Binary => Box::new(binview::BinviewHashGrouper::new()),

 _ => Box::new(row_encoded::RowEncodedHashGrouper::new()),
 }
 }
}

Reading that top to bottom: if the key columns can be laid out as fixed-width key rows, use that; otherwise if there is more than one key column, use generic row encoding; otherwise, for a single key, use the most specialized grouper the dtype allows — a direct hash table over the physical values for numbers and dates, a view-aware one for strings, and generic row encoding for anything exotic.

Single key: the values themselves

For one numeric or temporal key, no encoding is needed at all — the physical value is the key:

 rust
 
 
// crates/polars-expr/src/groups/single_key.rs:13
pub struct SingleKeyHashGrouper<T: PolarsDataType> {
 /// Each key with its group.
 table: HashTable<(T::Physical<'static>, IdxSize)>,
 num_groups: IdxSize,
 /// The group of the null key, IdxSize::MAX if there is none.
 null_idx: IdxSize,
 random_state: PlRandomState,
}

and the insert path is a textbook open-addressing hash table, with two hardware-friendly touches. First, the null key gets its own group index rather than being hashed, so nulls are never confused with a real value. Second, the work is done in blocks: the hashes of a whole block are computed (and prefetched) before the lookups happen, so the CPU has several independent memory accesses in flight:

 rust
 
 
// crates/polars-expr/src/groups/single_key.rs:64
 unsafe fn insert_impl(
 &mut self,
 arr: &T::Array,
 null_is_valid: bool,
 subset: &[IdxSize],
 mut on_group: impl FnMut(IdxSize),
 ) {

 rust
 
 
// crates/polars-expr/src/groups/single_key.rs:82
 let mut hashes = [0; BLOCK_SIZE];
 for block in subset.chunks(BLOCK_SIZE) {
 // Reserve up front so the table doesn't move after prefetching.
 self.table.reserve(block.len(), hasher);
 let prefetch = self.should_prefetch();
 for (hash, idx) in hashes.iter_mut().zip(block) {
 *hash = random_state.tot_hash_one(arr.value_unchecked(*idx as usize));
 if prefetch {
 self.table.prefetch_insert(*hash);
 }
 }

That “reserve up front so the table doesn’t move after prefetching” comment is the kind of detail that only shows up after a profiler: reallocating the table after issuing prefetches would invalidate the addresses you just prefetched.

Key rows: a hand-rolled row format

The newest and most interesting encoding is key_rows. Instead of using the generic polars-row encoder, Polars lays out the key columns itself into a row format with a fixed stride, and the layout is computed once per schema:

 rust
 
 
// crates/polars-expr/src/key_rows/layout.rs:72
/// The row format of keys of one schema. It depends only on the key dtypes: the null
/// bits are always present, also when nulls are not keys, so keys that disagree on
/// whether nulls match (the two sides of an anti join) share one layout.
///
/// Rows are written into zeroed words and null values are left zero, so equal keys
/// have equal words, including padding. The one exception is the second word of a
/// long view, which holds where its bytes are stored: equality skips that word and
/// compares the bytes instead.
#[derive(Debug, PartialEq)]
pub(crate) struct KeyRowLayout {
 pub(super) cols: Vec<ColLayout>,
 /// Byte offset of the null bits, one per column.
 pub(super) null_offset: usize,
 null_width: usize,
 pub(super) stride_words: usize,
 /// The first word of each view column, in column order.
 pub(super) view_words: Vec<usize>,
 /// Every word except the second half of each view.
 pub(super) plain_words: Vec<usize>,
}

Three design decisions are worth spelling out:

Nulls live in the row. One null bit per key column, always present even when nulls are not part of the key. That means the two sides of an anti join (where one side treats null as a key and the other does not) can share one layout and one hash table.
Columns are sorted by width, descending. The layout puts the widest columns first so that the whole row is a power-of-two number of 64-bit words, and so that comparisons can bail out early on the most discriminating word:

 rust
 
 
// crates/polars-expr/src/key_rows/layout.rs:90
 let null_width = cols.len().div_ceil(8).next_power_of_two();
 let mut fields: Vec<(usize, Option<usize>)> = widths
 .into_iter()
 .enumerate()
 .map(|(i, width)| (width, Some(i)))
 .chain([(null_width, None)])
 .collect();
 fields.sort_by_key(|(width, _)| Reverse(*width));

Equality is a word-wise XOR fold. Because rows are written into zeroed words, “equal” means “no differing bits”, and that is one instruction per word for the fixed-width part:

 rust
 
 
// crates/polars-expr/src/key_rows/layout.rs:170
 pub(super) unsafe fn rows_eq(
 &self,
 a: &[u64],
 b: &[u64],
 mut long_eq: impl FnMut(View, View) -> bool,
 ) -> bool {
 if self.view_words.is_empty() {
 return a.iter().zip(b).fold(0, |acc, (x, y)| acc | (x ^ y)) == 0;
 }
 for w in &self.plain_words {
 if a.get_unchecked(*w) != b.get_unchecked(*w) {
 return false;
 }
 }

Strings (Arrow “views”) are the awkward case: a view is a 16-byte struct where short strings are inlined and long strings point into a buffer. The second half of a view word holds a location, not data, so it is skipped by the word comparison and the bytes are compared instead.

Row encoding and binview: the fallbacks

Generic row encoding uses the polars-row crate and stores the encoded bytes in a byte-indexed map:

 rust
 
 
// crates/polars-expr/src/groups/row_encoded.rs:12
#[derive(Default)]
pub struct RowEncodedHashGrouper {
 idx_map: BytesIndexMap<()>,
}

 rust
 
 
// crates/polars-expr/src/groups/row_encoded.rs:23
 fn insert_key(&mut self, hash: u64, key: &[u8]) -> IdxSize {
 match self.idx_map.entry(hash, key) {
 Entry::Occupied(o) => o.index(),
 Entry::Vacant(v) => {
 let index = v.index();
 v.insert(());
 index
 },
 }
 }

For a single string key, there is a dedicated grouper that compares the Arrow views directly, so a 100 MB string column is never copied into a row buffer:

 rust
 
 
// crates/polars-expr/src/groups/binview.rs:33
 /// # Safety
 /// The view must be valid for the given buffer set.
 #[inline(always)]
 unsafe fn insert_key(
 &mut self,
 hash: u64,
 view: View,
 buffers: &Buffer<Buffer<u8>>,
 ) -> IdxSize {
 unsafe {
 match self.idx_map.entry_view(hash, view, buffers) {

The keys themselves

The Grouper does not take a DataFrame of keys; it takes a HashKeys, which is the encoded key material plus a precomputed hash per row:

 rust
 
 
// crates/polars-expr/src/hash_keys.rs:116
/// Represents a DataFrame plus a hash per row, intended for keys in grouping
/// or joining. The hashes may or may not actually be physically pre-computed,
/// this depends per type.
#[derive(Clone, Debug)]
pub enum HashKeys {
 RowEncoded(RowEncodedKeys),
 KeyRows(KeyRowKeys),
 Binview(BinviewKeys),
 Single(SingleKeys),
}

The from_df constructor is where the encoding decision is actually made, and it prefers key rows over everything else:

 rust
 
 
// crates/polars-expr/src/hash_keys.rs:130
 if !force_row_encoding
 && let Some(layout) = KeyRowLayout::new(df.columns().iter().map(|c| c.dtype()))
 {
 return Self::KeyRows(KeyRowKeys::from_columns(
 df.columns(),
 Arc::new(layout),
 &random_state,
 null_is_valid,
 ));
 }

 let first_col_variant = hash_keys_variant_for_dtype(df[0].dtype());
 let use_row_encoding = force_row_encoding
 || df.width() > 1
 || first_col_variant == HashKeysVariant::RowEncoded;

Precomputing the hash per row is what makes partitioned joins and the partitioned groupers possible: a key’s hash determines its partition, so a probe only has to look in one partition’s table.

One pass, many aggregations

Now the payoff. The eager engine groups once and then runs every aggregation against the same group structure:

 rust
 
 
// crates/polars-mem-engine/src/executors/group_by.rs:68
pub(super) fn group_by_helper(
 mut df: DataFrame,
 keys: Vec<Column>,
 aggs: &[Arc<dyn PhysicalExpr>],
 apply: Option<PlanCallback<DataFrame, DataFrame>>,
 state: &ExecutionState,
 maintain_order: bool,
 output_schema: &SchemaRef,
 slice: Option<(i64, usize)>,
) -> PolarsResult<DataFrame> {
 df.rechunk_mut_par();
 let gb = df.group_by_with_series(keys, true, maintain_order)?;

 rust
 
 
// crates/polars-mem-engine/src/executors/group_by.rs:15
pub(super) fn evaluate_aggs(
 df: &DataFrame,
 aggs: &[Arc<dyn PhysicalExpr>],
 groups: &GroupPositions,
 state: &ExecutionState,
) -> PolarsResult<Vec<Column>> {
 RAYON.install(|| {
 aggs.par_iter()
 .map(|expr| {
 let agg = expr.evaluate_on_groups(df, groups, state)?.finalize();
 polars_ensure!(agg.len() == groups.len(), agg_len = agg.len(), groups.len());
 Ok(agg)
 })
 .collect::<PolarsResult<Vec<_>>>()
 })
}

groups is computed once and then shared, read-only, by every aggregation expression, which are themselves free to run in parallel. The lazy engine does the same thing, one level down: an aggregation expression receives the shared GroupPositions and reduces its own input column against it:

 rust
 
 
// crates/polars-expr/src/expressions/aggregation.rs:232
 fn evaluate_on_groups_impl<'a>(
 &self,
 df: &DataFrame,
 groups: &'a GroupPositions,
 state: &ExecutionState,
 ) -> PolarsResult<AggregationContext<'a>> {
 let mut ac = self.input.evaluate_on_groups(df, groups, state)?;

The GroupedReduction trait is the streaming-engine equivalent, and it is designed so that many reductions can share one grouping structure:

 rust
 
 
// crates/polars-expr/src/reduce/mod.rs:163
 /// Updates the specified group with the given values.
 ///
 /// For order-sensitive grouped reductions, seq_id can be used to resolve
 /// order between calls/multiple reductions.
 fn update_group(
 &mut self,
 values: &[&Column],
 group_idx: IdxSize,
 seq_id: u64,
 ) -> PolarsResult<()>;

and the partition table of the streaming group-by shows the pattern in full: one grouper, N reductions, one pass over the rows:

 rust
 
 
// crates/polars-stream/src/nodes/group_by/table.rs:143
 pub fn add_rows(
 &mut self,
 input: usize,
 frame: &FrameWithKeys,
 rows: Option<&[IdxSize]>,
 ) -> PolarsResult<()> {
 let rows = rows_or_all(rows, &mut self.all_rows, frame.keys.len());
 self.group_idxs.clear();
 unsafe {
 self.grouper
 .insert_keys_subset(&frame.keys, rows, Some(&mut self.group_idxs));
 }
 let num_groups = self.grouper.num_groups();
 let params = &*self.params;
 let mut in_cols = Vec::new();
 for r_idx in &params.reductions_per_input[input] {
 let r = &mut self.reductions[*r_idx];
 r.resize(num_groups);
 in_cols.clear();
 in_cols.extend(
 params.grouped_reduction_cols[*r_idx]
 .iter()
 .map(|col| frame.frame.column(col).unwrap()),
 );
 // Cold rows only exist without order-sensitive reductions.
 unsafe { r.update_groups_subset(&in_cols, rows, &self.group_idxs, 0)? };
 }
 Ok(())
 }

The group indices computed by the grouper are handed to every reduction. r.resize(num_groups) grows the reduction’s state to the number of groups found so far; update_groups_subset then adds the values of the given rows to the right groups. Adding a tenth aggregation to a query costs a tenth reduction state and a tenth pass over the group’s rows — but not a tenth grouping pass.

The hot table

There is a second, more aggressive version of this idea in the streaming engine. Because a group-by over a large table has to hold the group table in memory, Polars splits the keys into hot (frequent) and cold (rare) ones:

 rust
 
 
// crates/polars-stream/src/nodes/group_by/mod.rs:1
//! The streaming group-by.
//!
//! While receiving input, each pipeline aggregates rows with frequent keys in small hot tables
//! and stores the other (cold) rows with their hashed keys in spillable frames. Groups evicted
//! from a hot table, and the hot tables at the end of the input, become pre-aggregates. Once all
//! input is received, the cold rows and pre-aggregates are split into hash partitions, which are
//! aggregated independently. With at most one partition per pipeline, all partitions are built
//! before sending. Otherwise the cold rows are first stored per partition, and the partitions are
//! built a few at a time while the results are sent.

The hot table is a HotGrouper, a sibling of Grouper with one crucial difference — it has a fixed size and evicts rather than growing:

 rust
 
 
// crates/polars-expr/src/hot_groups/mod.rs:21
/// A HotGrouper maps keys to groups, such that duplicate keys map to the same
/// group. Unlike a Grouper it has a fixed size and will cause evictions rather
/// than growing.
pub trait HotGrouper: Any + Send + Sync {
 /// Creates a new empty HotGrouper similar to this one, with the given size.
 fn new_empty(&self, groups: usize) -> Box<dyn HotGrouper>;

 /// Returns the number of groups in this HotGrouper.
 fn num_groups(&self) -> IdxSize;

 /// Returns the number of slots in the hot table.
 fn num_slots(&self) -> usize;

 /// Doubles the number of slots in the hot table. Group indices are unchanged.
 fn double(&mut self);

 /// Inserts the given keys into this Grouper, extending groups_idxs with
 /// the group index of keys[i].
 ///
 /// A missed key that is followed by a key with the same hash is inserted
 /// even if that evicts another key, as is every missed key with
 /// `force_hot`. This keeps sorted keys from first going cold.
 fn insert_keys(
 &mut self,
 keys: &HashKeys,
 hot_idxs: &mut Vec<IdxSize>,
 hot_group_idxs: &mut Vec<EvictIdx>,
 cold_idxs: &mut Vec<IdxSize>,
 force_hot: bool,
 );

The EvictIdx type is the trick that makes this work: a group index whose top bit is set means “this group was evicted, reset it before updating”. The reduction state carries the same flag, so an evicted group’s partial sum is discarded and rebuilt from the cold rows later. The note about “a missed key that is followed by a key with the same hash” is about sorted input: if the keys are sorted, the same key appears in a run, and evicting the first occurrence and re-inserting the second would defeat the purpose, so the grouper is allowed to evict on a hash collision instead.

The sink keeps both halves:

 rust
 
 
// crates/polars-stream/src/nodes/group_by/sink.rs:60
/// The state of one pipeline of the sink.
struct LocalSink {
 // Created when the input first sends this pipeline a non-empty morsel.
 hot_grouper_per_input: Vec<Option<Box<dyn HotGrouper>>>,
 hot_grouped_reductions: Vec<Box<dyn GroupedReduction>>,

 // The number of slots of each hot grouper, and the rows missed by them.
 hot_table_size: usize,
 miss_f2: F2Sketch,

 // The keys of the cold rows and the pre-aggregates.
 key_sketch: CardinalitySketch,

 // Per input, the cold rows not yet in a frame.
 cold_row_builders: Vec<Option<FrameWithKeysBuilder>>,
 // Each frame with the input it came from.
 cold_frames: Vec<(usize, SpillFrameWithKeys)>,
 // The estimated bytes of `cold_frames`.
 cold_bytes: u64,
 // In creation order.
 pre_aggs: Vec<PreAgg>,
}

The hot table grows only when the evidence says it should — the F2 sketch counts how often keys are missed, and the table is enlarged only if the missed keys repeat enough and dominate the misses:

 rust
 
 
// crates/polars-stream/src/nodes/group_by/sink.rs:47
/// The hot tables only grow if the frequently missed keys were missed at least this
/// many times each.
const HOT_TABLE_GROW_MIN_REPEAT: f64 = 4.0;

/// The hot tables only grow if the frequently missed keys make up at least this
/// fraction of the missed rows.
const HOT_TABLE_GROW_MIN_HEAVY_SHARE: f64 = 0.5;

The bottom line

The design in one sentence: grouping is a data structure, not a step. The group structure is built once, by a Grouper that picks an encoding suited to the key schema, and then it is a read-only input to every aggregation.

The trade-offs that come with it:

Encoding choice is per-query, not per-user. A single numeric key gets a direct hash table over the values; a multi-column key gets fixed-width key rows; strings get view comparisons; anything else falls back to generic row encoding. The cost is four implementations of one trait, and the benefit is that the common cases avoid copying key data at all.
The group structure is shared, so it is immutable during aggregation. Every aggregation reads it; none of them may reorder it. That is why evaluate_aggs can run the aggregations in parallel over one groups.
The hot table trades exactness of memory prediction for speed. It is a fixed-size cache of frequent keys, and it evicts. The evicted groups are not lost, they are re-derived from the spilled cold rows, which means the grouping work is done twice for evicted keys — a deliberate exchange of CPU for a bounded memory footprint.

Polars Hash-Shuffles Rows Between Streaming Stages

Jay Hubbert — Wed, 21 Oct 2026 05:40:00 +0000

TL;DR: A streaming group-by has to get every row with the same key into the same place, and a streaming join has to get both sides of a matching key into the same place. Polars does this with a hash shuffle, not a sort: each row’s key is hashed, the hash is mapped to one of N partitions, and each partition is then aggregated or joined on its own. Because a partition only ever contains one key’s worth of rows, the per-partition work is a plain hash table, with no comparison sort anywhere in the pipeline.

The classic way to co-locate equal keys is to sort. Sort the input by the key, then every run of equal keys is contiguous and a group-by or a merge join is a linear scan. That works, and it is what a lot of engines do. It is also a strange fit for a streaming engine, where the whole point is to never materialise the whole input: a sort has to hold the entire dataset (or spill all of it) before it can produce its first output row.

Polars’ answer is the shuffle. Same idea as a Spark repartition or a DataFusion RepartitionExec, but the implementation has some details worth reading.

The partition function

Everything rests on one function (crates/polars-utils/src/hashing.rs:109):

 rust
 
 
 /// Converts a hash to a partition. It is guaranteed that the output is
 /// in the range [0, n_partitions), and that independent HashPartitioners
 /// that we initialized with the same num_partitions and seed return the same
 /// partition.
 #[inline(always)]
 pub fn hash_to_partition(&self, hash: u64) -> usize {
 // Assuming r is a 64-bit random number, we note that
 // r / 2^64 is almost a uniform random number in [0, 1), and thus
 // floor(r * n_partitions / 2^64) is almost a uniform random integer in
 // [0, n_partitions). Despite being written with u128 multiplication this
 // compiles to a single mul / mulhi instruction on x86-x64/aarch64.
 let shuffled = hash.wrapping_mul(self.seed);
 ((shuffled as u128 * self.num_partitions as u128) >> 64) as usize
 }

Two details are doing real work here.

The first is the wrapping_mul by the seed. The incoming hash is a value hash — hash(42) == hash(42) for any key column — but value hashes have terrible distribution in their low bits. Numeric key columns in particular hash to something close to the value itself, so hash % n_partitions would send every value in a contiguous run to the same partition. Multiplying by a random odd number is a universal hash function for the high bits — the classic “fibonacci hashing” trick — and it is why the high bits are used for the partition index rather than the low ones.

The second detail is that the partition index is computed as floor(h * n / 2^64) rather than h % n. The comment explains why: the u128 multiply-and-shift compiles to a single mul/mulhi pair, so there is no division instruction on the hot path.

The seed is randomised per partitioner (crates/polars-utils/src/hashing.rs:82):

 rust
 
 
 #[inline]
 pub fn new(num_partitions: usize, mut seed: u64) -> Self {
 assert!(num_partitions > 0);
 // Make sure seeds bits are properly randomized, and is odd.
 const ARBITRARY1: u64 = 0x85921e81c41226a0;
 const ARBITRARY2: u64 = 0x3bc1d0faba166294;
 const ARBITRARY3: u64 = 0xfbde893e21a73756;
 seed = folded_multiply(seed ^ ARBITRARY1, ARBITRARY2);
 seed = folded_multiply(seed, ARBITRARY3);
 seed |= 1;
 Self {
 seed,
 num_partitions,
 }
 }

The | 1 is not cosmetic: the multiply-shift trick needs an odd multiplier to be a permutation, otherwise two different hashes could map to the same partition, and rows of different keys would collide into the same bucket. A collision is not a correctness bug (the hash table still compares the actual keys), but it destroys the point of having many partitions.

The other requirement is that both sides of a join must agree. Hence the doc comment: “independent HashPartitioners that we initialized with the same num_partitions and seed return the same partition”. In practice the engine always uses seed 0 (crates/polars-stream/src/nodes/joins/equi_join.rs:607):

 rust
 
 let partitioner = HashPartitioner::new(state.num_pipelines, 0);

so the number of partitions is the number of pipelines, and both sides of the join trivially agree. The group-by picks its own partition count but still uses seed 0 (crates/polars-stream/src/nodes/group_by/mod.rs:235).

Hashes are computed once, and kept with the data

The partition index of a row is derived from a 64-bit hash of its key. That hash is not recomputed every time the row is moved; it is computed once and carried alongside the row. The carrier is HashKeys (crates/polars-expr/src/hash_keys.rs:116):

 rust
 
 
/// Represents a DataFrame plus a hash per row, intended for keys in grouping
/// or joining. The hashes may or may not actually be physically pre-computed,
/// this depends per type.
#[derive(Clone, Debug)]
pub enum HashKeys {
 RowEncoded(RowEncodedKeys),
 KeyRows(KeyRowKeys),
 Binview(BinviewKeys),
 Single(SingleKeys),
}

There are four representations because hashing a u64 key column is completely different work from hashing a String key column. Single is a primitive numeric/temporal/categorical column, where the hash is essentially the value (so it is not physically stored). Binview is strings, which get a real hash. RowEncoded is the general case: the key columns are row-encoded into a byte buffer and then hashed.

Mapping hashes to partitions is (crates/polars-expr/src/hash_keys.rs:304):

 rust
 
 
 /// After this call partitions will be extended with the partition for each
 /// hash. Nulls are assigned IdxSize::MAX or a specific partition depending
 /// on whether partition_nulls is true.
 pub fn gen_partitions(
 &self,
 partitioner: &HashPartitioner,
 partitions: &mut Vec<IdxSize>,
 partition_nulls: bool,
 ) {
 unsafe {
 let null_p = if partition_nulls | self.null_is_valid() {
 partitioner.null_partition() as IdxSize
 } else {
 IdxSize::MAX
 };
 partitions.reserve(self.len());
 self.for_each_hash(|_idx, opt_h| {
 partitions.push_unchecked(
 opt_h
 .map(|h| partitioner.hash_to_partition(h) as IdxSize)
 .unwrap_or(null_p),
 );
 });
 }
 }

Nulls get their own treatment. HashPartitioner::null_partition() returns 0, and nulls are either routed there (when the join needs to match nulls against nulls) or marked with IdxSize::MAX and dropped from the partitioning entirely.

There is a second, faster variant that produces indices per partition rather than a partition per row (crates/polars-expr/src/hash_keys.rs:404):

 rust
 
 
 fn gen_idxs_per_partition_impl<const BUILD_SKETCHES: bool, const BUILD_F2: bool>(
 &self,
 partitioner: &HashPartitioner,
 partition_idxs: &mut [Vec<IdxSize>],
 sketches: &mut [CardinalitySketch],
 mut f2_sketch: Option<&mut F2Sketch>,
 partition_nulls: bool,
 ) {

This is the one the join uses, because it does the partition assignment and the cardinality estimation in a single pass over the hashes. The sketch is a side product: it estimates how many distinct keys are in each partition, so the hash table can be allocated at the right size instead of growing.## Splitting a frame into per-partition row lists

The core of the shuffle is split (crates/polars-stream/src/hash_shuffle/split.rs:43):

 rust
 
 
/// Groups the rows of `keys` (only those set in `mask`, if given) by partition: one pass for the
/// partitions and counts, a prefix sum, one pass placing the indices. Rows without a hash go to
/// partition 0. `None` with one partition and no mask: all rows, nothing computed.
pub fn split(
 keys: &HashKeys,
 mask: Option<&Bitmap>,
 partitioner: &HashPartitioner,
) -> Option<SplitRows> {
 let num_partitions = partitioner.num_partitions();
 if let Some(mask) = mask {
 assert_eq!(mask.len(), keys.len());
 }

 if num_partitions == 1 {
 let mask = mask?;
 let idxs: Vec<IdxSize> = mask.true_idx_iter().map(|i| i as IdxSize).collect();
 let offsets = vec![0, idxs.len() as IdxSize];
 return Some(SplitRows { idxs, offsets });
 }

 let mut partitions = Vec::new();
 keys.gen_partitions(partitioner, &mut partitions, true);
 // SAFETY: there is one partition per row, and the mask has one bit per row.
 let in_mask = |i: usize| mask.is_none_or(|m| unsafe { m.get_bit_unchecked(i) });

 let mut offsets: Vec<IdxSize> = vec![0; num_partitions + 1];
 for (i, p) in partitions.iter().enumerate() {
 if in_mask(i) {
 offsets[*p as usize + 1] += 1;
 }
 }
 let mut total = 0;
 for offset in &mut offsets {
 total += *offset;
 *offset = total;
 }

 let mut next = offsets[..num_partitions].to_vec();
 let mut idxs: Vec<IdxSize> = vec![0; total as usize];
 for (i, p) in partitions.iter().enumerate() {
 if in_mask(i) {
 let pos = &mut next[*p as usize];
 idxs[*pos as usize] = i as IdxSize;
 *pos += 1;
 }
 }

 Some(SplitRows { idxs, offsets })
}

This is a counting sort over the partition index, and it is worth reading closely because the data structure it produces is the entire shuffle buffer.

The output is SplitRows: one flat array of row indices, grouped by partition, with num_partitions + 1 offsets into it:

 rust
 
 
/// Row indices grouped by partition: one array, increasing within each partition (first/last
/// rely on it), and `num_partitions + 1` offsets into it.
pub struct SplitRows {
 idxs: Vec<IdxSize>,
 offsets: Vec<IdxSize>,
}

impl SplitRows {
 /// The rows of partition `p`, in increasing order.
 pub fn partition(&self, p: usize) -> &[IdxSize] {
 &self.idxs[self.offsets[p] as usize..self.offsets[p + 1] as usize]
 }
}

Two passes: the first counts how many rows land in each partition and turns the counts into a prefix sum; the second places each row index at its position. The result is that partition p’s rows are idxs[offsets[p]..offsets[p+1]] — a contiguous slice, so gathering them is a single take over the columns.

Note that the row indices within a partition are in increasing order. That is a free property of the counting sort, and it matters: it means the gathered rows of a partition keep their original relative order, so an order-sensitive aggregation still works on the shuffled data.

The memory cost of the shuffle is 4 bytes per row for the index array (with IdxSize = u32) plus a tiny offsets vector. That is dramatically cheaper than materialising a sorted copy of the data.

The per-partition buffers

The frames that get shuffled are not plain DataFrames. They carry their keys with them (crates/polars-stream/src/hash_shuffle/frame_with_keys.rs:17):

 rust
 
 
/// Rows with their hashed keys, and an optional mask of the rows that belong to it.
pub struct FrameWithKeys {
 pub frame: DataFrame,
 pub keys: HashKeys,
 pub mask: Option<Bitmap>,
}

The mask is an optimisation: if a frame is mostly rows that belong to the group-by, it can be stored whole with a bitmap marking the rows that do not, instead of being gathered row by row.

These frames are spillable. SpillFrameWithKeys wraps a SpillToken<FrameWithKeys>, so if the memory manager is under pressure the frame goes to disk and comes back later (crates/polars-stream/src/hash_shuffle/frame_with_keys.rs:88):

 rust
 
 
/// A `FrameWithKeys` that can spill, like `SpillFrame`: one `SpillToken`, its size known while
/// spilled.
pub struct SpillFrameWithKeys {
 token: SpillToken<FrameWithKeys>,
 num_rows: usize,
 estimated_size: usize,
}

Rows are moved between frames with a builder (crates/polars-stream/src/hash_shuffle/frame_with_keys.rs:172):

 rust
 
 
 /// Appends the rows `rows` of `frame` and `keys`. Panics unless the columns of `frame` are
 /// single chunks.
 ///
 /// # Safety
 /// The rows must be in-bounds of `frame` and `keys`, which have the same number of rows.
 pub unsafe fn gather_extend(&mut self, frame: &DataFrame, keys: &HashKeys, rows: &[IdxSize]) {
 unsafe {
 self.frame.gather_extend(frame, rows, ShareStrategy::Never);
 self.keys.gather_extend(keys, rows);
 }
 }

The frame and its keys are gathered together, so the keys stay row-aligned with the data. ShareStrategy::Never means string columns are copied rather than shared — the strings are moving to a new partition and must not keep the old buffer alive.

The exchange: group-by

The group-by node is the cleanest illustration of the whole mechanism, because it does the shuffle in two distinct phases.

Phase one (the sink) receives morsels and aggregates them. Rows with frequent keys go into small in-memory “hot tables”; everything else is stored cold, in spillable frames, with its keys (crates/polars-stream/src/nodes/group_by/sink.rs:152):

 rust
 
 
 /// Stores the cold rows `cold_idxs` of a slice `df` of a morsel of `input_idx`, and adds
 /// their hashes to the sketches. A whole morsel that may be stored as it is and has mostly
 /// cold rows becomes a frame with a mask of its cold rows; otherwise the cold rows are
 /// gathered into the builder of the input, which becomes a frame once it holds a morsel's
 /// worth of rows.
 async fn store_cold_rows(
 &mut self,
 input_idx: usize,
 df: DataFrame,
 keys: HashKeys,
 cold_idxs: &[IdxSize],
 may_store_whole: bool,
 spill_ctx: &MostRecentSpillContext,
 ) {

Phase two (the flush) happens once all input has arrived. This is where the partition count is decided (crates/polars-stream/src/nodes/group_by/mod.rs:233):

 rust
 
 
 /// Builds the results once all input is received. With at most one partition per pipeline
 /// all partitions are built here; otherwise the cold rows are stored per partition, and the
 /// partitions are built while the results are sent.
 fn start_flush(
 &self,
 sink: GroupBySink,
 state: &StreamingExecutionState,
 ) -> PolarsResult<Flush> {
 let (stats, frames, pre_aggs) = sink.finish_input(&self.params, &self.spill_ctx, state)?;
 let num_partitions = self.tuning.num_partitions(&stats, self.num_pipelines);
 let partitioner = HashPartitioner::new(num_partitions, 0);
 let pre_aggs = SplitPreAggs::new(pre_aggs, &partitioner, state)?;
 let num_frames = frames.len();
 let batches = make_batches(frames, self.tuning.batch_bytes());
 let reserve_groups = (stats.estimated_groups * 5 / 4 / num_partitions as u64) as usize;
 let build_at_once = num_partitions <= self.num_pipelines;

The number of partitions is a policy decision, and it is worth reading (crates/polars-stream/src/nodes/group_by/tuning.rs:101):

 rust
 
 
 /// The number of partitions: enough for every pipeline to have work, and enough that the
 /// results of any `num_pipelines` partitions fit in memory. Above `num_pipelines` it is a
 /// multiple of `num_pipelines`.
 pub fn num_partitions(&self, stats: &InputStats, num_pipelines: usize) -> usize {
 let work = stats.work();
 if work == 0 {
 return 1;
 }
 if let Some(n) = self.num_partitions {
 return n;
 }

 let t = num_pipelines.max(1) as u64;
 let n_rows = (work / self.min_rows_per_partition).clamp(1, t);
 let result_bytes = (2 * t)
 .saturating_mul(stats.estimated_groups)
 .saturating_mul(stats.group_bytes());
 let n_memory = result_bytes.div_ceil((budget() / 2).max(1));
 if n_memory <= t {
 return n_rows.max(n_memory) as usize;
 }

 let max_partitions = MAX_PARTITIONS.min(self.batch_bytes() as u64 / MIN_STORED_FRAME_BYTES);
 let n_cap = (max_partitions / t * t).max(2 * t);
 let n = n_memory.div_ceil(t).saturating_mul(t).min(n_cap);
 usize::try_from(n).unwrap()
 }

Two forces pull in opposite directions. More partitions means more parallelism but more per-partition overhead and more total hash-table memory, because each partition has its own table. Fewer partitions means cheaper tables but idle workers. The rule of thumb is: enough partitions that every pipeline has work (at least num_pipelines), but no more than the memory budget allows. Above num_pipelines, the count is rounded up to a multiple of num_pipelines so the work divides evenly across the workers.

The actual build has two shapes, decided by build_at_once (crates/polars-stream/src/nodes/group_by/mod.rs:261):

 rust
 
 
 let (outputs, stored, pre_aggs) = if build_at_once {
 let outputs =
 self.build_partitions(pre_aggs, batches, &partitioner, reserve_groups, state)?;
 actual_groups.add(outputs.iter().map(|out| out.height() as i64).sum());
 (outputs, Vec::new(), SplitPreAggs::default())
 } else {
 let stored = self.store_partitions(batches, &partitioner, state)?;
 (Vec::new(), stored, pre_aggs)
 };

If there are at most as many partitions as pipelines, every partition is built at once, in parallel, and the results are complete frames. If there are more partitions than pipelines, the partitions are stored first — one frame per partition per batch — and then built a few at a time while the results are streamed out.

The storing path is the interesting one, because it is the shuffle proper (crates/polars-stream/src/hash_shuffle/store.rs:49):

 rust
 
 
 /// Gathers this partition's rows of every frame of `batch` by index into one frame per input:
 /// reserved exactly, registered for spilling without waiting. Inputs without rows here are
 /// skipped. Each input's columns are those of its first frame in the batch. The frames must
 /// be split.
 pub fn write_batch(&mut self, batch: &[LoadedFrame]) {
 let p = self.index;
 for input in 0..self.frames_per_input.len() {
 let rows_per_frame = || {
 batch
 .iter()
 .filter(move |f| f.input == input)
 .map(move |f| (f, f.partition_rows(p).expect("frames must be split")))
 };
 let num_rows: usize = rows_per_frame().map(|(_, rows)| rows.len()).sum();
 if num_rows == 0 {
 continue;
 }

 let (first, _) = rows_per_frame().next().unwrap();
 let schema = first.frame.frame.schema().clone();
 let mut builder = FrameWithKeysBuilder::new(schema, &first.frame.keys);
 builder.reserve(num_rows);
 for (f, rows) in rows_per_frame() {
 if !rows.is_empty() {
 unsafe { builder.gather_extend(&f.frame.frame, &f.frame.keys, rows) };
 }
 }

 let frame = SpillFrameWithKeys::new_unregistered(builder.freeze_reset());
 self.spill_ctx.register_no_spill_check(&frame);
 self.estimated_size += frame.estimated_size() as u64;
 self.frames_per_input[input].push_back(frame);
 }
 }

For each partition, for each input, it gathers the partition’s rows out of every frame of the batch and concatenates them into one frame. That is the physical move: one gather per column per partition.

The batching matters for memory. make_batches (crates/polars-stream/src/hash_shuffle/batches.rs:17) sorts the frames so that the ones still in memory come first, then cuts them into batches of about batch_bytes:

 rust
 
 
/// Orders frames (those still in memory first, then by input) and cuts them into batches of
/// about `batch_bytes`, at least one frame each.
pub fn make_batches(
 frames: Vec<(usize, SpillFrameWithKeys)>,
 batch_bytes: usize,
) -> Vec<Vec<(usize, SpillFrameWithKeys)>> {

batch_bytes defaults to an eighth of the out-of-core memory budget (crates/polars-stream/src/hash_shuffle/batches.rs:18). So the shuffle proceeds in windows: load a window of cold frames, split each into partitions, write each partition’s share of the window into a partition-local frame, free the window, move to the next.

The build of one batch is overlapped with the loading of the next (crates/polars-stream/src/hash_shuffle/batches.rs:64):

 rust
 
 
/// Calls `build` on every partition once per batch, with the frames of that batch loaded. While
/// low-priority tasks build the partitions, high-priority tasks load, prepare and split the next
/// batch (each frame once) and free the previous one.
pub fn build_from_batches<S: Send>(

High-priority tasks load and split; low-priority tasks build the partitions. The split is the cheap part and gates the build, so it gets the higher priority.

The result of the shuffle is a set of per-partition frames. Each is aggregated independently into a PartitionTable, and the per-partition results are sent downstream as morsels (crates/polars-stream/src/nodes/group_by/flush.rs:171):

 rust
 
 
 TaskState::Building { table, stored } => {
 if let Some((input, frame)) = stored.pop_frame() {
 let frame = frame.load().await;
 let frame = self.params.prepare_frame(input, frame, state).await?;
 table.add_rows(input, &frame, None)?;
 } else {
 let frame = table.finish()?;
 self.actual_groups.add(frame.height() as i64);
 *task = if frame.height() > 0 {
 TaskState::Sending { frame, offset: 0 }
 } else {
 TaskState::Idle
 };
 }
 memory_manager().spill().await;
 },

The frames are consumed oldest-first, because the oldest ones are the most likely to still be in memory (they were the first to be written, so they were the first to be spilled, and the ones written last are the newest and therefore the most likely to be resident).

The exchange: join

The equi-join does the same shuffle with a different destination: the probe tables.

The build side is partitioned as it arrives (crates/polars-stream/src/nodes/joins/equi_join.rs:430):

 rust
 
 
 async fn partition_and_sink(
 mut recv: PortReceiver,
 local: &mut LocalBuilder,
 partitioner: HashPartitioner,
 params: &EquiJoinParams,
 state: &StreamingExecutionState,
 spill_ctx: &MostRecentSpillContext,
 ) -> PolarsResult<()> {

and each pipeline’s local builder keeps, per partition, the row indices of the build morsels that belong to it (crates/polars-stream/src/nodes/joins/equi_join.rs:653):

 rust
 
 
struct LocalBuilder {
 // The complete list of morsels and their computed hashes seen by this builder.
 morsels: Vec<(MorselSeq, SpillFrame, HashKeys)>,

 // A cardinality sketch per partition for the keys seen by this builder.
 sketch_per_p: Vec<CardinalitySketch>,

 // The key of each runtime filter seen by this builder.
 key_filters: Vec<KeyFilterBuilder>,

 // morsel_idxs_values_per_p[p][start..stop] contains the offsets into morsels[i]
 // for partition p, where start, stop are:
 // let start = morsel_idxs_offsets[i * num_partitions + p];
 // let stop = morsel_idxs_offsets[(i + 1) * num_partitions + p];
 morsel_idxs_values_per_p: Vec<Vec<IdxSize>>,
 morsel_idxs_offsets_per_p: Vec<usize>,
}

This is the same idea as SplitRows but stored per morsel, because the join needs to preserve the morsel boundaries: the build side’s rows must come out in the order the build input had them, because a left-outer join has to emit unmatched build rows in input order.

When the build side is complete, the partitions are turned into probe tables. Each partition is built by a separate task, and inside that task the rows are gathered from every local builder in morsel order (crates/polars-stream/src/nodes/joins/equi_join.rs:841):

 rust
 
 
 let p_morsel_idxs = &l.morsel_idxs_values_per_p[p]
 [p_morsel_idxs_start..p_morsel_idxs_stop];
 p_table.insert_keys_subset(keys, p_morsel_idxs, track_unmatchable);
 p_payload.gather_extend(&payload, p_morsel_idxs, ShareStrategy::Never);

The probe side then does the mirror image: partition the probe rows by hash, and probe each partition’s table (crates/polars-stream/src/nodes/joins/equi_join.rs:1040):

 rust
 
 
 async fn partition_and_probe(
 mut recv: PortReceiver,
 mut send: PortSender,
 partitions: &[ProbeTable],
 unordered_morsel_seq: &AtomicU64,
 partitioner: HashPartitioner,
 params: &EquiJoinParams,
 state: &StreamingExecutionState,
 ) -> PolarsResult<MorselSeq> {

with the partition assignment happening in one pass (crates/polars-stream/src/nodes/joins/equi_join.rs:1198):

 rust
 
 
 for p in partition_idxs.iter_mut() {
 p.clear();
 }
 hash_keys.gen_idxs_per_partition(
 &partitioner,
 &mut partition_idxs,
 &mut [],
 None,
 emit_unmatched,
 );

Note that this is not a second shuffle of the data. The probe rows stay where they are; only their indices are grouped by partition, and each partition’s indices are used to gather from that partition’s build-side payload. The build side did the physical move; the probe side only does the logical routing.

There is also an order-preserving mode, where the probe rows cannot be reordered at all and the probe has to follow the probe morsel’s order (crates/polars-stream/src/nodes/joins/equi_join.rs:1127):

 rust
 
 
 if params.preserve_order_probe {
 // To preserve the order we can't do bulk probes per partition and must follow
 // the order of the probe morsel. We can still group probes that are
 // consecutively on the same partition.
 probe_partitions.clear();
 hash_keys.gen_partitions(&partitioner, &mut probe_partitions, emit_unmatched);

Consecutive runs of the same partition are probed together, so an ordered join still gets most of the batching benefit, just not the full one.

Why this beats a sort

The comparison, for one input of n rows with g distinct keys:

	sort-based	hash shuffle
comparisons	`O(n log n)`	`O(n)` hashes
extra memory	whole sorted copy	4 bytes/row of indices + per-partition hash tables
first output	after the whole sort	after the first partition is built
ordering	total order available	only the original order within a partition

The “extra memory” row is the one people usually get wrong. A sort does not need to hold the sorted copy if it can spill, but it does need to hold the unsorted copy plus a sort key, and it must write the sorted output in order. The shuffle only needs an index array; the rows themselves are gathered into per-partition frames which replace the originals, so the total number of live row copies stays roughly constant.

The “first output” row is the streaming argument. A sort is a barrier: nothing comes out until everything is in. The shuffle is not — the group-by flush phase starts emitting partitions as soon as the first batch is built, and the join starts probing as soon as the build side finishes.

The trade-offs

Skew. A hash shuffle assumes the key distribution is uniform. If one key has 40% of the rows, one partition gets 40% of the work and the other workers idle. Polars mitigates this in the group-by with the hot table: frequent keys are aggregated in a small in-memory table before the shuffle, so a hot key never reaches a partition as a pile of rows (crates/polars-stream/src/nodes/group_by/sink.rs:107). The join has no such mitigation — a hot key simply makes one partition slow.

Memory of the hash tables. Each partition has its own hash table, sized from the cardinality sketches, so a partition with a million distinct keys gets a table sized for a million keys.

The extra copy. The shuffle physically moves every row into a partition-local frame. That is one gather per column per row, and for wide frames it is the dominant cost of the shuffle. It is still much cheaper than a sort, but it is not free.

The bottom line

The hash shuffle is what makes the streaming group-by and join possible without a sort barrier. The pieces are:

A fast partition function. Multiply by an odd random seed, take the high bits, avoid the division. HashPartitioner::hash_to_partition.
Hashes computed once and carried with the rows. HashKeys has a specialised representation per key type, so the hash is not recomputed on every shuffle.
A counting sort as the shuffle buffer. split produces one flat index array plus num_partitions + 1 offsets. 4 bytes per row, and the row order within a partition is preserved for free.
Spillable per-partition frames. FrameWithKeys plus SpillFrameWithKeys means the shuffle can run out of core, and the batch size is derived from the memory budget rather than being fixed.
A policy layer on top. The number of partitions is chosen from the estimated group count, the estimated bytes per group, and the memory budget — and is always a multiple of the pipeline count.

The knobs are environment variables if you want to experiment: POLARS_GROUP_BY_NUM_PARTITIONS forces the partition count, POLARS_GROUP_BY_MIN_ROWS_PER_PARTITION sets the work floor below which extra partitions are not worth it, and POLARS_GROUP_BY_BATCH_BYTES sets the size of the shuffle window.

Group-By on Opensource internals

Polars Estimates Cardinality With HyperLogLog

The register trick

The estimator, and what “HLL++” means here

The smaller sketch the streaming engine uses

Where the estimates are used

approx_n_unique

Deciding whether to dictionary-encode a Parquet column

Choosing the build side of a hash join

Publishing a bloom filter

The F2 sketch: counting heavy keys

The bottom line

Polars Encodes Group Keys as Rows to Make Hashing Cheap

Encoding a row

Strings are the awkward case

Hashing the bytes

The map that stores the groups

When no encoding is needed

Key rows: the fixed-width version

The bottom line

Polars Aggregates Faster When the Keys Are Already Sorted

The flag, not the check

The run-length scan

Making the scan parallel

What the hash path looks like

The metadata that travels with the groups

The streaming version

The bottom line

Polars Groups Columns So One Pass Feeds Many Aggregations

Two grouping engines, not one

The Grouper trait

Four encodings, one interface

Single key: the values themselves

Key rows: a hand-rolled row format

Row encoding and binview: the fallbacks

The keys themselves

One pass, many aggregations

The hot table

The bottom line

Polars Hash-Shuffles Rows Between Streaming Stages

The partition function

Hashes are computed once, and kept with the data

The per-partition buffers

The exchange: group-by

The exchange: join

Why this beats a sort

The trade-offs

The bottom line

`approx_n_unique`

The `Grouper` trait