Polars

Polars Hash-Shuffles Rows Between Streaming Stages

Oct 21, 2026 19 min read Jay Hubbert

Rows are partitioned by hash(key) into N buckets, so equal keys always land in the same bucket. Each bucket is aggregated or joined independently, which is a shuffle instead of a …

Polars Internals Streaming Group by Joins Performance

Polars' New Streaming Engine Works in Morsels

Oct 21, 2026 14 min read Jay Hubbert

The streaming engine never moves a whole DataFrame between operators. It moves a Morsel — a fixed-size slice of rows with a sequence number — over rendezvous channels, and a …

Polars Internals Streaming Concurrency Memory

Polars Kills Duplicate Subplans by Fingerprinting Them

Oct 21, 2026 14 min read Jay Hubbert

Polars does not compare subtrees with a recursive equality check. It hashes each plan node together with the canonical ids of its children, interns the result in a hash table of …

Polars Internals Optimizer Lazy Performance

Polars Simplifies the Expression Tree Before It Runs Anything

Oct 21, 2026 14 min read Jay Hubbert

Simplification is not a separate optimizer pass in Polars but a rewrite rule that runs interleaved with type coercion during conversion, using a context flag that tells the rule …

Polars Internals Optimizer Lazy

Polars Pushes Predicates Into the Parquet Scanner

Oct 21, 2026 13 min read Jay Hubbert

The filter is not a special case in the Parquet reader. Polars compiles the same expression tree a second time into a predicate over a synthetic frame of col_min / col_max / col_nc …

Polars Internals Optimizer Parquet Performance