Polars Parses CSV With Portable SIMD and simdutf8
Polars’ CSV reader scans 64 bytes at a time with portable SIMD, uses a carry-less multiply to compute quote parity, and hands each field to a per-dtype parse_bytes; strings are …
Polars
Polars Prunes Parquet Row Groups With Chunk Statistics, Not the Page Index
Polars prunes Parquet row groups with the per-column-chunk statistics in the footer: it materialises them as a DataFrame of col_min/col_max/col_nc columns, runs the predicate on …
Polars Does Interval Joins with a Sort and a Sweep
The interval join follows Khayyat et al.’s inequality-join algorithm: two sorts produce an L1 order (by start) and an L2 order (by end), and a sweep over the L2 order marks …
Polars Joins Two Tables with a Piecewise Merge Join
The merge join kernel walks a sorted build side and a sorted probe side with two cursors, comparing keys and emitting pairs. The streaming engine cuts the two inputs into pieces …
Polars Encodes Group Keys as Rows to Make Hashing Cheap
The row encoder writes each key column into a self-describing byte layout: a sentinel for null, big-endian bytes with the sign bit flipped for numbers, a terminator for strings. …