All posts published on this site.
Everything published here, newest first.
Arrow’s RowConverter encodes each key column into a byte layout where lexicographic row order equals byte order, turning a multi-column sort key comparison into a single memcmp.
The buffer layer of Polars is deliberately dumb: an immutable (ptr, length) view over a hand-rolled refcounted storage, with all growth pushed into the mutable builders.
DataFusion’s hash join probe partitions coordinate through a single AcqRel counter decrement; a subtle ordering bug was found with loom’s model checker and cannot be reproduced by …
Tiny batches destroy vectorized throughput; DataFusion merges them with arrow-rs BatchCoalescer, wrapped in a LIMIT-aware coalescer that slices instead of copying, and fused into …
DataFusion’s multi-level merge merges the sorted runs it can afford, spills the merged result back to disk, and repeats until one pass remains, re-spilling skewed runs whose widest …