COUNT(*) Is a Sum of Ones: How DataFusion Makes Counting the Cheapest Aggregate
COUNT(*) becomes COUNT(1) over a non-null literal, which lets DataFusion use a branchless index-only accumulator with no null buffer and no seen-values bookkeeping.
Datafusion
How DataFusion replaces a projection with an index instead of materializing it
The physical optimizer rewrites the plan after planning: projections are absorbed into scans as column indices, row indices become reader-provided virtual columns, and filters are …
Morsel-Driven IO: How DataFusion Reads Files in Pieces So One Slow File Cannot Stall the Scan
DataFusion’s morsel-driven IO breaks files into independently-processable pieces and shares the work queue between scan tasks, so an unbalanced file distribution does not leave …
Range Partitioning in DataFusion: Replacing the Hash Shuffle with Boundaries Chosen from the Data
DataFusion’s range partitioning cuts an ordered key space into N partitions at sampled boundaries, so a shuffle stays balanced even under skew and the partitions keep their order.
How DataFusion pushes filters through joins without breaking null semantics
Filters are split into conjuncts and pushed as far as the algebra allows; the preserved-side analysis is what stops DataFusion from pushing a predicate below a LEFT JOIN and …