DataFusion’s Parquet writer encodes row groups in parallel across columns, streams them to object storage in one pass, and — when the input plan is already ordered — records the …
The Parquet sink does not write morsels. It resizes them into row groups of a target size, encodes and compresses those row groups in parallel across columns, and hands the …
A SQL predicate is not one thing to DataFusion: it is simultaneously a candidate for statistics pruning and a candidate for decoder-level evaluation, and each has its own support …
The filter is not a special case in the Parquet reader. Polars compiles the same expression tree a second time into a predicate over a synthetic frame of col_min / col_max / col_nc …
DataFusion never reads a Parquet file row group by row group and hopes for the best: it builds an access plan, proves which row groups cannot match using footer statistics, the …