DataFusion pushes LIMIT into operators that can absorb it, so a ten-row query stops the scan, the join and the sort after ten rows instead of after a full 8192-row batch.
DataFusion rewrites filters into min/max expressions over per-container statistics and evaluates them in one vectorized batch across thousands of files.
DataFusion never reads a Parquet file row group by row group and hopes for the best: it builds an access plan, proves which row groups cannot match using footer statistics, the …
DataFusion’s push decoder interleaves byte-range fetches with decoding and re-checks a dynamic filter at every row-group boundary, so a filter that tightens mid-scan still prunes …
DataFusion prunes joins three ways: merge cursors that only move forward over key groups, a bitmap that lets the probe side skip containers whose key range cannot match, and a …