Instead of sorting N rows to keep k, DataFusion encodes sort keys in Arrow’s row format, keeps the k best rows in a binary heap, and publishes a dynamic filter so the scan below …
Exact DISTINCT becomes a GROUP BY on every column, and the approximate version keeps a 16 KiB HyperLogLog sketch per group, keyed by the hash of the value rather than the value …
DataFusion’s approx_percentile_cont compresses values into weighted centroids whose state can be merged across partitions — approximately, not exactly.
DataFusion evaluates a physical expression tree by walking it once per RecordBatch, dispatching on a Rust enum at each node, and handing whole arrays to Arrow kernels — never …
DataFusion validates the plan after every optimizer rule, using a two-level invariant system where the cheap structural checks always run and the expensive semantic checks are …