DataFusion’s proto layer serializes physical plans for distributed execution, and its deduplicating converter preserves the identity of shared expressions so a dynamic filter stays …
DataFusion invokes async UDFs once per Arrow batch (with an optional ideal_batch_size chunking), lifts them out of arbitrary expressions during physical planning, and rewrites …
DataFusion’s optimizer rewrites plan trees in place with Arc::make_mut, avoiding the clone-collect-rebuild cycle of the ownership-based TreeNode API, and skips the expensive …
COUNT(DISTINCT x) becomes a group-by on x plus a count of the resulting rows, count is rolled up as sum, and DISTINCT is stripped from aggregates whose merge is idempotent.
Nulls live in a side-channel validity bitmap, not in the value buffer, which lets DataFusion branch once on ‘has any nulls’ and skip whole loops, filters, and comparisons.