DataFusion’s bitwise sort-merge join tracks per-outer-row matches in a bit-packed BooleanBuffer, ORing filter results 64 bits at a time and using popcnt to count, so a semi join …
DataFusion does not count allocations — it counts reservations. Operators ask a MemoryPool for bytes; the pool decides; the reservation releases on drop.
DataFusion’s JoinSelection rule swaps join inputs when the left side’s estimated size exceeds the right’s, and any non-equi join predicate becomes a JoinFilter evaluated on …
DataFusion’s statistics-based file group splitter assigns files to partitions by best-fit on their min/max ranges, preserving the table’s ordering while balancing the file counts.
DataFusion aggregates are state machines: update_batch feeds them rows, state() serializes them, merge_batch combines them across partitions.