NOT IN in a Hash Join: Why DataFusion Decides UNKNOWN Per Build Row
DataFusion’s null-aware hash join tracks NOT IN’s three-valued logic with per-build-row bitmaps and a separate hash table of NULL-valued build rows, because the answer depends on …
Posts
All posts published on this site.
Everything published here, newest first.
The ASOF Join as a Merge Scan: How DataFusion Finds the Closest Preceding Row Without a Hash Table
DataFusion implements ASOF join as a forward-only merge over two sorted inputs, keeping exactly one candidate right row per equality group and replacing it as better matches …
Dictionary Columns Are a Hash Cache: How DataFusion Groups by Dictionary Index and What That Buys
A dictionary column is a pre-built hash cache: DataFusion hashes each distinct dictionary value once, scatters those hashes to the keys, and memoizes value-index to group-slot so …
DataFusion folds constants at plan time: ConstEvaluator, the algebra rules, and the cast-unwrapping trick
Constant folding happens in the logical optimizer via a dummy-batch evaluator, algebraic rules fire in a fixpoint loop, and casts are unwrapped into the literal when the conversion …
Embedded Projections: How DataFusion Removes a Whole Plan Node by Folding It Into the Join
DataFusion can fold a projection into the operator below it, so a hash join only builds the output columns that are actually used — removing both a plan node and a per-column take.