Skip to content
Opensource internals
  • Posts

Arrow

DataFusion's Branchless IN Filter: Why x IN (10, 20, 30) Is Faster Than Building a Hash Table

Oct 21, 2026 12 min read Jay Hubbert

DataFusion’s IN-list evaluation picks a filter by list size and type: a branchless comparison chain for short lists, a bitmap for narrow types, a hash set for wide ones.

DatafusionArrowSimdPerformanceInternals

Read more →

No Row-by-Row Interpreter: How DataFusion's Physical Expression Tree Compiles Into Whole-Batch Arrow Kernel Calls

Oct 21, 2026 15 min read Jay Hubbert

DataFusion evaluates a physical expression tree by walking it once per RecordBatch, dispatching on a Rust enum at each node, and handing whole arrays to Arrow kernels — never …

DatafusionArrowPerformanceInternals

Read more →

← Newer Page 3 of 3 Older →

© 2026 Jay Hubbert · Built with Hugo 0.168.0

RSS ·