Multi-Level Merge: How DataFusion Sorts More Data Than It Can Hold, One Pass at a Time
DataFusion’s multi-level merge merges the sorted runs it can afford, spills the merged result back to disk, and repeats until one pass remains, re-spilling skewed runs whose widest …