Skip to content
Opensource internals
  • Posts

Buffer manager

The Block Allocator: One mmap Arena, Per-Thread Free Lists and Lazy Commit

Oct 25, 2026 14 min read Jay Hubbert

DuckDB’s BlockAllocator reserves one mmap arena, addresses blocks by shifting, recycles them through per-thread free lists, commits them lazily, and estimates retention with a …

DuckdbMemoryBuffer managerPerformanceInternals

Read more →

The Temporary Memory Manager Solves a Convex Optimization to Split Memory

Oct 24, 2026 15 min read Jay Hubbert

DuckDB’s TemporaryMemoryManager models throughput versus materialization cost as a convex objective, computes its gradient analytically, and uses it to split one budget between …

DuckdbMemoryBuffer managerPerformanceInternals

Read more →

The Lock-Free Eviction Queue: Weak Pointers and Sequence Numbers

Oct 24, 2026 15 min read Jay Hubbert

The eviction queue is a lock-free MPMC queue of weak_ptr plus a monotonic sequence number; dead nodes are detected and purged in bulk every 4096 inserts.

DuckdbBuffer managerMemoryCachePerformance

Read more →

Pinning Is Ownership: DuckDB's Buffer Handle Is a shared_ptr

Oct 24, 2026 13 min read Jay Hubbert

Pinning a block in DuckDB is just holding a shared_ptr: eviction is impossible under a live handle, and the readers counter is only an eviction hint.

DuckdbBuffer managerMemoryCacheInternals

Read more →

© 2026 Jay Hubbert · Built with Hugo 0.168.0

RSS ·