Buffer-Manager on Opensource internals

The Block Allocator: One mmap Arena, Per-Thread Free Lists and Lazy Commit

Jay Hubbert — Sun, 25 Oct 2026 09:10:00 +0000

TL;DR: DuckDB does not malloc a 256 KiB block every time the buffer manager needs one. It reserves one big anonymous mmap arena up front, hands out fixed-size slices of it, and finds a slice’s identity with a shift — (pointer - base) >> 18 — because the block size is a power of two. Freed blocks go onto per-thread free lists, are batched back to shared queues in groups of 16, and are committed and decommitted lazily with madvise, so the arena’s address space is cheap but its resident pages are not. A rolling retention estimate packs an epoch and a count into one uint64 so the allocator can estimate how much it is really holding without walking a list.

The problem with `malloc(256 * 1024)`

Every time DuckDB’s buffer manager needs a block, it needs 256 KiB — that is DEFAULT_BLOCK_ALLOC_SIZE, and it is not negotiable, because the block size is baked into the on-disk format.

The obvious implementation is malloc(256 * 1024). It works, and it is a bad idea for a database that allocates and frees blocks in bursts from dozens of threads:

glibc will serve it from a per-thread arena, so it is not that bad, but each allocation still goes through the allocator’s bookkeeping and, for large chunks, mmap/munmap on the kernel.
munmap returns the pages to the kernel. The next allocation gets fresh zero pages, which means page faults and a cold cache on every reuse.
There is no way to ask “how much is this allocator holding right now?” without walking the allocator’s own data structures.

DuckDB’s answer is to own the address space itself.

One arena, reserved but not committed

The arena is one mmap of anonymous private memory:

 cpp
 
 
// src/storage/block_allocator.cpp:139
static data_ptr_t AllocateVirtualMemory(const idx_t size) {
#if INTPTR_MAX == INT32_MAX
	// Disable on 32-bit
	return nullptr;
#endif

#if defined(_WIN32)
	// This returns nullptr on failure
	return data_ptr_t(VirtualAlloc(nullptr, size, MEM_RESERVE, PAGE_NOACCESS));
#else
	auto flags = MAP_PRIVATE | MAP_ANONYMOUS;
#if defined(__linux__)
	// Reserve address space without charging the entire pool against the commit limit.
	flags |= MAP_NORESERVE;
#endif
	const auto ptr = mmap(nullptr, size, PROT_READ | PROT_WRITE, flags, -1, 0);
	return ptr == MAP_FAILED ? nullptr : data_ptr_cast(ptr);
#endif
}

MAP_NORESERVE is the interesting flag. Without it, the kernel counts the whole mapping against the process’s commit limit — and on Linux, overcommit_memory=2 (the strict setting) will refuse the mapping outright if the reserved size exceeds the limit. With it, the reservation is free: the kernel promises you the address range but does not promise the pages.

That matters because DuckDB’s pool can be much larger than the memory limit the user configured. The memory limit governs how much of the arena is supposed to be resident; the arena itself is just address space.

On Windows the equivalent is VirtualAlloc(..., MEM_RESERVE, PAGE_NOACCESS) — reserve the range, forbid access, and let the pages be committed later.

The pool is initialised lazily and published with a release store, so a failed reservation permanently disables the pool rather than being retried on every allocation:

 cpp
 
 
// src/storage/block_allocator.cpp:576
	const auto pointer = AllocateVirtualMemory(pool_size);
	if (!pointer) {
		pool_state.store(PoolState::UNAVAILABLE, std::memory_order_release);
		return false;
	}
	untouched = std::move(queue);
	pool_base = pointer;
	pool_state.store(PoolState::READY, std::memory_order_release);
	return true;
}

The free list is seeded with every block id in the pool, enqueued in batches of STANDARD_VECTOR_SIZE (2048) so the initial fill is a handful of bulk operations rather than thousands of pushes.

Finding a block with a shift

The arena is carved into fixed-size blocks. The block size is a power of two, and the allocator computes the shift once, in its constructor:

 cpp
 
 
// src/storage/block_allocator.cpp:493
BlockAllocator::BlockAllocator(Allocator &allocator_p, const idx_t block_size_p, const idx_t pool_size_p)
 : uuid(UUID::GenerateRandomUUID()), allocator(allocator_p), allocation_cache(make_uniq<AllocationCache>(allocator)),
 block_size(block_size_p), block_size_div_shift(CountZeros<idx_t>::Trailing(block_size)),
 pool_size(sizeof(void *) == 8 ? pool_size_p - pool_size_p % block_size : 0),
 pool_state(pool_size != 0 && DivBlockSize(pool_size) <= NumericLimits<uint32_t>::Maximum()
 ? PoolState::UNINITIALIZED
 : PoolState::UNAVAILABLE),
 touched(make_unsafe_uniq<BlockQueue>()), lifetime_state(make_shared_ptr<BlockAllocatorLifetimeState>()) {
	static_assert(std::tuple_size<decltype(retention_buckets)>::value == BlockAllocatorConfig::RETENTION_BUCKETS,
	 "Retention bucket storage must match the decay configuration");
	D_ASSERT(IsPowerOfTwo(block_size));
	InitializeRetention(RetentionTimeMillis());
}

CountZeros<idx_t>::Trailing(block_size) is the number of trailing zero bits of the block size — for 256 KiB that is 18. The pool size is rounded down to a whole number of blocks, and on 32-bit systems the pool is disabled entirely (pool_size becomes 0) because there is not enough address space to make it worthwhile.

With the shift in hand, the two conversions are a subtract and a shift:

 cpp
 
 
// src/storage/block_allocator.cpp:600
idx_t BlockAllocator::ModuloBlockSize(const idx_t n) const {
	return n & (block_size - 1);
}

idx_t BlockAllocator::DivBlockSize(const idx_t n) const {
	return n >> block_size_div_shift;
}

uint32_t BlockAllocator::GetBlockID(const data_ptr_t pointer) const {
	D_ASSERT(IsInPool(pointer));
	const auto offset = NumericCast<idx_t>(pointer - pool_base);
	D_ASSERT(ModuloBlockSize(offset) == 0);
	const auto block_id = NumericCast<uint32_t>(DivBlockSize(offset));
	VerifyBlockID(block_id);
	return block_id;
}

void BlockAllocator::VerifyBlockID(const uint32_t block_id) const {
	D_ASSERT(block_id < DivBlockSize(pool_size));
}

data_ptr_t BlockAllocator::GetPointer(const uint32_t block_id) const {
	VerifyBlockID(block_id);
	return pool_base + NumericCast<idx_t>(block_id) * block_size;
}

pointer → id is (pointer - pool_base) >> 18. id → pointer is pool_base + id * block_size. No lookup table, no hash, no map. The block id is its offset divided by the block size, and the division is a shift because the size is a power of two.

This is what makes the design work: the free lists store 32-bit integers, not pointers. A batch of 16 freed blocks is 64 bytes of bookkeeping that the CPU can push around in registers, and the address is recomputed on demand.

ModuloBlockSize is the mask for the opposite direction — it is the assertion that the pointer really is block-aligned.

Per-thread free lists

The allocator has two levels of free lists. At the top, two shared ConcurrentQueues of block ids — touched (blocks that have been used and returned) and untouched (blocks that were never touched). At the bottom, each thread keeps its own two vector<uint32_t> caches.

 cpp
 
 
// src/storage/block_allocator.cpp:226
	data_ptr_t Allocate() {
		auto pointer = TryAllocateFromLocal();
		if (pointer) {
			return pointer;
		}

		// We have run out of local blocks
		if (TryGetBatch(touched, *block_allocator->touched) || TryGetBatch(untouched, *block_allocator->untouched)) {
			// We have refilled local blocks
			pointer = TryAllocateFromLocal();
			D_ASSERT(pointer);
			return pointer;
		}

		// We have also run out of global blocks, use fallback allocator
		return block_allocator->allocator.AllocateData(block_allocator->block_size);
	}

The fast path is TryAllocateFromLocal, and it is a pop_back on a vector:

 cpp
 
 
// src/storage/block_allocator.cpp:292
	data_ptr_t TryAllocateFromLocal() {
		if (!touched.empty()) {
			const auto pointer = block_allocator->GetPointer(touched.back());
			touched.pop_back();
			block_allocator->UpdateCachedBlocks(-1, cache_slot);
			return pointer;
		}
		if (!untouched.empty()) {
			const auto pointer = block_allocator->GetPointer(untouched.back());
			OnFirstAllocation(pointer, block_allocator->block_size);
			untouched.pop_back();
			return pointer;
		}
		return nullptr;
	}

No atomics, no locks, no CAS. A thread that frees a block pushes the id onto its own vector; a thread that allocates pops from its own vector. The shared queues are only touched when a thread’s local cache runs dry, and then it takes a whole batch at once:

 cpp
 
 
// src/storage/block_allocator.cpp:308
	bool TryGetBatch(vector<uint32_t> &local, BlockQueue &global) {
		D_ASSERT(local.empty());
		local.resize(BlockAllocatorConfig::BATCH_SIZE);
		const auto size = global.q.try_dequeue_bulk(local.begin(), BlockAllocatorConfig::BATCH_SIZE);
		local.resize(size);
		if (size > 0 && RefersToSameObject(global, *block_allocator->touched)) {
			const auto now = RetentionTimeMillis();
			block_allocator->ReuseRetention(size, now);
			if (now >= next_decay_check) {
				next_decay_check = now + BlockAllocatorConfig::ACTIVE_DECAY_INTERVAL_MS;
				try {
					block_allocator->FreeInternal(optional_idx(), BlockAllocatorConfig::MAX_FLUSH_BLOCKS,
					 BlockAllocator::ReclaimMode::OPPORTUNISTIC);
				} catch (...) {
					// Failed reclamation leaves blocks available for reuse.
				}
			}
		}
		std::sort(local.begin(), local.end());
		return !local.empty();
	}

try_dequeue_bulk moves up to BATCH_SIZE (16) ids in one go, and the ids are then sorted — sorting matters later, because the reclamation path coalesces contiguous block ids into a single madvise call.

Freeing is the mirror image, with a batching threshold:

 cpp
 
 
// src/storage/block_allocator.cpp:244
	void Free(const data_ptr_t pointer) {
		touched.push_back(block_allocator->GetBlockID(pointer));
		block_allocator->UpdateCachedBlocks(1, cache_slot);
		if (touched.size() < BlockAllocatorConfig::FREE_THRESHOLD) {
			return;
		}

		// Upon reaching the threshold, we return a local batch to global
		std::sort(touched.begin(), touched.end());
		ReturnTouched(BlockAllocatorConfig::BATCH_SIZE);
	}

FREE_THRESHOLD is BATCH_SIZE * 2 = 32. A thread frees blocks; when it has 32 sitting locally, it sorts them and hands 16 back to the shared queue. The sort is not cosmetic — sorted ids make the contiguous-run coalescing in the reclamation path possible.

The two queues are distinguished by whether the block has ever been used. untouched blocks are still unmapped from the process’s point of view (never committed); touched blocks have resident pages. The distinction lets the allocator decommit a run of touched blocks with one syscall, and lets it skip the decommit entirely for blocks that were never committed.

Commit lazily, decommit with `madvise`

The arena’s pages are committed on first use and decommitted on release. On Linux that is madvise(MADV_DONTNEED); on macOS it is MADV_FREE_REUSABLE; on Windows it is VirtualFree(MEM_DECOMMIT).

 cpp
 
 
// src/storage/block_allocator.cpp:171
static void OnFirstAllocation(const data_ptr_t pointer, const idx_t size) {
	bool success = true;
#if defined(_WIN32)
	success = VirtualAlloc(pointer, size, MEM_COMMIT, PAGE_READWRITE);
#elif defined(__APPLE__)
	// Reclaimed pages remain reusable until explicitly claimed again.
	success = madvise(pointer, size, MADV_FREE_REUSE) == 0;
#endif
	if (!success) {
		throw InternalException("OnFirstAllocation failed");
	}
}

static bool OnDeallocation(const data_ptr_t pointer, const idx_t size) {
#if defined(_WIN32)
	return VirtualFree(pointer, size, MEM_DECOMMIT);
#elif defined(__APPLE__)
	return madvise(pointer, size, MADV_FREE_REUSABLE) == 0;
#elif defined(__MVS__)
	// the madvice functionality is not available on z/OS in any form
	return false;
#else
	return madvise(pointer, size, MADV_DONTNEED) == 0;
#endif
}

Note that on Linux OnFirstAllocation is a no-op: mmap with MAP_PRIVATE | MAP_ANONYMOUS already gives readable, writable pages, and the kernel commits them on first touch. The interesting half is OnDeallocation, which gives the pages back without giving up the address space. MADV_DONTNEED on an anonymous private mapping zeroes the pages and drops them; the next touch is a page fault and a fresh zero page. That is precisely the behaviour you want for a block cache: a cold block should read as zeros, not as the previous tenant’s data.

The decommit is done in runs. Because the ids are sorted, contiguous runs can be collapsed into one call:

 cpp
 
 
// src/storage/block_allocator.cpp:1086
bool BlockAllocator::FreeContiguousBlocks(const uint32_t block_id_start, const uint32_t block_id_end_including) const {
	const auto pointer = GetPointer(block_id_start);
	const auto num_blocks = block_id_end_including - block_id_start + 1;
	const auto size = num_blocks * block_size;
	return OnDeallocation(pointer, size);
}

One madvise for a run of 64 adjacent blocks instead of 64 syscalls. The whole point of sorting the ids on the way out of the thread-local cache is to make this possible.

There is a subtlety on the way back in. A block that was decommitted with MADV_DONTNEED and is then re-allocated gets fresh zero pages — but a block that was never touched still has its original (zero) pages. Both cases end up zeroed, so the allocator does not need to track “was this decommitted?” per block. The touched/untouched split exists for a different reason: it tells the reclamation path which blocks actually have resident pages worth releasing.

The packed retention estimate

Here is a small trick that I like a lot.

The allocator wants to know how much memory it is really holding, so it can decide when to start giving pages back to the kernel. Walking every free list to count would be O(pool size) and would need a lock.

Instead, DuckDB keeps a rolling, bucketed count of recently returned blocks. Time is divided into 500 ms buckets; each bucket holds how many blocks were returned during it. A bucket is packed into a single uint64 — 32 bits of epoch, 32 bits of count:

 cpp
 
 
// src/storage/block_allocator.cpp:58
static uint64_t PackRetentionBucket(const uint32_t epoch, const uint32_t count) {
	return (uint64_t(epoch) << 32) | count;
}

static uint32_t RetentionBucketEpoch(const uint64_t bucket) {
	return static_cast<uint32_t>(bucket >> 32);
}

static uint32_t RetentionBucketCount(const uint64_t bucket) {
	return static_cast<uint32_t>(bucket);
}

One atomic uint64 per bucket, updated with a compare-exchange. The epoch makes the bucket self-describing: you can tell when a count was recorded and how stale it is, without a separate array of timestamps.

There are three buckets, because the decay window is one second and the bucket width is 500 ms:

 cpp
 
 
// src/storage/block_allocator.cpp:119
idx_t BlockAllocator::RetentionTarget(const idx_t now_ms) const {
	const auto epoch = now_ms / BlockAllocatorConfig::RETENTION_INTERVAL_MS;
	idx_t weighted = 0;
	const auto offset = now_ms % BlockAllocatorConfig::RETENTION_INTERVAL_MS;
	for (auto &bucket : retention_buckets) {
		const auto current = bucket.load(std::memory_order_relaxed);
		const auto age = static_cast<uint32_t>(epoch - RetentionBucketEpoch(current));
		if (age >= retention_buckets.size()) {
			continue;
		}
		const auto age_ms = age == 0 ? 0 : (age - 1) * BlockAllocatorConfig::RETENTION_INTERVAL_MS + offset;
		const auto remaining_ms = BlockAllocatorConfig::RETENTION_DECAY_MS - age_ms;
		weighted += RetentionBucketCount(current) * remaining_ms;
	}
	return weighted / BlockAllocatorConfig::RETENTION_DECAY_MS;
}

The estimate is a linear decay, not a step function. Each bucket’s count is weighted by how much of the decay window it still has left, and the weighted sum is divided by the window length. A block returned one millisecond ago counts almost fully; one returned 900 ms ago counts about a tenth. That is the “bundled jemalloc” decay behaviour the config comment mentions.

Because the epoch is packed in, the bucket’s age is known without a timestamp array, and because the count saturates at the pool size, the arithmetic cannot overflow:

 cpp
 
 
// src/storage/block_allocator.cpp:79
void BlockAllocator::AddRetention(const idx_t count, const idx_t now_ms) const {
	const auto epoch = now_ms / BlockAllocatorConfig::RETENTION_INTERVAL_MS;
	const auto capacity = DivBlockSize(pool_size);
	auto &bucket = retention_buckets[epoch % retention_buckets.size()];
	auto current = bucket.load(std::memory_order_relaxed);
	do {
		const auto elapsed = static_cast<uint32_t>(epoch - RetentionBucketEpoch(current));
		// Modular ordering assumes generations are less than half the epoch range apart.
		if (elapsed >= (uint32_t(1) << 31)) {
			return;
		}
		const idx_t retained = elapsed == 0 ? RetentionBucketCount(current) : 0;
		const auto updated = PackRetentionBucket(
		 static_cast<uint32_t>(epoch),
		 NumericCast<uint32_t>(retained + MinValue(count, capacity - MinValue(capacity, retained))));
		if (updated == current || bucket.compare_exchange_weak(current, updated, std::memory_order_relaxed)) {
			return;
		}
	} while (true);
}

The elapsed >= 1 << 31 check is a wraparound guard: the epochs are compared modulo 2^32, so a generation that is more than half the epoch range old is ambiguous and the update is simply dropped. capacity is the number of blocks in the pool, so the count can never exceed it.

The result is an O(1) estimate — three atomic loads and a bit of arithmetic — of how many blocks the allocator is holding onto, with sub-second time resolution. No list walking, no locks.

The other size classes

Everything above is about the 256 KiB pool blocks. DuckDB also allocates plenty of other sizes, and those go through a separate, sharded cache of power-of-two classes:

 cpp
 
 
// src/storage/block_allocator.cpp:358
class AllocationCache {
	using Queue = duckdb_moodycamel::ConcurrentQueue<data_ptr_t>;
	static constexpr idx_t MIN_SHIFT = 19;
	static constexpr idx_t MAX_SHIFT = 26;
	static constexpr idx_t MAX_SHARDS = 256;
	static constexpr idx_t CPUS_PER_SHARD = 4;
	static constexpr idx_t MAX_LOCAL_SPARES = 16;
	// An admission ceiling, independent of pressure-driven reclamation.
	static constexpr idx_t MAX_CACHED_BYTES_PER_BUCKET = 1024ULL * 1024 * 1024;

MIN_SHIFT = 19 is 512 KiB, MAX_SHIFT = 26 is 64 MiB. The bucket index is again just a shift — CountZeros::Trailing(size) - MIN_SHIFT — so a size maps to its class with the same trick the pool uses for block ids. Sizes outside the range, or sizes that are not powers of two, are not cached at all and go straight to the fallback allocator.

The cache is sharded by CPU (CPUS_PER_SHARD = 4, up to 256 shards), so a thread almost always finds a free buffer in its own shard’s queue. When it does not, it steals from a neighbouring shard — but only after checking that the donor is above its spare allowance, so the cache never lets one hot thread drain a cold shard entirely.

What it costs

One mmap of the whole pool. The address space is reserved up front. On a 64-bit machine that is free; on 32-bit the pool is disabled.
One page fault per newly touched block. A block that was decommitted and comes back faults. That is the price of giving the pages back, and it is paid only for blocks that were idle long enough to be reclaimed.
A madvise syscall per contiguous run on reclamation. Not per block.
An atomic uint64 CAS per batch of frees to maintain the retention estimate, amortised over 16 blocks.
A per-thread cache that can hold memory the pool is not using. This is the interesting one. The per-thread lists mean a block freed on thread A may sit on thread A’s list while thread B allocates from the global queue. The retention buckets exist precisely to bound this: they estimate how much is being held and drive the reclamation path back toward the pool’s target.

And the win: no malloc, no munmap, no glibc arena, no allocator lock on the hot path. The buffer manager’s block allocation is a pop_back from a thread-local vector.

The bottom line

The ideas here are old-fashioned in the best way: pick a layout that makes the arithmetic trivial, and pay the costs in the places nobody looks at often.

Own the address space. One mmap(MAP_NORESERVE) gives you a pool without paying commit limits for it, and gives you a place to put the pages when you actually want them.
Make identity arithmetic. A power-of-two block size plus a shift turns pointer → id into a subtract and a shift, and turns the free lists into vectors of 32-bit integers.
Batch everything that touches shared state. Free lists are per-thread; the shared queues move 16 ids at a time; reclamation coalesces contiguous runs into a single madvise.
Estimate, do not measure, when measuring is expensive. The packed epoch-and-count bucket gives an O(1) answer to “how much am I holding?” that is good enough to make a decision, and costs one atomic CAS per batch.

None of this makes the buffer manager faster in the way a new compression codec would. It makes it boring, which is the point: an allocator that never calls into the C library’s allocator is an allocator whose behaviour you can reason about.

The Temporary Memory Manager Solves a Convex Optimization to Split Memory

Jay Hubbert — Sat, 24 Oct 2026 13:40:00 +0000

TL;DR: When several operators want memory at the same time, DuckDB does not divide the budget by the number of operators. Each operator registers a TemporaryMemoryState with a remaining size and a materialization penalty, and the manager minimises a convex cost function over the whole set. The gradient of that function has a closed form, so the manager can evaluate it in O(n) and then hand out the free memory to whichever operator has the most negative gradient — five rounds per operator. The result is deterministic, needs no search, and gives memory to whoever is closest to finishing.

The query that breaks an even split

You write a join between two large tables. The hash join’s build side does not fit in memory, so it spills. At the same moment the scan feeding the probe side is buffering vectors, and the aggregate on top of it is holding a hash table. Three operators, one memory limit.

The naive answer is to give each a third. That is wrong in two directions at once. The aggregate might need 1 MB and be done in a millisecond; the hash join might need 40 GB and run for an hour. An even split starves the join into spilling when it did not have to, while handing the aggregate 30 GB it will never touch.

What you actually want is a rule that answers: given the operators that are running right now, and the bytes each of them still needs, how should the budget be divided so that the system finishes fastest?

DuckDB answers that with a small optimisation problem, solved on every reservation change.

The state: what an operator tells the manager

An operator that might need to spill registers a TemporaryMemoryState. Three numbers matter:

remaining size (s_i): how many bytes this operator would need if it could hold everything in memory.
reservation (r_i): how many bytes it has actually been granted.
materialization penalty (p_i): how expensive it is to spill this operator’s data, in arbitrary units.

 cpp
 
 
// src/include/duckdb/storage/temporary_memory_manager.hpp:59
	//! The remaining size needed if it could fit fully in memory
	atomic<idx_t> remaining_size;
	//! The minimum reservation for this state
	atomic<idx_t> minimum_reservation;
	//! How much memory this operator has reserved
	atomic<idx_t> reservation;
	//! The weight used for determining the reservation for this state
	atomic<idx_t> materialization_penalty;

The penalty is not an abstraction. The hash join sets it to the tuple width of its probe side — a wide row is expensive to spill to disk, a narrow row is cheap:

 cpp
 
 
// src/execution/operator/join/physical_hash_join.cpp:1066
	gstate.temporary_memory_state->SetMinimumReservation(max_partition_ht_size + gstate.probe_side_requirement);

	bool all_constant;
	gstate.temporary_memory_state->SetMaterializationPenalty(GetTupleWidth(gstate.probe_types, all_constant));
	gstate.temporary_memory_state->SetRemainingSize(gstate.total_size);

The manager keeps every live state in a set, and every state is registered with a guaranteed floor so that no operator can be starved to zero:

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:105

unique_ptr<TemporaryMemoryState> TemporaryMemoryManager::Register(ClientContext &context) {
	const annotated_lock_guard<annotated_mutex> guard(lock);
	UpdateConfiguration(context);

	auto result = unique_ptr<TemporaryMemoryState>(new TemporaryMemoryState(*this, DefaultMinimumReservation()));
	SetRemainingSize(*result, result->GetMinimumReservation());
	SetReservation(*result, result->GetMinimumReservation());
	active_states.insert(*result);

	Verify();
	return result;
}

The objective function

Write t_i = r_i / s_i for the throughput of state i: the fraction of the work it can do without materialising anything. t_i is in (0, 1] by construction — the reservation never exceeds the remaining size.

Two quantities summarise the whole system.

Materialization cost. Every operator that only gets t_i of its memory must materialise the other 1 - t_i. Weight that by how expensive that operator’s materialisation is:

text

M(r) = Σ_i p_i · (1 - t_i) = Σ_i p_i · (1 - r_i / s_i)

Throughput. One operator crawling at 10 % does not get rescued by another one running at 100 %. The system is only as fast as its geometric mean, so the shortfall is measured against the geometric mean of the throughputs:

text

T(r) = ( Π_i t_i )^(1/n)

The cost the manager minimises is the product of the two:

text

C(r) = M(r) · (1 - T(r))

Read it as: the amount of spilling we will have to do, times how far the system as a whole is from running at full speed. If everything fits, T = 1 and the cost is zero. If one operator is starved, T drops and the cost rises sharply — and it rises for everyone, because the geometric mean is the one place where a single bad operator contaminates the whole.

Why this objective is convex

This matters, because it is what makes a gradient useful.

With s_i and p_i fixed, M(r) is an affine function of r, and it is non-negative on the feasible box 0 < r_i ≤ s_i because t_i ≤ 1.

T(r) is the geometric mean of the positive affine functions r_i / s_i. The geometric mean is concave on the non-negative orthant, and it is non-decreasing in each argument, so the composition is concave. Therefore 1 - T(r) is convex, and non-negative (since T ≤ 1).

The product of two non-negative convex functions is convex. So C(r) is convex on the feasible region.

That is the licence to use first-order information: at any point, the gradient tells you the direction in which the cost falls fastest, and there is exactly one local minimum to find.

The closed-form gradient

Here is the part that is genuinely nice. The gradient of C with respect to each reservation has a formula you can write on a napkin — no numerical differentiation, no finite differences, no Newton iterations.

M depends on r_i linearly: ∂M/∂r_i = -p_i / s_i.

T is an exponential of a mean of logarithms, so ∂T/∂r_i = T / (n · r_i).

Chain-ruling on C = M · (1 - T):

text

∂C/∂r_i = (∂M/∂r_i)·(1 - T) - M·(∂T/∂r_i)
 = -p_i·(1 - T)/s_i - T·M/(n·r_i)

Both terms are non-positive. That makes sense: giving an operator more memory never increases the cost. What differs between operators is the magnitude, and that magnitude is the marginal benefit of one more byte.

And here is the code, computing exactly that:

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:185
static void ComputeDerivatives(const vector<reference<const TemporaryMemoryState>> &states, const vector<idx_t> &res,
 vector<double> &der, const idx_t n) {
	// Cost function takes "throughput" (reservation / size) of each operator as its principal input
	double log_throughput_prod = 0;
	double mat_cost = 0;
	for (idx_t i = 0; i < n; i++) {
		auto &state = states[i].get();
		const auto resd = static_cast<double>(res[i]);
		const auto sizd = static_cast<double>(MaxValue<idx_t>(state.GetRemainingSize(), 1));
		const auto pend = static_cast<double>(state.GetMaterializationPenalty());
		D_ASSERT(resd > 0);
		D_ASSERT(resd <= sizd);
		const auto throughput = resd / sizd;
		log_throughput_prod += std::log(throughput);
		mat_cost += pend * (1 - throughput); // Materialization cost: sum of (1 - throughput)
	}
	const double nd = static_cast<double>(n); // n as double for convenience
	const double throughput_geomean = std::exp(log_throughput_prod / nd);
	const double tp_mult = 1 - throughput_geomean; // Throughput multiplier: 1 - geomean throughputs

	// Cost function: materialization cost * (1 - throughput multiplier), but we don't actually need to compute it
	// here. We need to compute the derivative with respect to every reservation, stored in "der"
	// Just use https://www.derivative-calculator.net with this (n = 3) to see what's going on
	// (3 - (a_1/s_1)-(a_2/s_2)-(a_3/s_3))*(1-((a_1/s_1)*(a_2/s_2)*(a_3/s_3))^(1/3))
	const double intermediate = -(throughput_geomean * mat_cost) / nd;
	for (idx_t i = 0; i < n; i++) {
		auto &state = states[i].get();
		const auto resd = static_cast<double>(res[i]);
		const auto sizd = static_cast<double>(MaxValue<idx_t>(state.GetRemainingSize(), 1));
		const auto pend = static_cast<double>(state.GetMaterializationPenalty());
		der[i] = intermediate / resd - pend * tp_mult / sizd;
	}
}

Read the last line against the formula. intermediate / resd is -T·M/(n·r_i). pend * tp_mult / sizd is p_i·(1 - T)/s_i, subtracted, so the whole derivative is -T·M/(n·r_i) - p_i·(1 - T)/s_i — exactly ∂C/∂r_i.

Two details worth noticing. The product of throughputs is never formed; the code accumulates log(t_i) and exponentiates once at the end, so a thousand operators each running at a thousandth of their requirement cannot underflow the intermediate product to zero before the 1/n-th root is taken. And the loop is O(n) with one log per operator — the derivative of a product of a hundred terms is not a hundred-term product of derivatives.

From gradient to split

ComputeReservation is the function that actually splits the money. It collects the live states, computes an initial reservation for each, and then runs a greedy marginal-allocation loop.

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:219
idx_t TemporaryMemoryManager::ComputeReservation(const TemporaryMemoryState &temporary_memory_state) const {
	static constexpr idx_t OPTIMIZATION_ITERATIONS_MULTIPLIER = 5;

	// Use vectors for ease
	optional_idx state_index;
	vector<reference<const TemporaryMemoryState>> states;
	vector<idx_t> res;
	vector<double> der;
	idx_t sum_of_initial_res = 0;

	idx_t i = 0;
	for (auto &state : active_states) {
		const auto initial_reservation = ComputeInitialReservation(state);
		sum_of_initial_res += initial_reservation;
		if (RefersToSameObject(state.get(), temporary_memory_state)) {
			state_index = i;
		} else if (initial_reservation >= state.get().GetRemainingSize()) {
			continue;
		}
		states.emplace_back(state);
		res.push_back(initial_reservation);
		der.push_back(NumericLimits<double>::Maximum());
		i++;
	}
	const idx_t n = i;

	if (sum_of_initial_res >= memory_limit) {
		return res[state_index.GetIndex()];
	}
	const auto free_memory = memory_limit - sum_of_initial_res;

States that are already fully satisfied (initial_reservation >= remaining size) are dropped from the optimisation entirely — there is nothing to decide about them. If the sum of the floors already exceeds the limit, the answer is simply the floor and the optimiser is skipped.

Then the loop itself:

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:250
	// Distribute memory in OPTIMIZATION_ITERATIONS
	idx_t remaining_memory = free_memory;
	const idx_t optimization_iterations = OPTIMIZATION_ITERATIONS_MULTIPLIER * n;
	for (idx_t opt_idx = 0; opt_idx < optimization_iterations && remaining_memory > 0; opt_idx++) {
		ComputeDerivatives(states, res, der, n);

		// Find the index of the state with the lowest derivative
		idx_t min_idx = 0;
		double min_der = NumericLimits<double>::Maximum();
		bool found_candidate = false;
		for (i = 0; i < n; i++) {
			auto &state = states[i].get();
			if (res[i] >= state.GetRemainingSize()) {
				continue; // We can't increase the reservation of "maxed" states, so we skip these
			}
			if (!found_candidate || der[i] < min_der) {
				min_idx = i;
				min_der = der[i];
				found_candidate = true;
			}
		}
		if (!found_candidate) {
			break;
		}

The scheme is plain marginal benefit allocation. Each round:

Evaluate the gradient at the current reservation vector.
Find the state with the lowest (most negative) derivative — the state for which one more byte buys the most.
Give that state a chunk of the remaining memory.
Repeat, 5 × n times.

Because the gradient is negative everywhere, “lowest” is unambiguous: it is the state with the largest |∂C/∂r_i|. And because C is convex, the marginal benefit of a byte only decreases as you give that operator more — so the greedy choice cannot be improved by revisiting an earlier decision in a way the gradient does not already capture.

The chunk size is chosen so the free memory is exhausted in a fixed number of rounds rather than all at once:

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:274
		auto &min_state = states[min_idx].get();

		// This is how much memory we will distribute in this round
		const auto iter_memory = ExactNumericCast<idx_t>(
		 std::ceil(static_cast<double>(remaining_memory) / static_cast<double>(optimization_iterations - opt_idx)));

		// Compute how much we can add
		const auto state_room = min_state.GetRemainingSize() - res[min_idx];
		const auto delta = MinValue(iter_memory, state_room);
		D_ASSERT(delta <= remaining_memory);

		// Update counts
		res[min_idx] += delta;
		remaining_memory -= delta;

Note state_room: an operator is never given more than it asked for. A reservation above the remaining size would make t_i > 1, which would break both the model and the D_ASSERT(resd <= sizd) in ComputeDerivatives.

There is one more wrinkle, and it is a nice one. ComputeReservation is called once per state, in whatever order the operators happen to ask. So a naive implementation would be order-dependent: the first state to ask would soak up the free memory. DuckDB handles this with a second phase: it re-evaluates the derivative with every state at its maximum possible reservation, sorts the states by that derivative, and only then applies the per-state caps in that order.

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:296
	// We computed how the memory should be assigned to the states,
	// but we did not yet take into account the upper bound of MAXIMUM_FREE_MEMORY_RATIO * free_memory.
	// We don't simply bound this state's reservation because order matters,
	// and this function is called in arbitrary order for all active states.
	// So, we order the states by derivative (as if they got the max reservation),
	// and assign the states with the lowest derivate first.
	// This way, order does not matter _as much_, and the "best" states will _likely_ get more memory
	vector<idx_t> idxs(n, 0);
	vector<idx_t> max_res(n, 0);
	for (i = 0; i < n; i++) {
		idxs[i] = i;
		max_res[i] = states[i].get().GetRemainingSize() - 1;
	}
	ComputeDerivatives(states, max_res, der, n);
	std::sort(idxs.begin(), idxs.end(), [&](const idx_t &lhs, const idx_t &rhs) { return der[lhs] < der[rhs]; });

The comment is candid about it: order does not matter as much. It is a heuristic that makes the answer approximately order-independent rather than exactly so.

The fairness floors

An optimisation that maximises throughput can starve an operator. Two constants stop that.

 cpp
 
 
// src/include/duckdb/storage/temporary_memory_manager.hpp:83
	//! 512 blocks per state per thread, which is 0.125GB per thread for DEFAULT_BLOCK_ALLOC_SIZE.
	static constexpr idx_t MINIMUM_RESERVATION_PER_STATE_PER_THREAD = 512ULL * DEFAULT_BLOCK_ALLOC_SIZE;
	//! 1/16th of the available main memory.
	static constexpr idx_t MINIMUM_RESERVATION_MEMORY_LIMIT_DIVISOR = 16ULL;

	//! The maximum ratio of the memory limit that we reserve using the TemporaryMemoryManager
	static constexpr double MAXIMUM_MEMORY_LIMIT_RATIO = 0.9;
	//! The maximum ratio of the remaining memory that we reserve per TemporaryMemoryState
	static constexpr double MAXIMUM_FREE_MEMORY_RATIO = 0.9;

DEFAULT_BLOCK_ALLOC_SIZE is 262144, so 512 blocks is 128 MiB per thread. The floor for a state is the smaller of “128 MiB × number of threads” and “one sixteenth of the whole memory limit”:

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:74
idx_t TemporaryMemoryManager::DefaultMinimumReservation() const {
	return MinValue(num_threads * MINIMUM_RESERVATION_PER_STATE_PER_THREAD,
	 memory_limit / MINIMUM_RESERVATION_MEMORY_LIMIT_DIVISOR);
}

On an 8-core machine with a 16 GiB limit both terms are 1 GiB, so every registered operator is guaranteed 1 GiB. The floor is also capped by the operator’s own remaining size — you cannot reserve more than the operator needs.

The manager itself never sees the whole buffer pool budget:

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:89
void TemporaryMemoryManager::UpdateConfiguration(ClientContext &context) {
	auto &buffer_manager = BufferManager::GetBufferManager(context);
	auto &task_scheduler = TaskScheduler::GetScheduler(context);

	memory_limit =
	 LossyNumericCast<idx_t>(MAXIMUM_MEMORY_LIMIT_RATIO * static_cast<double>(buffer_manager.GetMaxMemory()));
	has_temporary_directory = buffer_manager.HasTemporaryDirectory();
	num_threads = task_scheduler.NumberOfThreads();
	num_connections = ConnectionManager::Get(context).GetConnectionCount();
	query_max_memory = buffer_manager.GetOperatorMemoryLimit();

90 % of the buffer pool’s max memory, so there is always headroom for the buffer manager’s own accounting. And MAXIMUM_FREE_MEMORY_RATIO caps any single state at 90 % of the free memory — one operator cannot swallow the budget and starve the rest.

There is also a special case that short-circuits the whole model: if there is no temporary directory, nothing can be spilled, so there is no reason to limit anybody.

 cpp
 
 
// src/storage/temporary_memory_manager.cpp:118
void TemporaryMemoryManager::UpdateState(ClientContext &context, TemporaryMemoryState &temporary_memory_state) {
	UpdateConfiguration(context);

	// The lower bound for the reservation of this state is either the minimum reservation or the remaining size
	const auto lower_bound =
	 MinValue(temporary_memory_state.GetMinimumReservation(), temporary_memory_state.GetRemainingSize());

	if (temporary_memory_state.GetRemainingSize() == 0) {
		// Sometimes set to 0 to denote end of state (before actually deleting the state)
		SetReservation(temporary_memory_state, 0);
	} else if (Settings::Get<DebugForceExternalSetting>(context)) {
		// We're forcing external processing. Give it the minimum
		SetReservation(temporary_memory_state, lower_bound);
	} else if (!has_temporary_directory) {
		// We cannot offload, so we cannot limit memory usage. Set reservation equal to the remaining size
		SetReservation(temporary_memory_state, temporary_memory_state.GetRemainingSize());
	} else if (reservation - temporary_memory_state.GetReservation() + lower_bound >= memory_limit) {
		// We overshot. Set reservation equal to the minimum
		SetReservation(temporary_memory_state, lower_bound);
	} else {
		// The upper bound for the reservation of this state is the minimum of:
		// 1. Remaining size of the state
		// 2. The max memory per query
		// 3. MAXIMUM_FREE_MEMORY_RATIO * free memory
		auto upper_bound = MinValue(temporary_memory_state.GetRemainingSize(), query_max_memory);
		const auto free_memory = memory_limit - (reservation - temporary_memory_state.GetReservation());
		upper_bound = MinValue(upper_bound,
		 LossyNumericCast<idx_t>(MAXIMUM_FREE_MEMORY_RATIO * static_cast<double>(free_memory)));
		upper_bound = MinValue(upper_bound, free_memory);

		idx_t new_reservation;
		if (lower_bound >= upper_bound) {
			new_reservation = lower_bound;
		} else {
			new_reservation = remaining_size > memory_limit ? ComputeReservation(temporary_memory_state) : upper_bound;
		}

		SetReservation(temporary_memory_state, new_reservation);
	}

	Verify();
}

Notice that the optimiser is only invoked when the system is actually under pressure (remaining_size > memory_limit). When everything fits, the upper bound is simply “all of it” and there is nothing to optimise.

What it costs

The optimisation is not free, so here is the bill.

ComputeDerivatives is O(n) with one log and one exp per call.
The allocation loop runs 5 × n rounds, each calling ComputeDerivatives, so the whole thing is O(n²) with a small constant. With three operators that is 15 rounds; with thirty it is 150.
ComputeReservation is called under the manager’s lock, from UpdateState, which is called whenever an operator’s remaining size changes.

For the operator counts DuckDB actually sees — a handful of concurrent operators per query, a few dozen at the very worst — this is tens of microseconds, paid on reservation changes rather than per tuple. The alternative, a heuristic like “half the memory to the biggest operator”, is cheaper and measurably worse at the boundary between “just barely fits” and “just barely spills”. Spilling a hash table to disk is expensive enough that being right about the threshold pays for the optimiser many times over.

Two honest caveats:

The model treats s_i and p_i as constants during the optimisation. If an operator’s remaining size changes mid-flight — a hash join that discovers its build side is larger than it thought — the whole split has to be recomputed. DuckDB handles this by re-running UpdateState whenever the remaining size changes.
The final phase is a heuristic for order-independence, not a proof of it. Two states with very similar derivatives can still end up with different reservations depending on the order in which they ask.

The bottom line

Memory allocation between concurrent operators is an optimisation problem, and treating it as one pays off.

The three ideas worth carrying away:

Model the tradeoff explicitly. “Bytes needed” and “bytes granted” give a throughput; throughputs combine into a geometric mean, and the shortfall of that mean is what a starved operator does to the whole query.
Get the gradient in closed form. The cost is a product of an affine function and a concave-composed term, so its derivative is a two-line formula. No numerical differentiation, no black-box solver, and the gradient costs one pass.
Greedy marginal allocation on a convex objective is enough. Pick the variable with the most negative partial derivative, give it memory, repeat. Five rounds per operator is a heuristic, but the convexity is what makes it a good one.

The floors are the other half of the story. An optimiser that maximises throughput will happily starve an operator to zero, so DuckDB pairs the optimisation with a hard per-state minimum and a hard per-state maximum. The best split of a budget you do not own is the one that does not require anyone to go hungry.

The Lock-Free Eviction Queue: Weak Pointers and Sequence Numbers

Jay Hubbert — Sat, 24 Oct 2026 13:10:00 +0000

TL;DR: The eviction queue is a duckdb_moodycamel::ConcurrentQueue of BufferEvictionNode{ weak_ptr<BlockMemory> memory_p; idx_t handle_sequence_number; }. Because the queue stores only a weak pointer, enqueuing a block costs nothing and never keeps a dead block alive — the queue is allowed to accumulate garbage. An entry is stale if the weak pointer has expired (the block was destroyed) or if its sequence number no longer matches the block’s current eviction_seq_num (a newer entry superseded it). Staleness is therefore detectable without taking the block lock. Garbage collection is amortised: a purge runs every 4096 insertions and uses try_dequeue_bulk + enqueue_bulk with a separate producer token, so the survivors land in a sub-queue the consumer token has already walked past and FIFO order is preserved.

The scenario

Eight threads are scanning eight different tables. The memory limit is 4 GB. Every time a scan finishes a block it hands the block back to the pool; every time a scan wants a new block it takes one. Meanwhile, the pool is over its limit and needs to make room.

The pool’s job is to keep a list of blocks it could throw away, in roughly the order it should throw them away. That list has to survive:

concurrency — many threads adding and removing entries at once;
blocks that die — a BlockHandle can be destroyed while its entry is still in the queue, leaving a dangling reference;
duplicate entries — the same block can be enqueued, evicted-and-reloaded, and enqueued again, so the queue can hold several entries for the same block, only one of which is current.

A naive std::queue<shared_ptr<BlockHandle>> fails on all three counts: it needs a lock, and it cannot represent “the block this entry refers to is gone”.

The node: a weak pointer and a version number

Here is the whole type:

 cpp
 
 
// src/include/duckdb/storage/buffer/buffer_pool.hpp:27
struct BufferEvictionNode {
	BufferEvictionNode() {
	}
	BufferEvictionNode(weak_ptr<BlockMemory> block_memory_p, idx_t eviction_seq_num);

	weak_ptr<BlockMemory> memory_p;
	idx_t handle_sequence_number;

	bool IsDeadNode(optional_idx debug_sleep_micros = optional_idx());
};

Two fields. memory_p is a weak pointer to the block’s BlockMemory; handle_sequence_number is a version number.

Why weak? Because the queue must not keep a block alive. If the queue held a shared_ptr, then a block could never be freed while its entry sat in the queue, and the queue would grow without bound as blocks are loaded and unloaded. With a weak_ptr, the entry is allowed to dangle — the entry survives the block, and the purger notices.

Why a sequence number? Because a block can be enqueued more than once. Suppose a block is unpinned (enqueued, sequence 1), then pinned and unpinned again (enqueued, sequence 2). Now the queue holds two entries for the same block. Only the second is current. Without a version, the sweeper would happily unload the block twice, or unload it while somebody had re-pinned it.

The version lives on the block itself:

 cpp
 
 
// src/include/duckdb/storage/buffer/block_handle.hpp:120
	//! Returns the eviction sequence number.
	idx_t GetEvictionSequenceNumber() const {
		return eviction_seq_num.load(std::memory_order_acquire);
	}
	//! Increments the eviction sequence number prior to returning it.
	idx_t NextEvictionSequenceNumber(BlockLock &l) {
		VerifyMutex(l);
		auto new_sequence_number = eviction_seq_num.load(std::memory_order_relaxed) + 1;
		// Publish preceding dead-node accounting to lock-free queue readers.
		eviction_seq_num.store(new_sequence_number, std::memory_order_release);
		return new_sequence_number;
	}

and the declaration carries the invariant in a comment:

 cpp
 
 
// src/include/duckdb/storage/buffer/block_handle.hpp:242
	//! The internal eviction sequence number. Monotonic: it is never reset, so an eviction queue
	//! entry is stale if and only if its sequence number differs from this one. Writers hold the block lock.
	atomic<idx_t> eviction_seq_num;

Monotonic. Never reset. That is the key property. Because the counter only ever goes up, “the entry’s number differs from the block’s current number” is an unambiguous test for “this entry is from an older generation”. No ABA problem, no epoch, no generation list.

Two ways a node dies

The liveness test is short:

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:42
BufferEvictionNode::BufferEvictionNode(weak_ptr<BlockMemory> block_memory_p, idx_t eviction_seq_num)
 : memory_p(std::move(block_memory_p)), handle_sequence_number(eviction_seq_num) {
	D_ASSERT(!memory_p.expired());
}

bool BufferEvictionNode::IsDeadNode(optional_idx debug_sleep_micros) {
	auto shared_memory_p = memory_p.lock();
	if (debug_sleep_micros.IsValid()) {
		ThreadUtil::SleepMicroSeconds(debug_sleep_micros.GetIndex());
	}
	if (!shared_memory_p) {
		return true;
	}
	if (handle_sequence_number != shared_memory_p->GetEvictionSequenceNumber()) {
		return true;
	}
	return false;
}

Two death certificates:

!shared_memory_p — the weak pointer could not be upgraded. The BlockMemory was destroyed, so the block is gone entirely. The entry is garbage.
handle_sequence_number != shared_memory_p->GetEvictionSequenceNumber() — the block is alive, but this entry is from an older generation. A newer entry for the same block exists, and that one is the live one.

Note that the second test reads GetEvictionSequenceNumber() with memory_order_acquire while the writer used memory_order_release. That pairing is what makes the test well-defined: if the reader sees the new number, it also sees everything the writer did before storing it.

And note what is not here: no lock. IsDeadNode touches only the weak pointer’s control block and one atomic load. It can be called on a queue full of blocks that other threads are concurrently locking, loading and unloading. The staleness test is entirely lock-free.

The debug_sleep_micros parameter is a debug-only hook: it makes the thread sleep between upgrading the weak pointer and checking the sequence number, so that a race between a block’s destruction and the purger can be provoked on purpose. There is a whole DuckDB setting for it.

The queue

The container is a third-party lock-free MPMC queue:

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:61
typedef duckdb_moodycamel::ConcurrentQueue<BufferEvictionNode> eviction_queue_t;

duckdb_moodycamel::ConcurrentQueue is Cameron Desrochers’ implementation, vendored in third_party/concurrentqueue. Its own header describes it as “a C++11 implementation of a multi-producer, multi-consumer lock-free queue”, and points at a write-up of the design.

And DuckDB keeps a no-threads fallback so that single-threaded builds still compile:

 cpp
 
 
// src/include/duckdb/parallel/concurrentqueue.hpp:58
template <typename T>
class ConcurrentQueue {
private:
	//! The standard library queue.
	std::queue<T, std::deque<T>> q;

public:
	//! Default constructor.
	ConcurrentQueue() = default;
	//! Constructor reserving capacity.
	explicit ConcurrentQueue(size_t capacity) {
		q.reserve(capacity);
	}

The real queue is not a ring buffer. It is a linked list of per-producer sub-queues, which is what makes the producer/consumer token trick possible.

Adding to the queue

Here is the enqueue path, from the BufferPool down to the queue:

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:274
bool BufferPool::AddToEvictionQueue(BlockLock &lock, shared_ptr<BlockHandle> &handle) {
	auto &memory = handle->GetMemory();
	// Verify the caller passed this block's lock before we mutate any of its state.
	// The block lock is held throughout: Unpin holds it; ConvertToPersistent acquires the
	// (uncontended) lock of the freshly created block before calling.
	memory.VerifyMutex(lock);
	auto &queue = GetEvictionQueueForBlockMemory(memory);

	D_ASSERT(memory.GetReaders() == 0);
	if (memory.HasLiveQueueEntry(lock)) {
		// Count the previous live entry before bumping the sequence number. PurgeIteration
		// reads sequence numbers without the block lock; bumping first could let it see the
		// previous entry as stale and decrement dead_nodes before this matching increment.
		queue.IncrementDeadNodes();
	}

	auto ts = memory.NextEvictionSequenceNumber(lock);

Three things in this excerpt, and all three are about the race between the lock-free purger and this locked writer.

HasLiveQueueEntry tracks whether the block already has an entry in the queue that has not been counted as dead. If it does, the previous entry is about to become stale (we are about to bump the sequence number), so the dead-node counter is incremented now.

The ordering comment is the interesting part:

Count the previous live entry before bumping the sequence number. PurgeIteration reads sequence numbers without the block lock; bumping first could let it see the previous entry as stale and decrement dead_nodes before this matching increment.

This is a genuine race, and the code closes it by ordering the two operations. If we bumped the sequence number first, a concurrent purger could observe the old entry as stale, decide it is dead, and decrement total_dead_nodes — and then our increment would arrive and push the counter up by one, leaving the counter permanently one too high. By counting the dead node first, the counter is momentarily one too low rather than one too high, and the purger’s decrement corrects it.

NextEvictionSequenceNumber is called last, and it bumps the counter that makes the previous entry stale. After this point, the new entry is the only live one.

Then the actual enqueue:

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:144
bool EvictionQueue::AddToEvictionQueue(BufferEvictionNode &&node) {
	q.enqueue(std::move(node));
	return ++evict_queue_insertions % INSERT_INTERVAL == 0;
}

One line of real work, and then the amortisation trigger: the queue counts its own insertions and reports “time to purge” every 4096 of them.

The purge

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:154
void EvictionQueue::Purge() {
	// only one thread purges the queue, all other threads early-out
	unique_lock<mutex> guard(purge_lock, std::try_to_lock);
	if (!guard.owns_lock()) {
		return;
	}

	// we purge INSERT_INTERVAL * PURGE_SIZE_MULTIPLIER nodes
	idx_t purge_size = INSERT_INTERVAL * PURGE_SIZE_MULTIPLIER;

	// get an estimate of the queue size as-of now
	idx_t approx_q_size = q.size_approx();

	// early-out, if the queue is not big enough to justify purging
	// - we want to keep the LRU characteristic alive
	if (approx_q_size < purge_size * EARLY_OUT_MULTIPLIER) {
		return;
	}

The purge is not a lock-free operation, and it does not need to be: it is a rare, batch-oriented garbage collection, and it is explicitly serialised. std::try_to_lock means that if another thread is already purging, this thread just gives up — the purge will happen anyway.

The early-out is a heuristic with a stated motive: “we want to keep the LRU characteristic alive”. Purging a queue that is mostly alive means re-inserting most of its nodes, which costs time and disturbs the order. So the purge only runs when the queue is big.

The constants:

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:114
private:
	//! We trigger a purge of the eviction queue every INSERT_INTERVAL insertions
	constexpr static idx_t INSERT_INTERVAL = 4096;
	//! We multiply the base purge size by this value.
	constexpr static idx_t PURGE_SIZE_MULTIPLIER = 2;
	//! We multiply the purge size by this value to determine early-outs. This is the minimum queue size.
	//! We never purge below this point.
	constexpr static idx_t EARLY_OUT_MULTIPLIER = 4;
	//! We multiply the approximate alive nodes by this value to test whether our total dead nodes
	//! exceed their allowed ratio. Must be greater than 1.
	constexpr static idx_t ALIVE_NODE_MULTIPLIER = 4;

So: purge every 4096 insertions, purge 8192 nodes at a time, and only bother if the queue has at least 32768 nodes in it.

The bulk purge, and the FIFO trick

Here is the actual garbage collection:

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:215
void EvictionQueue::PurgeIteration(const idx_t purge_size) {
	// if this purge is significantly smaller or bigger than the previous purge, then
	// we need to resize the purge_nodes vector. Note that this barely happens, as we
	// purge queue_insertions * PURGE_SIZE_MULTIPLIER nodes
	idx_t previous_purge_size = purge_nodes.size();
	if (purge_size < previous_purge_size / 2 || purge_size > previous_purge_size) {
		purge_nodes.resize(purge_size);
	}

	// Dequeue using consumer token — progresses through sub-queues sequentially
	const idx_t actually_dequeued = q.try_dequeue_bulk(purge_consumer_token, purge_nodes.begin(), purge_size);
	if (actually_dequeued == 0) {
		return;
	}

	idx_t dead_count = 0;
	idx_t alive_count = 0;
	auto raw_sleep_micros = debug_eviction_queue_sleep.load(std::memory_order_relaxed);
	optional_idx debug_sleep_micros = raw_sleep_micros > 0 ? optional_idx(raw_sleep_micros) : optional_idx();
	for (idx_t i = 0; i < actually_dequeued; i++) {
		auto &node = purge_nodes[i];
		if (node.IsDeadNode(debug_sleep_micros)) {
			dead_count++;
		} else {
			// Move alive nodes to the front for bulk re-enqueue
			purge_nodes[alive_count++] = std::move(node);
		}
	}

	total_dead_nodes -= dead_count;

	// Re-enqueue alive nodes via producer token — goes into a dedicated sub-queue
	// that the consumer token has already passed
	if (alive_count > 0) {
		if (!purge_producer_token.valid()) {
			throw OutOfMemoryException("Failed to allocate eviction queue producer");
		}
		q.enqueue_bulk(purge_producer_token, purge_nodes.begin(), alive_count);
	}
}

The algorithm is: dequeue a batch, partition it into dead and alive, and re-enqueue the alive ones.

The partition is the compaction trick: alive nodes are moved to the front of the purge_nodes vector (purge_nodes[alive_count++] = std::move(node)), so the survivors end up packed in [0, alive_count) and can be re-enqueued in one bulk call.

Now the part that deserves a slow look: why a separate producer token?

The ConcurrentQueue in moodycamel’s design is a list of per-producer sub-queues, and a consumer token walks them in a fixed order, remembering which one it was on. If the purger re-enqueued the survivors into the same sub-queue the consumer was about to visit, the survivors would be dequeued again immediately, in a loop, forever. The purger would make no progress.

The fix is to give the purge its own producer token. That token is bound to a different sub-queue — one the consumer token has already walked past. So the survivors are pushed behind the consumer’s current position in the traversal, and the consumer will come back around to them only after it has finished with the sub-queues it has not yet visited.

The result: the purge preserves the queue’s FIFO order. The oldest blocks stay oldest, and the LRU characteristic the comment mentioned is preserved across a GC cycle.

The total_dead_nodes bookkeeping also matters: the purger decrements the counter by exactly the number of dead nodes it removed. That counter is what drives the “purge more aggressively” loop in Purge() — when the dead-to-alive ratio gets bad, the purger keeps going until it either reaches a healthy ratio or runs out of queue.

The eviction sweep itself

For completeness, here is the consumer side — the part that actually unloads blocks:

 cpp
 
 
// src/storage/buffer/buffer_pool.cpp:455
template <typename FN>
void EvictionQueue::IterateUnloadableBlocks(FN fn) {
	auto debug_sleep_micros = debug_eviction_queue_sleep.load(std::memory_order_relaxed);
	for (;;) {
		// get a block to unpin from the queue
		BufferEvictionNode node;
		if (!q.try_dequeue(node)) {
			// we could not dequeue any eviction node, so we try one more time,
			// but more aggressively
			if (!TryDequeueWithLock(node)) {
				return;
			}
		}

		// get a reference to the underlying block pointer
		auto handle = node.memory_p.lock();
		if (debug_sleep_micros > 0) {
			// Debug race conditions regarding the ownership of the BlockMemory.
			// Note that for this to trigger we need at least one purge iteration with the setting active.
			ThreadUtil::SleepMicroSeconds(debug_sleep_micros);
		}
		if (!handle) {
			DecrementDeadNodes();
			continue;
		}

		// we might be able to free this block: grab the mutex and check if we can free it
		auto lock = handle->GetLock();
		if (node.handle_sequence_number != handle->GetEvictionSequenceNumber()) {
			// A newer entry superseded this node: it was counted as dead when that entry was added.
			DecrementDeadNodes();
			continue;
		}

The same sequence-number test appears here, but now it is inside the block lock. The difference matters:

In the purger (lock-free), a mismatch means “this entry is stale, drop it”.
Here (holding the lock), a mismatch means “this entry is stale, drop it — and it was already counted as dead when the newer entry was added”, so the counter is decremented.

And when the entry is current, the block is unloaded, under the lock, and the block loses its queue entry (SetHasLiveQueueEntry(lock, false)) so that a later unpin will re-enqueue it with a fresh sequence number.

Two failure paths are handled with care. If the block cannot be offloaded because there is no temporary directory, the node is re-enqueued and the sweep stops — the block is not lost. And if the unload itself throws, the node is re-enqueued and the exception is rethrown, so a failed unload never leaves a block permanently un-evictable.

Costs and trade-offs

The queue can grow without bound between purges. That is the deliberate trade: a weak_ptr entry costs 16 bytes and never keeps anything alive, so a queue with a million stale entries costs 16 MB and nothing else. The purger is the garbage collector, and it runs on a schedule rather than on every operation.
Purging is O(dead nodes), amortised. Every 4096 insertions, up to 8192 nodes are examined. Each examination is one weak-pointer upgrade (an atomic CAS on the control block) plus one atomic load. There is no per-operation cost beyond the enqueue itself.
The weak-pointer upgrade is not free. memory_p.lock() is an atomic compare-and-swap on the control block, and it will fail for dead nodes. But it is lock-free and wait-free, and it is the only way to know whether the block is still alive.
The sequence number is a uint64_t on the block. It costs 8 bytes per block and one atomic increment per enqueue. In exchange, the purger needs no lock and no epoch-based scheme.
The dead-node counter is approximate by design. It is maintained under a mixture of locked and lock-free operations, and the source comments are explicit about the race and about which way the error is allowed to go. It is used to decide how hard to purge, not for correctness.
The pool is not one queue but several. FileBufferTypeToEvictionQueueTypeIdx maps buffer types onto three queue types — BLOCK/EXTERNAL_FILE (cheap to discard), MANAGED_BUFFER (must be spilled to a temp file), and TINY_BUFFER (last resort) — and the sweeper drains them in that order. MANAGED_BUFFER alone is split into six queues so that the radix-partitioned row-group allocator can keep a partition’s blocks together. All of that priority lives in which queue a block goes into, not in the queue algorithm itself.
The debug sleep hook exists because these races are real. IsDeadNode takes an optional sleep between upgrading the weak pointer and reading the sequence number, precisely so a test can widen that window and check that the accounting still balances.

The bottom line

The eviction queue is a good example of a design where the data structure is dumb and the protocol is clever. The queue itself is a general-purpose lock-free MPMC container that knows nothing about blocks. All the block-specific intelligence lives in two fields of the node: a weak pointer, so the queue never keeps anything alive, and a monotonic version number, so a stale entry can be recognised without a lock.

The version number is the trick worth stealing. A weak pointer alone tells you “the block is gone”. A weak pointer plus a monotonically increasing per-block counter tells you “this entry is either from a dead block or from a previous generation of a live one” — and both of those questions can be answered with two atomic operations and no lock. No epochs, no hazard pointers, no generation lists.

The second trick worth stealing is the separate producer token. Garbage collection that re-inserts survivors into the same queue it is draining is a livelock waiting to happen; giving the collector its own producer token so the survivors land behind the consumer’s current position is a two-line fix that preserves the queue’s ordering guarantees.

And the general lesson about counters: DuckDB maintains a total_dead_nodes counter across a boundary between locked and lock-free code, and the source comments spend more words explaining which way the error is allowed to go than the arithmetic itself. When a counter is only a heuristic, say so — loudly — and decide deliberately whether it may over- or under-count. The decision is written down: under-count first, let the purger’s decrement fix it up. That is a much better failure mode than the opposite.

Pinning Is Ownership: DuckDB's Buffer Handle Is a shared_ptr

Jay Hubbert — Sat, 24 Oct 2026 12:40:00 +0000

TL;DR: In DuckDB, “pinning” a block means holding a shared_ptr to it. A BufferHandle owns a shared_ptr<BlockHandle>; the BlockHandle owns a shared_ptr<BlockMemory>; and the BlockMemory is the only place the actual FileBuffer bytes live. While a reader holds the handle, the bytes cannot be destroyed, so the block cannot be evicted underneath it. The readers atomic counter is not the safety mechanism — it is an advisory signal that lets the eviction sweeper skip pinned blocks without taking a lock. The safety comes free from the ownership graph.

The scenario

You run a query over a table that is much bigger than your memory limit:

 sql
 
SELECT sum(o_totalprice) FROM orders;

The buffer pool is under pressure. The eviction sweeper is running, deciding which blocks to throw out to make room for the next one. Meanwhile, the scan is holding several blocks open and is about to read a DECIMAL out of one of them.

The question this article is about is narrow and important: how does DuckDB guarantee that the eviction sweeper cannot pull the rug out from under that read?

The naive answer — “it keeps a counter of active readers and refuses to evict a block with a non-zero count” — is how a lot of buffer managers work, and it is subtly wrong. A counter is a hint. Between the moment the sweeper reads the counter and the moment it acts on it, the count can change. Any design that relies on the counter for correctness has a race in it.

DuckDB does not rely on the counter for correctness. It relies on ownership.

The ownership chain

Start at the top. This is the entire BufferHandle class:

 cpp
 
 
// src/include/duckdb/storage/buffer/buffer_handle.hpp:53
	const shared_ptr<BlockHandle> &GetBlockHandle() const {
		return handle;
	}

private:
	//! The block handle
	shared_ptr<BlockHandle> handle;
	//! The managed buffer node
	optional_ptr<FileBuffer> node;

Two members. handle is a shared_ptr<BlockHandle> — a shared ownership handle to the block’s metadata. node is a raw optional_ptr<FileBuffer> — a non-owning pointer to the bytes.

That split is the whole design. The handle owns the block; the block owns the bytes; the handle merely points at the bytes. The pointer is a convenience (it is what Ptr() returns), not a lifetime guarantee.

Here is the BlockHandle side:

 cpp
 
 
// src/include/duckdb/storage/buffer/block_handle.hpp:292
	//! Returns a const reference to the memory of a block.
	const BlockMemory &GetMemory() const {
		return memory;
	}
	//! Returns a reference to the memory of a block.
	BlockMemory &GetMemory() {
		return memory;
	}
	//! Returns a weak pointer to the memory of a block.
	weak_ptr<BlockMemory> GetMemoryWeak() const {
		return weak_ptr<BlockMemory>(memory_p);
	}

and the members at the bottom of the class:

 cpp
 
 
// src/include/duckdb/storage/buffer/block_handle.hpp:324
	//! Pointer to the underlying memory of the block.
	const shared_ptr<BlockMemory> memory_p;
	//! Memory for fast access to the block memory.
	BlockMemory &memory;

There are two views of the same BlockMemory:

memory_p is a shared_ptr<BlockMemory> — the owner.
memory is a BlockMemory & — a raw reference, for fast access without an atomic refcount bump.

And BlockMemory is where the bytes actually live:

 cpp
 
 
// src/include/duckdb/storage/buffer/block_handle.hpp:234
	//! The number of concurrent readers. Writers hold the block lock; reads may be unlocked.
	atomic<int32_t> readers;
	//! The memory tag.
	const MemoryTag tag;
	//! The file buffer type.
	const FileBufferType buffer_type;
	//! A pointer to the loaded data, if any.
	unique_ptr<FileBuffer> buffer;

buffer is a unique_ptr<FileBuffer>. That is the object that owns the memory the query is reading.

So the chain is:

text

BufferHandle
 └─ shared_ptr<BlockHandle>
 └─ shared_ptr<BlockMemory> (memory_p)
 └─ unique_ptr<FileBuffer> (buffer)
 └─ the actual bytes

Three levels of ownership, and every level is a shared_ptr or a unique_ptr. When the last shared_ptr to the BlockMemory goes away, the BlockMemory is destroyed, its unique_ptr<FileBuffer> is destroyed, and the bytes are released. There is no other path to freeing the bytes, because buffer is private to BlockMemory and there is exactly one BlockMemory per block.

This is the invariant: the bytes cannot be freed while any BufferHandle exists, because a BufferHandle transitively holds the only shared_ptr chain to them.

Why the readers counter is only a hint

Here is the counter’s implementation:

 cpp
 
 
// src/include/duckdb/storage/buffer/block_handle.hpp:79
	//! Increments the number of readers prior to returning it.
	int32_t IncrementReaders(BlockLock &l) {
		VerifyMutex(l);
		// Tiny buffers do not decrement on unpin, so their reader count can wrap.
		auto new_readers = static_cast<int32_t>(static_cast<uint32_t>(readers.load(std::memory_order_relaxed)) + 1);
		readers.store(new_readers, std::memory_order_relaxed);
		return new_readers;
	}
	//! Decrements the number of readers prior to returning it.
	int32_t DecrementReaders(BlockLock &l) {
		VerifyMutex(l);
		auto new_readers = static_cast<int32_t>(static_cast<uint32_t>(readers.load(std::memory_order_relaxed)) - 1);
		readers.store(new_readers, std::memory_order_relaxed);
		return new_readers;
	}

Two things to notice.

The comment about tiny buffers. “Tiny buffers do not decrement on unpin, so their reader count can wrap.” A TINY_BUFFER is a small allocation that is not managed by the pool; it is never unloaded, so the counter would only ever grow. Rather than special-case it, the code explicitly allows the int32_t to wrap around by doing the arithmetic in uint32_t and casting back. This is a deliberate decision to make a hint wrong rather than to add a branch to a hot path. And it is safe precisely because the hint is not load-bearing for correctness.

The memory ordering is relaxed. The counter is not a synchronisation primitive. It is a number that is approximately right. If two threads increment it concurrently, the result is some number that is not exactly the number of readers — and that is fine, because the counter is only used to answer “should I bother looking at this block?”.

And here is where it is consumed:

 cpp
 
 
// src/storage/buffer/block_handle.cpp:105
CanUnloadResult BlockMemory::CanUnload() const {
	if (GetState() == BlockState::BLOCK_UNLOADED) {
		// The block has already been unloaded.
		return CanUnloadResult::ALREADY_UNLOADED;
	}
	if (GetReaders() > 0) {
		// There are active readers.
		return CanUnloadResult::PINNED;
	}
	if (BlockId() >= MAXIMUM_BLOCK && MustWriteToTemporaryFile() && !GetBufferManager().HasTemporaryDirectory()) {
		// The block memory cannot be destroyed upon eviction/unpinning.
		// In order to unload this block we need to write it to a temporary buffer.
		// However, no temporary directory is specified, hence, we cannot unload.
		return CanUnloadResult::NO_TEMP_DIRECTORY;
	}
	return CanUnloadResult::CAN_UNLOAD;
}

GetReaders() > 0 → PINNED. The sweeper skips it.

But how is this safe? The answer is that the sweeper does not free the bytes by destroying the FileBuffer out from under a reader. It frees them by unloading the block — and unloading requires taking the block’s lock, and the block’s state transition to BLOCK_UNLOADED is guarded by that lock.

The real safety net is the combination:

A reader holds a shared_ptr chain to the BlockMemory, so the BlockMemory — and therefore the FileBuffer — cannot be destroyed.
The sweeper’s decision to unload is made under the block lock.
A reader that is about to read takes the block lock when it loads or increments the reader count, so it cannot race with an unload.

The counter, meanwhile, tells the sweeper “don’t even bother taking the lock for this one”. It is an optimisation. If the counter were wrong, the sweeper would take the lock, see the true state, and do the right thing anyway.

Pinning: the double-checked load

Here is Pin, the function that turns a BlockHandle into a BufferHandle:

 cpp
 
 
// src/storage/standard_buffer_manager.cpp:369
BufferHandle StandardBufferManager::Pin(const QueryContext &context, shared_ptr<BlockHandle> &handle) {
	// we need to be careful not to return the BufferHandle to this block while holding the BlockHandle's lock
	// as exiting this function's scope may cause the destructor of the BufferHandle to be called while holding the lock
	// the destructor calls Unpin, which grabs the BlockHandle's lock again, causing a deadlock
	BufferHandle buf;

	idx_t required_memory;
	auto &block_memory = handle->GetMemory();
	{
		// lock the block
		auto lock = block_memory.GetLock();
		// check if the block is already loaded
		if (block_memory.GetState() == BlockState::BLOCK_LOADED) {
			// the block is loaded, increment the reader count and set the BufferHandle
			buf = handle->Load(lock, context);
		}
		required_memory = block_memory.GetMemoryUsage();
	}

	if (buf.IsValid()) {
		return buf; // the block was already loaded, return it without holding the BlockHandle's lock
	}

Read the comment at the top. It is a self-deadlock warning, and it is the most instructive line in the whole file:

we need to be careful not to return the BufferHandle to this block while holding the BlockHandle’s lock as exiting this function’s scope may cause the destructor of the BufferHandle to be called while holding the lock the destructor calls Unpin, which grabs the BlockHandle’s lock again, causing a deadlock

This is the price of the ownership design. BufferHandle’s destructor calls Unpin, which takes the block lock. If the handle were returned while the lock was held, the temporary’s destructor would run at the end of the scope — with the lock still held — and Unpin would try to take the same non-recursive std::mutex again. Boom.

So Pin deliberately returns the handle outside the lock’s scope. The lock is acquired in an inner block { ... } precisely so that it is released before the return.

Continue reading the function and you see the classic double-checked pattern:

 cpp
 
 
// src/storage/standard_buffer_manager.cpp:392
	// evict blocks until we have space for the current block
	unique_ptr<FileBuffer> reusable_buffer;
	auto reservation =
	 EvictBlocksOrThrow(context, block_memory.GetMemoryTag(), required_memory, &reusable_buffer,
	 "failed to pin block of size %s%s", StringUtil::BytesToHumanReadableString(required_memory));

	// lock the handle again and repeat the check (in case anybody loaded in the meantime)
	auto lock = block_memory.GetLock();
	// check if the block is already loaded
	if (block_memory.GetState() == BlockState::BLOCK_LOADED) {
		// the block is loaded, increment the reader count and return a pointer to the handle
		reservation.Resize(0);
		buf = handle->Load(lock, context);
	} else {

Between releasing the lock and re-acquiring it, we went and evicted blocks to make room — which takes an unbounded amount of time and releases the lock entirely. Another thread may have loaded the same block in the meantime. So the state is checked again under the lock. If somebody beat us to it, the memory reservation we took is refunded (reservation.Resize(0)) and we just take a reader count on the already-loaded block.

The reservation refund is a nice touch: the memory is charged to the pool the moment we decide we might need it, and un-charged the moment we discover we don’t. The charging is an RAII token, so the refund is a single call.

And here is the actual load:

 cpp
 
 
// src/storage/buffer/block_handle.cpp:214
BufferHandle BlockHandle::Load(BlockLock &l, QueryContext context, unique_ptr<FileBuffer> reusable_buffer) {
	memory.VerifyMutex(l);
	if (memory.GetState() == BlockState::BLOCK_LOADED) {
		// The block has already been loaded.
		D_ASSERT(memory.GetBuffer());
		memory.IncrementReaders(l);
		return BufferHandle(shared_from_this(), memory.GetBuffer());
	}

	if (!memory.CanReload()) {
		// the buffer was destroyed upon unpin/evict, or the handle was replaced and its disk block
		// may be rewritten in place - the caller must re-resolve the block id
		return BufferHandle();
	}

BufferHandle(shared_from_this(), memory.GetBuffer()). The shared_from_this() is the crux: the BlockHandle hands out a shared_ptr to itself, and the BufferHandle stores it. That is what makes the whole chain hold.

Note also memory.IncrementReaders(l) — the counter is bumped here, under the block lock, at the same moment the handle is created. So the counter and the ownership are established atomically with respect to the lock. A sweeper that takes the lock and sees readers == 0 genuinely knows that at that instant no reader exists.

Unpinning

 cpp
 
 
// src/storage/standard_buffer_manager.cpp:459
void StandardBufferManager::Unpin(shared_ptr<BlockHandle> &handle) {
	bool purge = false;
	auto &block_memory = handle->GetMemory();
	{
		auto lock = block_memory.GetLock();
		if (!block_memory.GetBuffer(lock) || block_memory.GetBufferType() == FileBufferType::TINY_BUFFER) {
			return;
		}
		D_ASSERT(block_memory.GetReaders() > 0);
		auto new_readers = block_memory.DecrementReaders(lock);
		if (new_readers == 0) {
			VerifyZeroReaders(lock, handle);
			if (block_memory.MustAddToEvictionQueue()) {
				purge = buffer_pool.AddToEvictionQueue(lock, handle);
			} else {
				block_memory.Unload(lock);
			}
		}
	}

	// We do not have to keep the handle locked while purging.
	if (purge) {
		PurgeQueue(*handle);
	}
}

Three things:

Tiny buffers return early. They were never added to the pool, so there is nothing to unpin. (And, as we saw, that is why their reader counter is allowed to wrap.)
When the count hits zero, the block goes on the eviction queue. Note the order: the count is decremented first, and only then is the block offered to the sweeper. That ordering is what makes the counter a valid hint.
The purge happens outside the lock. The comment says it: “We do not have to keep the handle locked while purging.” A purge is a full sweep of the eviction queue — it can take a while, and holding the block’s lock through it would block every other thread that wants to pin the same block. The purge flag is set under the lock; the call happens after the lock is released.

And here is where the handle gives up its ownership:

 cpp
 
 
// src/storage/buffer/buffer_handle.cpp:35
void BufferHandle::Destroy() {
	if (!handle || !IsValid()) {
		return;
	}
	handle->GetMemory().GetBufferManager().Unpin(handle);
	handle.reset();
	node = nullptr;
}

Unpin first, then handle.reset(). The reset is what drops the shared_ptr — and if that was the last reference to the BlockMemory, the bytes are freed right there. If the block was still in use elsewhere, the shared_ptr count simply stays above zero and the bytes survive.

Note the order: Unpin before reset. If it were the other way round, the BlockMemory could be destroyed before Unpin had decremented the counter, and Unpin would be touching freed memory.

Costs and trade-offs

The reference counting is an atomic increment on the hot path. Every Pin that finds the block already loaded does one shared_from_this() (an atomic increment on the BlockHandle’s control block) and one shared_ptr copy into the BufferHandle. Every Unpin does the matching decrement. On a hot scan, that is a couple of atomic read-modify-writes per block — cheap, but not free. It is worth it because it replaces a correctness mechanism (locking, reference counting by hand, hazard pointers) with a safety mechanism (the type system).
shared_ptr is not free in code size either. The enable_shared_from_this base class on BlockHandle, the weak_ptr accessor for the eviction queue, the ColumnBindingReplacer-style plumbing — all of it exists because the block is a shared_ptr citizen.
The readers counter is still worth keeping. It is an atomic<int32_t> with relaxed ordering, and it lets the eviction sweeper skip pinned blocks without taking a lock and without constructing a shared_ptr from the weak one. On a pool with thousands of blocks, most of which may be pinned, that saves a lot of lock traffic.
The counter is allowed to lie. Tiny buffers wrap it. The relaxed ordering means concurrent increments can lose a count. Both are fine because nothing depends on its exactness. This is the design paying off: an approximate counter is cheap and safe, and a precise counter would need a lock.
The Pin path takes the block lock at least once and often twice. The double-checked pattern exists because the eviction step in between is slow. If the block was already loaded, only the first lock is taken.
The deadlock hazard is real and is handled by a comment and by scope. Returning a handle whose destructor takes the same lock is the kind of bug you find in production once. The fix here — an inner scope — is subtle enough to deserve the three-line comment that explains it.

The bottom line

There are two ways to keep a block alive while someone reads it.

The first is a counter: “readers: 3, do not evict”. The second is ownership: “here is a shared_ptr; the memory is yours until you let go of it”. The first requires every participant to agree on the protocol and to update the counter at exactly the right moment; the second requires nothing but the discipline to use shared_ptr correctly.

DuckDB uses the second for safety and the first for speed, and it is explicit about which is which. The readers counter is relaxed, is allowed to wrap, and is consulted only to decide whether to bother taking a lock. The shared_ptr chain is the actual guarantee: BufferHandle → BlockHandle → BlockMemory → FileBuffer → the bytes, with a shared_ptr at every level that has a lifetime to protect.

The general lesson: if you need a lock to make your data structure safe, you probably have the wrong data structure. A lock-free shared_ptr is not as free as a raw pointer, but it is free enough, and it buys you the freedom to let the sweeper guess wrong occasionally without corrupting anything.

Buffer-Manager on Opensource internals

The Block Allocator: One mmap Arena, Per-Thread Free Lists and Lazy Commit

The problem with malloc(256 * 1024)

One arena, reserved but not committed

Finding a block with a shift

Per-thread free lists

Commit lazily, decommit with madvise

The packed retention estimate

The other size classes

What it costs

The bottom line

The Temporary Memory Manager Solves a Convex Optimization to Split Memory

The query that breaks an even split

The state: what an operator tells the manager

The objective function

Why this objective is convex

The closed-form gradient

From gradient to split

The fairness floors

What it costs

The bottom line

The Lock-Free Eviction Queue: Weak Pointers and Sequence Numbers

The scenario

The node: a weak pointer and a version number

Two ways a node dies

The queue

Adding to the queue

The purge

The bulk purge, and the FIFO trick

The eviction sweep itself

Costs and trade-offs

The bottom line

Pinning Is Ownership: DuckDB's Buffer Handle Is a shared_ptr

The scenario

The ownership chain

Why the readers counter is only a hint

Pinning: the double-checked load

Unpinning

Costs and trade-offs

The bottom line

The problem with `malloc(256 * 1024)`

Commit lazily, decommit with `madvise`