Memory-Tracking on Opensource internals

ClickHouse Intercepts Every Allocation to Charge It

Jay Hubbert — Sun, 25 Oct 2026 09:00:00 +0000

TL;DR: To charge memory you need to know that an allocation happened and how many bytes it was. ClickHouse solves the first problem by replacing operator new/operator delete and interposing the libc malloc family; the second by charging the actual size class (je_nallocx), by passing the size explicitly to free, and by asking jemalloc for the size of a pointer at free time (je_sallocx).

Memory accounting has two halves, and they are not equally hard.

The easy half is noticing that an allocation happened. Every allocator funnels through a small number of entry points, and a program can replace them.

The hard half is knowing how many bytes to charge. malloc(100) may consume 112 bytes, because allocators work in size classes. And free(p) does not say how big p was — the standard free has no size parameter at all. Charge the requested size and you under-count by the rounding on every allocation; charge the wrong thing on free and the counters drift apart permanently.

ClickHouse’s answer is three-layered, and each layer solves a slightly different version of the same problem.

Layer 1: the C++ operators

The most direct interception point is operator new. ClickHouse replaces all eight overloads:

 cpp
 
 
// src/Common/AllocationInterceptors.cpp:54
void * operator new(std::size_t size)
{
 AllocationTrace trace;
 std::size_t actual_size = Memory::trackMemory(size, trace);
 void * ptr = Memory::newNoExcept(size);
 if (!ptr) [[unlikely]]
 {
 [[maybe_unused]] auto rollback_trace = CurrentMemoryTracker::free(actual_size);
 throw std::bad_alloc{};
 }
 trace.onAlloc(ptr, actual_size);
 return ptr;
}

Three things to notice.

The charge happens before the allocation. Memory::trackMemory is the first call. If the memory tracker refuses, Memory::newNoExcept is never reached and no bytes are touched.

The rollback on failure. If the underlying allocator returns null, the charge is undone before the std::bad_alloc propagates. A tracker that charges for an allocation that never happened is a leak that never gets collected.

Memory::newNoExcept is a raw allocator call, deliberately bypassing the tracking wrapper in malloc.cpp:

 cpp
 
 
// src/Common/AllocationInterceptors.h:12
/// These macros provide direct access to the underlying allocator,
/// bypassing the tracking wrappers in malloc.cpp.
/// Use them in code that does its own memory tracking (e.g. Allocator, AllocatorWithMemoryTracking)
/// to avoid double-counting.

This is the double-counting problem, and it is worth dwelling on. ClickHouse interposes malloc at the libc level. If operator new called malloc, the bytes would be charged twice — once by operator new’s own trackMemory, and once by the interposed malloc. So operator new calls je_malloc directly:

 cpp
 
 
// src/Common/AllocationInterceptors.h:21
#define __real_malloc(size) je_malloc(size)
#define __real_calloc(nmemb, size) je_calloc(nmemb, size)
#define __real_realloc(ptr, size) je_realloc(ptr, size)
#define __real_posix_memalign(memptr, alignment, size) je_posix_memalign(memptr, alignment, size)
#define __real_aligned_alloc(alignment, size) je_aligned_alloc(alignment, size)
#define __real_free je_free

The same __real_* escape hatch is what makes the scheme work under sanitizers, which have their own malloc interception.

The size problem, part one: charge the real size class

Memory::trackMemory does not charge the requested size. It charges what the allocator will actually consume:

 cpp
 
 
// src/Common/memory.h:92
template <std::same_as<std::align_val_t>... TAlign>
requires DB::OptionalArgument<TAlign...>
inline ALWAYS_INLINE size_t getActualAllocationSize(size_t size, TAlign... align [[maybe_unused]])
{
 size_t actual_size = size;

#if USE_JEMALLOC
 /// The nallocx() function allocates no memory, but it performs the same size computation as the mallocx() function
 /// @note je_mallocx() != je_malloc(). It's expected they don't differ much in allocation logic.
 size_t size_for_nallocx = size;
 if (size_for_nallocx == 0) [[unlikely]]
 size_for_nallocx = 1;

 if constexpr (sizeof...(TAlign) == 1)
 {
 actual_size = je_nallocx(size_for_nallocx, MALLOCX_ALIGN(alignToSizeT(align...)));
 }
 else
 {
 actual_size = je_nallocx(size_for_nallocx, 0);
 }
#endif

 return actual_size;
}

je_nallocx computes jemalloc’s size class for a request without allocating anything. So the tracker is charged the number of bytes that will actually be consumed — 112 for a malloc(100), 8192 for a malloc(5000), and so on — and the accounting matches what the process really spends.

This is why jemalloc is not optional for the accounting to be exact. Without it, actual_size is just size, and the tracked total systematically under-counts by the size-class rounding.

Note the size_for_nallocx == 0 special case: a zero-byte allocation is legal in C, and jemalloc’s nallocx would misbehave on it.

The size problem, part two: the free side

The free side is where the size is genuinely unknown. There are three solutions in the tree, and they are used in different places.

Solution A: pass the size explicitly

ClickHouse’s own Allocator does not have a free(void*). It has a free(void*, size_t):

 cpp
 
 
// src/Common/Allocator.h:49
 /// Allocate memory range.
 void * alloc(size_t size, size_t alignment = 0);

 /// Free memory range.
 void free(void * buf, size_t size, size_t alignment = 0);

 /** Enlarge memory range.
 * Data from old range is moved to the beginning of new range.
 * Address of memory range could change.
 */
 void * realloc(void * buf, size_t old_size, size_t new_size, size_t alignment = 0);

The header calls this out as a deliberate difference from std::allocator:

 cpp
 
 
// src/Common/Allocator.h:34
 * The interface is different from std::allocator
 * - the presence of the method realloc, which for large chunks of memory uses mremap;
 * - passing the size into the `free` method;
 * - by the presence of the `alignment` argument;
 * - the possibility of zeroing memory (used in hash tables);

The free implementation is correspondingly explicit:

 cpp
 
 
// src/Common/Allocator.cpp:143
template <bool clear_memory_, bool populate>
void Allocator<clear_memory_, populate>::free(void * buf, size_t size, size_t /* alignment */)
{
 try
 {
 checkSize(size);
 freeImpl(buf);
 auto trace = CurrentMemoryTracker::free(size);
 trace.onFree(buf, size);
 }
 catch (...)
 {
 DB::tryLogCurrentException("Allocator::free");
 throw;
 }
}

Note that freeImpl is called before the tracker is credited back. If the tracker somehow threw, the memory would already be gone — and free is documented as not throwing in practice, but the try/catch is there to make sure a logging failure does not turn into a std::terminate.

AllocatorWithMemoryTracking does the same for the standard containers, with the element count times the element size:

 cpp
 
 
// src/Common/AllocatorWithMemoryTracking.h:61
 [[nodiscard]] T * allocate(size_t n)
 {
 if (n > std::numeric_limits<size_t>::max() / sizeof(T)) [[unlikely]]
 throwBadAllocFromAllocatorWithMemoryTracking();

 constexpr size_t alignment = alignof(T) > alignof(std::max_align_t) ? alignof(T) : 0;
 return static_cast<T *>(allocateWithMemoryTracking(n * sizeof(T), alignment));
 }

 void deallocate(T * p, size_t n) noexcept
 {
 deallocateWithMemoryTracking(p, n * sizeof(T));
 }

The overflow check on the multiply is the kind of thing that only matters once every hour, but when it matters it matters.

Solution B: the sized `operator delete`

C++17 added sized deallocation:

 cpp
 
 
// src/Common/AllocationInterceptors.cpp:209
void operator delete(void * ptr, std::size_t size) noexcept
{
 AllocationTrace trace;
 std::size_t actual_size = Memory::untrackMemory(ptr, trace, size);
 trace.onFree(ptr, actual_size);
 Memory::deleteSized(ptr, size);
}

and ClickHouse defines both the sized and the unsized variants, as the standard requires:

 cpp
 
 
// src/Common/AllocationInterceptors.cpp:168
/// C++17 std 21.6.2.1 (11)
/// If a function without a size parameter is defined, the program should also define the corresponding function with a size parameter.
/// If a function with a size parameter is defined, the program shall also define the corresponding version without the size parameter.
///
/// cppreference:
/// It's unspecified whether size-aware or size-unaware version is called when deleting objects of
/// incomplete type and arrays of non-class and trivially-destructible class types.

The comment is a genuine portability hazard that ClickHouse had to think about: for some types the compiler may call either overload, so both must work.

Solution C: ask jemalloc

The unsized operator delete(void*) has no size at all. jemalloc does: given a pointer it owns, je_sallocx reports the usable size.

 cpp
 
 
// src/Common/memory.h:153
template <std::same_as<std::align_val_t>... TAlign>
requires DB::OptionalArgument<TAlign...>
inline ALWAYS_INLINE size_t untrackMemory(void * ptr [[maybe_unused]], AllocationTrace & trace, std::size_t size [[maybe_unused]] = 0, TAlign... align [[maybe_unused]]) noexcept
{
 std::size_t actual_size = 0;

 try
 {
#if USE_JEMALLOC

 /// @note It's also possible to use je_malloc_usable_size() here.
 if (ptr != nullptr) [[likely]]
 {
 if constexpr (sizeof...(TAlign) == 1)
 actual_size = je_sallocx(ptr, MALLOCX_ALIGN(alignToSizeT(align...)));
 else
 actual_size = je_sallocx(ptr, 0);
 }
#else
 if (size)
 actual_size = size;
# if defined(_GNU_SOURCE)
 /// It's innaccurate resource free for sanitizers. malloc_usable_size() result is greater or equal to allocated size.
 else
 actual_size = malloc_usable_size(ptr);
# elif defined(OS_DARWIN)
 else
 actual_size = malloc_size(ptr);
# endif
#endif
 trace = CurrentMemoryTracker::free(actual_size);
 }
 catch (...) // NOLINT(bugprone-empty-catch) Ok: operator delete must not throw
 {
 }

 return actual_size;
}

This function is the linchpin of the whole scheme, and it has three properties worth naming:

It is noexcept. operator delete must not throw, and the empty catch is the enforcement. If the tracker threw here, the program’s delete would throw, and the accounting would be broken anyway.
It falls back to malloc_usable_size when the size is not passed. That path is marked as inaccurate for sanitizers, and the comment says so.
It uses je_sallocx, not je_malloc_usable_size. The comment notes the alternative exists but does not use it — sallocx is the cheaper size-class query.

Layer 2: the libc symbols

Interposing operator new only covers C++ code that goes through new. Anything that calls malloc directly — a C library, a third-party dependency, a std::vector inside a library that was compiled elsewhere — is invisible.

ClickHouse therefore interposes the libc allocation symbols themselves. On Linux and FreeBSD, the whole malloc family is replaced:

 cpp
 
 
// src/Common/malloc.cpp:30
void * malloc(size_t size)
{
 AllocationTrace trace;
 size_t actual_size = Memory::trackMemoryFromC(size, trace);
 void * ptr = je_malloc(size);
 if (unlikely(!ptr))
 {
 trace = CurrentMemoryTracker::free(actual_size);
 return nullptr;
 }
 trace.onAlloc(ptr, actual_size);
 return ptr;
}

The file’s header comment explains the mechanism and, importantly, why it changed:

 cpp
 
 
// src/Common/malloc.cpp:14
/// Define standard allocation functions that perform memory tracking
/// and delegate to jemalloc via je_* prefixed functions.
///
/// This replaces the previous --wrap linker approach with direct symbol
/// interposition: malloc.cpp compiles to a dedicated object (clickhouse_malloc.o)
/// that appears before libjemalloc.a in the link order.
///
/// jemalloc with je_ prefix does not export valloc/memalign/pvalloc,
/// so we implement them via je_posix_memalign / je_aligned_alloc.

The --wrap linker approach used to be the way to do this. Direct symbol interposition is simpler: define malloc in an object file that comes first in the link order, and the dynamic linker binds every reference to it.

There is a second layer of trickiness on glibc: glibc’s internal calls to malloc do not go through the public symbol, so they would escape interception. ClickHouse aliases them:

 cpp
 
 
// src/Common/malloc.cpp:243
/// On Linux (non-musl), glibc internally calls __libc_malloc etc.
/// We provide aliases so those internal calls also go through our wrappers.
#if !defined(USE_MUSL) && defined(OS_LINUX)

void * __libc_malloc(size_t size) __attribute__((alias("malloc"))); // NOLINT(bugprone-reserved-identifier,cert-dcl37-c,cert-dcl51-cpp)
void __libc_free(void * ptr) __attribute__((alias("free"))); // NOLINT(bugprone-reserved-identifier,cert-dcl37-c,cert-dcl51-cpp)
void * __libc_calloc(size_t nmemb, size_t size) __attribute__((alias("calloc"))); // NOLINT(bugprone-reserved-identifier,cert-dcl37-c,cert-dcl51-cpp)
void * __libc_realloc(void * ptr, size_t size) __attribute__((alias("realloc"))); // NOLINT(bugprone-reserved-identifier,cert-dcl37-c,cert-dcl51-cpp)
void * __libc_memalign(size_t alignment, size_t size) __attribute__((alias("memalign"))); // NOLINT(bugprone-reserved-identifier,cert-dcl37-c,cert-dcl51-cpp)
void * __libc_valloc(size_t size) __attribute__((alias("valloc"))); // NOLINT(bugprone-reserved-identifier,cert-dcl37-c,cert-dcl51-cpp)
void * __libc_pvalloc(size_t size) __attribute__((alias("pvalloc"))); // NOLINT(bugprone-reserved-identifier,cert-dcl37-c,cert-dcl51-cpp)

The whole family is covered: malloc, free, calloc, realloc, posix_memalign, aligned_alloc, valloc, memalign, pvalloc and reallocarray.

The C API cannot throw

The libc-level wrappers use a different tracking entry point:

 cpp
 
 
// src/Common/memory.h:142
/// We cannot throw from C API
template <std::same_as<std::align_val_t>... TAlign>
requires DB::OptionalArgument<TAlign...>
inline ALWAYS_INLINE size_t trackMemoryFromC(size_t size, AllocationTrace & trace, TAlign... align)
{
 [[maybe_unused]] MemoryTrackerUntrackedAllocationsBlockerInThread blocker;
 std::size_t actual_size = getActualAllocationSize(size, align...);
 trace = CurrentMemoryTracker::allocNoThrow(actual_size);
 return actual_size;
}

Two differences from the C++ path:

allocNoThrow instead of allocThrow — the tracker may count the allocation but must not refuse it, because a C library’s malloc has no way to report “out of memory” other than returning null.
A MemoryTrackerUntrackedAllocationsBlockerInThread scope. That blocker suppresses the MemoryAllocatedWithoutCheck telemetry, which would otherwise report every C-library allocation as an unchecked one.

`realloc` is the interesting one

realloc is where accounting usually breaks, because the old block may be kept (in place) or freed and a new one allocated. ClickHouse handles it by charging the new size first and freeing the old size after:

 cpp
 
 
// src/Common/malloc.cpp:73
void * realloc(void * ptr, size_t size)
{
 size_t old_actual_size = 0;
 if (ptr)
 old_actual_size = je_sallocx(ptr, 0);

 AllocationTrace trace;
 size_t actual_size = Memory::trackMemoryFromC(size, trace);
 void * res = je_realloc(ptr, size);
 if (unlikely(!res))
 {
 trace = CurrentMemoryTracker::free(actual_size);
 /// With jemalloc opt.zero_realloc:free (the default), realloc(ptr, 0)
 /// frees ptr and returns nullptr. That is not a failure — untrack the old allocation.
 if (size == 0 && ptr)
 {
 AllocationTrace free_trace = CurrentMemoryTracker::free(old_actual_size);
 free_trace.onFree(ptr, old_actual_size);
 }
 return nullptr;
 }

 if (ptr)
 {
 AllocationTrace free_trace = CurrentMemoryTracker::free(old_actual_size);
 free_trace.onFree(ptr, old_actual_size);
 }
 trace.onAlloc(res, actual_size);
 return res;
}

The size == 0 special case is a genuine jemalloc subtlety, documented in the comment: with the default opt.zero_realloc:free, realloc(ptr, 0) frees the pointer and returns null. That is not a failure, so the old allocation must be untracked even though the function “failed”.

And ClickHouse’s own Allocator::realloc takes the other approach — charge the whole new size up front, because it does not know which of the two behaviours the underlying realloc will pick:

 cpp
 
 
// src/Common/Allocator.cpp:172
 if (alignment <= MALLOC_MIN_ALIGNMENT) [[likely]]
 {
 /// Resize malloc'd memory region with no special alignment requirement.
 /// Realloc can do 2 possible things:
 /// - expand existing memory region
 /// - allocate new memory block and free the old one
 /// Because we don't know which option will be picked we need to make sure there is enough
 /// memory for all options
 auto trace_alloc = CurrentMemoryTracker::alloc(new_size);

Charging new_size up front and then crediting old_size back means the tracker briefly holds old_size + new_size bytes for one buffer. That is deliberate: it guarantees the limit is never violated by the realloc itself, at the cost of a transient over-count.

Layer 3: macOS

On Linux the symbol interposition works. On macOS it does not, and the reason is structural:

 cpp
 
 
// src/Common/malloc.cpp:278
/// macOS counterpart of the symbol interposition above.
///
/// On Linux/FreeBSD we override the libc allocation symbols so every raw malloc/free is tracked.
/// That does not work on Mach-O: `free` is dispatched through the "malloc zone" that owns the
/// pointer, so overriding the `free` symbol with one that calls `je_free` would corrupt pointers
/// belonging to other zones, and dyld interposition cannot reach the malloc calls libSystem makes
/// internally. Instead we lean on the fact that jemalloc installs itself as the process' default
/// malloc zone (contrib/jemalloc/src/zone.c): every allocation and free flows through that zone's
/// callbacks, so wrapping them tracks the whole process while staying zone-consistent.
///
/// `initializeJemallocZoneMemoryTracking` must run once, after jemalloc has registered its zone.

So on macOS, ClickHouse replaces the zone callbacks instead:

 cpp
 
 
// src/Common/malloc.cpp:350
static void * trackedZoneMalloc(malloc_zone_t * zone, size_t size)
{
 ReentrancyGuard guard;
 if (!guard.engaged) [[unlikely]]
 return saved_zone.malloc(zone, size);

 AllocationTrace trace;
 size_t actual_size = Memory::trackMemoryFromC(size, trace);
 void * ptr = saved_zone.malloc(zone, size);
 if (ptr == nullptr) [[unlikely]]
 {
 [[maybe_unused]] auto rollback_trace = CurrentMemoryTracker::free(actual_size);
 return nullptr;
 }
 trace.onAlloc(ptr, actual_size);
 return ptr;
}

The ReentrancyGuard exists because of a macOS-specific bootstrapping problem:

 cpp
 
 
// src/Common/malloc.cpp:301
/// Reentrancy guard. macOS allocates thread-local storage lazily, on first access per thread, by
/// calling malloc. Our tracking touches thread-locals (CurrentMemoryTracker), so the first
/// allocation on a new thread would recurse: malloc -> track -> TLV init -> malloc -> ... until the
/// stack overflows. We detect the reentry with a pthread key, whose thread-specific slot lives in a
/// preallocated array and never itself calls malloc, and pass such allocations straight through
/// untracked. This is macOS-specific: on Linux TLS is set up at thread creation, not lazily.

And the guard’s cost is documented honestly, including the drift it causes:

 cpp
 
 
// src/Common/malloc.cpp:303
/// Known limitation: the pass-through allocation (the thread's ~40 KiB dyld TLV block) is charged
/// to no tracker, but its eventual free runs the normal tracked path and subtracts it. Charging it
/// is impossible - MemoryTracker::allocImpl itself reads thread-locals, so it would re-enter the
/// same lazy TLV init. The effect is one ~40 KiB block per thread: a static under-count while the
/// thread lives, and a ~40 KiB downward drift of the *global* tracker per completed thread
/// create->destroy cycle. It does not affect per-query max_memory_usage (TLV frees happen at thread
/// teardown, charged to the global tracker, not a query's). Measured on a server that spun up ~865
/// threads: ~35 MiB total, and no drift was realized under 1000-connection churn because pooled
/// threads persist. Accepted as a bounded, macOS-only, dev-platform limitation rather than paying
/// a per-free pointer-set lookup on the allocation hot path.

That is a bug report, a root-cause analysis, a measurement, and a decision to accept the trade — in one comment. The “~40 KiB downward drift of the global tracker per completed thread create->destroy cycle” is a real accounting error, on a real platform, and the comment says so.

Throwing or not

Not every interception point is allowed to refuse an allocation. The C++ operator new can, but only above a size threshold, and only if the server opted in:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:133
AllocationTrace CurrentMemoryTracker::allocThrow(Int64 size)
{
 const bool enforce_memory_limit = static_cast<UInt64>(size) >= min_allocation_size_to_throw_on_memory_limit.load(std::memory_order_relaxed);
 return allocImpl(size, enforce_memory_limit);
}

The threshold defaults to disabled, encoded as UINT64_MAX so the hot path is a single comparison:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:18
namespace
{
 std::atomic<UInt64> min_allocation_size_to_throw_on_memory_limit{std::numeric_limits<UInt64>::max()};
}

and the setter maps the user-facing 0 onto that sentinel:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:139
void CurrentMemoryTracker::setMinAllocationSizeBytesToThrow(UInt64 value)
{
 min_allocation_size_to_throw_on_memory_limit.store(
 value == 0 ? std::numeric_limits<UInt64>::max() : value,
 std::memory_order_relaxed);
}

The server setting explains why the default is “never”:

 cpp
 
 
// src/Core/ServerSettings.cpp:438
 DECLARE(UInt64, min_allocation_size_to_throw_on_memory_limit, 0, R"(
 Minimum size, in bytes, of a generic C++ allocation (the kind made by standard containers, strings, `std::vector` growth, smart pointers, etc.) that is allowed to raise `MEMORY_LIMIT_EXCEEDED` once `max_server_memory_usage` is reached. Smaller generic allocations are still counted against the memory tracker but are allowed to succeed even past the limit, which reduces spurious failures during cleanup and exception-handling paths near OOM.

 Allocations made by ClickHouse's own large buffers — column data, hash tables, arenas, query pipelines, IO buffers — always honour the memory limit and may throw regardless of this value. This setting only affects implicit allocations that flow through `operator new`.

 A value of `0` (default) preserves the legacy behaviour: implicit `operator new` allocations are never refused by the memory tracker, and only explicit ClickHouse allocations can raise `MEMORY_LIMIT_EXCEEDED`.

 Note, to avoid side effects it is recommended to set value greater then `max_untracked_memory`.
)", 0) \

So the split is:

Allocation site	Throws on limit?
`operator new` (containers, strings, `std::vector`)	Only if `min_allocation_size_to_throw_on_memory_limit` is set and the size is above it
libc `malloc` family	Never — the C API cannot report it
ClickHouse’s own `Allocator` (columns, hash tables, arenas)	Always

The reasoning is sound: refusing a two-byte std::string allocation during exception unwinding is how you turn a recoverable out-of-memory into a std::terminate. ClickHouse’s own big buffers are different — if a hash table cannot grow, the query has to fail now, cleanly, while it still holds a coherent state.

Sampling

Charging every allocation is cheap; tracing every allocation is not. The AllocationTrace returned by the tracker carries a probability, and the trace is only taken if the pointer hashes into the sample:

 cpp
 
 
// src/Common/MemoryTracker.cpp:101
bool shouldTrackAllocation(Float64 probability, void * ptr)
{
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wimplicit-const-int-float-conversion"
 return static_cast<double>(intHash64(uintptr_t(ptr))) < static_cast<double>(std::numeric_limits<uint64_t>::max()) * probability;
#pragma clang diagnostic pop
}

The hash is of the pointer, not of the size or the call site. That gives two useful properties: the same allocation site is sampled consistently across a run (so a hot site shows up reliably), and no shared state is needed to decide.

The AllocationTrace itself is designed so that the common case — not sampling — costs one branch:

 cpp
 
 
// src/Common/AllocationTrace.h:13
 ALWAYS_INLINE void onAlloc(void * ptr, size_t size) const
 {
 if (likely(sample_probability <= 0))
 return;

 onAllocImpl(ptr, size);
 }

Overflow guards

Two overflow guards protect the accounting from a size computation that has gone wrong.

The first is in the allocator:

 cpp
 
 
// src/Common/Allocator.cpp:122
void checkSize(size_t size)
{
 /// More obvious exception in case of possible overflow (instead of just "Cannot mmap").
 if (size >= MAX_ALLOCATION_SIZE) [[unlikely]]
 throw DB::Exception(DB::ErrorCodes::LOGICAL_ERROR, "Too large size ({}) passed to allocator. It indicates an error.", size);
}

with

 cpp
 
 
// src/Common/Allocator.h:24
/// The allocators refuse anything of this size or larger: such an allocation can never succeed,
/// and a size that large is a sign of an overflow in the computation of the size.
static constexpr size_t MAX_ALLOCATION_SIZE = 0x8000000000000000ULL;

0x8000000000000000 is 2^63 — the point where a signed 64-bit size would go negative. An allocation that large can never succeed, so refusing it with a clear message beats letting it turn into a nonsense size.

The second is in the tracker itself:

 cpp
 
 
// src/Common/MemoryTracker.cpp:349
 if (size < 0)
 throw DB::Exception(DB::ErrorCodes::LOGICAL_ERROR, "Negative size ({}) is passed to MemoryTracker. It is a bug.", size);

Prefaulting

One more thing the interception layer does: large allocations are pre-faulted, so that the pages exist before anyone touches them.

 cpp
 
 
// src/Common/Allocator.cpp:59
void prefaultPages([[maybe_unused]] void * buf_, [[maybe_unused]] size_t len_)
{
#if defined(MADV_POPULATE_WRITE)
 if (len_ < POPULATE_THRESHOLD)
 return;

 if (!is_supported_by_kernel) [[unlikely]]
 return;

 auto [buf, len] = adjustToPageSize(buf_, len_, ::getPageSize());
 ::madvise(buf, len, MADV_POPULATE_WRITE);
#endif
}

and the threshold is, in the source’s own words, arbitrary:

 cpp
 
 
// src/Common/Allocator.cpp:132
/// Constant is chosen almost arbitrarily, what I observed is 128KB is too small, 1MB is almost indistinguishable from 64MB and 1GB is too large.
extern const size_t POPULATE_THRESHOLD = std::max(Int64{16 * 1024 * 1024}, ::getPageSize());

16 MiB. The madvise is skipped on kernels older than 5.14, which do not have MADV_POPULATE_WRITE.

This is not memory tracking, but it happens in the same place, and it is worth knowing that the allocator is not just handing out addresses — for large buffers it is making sure the pages are real.

What it costs

Every allocation pays a tracker call. Not a sample, not a probability — a full walk up the four-level chain (mitigated by the per-thread batching, covered in the sibling article). The interception layer is the reason that batching exists.

je_nallocx and je_sallocx are not free. They are cheap size-class computations, but they are called on every allocation and every free respectively. Without jemalloc the fallback is malloc_usable_size, which the source describes as inaccurate under sanitizers.

The macOS path has a known drift. ~40 KiB per thread, documented above.

The operator new path can throw. Not by default, but if min_allocation_size_to_throw_on_memory_limit is set, a generic C++ allocation above the threshold can raise MEMORY_LIMIT_EXCEEDED from somewhere deep inside a library.

What to go do about it

The one thing to remember: the number ClickHouse reports is the size class the allocator actually consumed, not the size the code asked for. If you are comparing system.processes.memory_usage against the sum of your column sizes, expect the tracked number to be larger.

The part people get wrong: max_memory_usage does not bound every allocation on the query’s path. operator new allocations are counted but not refused by default, and libc allocations are counted but never refused. Only ClickHouse’s own big buffers are hard-bounded. If you want a hard bound on generic allocations too:

 sql
 
 
SELECT value FROM system.server_settings
WHERE name = 'min_allocation_size_to_throw_on_memory_limit';

Setting it to a value above max_untracked_memory is the recommendation in the setting’s own documentation, and it is worth following: below that threshold, the batching would flush a small allocation as part of a batch larger than the threshold, and the accounting would be confusing.

What to go do about it. If you suspect the tracked total and the real RSS disagree, compare them directly:

 sql
 
 
SELECT
 (SELECT value FROM system.metrics WHERE metric = 'MemoryTracking') AS tracked,
 (SELECT value FROM system.metrics WHERE metric = 'MemoryTrackingUncorrected') AS uncorrected;

MemoryTracking is the corrected value, MemoryTrackingUncorrected is the raw counter. A large gap between them and the process’s RSS is the accounting drift the sibling articles describe.

ClickHouse Charges Memory Through a Hierarchy of Trackers

Jay Hubbert — Sun, 25 Oct 2026 09:00:00 +0000

TL;DR: Every allocation in ClickHouse is charged to a chain of MemoryTracker objects: total_memory_tracker (global) → the user’s tracker → the query’s tracker → the thread’s tracker. Each link has its own hard limit, and the allocation is offered to each link in turn. A limit hit anywhere in the chain aborts the allocation with a MEMORY_LIMIT_EXCEEDED exception.

A columnar database that runs 200 concurrent queries on 64 cores has a bookkeeping problem that a row store does not. It has to answer four questions at the same time:

Is the server as a whole about to run out of RAM?
Is this user about to run out of RAM?
Is this one query about to run out of RAM?
And, at the lowest level, which thread should be charged for the bytes?

Answering them with four independent counters is the obvious approach and it is wrong, for two reasons. First, a query’s memory is not owned by one thread: a SELECT fans out across a thread pool, blocks are handed between threads, and the final aggregation happens on yet another one. Second, the four answers are not independent — a query’s limit is meaningless if the server’s limit is lower.

ClickHouse’s answer is a tree of counters with a parent pointer, where one allocation walks up the tree and is offered to every level.

The four levels

The levels are enumerated, and the enumeration order is the tree order — a lower value is closer to the root:

 cpp
 
 
// src/Common/VariableContext.h:7
enum class VariableContext : uint8_t
{
 Global = 0,
 User, /// Group of processes
 Process, /// For example, a query or a merge
 Thread, /// A thread of a process

 Max, /// size of the enum, not a real level
};

There are exactly two global-scope trackers, created at static-init time:

 cpp
 
 
// src/Common/MemoryTracker.cpp:160
MemoryTracker total_memory_tracker(nullptr, VariableContext::Global);
MemoryTracker background_memory_tracker(&total_memory_tracker, VariableContext::User, false);

total_memory_tracker is the server total: it has no parent, and its hard limit is the server’s max_server_memory_usage. background_memory_tracker is a User-level tracker for work that is not a user query — merges, mutations, and other background jobs:

 cpp
 
 
// src/Interpreters/ThreadStatusExt.cpp:275
ThreadGroupPtr ThreadGroup::createForMergeMutate(ContextPtr storage_context)
{
 const Int32 os_threads_nice_value = storage_context->getServerSettings()[ServerSetting::os_threads_nice_value_merge_mutate];
 auto group = create(storage_context, os_threads_nice_value);
 group->memory_tracker.setDescription("Background process (mutate/merge)");
 group->memory_tracker.setParent(&background_memory_tracker);
 return group;
}

Note the third constructor argument false on background_memory_tracker: background work does not get a “peak memory usage” log line in the destructor.

The per-user and per-query trackers are created at query start. The per-user one lives in ProcessListForUser:

 cpp
 
 
// src/Interpreters/ProcessList.h:350
 MemoryTracker user_memory_tracker{VariableContext::User};

and the per-query one lives in ThreadGroup, which is created per query:

 cpp
 
 
// src/Common/ThreadStatus.h:103
 MemoryTracker memory_tracker{VariableContext::Process};

The chain, at steady state, is:

 text
 
 
 total_memory_tracker VariableContext::Global
 hard limit: max_server_memory_usage
 ^ parent
 |
 user_memory_tracker VariableContext::User
 hard limit: max_memory_usage_for_user
 ^ parent
 |
 ThreadGroup::memory_tracker VariableContext::Process
 hard limit: max_memory_usage
 ^ parent
 |
 ThreadStatus::memory_tracker VariableContext::Thread
 no hard limit (see below)

A fourth level, Thread, exists per OS thread. It carries no limit, and the source is explicit about why:

 cpp
 
 
// src/Interpreters/ProcessList.cpp:385
 /// NOTE: Do not set the limit for thread-level memory tracker since it could show unreal values
 /// since allocation and deallocation could happen in different threads

That comment is the whole reason the tree is a tree and not a flat set of counters. A block allocated on thread A and freed on thread B would make a per-thread counter go negative. The thread tracker is therefore a routing node, not a limit node: it exists so that the chain has a leaf the thread’s allocations can be charged to, and so that the blocker’s level has something to compare against.

The parent pointer

Each tracker holds a single atomic pointer to its parent, and the header explains exactly why it is atomic and why the ordering matters:

 cpp
 
 
// src/Common/MemoryTracker.h:91
 /// Singly-linked list. All information will be passed to subsequent memory trackers also (it allows to implement trackers hierarchy).
 /// In terms of tree nodes it is the list of parents. Lifetime of these trackers should "include" lifetime of current tracker.
 /// Requires acquire-release:
 /// 1. Thread A constructs MemoryTracker object and attaches MemoryTracker pointer
 /// (e.g. ProcessList::insert where the user's MemoryTracker is constructed right before calling setParent).
 /// 2. Thread B traverses the chain and dereferences each pointer (e.g. another thread in thread group).
 /// 3. If Thread B sees a pointer, it should be guaranteed to see the object's memory without data races.
 /// Hence, we need the Thread A's pointer store to synchronize-with the Thread B's pointer load.
 std::atomic<MemoryTracker *> parent {};

setParent is the only place the link is written, and it does two things that are easy to get wrong: it flushes the thread’s deferred bytes before moving the pointer, and it attaches the child to the new parent before publishing the pointer, so a concurrent reader that sees the new parent also sees the child counted:

 cpp
 
 
// src/Common/MemoryTracker.cpp:1058
void MemoryTracker::setParent(MemoryTracker * elem)
{
 /// Untracked memory shouldn't be accounted to a query or a user if it was allocated before the thread was attached
 /// to a query thread group or a user group, because this memory will be (🤞) freed outside of these scopes.
 if (level == VariableContext::Thread && DB::current_thread)
 DB::current_thread->flushUntrackedMemory();

 /// Attach before publishing, so the new parent counts this child before any of its bytes arrive.
 if (elem && level == VariableContext::Process)
 elem->attachChild();
 auto * old_parent = parent.exchange(elem, std::memory_order_acq_rel);
 if (old_parent && level == VariableContext::Process)
 old_parent->detachChild();

The last four lines are the interesting ones. A Process-level tracker counts its children:

 cpp
 
 
// src/Common/MemoryTracker.cpp:737
void MemoryTracker::attachChild()
{
 if (level == VariableContext::User)
 children_count.fetch_add(1);
}

and when the last child goes away, whatever residual is left on the user is settled, so a user does not accumulate a permanent residue:

 cpp
 
 
// src/Common/MemoryTracker.cpp:744
void MemoryTracker::detachChild()
{
 if (level != VariableContext::User || children_count.fetch_sub(1) != 1)
 return;

 /// Read before checking again: a child attached meanwhile is either seen below or charges after the read.
 Int64 residual = amount.load();
 if (residual == 0 || children_count.load() != 0)
 return;

Walking the chain

The whole design reduces to one recursive function. MemoryTracker::allocImpl charges this level, then hands the same bytes to the parent:

 cpp
 
 
// src/Common/MemoryTracker.cpp:391
 Int64 will_be = size ? size + amount.fetch_add(size, std::memory_order_relaxed) : amount.load(std::memory_order_relaxed);
 Int64 will_be_rss = 0;
 if (level == VariableContext::Global)
 will_be_rss = size ? size + rss.fetch_add(size, std::memory_order_relaxed) : rss.load(std::memory_order_relaxed);

 Int64 current_hard_limit = hard_limit.load(std::memory_order_relaxed);
 bool memory_limit_exceeded_ignored = false;

The amount.fetch_add happens before the limit check, which means the counter is momentarily over the limit while the check runs. The comment above it says so explicitly:

 cpp
 
 
// src/Common/MemoryTracker.cpp:387
 /** Using memory_order_relaxed means that if allocations are done simultaneously,
 * we allow exception about memory limit exceeded to be thrown only on next allocation.
 * So, we allow over-allocations.
 */

This is a deliberate trade: relaxed ordering avoids the cost of an acquire-release fence on the hottest path in the server, at the price of letting several threads briefly overshoot the limit before any of them is refused. The overshoot is bounded by the number of threads racing, not by the size of the allocations.

After the check, the chain continues:

 cpp
 
 
// src/Common/MemoryTracker.cpp:574
 AllocationTrace allocation_trace(_sample_probability);
 if (auto * loaded_next = parent.load(std::memory_order_acquire))
 {
 MemoryTracker * tracker = level == VariableContext::Process ? this : query_tracker;
 try
 {
 allocation_trace = loaded_next->allocImpl(size, enforce_memory_limit, tracker, _sample_probability);
 }
 catch (...)
 {
 decrementLocalUsage(size);
 throw;
 }
 }

Note the catch: if the parent refuses the allocation, this level has already added the bytes to its own counter, so it must undo them before rethrowing. That single catch block is the difference between “the query hit its limit” and “the server now permanently believes the query is holding 50 GB”. The unit tests in src/Common/tests/gtest_memory_tracker.cpp exist specifically to pin this down — ParentLimitFailureRollsBackHierarchy, ProcessLimitFailureRollsBackDescendants, RepeatedParentLimitFailuresDoNotAccumulate.

Carrying the query identity down the chain

The query_tracker argument is a small but load-bearing trick. The Global level needs to know which query asked for the memory, because the overcommit mechanism has to pick a victim query to kill — and the global tracker has no idea which query that is. So the Process-level tracker passes itself down as the “query tracker” and the global level keeps it:

 cpp
 
 
// src/Common/MemoryTracker.cpp:377
 /// Since the MemoryTrackerBlockerInThread should respect the level, we should go to the next parent.
 if (auto * loaded_next = parent.load(std::memory_order_acquire))
 {
 MemoryTracker * tracker = level == VariableContext::Process ? this : query_tracker;
 return loaded_next->allocImpl(size, enforce_memory_limit, tracker, _sample_probability);
 }

The same expression appears twice — once in the blocked path above and once in the normal path. The invariant is: at the Global level, query_tracker is the innermost Process-level tracker that was on the chain, and at any other level it is passed through unchanged.

Where the limits come from

The limits are session settings, applied when the query is registered:

 cpp
 
 
// src/Interpreters/ProcessList.cpp:322
 /// Track memory usage for all simultaneously running queries from single user.
 if (starts_a_new_period)
 user_process_list.user_memory_tracker.setHardLimit(settings[Setting::max_memory_usage_for_user]);
 else
 user_process_list.user_memory_tracker.setOrRaiseHardLimit(settings[Setting::max_memory_usage_for_user]);
 user_process_list.user_memory_tracker.setSoftLimit(settings[Setting::memory_overcommit_ratio_denominator_for_user]);
 user_process_list.user_memory_tracker.setDescription("User");

and for the query itself:

 cpp
 
 
// src/Interpreters/ProcessList.cpp:359
 /// Set query-level memory trackers
 thread_group->memory_tracker.setOrRaiseHardLimit(settings[Setting::max_memory_usage]);
 configureMemoryTrackerFromSettings(query_context->hasTraceCollector(), thread_group->memory_tracker, settings);

Two different setters, and the difference is deliberate. setHardLimit replaces the limit outright; setOrRaiseHardLimit only ever raises it:

 cpp
 
 
// src/Common/MemoryTracker.cpp:1034
void MemoryTracker::setOrRaiseHardLimit(Int64 value)
{
 /// This is just atomic set to maximum.
 Int64 old_value = hard_limit.load(std::memory_order_relaxed);
 while ((value == 0 || old_value < value) && !hard_limit.compare_exchange_weak(old_value, value))
 ;
}

The value == 0 case is the interesting one: zero means “unlimited”, and “unlimited” is encoded as raise to infinity. So a user who runs one query with max_memory_usage_for_user = 10 GiB and then a second with max_memory_usage_for_user = 0 does not get an unlimited user — the user stays capped at 10 GiB. The starts_a_new_period flag above decides when the cap is reset: a “period” ends when the user’s last query leaves the list.

The setSoftLimit line is the overcommit knob. memory_overcommit_ratio_denominator defaults to 1 GiB:

 cpp
 
 
// src/Core/Settings.cpp:4623
 DECLARE(UInt64, memory_overcommit_ratio_denominator, 1_GiB, R"(
 It represents the soft memory limit when the hard limit is reached on the global level.
 This value is used to compute the overcommit ratio for the query.
 Zero means skip the query.
 Read more about [memory overcommit](/concepts/features/configuration/settings/memory-overcommit).
)", 0, \
 {"22.5", 0, 1073741824, "Enable memory overcommit feature by default"}) \

The soft limit is not a cap. It is the denominator of the ratio used to rank queries when the server has to choose one to kill.

Where the thread tracker joins

A thread attaches to a query’s group when the group is created, and the thread’s tracker is reparented at that moment:

 cpp
 
 
// src/Interpreters/ThreadStatusExt.cpp:446
 /// Reparenting the memory tracker flushes the untracked balance the thread carried in, so the
 /// counters must be reparented after it, or those bytes are reported as this group's.
 memory_tracker.setParent(&thread_group->memory_tracker);
 performance_counters.setParent(&thread_group->performance_counters);

The comment records the ordering constraint precisely: the reparenting flushes the thread’s deferred (untracked) balance, and that flush has to land on the new parent, so the tracker must be reparented before the performance counters are. The inverse ordering would report the flushed bytes against the wrong scope.

When the thread detaches — a query finishing, or a pool thread returning to the pool — the tracker goes back to the global tracker and the deferred bytes are flushed one last time:

 cpp
 
 
// src/Interpreters/ThreadStatusExt.cpp:481
void ThreadStatus::detachFromGroup()
{
 if (!thread_group)
 return;

 LockMemoryExceptionInThread lock_memory_tracker(VariableContext::Global);

 /// flush untracked memory before resetting memory_tracker parent
 flushUntrackedMemory();

The LockMemoryExceptionInThread at the top matters: detaching happens during stack unwinding when a query is cancelled, and an exception thrown from a destructor during unwinding is std::terminate. The lock is the mechanism that says “no memory-limit exceptions from here on”.

The free path

Freeing is not symmetric with allocating, and the difference is where most of the subtle bugs live. MemoryTracker::free takes the bytes off this level first, and then only passes on what was actually there:

 cpp
 
 
// src/Common/MemoryTracker.cpp:890
AllocationTrace MemoryTracker::free(Int64 size, double _sample_probability)
{
 if (_sample_probability < 0)
 _sample_probability = sample_probability.load(std::memory_order_relaxed);

 if (!isSizeOkForSampling(size))
 _sample_probability = 0;

 if (MemoryTrackerBlockerInThread::isBlocked(level))
 {
 if (level == VariableContext::Global)
 {
 /// For global memory tracker always update memory usage.
 amount.fetch_sub(size, std::memory_order_relaxed);
 rss.fetch_sub(size, std::memory_order_relaxed);
 auto metric_loaded = metric.load(std::memory_order_relaxed);
 if (metric_loaded != CurrentMetrics::end())
 CurrentMetrics::sub(metric_loaded, size);
 }

The removal itself saturates at zero for the User and Process levels, because a query may free memory it never held (a cache eviction, a pool object freed by a different query’s thread):

 cpp
 
 
// src/Common/MemoryTracker.cpp:607
 else
 {
 /// A query may free memory it never held (e.g. a cache eviction): take only what is there, in one
 /// compare-exchange so that concurrent frees cannot together remove more than is there.
 Int64 current = amount.load(std::memory_order_relaxed);
 do
 {
 accounted_size = std::min(size, std::max<Int64>(current, 0));
 }
 while (!amount.compare_exchange_weak(current, current - accounted_size, std::memory_order_relaxed));
 }

and the “freed more than we held” residue is not silently dropped — it goes to the server total, and the deficit is remembered:

 cpp
 
 
// src/Common/MemoryTracker.cpp:924
 /// A query freeing what it never held does not free it for its user either; only the server total holds it.
 if (accounted_size < size && loaded_next->level == VariableContext::User)
 {
 std::ignore = total_memory_tracker.free(size - accounted_size, _sample_probability);
#ifdef DEBUG_OR_SANITIZER_BUILD
 freed_beyond_held.fetch_add(size - accounted_size, std::memory_order_relaxed);
#endif
 size = accounted_size;
 }

The freed_beyond_held counter exists only in debug builds, and it is there to explain a specific false positive: thread A of a query allocates 2 MiB that stays in its untracked (deferred) buffer, and thread B frees that block. The free reaches the server total first, so the query now holds 2 MiB less than it should, and when A finally flushes, the query is charged for a block that is already gone. That shows up as drift at the end of the query. The header documents it in full:

 cpp
 
 
// src/Common/MemoryTracker.h:158
 /// What a query root freed beyond what it held, which went straight to the server total (see `free`). For example,
 /// thread A of a query allocates a 2 MiB block that stays in its untracked memory, below `max_untracked_memory`, and
 /// thread B frees the block and flushes: the query holds less than 2 MiB, so the rest goes here. When A flushes later,
 /// the query is charged for a block that is already freed, which shows as drift at the end of the query but is not.
 std::atomic<Int64> freed_beyond_held = 0;

Settling what a query leaves behind

When a query’s tracker is destroyed, whatever it still holds is taken off its user — so that a user does not end up charged for a query’s leftovers:

 cpp
 
 
// src/Common/MemoryTracker.cpp:697
void MemoryTracker::settleDriftOnQueryEnd()
{
 Int64 drift = amount.load(std::memory_order_relaxed);
 if (drift == 0)
 return;

 /// Nested groups (a view, a merge inside `OPTIMIZE`) leave their drift to the query above them.
 auto * user_tracker = parent.load(std::memory_order_relaxed);
 if (!user_tracker || user_tracker->level != VariableContext::User)
 return;

 /// Another query of the same user may be releasing the same bytes right now, so count what was actually removed.
 Int64 settled = user_tracker->adjustLocally(drift);
 adjustLocally(drift);
 if (settled == 0)
 return;

 ProfileEvents::increment(ProfileEvents::QueryMemoryDriftSettled);
 ProfileEvents::increment(ProfileEvents::QueryMemoryDriftSettledBytes, std::abs(settled));
}

The “nested groups” line is a subtlety worth pausing on. A SELECT from a view, or an OPTIMIZE that runs a merge, creates a nested thread group. The nested group’s tracker is not attached to the user — it charges through the group above it, so the user is not billed twice:

 cpp
 
 
// src/Interpreters/ProcessList.cpp:337
 /// A nested group (a view, a flush run by `SYSTEM FLUSH ASYNC INSERT QUEUE`, a dictionary load) is
 /// charged through the group above it, and would be charged to the user twice.
 if (!thread_group->isNested())
 {
 thread_group->memory_tracker.reparent(&user_process_list.user_memory_tracker);
 thread_group->memory_pressure_monitor.setParent(user_process_list.user_memory_pressure_monitor);
 }

isNested() is just “does this group have a parent group”:

 cpp
 
 
// src/Common/ThreadStatus.h:157
 bool isNested() const { return parent != nullptr; }

And the settleDriftOnQueryEnd guard user_tracker->level != VariableContext::User is what implements that: a nested group’s parent is a Process-level tracker, not a User-level one, so the nested group returns early and leaves its drift to the query above it, which does have a User parent and does settle.

The hot fields

The tracker’s own layout is hand-tuned, because amount is written on every allocation and every free:

 cpp
 
 
// src/Common/MemoryTracker.h:72
 /// Hot: every field touched on the alloc/free path. Packed into a single cache line
 /// (24B writes + 40B reads = 64B exactly) so each allocation needs at most one line
 /// in flight. The line bounces among writer cores either way; co-locating the
 /// read-mostly limits/parent/metric here means writers get them for free with the
 /// same fetch. The only cost is reader-only cores re-fetching when allocator writes
 /// invalidate, but allocations dominate over pure reads in practice.
 struct
 {
 /// Hot writes: updated on every alloc/free flush.
 alignas(DB::CH_CACHE_LINE_SIZE) std::atomic<Int64> amount {0};
 std::atomic<Int64> peak {0};
 std::atomic<Int64> rss {0};

Twenty-four bytes of writes (three Int64 counters) plus forty bytes of reads (three limits, the parent pointer, the metric) fills a 64-byte cache line exactly. The whole struct is aligned to the cache line:

 cpp
 
 
// src/Common/MemoryTracker.h:421
static_assert(std::alignment_of_v<MemoryTracker> == DB::CH_CACHE_LINE_SIZE);

and the “cold” fields — the description string, the fault-injection probability, the sampling configuration — live in a second cache line that is never touched on the alloc/free path.

Metrics and observability

A tracker can be attached to a CurrentMetrics counter, which is how background work reports itself:

 cpp
 
 
// src/Common/MemoryTracker.h:337
 /// The memory consumption could be shown in realtime via CurrentMetrics counter
 void setMetric(CurrentMetrics::Metric metric_)
 {
 metric.store(metric_, std::memory_order_relaxed);
 }

The server total is not a metric but a tracker, and it is reported:

 cpp
 
 
// src/Common/CurrentMetrics.cpp:75
 M(MemoryTracking, "Total amount of memory (bytes) tracked by the server. By default the background memory worker corrects this value on every tick from an external measurement of the memory the process really uses (see the `memory_worker_correct_memory_tracker` server setting), so it follows the measured usage rather than a plain count of allocations. See `MemoryTrackingUncorrected` for the value with no corrections applied.") \

The distinction matters: MemoryTracking is corrected against the real RSS by the background memory worker, while MemoryTrackingUncorrected is the raw counter of the tree. The gap between them is the accounting drift plus the memory the process uses that never went through the counter at all — the pages the allocator retains.

Per-query usage is visible in the process list and the query log:

 sql
 
 
SELECT query_id, memory_usage, peak_memory_usage
FROM system.processes
WHERE memory_usage > 0
ORDER BY memory_usage DESC;

 sql
 
 
SELECT
 query_id,
 formatReadableSize(memory_usage) AS used,
 formatReadableSize(peak_memory_usage) AS peak,
 exception
FROM system.query_log
WHERE type = 'QueryFinish' AND memory_usage > 0
ORDER BY peak_memory_usage DESC
LIMIT 10;

What the hierarchy buys and what it costs

What it buys. One allocation is checked against four independent budgets without any of them knowing about the others. The user’s limit works even when the user’s queries run on 32 different threads. The server’s limit works even when the server has 200 users. And because the check happens before the bytes are handed to the allocator, an over-budget allocation costs nothing but the counter update.

What it costs.

The chain walk is a pointer chase on the hottest path in the server. Four levels means four atomic loads, four fetch_adds, four limit checks, and four fetch_subs on the free side — per allocation. This is why the thread level batches allocations instead of charging them one by one, and why the whole tracker struct is packed into a single cache line.
The counters can overshoot. relaxed ordering plus the add-then-check order means the reported usage can exceed the limit by whatever several racing threads allocated. The tests acknowledge this with a tolerance rather than an equality.

The tree is mutable at run time. reparent moves a tracker and its bytes between subtrees, and it has to check the new ancestors’ limits before the move, because moving is not an allocation and would not go through the normal path:

 cpp
 
 
// src/Common/MemoryTracker.cpp:792
 /// Moving is not an allocation, so check the limits here, before anything changes.
 for (const auto * tracker = new_parent; size > 0 && tracker && tracker != common; tracker = tracker->parent.load(std::memory_order_relaxed))
 {
 Int64 limit = tracker->hard_limit.load(std::memory_order_relaxed);
 Int64 will_be = tracker->amount.load(std::memory_order_relaxed) + size;
 if (limit && will_be > limit)
 {
 const auto * description = tracker->description_ptr.load(std::memory_order_relaxed);
 throw DB::Exception(
 DB::ErrorCodes::MEMORY_LIMIT_EXCEEDED,
 "{}{}exceeded: would use {} (attempt to move {} already allocated), maximum: {}",

The Thread level is not a limit and the source says so. Any attempt to put one there produces numbers that “could show unreal values”.

The part people get wrong. A query that fails with MEMORY_LIMIT_EXCEEDED did not necessarily hit its own max_memory_usage. The chain is checked from the leaf up, and the global tracker checks both its own byte counter and the process RSS:

 cpp
 
 
// src/Common/MemoryTracker.cpp:427
 if (unlikely(
 current_hard_limit && (will_be > current_hard_limit || (level == VariableContext::Global && will_be_rss > current_hard_limit))))

So a server-level max_server_memory_usage can refuse an allocation that is perfectly within the query’s budget, and the message says which one it was.

What to go do about it. Read the exception text. The global-level message carries the RSS, the untracked-memory total and the overcommit decision; the query-level message carries only the numbers. The difference tells you whether you have a query problem or a server problem, and no amount of max_memory_usage tuning fixes the latter.

ClickHouse Batches Memory Accounting Per Thread

Jay Hubbert — Sun, 25 Oct 2026 09:00:00 +0000

TL;DR: Charging a shared counter on every malloc is a cache-line ping-pong across every core. So ClickHouse keeps a per-thread balance of untracked bytes, and only when that balance exceeds 4 MiB (or the per-CPU budget runs out) does it flush the whole batch to the tracker hierarchy in a single charge. The cost: the tracked total can lag reality by up to 4 MiB per thread.

Every allocation in ClickHouse goes through a chain of four memory trackers, and each one does an atomic read-modify-write on a shared counter. Multiply that by four levels and by the number of allocations a query makes — a columnar engine allocates a lot, because every column is a separate buffer and every hash table growth is a reallocation — and you get the classic scalability wall: the counter is in a cache line, every core wants to write it, and the line ping-pongs between cores at the rate of the allocation rate.

This is the same problem that malloc has with its own arena metadata, and the same answer: don’t share, batch locally, flush rarely.

The thread-local pointer

First, the plumbing. ClickHouse does not look up “the current tracker” through a registry or a map; it reads a single pointer out of thread-local storage. But it is not a plain thread_local — it is a fiber-local:

 cpp
 
 
// src/Common/CurrentThread.h:39
/**
 * We use **constinit** here to tell the compiler the current_thread variable is initialized.
 * If we didn't help the compiler, then it would most likely add a check before every use of the variable to initialize it if needed.
 * Instead it will trust that we are doing the right thing (and we do initialize it to nullptr) and emit more optimal code.
 * This is noticeable in functions like CurrentMemoryTracker::free and CurrentMemoryTracker::allocImpl
 * See also:
 * - https://en.cppreference.com/w/cpp/language/constinit
 * - https://github.com/ClickHouse/ClickHouse/pull/40078
 */
extern constinit FiberLocal<ThreadStatus *, FiberLocalSlot::CURRENT_THREAD> current_thread;

The constinit matters: without it the compiler would emit a guard variable check on every read of current_thread, and this is read on every allocation and every free. The comment says so explicitly.

FiberLocal is ClickHouse’s own thread-local storage abstraction, and it is not a wrapper around thread_local — on x86-64 it is a hand-written segment-relative load:

 cpp
 
 
// src/Common/FiberLocal.h:172
 template <size_t slot>
 static uintptr_t load() noexcept
 {
 static_assert(slot < slot_count);
 uintptr_t value = 0;
#if defined(__x86_64__) && defined(__ELF__)
 __asm__ __volatile__(
 "movq %%fs:FiberLocalStorageThreadStorage@tpoff+%c1, %0"
 : "=r"(value)
 : "i"(slot * sizeof(void *))
 : "memory");

One movq from the FS segment, no function call, no TLS lookup helper. The slots are an array of words, and the whole array is swapped on a fiber switch:

 cpp
 
 
// src/Common/FiberLocal.h:224
 alignas(DB::CH_CACHE_LINE_SIZE) std::array<uintptr_t, slot_count> slots = fiberLocalSlotDefaults();
 static_assert(
 sizeof(slots) <= 2 * DB::CH_CACHE_LINE_SIZE,
 "One slot is one word, and a context switch swaps the whole arena: keep it within two cache lines");

The tracker itself lives in the ThreadStatus object hanging off that pointer, together with the local balance:

 cpp
 
 
// src/Common/ThreadStatus.h:209
 MemoryTracker memory_tracker{VariableContext::Thread};
 /// Small amount of untracked memory (per thread atomic-less counter)
 UntrackedMemoryCounter untracked_memory;
 /// MemoryTrackerBlockerInThread state corresponding to untracked_memory.
 VariableContext untracked_memory_blocker_level = VariableContext::Max;
 /// Each thread could new/delete memory in range of (-untracked_memory_limit, untracked_memory_limit) without access to common counters.
 Int64 untracked_memory_limit = 4 * 1024 * 1024;
 /// Per-CPU untracked memory
 PerCPUMemoryThreadState per_cpu_untracked_memory;

Four things to notice:

untracked_memory is an UntrackedMemoryCounter — “per thread atomic-less counter”.
The limit is 4 MiB, and it is signed: the range is (-limit, +limit), so the counter tracks both deferred allocations and deferred frees.
untracked_memory_blocker_level remembers which blocker level the deferred bytes belong to.
There is a per-CPU state too, which is the second-level batching.

The atomic-less counter

The counter is deliberately not an atomic read-modify-write, because only one thread ever writes it:

 cpp
 
 
// src/Common/UntrackedMemoryRegistry.h:14
class ThreadLocalCounter
{
public:
 ThreadLocalCounter() = default;

 ThreadLocalCounter(const ThreadLocalCounter &) = delete;
 ThreadLocalCounter & operator=(ThreadLocalCounter &) = delete;
 ThreadLocalCounter(ThreadLocalCounter &&) = delete;
 ThreadLocalCounter & operator=(ThreadLocalCounter &&) = delete;

 ALWAYS_INLINE inline Int64 load() const noexcept { return value.load(std::memory_order_relaxed); }
 ALWAYS_INLINE inline void store(Int64 v) noexcept { value.store(v, std::memory_order_relaxed); }

 ALWAYS_INLINE inline Int64 add(Int64 delta) noexcept
 {
 /// Not a real atomic RMW for performance considerations.
 /// All writes (add, store) come from a single thread, so this is fine.
 const Int64 new_value = value.load(std::memory_order_relaxed) + delta;
 value.store(new_value, std::memory_order_relaxed);
 return new_value;
 }

private:
 std::atomic<Int64> value{0};
};

It is still an std::atomic<Int64> — so that a reader on another thread (e.g. the OOM message summing all threads) does not see a torn value — but the write path is a plain load, add and store. That is the entire point of the class: the write is thread-local, so it does not need to be atomic, but it is read cross-thread, so it must not be torn.

The counter is also registered in a global intrusive list, so the server can sum all of them:

 cpp
 
 
// src/Common/UntrackedMemoryRegistry.h:49
class UntrackedMemoryRegistry
{
public:
 static UntrackedMemoryRegistry & instance();

 void add(UntrackedMemoryCounter * counter);
 void remove(UntrackedMemoryCounter * counter);

 Int64 sum() const;

private:
 UntrackedMemoryRegistry() = default;

 mutable std::mutex mutex;
 boost::intrusive::list<UntrackedMemoryCounter> counters;
};

That sum() is what produces the Untracked memory across all threads: line in the server-level out-of-memory message. It is the one number that tells you whether the server’s tracked total is honest.

The allocation path

Now the actual batching. CurrentMemoryTracker::allocImpl is where the decision is made. First, the tracker is found:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:34
MemoryTracker * getMemoryTracker()
{
 if (auto * thread_memory_tracker = DB::CurrentThread::getMemoryTracker())
 return thread_memory_tracker;

 /// total_memory_tracker can be used before MainThreadStatus is initialized,
 /// but only after its own initialization and before teardown.
 if (DB::MainThreadStatus::initialized() || isTotalMemoryTrackerInitialized())
 return &total_memory_tracker;

 return nullptr;
}

Then the batch is updated, and only flushed if it has grown past the limit:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:83
 DB::PerCPUMemoryThreadState previous_per_cpu = cur_thread->per_cpu_untracked_memory;
 Int64 new_untracked_memory = cur_thread->untracked_memory.add(size);
 Int64 previous_untracked_memory = new_untracked_memory - size;

 /// Flush when the per-thread cap is hit, or when the per-CPU budget cannot cover the deferral.
 if (new_untracked_memory > cur_thread->untracked_memory_limit
 || !DB::per_cpu_memory.sync(new_untracked_memory, cur_thread->per_cpu_untracked_memory))
 {
 DB::per_cpu_memory.release(cur_thread->per_cpu_untracked_memory);

 cur_thread->untracked_memory.store(0);

 try
 {
 /// We cannot return the AllocationTrace from here, since its sample_probability was calculated on the (batched) flushed size, which may not match the original allocation size.
 if (new_untracked_memory > 0)
 std::ignore = memory_tracker->allocImpl(new_untracked_memory, enforce_memory_limit, /*query_tracker=*/ nullptr, /*_sample_probability=*/ 0.0);
 else
 std::ignore = memory_tracker->free(-new_untracked_memory, /*_sample_probability=*/ 0.0);
 }
 catch (...)
 {
 cur_thread->untracked_memory.add(previous_untracked_memory);
 DB::per_cpu_memory.rollback(cur_thread->per_cpu_untracked_memory, previous_per_cpu);
 throw;
 }
 }

Read it carefully — it is the whole mechanism in one block.

The fast path is the first two lines: untracked_memory.add(size). That is a load, an add, a store, on a word that lives in this thread’s own cache lines. No atomic RMW, no cache-line transfer, no contention with other cores. That is the entire optimization.

The flush condition is new_untracked_memory > untracked_memory_limit — the per-thread cap — or the per-CPU budget saying “no more deferral”.

The flush itself is a single memory_tracker->allocImpl(new_untracked_memory, ...). One call, one charge, one walk up the four-level chain — for 4 MiB worth of allocations. That is a 4 MiB amortisation factor on the entire tracker machinery.

The catch restores the previous balance if the flush throws. This is the case where the server limit is hit: the flush throws MEMORY_LIMIT_EXCEEDED, and the thread has to put its 4 MiB back so that a later flush does not double-count it. The previous_untracked_memory was captured on the way in precisely so this restore is exact.

The _sample_probability of 0.0 in the flush call is a comment in disguise. The AllocationTrace returned by allocImpl carries a sampling probability computed for the batched size, but the caller wanted a trace for the original allocation. Returning the batch’s trace would attribute a stack trace to the wrong allocation. So the flush discards its trace, and the caller computes its own:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:111
 return AllocationTrace(cur_thread->getEffectiveSampleProbability(size));

The blocker level

One subtlety: the deferred bytes belong to a particular blocker level. If the blocker’s level changes while bytes are pending, the pending bytes have to be flushed first, because they were accumulated under the old level’s rules:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:71
 /// Make sure we do memory tracker calls with the correct level in MemoryTrackerBlockerInThread.
 /// E.g. suppose allocImpl is called twice: first for 2 MB with blocker set to
 /// VariableContext::User, then for 3 MB with no blocker. This should increase the
 /// Global memory tracker by 5 MB and the User memory tracker by 3 MB. So we can't group
 /// these two calls into one memory_tracker->allocImpl call.
 VariableContext blocker_level = MemoryTrackerBlockerInThread::getLevel();
 if (blocker_level != cur_thread->untracked_memory_blocker_level)
 {
 cur_thread->flushUntrackedMemory();
 }
 cur_thread->untracked_memory_blocker_level = blocker_level;

The example in the comment is the clearest possible statement of why batching is not always safe: 2 MB under a User-level blocker plus 3 MB without a blocker must not become one 5 MB charge, because the first 2 MB should not be charged to the user and the second 3 MB should be. So the level change forces a flush and the two charges stay separate.

The free path

The free side is symmetric, and has the same structure:

 cpp
 
 
// src/Common/CurrentMemoryTracker.cpp:170
 Int64 new_untracked_memory = cur_thread->untracked_memory.add(-size);

 /// Flush when the per-thread cap is hit, or when the per-CPU budget cannot cover the deferral.
 if (new_untracked_memory < -cur_thread->untracked_memory_limit
 || !DB::per_cpu_memory.sync(new_untracked_memory, cur_thread->per_cpu_untracked_memory))
 {
 DB::per_cpu_memory.release(cur_thread->per_cpu_untracked_memory);

 cur_thread->untracked_memory.store(0);
 /// We cannot return the AllocationTrace from here, since its sample_probability was calculated on the (batched) flushed size, which may not match the original allocation size.
 if (new_untracked_memory > 0)
 std::ignore = memory_tracker->allocImpl(new_untracked_memory, /*enforce_memory_limit=*/ false, /*query_tracker=*/ nullptr, /*_sample_probability=*/ 0.0);
 else
 std::ignore = memory_tracker->free(-new_untracked_memory, /*_sample_probability=*/ 0.0);
 }

Two differences from the allocation path:

The condition is < -limit, not > +limit. A negative balance means deferred frees — memory that was allocated (and charged) elsewhere and is being freed here, so the balance can go negative.
The flush uses enforce_memory_limit = false for a positive balance. A deferred free must never be the thing that trips the memory limit: the bytes are already accounted for, and refusing the free would leave the tracker permanently over-charged. Freeing is always allowed.

The per-CPU budget

The per-thread cap alone is not enough. max_untracked_memory is 4 MiB per thread, and a server with 1 000 threads can therefore hide 4 GB of untracked memory. The per-CPU budget is the second line of defence:

 cpp
 
 
// src/Common/PerCPUMemory.h:26
/// Per-CPU bound on the deferred (un-flushed) untracked memory the global MemoryTracker cannot yet
/// see, capping it in both directions: deferred allocations (-> OOM) and deferred frees (-> the
/// tracker over-reports usage).
///
/// Each CPU has two one-sided counters: `allocated` (>= 0) and `freed` (<= 0). A thread's
/// untracked_memory contributes to exactly one of them, by sign, batched so the shared counter is
/// touched only after drifting by `buffer`. When the occupied side leaves [-capacity, capacity]
/// the thread must flush, bounding the deferred bytes of every thread on the CPU, parked included.
///
/// Two separate counters, not one signed net, because a signed net would let a deferred free cancel
/// a deferred allocation: the allocation passes admission, then the free flushes on its own (its
/// thread exits or hits its per-thread cap) and the allocation is left untracked. Each thread can
/// leave up to max_untracked_memory untracked that way, i.e. nthreads * max_untracked_memory total —
/// the ~40 GiB overcommit this exists to remove.
///
/// Each counter is the sum of all threads' contributions to it (PerCPUMemoryThreadState::contributed):
/// every add is matched by a later subtract on flush, or a move to the other side/CPU on a sign
/// change or migration, so it equals the real deferred bytes on that side and the cap stays accurate.
///
/// The capacity check races with concurrent updates, so a counter may briefly exceed capacity —
/// harmless, since it only gates when bytes flush into the MemoryTracker, which enforces the real limit.
///
/// Bound per CPU: 0 <= allocated <= capacity, -capacity <= freed <= 0 (+ in-flight), so deferred
/// memory is ~ ncpu * max_per_cpu_untracked_memory + nthreads * per_cpu_untracked_memory_thread_buffer.
class PerCPUMemory

Read that comment twice. The “two separate counters, not one signed net” paragraph is the whole reason for the design, and the failure it prevents is subtle:

A thread allocates 3 MiB (deferred, not yet flushed) and then frees it (also deferred). With a single signed counter, the two cancel out and the counter returns to zero.
But the free was deferred against a block that was allocated by someone else — or, more simply, the free flushes on its own (the thread exits, or hits its per-thread cap in the negative direction) while the allocation is still sitting in the buffer.
The allocation is then left untracked forever: it passed admission, but nothing ever flushes it.

With one counter per side, the allocation sits on the allocated side and the free on the freed side, and each is accounted independently.

The constants:

 cpp
 
 
// src/Common/PerCPUMemory.h:52
public:
 static constexpr Int64 DEFAULT_BUDGET = 8 * 1024 * 1024;
 static constexpr Int64 DEFAULT_THREAD_BUFFER = 32 * 1024;
 /// max_per_cpu_untracked_memory == 0 maps here: no side ever leaves the range, disabling the
 /// per-CPU bound (only the per-thread cap applies).
 static constexpr Int64 UNLIMITED_BUDGET = std::numeric_limits<Int64>::max() / 2;

8 MiB per CPU, 32 KiB of per-thread slack. The slack is the price of not touching the shared per-CPU counter on every single allocation:

 cpp
 
 
// src/Common/PerCPUMemory.h:83
 [[nodiscard]] ALWAYS_INLINE bool sync(Int64 untracked_memory, PerCPUMemoryThreadState & state)
 {
 const Int64 buffer_now = buffer.load(std::memory_order_relaxed);
 if (likely(std::abs(untracked_memory - state.contributed) < buffer_now))
 return true;

 return publish(untracked_memory, state);
 }

state.contributed is what this thread last published to its CPU’s counter. If the thread’s own balance has not drifted more than buffer (32 KiB) from what it already published, nothing needs to happen — the shared counter already accounts for this thread. Only a drift of more than 32 KiB forces an update of the shared counter.

Publishing

 cpp
 
 
// src/Common/PerCPUMemory.h:92
 [[nodiscard]] ALWAYS_INLINE bool publish(Int64 untracked_memory, PerCPUMemoryThreadState & state)
 {
 const int cpu = sched_getcpu();
 if (unlikely(static_cast<unsigned>(cpu) >= static_cast<unsigned>(cpu_count)))
 return false;

 Int64 occupied = 0;
 /// Fast path only for a genuine same-side delta. untracked_memory != 0 diverts a now-balanced
 /// thread to the slow branch: it holds nothing and must always be admitted, but the delta path
 /// would test the shared counter and could force a flush under other threads' pressure.
 const bool same_side = cpu == state.contributed_on_cpu
 && untracked_memory != 0
 && (untracked_memory > 0) == (state.contributed > 0);
 if (likely(same_side))
 {
 const Int64 delta = untracked_memory - state.contributed;
 std::atomic<Int64> & side = untracked_memory > 0 ? slots[cpu].allocated : slots[cpu].freed;
 occupied = side.fetch_add(delta, std::memory_order_relaxed) + delta;
 state.contributed = untracked_memory;
 }
 else
 {
 /// Migrated or changed sign: add to the new CPU/side before removing from the old, so
 /// the bytes are never charged to no CPU (a brief double-charge only tightens the bound).
 if (untracked_memory > 0)
 occupied = slots[cpu].allocated.fetch_add(untracked_memory, std::memory_order_relaxed) + untracked_memory;
 else if (untracked_memory < 0)
 occupied = slots[cpu].freed.fetch_add(untracked_memory, std::memory_order_relaxed) + untracked_memory;
 release(state);
 state.contributed = untracked_memory;
 state.contributed_on_cpu = cpu;
 }

 /// allocated >= 0 and freed <= 0, so one symmetric check covers both sides.
 const Int64 capacity_now = capacity.load(std::memory_order_relaxed);
 return -capacity_now <= occupied && occupied <= capacity_now;
 }

Three cases, all of them real:

Same CPU, same side: the fast path. One fetch_add of the delta.
Migrated to another CPU, or the sign flipped: the slow path. The bytes are added to the new side before being removed from the old, so they are never charged to no CPU. A brief double-charge is acceptable — it only tightens the bound, never loosens it.
The CPU id is out of range: return false, which means “flush”. This is the safe direction: the caller flushes to the real tracker.

Note sched_getcpu(). This is Linux-only; on other platforms the whole PerCPUMemory class is a stub whose sync always returns true, which disables the per-CPU bound and leaves only the per-thread cap.

Rolling back

If the flush throws, the per-CPU accounting has to be undone:

 cpp
 
 
// src/Common/PerCPUMemory.h:143
 /// Re-add the pre-sync() snapshot after a failed flush; unconditional (unlike the gated
 /// sync) because the bytes are still live. May transiently exceed capacity; the next sync
 /// corrects it.
 ALWAYS_INLINE void rollback(PerCPUMemoryThreadState & state, const PerCPUMemoryThreadState & saved)
 {
 release(state);
 if (saved.contributed_on_cpu >= 0)
 {
 if (saved.contributed > 0)
 slots[saved.contributed_on_cpu].allocated.fetch_add(saved.contributed, std::memory_order_relaxed);
 else if (saved.contributed < 0)
 slots[saved.contributed_on_cpu].freed.fetch_add(saved.contributed, std::memory_order_relaxed);
 }
 state = saved;
 }

The asymmetry between sync (gated by the buffer) and rollback (unconditional) is deliberate and commented: the bytes are still live, so the counter must reflect them whether or not the buffer was crossed.

Flushing explicitly

Two situations force a flush regardless of the limits. One is a change of blocker level (shown above). The other is the thread finishing or changing its group:

 cpp
 
 
// src/Common/ThreadStatus.cpp:263
void ThreadStatus::flushUntrackedMemory()
{
 /// The deferred bytes our contribution accounted for are about to be tracked, so remove it.
 per_cpu_memory.release(per_cpu_untracked_memory);

 Int64 current_untracked_memory = untracked_memory.load();
 if (current_untracked_memory == 0)
 return;

 MemoryTrackerBlockerInThread blocker(untracked_memory_blocker_level);
 untracked_memory.store(0);
 memory_tracker.adjustWithUntrackedMemory(current_untracked_memory);
}

The order is important: the per-CPU contribution is released first (because those bytes are about to be tracked for real), then the balance is zeroed, and only then is the tracker adjusted. adjustWithUntrackedMemory is a thin wrapper that picks the right direction:

 cpp
 
 
// src/Common/MemoryTracker.cpp:669
void MemoryTracker::adjustWithUntrackedMemory(Int64 untracked_memory)
{
 if (untracked_memory > 0)
 std::ignore = allocImpl(untracked_memory, /*enforce_memory_limit*/ false);
 else
 std::ignore = free(-untracked_memory);
}

enforce_memory_limit = false again: the bytes were already allocated, and a flush must never be the thing that refuses them.

The limit is a session setting

The 4 MiB default is max_untracked_memory, and it is per-query:

 cpp
 
 
// src/Core/Settings.cpp:4656
 DECLARE(UInt64, max_untracked_memory, (4 * 1024 * 1024), R"(
 Small allocations and deallocations are grouped in thread local variable and tracked or profiled only when an amount (in absolute value) becomes larger than the specified value. If the value is higher than 'memory_profiler_step' it will be effectively lowered to 'memory_profiler_step'.
)", 0) \

and it is lowered further if the memory profiler’s step is smaller, so that profiling granularity is preserved:

 cpp
 
 
// src/Interpreters/ThreadStatusExt.cpp:394
 untracked_memory_limit = settings[Setting::max_untracked_memory];
 if (settings[Setting::memory_profiler_step] && settings[Setting::memory_profiler_step] < static_cast<UInt64>(untracked_memory_limit))
 untracked_memory_limit = settings[Setting::memory_profiler_step];

The per-CPU bounds are server-wide:

 cpp
 
 
// src/Core/ServerSettings.cpp:432
 DECLARE(UInt64, max_per_cpu_untracked_memory, (8 * 1024 * 1024), R"(
 Upper bound, in bytes, on the untracked memory all threads running on one CPU may hold at once before it is flushed to the memory tracker. While `max_untracked_memory` bounds a single thread, this bounds the per-CPU total, so many threads cannot multiply their per-thread allowance into a large server-wide overcommit. The total untracked memory is therefore bounded by roughly `number_of_cpus * max_per_cpu_untracked_memory`. A value of `0` disables the per-CPU bound (only the per-thread `max_untracked_memory` applies). Linux only.
)", 0) \
 DECLARE(UInt64, per_cpu_untracked_memory_thread_buffer, (32 * 1024), R"(
 Amount of untracked memory, in bytes, each thread may hold without touching the shared per-CPU budget. It amortizes the cost of the per-CPU bookkeeping for small allocations and is the slack added on top of `max_per_cpu_untracked_memory * number_of_cpus` in the worst case. A value of `0` removes the slack: every allocation updates the shared per-CPU counter (most precise, but more contention). Linux only.
)", 0) \

The worst case is spelled out: number_of_cpus * 8 MiB + nthreads * 32 KiB. On a 64-core box with 1 000 threads, that is 512 MiB + 32 MiB ≈ 544 MiB of memory that the server’s tracked total does not know about at any given instant.

What it costs

The tracked total lags. Up to 4 MiB per thread and up to 8 MiB per CPU can be unaccounted for at any instant. system.processes.memory_usage and system.query_log.memory_usage are therefore lower bounds on what is actually held, not exact figures.

A freed block can be untracked before its allocation is. This is the freed_beyond_held case, and it is documented in the tracker header:

 cpp
 
 
// src/Common/MemoryTracker.h:154
#ifdef DEBUG_OR_SANITIZER_BUILD
 std::atomic_bool drift_expected = false;
 /// A starting point, lowered as flagged sites get fixed.
 static constexpr Int64 drift_warn_threshold = 1024 * 1024;
 /// What a query root freed beyond what it held, which went straight to the server total (see `free`). For example,
 /// thread A of a query allocates a 2 MiB block that stays in its untracked memory, below `max_untracked_memory`, and
 /// thread B frees the block and flushes: the query holds less than 2 MiB, so the rest goes here. When A flushes later,
 /// the query is charged for a block that is already freed, which shows as drift at the end of the query but is not.
 std::atomic<Int64> freed_beyond_held = 0;
#endif

The drift is bounded and it is settled when the query ends:

 cpp
 
 
// src/Common/MemoryTracker.cpp:719
Int64 MemoryTracker::unexpectedDrift() const
{
#ifdef DEBUG_OR_SANITIZER_BUILD
 auto * loaded_parent = parent.load(std::memory_order_relaxed);
 if (!loaded_parent || loaded_parent->level != VariableContext::User || drift_expected.load(std::memory_order_relaxed))
 return 0;
 Int64 drift = amount.load(std::memory_order_relaxed);
 /// Threads batch what they allocate and free (`max_untracked_memory`), so a free can arrive before the allocation
 /// it matches. It goes to the server total, and the allocation, arriving later, stays here without being drift.
 if (drift > 0)
 drift = std::max<Int64>(drift - freed_beyond_held.load(std::memory_order_relaxed), 0);
 return std::abs(drift) > drift_warn_threshold ? drift : 0;
#else
 return 0;
#endif
}

This is debug-build-only, and the threshold is 1 MiB — a smaller drift is noise.

The flush can throw. If the server limit is reached exactly when a thread flushes its 4 MiB, the flush throws MEMORY_LIMIT_EXCEEDED from inside an allocation that was going to succeed. This is the correct behaviour — the server really is out of memory — but it means the exception can surface at a confusing place.

sched_getcpu() per publish. The per-CPU mechanism needs the CPU id, which is a vDSO call on Linux but not free. The 32 KiB buffer exists precisely to make that call rare.

Measuring it

ClickHouse ships a benchmark for exactly this path:

 cpp
 
 
// src/Common/examples/memory_tracker_benchmark.cpp:1
/// Benchmark of CurrentMemoryTracker::{alloc,free} under a 4-level nested
/// MemoryTracker hierarchy (Global -> User -> Process -> Thread).
///
/// Usage:
/// clickhouse-examples memory_tracker_benchmark
/// [--threads N] [--ops N] [--size N]
/// [--mode pair|alloc-only|free-only|new-delete]

The pair mode measures the tracker alone; the new-delete mode adds jemalloc on top, and the difference between them is the allocator’s share:

 cpp
 
 
// src/Common/examples/memory_tracker_benchmark.cpp:180
 case Mode::NewDelete:
 for (Int64 i = 0; i < ops; ++i)
 {
 /// Goes through operator new -> Memory::trackMemory ->
 /// CurrentMemoryTracker::allocNoThrow -> jemalloc;
 /// delete[] mirrors with Memory::untrackMemory + jefree.
 /// Use volatile asm to prevent the compiler from eliding the
 /// pair, but skip touching the memory so we measure the
 /// allocator + tracker cost, not page-fault cost.
 char * ptr = new char[static_cast<size_t>(size)];
 clobber(ptr);
 delete[] ptr;
 ++done;
 }

Note that the benchmark constructs a ThreadStatus on the worker’s stack, so that current_thread is non-null and the per-thread and per-CPU paths are exercised:

 cpp
 
 
// src/Common/examples/memory_tracker_benchmark.cpp:141
 /// Construct ThreadStatus on the worker stack so `current_thread` is
 /// non-null and `CurrentMemoryTracker` exercises the per-thread +
 /// per-CPU code paths. Re-parent the thread tracker to the process
 /// (query) level so the chain is Thread -> Process -> User -> Global.
 ThreadStatus thread_status;
 CurrentThread::get().memory_tracker.setParent(process_tracker);

What to go do about it

The one thing to remember: the memory numbers you read are a lower bound, and the gap is bounded and known. If you need exact accounting — for a profiler, or for a strict budget — set max_untracked_memory to 0:

 sql
 
SELECT max_untracked_memory FROM system.settings WHERE name = 'max_untracked_memory';

 sql
 
 
SELECT count()
FROM events
GROUP BY user_id
SETTINGS max_untracked_memory = 0;

That turns every allocation into an individual charge up the four-level chain, which is exactly what you want for accuracy and exactly what you do not want for throughput.

The part people get wrong: setting max_untracked_memory = 0 to “get accurate numbers” is a performance cliff, not a tuning knob. The tracker chain is on the hot path of every allocation, and the batching is what keeps it from dominating query time. Measure before you do it.

What to go do about it: if you are chasing a discrepancy between system.processes.memory_usage and the RSS of the process, the answer is usually in the server-level out-of-memory message’s Untracked memory across all threads figure, not in the per-query number.

Memory-Tracking on Opensource internals

ClickHouse Intercepts Every Allocation to Charge It

Layer 1: the C++ operators

The size problem, part one: charge the real size class

The size problem, part two: the free side

Solution A: pass the size explicitly

Solution B: the sized operator delete

Solution C: ask jemalloc

Layer 2: the libc symbols

The C API cannot throw

realloc is the interesting one

Layer 3: macOS

Throwing or not

Sampling

Overflow guards

Prefaulting

What it costs

What to go do about it

ClickHouse Charges Memory Through a Hierarchy of Trackers

The four levels

The parent pointer

Walking the chain

Carrying the query identity down the chain

Where the limits come from

Where the thread tracker joins

The free path

Settling what a query leaves behind

The hot fields

Metrics and observability

What the hierarchy buys and what it costs

ClickHouse Batches Memory Accounting Per Thread

The thread-local pointer

The atomic-less counter

The allocation path

The blocker level

The free path

The per-CPU budget

Publishing

Rolling back

Flushing explicitly

The limit is a session setting

What it costs

Measuring it

What to go do about it

Solution B: the sized `operator delete`

`realloc` is the interesting one