The Consecutive-Rows Fast Path: A DELETE That Stores Zero Row IDs

Jay Hubbert — Sun, 25 Oct 2026 10:40:00 +0000

TL;DR: A DELETE has to remember which rows it removed, so that a ROLLBACK can put them back. The obvious implementation is an array of row ids. DuckDB notices that a delete almost always hits a contiguous range of row ids inside a vector, sets a one-bit flag, and allocates no array at all. The undo entry shrinks to a fixed-size struct, and at commit time a whole-vector delete collapses into a single 8-byte transaction id shared by all 2048 rows of the vector.

The scenario

You run:

 sql
 
DELETE FROM events WHERE ts < '2026-01-01';

and the predicate happens to match 3 million rows. DuckDB does not rewrite the table - the rows stay exactly where they are on disk. Instead it writes a delete marker into the row group’s version information so that later scans skip them.

But a transaction is allowed to be rolled back. If your DELETE fails halfway through (a constraint violation on some other table, say), DuckDB has to un-delete the rows it already marked. That means the transaction’s undo log has to carry enough information to reconstruct exactly which rows were deleted.

The naive encoding is an array of row ids, 8 bytes each. For 3 million rows that is 24 MB of pure bookkeeping, and it has to be written, kept alive, and walked on every rollback.

DuckDB does much better, and the trick is a single observation about what deletes look like.

Deletes are bucketed by vector before they reach the undo log

DuckDB stores data in vectors of STANDARD_VECTOR_SIZE = 2048 rows. A row id is decomposed into a vector index and an offset within the vector. The delete path uses that decomposition directly: it accumulates row offsets for the current vector and flushes them when the scan moves to the next vector.

 cpp
 
 
// src/storage/table/row_group.cpp:2177
void VersionDeleteState::Delete(row_t row_id) {
	D_ASSERT(row_id >= 0);
	idx_t vector_idx = UnsafeNumericCast<idx_t>(row_id) / STANDARD_VECTOR_SIZE;
	idx_t idx_in_vector = UnsafeNumericCast<idx_t>(row_id) - vector_idx * STANDARD_VECTOR_SIZE;
	if (current_chunk != vector_idx) {
		Flush();

		current_chunk = vector_idx;
		chunk_row = vector_idx * STANDARD_VECTOR_SIZE;
	}
	rows[count++] = UnsafeNumericCast<row_t>(idx_in_vector);
}

Two things to notice. First, rows[] holds offsets within the vector, not absolute row ids, so each entry fits in 16 bits (a vector has 2048 rows; the offset needs 11). Second, the delete statement walks rows in order, so the offsets it accumulates are usually 0, 1, 2, 3, ... - consecutive.

When the buffer for one vector is full, the state is flushed: the actual delete markers are written into the version info, and only the rows that were really deleted (some may have been deleted already, e.g. by a USING clause) are pushed into the undo log.

 cpp
 
 
// src/storage/table/row_group.cpp:2190
void VersionDeleteState::Flush() {
	if (count == 0) {
		return;
	}
	// it is possible for delete statements to delete the same tuple multiple times when combined with a USING clause
	// in the current_info->Delete, we check which tuples are actually deleted (excluding duplicate deletions)
	// this is returned in the actual_delete_count
	auto actual_delete_count = info.DeleteRows(current_chunk, transaction.GetTransactionId(), rows, count);
	delete_count += actual_delete_count;
	if (deleted_ids) {
		if (!deleted_ids->has_skipped_rows && actual_delete_count != count) {
			// the first skipped rows - all rows flushed before were deleted, they are the preceding input rows
			deleted_ids->has_skipped_rows = true;
			deleted_ids->row_ids.assign(deleted_ids->input_ids, deleted_ids->input_ids + input_offset);
		}
		if (deleted_ids->has_skipped_rows) {
			// DeleteRows moves the rows that were actually deleted to the front
			for (idx_t i = 0; i < actual_delete_count; i++) {
				deleted_ids->row_ids.push_back(UnsafeNumericCast<row_t>(base_row + chunk_row) + rows[i]);
			}
		}
		input_offset += count;
	}
	if (transaction.transaction && actual_delete_count > 0) {
		// now push the delete into the undo buffer, but only if any deletes were actually performed
		transaction.transaction->PushDelete(table_entry, info.GetOrCreateVersionInfo(), current_chunk, rows,
		 actual_delete_count, base_row + chunk_row);
	}
	count = 0;
}

PushDelete is where the interesting decision is made.

The undo entry for a delete is a DeleteInfo. It is not a pointer-chasing heap object; it is a flat struct written into the transaction’s bump-allocated undo buffer, with a flexible array member at the end.

 cpp
 
 
// src/include/duckdb/transaction/delete_info.hpp:17
struct DeleteInfo {
	DuckTableEntry *table;
	RowVersionManager *version_info;
	idx_t vector_idx;
	idx_t count;
	idx_t base_row;
	//! Whether or not row ids are consecutive (0, 1, 2, ..., count).
	//! If this is true no rows are stored and `rows` should not be accessed.
	bool is_consecutive;

	uint16_t *GetRows() {
		if (is_consecutive) {
			throw InternalException("DeleteInfo is consecutive - rows are not accessible");
		}
		return rows;
	}
	const uint16_t *GetRows() const {
		if (is_consecutive) {
			throw InternalException("DeleteInfo is consecutive - rows are not accessible");
		}
		return rows;
	}

private:
	//! The per-vector row identifiers (actual row id is base_row + rows[x])
	uint16_t rows[1];
};

Read the field list as a header plus a payload:

table, version_info, vector_idx - which vector of which table is affected.
base_row - the absolute row id of offset 0 in this vector. The payload holds offsets, so the absolute row id of entry x is base_row + rows[x].
count - how many rows were deleted.
is_consecutive - the flag that changes everything.
rows[1] - a flexible array member. When the delete is not consecutive, count uint16_t offsets are stored immediately after the header.

The GetRows() accessors are the load-bearing part of the design: if is_consecutive is set, they throw. A caller cannot silently read a row array that was never written. The flag is not “advisory” - it is the only way the code is allowed to distinguish the two layouts, and the type system plus the exception make forgetting to check it a loud failure rather than a use-after-uninitialized-read.

The fast path: detect consecutiveness, then allocate nothing

 cpp
 
 
// src/transaction/duck_transaction.cpp:101
void DuckTransaction::PushDelete(DuckTableEntry &table_entry, RowVersionManager &info, idx_t vector_idx, row_t rows[],
 idx_t count, idx_t base_row) {
	bool is_consecutive = true;
	// check if the rows are consecutive
	for (idx_t i = 0; i < count; i++) {
		if (rows[i] != row_t(i)) {
			is_consecutive = false;
			break;
		}
	}
	idx_t alloc_size = sizeof(DeleteInfo);
	if (!is_consecutive) {
		// if rows are not consecutive we need to allocate row identifiers
		alloc_size += sizeof(uint16_t) * count;
	}

	auto undo_entry = undo_buffer.CreateEntry(UndoFlags::DELETE_TUPLE, alloc_size);
	auto delete_info = reinterpret_cast<DeleteInfo *>(undo_entry.GetDataMutable());
	delete_info->version_info = &info;
	delete_info->vector_idx = vector_idx;
	delete_info->table = &table_entry;
	delete_info->count = count;
	delete_info->base_row = base_row;
	delete_info->is_consecutive = is_consecutive;
	if (!is_consecutive) {
		// if rows are not consecutive
		auto delete_rows = delete_info->GetRows();
		for (idx_t i = 0; i < count; i++) {
			delete_rows[i] = NumericCast<uint16_t>(rows[i]);
		}
	}
}

Read it line by line and the whole design falls out:

The loop tests rows[i] != row_t(i) and breaks on the first mismatch. For the common case (a contiguous range), it exits after one iteration having proved 0 == 0. The check costs one comparison in the good case, and count comparisons in the bad case.
alloc_size is sizeof(DeleteInfo) - the header plus the one-element flexible array - when the rows are consecutive. There is no payload at all. The undo entry is a fixed-size struct.
When the rows are not consecutive, the payload is sizeof(uint16_t) * count bytes. Two bytes per deleted row, not eight. That is because the offsets are relative to base_row and a vector only has 2048 rows.
undo_buffer.CreateEntry bump-allocates exactly alloc_size bytes. No second allocation, no pointer to chase, no vector of nodes.

So the cost of a delete’s undo entry is:

Case	Undo bytes per vector
Consecutive (the common case)	`sizeof(DeleteInfo)` - fixed, independent of `count`
Not consecutive	`sizeof(DeleteInfo) + 2 * count`

The constant-id case: deleting a whole vector

There is a second, even better representation, and it lives one layer down, in the row group’s version info. Every vector of a row group carries a ChunkVectorInfo, which stores the delete transaction id of its rows. That id can be stored three ways:

 cpp
 
 
// src/include/duckdb/storage/table/chunk_info.hpp:47
//! The storage state of the delete side of a ChunkVectorInfo. Exhaustive and mutually exclusive:
//! CONSTANT - all rows share constant_delete_id (NOT_DELETED_ID for none-deleted, or a single delete id)
//! MASKED - partially deleted, all deleted rows share one committed id (mask_delete_id): which rows are
//! deleted is stored in deleted_mask, deleted_data freed
//! ARRAY - per-row delete ids materialized in deleted_data
enum class DeleteIdState : uint8_t { CONSTANT, MASKED, ARRAY };

The CONSTANT state is the one we want: every row in the vector shares a single delete transaction id. No per-row array, no bitmask - one 8-byte number decides the fate of all 2048 rows.

Reaching that state from a delete is the whole point of the consecutive-rows flag. Look at the order of the checks in ChunkVectorInfo::Delete:

 cpp
 
 
// src/storage/table/chunk_info.cpp:384
idx_t ChunkVectorInfo::Delete(transaction_t transaction_id, row_t rows[], idx_t count) {
	if (HasConstantDeleteId() && ConstantDeleteId() != NOT_DELETED_ID) {
		// all rows in this vector share the same deleted id - the rows we are trying to delete are already deleted
		if (ConstantDeleteId() == transaction_id) {
			// the rows were deleted by this transaction already - skip
			return 0;
		}
		// the rows were deleted by another transaction - conflict
		throw TransactionException("Conflict on tuple deletion!");
	}
	if (HasConstantDeleteId() && count == STANDARD_VECTOR_SIZE && DeletesEntireVector(rows, count)) {
		// no rows were deleted yet and we are deleting the entire vector
		// all rows share the same deleted id - store it as a constant instead of materializing per-row delete ids
		constant_delete_id = transaction_id;
		return count;
	}

The second branch is the whole-vector case. Three conditions:

HasConstantDeleteId() - the vector is not already partially deleted, so there is no per-row array to reconcile.
count == STANDARD_VECTOR_SIZE - this delete covers all 2048 rows of the vector.
DeletesEntireVector(rows, count) - and it really covers all of them, not just 2048 of the rows that happen to be in this vector.

The helper is the same “is it 0,1,2,…?” test as in PushDelete:

 cpp
 
 
// src/storage/table/chunk_info.cpp:374
static bool DeletesEntireVector(const row_t rows[], idx_t count) {
	D_ASSERT(count == STANDARD_VECTOR_SIZE);
	for (idx_t i = 0; i < count; i++) {
		if (rows[i] != row_t(i)) {
			return false;
		}
	}
	return true;
}

And note where the branch sits: before the general path materializes the per-row delete id array. The general path continues with allocator.GetHandle(GetInitializedDeletedPointer()), which is what allocates 2048 × 8 bytes of per-row transaction ids. The whole-vector branch returns before that happens. So deleting an entire vector costs:

one 8-byte transaction id, versus
2048 × 8 = 16 KB of per-row ids.

A 2048x reduction in the version-info footprint, and the per-row array is never allocated in the first place.

At scan time, the constant state is also the cheapest possible check. Instead of testing 2048 ids, the scan compares one id against the snapshot:

 cpp
 
 
// src/storage/table/chunk_info.cpp:69
template <class INSERT_OP, class DELETE_OP>
idx_t ChunkVectorInfo::TemplatedGetSelVector(const SnapshotView &view, optional_ptr<SelectionVector> sel_vector,
 idx_t max_count) const {
	switch (delete_state) {
	case DeleteIdState::CONSTANT: {
		// all tuples have the same deleted id
		if (DELETE_OP::IsDeleted(view, ConstantDeleteId())) {
			// all tuples are deleted
			return 0;
		}
		// no tuples are deleted: we only have to check the inserted ids
		if (HasConstantInsertionId()) {
			// all tuples have the same inserted id as well
			if (INSERT_OP::UseInsertedVersion(view, ConstantInsertId())) {
				return max_count;
			} else {
				return 0;
			}
		}

One comparison decides whether the entire vector is visible.

The flag also pays off at commit time

The CONSTANT state can also be reached after the fact, at commit. That is where the is_consecutive flag on the undo entry earns its keep.

 cpp
 
 
// src/storage/table/chunk_info.cpp:427
void ChunkVectorInfo::CommitDelete(transaction_t commit_id, const DeleteInfo &info) {
	if (info.is_consecutive && info.count == STANDARD_VECTOR_SIZE) {
		// the delete covers the entire vector - all rows share the same deleted id
		// we can store the deleted id as a constant and free any per-row delete ids
		FreeDeleteData();
		constant_delete_id = commit_id;
		return;
	}
	if (HasConstantDeleteId() && ConstantDeleteId() == commit_id) {
		// all rows already share this exact deleted id - nothing to do
		return;
	}
	// we are materializing / modifying per-row delete ids - re-arm the compression check
	recheck_compression = true;
	bool all_equal = true;
	{
		auto segment = allocator.GetHandle(GetInitializedDeletedPointer());
		auto deleted = segment.GetPtr<transaction_t>();

		if (info.is_consecutive) {
			for (idx_t i = 0; i < info.count; i++) {
				deleted[i] = commit_id;
			}
		} else {
			auto rows = info.GetRows();
			for (idx_t i = 0; i < info.count; i++) {
				deleted[rows[i]] = commit_id;
			}
		}

Three things happen here, and they are all consequences of the one flag:

info.is_consecutive && info.count == STANDARD_VECTOR_SIZE -> the delete covered a whole vector -> FreeDeleteData() throws away the 16 KB per-row array and replaces it with a single constant_delete_id.
info.is_consecutive (but a partial vector) -> the loop writes commit_id to deleted[0..count). The offsets are implicit: they are the loop index. The undo entry did not have to store them.
Otherwise -> info.GetRows() is used, which is the one case where the payload exists.

Note that branch 3 is the only one that calls GetRows(), which is exactly why the accessor can throw when is_consecutive is set: in that situation the code never asks for the rows.

Rollback and the WAL both reconstruct the row ids from the flag

The flag has to be understood by every consumer of the undo entry, because the row ids are not stored anywhere.

The WAL writer reconstructs them arithmetically:

 cpp
 
 
// src/transaction/wal_write_state.cpp:189
void WALWriteState::WriteDelete(DeleteInfo &info) {
	// switch to the current table, if necessary
	SwitchTable(*info.table, UndoFlags::DELETE_TUPLE);

	if (!delete_chunk) {
		delete_chunk = make_uniq<DataChunk>();
		vector<LogicalType> delete_types = {LogicalType::ROW_TYPE};
		delete_chunk->Initialize(Allocator::DefaultAllocator(), delete_types);
	}
	auto rows = FlatVector::GetDataMutable<row_t>(delete_chunk->data[0]);
	if (info.is_consecutive) {
		for (idx_t i = 0; i < info.count; i++) {
			rows[i] = UnsafeNumericCast<int64_t>(info.base_row + i);
		}
	} else {
		auto delete_rows = info.GetRows();
		for (idx_t i = 0; i < info.count; i++) {
			rows[i] = UnsafeNumericCast<int64_t>(info.base_row) + delete_rows[i];
		}
	}
	delete_chunk->SetChildCardinality(info.count);
	log.WriteDelete(*delete_chunk);
}

info.base_row + i. The row ids are implicit in the arithmetic. The WAL still gets the full list of deleted row ids - the WAL format has no notion of “a consecutive range” - but it is recomputed on the fly instead of being stored in memory.

Rollback is the mirror image. It does not “un-delete” anything: it re-applies the delete info with the sentinel id NOT_DELETED_ID, which means “never deleted”.

 cpp
 
 
// src/transaction/rollback_state.cpp:38
	case UndoFlags::DELETE_TUPLE: {
		auto info = reinterpret_cast<DeleteInfo *>(data);
		// reset the deleted flag on rollback
		info->version_info->CommitDelete(info->vector_idx, NOT_DELETED_ID, *info);
		break;
	}

Because CommitDelete understands is_consecutive, the rollback path can restore a consecutive delete by writing NOT_DELETED_ID into deleted[0..count) - again without ever touching a stored array.

What it costs

The check is O(count) in the bad case, O(1) in the good case. PushDelete breaks out of the loop on the first non-consecutive row, so a scan that produces a contiguous range pays one comparison.
16-bit offsets only work because vectors are 2048 rows. The payload is relative to base_row, so it never has to hold an absolute row id. If DuckDB ever grew vectors past 65536 rows this scheme would need to widen.
The flag only pays off for deletes that are contiguous within one vector. A delete that skips rows - DELETE ... WHERE x % 2 = 0 - produces offsets 0, 2, 4, ... and falls back to the array. It still costs 2 bytes per row rather than 8, but it is not free.
Consecutiveness is per vector, not per statement. A delete that matches 3 million rows produces 3 million / 2048 = roughly 1464 undo entries. Each one is a fixed-size struct. The undo log is a handful of blocks, not 24 MB.
The whole-vector constant case has a precondition: the delete must have covered all 2048 rows of the vector. A DELETE with a filter that happens to match 2047 of 2048 rows does not get the constant state, and pays the per-row array.

The bottom line

The lesson is not “deletes are cheap”. It is: represent the common case implicitly and make the uncommon case pay.

DuckDB’s delete machinery has three representations for “which rows are gone”:

A single transaction id for the whole vector (CONSTANT) - 8 bytes for 2048 rows.
A bitmask plus one shared id (MASKED) - 2048 bits plus 8 bytes.
A full array of per-row ids (ARRAY) - 16 KB.

And it has two representations for “which rows did this transaction delete, so we can roll back”:

A flag saying “the offsets were 0, 1, 2, …, count-1” - zero payload bytes.
An array of uint16_t offsets - two bytes per row.

In both cases the implicit representation is not just an optimisation, it is the only one that is correct in the common case, and the code is written so that getting it wrong fails loudly (GetRows() throws) rather than silently reading memory that was never written.

The general principle: when the shape of your data is predictable, encode the shape and pay for the exception. A contiguous range of row ids is the single most predictable thing about a DELETE, and it turns a 24 MB undo entry into a 48-byte one.

Delete on Opensource internals