Internals on Opensource internals

PostgreSQL Stores a Row

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Every database gives you a row and every database lies a little about what that row is. MySQL gives you a record copied into a row buffer. SQLite gives you a record in a b-tree leaf. PostgreSQL gives you something stranger and more interesting: a 23-byte header followed by a payload, living at some byte offset on an 8 kB page, reached through a 4-byte indirection that lives in a completely different part of that same page.

That indirection is the reason PostgreSQL can move tuples around during VACUUM without invalidating a single index entry. The header is the reason an UPDATE can leave a trail of old versions behind. The slotted page is the reason a table’s rows are not laid out contiguously — and why reclaiming the space left behind by dead rows is a whole subsystem.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

A row is not a record

Let’s start with the container. PostgreSQL stores a table as an array of fixed-size pages — 8 kB by default, configurable at initdb time. The heap access method never stores anything that isn’t on a page, and it never stores a row that spans two pages. A row that doesn’t fit in 8 kB gets pushed out of line entirely (more on that later).

Each page is a slotted page. From src/include/storage/bufpage.h:

 c
 
 
/*
 * a postgres disk page is always a slotted page of the form:
 *
 * +----------------+---------------------------------+
 * | PageHeaderData | linp1 linp2 linp3 ... |
 * +-----------+----+---------------------------------+
 * | ... linpN |									 |
 * +-----------+--------------------------------------+
 * |		 ^ pd_lower							 |
 * |												 |
 * |			 v pd_upper							 |
 * +-------------+------------------------------------+
 * |			 | tupleN ... |
 * +-------------+------------------+-----------------+
 * |	 ... tuple3 tuple2 tuple1 | "special space" |
 * +--------------------------------+-----------------+
 *									^ pd_special
 */

Five parts, and they grow in opposite directions:

Region	Size on an 8 kB page	Grows
`PageHeaderData`	24 bytes	fixed
Line pointer array	4 bytes per row	upwards from `pd_lower`
Free space	whatever is left	—
Tuples	variable	downwards from `pd_upper`
Special space	0 for heap pages	fixed

The header is small enough to write out in full:

 c
 
 
typedef struct PageHeaderData
{
	PageXLogRecPtr pd_lsn;		/* LSN: next byte after last byte of xlog
								 * record for last change to this page */
	uint16		pd_checksum;	/* checksum */
	uint16		pd_flags;		/* flag bits, see below */
	LocationIndex pd_lower;		/* offset to start of free space */
	LocationIndex pd_upper;		/* offset to end of free space */
	LocationIndex pd_special;	/* offset to start of special space */
	uint16		pd_pagesize_version;
	TransactionId pd_prune_xid; /* oldest prunable XID, or zero if none */
	ItemIdData	pd_linp[FLEXIBLE_ARRAY_MEMBER]; /* line pointer array */
} PageHeaderData;

pd_lower and pd_upper are the entire free-space bookkeeping. pd_lower marks the end of the line pointer array, pd_upper marks the start of the tuple area; the gap between them is what’s available. There is no free-list, no extent map, no per-page allocation structure. Free space is just an interval, and it is used from both ends.

A heap page has pd_special == BLCKSZ, because the “special space” is where an index access method stashes its own per-page state. A btree leaf page, for instance, ends with a BTPageOpaqueData holding the sibling links and the tree level. Heap pages have nothing to say for themselves.

Line pointers: the indirection that makes everything possible

The line pointer array is where the “row” abstraction starts to leak. Each entry is four bytes (src/include/storage/itemid.h):

 c
 
 
typedef struct ItemIdData
{
	unsigned	lp_off:15,		/* offset to tuple (from start of page) */
				lp_flags:2,		/* state of line pointer, see below */
				lp_len:15;		/* byte length of tuple */
} ItemIdData;

Fifteen bits of byte offset, fifteen bits of length, two bits of state. lp_flags has four values:

 c
 
 
#define LP_UNUSED		0		/* unused (should always have lp_len=0) */
#define LP_NORMAL		1		/* used (should always have lp_len>0) */
#define LP_REDIRECT		2		/* HOT redirect (should have lp_len=0) */
#define LP_DEAD			3		/* dead, may or may not have storage */

Note the convention, stated in the header comment: by convention lp_len == 0 in every line pointer that does not have storage, independently of its lp_flags state. A line pointer can be “in use” and still point at nothing. That is the whole trick.

So a row is addressed by a TID — a tuple identifier — which is a (block number, offset number) pair. The block number says which page; the offset number says which line pointer. The line pointer then says which byte range the tuple occupies. Three levels of addressing, and only the last one is a physical location.

This indirection is not incidental. It’s what lets VACUUM slide tuples around within a page, or move them between pages, and leave every index entry untouched — the index stores a TID, and the TID still resolves to the right line pointer, and the line pointer has been updated to point at the tuple’s new home.

Two details worth knowing:

The number of line pointers is derived, not stored. It is (pd_lower - SizeOfPageHeaderData) / sizeof(ItemIdData). There’s no counter; the array simply grows.
A line pointer’s lp_off is overloaded. For LP_NORMAL it’s a byte offset. For LP_REDIRECT it’s the offset number of another line pointer on the same page. Same field, completely different meaning.

The tuple header: 23 bytes

Now the header at the front of every heap tuple (src/include/access/htup_details.h):

 c
 
 
struct HeapTupleHeaderData
{
	union
	{
		HeapTupleFields t_heap;
		DatumTupleFields t_datum;
	}			t_choice;

	ItemPointerData t_ctid;		/* current TID of this or newer tuple (or a
								 * speculative insertion token) */

	/* Fields below here must match MinimalTupleData! */

#define FIELDNO_HEAPTUPLEHEADERDATA_INFOMASK2 2
	uint16		t_infomask2;	/* number of attributes + various flags */

#define FIELDNO_HEAPTUPLEHEADERDATA_INFOMASK 3
	uint16		t_infomask;		/* various flag bits, see below */

#define FIELDNO_HEAPTUPLEHEADERDATA_HOFF 4
	uint8		t_hoff;			/* sizeof header incl. bitmap, padding */

	/* ^ - 23 bytes - ^ */

#define FIELDNO_HEAPTUPLEHEADERDATA_BITS 5
	uint8		t_bits[FLEXIBLE_ARRAY_MEMBER];	/* bitmap of NULLs */

	/* MORE DATA FOLLOWS AT END OF STRUCT */
};

Twenty-three bytes. Here is what each byte is doing, on a 64-bit machine with MAXIMUM_ALIGNOF == 8:

Offset	Bytes	Field	Meaning
0	4	`t_xmin`	XID of the inserting transaction
4	4	`t_xmax`	XID of the deleting or locking transaction
8	4	`t_cid`	insert and/or delete command ID (overlays `t_xvac`)
12	4	`t_ctid.ip_blkid`	block number of this or a newer version
16	2	`t_ctid.ip_posid`	offset number of this or a newer version
18	2	`t_infomask2`	attribute count + HOT/heap-only flags
20	2	`t_infomask`	content and visibility flags
22	1	`t_hoff`	offset of the user data

ItemPointerData is six bytes, not eight, and it gets there by fiat:

 c
 
 
/* If compiler understands packed and aligned pragmas, use those */
#if defined(pg_attribute_packed) && defined(pg_attribute_aligned)
			pg_attribute_packed()
			pg_attribute_aligned(2)
#endif
ItemPointerData;

The comment above it explains why: “because there is an item pointer in each tuple header and index tuple header on disk, it’s very important not to waste space with structure padding bytes.” Two bytes saved here, on every row, forever.

Five virtual fields in three physical ones

The header stores five logical fields — Xmin, Cmin, Xmax, Cmax, Xvac — in three physical slots. From the header comment:

We store five “virtual” fields Xmin, Cmin, Xmax, Cmax, and Xvac in three physical fields. Xmin and Xmax are always really stored, but Cmin, Cmax and Xvac share a field. This works because we know that Cmin and Cmax are only interesting for the lifetime of the inserting and deleting transaction respectively.

Cmin and Cmax are command IDs — counters that order the statements within a single transaction. If the same transaction inserts and then deletes a row, both values have to survive. PostgreSQL solves this with a “combo CID”: a synthetic value that maps back to both, using local state in the originating backend. The HEAP_COMBOCID bit in t_infomask says “this field is a combo”.

Before 8.3, HeapTupleHeaderData had separate cmin and cmax fields. Removing one 4-byte field from every row in every table was worth the combo-CID machinery.

t_infomask

Sixteen bits, split into two groups. The low four describe the physical contents of the tuple:

 c
 
 
#define HEAP_HASNULL			0x0001	/* has null attribute(s) */
#define HEAP_HASVARWIDTH		0x0002	/* has variable-width attribute(s) */
#define HEAP_HASEXTERNAL		0x0004	/* has external stored attribute(s) */
#define HEAP_HASOID_OLD			0x0008	/* has an object-id field */

The upper twelve are about visibility — who can see this row, and what happened to it:

 c
 
 
#define HEAP_XMAX_KEYSHR_LOCK	0x0010	/* xmax is a key-shared locker */
#define HEAP_COMBOCID			0x0020	/* t_cid is a combo CID */
#define HEAP_XMAX_EXCL_LOCK		0x0040	/* xmax is exclusive locker */
#define HEAP_XMAX_LOCK_ONLY		0x0080	/* xmax, if valid, is only a locker */

#define HEAP_XMIN_COMMITTED		0x0100	/* t_xmin committed */
#define HEAP_XMIN_INVALID		0x0200	/* t_xmin invalid/aborted */
#define HEAP_XMIN_FROZEN		(HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID)
#define HEAP_XMAX_COMMITTED		0x0400	/* t_xmax committed */
#define HEAP_XMAX_INVALID		0x0800	/* t_xmax invalid/aborted */
#define HEAP_XMAX_IS_MULTI		0x1000	/* t_xmax is a MultiXactId */
#define HEAP_UPDATED			0x2000	/* this is UPDATEd version of row */
#define HEAP_MOVED_OFF			0x4000	/* moved to another place by pre-9.0
										 * VACUUM FULL; kept for binary
										 * upgrade support */
#define HEAP_MOVED_IN			0x8000	/* moved from another place by pre-9.0
										 * VACUUM FULL; kept for binary
										 * upgrade support */

HEAP_XMIN_COMMITTED and friends are hint bits. They cache a fact that could otherwise require looking up the transaction’s commit record in CLOG. Once a reader has established that the inserting transaction committed, it stamps the tuple and everyone afterwards gets the answer for free. The rules are conservative — a hint is only set when the answer is known-good — and the code is careful never to set a hint that would be wrong if the buffer is later written to disk:

 c
 
 
	if (TransactionIdIsValid(xid))
	{
		if (BufferIsPermanent(buffer))
		{
			/* NB: xid must be known committed here! */
			XLogRecPtr	commitLSN = TransactionIdGetCommitLSN(xid);

			if (XLogNeedsFlush(commitLSN) &&
				BufferGetLSNAtomic(buffer) < commitLSN)
			{
				/* not flushed and no LSN interlock, so don't set hint */
				return;
			}
		}
	}

Notice the subtlety: if the commit record isn’t on disk yet, the hint isn’t set, because the page might reach disk before the commit record does, and then a crash would leave a tuple claiming to be committed when it isn’t.

t_infomask2

 c
 
 
#define HEAP_NATTS_MASK			0x07FF	/* 11 bits for number of attributes */
/* bits 0x1800 are available */
#define HEAP_KEYS_UPDATED		0x2000	/* tuple was updated and key cols
										 * modified, or tuple deleted */
#define HEAP_HOT_UPDATED		0x4000	/* tuple was HOT-updated */
#define HEAP_ONLY_TUPLE			0x8000	/* this is heap-only tuple */

The low eleven bits hold the attribute count, which is why t_infomask2 is a uint16 and the number of columns is limited. The other three bits carry the HOT state machine, which we will come back to.

The null bitmap and the 23/24 trick

HEAP_HASNULL in t_infomask means a null bitmap is present, immediately after the fixed header, one bit per column:

 c
 
 
static inline int
BITMAPLEN(int NATTS)
{
	return (NATTS + 7) / 8;
}

The bitmap is inverted — a 1 bit means not null:

 c
 
 
/*
 * Check a tuple's null bitmap to determine whether the attribute is null.
 * Note that a 0 in the null bitmap indicates a null, while 1 indicates
 * non-null.
 */
static inline bool
att_isnull(int ATT, const uint8 *BITS)
{
	return !(BITS[ATT >> 3] & (1 << (ATT & 0x07)));
}

Attribute 0 is the low bit of byte 0. (This is the sort of detail that makes hand-parsing a heap page a mildly masochistic exercise. The pageinspect extension exists for a reason.)

Now, t_hoff is the offset from the start of the tuple to the first byte of user data, and it must be a multiple of MAXALIGN — 8 bytes on a 64-bit machine. The reason is that the data area must be 8-byte aligned so that fetchatt() can do a direct aligned load of an int8 or double column.

The calculation, in heap_form_tuple() (src/backend/access/common/heaptuple.c):

 c
 
	len = offsetof(HeapTupleHeaderData, t_bits);

	if (hasnull)
		len += BITMAPLEN(numberOfAttributes);

	hoff = len = MAXALIGN(len); /* align user data safely */

offsetof(HeapTupleHeaderData, t_bits) is 23. So:

No NULLs: len = 23 → MAXALIGN(23) = 24 → t_hoff = 24. Byte 23 is pure padding.
1–8 columns, some NULL: len = 23 + 1 = 24 → MAXALIGN(24) = 24 → t_hoff = 24. The bitmap lands exactly in the padding byte.
9–16 columns, some NULL: len = 23 + 2 = 25 → MAXALIGN(25) = 32 → t_hoff = 32. Seven bytes of padding.

So the header is a 23-byte struct that behaves like a 24-byte header. The one-byte discrepancy is deliberate: a 24-byte header is 8-byte aligned, so the data area inherits that alignment for free, and any 4-byte-aligned field inside it is also correctly aligned. Had the header been 24 bytes and the bitmap 1 byte, the data would start at 25 — unaligned.

There is a hard limit hiding here. t_hoff is a uint8, so it can hold at most 255. The comment in htup_details.h spells out the constraint:

The key limit on this value is that the size of the fixed overhead for a tuple, plus the size of the null-values bitmap (at 1 bit per column), plus MAXALIGN alignment, must fit into t_hoff which is uint8. On most machines the upper limit without making t_hoff wider would be a little over 1700.

Which is why MaxTupleAttributeNumber is 1664 and MaxHeapAttributeNumber is 1600. PostgreSQL’s real column limit is a storage-layout limit, not an arbitrary one.

The data area

After t_hoff come the column values, packed in attribute order. The rules come out of fill_val() in heaptuple.c:

NULL columns cost zero bytes. fill_val sets the bitmap bit and returns before writing any data. The space is genuinely not reserved.
By-value columns (int4, bool, oid, …) are aligned to their attalign and stored inline.
Fixed-length by-reference columns (numeric, uuid, …) are aligned and copied inline.
Short varlena (a text of 126 bytes or less) gets no alignment padding at all — it can sit at any byte offset.
Full 4-byte-header varlena is aligned, because the 4-byte length word must be naturally aligned.
cstring (attlen == -2) never needs alignment.

The interesting one is #4. Here is the code:

 c
 
 
		else if (VARATT_IS_SHORT(val))
		{
			/* no alignment for short varlenas */
			data_length = VARSIZE_SHORT(val);
			memcpy(data, val, data_length);
		}
		else if (att->attispackable && VARATT_CAN_MAKE_SHORT(val))
		{
			/* convert to short varlena -- no alignment */
			data_length = VARATT_CONVERTED_SHORT_SIZE(val);
			SET_VARSIZE_SHORT(data, data_length);
			memcpy(data + 1, VARDATA(val), data_length - 1);
		}
		else
		{
			/* full 4-byte header varlena */
			data = (char *) att_nominal_alignby(data, att->attalignby);
			data_length = VARSIZE(val);
			memcpy(data, val, data_length);
		}

A text value whose payload is 126 bytes or less is stored with a one-byte header and needs no padding. A payload of 127 bytes cannot be short — the limit is VARATT_SHORT_MAX (0x7F) — so it falls back to the 4-byte-header format, which is aligned, and the value suddenly costs several bytes more than the one below it. It is a real, if obscure, discontinuity in the space cost of text columns.

Variable-length values: the four header formats

Every variable-length value shares the varlena concept, but there are four on-disk encodings, distinguished by the low bits of the first header byte (src/include/varatt.h):

 c
 
 
typedef struct
{
	struct						/* Normal varlena (4-byte length) */
	{
		uint32		va_header;
		char		va_data[FLEXIBLE_ARRAY_MEMBER];
	}			va_4byte;
	struct						/* Compressed-in-line format */
	{
		uint32		va_header;
		uint32		va_tcinfo;	/* Original data size (excludes header) and
								 * compression method; see va_extinfo */
		char		va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
	}			va_compressed;
} varattrib_4b;

typedef struct
{
	uint8		va_header;
	char		va_data[FLEXIBLE_ARRAY_MEMBER]; /* Data begins here */
} varattrib_1b;

typedef struct
{
	uint8		va_header;		/* Always 0x80 or 0x01 */
	uint8		va_tag;			/* Type of datum */
	char		va_data[FLEXIBLE_ARRAY_MEMBER]; /* Type-specific data */
} varattrib_1b_e;

4-byte header — the general case. The header stores the length in its upper 30 bits and flags in the low bits (compressed / external).
1-byte header — the “short” format, for payloads up to VARATT_SHORT_MAX (0x7F = 127) bytes. One bit of the header is reserved as a flag; the other seven hold the length.
1-byte header + tag — the external (TOAST pointer) format. Also a one-byte header, so TOAST pointers cost no alignment.

TOAST: when a row doesn’t fit

A tuple can be at most MaxHeapTupleSize:

#define MaxHeapTupleSize (BLCKSZ - MAXALIGN(SizeOfPageHeaderData + sizeof(ItemIdData)))

With BLCKSZ 8192, a 24-byte page header and a 4-byte line pointer, that’s 8192 − 32 = 8160 bytes.

But TOAST kicks in much earlier. From src/include/access/heaptoast.h:

 c
 
 
/*
 * Find the maximum size of a tuple if there are to be N tuples per page.
 */
#define MaximumBytesPerTuple(tuplesPerPage) \
	MAXALIGN_DOWN((BLCKSZ - \
				 MAXALIGN(SizeOfPageHeaderData + (tuplesPerPage) * sizeof(ItemIdData))) \
				 / (tuplesPerPage))

#define TOAST_TUPLES_PER_PAGE	4

#define TOAST_TUPLE_THRESHOLD	MaximumBytesPerTuple(TOAST_TUPLES_PER_PAGE)

#define TOAST_TUPLE_TARGET		TOAST_TUPLE_THRESHOLD

TOAST_TUPLE_THRESHOLD works out to 2032 bytes: the largest tuple such that four of them fit on a page. If a tuple exceeds that, the toaster takes over.

The toaster’s strategy, in order:

Compress every compressible column, in place. Compression is tried per-column, and a column is only kept compressed if it actually got smaller. A text column that compresses to 40% of its size stays compressed inline, with a 4-byte header that records both the compressed length and the original length.
Move EXTENDED and EXTERNAL columns out of line into the table’s TOAST table, replacing them in the row with an 18-byte pointer.
As a last resort, move PLAIN and MAIN columns out of line too, up to TOAST_TUPLE_TARGET_MAIN (8160 bytes — the largest tuple that fits on a page at all). The user said “keep this inline if at all possible”, so this only happens when nothing else works.

The TOAST pointer itself is the varattrib_1b_e format: a one-byte header, a tag byte, and a payload:

 c
 
 
typedef struct varatt_external_oid
{
	int32		va_rawsize;		/* Original data size (includes header) */
	uint32		va_extinfo;		/* External saved size (without header) and
								 * additional flag bits */
	Oid			va_valueid;		/* Unique ID of value within TOAST table */
	Oid			va_toastrelid;	/* RelID of TOAST table containing it */
} varatt_external_oid;

The key design decision: the pointer identifies the value by (toast table, value id), not by the physical location of the row that contains it. That is what makes TOAST pointers relocatable — a row can be moved by VACUUM FULL, CLUSTER, or ALTER TABLE ... SET TABLESPACE without rewriting its out-of-line data.

The TOAST table

Every table with toastable columns gets a companion table named pg_toast_<oid>, created in src/backend/catalog/toasting.c:

 c
 
 
	tupdesc = CreateTemplateTupleDesc(3);
	TupleDescInitEntry(tupdesc, (AttrNumber) 1,
					 "chunk_id",
					 toast_chunkid_typid,
					 -1, 0);
	TupleDescInitEntry(tupdesc, (AttrNumber) 2,
					 "chunk_seq",
					 INT4OID,
					 -1, 0);
	TupleDescInitEntry(tupdesc, (AttrNumber) 3,
					 "chunk_data",
					 BYTEAOID,
					 -1, 0);

Three columns: an identifier for the value, a chunk sequence number, and the chunk payload. Large values are split across multiple rows.

The chunk size is bounded by what has to fit on a page:

 c
 
 
#define TOAST_OID_MAX_CHUNK_SIZE	\
	(EXTERN_TUPLE_MAX_SIZE -							\
	 MAXALIGN(SizeofHeapTupleHeader) -					\
	 sizeof(Oid) -										\
	 sizeof(int32) -									\
	 VARHDRSZ)

EXTERN_TUPLE_MAX_SIZE is MaximumBytesPerTuple(4) = 2032, so the chunk payload is 2032 − 24 − 4 − 4 − 4 = 1996 bytes for an oid value id (and 1992 for the oid8 variant).

One last detail that is easy to miss:

 c
 
 
	/*
	 * Ensure that the toast table doesn't itself get toasted, or we'll be
	 * toast :-(. This is essential for chunk_data because type bytea is
	 * toastable; hit the other two just to be sure.
	 */
	TupleDescAttr(tupdesc, 0)->attstorage = TYPSTORAGE_PLAIN;
	TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
	TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;

The TOAST table is explicitly not toastable. A row cannot contain a pointer to a value that lives in a table that itself contains pointers to values.

UPDATE: the version chain

Now that we know what a tuple looks like, we can see what an UPDATE does. PostgreSQL never modifies a row in place (with one narrow exception for system catalogs). It writes a new version and links the old one to it.

The link is t_ctid. From the header comment:

A word about t_ctid: whenever a new tuple is stored on disk, its t_ctid is initialized with its own TID (location). If the tuple is ever updated, its t_ctid is changed to point to the replacement version of the tuple.

So t_ctid is either a self-reference (this is the newest version) or a pointer to a successor. The header comment also warns about the obvious hazard:

Beware however that VACUUM might erase the pointed-to (newer) tuple before erasing the pointing (older) tuple. Hence, when following a t_ctid link, it is necessary to check to see if the referenced slot is empty or contains an unrelated tuple. Check that the referenced tuple has XMIN equal to the referencing tuple’s XMAX to verify that it is actually the descendant version.

That XMIN/XMAX check is the chain-integrity test, and it appears everywhere — in the index scan, in the pruner, and in the row-locking code. It’s how a reader knows it hasn’t followed a link into a recycled slot.

The actual update, in heap_update() (src/backend/access/heap/heapam.c):

 c
 
 
	newtup->t_data->t_infomask &= ~(HEAP_XACT_MASK);
	newtup->t_data->t_infomask2 &= ~(HEAP2_XACT_MASK);
	HeapTupleHeaderSetXmin(newtup->t_data, xid);
	HeapTupleHeaderSetCmin(newtup->t_data, cid);
	newtup->t_data->t_infomask |= HEAP_UPDATED | infomask_new_tuple;
	newtup->t_data->t_infomask2 |= infomask2_new_tuple;
	HeapTupleHeaderSetXmax(newtup->t_data, xmax_new_tuple);

The new version gets the updater’s XID as its xmin. Then, at the very end of the operation, the old version is finalized:

 c
 
 
	/* Clear obsolete visibility flags, possibly set by ourselves above... */
	oldtup.t_data->t_infomask &= ~(HEAP_XMAX_BITS | HEAP_MOVED);
	oldtup.t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
	/* ... and store info about transaction updating this tuple */
	Assert(TransactionIdIsValid(xmax_old_tuple));
	HeapTupleHeaderSetXmax(oldtup.t_data, xmax_old_tuple);
	oldtup.t_data->t_infomask |= infomask_old_tuple;
	oldtup.t_data->t_infomask2 |= infomask2_old_tuple;
	HeapTupleHeaderSetCmax(oldtup.t_data, cid, iscombo);

	/* record address of new tuple in t_ctid of old one */
	oldtup.t_data->t_ctid = heaptup->t_self;

One line — oldtup.t_data->t_ctid = heaptup->t_self — is the entire mechanism of MVCC at the storage layer. The old version records where its successor lives, and the old version’s xmax records who did the update.

Notice also HEAP_XMAX_BITS, which is masked out before the new value is written:

 c
 
 
#define HEAP_XMAX_BITS (HEAP_XMAX_COMMITTED | HEAP_XMAX_INVALID | \
						HEAP_XMAX_IS_MULTI | HEAP_LOCK_MASK | HEAP_XMAX_LOCK_ONLY)

Everything about xmax is recomputed from scratch on every write, because the lock state can change independently of the row’s identity.

Visibility: the rule

Visibility is decided by HeapTupleSatisfiesMVCC(), and the rule is compact:

the tuple is visible iff xmin is committed-and-not-in-snapshot AND (xmax is invalid / lock-only / in-snapshot-and-not-committed)

The Cmin/Cmax comparison against the snapshot’s curcid is what distinguishes “my own earlier command updated this row” from “my own later command will update this row” — the TM_SelfModified versus TM_Invisible split. And one detail that is easy to miss:

 c
 
 
	if (tuple->t_infomask & HEAP_XMAX_COMMITTED)
	{
		if (HEAP_XMAX_IS_LOCKED_ONLY(tuple->t_infomask))
			return TM_Ok;
		if (!ItemPointerEquals(&htup->t_self, &tuple->t_ctid))
			return TM_Updated;	/* updated by other */
		else
			return TM_Deleted;	/* deleted by other */
	}

The same t_ctid field that carries the HOT chain also distinguishes UPDATE from DELETE. t_ctid != t_self means “this version was updated”; t_ctid == t_self means “this version was deleted”. One field, two questions, answered by the same comparison.

HOT: updating without touching the index

The obvious cost of an UPDATE is that it creates a new index entry for the new version. Heap-Only Tuples avoid that. From the manual:

To help reduce the overhead of updates, PostgreSQL has an optimization called heap-only tuples (HOT).

HOT applies when both of these hold:

The update does not modify any column referenced by an index (excluding summarizing indexes like BRIN).
There is sufficient free space on the page containing the old row.

Then the new version is written to the same page, no index entry is created, and the chain is walked from the root tuple via t_ctid. The flags make this possible:

 c
 
 
	if (use_hot_update)
	{
		/* Mark the old tuple as HOT-updated */
		HeapTupleSetHotUpdated(&oldtup);
		/* And mark the new tuple as heap-only */
		HeapTupleSetHeapOnly(heaptup);
		/* Mark the caller's copy too, in case different from heaptup */
		HeapTupleSetHeapOnly(newtup);
	}

HEAP_HOT_UPDATED marks the predecessor (“my successor is heap-only”); HEAP_ONLY_TUPLE marks the successor (“no index entry points at me”). The chain looks like this:

text

 btree index entry ──► (blk, offnum=1) root tuple
 t_ctid = (blk, offnum=2)
 t_infomask2 |= HEAP_HOT_UPDATED
 lp[2] ──► version 2
 t_ctid = (blk, offnum=3)
 t_infomask2 |= HEAP_HOT_UPDATED
 lp[3] ──► version 3
 t_ctid = (blk, offnum=3) (self — end)
 t_infomask2 |= HEAP_ONLY_TUPLE

The reader follows the chain in heap_hot_search_buffer() (src/backend/access/heap/heapam_indexscan.c):

 c
 
 
		/* Check to see if HOT chain continues past this tuple; if so fetch
		 * the next offnum and loop around.
		 */
		if (HeapTupleIsHotUpdated(heapTuple))
		{
			Assert(ItemPointerGetBlockNumber(&heapTuple->t_data->t_ctid) ==
				 blkno);
			offnum = ItemPointerGetOffsetNumber(&heapTuple->t_data->t_ctid);
			at_chain_start = false;
			prev_xmax = HeapTupleHeaderGetUpdateXid(heapTuple->t_data);
		}
		else
			break;				/* end of chain */

Note the assertion: a HOT chain never crosses a page boundary. The successor is always in the same block. That is why HOT requires free space on the same page — a chain that hopped pages would need a different mechanism to find the next version.

The chain-integrity check follows immediately after:

 c
 
 
		/*
		 * The xmin should match the previous xmax value, else chain is
		 * broken.
		 */
		if (TransactionIdIsValid(prev_xmax) &&
			!TransactionIdEquals(prev_xmax,
								 HeapTupleHeaderGetXmin(heapTuple->t_data)))
			break;

Pruning: reclaiming the space

Dead versions accumulate. When a row is updated ten times, there are ten tuples on the page, and nine of them are garbage once no transaction can see them.

Pruning happens opportunistically, during ordinary scans, via heap_page_prune_opt(). The trigger is fillfactor-aware:

 c
 
 
	minfree = RelationGetTargetPageFreeSpace(relation,
											 HEAP_DEFAULT_FILLFACTOR);
	minfree = Max(minfree, BLCKSZ / 10);

	if (PageIsFull(page) || PageGetHeapFreeSpace(page) < minfree)

and it needs a cleanup lock, not merely an exclusive lock, because pruning can destroy line pointers other backends may hold references to.

The chain surgery is in heap_prune_chain(). Its header comment is the clearest statement of the algorithm in the whole tree:

The root line pointer is redirected to the tuple immediately after the latest DEAD tuple. If all tuples in the chain are DEAD, the root line pointer is marked LP_DEAD.

Three outcomes:

Situation	Root line pointer	Intermediate line pointers
No dead member	unchanged	unchanged
All members dead	→ `LP_DEAD`	→ `LP_UNUSED`
Some dead prefix	→ `LP_REDIRECT`	→ `LP_UNUSED`

The important subtlety: the t_ctid links are not rewritten during pruning. The chain is repaired by replacing the root line pointer’s storage with a redirect, not by patching t_ctid:

 c
 
		ItemIdSetRedirect(fromlp, tooff);

with

 c
 
 
#define ItemIdSetRedirect(itemId, link) \
( \
	(itemId)->lp_flags = LP_REDIRECT, \
	(itemId)->lp_off = (link), \
	(itemId)->lp_len = 0 \
)

The root keeps its offset number, so index entries pointing at it stay valid, but loses its storage (lp_len = 0) and repurposes lp_off to hold the offset number of the target. The reader follows the redirect at the start of a chain:

 c
 
 
		/* check for unused, dead, or redirected items */
		if (!ItemIdIsNormal(lp))
		{
			/* We should only see a redirect at start of chain */
			if (ItemIdIsRedirected(lp) && at_chain_start)
			{
				/* Follow the redirect */
				offnum = ItemIdGetRedirect(lp);
				at_chain_start = false;
				continue;
			}
			/* else must be end of chain */
			break;
		}

Now, the freed tuple bytes are still sitting there in the middle of the page, unreachable. PageRepairFragmentation() moves the survivors downward to close the gap:

 c
 
 
		/* shift the target offset down by the length of this tuple */
		upper -= itemidptr->alignedlen;
		/* point the copy_head to the start of this tuple */
		copy_head = itemidptr->itemoff;

		/* update the line pointer to reference the new offset */
		lp->lp_off = upper;

Note that lp_off is rewritten for every moved tuple. The line pointers are the only thing that has to stay stable; the tuples can go anywhere.

Two side structures

The free space map

The FSM is a per-relation array of pages, each page holding a tree of free-space values per heap page. It’s consulted when a tuple needs to be placed and the current target page is full, to find a page with room.

The visibility map

The VM is a bitmap with two bits per heap page — all-visible and all-frozen:

 c
 
 
/* Number of bits for one heap page */
#define BITS_PER_HEAPBLOCK 2

/* Flags for bit map */
#define VISIBILITYMAP_ALL_VISIBLE	0x01
#define VISIBILITYMAP_ALL_FROZEN	0x02

The VM page is nearly all bitmap:

 c
 
 
/*
 * Size of the bitmap on each visibility map page, in bytes. There's no
 * extra headers, so the whole page minus the standard page header is
 * used for the bitmap.
 */
#define MAPSIZE (BLCKSZ - MAXALIGN(SizeOfPageHeaderData))

8168 bytes of bitmap per VM page, four heap pages per byte, so 32 672 heap pages per VM page. The VM fork is about 1/32672 the size of the heap fork.

When every tuple on a heap page is visible to everyone, that page’s all-visible bit is set, and readers can skip the page entirely. This is why SELECT count(*) on an untouched table is nearly instant.

Look at it yourself

The pageinspect extension exposes all of this. It lives in contrib/pageinspect/ and is not installed by default. The raw-page functions require superuser:

 sql
 
 
CREATE EXTENSION pageinspect;

-- the line pointer array, and the header of every tuple on the page
SELECT lp, lp_off, lp_len, lp_flags, lp_flags = 1 AS normal,
 t_xmin, t_xmax, t_ctid, t_hoff
FROM heap_page_items(get_raw_page('mytable', 0));

-- the decoded flag names
SELECT (heap_tuple_infomask_flags(t_infomask, t_infomask2)).*
FROM heap_page_items(get_raw_page('mytable', 0));

-- where does each column live inside the tuple
SELECT t_ctid, s.*
FROM heap_page_items(get_raw_page('mytable', 0)) h,
 LATERAL unnest(tuple_data_split('mytable'::regclass,
 h.t_data, h.t_infomask,
 h.t_infomask2, h.t_bits)) s;

heap_page_items gives you the line pointer array and the raw tuple headers; tuple_data_split gives you the byte range of every column within a tuple. These two together are enough to verify everything in this article by hand.

What it all means in practice

A few consequences fall out of the layout, and they’re worth having in your head when you’re reasoning about a slow table:

Row width is expensive in a way that row count is not. The 23-byte header is paid by every row, but the alignment rules mean a table with one wide text column can waste more space in padding than a table with many narrow columns. MaxHeapTuplesPerPage is 291 on an 8 kB page:

 c
 
 
#define MaxHeapTuplesPerPage	\
	((int) ((BLCKSZ - SizeOfPageHeaderData) / \
			(MAXALIGN(SizeofHeapTupleHeader) + sizeof(ItemIdData))))

(8192 − 24) / (24 + 4) = 291. A page of 291 minimal rows has 291 tuples of 24 bytes plus 291 line pointers of 4 bytes, in 8192 bytes.

HOT updates are dramatically cheaper than cold ones. If you can update a row without touching an indexed column, and there’s room on the page, PostgreSQL writes a new tuple and nothing else. No index churn, no dead-index-entry cleanup. This is why fillfactor matters: a lower fillfactor leaves room on the page for the new version, and the chance of a HOT update goes up.

The 4-byte line pointer is a per-row tax on every page. It is the price of the indirection that makes VACUUM, HOT, and page defragmentation possible. It’s also why MaxHeapTupleSize is 8160 and not 8168.

An UPDATE never reuses the old tuple’s bytes. It appends a new version and leaves the old one for VACUUM to reclaim. A table that is updated constantly will bloat until vacuum catches up, and the bloat is not fragmentation — it’s whole tuples.

The t_ctid field is the load-bearing structure. It answers three questions at once: where is the newest version of this row, was this version updated or deleted, and where does the next HOT chain member live. Six bytes, three jobs.

PostgreSQL's time and timetz Implementation in C

Jay Hubbert — Mon, 12 Oct 2026 10:21:30 +0000

A PostgreSQL time is a 64-bit integer counting microseconds since midnight. A timetz is the same integer plus a 4-byte timezone offset. So far so simple. The interesting part is the mismatch: the in-memory struct TimeTzADT is 16 bytes, but the type declares itself 12 bytes wide, and both numbers are correct — one is the padded struct, the other is what actually gets copied into a tuple.

This is a walk through the C implementation, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entries

From src/include/catalog/pg_type.dat, time (lines 291–295) and timetz (lines 322–328):

 text
 
 
{ oid => '1083', array_type_oid => '1183', descr => 'time of day',
 typname => 'time', typlen => '8', typbyval => 't', typcategory => 'D',
 typinput => 'time_in', typoutput => 'time_out', typreceive => 'time_recv',
 typsend => 'time_send', typmodin => 'timetypmodin',
 typmodout => 'timetypmodout', typalign => 'd' },

{ oid => '1266', array_type_oid => '1270',
 descr => 'time of day with time zone',
 typname => 'timetz', typlen => '12', typbyval => 'f', typcategory => 'D',
 typinput => 'timetz_in', typoutput => 'timetz_out',
 typreceive => 'timetz_recv', typsend => 'timetz_send',
 typmodin => 'timetztypmodin', typmodout => 'timetztypmodout',
 typalign => 'd' },

The contrast is the whole story:

Field	`time`	`timetz`
`typlen`	`8`	`12`
`typbyval`	`t` (by value)	`f` (by reference)
`typalign`	`d` (8 bytes)	`d` (8 bytes)
array oid	`1183`	`1270`

time is an 8-byte integer passed by value. timetz is a 12-byte value passed by reference — and that combination is where the padding trap lives.

The C representation

From src/include/utils/date.h:

 c
 
 
typedef int64 TimeADT;

typedef struct
{
	TimeADT		time;			/* all time units other than months and years */
	int32		zone;			/* numeric time zone, in seconds */
} TimeTzADT;

/*
 * sizeof(TimeTzADT) will be 16 on most platforms due to alignment padding.
 * However, timetz's typlen is 12 according to pg_type. In most places
 * we can get away with using sizeof(TimeTzADT), but where it's important
 * to match the declared typlen, use TIMETZ_TYPLEN.
 */
#define TIMETZ_TYPLEN		12

time occupies bytes 0–7 and zone bytes 8–11, so the useful data is exactly 12 bytes. But the compiler pads the struct to 16 so that an int64 member stays 8-byte aligned. The TIMETZ_TYPLEN macro exists precisely to keep the two numbers straight.

Microseconds since midnight

time stores h/m/s as a flat microsecond count. The conversion is a one-liner (tm2time()):

 c
 
 
int
tm2time(struct pg_tm *tm, fsec_t fsec, TimeADT *result)
{
	*result = ((((tm->tm_hour * MINS_PER_HOUR + tm->tm_min) * SECS_PER_MINUTE) + tm->tm_sec)
			 * USECS_PER_SEC) + fsec;
	return 0;
}

The constants come from src/include/datatype/timestamp.h: USECS_PER_SEC = 1000000, USECS_PER_DAY = 86400000000. The inverse, time2tm(), divides the count back out into hour, minute, second, and fraction.

time_in() parses with DecodeTimeOnly() (the time-only sibling of DecodeDateTime()), then calls tm2time() and AdjustTimeForTypmod(). time_out() calls time2tm() and EncodeTimeOnly(). The binary format is just the 8 bytes: time_recv() uses pq_getmsgint64(), time_send() uses pq_sendint64().

The range check is generous on purpose. time_overflows() allows hour up to 24 and sec up to 60, then verifies the total does not exceed USECS_PER_DAY:

 c
 
 
	/*
	 * Because we allow, eg, hour = 24 or sec = 60, we must check separately
	 * that the total time value doesn't exceed 24:00:00.
	 */
	if ((((((hour * MINS_PER_HOUR + min) * SECS_PER_MINUTE)
		 + sec) * USECS_PER_SEC) + fsec) > USECS_PER_DAY)
		return true;

So '24:00:00'::time is a legal value.

Typmod and rounding

Both types accept a precision modifier, capped at MAX_TIME_PRECISION = 6 microsecond digits. AdjustTimeForTypmod() rounds (it does not truncate) using a scale and offset table:

 c
 
 
	static const int64 TimeScales[MAX_TIME_PRECISION + 1] = {
		INT64CONST(1000000), INT64CONST(100000), INT64CONST(10000),
		INT64CONST(1000), INT64CONST(100), INT64CONST(10), INT64CONST(1)
	};

	static const int64 TimeOffsets[MAX_TIME_PRECISION + 1] = {
		INT64CONST(500000), INT64CONST(50000), INT64CONST(5000),
		INT64CONST(500), INT64CONST(50), INT64CONST(5), INT64CONST(0)
	};

	if (typmod >= 0 && typmod <= MAX_TIME_PRECISION)
	{
		if (*time >= INT64CONST(0))
			*time = ((*time + TimeOffsets[typmod]) / TimeScales[typmod]) *
				TimeScales[typmod];
		else
			*time = -((((-*time) + TimeOffsets[typmod]) / TimeScales[typmod]) *
					 TimeScales[typmod]);
	}

The offset is half a unit of the target precision, which is why '12:00:00.123456'::time(3) rounds to .123 but .1235 rounds up. The negative branch negates, rounds, and re-negates, so rounding is symmetric around zero.

timetz: the 12-byte puzzle

timetz reuses the time machinery and adds zone, an int32 offset in seconds. tm2timetz() is tm2time() plus one assignment:

 c
 
 
int
tm2timetz(struct pg_tm *tm, fsec_t fsec, int tz, TimeTzADT *result)
{
	result->time = ((((tm->tm_hour * MINS_PER_HOUR + tm->tm_min) * SECS_PER_MINUTE) + tm->tm_sec) *
					USECS_PER_SEC) + fsec;
	result->zone = tz;

	return 0;
}

The offset is bounded by TZDISP_LIMIT (src/include/datatype/timestamp.h):

 c
 
 
#define MAX_TZDISP_HOUR		15	/* maximum allowed hour part */
#define TZDISP_LIMIT		((MAX_TZDISP_HOUR + 1) * SECS_PER_HOUR)

timetz_recv() rejects anything at or beyond that bound, and timetz_send() writes the 8-byte time followed by the 4-byte zone — 12 bytes total.

Here is the subtlety. timetz is typbyval = f, so a Datum holds a pointer. In a heap tuple, though, a fixed-length by-reference attribute is stored inline. In fill_val() (src/backend/access/common/heaptuple.c):

 c
 
 
	else
	{
		/* fixed-length pass-by-reference */
		data = (char *) att_nominal_alignby(data, att->attalignby);
		Assert(att->attlen > 0);
		data_length = att->attlen;
		memcpy(data, DatumGetPointer(datum), data_length);
	}

att->attlen is 12, so only 12 bytes are copied out of the 16-byte TimeTzADT. The 4 padding bytes are simply dropped. When the value is read back, fetch_att() returns PointerGetDatum(T) — a pointer straight into the tuple’s data area — and the caller casts it to TimeTzADT *. That is safe only because time and zone live in the first 12 bytes.

The declared typalign is d (8 bytes), so the field starts at an 8-byte-aligned offset and the next attribute is aligned again, giving an effective 16-byte stride. The on-disk representation is 12 bytes; the in-memory struct is 16. Both are correct, for different jobs.

Comparing timetz

timetz cannot be compared as a plain integer, because two values with the same local time but different zones are different instants. The comparator normalizes to a GMT-equivalent time-of-day first (timetz_cmp_internal()):

 c
 
 
	/* Primary sort is by true (GMT-equivalent) time */
	t1 = time1->time + (time1->zone * USECS_PER_SEC);
	t2 = time2->time + (time2->zone * USECS_PER_SEC);

	if (t1 > t2)
		return 1;
	if (t1 < t2)
		return -1;

	/*
	 * If same GMT time, sort by timezone; we only want to say that two
	 * timetz's are equal if both the time and zone parts are equal.
	 */
	if (time1->zone > time2->zone)
		return 1;
	...

The sign convention is worth pinning down: zone is stored in seconds west of Greenwich (positive west, the SQL convention), and EncodeTimezone() negates it for display (tz <= 0 ? '+' : '-'). Adding zone * USECS_PER_SEC to the local time therefore yields the UTC time-of-day, which is what “true” means here. Two timetz values are equal only when both the normalized time and the raw zone match.

Arithmetic

time has no infinity sentinels, but it does wrap. time_pl_interval() adds an interval’s time field and folds the result back into a day:

 c
 
 
	result = time + span->time;
	result -= result / USECS_PER_DAY * USECS_PER_DAY;
	if (result < INT64CONST(0))
		result += USECS_PER_DAY;

interval_time() does the same modulo fold, and its comment explains the choice for negative values: '-2 hours' becomes '22:00:00'. time_mi_time() returns an Interval whose time field is the raw difference.

Look at it yourself

The catalog rows make the layout explicit:

 sql
 
 
SELECT typname, typlen, typbyval, typalign, typarray
FROM pg_type
WHERE typname IN ('date', 'time', 'timetz');

pageinspect will show you the exact byte counts — 8 for time, 12 for timetz:

 sql
 
 
CREATE EXTENSION pageinspect;
CREATE TABLE t(tt time, tz timetz);
INSERT INTO t VALUES ('12:34:56.789', '12:34:56+02');

SELECT s.t_data
FROM heap_page_items(get_raw_page('t', 0)) h,
 LATERAL unnest(tuple_data_split('t'::regclass, h.t_data,
 h.t_infomask, h.t_infomask2,
 h.t_bits)) AS s(t_data);

The zone sign is visible in output, and the precision rounding is visible too:

 sql
 
 
SELECT '12:34:56.789'::timetz; -- 12:34:56.789+00
SELECT '12:00:00.1235'::time(3); -- 12:00:00.124
SELECT '24:00:00'::time; -- 24:00:00

What it all means

time is int64 microseconds since midnight; timetz adds a 4-byte zone offset in seconds.
time is typlen 8, typbyval t, typalign d; timetz is typlen 12, typbyval f, typalign d.
The TimeTzADT struct is 16 bytes with padding, but only 12 bytes are stored in a tuple; the padding is dropped and the pointer reads back into the first 12 bytes.
Precision (typmod) rounds half-up, capped at 6 digits.
timetz comparison normalizes to UTC time-of-day, then breaks ties by raw zone, so equality requires both to match.
time arithmetic wraps modulo one day, not modulo 24:00:00 as an instant.

PostgreSQL Speeds Up Hash Joins

Jay Hubbert — Sun, 11 Oct 2026 09:00:00 +0000

A hash join is the simplest join algorithm in the database: build a hash table from the inner relation, then probe it once per outer row. There is no sort, no merge, no nested loop over the inner side.

And yet it is one of the most performance-sensitive pieces of the executor, because it does the same tiny bit of work an enormous number of times: compute a hash value, look it up, compare the key. For a join between a million-row hash table and a ten-million-row outer relation, that is ten million hash computations and ten million bucket scans, and each one is measured in tens of nanoseconds. Everything else is noise.

So PostgreSQL has spent the last few years shaving that loop down. The strategy has been consistent: move the hashing out of the general-purpose expression interpreter and into specialised C functions, selected by pattern-matching the compiled expression’s step list at startup time.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

Where the time goes

A hash join has two sides and two kinds of work.

Building the hash table. Every inner row’s join key has to be hashed and the row inserted into a bucket chain. This is done once per inner row.

Probing. Every outer row’s join key has to be hashed, the resulting bucket found, and the bucket chain walked, comparing keys until a match is found. This is done once per outer row, and it is the side that dominates: a hash join between a small inner table and a huge outer relation does enormously more probing than building.

Both sides need the same hash function for a given join key, and PostgreSQL has to compute it from an arbitrary expression (the join key can be a.b + 1, or lower(name), or a plain column).

That expression is evaluated by the expression interpreter (src/backend/executor/execExprInterp.c), which compiles an expression tree into a flat list of steps and then runs them through a threaded dispatch loop. The interpreter is fast, but it has a fixed per-step overhead, and for a hash join the step list for “hash column 3” is tiny — four or five steps for what is, in machine code, about six instructions.

The opportunity is obvious: if you know the shape of the step list before you run it, you can write a function that does the whole thing without a dispatch loop at all.

Hashing as an expression

The first move was to make hashing an expression operation at all. Commit adf97c15623 (“Speed up Hash Join by making ExprStates support hashing”, August 2024) introduced a family of new opcodes and the ExecBuildHash32Expr() constructor. Its commit message explains the motivation:

This allows both faster hashing and also JIT compilation of these expressions. This is especially useful when hash joins have multiple join keys as the previous code called ExecEvalExpr on each hash join key individually and that was inefficient as tuple deformation would have only taken into account one key at a time, which could lead to walking the tuple once for each join key. With the new code, we’ll determine the maximum attribute required and deform the tuple to that point only once.

Some performance tests done with this change have shown up to a 20% performance increase of a query containing a Hash Join without JIT compilation and up to a 26% performance increase when JIT is enabled and optimization and inlining were performed by the JIT compiler.

The opcodes themselves, from src/include/executor/execExpr.h:

 c
 
 
	/* evaluation steps for hashing */
	EEOP_HASHDATUM_SET_INITVAL,
	EEOP_HASHDATUM_FIRST,
	EEOP_HASHDATUM_FIRST_STRICT,
	EEOP_HASHDATUM_NEXT32,
	EEOP_HASHDATUM_NEXT32_STRICT,

Five opcodes, with a clear division of labour:

SET_INITVAL seeds the hash with a caller-supplied 32-bit value.
FIRST hashes the first key and replaces the running value.
NEXT32 hashes the next key and combines it with the running value.
The _STRICT variants abort the whole computation when the key is NULL, instead of treating NULL as a zero hash.

The combination is a rotate-then-XOR, from the interpreter:

 c
 
 
		EEO_CASE(EEOP_HASHDATUM_NEXT32)
		{
			FunctionCallInfo fcinfo = op->d.hashdatum.fcinfo_data;
			uint32		existinghash;

			existinghash = DatumGetUInt32(op->d.hashdatum.iresult->value);
			/* combine successive hash values by rotating */
			existinghash = pg_rotate_left32(existinghash, 1);

			/* leave the hash value alone on NULL inputs */
			if (!fcinfo->args[0].isnull)
			{
				uint32		hashvalue;

				/* execute hash func and combine with previous hash value */
				hashvalue = DatumGetUInt32(op->d.hashdatum.fn_addr(fcinfo));
				existinghash = existinghash ^ hashvalue;
			}

			*op->resvalue = UInt32GetDatum(existinghash);
			*op->resnull = false;

			EEO_NEXT();
		}

And the FIRST variant, which has to cope with NULL by storing zero so that a subsequent NEXT32 still has something to combine with:

 c
 
 
		EEO_CASE(EEOP_HASHDATUM_FIRST)
		{
			FunctionCallInfo fcinfo = op->d.hashdatum.fcinfo_data;

			/*
			 * Save the Datum on non-null inputs, otherwise store 0 so that
			 * subsequent NEXT32 operations combine with an initialized value.
			 */
			if (!fcinfo->args[0].isnull)
				*op->resvalue = op->d.hashdatum.fn_addr(fcinfo);
			else
				*op->resvalue = (Datum) 0;

			*op->resnull = false;

			EEO_NEXT();
		}

Building the hash expression

ExecBuildHash32Expr() (src/backend/executor/execExpr.c) is the constructor. It takes the list of key expressions, the hash functions to use, and builds a step list:

 c
 
 
ExprState *
ExecBuildHash32Expr(TupleDesc desc, const TupleTableSlotOps *ops,
					const Oid *hashfunc_oids, const List *collations,
					const List *hash_exprs, const bool *opstrict,
					PlanState *parent, uint32 init_value)
{
	ExprState *state = makeNode(ExprState);
	ExprEvalStep scratch = {0};
	NullableDatum *iresult = NULL;
	List	 *adjust_jumps = NIL;
	ListCell *lc;
	ListCell *lc2;
	intptr_t	strict_opcode;
	intptr_t	opcode;
	int			num_exprs = list_length(hash_exprs);

	Assert(num_exprs == list_length(collations));

	state->parent = parent;

	/* Insert setup steps as needed. */
	ExecCreateExprSetupSteps(state, unconstify(Node *, (const Node *) hash_exprs));

	/*
	 * Make a place to store intermediate hash values between subsequent
	 * hashing of individual expressions. We only need this if there is more
	 * than one expression to hash or an initial value plus one expression.
	 */
	if ((int64) num_exprs + (init_value != 0) > 1)
		iresult = palloc_object(NullableDatum);

The iresult slot is where the running hash value lives between keys. Notice the comment in the header of the function:

If any hash_expr yields NULL and the corresponding hash operator is strict, the created ExprState will return NULL. (If the operator is not strict, we treat NULL values as having a hash value of zero. The hash functions themselves are always treated as strict.)

The two modes matter, and the difference is the subject of a bug we will come back to.

The hash join node wires it up in ExecInitHashJoin() (src/backend/executor/nodeHashjoin.c):

 c
 
 
		/*
		 * Build ExprStates to obtain hash values for either side of the join.
		 * Note: must build the ExprStates before ExecHashTableCreate() so we
		 * properly attribute any SubPlans that exist in the hash expressions
		 * to the correct PlanState.
		 */
		nkeys = list_length(node->hashoperators);

		outer_hashfuncid = palloc_array(Oid, nkeys);
		inner_hashfuncid = palloc_array(Oid, nkeys);
		hash_strict = palloc_array(bool, nkeys);
		...
		/*
		 * Build an ExprState to generate the hash value for the expressions
		 * on the outer side of the join.
		 */
		hjstate->hj_OuterHash =
			ExecBuildHash32Expr(hjstate->js.ps.ps_ResultTupleDesc,
								hjstate->js.ps.resultops,
								outer_hashfuncid,
								node->hashcollations,
								node->hashkeys,
								hash_strict,
								&hjstate->js.ps,
								0);

		/* As above, but for the inner side of the join */
		hashstate->hash_expr =
			ExecBuildHash32Expr(hashstate->ps.ps_ResultTupleDesc,
								hashstate->ps.resultops,
								inner_hashfuncid,
								node->hashcollations,
								hash->hashkeys,
								hash_strict,
								&hashstate->ps,
								0);

Two ExprStates, one per side, each evaluating a list of key expressions into a single 32-bit hash.

The inner side is evaluated when the hash table is built (src/backend/executor/nodeHash.c):

 c
 
 
		slot = ExecProcNode(outerNode);
		if (TupIsNull(slot))
			break;
		/* We have to compute the hash value */
		econtext->ecxt_outertuple = slot;

		ResetExprContext(econtext);

		hashdatum = ExecEvalExprSwitchContext(node->hash_expr, econtext,
											 &isnull);

		if (!isnull)
		{
			/* normal case with a non-null join key */
			uint32		hashvalue = DatumGetUInt32(hashdatum);
			int			bucketNumber;

			bucketNumber = ExecHashGetSkewBucket(hashtable, hashvalue);
			if (bucketNumber != INVALID_SKEW_BUCKET_NO)
			{
				/* It's a skew tuple, so put it into that hash table */
				ExecHashSkewTableInsert(hashtable, slot, hashvalue,
										bucketNumber);
			}
			else
			{
				/* Not subject to skew optimization, so insert normally */
				ExecHashTableInsert(hashtable, slot, hashvalue);
			}
			hashtable->totalTuples += 1;
		}

And the outer side, once per probe, in nodeHashjoin.c:

 c
 
 
			*hashvalue = DatumGetUInt32(ExecEvalExprSwitchContext(hjstate->hj_OuterHash,
																 econtext,
																 &isnull));

ExecEvalExprSwitchContext() is a thin wrapper that switches to the per-tuple memory context and then calls state->evalfunc():

 c
 
 
static inline Datum
ExecEvalExprSwitchContext(ExprState *state,
						 ExprContext *econtext,
						 bool *isNull)
{
	Datum		retDatum;
	MemoryContext oldContext;

	oldContext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);
	retDatum = state->evalfunc(state, econtext, isNull);
	MemoryContextSwitchTo(oldContext);
	return retDatum;
}

So state->evalfunc is the hook. Whatever it points at is what runs ten million times.

The interpreter

Before the optimisation, evalfunc always pointed at ExecInterpExpr(), the threaded interpreter. Its header comment explains the dispatch strategy:

This file provides either a “direct threaded” (for gcc, clang and compatible) or a “switch threaded” (for all compilers) implementation of expression evaluation. The former is amongst the fastest known methods of interpreting programs without resorting to assembly level work, or just-in-time compilation, but it requires support for computed gotos.

In direct threading, we use gcc’s label-as-values extension - also adopted by some other compilers - to replace ExprEvalStep->opcode with the address of the block implementing the instruction. Dispatch to the next instruction is done by a “computed goto”. This allows for better branch prediction (as the jumps are happening from different locations) and fewer jumps (as no preparatory jump to a common dispatch location is needed).

Direct threading is the classic “threaded code” technique: each opcode’s implementation ends with a jump to the next opcode’s implementation, so there is no central dispatch point at all. The interpreter is about as fast as an interpreter gets.

But it is still an interpreter. Every step pays for the dispatch.

And here is the key observation from ExecReadyInterpretedExpr():

For very simple instructions the overhead of the full interpreter “startup”, as minimal as it is, is noticeable. Therefore ExecReadyInterpretedExpr will choose to implement certain simple opcode patterns using special fast-path routines (ExecJust*).

Pattern-matching the step list

ExecReadyInterpretedExpr() (src/backend/executor/execExprInterp.c) runs once, when the expression is prepared. Its job is to look at the compiled step list and decide: “is this one of the shapes I know how to do without an interpreter?”

 c
 
 
	/*
	 * Select fast-path evalfuncs for very simple expressions. "Starting up"
	 * the full interpreter is a measurable overhead for these, and these
	 * patterns occur often enough to be worth optimizing.
	 */
	if (state->steps_len == 5)
	{
		ExprEvalOp	step0 = state->steps[0].opcode;
		ExprEvalOp	step1 = state->steps[1].opcode;
		ExprEvalOp	step2 = state->steps[2].opcode;
		ExprEvalOp	step3 = state->steps[3].opcode;

		if (step0 == EEOP_INNER_FETCHSOME &&
			step1 == EEOP_HASHDATUM_SET_INITVAL &&
			step2 == EEOP_INNER_VAR &&
			step3 == EEOP_HASHDATUM_NEXT32)
		{
			state->evalfunc_private = (void *) ExecJustHashInnerVarWithIV;
			return;
		}
	}
	else if (state->steps_len == 4)
	{
		ExprEvalOp	step0 = state->steps[0].opcode;
		ExprEvalOp	step1 = state->steps[1].opcode;
		ExprEvalOp	step2 = state->steps[2].opcode;

		if (step0 == EEOP_OUTER_FETCHSOME &&
			step1 == EEOP_OUTER_VAR &&
			step2 == EEOP_HASHDATUM_FIRST)
		{
			state->evalfunc_private = (void *) ExecJustHashOuterVar;
			return;
		}
		else if (step0 == EEOP_INNER_FETCHSOME &&
				 step1 == EEOP_INNER_VAR &&
				 step2 == EEOP_HASHDATUM_FIRST)
		{
			state->evalfunc_private = (void *) ExecJustHashInnerVar;
			return;
		}
		else if (step0 == EEOP_OUTER_FETCHSOME &&
				 step1 == EEOP_OUTER_VAR &&
				 step2 == EEOP_HASHDATUM_FIRST_STRICT)
		{
			state->evalfunc_private = (void *) ExecJustHashOuterVarStrict;
			return;
		}
	}
	else if (state->steps_len == 3)
	{
		...
		else if (step0 == EEOP_INNER_VAR &&
				 step1 == EEOP_HASHDATUM_FIRST)
		{
			state->evalfunc_private = (void *) ExecJustHashInnerVarVirt;
			return;
		}
		else if (step0 == EEOP_OUTER_VAR &&
				 step1 == EEOP_HASHDATUM_FIRST)
		{
			state->evalfunc_private = (void *) ExecJustHashOuterVarVirt;
			return;
		}
	}

This is the whole trick, and it is worth reading slowly.

It matches on steps_len first. The step count is a cheap necessary condition, and it buckets the cases: five steps, four steps, three steps, two steps.

It then matches on the exact opcode sequence. Not “contains a HASHDATUM somewhere” — the exact sequence, in order. EEOP_INNER_FETCHSOME, EEOP_INNER_VAR, EEOP_HASHDATUM_FIRST and nothing else.

Note what is not matched. There is no pattern for EEOP_INNER_FETCHSOME, EEOP_INNER_VAR, EEOP_HASHDATUM_NEXT32 — the non-strict “next key” case. There is no pattern with two key expressions. There is no pattern for a hash key that is an expression rather than a plain column. Every one of those falls through to the general interpreter.

The last step is always a terminator — EEOP_DONE_RETURN or EEOP_DONE_NO_RETURN, which is asserted at the top of ExecReadyInterpretedExpr(). That is why a four-opcode expression has steps_len == 5 (four real opcodes plus the terminator). The count includes the terminator.

The result is stored in evalfunc_private, not evalfunc. Look at the surrounding code:

 c
 
 
	/*
	 * First time through, check whether attribute matches Var. Might not be
	 * ok anymore, due to schema changes. We do that by setting up a callback
	 * that does checking on the first call, which then sets the evalfunc
	 * callback to the actual method of execution.
	 */
	state->evalfunc = ExecInterpExprStillValid;

The first call goes through a validity check (the schema might have changed since the plan was built), and only then does the real function get installed:

 c
 
 
Datum
ExecInterpExprStillValid(ExprState *state, ExprContext *econtext, bool *isNull)
{
	/*
	 * First time through, check whether attribute matches Var. Might not be
	 * ok anymore, due to schema changes.
	 */
	CheckExprStillValid(state, econtext);

	/* skip the check during further executions */
	state->evalfunc = (ExprStateEvalFunc) state->evalfunc_private;

	/* and actually execute */
	return state->evalfunc(state, econtext, isNull);
}

So evalfunc_private holds the chosen implementation, and evalfunc holds the trampoline that installs it. Elegant, and it means the one-time schema check is not paid on every call.

The specialised functions

Now the payoff. The four-step inner case:

 c
 
 
/* implementation of ExecJustHash(Inner|Outer)Var */
static pg_always_inline Datum
ExecJustHashVarImpl(ExprState *state, TupleTableSlot *slot, bool *isnull)
{
	ExprEvalStep *fetchop = &state->steps[0];
	ExprEvalStep *var = &state->steps[1];
	ExprEvalStep *hashop = &state->steps[2];
	FunctionCallInfo fcinfo = hashop->d.hashdatum.fcinfo_data;
	int			attnum = var->d.var.attnum;

	CheckOpSlotCompatibility(fetchop, slot);
	slot_getsomeattrs(slot, fetchop->d.fetch.last_var);

	fcinfo->args[0].value = slot->tts_values[attnum];
	fcinfo->args[0].isnull = slot->tts_isnull[attnum];

	*isnull = false;

	if (!fcinfo->args[0].isnull)
		return hashop->d.hashdatum.fn_addr(fcinfo);
	else
		return (Datum) 0;
}

That is the entire hash computation, for the common case of “hash one plain column of the inner relation”:

Deform the tuple as far as needed (slot_getsomeattrs).
Copy the attribute value and its NULL flag into the function call frame.
Call the hash function.
If the value was NULL, return 0 (matching EEOP_HASHDATUM_FIRST’s “store 0 on NULL” behaviour).

No dispatch loop. No EEO_NEXT(). No step array traversal. The compiler can see the whole thing.

The outer and inner wrappers are one-liners:

 c
 
 
/* implementation for hashing an outer Var */
static Datum
ExecJustHashOuterVar(ExprState *state, ExprContext *econtext, bool *isnull)
{
	return ExecJustHashVarImpl(state, econtext->ecxt_outertuple, isnull);
}

/* implementation for hashing an inner Var */
static Datum
ExecJustHashInnerVar(ExprState *state, ExprContext *econtext, bool *isnull)
{
	return ExecJustHashVarImpl(state, econtext->ecxt_innertuple, isnull);
}

The strict variant

The strict case is different, and the difference is exactly the bug discussed below. In strict mode a NULL key means “the whole hash is NULL” — the join key does not match anything:

 c
 
 
/*
 * implementation for hashing an outer Var. Returns NULL on NULL input.
 */
static Datum
ExecJustHashOuterVarStrict(ExprState *state, ExprContext *econtext,
						 bool *isnull)
{
	ExprEvalStep *fetchop = &state->steps[0];
	ExprEvalStep *var = &state->steps[1];
	ExprEvalStep *hashop = &state->steps[2];
	FunctionCallInfo fcinfo = hashop->d.hashdatum.fcinfo_data;
	int			attnum = var->d.var.attnum;

	CheckOpSlotCompatibility(fetchop, econtext->ecxt_outertuple);
	slot_getsomeattrs(econtext->ecxt_outertuple, fetchop->d.fetch.last_var);

	fcinfo->args[0].value = econtext->ecxt_outertuple->tts_values[attnum];
	fcinfo->args[0].isnull = econtext->ecxt_outertuple->tts_isnull[attnum];

	if (!fcinfo->args[0].isnull)
	{
		*isnull = false;
		return hashop->d.hashdatum.fn_addr(fcinfo);
	}
	else
	{
		/* return NULL on NULL input */
		*isnull = true;
		return (Datum) 0;
	}
}

The seeded variant

The five-step pattern is for a hash seeded with an initial value — used by parallel hash aggregation, which seeds the hash with a worker-specific IV:

 c
 
 
/*
 * implementation for hashing an inner Var, seeding with an initial value.
 */
static Datum
ExecJustHashInnerVarWithIV(ExprState *state, ExprContext *econtext,
						 bool *isnull)
{
	ExprEvalStep *fetchop = &state->steps[0];
	ExprEvalStep *setivop = &state->steps[1];
	ExprEvalStep *innervar = &state->steps[2];
	ExprEvalStep *hashop = &state->steps[3];
	FunctionCallInfo fcinfo = hashop->d.hashdatum.fcinfo_data;
	int			attnum = innervar->d.var.attnum;
	uint32		hashkey;

	CheckOpSlotCompatibility(fetchop, econtext->ecxt_innertuple);
	slot_getsomeattrs(econtext->ecxt_innertuple, fetchop->d.fetch.last_var);

	fcinfo->args[0].value = econtext->ecxt_innertuple->tts_values[attnum];
	fcinfo->args[0].isnull = econtext->ecxt_innertuple->tts_isnull[attnum];

	hashkey = DatumGetUInt32(setivop->d.hashdatum_initvalue.init_value);
	hashkey = pg_rotate_left32(hashkey, 1);

	if (!fcinfo->args[0].isnull)
	{
		uint32		hashvalue;

		hashvalue = DatumGetUInt32(hashop->d.hashdatum.fn_addr(fcinfo));
		hashkey = hashkey ^ hashvalue;
	}

	*isnull = false;
	return UInt32GetDatum(hashkey);
}

Note the step indices: steps[1] is the SET_INITVAL step and steps[2] is the INNER_VAR step. The initial value is stored in the step’s own payload and is fetched directly, not through the generic *op->resvalue indirection.

The virtual-slot variant

The three-step pattern has no FETCHSOME step at all. That absence is meaningful, and it is the reason the virtual-slot variant can skip slot_getsomeattrs() entirely:

 c
 
 
/* implementation for ExecJustHash(Inner|Outer)VarVirt */
static pg_always_inline Datum
ExecJustHashVarVirtImpl(ExprState *state, TupleTableSlot *slot, bool *isnull)
{
	ExprEvalStep *var = &state->steps[0];
	ExprEvalStep *hashop = &state->steps[1];
	FunctionCallInfo fcinfo = hashop->d.hashdatum.fcinfo_data;
	int			attnum = var->d.var.attnum;

	fcinfo->args[0].value = slot->tts_values[attnum];
	fcinfo->args[0].isnull = slot->tts_isnull[attnum];

	*isnull = false;

	if (!fcinfo->args[0].isnull)
		return hashop->d.hashdatum.fn_addr(fcinfo);
	else
		return (Datum) 0;
}

No slot_getsomeattrs. No CheckOpSlotCompatibility. It reads slot->tts_values[attnum] directly.

Why is that safe? Because a virtual slot has no on-disk tuple to deform. Its values are already materialised in the tts_values[] array. The interpreter knows this, and the code that emits the setup steps knows it too:

 c
 
 
	/* if the slot is known to always virtual we never need to deform */
	if (op->d.fetch.fixed && op->d.fetch.kind == &TTSOpsVirtual)
		return false;

ExecComputeSlotInfo() returns false — meaning “do not push a FETCHSOME step” — when the slot is known to be virtual. That is why the virtual case produces a three-step expression rather than a four-step one, and why the pattern matcher can tell them apart by step count alone.

The ExecJustVarVirtImpl() counterpart says it explicitly:

 c
 
 
	/*
	 * As it is guaranteed that a virtual slot is used, there never is a need
	 * to perform tuple deforming (nor would it be possible). Therefore
	 * execExpr.c has not emitted an EEOP_*_FETCHSOME step. Verify, as much as
	 * possible, that that determination was accurate.
	 */
	Assert(TTS_IS_VIRTUAL(slot));
	Assert(TTS_FIXED(slot));
	Assert(attnum >= 0 && attnum < slot->tts_nvalid);

For a heap tuple slot, slot_getsomeattrs() is not free — it is the tuple deformation machinery, which unpacks varlena headers, follows TOAST pointers, and so on. Skipping it when it is not needed is a real saving.

The payoff

Commit 50416cc4843 (“Speedup Hash Joins with dedicated functions for ExprState hashing”, December 2024) is the one that introduced the ExecJust*Hash* family. Its commit message:

Hashing of a single Var is a very common operation for ExprState to perform. Here we add dedicated ExecJust* functions which helps speed up Hash Joins by removing the interpretation overhead in ExecInterpExpr().

This change currently only affects Hash Joins on a single column. Hash Joins with multiple join keys or an expression still run through ExecInterpExpr().

Some testing has shown up to 10% query performance increases on recent AMD hardware and nearly 7% increase on an Apple M2 for a query performing a hash join with a large number of lookups on a small hash table.

Two things to note. First, the scope: single column only. Second, the shape of the workload that benefits: “a large number of lookups on a small hash table” — i.e. probe-dominated, which is exactly the case where per-probe overhead matters.

Hardcoding the NULL check

The same “specialise the common case” philosophy was applied to strict function calls. Commit d35d32d7112 (“Add special case fast-paths for strict functions”, March 2025):

Many STRICT function calls will have one or two arguments, in which case we can speed up checking for NULL input by avoiding setting up a loop over the arguments. This adds EEOP_FUNCEXPR_STRICT_1 and the corresponding EEOP_FUNCEXPR_STRICT_2 for functions with one and two arguments respectively.

The generic strict function step loops:

 c
 
 
		/* strict function call with more than two arguments */
		EEO_CASE(EEOP_FUNCEXPR_STRICT)
		{
			FunctionCallInfo fcinfo = op->d.func.fcinfo_data;
			NullableDatum *args = fcinfo->args;
			int			nargs = op->d.func.nargs;
			Datum		d;

			Assert(nargs > 2);

			/* strict function, so check for NULL args */
			for (int argno = 0; argno < nargs; argno++)
			{
				if (args[argno].isnull)
				{
					*op->resnull = true;
					goto strictfail;
				}
			}
			fcinfo->isnull = false;
			d = op->d.func.fn_addr(fcinfo);
			*op->resvalue = d;
			*op->resnull = fcinfo->isnull;

	strictfail:
			EEO_NEXT();
		}

A loop, with a goto, and a nargs that is not known at compile time. The one-argument version:

 c
 
 
		/* strict function call with one argument */
		EEO_CASE(EEOP_FUNCEXPR_STRICT_1)
		{
			FunctionCallInfo fcinfo = op->d.func.fcinfo_data;
			NullableDatum *args = fcinfo->args;

			Assert(op->d.func.nargs == 1);

			/* strict function, so check for NULL args */
			if (args[0].isnull)
				*op->resnull = true;
			else
			{
				Datum		d;

				fcinfo->isnull = false;
				d = op->d.func.fn_addr(fcinfo);
				*op->resvalue = d;
				*op->resnull = fcinfo->isnull;
			}

			EEO_NEXT();
		}

and the two-argument version:

 c
 
 
		/* strict function call with two arguments */
		EEO_CASE(EEOP_FUNCEXPR_STRICT_2)
		{
			FunctionCallInfo fcinfo = op->d.func.fcinfo_data;
			NullableDatum *args = fcinfo->args;

			Assert(op->d.func.nargs == 2);

			/* strict function, so check for NULL args */
			if (args[0].isnull || args[1].isnull)
				*op->resnull = true;
			else
			{
				Datum		d;

				fcinfo->isnull = false;
				d = op->d.func.fn_addr(fcinfo);
				*op->resvalue = d;
				*op->resnull = fcinfo->isnull;
			}

			EEO_NEXT();
		}

The same trick appears for aggregate transition functions’ strict input checks:

 c
 
 
		/* special case for just one argument */
		EEO_CASE(EEOP_AGG_STRICT_INPUT_CHECK_ARGS_1)
		{
			NullableDatum *args = op->d.agg_strict_input_check.args;
			PG_USED_FOR_ASSERTS_ONLY int nargs = op->d.agg_strict_input_check.nargs;

			Assert(nargs == 1);

			if (args[0].isnull)
				EEO_JUMP(op->d.agg_strict_input_check.jumpnull);
			EEO_NEXT();
		}

versus the general case, which loops over the arguments.

And the choice of opcode happens once, at expression-build time:

 c
 
 
	/* Insert appropriate opcode depending on strictness and stats level */
	if (pgstat_track_functions <= flinfo->fn_stats)
	{
		if (flinfo->fn_strict && nargs > 0)
		{
			/* Choose nargs optimized implementation if available. */
			if (nargs == 1)
				scratch->opcode = EEOP_FUNCEXPR_STRICT_1;
			else if (nargs == 2)
				scratch->opcode = EEOP_FUNCEXPR_STRICT_2;
			else
				scratch->opcode = EEOP_FUNCEXPR_STRICT;
		}
		else
			scratch->opcode = EEOP_FUNCEXPR;
	}

Note the pgstat_track_functions condition: if per-function statistics collection is enabled, the specialised opcodes are not used, because they bypass the FUSAGE variants that do the counting.

The NULL-key bug

Specialisation is where bugs live. Commit 60826a352d4 (“Fix Hash Join performance issue when hashing NULL values”, July 2026) fixed a real one. Its message is a model of a good bug report:

adf97c156 allowed expression evaluation to perform hashing, and subsequently 9ca67658d fixed a memory stomping bug in that commit that caused unrelated-to-hashing expression op steps to stomp on the intermediate hash value. The intermediate hash value needs to be maintained when hashing multiple hash keys. 9ca67658d didn’t quite get things right when in “strict” mode when it aborted hashing early after encountering a NULL hash key. What was meant to happen was that the expression returns NULL directly to indicate to the caller the value hashed to NULL. The problem was that any EEOP_HASHDATUM_FIRST_STRICT or EEOP_HASHDATUM_NEXT32_STRICT op step that didn’t belong to the final key to be hashed would have its op->resnull and op->resvalue pointing to the location to store the intermediate hash value. That’s correct for non-NULLs since we bit-rotate the intermediate value and continue hashing, but with the strict case, when we get a NULL key, we immediately jump to the “jumpdone” step. The problem is the jumpdone step expects the ExprState resnull and resvalue fields to be set (as they would be if we didn’t abort hashing early due to the NULL), but when we aborted early, the ExprState fields never got set. This would result in inserting records into the hash table that would never match to any join partner, which is a waste of CPU and memory.

Let me unpack that, because it is subtle and it is a nice illustration of why specialisation is dangerous.

There are two “output slots” in play:

op->resvalue / op->resnull — where this step writes its result. For a key that is not the last one, this points at the intermediate hash slot (iresult), because the running hash must be preserved for the next key.
state->resvalue / state->resnull — the final result of the whole expression.

For a multi-key hash, the first key’s HASHDATUM_FIRST_STRICT step writes to the intermediate slot, and only the last key’s step writes to state->resvalue.

Now, in strict mode, when a NULL key is found, the interpreter aborts immediately:

 c
 
 
		EEO_CASE(EEOP_HASHDATUM_FIRST_STRICT)
		{
			FunctionCallInfo fcinfo = op->d.hashdatum.fcinfo_data;

			if (fcinfo->args[0].isnull)
			{
				/*
				 * With strict we have the expression return NULL instead of
				 * ignoring NULL input values. We've nothing more to do after
				 * finding a NULL.
				 */
				state->resnull = true;
				state->resvalue = (Datum) 0;
				EEO_JUMP(op->d.hashdatum.jumpdone);
			}

(note: this is the fixed version; the bug was *op->resnull = true; *op->resvalue = (Datum) 0;).

The bug: when the NULL was found in the first key of a multi-key hash, the step wrote to *op->resnull / *op->resvalue — the intermediate slot — and then jumped to the “done” step. The “done” step reads state->resnull / state->resvalue. Those were never written, so they held whatever was left over from the previous evaluation.

Consequences: the caller saw a non-NULL hash value for a tuple whose join key was NULL. That tuple got inserted into the hash table, where it could never match anything — pure waste of memory and CPU, in the hash-join build phase.

The fix, in both _STRICT opcodes, is to write to the expression’s result fields directly:

 c
 
 
				state->resnull = true;
				state->resvalue = (Datum) 0;

instead of *op->resnull / *op->resvalue. One character of difference — op-> versus state-> — with a real performance consequence.

The commit also notes the blast radius:

Although Hash Agg and Hashed Subplans do use hashing from ExprStates, those were unaffected by this bug, as neither of those uses the STRICT op steps.

Which is precisely why it survived: hash aggregate and hashed subplans build their hash expressions with init_value == 0 and non-strict operators, so they never take the _STRICT path.

The reporter was a user: “Reported-by: Dan Stefura”.

The hash table side

Specialising the hash computation is only half of it. The probe side walks a bucket chain, and for each candidate it has to re-check the join qual. From ExecScanHashBucket() in nodeHash.c:

 c
 
 
	while (hashTuple != NULL)
	{
		if (hashTuple->hashvalue == hashvalue)
		{
			TupleTableSlot *inntuple;

			/* insert hashtable's tuple into exec slot so ExecQual sees it */
			inntuple = ExecStoreMinimalTuple(HJTUPLE_MINTUPLE(hashTuple),
											 hjstate->hj_HashTupleSlot,
											 false);	/* do not pfree */
			econtext->ecxt_innertuple = inntuple;

			if (ExecQualAndReset(hjclauses, econtext))
			{
				hjstate->hj_CurTuple = hashTuple;
				return true;
			}
		}

		hashTuple = hashTuple->next.unshared;
	}

Two things happen per candidate: a cheap 32-bit hash comparison, and only on a match, the full join qual (hjclauses) is evaluated. The 32-bit comparison is what makes the bucket scan cheap — a hash collision is expected roughly once in 2^32 rows, so the qual almost never runs.

ExecQualAndReset() resets the expression context after each evaluation, so the qual’s temporary memory does not accumulate across candidates.

Memoize: a hash join that remembers

The Memoize node (src/backend/executor/nodeMemoize.c) is a materialised cache in front of a subplan, and it is the other place where hash-join-style machinery shows up: a hash table keyed by the subplan’s parameters, with an LRU eviction policy.

The probe slot

The design is unusual: the hash and equality functions do not use the stored key at all. They read the probe slot instead, which is populated before every lookup:

 c
 
 
/*
 * MemoizeHash_hash
 *		Hash function for simplehash hashtable. 'key' is unused here as we
 *		require that all table lookups first populate the MemoizeState's
 *		probeslot with the key values to be looked up.
 */
static uint32
MemoizeHash_hash(struct memoize_hash *tb, const MemoizeKey *key)

The population function is prepare_probe_slot():

 c
 
 
static inline void
prepare_probe_slot(MemoizeState *mstate, MemoizeKey *key)
{
	TupleTableSlot *pslot = mstate->probeslot;
	TupleTableSlot *tslot = mstate->tableslot;
	int			numKeys = mstate->nkeys;

	ExecClearTuple(pslot);

	if (key == NULL)
	{
		ExprContext *econtext = mstate->ss.ps.ps_ExprContext;
		MemoryContext oldcontext;

		oldcontext = MemoryContextSwitchTo(econtext->ecxt_per_tuple_memory);

		/* Set the probeslot's values based on the current parameter values */
		for (int i = 0; i < numKeys; i++)
			pslot->tts_values[i] = ExecEvalExpr(mstate->param_exprs[i],
												econtext,
												&pslot->tts_isnull[i]);

		MemoryContextSwitchTo(oldcontext);
	}
	else
	{
		/* Process the key's MinimalTuple and store the values in probeslot */
		ExecStoreMinimalTuple(key->params, tslot, false);
		slot_getallattrs(tslot);
		memcpy(pslot->tts_values, tslot->tts_values, sizeof(Datum) * numKeys);
		memcpy(pslot->tts_isnull, tslot->tts_isnull, sizeof(bool) * numKeys);
	}

	ExecStoreVirtualTuple(pslot);
}

Note the two paths. When looking up the current parameter values (key == NULL), the parameter expressions are evaluated directly into the probe slot’s arrays — no tuple copying at all. When re-hashing an entry that is being evicted, the stored key tuple is re-materialised into the probe slot so that the same hash function can be used.

And the probe slot is turned into a virtual tuple at the end:

 c
 
	ExecStoreVirtualTuple(pslot);

which is the same trick the hash join’s virtual-slot fast path relies on: no deformation needed, values already materialised.

The LRU eviction

The cache is bounded by mem_limit, and when it is exceeded, entries are evicted from the least-recently-used end of a doubly-linked list:

 c
 
 
/*
 * cache_reduce_memory
 *		Evict older and less recently used items from the cache in order to
 *		reduce the memory consumption back to something below the
 *		MemoizeState's mem_limit.
 */
static bool
cache_reduce_memory(MemoizeState *mstate, MemoizeKey *specialkey)
{
	...
	/* Start the eviction process starting at the head of the LRU list. */
	dlist_foreach_modify(iter, &mstate->lru_list)
	{
		MemoizeKey *key = dlist_container(MemoizeKey, lru_node, iter.cur);
		MemoizeEntry *entry;

		/*
		 * Populate the hash probe slot in preparation for looking up this LRU
		 * entry.
		 */
		prepare_probe_slot(mstate, key);

		/*
		 * Ideally the LRU list pointers would be stored in the entry itself
		 * rather than in the key. Unfortunately, we can't do that as the
		 * simplehash.h code may resize the table and allocate new memory for
		 * entries which would result in those pointers pointing to the old
		 * buckets. However, it's fine to use the key to store this as that's
		 * only referenced by a pointer in the entry, which of course follows
		 * the entry whenever the hash table is resized. Since we only have a
		 * pointer to the key here, we must perform a hash table lookup to
		 * find the entry that the key belongs to.
		 */
		entry = memoize_lookup(mstate->hashtable, NULL);
		...
		remove_cache_entry(mstate, entry);

		evictions++;

		/* Exit if we've freed enough memory */
		if (mstate->mem_used <= mstate->mem_limit)
			break;
	}

Note the comment about why the LRU list pointer lives in the key rather than the entry: simplehash.h may resize the table and reallocate its buckets, invalidating pointers stored inside the entries. The key is referenced by pointer from the entry, so it moves with the entry.

The eviction loop has to do a full hash-table lookup per evicted entry (just to find the entry that belongs to the key it is evicting) — the price of that design decision.

16 bytes per tuple

Commit 609f969f61b (“Reduce memory overheads for storing a Memoize tuple”, August 2026) is a nice piece of allocator-level thinking. Each cached tuple used to be a separately allocated structure plus a MinimalTuple. Now the “next tuple” pointer is stored in 8 extra bytes allocated with the tuple:

 c
 
 
/*
 * The number of extra bytes we request from ExecCopySlotMinimalTupleExtra to
 * allow storage of the pointer to the next cached tuple for a MemoizeEntry.
 */
#define MEMOIZE_NEXT_TUPLE_EXTRA_BYTES MAXALIGN(sizeof(MinimalTuple))

MAXALIGN(sizeof(MinimalTuple)) is 8 bytes on a 64-bit machine. And the pointer lives before the tuple:

 c
 
 
/*
 * Calculate the address of the "next" pointer from the MinimalTuple
 */
#define MemoizeNextTupleAddress(t) \
	((MinimalTuple *) ((char *) (t) - MEMOIZE_NEXT_TUPLE_EXTRA_BYTES))

ExecCopySlotMinimalTupleExtra() allocates the tuple with that much extra space, already zeroed:

 c
 
 
/*
 * ExecCopySlotMinimalTupleExtra - return MinimalTuple allocated in caller's
 * context, with extra bytes (maxaligned and zeroed) before the tuple for data
 * the caller wishes to store along with the tuple (without requiring the
 * caller to make an additional allocation).
 */
static inline MinimalTuple
ExecCopySlotMinimalTupleExtra(TupleTableSlot *slot, Size extra)
{
	return slot->tts_ops->copy_minimal_tuple(slot, extra);
}

And the cache store path:

 c
 
 
	/*
	 * Form a MinimalTuple with extra space to store a "next" pointer so that
	 * we can form a singly linked list of tuples belonging to this
	 * MemoizeEntry.
	 */
	mintuple = ExecCopySlotMinimalTupleExtra(slot,
											 MEMOIZE_NEXT_TUPLE_EXTRA_BYTES);

	/*
	 * No need to use MemoizeSetNextTuple to point the next tuple to NULL as
	 * ExecCopySlotMinimalTupleExtra zeros the extra bytes.
	 */

The commit message quantifies the win:

This uses 16 fewer bytes per tuple stored in each MemoizeEntry. 24 bytes can be saved by not doing palloc_object(MemoizeTuple), and instead we can store the pointer to the next tuple by requesting that ExecCopySlotMinimalTupleExtra() allocate an extra 8 bytes per MinimalTuple, a net saving of 16 bytes per cached tuple, which equates to 33% less memory in the most extreme case (assuming very few entries with many tuples and the narrowest possible MinimalTuple).

Saving memory in Memoize allows more entries to exist concurrently and reduces the chances that an entry will be evicted from the cache due to memory pressure. That increases the chances of getting cache hits.

The accounting macro follows:

 c
 
 
/* Helper macros for memory accounting */
#define EMPTY_ENTRY_MEMORY_BYTES(e)		(sizeof(MemoizeEntry) + \
										 sizeof(MemoizeKey) + \
										 (e)->key->params->t_len)
#define CACHE_TUPLE_BYTES(t)			((t)->t_len + \
										 MEMOIZE_NEXT_TUPLE_EXTRA_BYTES)

This is a micro-optimisation with a macro-level payoff: 16 bytes out of a typical tuple’s ~48 bytes of overhead is a third of the per-tuple memory, and memory per cached tuple is the cache’s capacity.

The JIT has its own copy

One more wrinkle. PostgreSQL’s JIT compiler (src/backend/jit/llvm/) compiles the same step list, but it is a completely separate implementation with its own switch over the same opcodes. So every change to the opcode set has to be made twice.

The strict-function specialisation had to be mirrored:

 c
 
 
			case EEOP_FUNCEXPR:
			case EEOP_FUNCEXPR_STRICT:
			case EEOP_FUNCEXPR_STRICT_1:
			case EEOP_FUNCEXPR_STRICT_2:
				{
					FunctionCallInfo fcinfo = op->d.func.fcinfo_data;
					LLVMValueRef v_fcinfo_isnull;
					LLVMValueRef v_retval;

					if (opcode == EEOP_FUNCEXPR_STRICT ||
						opcode == EEOP_FUNCEXPR_STRICT_1 ||
						opcode == EEOP_FUNCEXPR_STRICT_2)
					{

and the hashing opcodes likewise:

 c
 
 
			case EEOP_HASHDATUM_SET_INITVAL:
				{
					LLVMValueRef v_initvalue;

					v_initvalue = l_datum_const(op->d.hashdatum_initvalue.init_value);

					LLVMBuildStore(b, v_initvalue, v_resvaluep);
					LLVMBuildStore(b, l_sbool_const(0), v_resnullp);
					LLVMBuildBr(b, opblocks[opno + 1]);
					break;
				}

			case EEOP_HASHDATUM_FIRST:
			case EEOP_HASHDATUM_FIRST_STRICT:
			case EEOP_HASHDATUM_NEXT32:
			case EEOP_HASHDATUM_NEXT32_STRICT:

Note that the JIT has to be careful about when it rotates the running hash, and it differs deliberately between strict and non-strict mode:

 c
 
 
					/*
					 * When performing the next hash and not in strict mode we
					 * perform a rotation of the previously stored hash value
					 * before doing the NULL check. We want to do this even
					 * when we receive a NULL Datum to hash. In strict mode,
					 * we do this after the NULL check so as not to waste the
					 * effort of rotating the bits when we're going to throw
					 * away the hash value and return NULL.
					 */

This is the same subtlety that produced the NULL-key bug, and the JIT had to be written with the same care.

The JIT also changes which fast path applies. ExecReadyExpr() is the single entry point:

 c
 
 
static void
ExecReadyExpr(ExprState *state)
{
	if (jit_compile_expr(state))
		return;

	ExecReadyInterpretedExpr(state);
}

If the JIT compiled the expression, ExecReadyInterpretedExpr() is never called, so the ExecJust* fast paths are not installed at all when JIT is active. The JIT’s own code is the fast path in that case — which is why the adf97c15623 commit measured a 26% improvement with JIT enabled versus 20% without.

Caveats

Only very specific opcode shapes are specialised. A hash join on two columns, or on an expression like a + 1, or on a column with a collation-sensitive hash function, falls through to the general interpreter. The commit message is explicit: “This change currently only affects Hash Joins on a single column. Hash Joins with multiple join keys or an expression still run through ExecInterpExpr().”

The strict fast paths only cover 1 and 2 arguments. EEOP_FUNCEXPR_STRICT still loops. And the specialised opcodes are skipped entirely when pgstat_track_functions is enabled, because they bypass the statistics-collecting variants.

The JIT and the interpreter are separate implementations of the same specification. Every opcode change must be made twice, and the two can disagree — as they nearly did around the NULL-key bug.

Specialisation is where bugs live. The op->resnull versus state->resnull distinction is invisible at the level of “return NULL if the key is NULL”, but the difference between writing the intermediate slot and the final slot is the difference between a correct result and a hash table full of rows that can never match.

pfree is not the only thing that got faster. The hash join’s build side also got faster, because ExecBuildHash32Expr() deforms the tuple once to the highest needed attribute rather than once per key.

What it all means in practice

The pattern is “specialise the shape you actually have”. PostgreSQL’s expression evaluation is a general interpreter with a general dispatcher, and it is used for everything from qual evaluation to aggregate transitions to hash keys. Rather than making the interpreter faster in general, the strategy is to notice the handful of shapes that dominate real workloads and hand-write those.

Matching on step count first is a cheap filter that pays for itself. steps_len == 5 narrows the space before any opcode is examined, and the check happens exactly once per expression.

The specialised functions are not “clever”. They are straight-line code that a compiler can see through — no dispatch, no goto, no loop over nargs. The win comes from removing the interpreter’s per-step overhead, not from doing anything clever.

The virtual-slot trick is about knowing what you don’t have to do. A virtual slot has no tuple to deform, so the deformation step is not emitted, so the fast path does not perform it. Absence of a step is the signal.

Memory per cached entry is cache capacity. Saving 16 bytes per Memoize tuple is not a micro-optimisation; in a cache bounded by work_mem, it directly translates into more entries surviving, fewer evictions, and more cache hits.

A 10% win on a probe-dominated hash join is worth a lot. The commit measured “up to 10% query performance increases on recent AMD hardware” for a workload with many probes against a small hash table. That is the shape of workload where per-probe overhead is the whole cost.

PostgreSQL's numeric Implementation in C

Jay Hubbert — Thu, 08 Oct 2026 10:09:37 +0000

numeric is PostgreSQL’s exact decimal type: arbitrary precision, no rounding surprises, no binary-vs-decimal conversion error. It is also the type where the storage format and the arithmetic format are deliberately different, and the difference is the whole design.

The short version: numeric stores an array of digits in base 10000, with a separate sign and a “weight” that says where the decimal point goes. It comes in two on-disk header sizes. And the in-memory working form, NumericVar, keeps a spare leading digit so that a carry out of the top of the number never forces a repalloc.

This is a walk through src/backend/utils/adt/numeric.c and src/include/utils/numeric.h, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entry

From src/include/catalog/pg_type.dat:

 text
 
 
{ oid => '1700', array_type_oid => '1231',
 descr => '\'numeric(precision, scale)\' arbitrary precision number',
 typname => 'numeric', typlen => '-1', typbyval => 'f', typcategory => 'N',
 typinput => 'numeric_in', typoutput => 'numeric_out',
 typreceive => 'numeric_recv', typsend => 'numeric_send',
 typmodin => 'numerictypmodin', typmodout => 'numerictypmodout',
 typalign => 'i', typstorage => 'm' },

Read against the float types, the differences are the point:

Field	Value	Meaning
`typlen`	`-1`	varlena — variable length, 4-byte or 1-byte header
`typbyval`	`f`	by reference; the `Datum` is a pointer
`typalign`	`i`	4-byte alignment
`typstorage`	`m`	`TYPSTORAGE_MAIN`: toastable, but prefer to keep inline
array oid	1231	`_numeric`
typmod I/O	`numerictypmodin/out`	supports `numeric(p,s)`

typstorage => 'm' is the interesting one. A numeric can be compressed and moved out of line, but the type asks the toaster to try harder to keep it in the main tuple than an 'x' (EXTENDED) column would — a reasonable choice for a type that is often small and whose exact value you usually want on the page.

On disk: two headers over base-10000 digits

The representation is introduced by a long comment at the top of numeric.c, which settles the base first:

 c
 
 
#if 1
#define NBASE		10000
#define HALF_NBASE	5000
#define DEC_DIGITS	4			/* decimal digits per NBASE digit */
#define MUL_GUARD_DIGITS	2	/* these are measured in NBASE digits */
#define DIV_GUARD_DIGITS	4

typedef int16 NumericDigit;
#endif

Base 10000 is not an accident: it is a power of ten, so decimal I/O and rounding are trivial, and it is comfortably below sqrt(INT_MAX), leaving headroom for mul_var/div_var to postpone carries. The comment notes that other bases (10, 100) exist only as historical #if 0 blocks and are no longer supported; the on-disk format is always base 10000.

The stored value is a sign-magnitude digit array with two possible headers:

 c
 
 
struct NumericShort
{
	uint16		n_header;		/* Sign + display scale + weight */
	NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */
};

struct NumericLong
{
	uint16		n_sign_dscale;	/* Sign + display scale */
	int16		n_weight;		/* Weight of 1st digit	*/
	NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */
};

union NumericChoice
{
	uint16		n_header;		/* Header word */
	struct NumericLong n_long;	/* Long form (4-byte header) */
	struct NumericShort n_short;	/* Short form (2-byte header) */
};

struct NumericData
{
	int32		vl_len_;		/* varlena header (do not touch directly!) */
	union NumericChoice choice; /* choice of format */
};

Which header you get is encoded in the top two bits of the first word:

 c
 
 
#define NUMERIC_SIGN_MASK	0xC000
#define NUMERIC_POS			0x0000
#define NUMERIC_NEG			0x4000
#define NUMERIC_SHORT		0x8000
#define NUMERIC_SPECIAL		0xC000

So 0xC000 means “special” (NaN or infinity), 0x8000 means the short form, and 0x0000/0x4000 mean the long form with positive/negative sign. Because both SHORT and SPECIAL have the high bit set, the code can test a single bit instead of comparing against each constant:

#define NUMERIC_HEADER_IS_SHORT(n)	(((n)->choice.n_header & 0x8000) != 0)

The short header is the clever one. Its 16 bits are packed as 1 sign bit, 6 bits of display scale, and 7 bits of weight:

 c
 
 
#define NUMERIC_SHORT_SIGN_MASK			0x2000
#define NUMERIC_SHORT_DSCALE_MASK		0x1F80
#define NUMERIC_SHORT_DSCALE_SHIFT		7
#define NUMERIC_SHORT_WEIGHT_MASK		0x003F
#define NUMERIC_SHORT_WEIGHT_MAX		NUMERIC_SHORT_WEIGHT_MASK
#define NUMERIC_SHORT_WEIGHT_MIN		(-(NUMERIC_SHORT_WEIGHT_MASK+1))

The long header instead stores a full int16 weight and a 14-bit display scale (NUMERIC_DSCALE_MASK == 0x3FFF). The two header sizes are:

 c
 
 
#define NUMERIC_HDRSZ	(VARHDRSZ + sizeof(uint16) + sizeof(int16)) /* 8 */
#define NUMERIC_HDRSZ_SHORT (VARHDRSZ + sizeof(uint16)) /* 6 */

The number of digits is never stored — it is derived from the varlena length:

 c
 
 
#define NUMERIC_NDIGITS(num) \
	((VARSIZE(num) - NUMERIC_HEADER_SIZE(num)) / sizeof(NumericDigit))

Special values are distinguished by the two bits just below the NUMERIC_SPECIAL flag: 0xC000 is NaN, 0xD000 positive infinity, 0xF000 negative infinity (NUMERIC_NAN, NUMERIC_PINF, NUMERIC_NINF). In this tree they are always written with the 2-byte short header; the comment notes that 4-byte NaNs may still exist on disk after a pg_upgrade.

In memory: NumericVar

Arithmetic is not done on the packed form. It is done on NumericVar, which keeps the digit buffer separate from the metadata:

 c
 
 
typedef struct NumericVar
{
	int			ndigits;		/* # of digits in digits[] - can be 0! */
	int			weight;			/* weight of first digit */
	int			sign;			/* NUMERIC_POS, _NEG, _NAN, _PINF, or _NINF */
	int			dscale;			/* display scale */
	NumericDigit *buf;			/* start of palloc'd space for digits[] */
	NumericDigit *digits;		/* base-NBASE digits */
} NumericVar;

Two things to notice. First, weight is the base-10000 exponent of the first digit, so the value is `digits[0] * 10000^weight + digits[1] * 10000^(weight-1)

…; weightcan be negative. Second,bufanddigitsare distinct pointers.alloc_var` deliberately leaves one unused digit in front:

 c
 
 
static void
alloc_var(NumericVar *var, int ndigits)
{
	digitbuf_free(var->buf);
	var->buf = digitbuf_alloc(ndigits + 1);
	var->buf[0] = 0;			/* spare digit for rounding */
	var->digits = var->buf + 1;
	var->ndigits = ndigits;
}

That spare slot is where a carry out of the top digit lands, and where round_var can write when a value rounds up to one more digit. Because the buffer is separate from the struct, the same variable can be passed as both input and output to add_var, mul_var, and friends — the comment calls this out explicitly.

The distinction between init_var_from_num and set_var_from_num is a real API choice: the former aliases the packed digits (no copy, faster, but you must not modify them, because the changes would propagate back into the Numeric), while the latter copies into a fresh buffer.

Input and output

numeric_in is more involved than you might expect. It skips whitespace, handles a sign, recognizes NaN/Infinity (the same spellings float8in accepts), and then — in this tree — also recognizes non-decimal integer prefixes:

 c
 
 
		if (cp[0] == '0')
		{
			switch (cp[1])
			{
				case 'x':
				case 'X':
					base = 16;
					break;
				case 'o':
				case 'O':
					base = 8;
					break;
				case 'b':
				case 'B':
					base = 2;
					break;
				default:
					base = 10;
			}
		}

So '0x10'::numeric parses as 16, and '0b101'::numeric as 5. Decimal input goes through set_var_from_str, which extracts the decimal digits into a temporary byte array, computes the decimal weight and scale, and then packs them into base-10000 digits. Either way, apply_typmod enforces numeric(p,s) before make_result packs the value for storage.

numeric_out reverses this: special values print as NaN / Infinity / -Infinity, and everything else goes through get_str_from_var, which walks the digit array and emits exactly dscale digits after the point.

Binary I/O is a flat sequence of int16s, documented in the numeric_recv comment:

text

External format is a sequence of int16's:
ndigits, weight, sign, dscale, NumericDigits.

numeric_recv validates every field — the sign must be one of the five legal values, the scale must fit NUMERIC_DSCALE_MASK, and every digit must be in [0, NBASE) — before rebuilding the value.

The numeric(p,s) typmod is packed into 32 bits: precision in the high 16, scale (which may be negative) in the low 11, plus VARHDRSZ as a historical offset:

 c
 
 
static inline int32
make_numeric_typmod(int precision, int scale)
{
	return ((precision << 16) | (scale & 0x7ff)) + VARHDRSZ;
}

make_result: stripping zeros and choosing a header

The packing function is where the “zero has no digits” invariant lives. make_result_safe truncates leading and trailing zero digits, forces a zero result to weight = 0 and positive sign, and then picks the header:

 c
 
 
	/* truncate leading zeroes */
	while (n > 0 && *digits == 0)
	{
		digits++;
		weight--;
		n--;
	}
	/* truncate trailing zeroes */
	while (n > 0 && digits[n - 1] == 0)
		n--;

	/* If zero result, force to weight=0 and positive sign */
	if (n == 0)
	{
		weight = 0;
		sign = NUMERIC_POS;
	}

It then uses NUMERIC_CAN_BE_SHORT(var->dscale, weight) to decide between the 6-byte and 8-byte headers, and finally checks that the packed fields actually round-trip — if the weight or scale did not survive the narrow fields, it raises “value overflows numeric format”. The consequence of the zero-stripping is that 1.0 and 1.00 compare equal but have different dscale, and hash_numeric deliberately does not hash the scale:

 c
 
 
	/*
	 * Note that we don't hash on the Numeric's scale, since two numerics can
	 * compare equal but have different scales. ...
	 */

Two non-obvious decisions

Sorting abbreviates values into 64 bits. Sorting numeric is slow, so the sortsupport path packs a value into an int64 key: 7 bits of excess-44 weight plus four 14-bit base-10000 digit words. The comment in numeric_abbrev_convert_var is candid about the trade-off — it compares at least 13 decimal digits, is only meaningful in the range 10^-176 to 10^332, and the abbreviation is negated relative to the value so that the NaN/infinity ordering falls out for free:

 c
 
 
	/* the abbrev is negated relative to the original */
	if (var->sign == NUMERIC_POS)
		result = -result;

Because abbreviation can be a pessimization on low-cardinality data, numeric_abbrev_abort uses a HyperLogLog sketch to estimate how many distinct values it has seen and aborts if the cardinality is too low to pay off.

SUM() uses a dedicated accumulator. NumericSumAccum stores digits as int32 rather than int16, so it can add up to NBASE - 1 values before it has to propagate carry. It also accumulates positive and negative values in separate buffers:

 c
 
 
typedef struct NumericSumAccum
{
	int			ndigits;
	int			weight;
	int			dscale;
	int			num_uncarried;
	bool		have_carry_space;
	int32	 *pos_digits;
	int32	 *neg_digits;
} NumericSumAccum;

The comment explains why: adding to one running value would require deciding add-vs-subtract and renormalizing on every input, whereas keeping the two signs apart is “simpler and faster”; accum_sum_final combines them once at the end.

Rounding, in round_var, is half-away-from-zero on the magnitude (extra >= pow10 / 2), and it accepts a negative rscale, which rounds before the decimal point:

Round the value of a variable to no more than rscale decimal digits after the decimal point. NOTE: we allow rscale < 0 here, implying rounding before the decimal point.

Look at it yourself

The header choice and the typmod are both visible from SQL:

 sql
 
 
SELECT typname, typlen, typbyval, typalign, typstorage
FROM pg_type WHERE typname = 'numeric';

-- the short header is used for small, modest-scale values
SELECT pg_column_size(1.5::numeric),
 pg_column_size(0.000000000000000000000000000000001::numeric),
 pg_column_size(123456789012345678901234567890::numeric);

-- zero stores no digits at all
SELECT pg_column_size(0::numeric);

-- scale is metadata: equal values, different dscale
SELECT 1.0::numeric = 1.00::numeric;

-- non-decimal integer input
SELECT '0x10'::numeric, '0b101'::numeric, '0o17'::numeric;

-- typmod limits
SELECT 123.456::numeric(5,2); -- 123.46

pg_column_size(0::numeric) is small because make_result stripped the zero digit away and emitted a header with ndigits == 0. The numeric(5,2) case shows apply_typmod calling round_var to two decimal places before storage.

Key takeaways

numeric is base-10000 sign-magnitude, not IEEE. Digits are int16 values in [0, 10000), with an explicit weight locating the decimal point.
Two on-disk headers. The short form packs sign, 6-bit scale, and 7-bit weight into one word; the long form carries an int16 weight and a 14-bit scale. A single high bit selects between them.
The arithmetic form is different. NumericVar holds the digit buffer separately and reserves a leading digit for carries, which is what lets the same variable be input and output.
Zero has no digits. make_result strips leading and trailing zeros, so ndigits == 0 is a valid, canonical zero, and values with different scales can compare equal.
numeric is toastable but MAIN, and it supports both the numeric(p,s) typmod and, in this tree, non-decimal integer literals.

PostgreSQL Waits for Locks

Jay Hubbert — Tue, 06 Oct 2026 09:00:00 +0000

Every lock in PostgreSQL is a bet. When a backend wants a resource that somebody else has, it has to choose: burn CPU waiting, or go to sleep and let the kernel schedule somebody else. Both are wrong sometimes. Spinning wastes a core and burns power; sleeping costs a context switch, wakes up in the wrong place, and — worst of all — adds the victim’s release time to the waiter’s latency.

PostgreSQL’s answer is a ladder. It spins in user space with an architecture-specific pause instruction, then sleeps for a randomized, exponentially growing interval, and only then blocks in the kernel on a POSIX semaphore. Every rung of that ladder is a separate piece of code with a separate set of trade-offs, and the interesting part of the design is where the rungs are.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The ladder, in one table

Layer	Object	Waits by	Kernel involved?
1	Spinlock (`slock_t`)	`lock; xchgb` in a loop, then `pg_usleep()`	Only after ~1000 sleeps
2	Atomic (`pg_atomic_uint32`)	Compare-and-swap loop	No
3	LWLock	Spin, then block on a POSIX semaphore	Yes
4	Latch	Sleep in `epoll_wait()`/`poll()`	Yes
5	Heavyweight lock	Block on the lock manager’s semaphore	Yes

The ladder is not uniform, and that is the point. A spinlock protects something so small that the only correct thing to do is spin. An LWLock protects something that might take milliseconds. A latch is not a lock at all — it is a one-way door that a backend waits on until somebody opens it.

Layer 1: the spinlock

The hardware primitives

A spinlock is one byte. The contract is spelled out in src/include/storage/s_lock.h:

int TAS(slock_t *lock) Atomic test-and-set instruction. Attempt to acquire the lock, but do not wait. Returns 0 if successful, nonzero if unable to acquire the lock.

int TAS_SPIN(slock_t *lock) Like TAS(), but this version is used when waiting for a lock previously found to be contended. By default, this is the same as TAS(), but on some architectures it’s better to poll a contended lock using an unlocked instruction and retry the atomic test-and-set only when it appears free.

On x86_64 the whole thing is one instruction pair:

 c
 
 
// src/include/storage/s_lock.h
static inline int
tas(volatile slock_t *lock)
{
	slock_t		_res = 1;

	__asm__ __volatile__(
	"	lock			\n"
	"	xchgb	%0,%1	\n"
:		"+q"(_res), "+m"(*lock)
:		/* no inputs */
:		"memory", "cc");
	return (int) _res;
}

lock xchg is a read-modify-write with the bus held, so it is atomic with respect to every other core. The old value comes back in %0, and the return value tells the caller whether it got the lock (old value zero) or found it taken.

The free test

The interesting part is TAS_SPIN, and on x86_64 it is a one-liner:

 c
 
 
// src/include/storage/s_lock.h
/*
 * On Intel EM64T, it's a win to use a non-locking test before the xchg proper,
 * but only when spinning.
 *
 * See also Implementing Scalable Atomic Locks for Multi-Core Intel(tm) EM64T
 * and IA32, by Michael Chynoweth and Mary R. Lee. As of this writing, it is
 * available at:
 * http://software.intel.com/en-us/articles/implementing-scalable-atomic-locks-for-multi-core-intel-em64t-and-ia32-architectures
 */
#define TAS_SPIN(lock) (*(lock) ? 1 : TAS(lock))

Read it as: if the lock byte is non-zero, it is taken, so return “failed” without touching the bus. Only if it looks free do we pay for the lock xchgb.

The reason this only happens while spinning is subtle. A lock-prefixed instruction on x86 is a full barrier: it drains the store buffer and serializes against other cores’ atomic operations. If you issue one for every failed attempt, you are asking the memory subsystem to arbitrate between every spinner on the machine — which is exactly the contention you are trying to avoid. A plain load is free (it can be satisfied from the local cache) and lets the spinner’s core keep running at full speed.

The “only when spinning” part is about the first acquisition. On the first attempt, the lock is very likely free, and the cost of the lock xchgb is paid anyway; doing a speculative plain load first would just add a branch and a potential cache miss on the fast path. And there is a correctness-adjacent reason too: the free test is a prediction. Between the load and the xchgb, another core could take the lock. That is fine — the xchgb is still the arbiter, and it will report the failure — but it means the free test can produce false “free” answers, and only the atomic instruction may decide.

The 32-bit x86 version takes the opposite approach and does the free test inside tas() itself:

 c
 
 
// src/include/storage/s_lock.h
static inline int
tas(volatile slock_t *lock)
{
	slock_t		_res = 1;

	/*
	 * Use a non-locking test before asserting the bus lock. Note that the
	 * extra test appears to be a small loss on some x86 platforms and a small
	 * win on others; it's by no means clear that we should keep it.
	 *
	 * When this was last tested, we didn't have separate TAS() and TAS_SPIN()
	 * macros. Nowadays it probably would be better to do a non-locking test
	 * in TAS_SPIN() but not in TAS(), like on x86_64, but no-one's done the
	 * testing to verify that. Without some empirical evidence, better to
	 * leave it alone.
	 */
	__asm__ __volatile__(
	"	cmpb	$0,%1		\n"
	"	jne		TAS%=_out	\n"
	"	lock				\n"
	"	xchgb	%0,%1		\n"
	"TAS%=_out: \n"
:		"+q"(_res), "+m"(*lock)
:		/* no inputs */
:		"memory", "cc");
	return (int) _res;
}

Note the comment’s honesty: “it’s by no means clear that we should keep it”. This is the kind of micro-optimization that is kept because it was measured once, long ago, and nobody has had the heart to re-measure it.

The pause instruction

Between attempts, the spinner has to do something. Doing nothing is bad: modern x86 cores issue speculatively and out of order, so a tight loop that keeps polling a memory location hammers the memory subsystem and the core’s pipeline. The answer is a hint instruction that tells the CPU “you are in a spin-wait loop”.

On x86 that is PAUSE, and PostgreSQL spells it the classic way:

 c
 
 
// src/include/storage/s_lock.h
static inline void
spin_delay(void)
{
	/*
	 * Adding a PAUSE in the spin delay loop is demonstrably a no-op on
	 * Opteron, but it may be of some use on EM64T, so we keep it.
	 */
	__asm__ __volatile__(
	" rep; nop			\n");
}

The 32-bit x86 version carries the full Intel explanation:

 c
 
 
// src/include/storage/s_lock.h
static inline void
spin_delay(void)
{
	/*
	 * This sequence is equivalent to the PAUSE instruction ("rep" is
	 * ignored by old IA32 processors if the following instruction is
	 * not a string operation); the IA-32 Architecture Software
	 * Developer's Manual, Vol. 3, Section 7.7.2 describes why using
	 * PAUSE in the inner loop of a spin lock is necessary for good
	 * performance:
	 *
	 * The PAUSE instruction improves the performance of IA-32
	 * processors supporting Hyper-Threading Technology when
	 * executing spin-wait loops and other routines where one
	 * thread is accessing a shared lock or semaphore in a tight
	 * polling loop. When executing a spin-wait loop, the
	 * processor can suffer a severe performance penalty when
	 * exiting the loop because it detects a possible memory order
	 * violation and flushes the core processor's pipeline. The
	 * PAUSE instruction provides a hint to the processor that the
	 * code sequence is a spin-wait loop. The processor uses this
	 * hint to avoid the memory order violation and prevent the
	 * pipeline flush. In addition, the PAUSE instruction
	 * de-pipelines the spin-wait loop to prevent it from
	 * consuming execution resources excessively.
	 */
	__asm__ __volatile__(
	" rep; nop			\n");
}

“De-pipelines the spin-wait loop to prevent it from consuming execution resources excessively” is the key phrase: PAUSE exists so that a spinning thread does not steal issue slots from the thread that is about to release the lock. On a hyper-threaded machine that matters enormously, because the spinner and the holder may share a physical core.

On ARM64 the answer is ISB:

 c
 
 
// src/include/storage/s_lock.h
static inline void
spin_delay(void)
{
	/*
	 * Using an ISB instruction to delay in spinlock loops appears beneficial
	 * on high-core-count ARM64 processors. It seems mostly a wash for smaller
	 * gear, and ISB doesn't exist at all on pre-v7 ARM chips.
	 */
	__asm__ __volatile__(
	" isb;				\n");
}

ISB flushes the instruction pipeline, which has the same effect as PAUSE: it stops the core from speculatively executing past the spin loop.

PowerPC is interesting because it does not define SPIN_DELAY() at all, so the generic fallback in s_lock.h applies:

 c
 
 
// src/include/storage/s_lock.h
#if !defined(SPIN_DELAY)
#define SPIN_DELAY()	((void) 0)
#endif		/* SPIN_DELAY */

Instead, PowerPC gets its memory ordering from the lwarx/stwcx. compare-and-swap loop, and uses lwsync as its barrier:

 c
 
 
// src/include/storage/s_lock.h
/*
 * NOTE: per the Enhanced PowerPC Architecture manual, v1.0 dated 7-May-2002,
 * an isync is a sufficient synchronization barrier after a lwarx/stwcx loop.
 * But if the spinlock is in ordinary memory, we can use lwsync instead for
 * better performance.
 */
static inline int
tas(volatile slock_t *lock)
{
	slock_t _t;
	int _res;

	__asm__ __volatile__(
"	lwarx %0,0,%3,1	\n"
"	cmpwi %0,0		\n"
"	bne TAS%=_fail	\n"
"	addi %0,%0,1		\n"
"	stwcx. %0,0,%3		\n"
"	beq TAS%=_ok	\n"
"TAS%=_fail: \n"
"	li %1,1		\n"
"	b TAS%=_out	\n"
"TAS%=_ok: \n"
"	lwsync				\n"
"	li %1,0		\n"
"TAS%=_out: \n"
:	"=&b"(_t), "=r"(_res), "+m"(*lock)
:	"r"(lock)
:	"memory", "cc");
	return _res;
}

/*
 * PowerPC S_UNLOCK is almost standard but requires a "sync" instruction.
 * But we can use lwsync instead for better performance.
 */
#define S_UNLOCK(lock)	\
do \
{ \
	__asm__ __volatile__ ("	lwsync \n" ::: "memory"); \
	*((volatile slock_t *) (lock)) = 0; \
} while (0)

lwarx is “load with reservation” and stwcx. is “store with conditional”, which together implement a compare-and-swap on PowerPC’s older architectures. The lwsync after a successful acquisition is the acquire barrier; the lwsync before the store in S_UNLOCK is the release barrier.

The memory ordering contract

The header is unusually explicit about what the compiler and the hardware are allowed to do:

It is the responsibility of these macros to make sure that the compiler does not re-order accesses to shared memory to precede the actual lock acquisition, or follow the lock release.

On platforms with weak memory ordering, the TAS(), TAS_SPIN(), and S_UNLOCK() macros must further include hardware-level memory fence instructions to prevent similar re-ordering at the hardware level. TAS() and TAS_SPIN() must guarantee that loads and stores issued after the macro are not executed until the lock has been obtained. Conversely, S_UNLOCK() must guarantee that loads and stores issued before the macro have been executed before the lock is released.

Two different barriers are needed, and the difference is why S_UNLOCK() is not just *(lock) = 0 everywhere:

 c
 
 
// src/include/storage/s_lock.h
/*
 * Default implementation of S_UNLOCK() for gcc/icc.
 *
 * Note that this implementation is unsafe for any platform that can reorder
 * a memory access (either load or store) after a following store. That
 * happens not to be possible on x86 and most legacy architectures (some are
 * single-processor!), but many modern systems have weaker memory ordering.
 * Those that do must define their own version of S_UNLOCK() rather than
 * relying on this one.
 */
#if !defined(S_UNLOCK)
#define S_UNLOCK(lock)	\
	do { __asm__ __volatile__("" : : : "memory"); *(lock) = 0; } while (0)
#endif		/* !defined(S_UNLOCK) */

The default is a compiler barrier only. On x86 that is enough, because the hardware is total-store-ordered: stores cannot be reordered with other stores, and the lock prefix on the acquisition side prevents the hardware from moving anything past it. On ARM or PowerPC you need an explicit instruction, which is why those architectures define their own S_UNLOCK.

The generic spin loop

Everything above is platform-specific. The portable part is in src/backend/storage/lmgr/s_lock.c:

 c
 
 
// src/backend/storage/lmgr/s_lock.c
#ifdef USE_DEFAULT_S_LOCK
/*
 * s_lock(lock) - platform-independent portion of waiting for a spinlock.
 */
int
s_lock(volatile slock_t *lock, const char *file, int line, const char *func)
{
	SpinDelayStatus delayStatus;

	init_spin_delay(&delayStatus, file, line, func);

	while (TAS_SPIN(lock))
	{
		perform_spin_delay(&delayStatus);
	}

	finish_spin_delay(&delayStatus);

	return delayStatus.delays;
}
#endif

The fast path is one TAS_SPIN that succeeds. The slow path calls perform_spin_delay() in a loop until the lock is acquired, and the number of delays is returned so that the caller can report it (this is how a “stuck spinlock” panic gets a sensible message).

perform_spin_delay()

This is the function that actually implements the ladder:

 c
 
 
// src/backend/storage/lmgr/s_lock.c
/*
 * Wait while spinning on a contended spinlock.
 */
void
perform_spin_delay(SpinDelayStatus *status)
{
	/* CPU-specific delay each time through the loop */
	SPIN_DELAY();

	/* Block the process every spins_per_delay tries */
	if (++(status->spins) >= spins_per_delay)
	{
		if (++(status->delays) > NUM_DELAYS)
			s_lock_stuck(status->file, status->line, status->func);

		if (status->cur_delay == 0) /* first time to delay? */
			status->cur_delay = MIN_DELAY_USEC;

		/*
		 * Once we start sleeping, the overhead of reporting a wait event is
		 * justified. Actively spinning easily stands out in profilers, but
		 * sleeping with an exponential backoff is harder to spot...
		 *
		 * We might want to report something more granular at some point, but
		 * this is better than nothing.
		 */
		pgstat_report_wait_start(WAIT_EVENT_SPIN_DELAY);
		pg_usleep(status->cur_delay);
		pgstat_report_wait_end();

#if defined(S_LOCK_TEST)
		fprintf(stdout, "*");
		fflush(stdout);
#endif

		/* increase delay by a random fraction between 1X and 2X */
		status->cur_delay += (int) (status->cur_delay *
									pg_prng_double(&pg_global_prng_state) + 0.5);
		/* wrap back to minimum delay when max is exceeded */
		if (status->cur_delay > MAX_DELAY_USEC)
			status->cur_delay = MIN_DELAY_USEC;

		status->spins = 0;
	}
}

Three phases per iteration:

SPIN_DELAY() — the architecture-specific pause. Executed on every pass through the loop, not just every spins_per_delay tries.
Every spins_per_delay attempts, sleep for cur_delay microseconds via pg_usleep().
Multiply the delay by a random factor between 1× and 2×.

The constants:

 c
 
 
// src/backend/storage/lmgr/s_lock.c
#define MIN_SPINS_PER_DELAY 10
#define MAX_SPINS_PER_DELAY 1000
#define NUM_DELAYS			1000
#define MIN_DELAY_USEC		1000L
#define MAX_DELAY_USEC		1000000L

So: spin up to spins_per_delay times (default 100), then sleep 1 ms, then sleep between 1 and 2 ms, then between 2 and 4 ms, and so on, capped at 1 second — and after 1000 delays, declare the spinlock stuck and PANIC.

Note the granularity of the backoff: cur_delay += (int) (cur_delay * pg_prng_double(...) + 0.5) multiplies by a random number in [1.0, 2.0), rounded. That is a jittered exponential backoff, and the jitter is the whole point.

Why the randomness matters

The file header explains the starvation problem:

Once we do decide to block, we use randomly increasing pg_usleep() delays. The first delay is 1 msec, then the delay randomly increases to about one second, after which we reset to 1 msec and start again. The idea here is that in the presence of heavy contention we need to increase the delay, else the spinlock holder may never get to run and release the lock. (Consider situation where spinlock holder has been nice’d down in priority by the scheduler — it will not get scheduled until all would-be acquirers are sleeping, so if we always use a 1-msec sleep, there is a real possibility of starvation.) But we can’t just clamp the delay to an upper bound, else it would take a long time to make a reasonable number of tries.

Two separate effects are at work:

Un-jittered backoff would synchronise the spinners. If every waiter sleeps for exactly 1 ms, they all wake up at exactly the same time, all collide, and all go back to sleep for exactly 1 ms. The lock holder gets one collision per millisecond instead of a smooth trickle of retries. Worse, the collisions mean the lock’s cache line ping-pongs between cores.
Un-jittered backoff starves the holder. On a machine with more runnable threads than cores, a spinner that wakes up every millisecond is runnable every millisecond. The holder, which needs the CPU to release the lock, gets scheduled less often the more spinners there are. The jitter breaks that pattern: spinners wake at uncorrelated times, so the holder gets a chance to run between them.

The pg_prng_double() call uses the backend’s global PRNG state, so the jitter differs between backends — two spinners on the same lock will not pick the same delay.

The adaptive spin count

How many times should we spin before sleeping? Too few and we pay a kernel round trip for a lock that would have been free in a microsecond; too many and we burn CPU on a lock that is never going to be released soon. PostgreSQL guesses, and then learns.

The guess starts at 100:

 c
 
 
// src/include/storage/s_lock.h
/* Support for dynamic adjustment of spins_per_delay */
#define DEFAULT_SPINS_PER_DELAY 100

The learning happens in two steps. First, locally, at the end of each contended acquisition:

 c
 
 
// src/backend/storage/lmgr/s_lock.c
/*
 * After acquiring a spinlock, update estimates about how long to loop.
 *
 * If we were able to acquire the lock without delaying, it's a good
 * indication we are in a multiprocessor. If we had to delay, it's a sign
 * (but not a sure thing) that we are in a uniprocessor. Hence, we
 * decrement spins_per_delay slowly when we had to delay, and increase it
 * rapidly when we didn't. It's expected that spins_per_delay will
 * converge to the minimum value on a uniprocessor and to the maximum
 * value on a multiprocessor.
 *
 * Note: spins_per_delay is local within our current process. We want to
 * average these observations across multiple backends, since it's
 * relatively rare for this function to even get entered, and so a single
 * backend might not live long enough to converge on a good value. That
 * is handled by the two routines below.
 */
void
finish_spin_delay(SpinDelayStatus *status)
{
	if (status->cur_delay == 0)
	{
		/* we never had to delay */
		if (spins_per_delay < MAX_SPINS_PER_DELAY)
			spins_per_delay = Min(spins_per_delay + 100, MAX_SPINS_PER_DELAY);
	}
	else
	{
		if (spins_per_delay > MIN_SPINS_PER_DELAY)
			spins_per_delay = Max(spins_per_delay - 1, MIN_SPINS_PER_DELAY);
	}
}

The asymmetry is the interesting part: +100 when we got the lock without sleeping, −1 when we had to sleep. The reasoning is that “we never had to delay” is a strong, cheap signal that spinning is worth it, whereas “we had to delay” is a weak signal that it is not — maybe the lock holder was descheduled, maybe it was just unlucky. So the estimate ratchets up quickly and down slowly.

Second, globally. The local value is folded into a shared estimate at backend exit:

 c
 
 
// src/backend/storage/lmgr/s_lock.c
/*
 * Update shared estimate of spins_per_delay during backend exit.
 *
 * NB: this has to be pretty fast as it is called while holding a spinlock
 */
int
update_spins_per_delay(int shared_spins_per_delay)
{
	/*
	 * We use an exponential moving average with a relatively slow adaption
	 * rate, so that noise in any one backend's result won't affect the shared
	 * value too much. As long as both inputs are within the allowed range,
	 * the result must be too, so we need not worry about clamping the result.
	 *
	 * We deliberately truncate rather than rounding; this is so that single
	 * adjustments inside a backend can affect the shared estimate (see the
	 * asymmetric adjustment rules above).
	 */
	return (shared_spins_per_delay * 15 + spins_per_delay) / 16;
}

(shared * 15 + local) / 16 is an exponential moving average with a weight of 1/16 on the new sample. So a single backend’s experience moves the shared estimate by at most 1/16 of the distance between the two values — and with the truncation, it moves it by at least one unit, which is why the comment mentions “we deliberately truncate rather than rounding”.

The shared value lives in ProcGlobal->spins_per_delay and is read into each backend’s local copy at startup:

 c
 
 
// src/backend/storage/lmgr/proc.c
	/* Update shared estimate of spins_per_delay */
	ProcGlobal->spins_per_delay = update_spins_per_delay(ProcGlobal->spins_per_delay);

	SpinLockRelease(&ProcGlobal->freeProcsLock);

So the estimate is a genuinely global, slowly-adapting, cross-backend number — and it converges, over time, to 1000 (spin hard) on a machine where locks are usually free, or to 10 (give up quickly) on a machine where they are not.

Stuck spinlock detection

If the lock is never released, the loop must terminate. After NUM_DELAYS (1000) sleeps, the backend panics:

 c
 
 
// src/backend/storage/lmgr/s_lock.c
/*
 * s_lock_stuck() - complain about a stuck spinlock
 */
static void
s_lock_stuck(const char *file, int line, const char *func)
{
	if (!func)
		func = "(unknown)";
#if defined(S_LOCK_TEST)
	fprintf(stderr,
			"\nStuck spinlock detected at %s, %s:%d.\n",
			func, file, line);
	exit(1);
#else
	elog(PANIC, "stuck spinlock detected at %s, %s:%d",
		 func, file, line);
#endif
}

The file header explains the timeout philosophy:

We time out and declare error after NUM_DELAYS delays (thus, exactly that many tries). With the given settings, this will usually take 2 or so minutes. It seems better to fix the total number of tries (and thus the probability of unintended failure) than to fix the total time spent.

Fixing the number of tries rather than the elapsed time means the failure probability does not depend on how loaded the machine is. And the message includes __FILE__, __LINE__ and __func__ of the spinlock’s acquisition site, which is the first thing you want to know when the postmaster takes a PANIC.

Note also that the spin delay is a wait event:

 c
 
 
// src/backend/storage/lmgr/s_lock.c
		pgstat_report_wait_start(WAIT_EVENT_SPIN_DELAY);
		pg_usleep(status->cur_delay);
		pgstat_report_wait_end();

The event is called SpinDelay in the wait event catalogue (src/backend/utils/activity/wait_event_names.txt):

text

SPIN_DELAY	"Waiting while acquiring a contended spinlock."

So pg_stat_activity will show a backend in SpinDelay state — but only when it is sleeping, not when it is actively spinning. The comment in the code explains why the reporting is only done around the sleep: “Actively spinning easily stands out in profilers, but sleeping with an exponential backoff is harder to spot”. If you see a backend burning CPU with no wait event at all, that is a spinner.

Layer 2: the atomics

Spinlocks are the blunt instrument. PostgreSQL also has a proper compare-and-swap API, in src/include/port/atomics.h and the per-architecture headers under src/include/port/atomics/.

The x86 implementations

 c
 
 
// src/include/port/atomics/arch-x86.h
#define PG_HAVE_ATOMIC_COMPARE_EXCHANGE_U32
static inline bool
pg_atomic_compare_exchange_u32_impl(volatile pg_atomic_uint32 *ptr,
									uint32 *expected, uint32 newval)
{
	char	ret;

	/*
	 * Perform cmpxchg and use the zero flag which it implicitly sets when
	 * equal to measure the success.
	 */
	__asm__ __volatile__(
	"	lock				\n"
	"	cmpxchgl	%4,%5	\n"
	" setz		%2		\n"
:		"=a" (*expected), "=m"(ptr->value), "=q" (ret)
:		"a" (*expected), "r" (newval), "m"(ptr->value)
:		"memory", "cc");
	return (bool) ret;
}

#define PG_HAVE_ATOMIC_FETCH_ADD_U32
static inline uint32
pg_atomic_fetch_add_u32_impl(volatile pg_atomic_uint32 *ptr, int32 add_)
{
	uint32 res;
	__asm__ __volatile__(
	"	lock				\n"
	"	xaddl	%0,%1		\n"
:		"=q"(res), "=m"(ptr->value)
:		"0" (add_), "m"(ptr->value)
:		"memory", "cc");
	return res;
}

Two instructions, two ideas:

lock cmpxchgl compares the accumulator (%eax) with the destination; if they are equal, the destination is loaded with the source. The old destination value is written back into %eax on failure — which is where the “expected” value comes from. The lock prefix makes the whole thing atomic.
setz reads the zero flag that cmpxchg implicitly sets when the comparison succeeded, and turns it into a 0-or-1 byte. This is the trick the comment refers to: “Perform cmpxchg and use the zero flag which it implicitly sets when equal to measure the success.” Rather than branching (which would be a mispredict), the flag is materialized as a value with setcc.
lock xaddl is “exchange and add”: it atomically adds the source to the destination and returns the old value. One instruction, no loop, no retry.

Note the memory clobbers: "memory", "cc". The memory clobber tells the compiler that the asm may read or write arbitrary memory, so it cannot keep anything in a register across it; the cc clobber tells it that the condition codes were modified.

The API contract

 c
 
 
// src/include/port/atomics.h
/*
 * pg_atomic_compare_exchange_u32 - CAS operation
 *
 * Atomically compare the current value of ptr with *expected and store newval
 * iff ptr and *expected have the same value. The current value of *ptr will
 * always be stored in *expected.
 *
 * Return true if values have been exchanged, false otherwise.
 *
 * Full barrier semantics.
 */

That third sentence — “The current value of *ptr will always be stored in *expected” — is the reason the CAS-loop implementations of the other operations are so short.

The CAS loops

Not every operation has a dedicated instruction on every architecture, so generic.h builds them out of the CAS primitive:

 c
 
 
// src/include/port/atomics/generic.h
#if !defined(PG_HAVE_ATOMIC_FETCH_AND_U32) && defined(PG_HAVE_ATOMIC_COMPARE_EXCHANGE_U32)
#define PG_HAVE_ATOMIC_FETCH_AND_U32
static inline uint32
pg_atomic_fetch_and_u32_impl(volatile pg_atomic_uint32 *ptr, uint32 and_)
{
	uint32 old;
	old = ptr->value;			/* ok if read is not atomic */
	while (!pg_atomic_compare_exchange_u32_impl(ptr, &old, old & and_))
		/* skip */;
	return old;
}
#endif

#if !defined(PG_HAVE_ATOMIC_FETCH_OR_U32) && defined(PG_HAVE_ATOMIC_COMPARE_EXCHANGE_U32)
#define PG_HAVE_ATOMIC_FETCH_OR_U32
static inline uint32
pg_atomic_fetch_or_u32_impl(volatile pg_atomic_uint32 *ptr, uint32 or_)
{
	uint32 old;
	old = ptr->value;			/* ok if read is not atomic */
	while (!pg_atomic_compare_exchange_u32_impl(ptr, &old, old | or_))
		/* skip */;
	return old;
}
#endif

The neat part is that the retry needs no extra load. A failed cmpxchg writes the current value of *ptr back into *expected, which is the same variable the loop uses as its source of truth. So after a failure, old already holds the fresh value, and the next iteration computes old & and_ (or old | or_) from it. The comment /* ok if read is not atomic */ on the initial load acknowledges the one non-atomic read in the function: it is only a starting guess, and the CAS loop corrects it immediately.

The same pattern gives you fetch_sub, exchange, and add_fetch:

 c
 
 
// src/include/port/atomics/generic.h
#if !defined(PG_HAVE_ATOMIC_EXCHANGE_U32) && defined(PG_HAVE_ATOMIC_COMPARE_EXCHANGE_U32)
#define PG_HAVE_ATOMIC_EXCHANGE_U32
static inline uint32
pg_atomic_exchange_u32_impl(volatile pg_atomic_uint32 *ptr, uint32 xchg_)
{
	uint32 old;
	old = ptr->value;			/* ok if read is not atomic */
	while (!pg_atomic_compare_exchange_u32_impl(ptr, &old, xchg_))
		/* skip */;
	return old;
}
#endif

Memory barriers

 c
 
 
// src/include/port/atomics/arch-x86.h
/*
 * Both 32 and 64 bit x86 do not allow loads to be reordered with other loads,
 * or stores to be reordered with other stores, but a load can be performed
 * before a subsequent store.
 *
 * Technically, some x86-ish chips support uncached memory access and/or
 * special instructions that are weakly ordered. In those cases we'd need
 * the read and write barriers to be lfence and sfence. But since we don't
 * do those things, a compiler barrier should be enough.
 *
 * "lock; addl" has worked for longer than "mfence". It's also rumored to be
 * faster in many scenarios.
 */
#if defined(__GNUC__) || defined(__INTEL_COMPILER)
#if defined(__i386__)
#define pg_memory_barrier_impl()		\
	__asm__ __volatile__ ("lock; addl $0,0(%%esp)" : : : "memory", "cc")
#elif defined(__x86_64__)
#define pg_memory_barrier_impl()		\
	__asm__ __volatile__ ("lock; addl $0,0(%%rsp)" : : : "memory", "cc")
#endif
#endif /* defined(__GNUC__) || defined(__INTEL_COMPILER) */

#define pg_read_barrier_impl()		pg_compiler_barrier_impl()
#define pg_write_barrier_impl()		pg_compiler_barrier_impl()

lock; addl $0, 0(%rsp) is the classic x86 full barrier: it adds zero to a stack location, but the lock prefix makes it an atomic read-modify-write, which is a full memory barrier. And on x86, a read or write barrier is just a compiler barrier, because the hardware’s memory model is strong enough that you never need more than that.

The 64-bit fallback

On a 32-bit platform without a 64-bit CAS instruction, fallback.h provides an emulated 64-bit atomic — and the emulation is, of course, a spinlock:

 c
 
 
// src/include/port/atomics/fallback.h
/*
 * fallback.h
 * Fallback for platforms without 64 bit atomics support. Slower
 * than native atomics support, but not unusably slow.
 */

#if !defined(PG_HAVE_ATOMIC_U64_SUPPORT)

#define PG_HAVE_ATOMIC_U64_SIMULATION

#define PG_HAVE_ATOMIC_U64_SUPPORT
typedef struct pg_atomic_uint64
{
	int			sema;
	volatile uint64 value;
} pg_atomic_uint64;

And the implementation, in src/backend/port/atomics.c:

 c
 
 
// src/backend/port/atomics.c
bool
pg_atomic_compare_exchange_u64_impl(volatile pg_atomic_uint64 *ptr,
									uint64 *expected, uint64 newval)
{
	bool		ret;

	/*
	 * Do atomic op under a spinlock. It might look like we could just skip
	 * the cmpxchg if the lock isn't available, but that'd just emulate a
	 * 'weak' compare and swap. I.e. one that allows spurious failures. Since
	 * several algorithms rely on a strong variant and that is efficiently
	 * implementable on most major architectures let's emulate it here as
	 * well.
	 */
	SpinLockAcquire((slock_t *) &ptr->sema);

	/* perform compare/exchange logic */
	ret = ptr->value == *expected;
	*expected = ptr->value;
	if (ret)
		ptr->value = newval;

	/* and release lock */
	SpinLockRelease((slock_t *) &ptr->sema);

	return ret;
}

The comment is a nice piece of engineering reasoning: a lock-free implementation would be tempted to skip the compare-and-swap when the lock is contended, but that would be a weak CAS — one that may fail spuriously. Several algorithms in PostgreSQL depend on the strong variant, so the fallback implements the strong one, at the cost of spinning.

Note that this is a recursive layering: the 64-bit atomic’s implementation uses the 32-bit spinlock, which uses S_LOCK, which uses perform_spin_delay(). The whole ladder is recursive.

Layer 3: the LWLock

An LWLock is a shared-memory reader/writer lock. It is “lightweight” compared to the heavyweight lock manager in the sense that it has no owner, no lock table, no deadlock detection, and no interaction with pg_locks.

The state word

Everything about an LWLock’s state lives in one 32-bit word:

 c
 
 
// src/include/storage/lmgr/lwlock.h
typedef struct LWLock
{
	uint16		tranche;		/* tranche ID */
	pg_atomic_uint32 state;		/* state of exclusive/nonexclusive lockers */
	proclist_head waiters;		/* list of waiting PGPROCs */
#ifdef LOCK_DEBUG
	pg_atomic_uint32 nwaiters;	/* number of waiters */
	struct PGPROC *owner;		/* last exclusive owner of the lock */
#endif
} LWLock;

And the layout of that word:

 c
 
 
// src/backend/storage/lmgr/lwlock.c
#define LW_FLAG_HAS_WAITERS			((uint32) 1 << 31)
#define LW_FLAG_WAKE_IN_PROGRESS	((uint32) 1 << 30)
#define LW_FLAG_LOCKED				((uint32) 1 << 29)
#define LW_FLAG_BITS				3
#define LW_FLAG_MASK				(((1<<LW_FLAG_BITS)-1)<<(32-LW_FLAG_BITS))

/* assumes MAX_BACKENDS is a (power of 2) - 1, checked below */
#define LW_VAL_EXCLUSIVE			(MAX_BACKENDS + 1)
#define LW_VAL_SHARED				1

/* already (power of 2)-1, i.e. suitable for a mask */
#define LW_SHARED_MASK				MAX_BACKENDS
#define LW_LOCK_MASK				(MAX_BACKENDS | LW_VAL_EXCLUSIVE)

Three flags in the top three bits, and the bottom bits hold the holder count:

Bits	Meaning
31	`LW_FLAG_HAS_WAITERS` — somebody is queued
30	`LW_FLAG_WAKE_IN_PROGRESS` — a wakeup is in flight
29	`LW_FLAG_LOCKED` — the wait list is locked
28–0	shared holder count, or `LW_VAL_EXCLUSIVE` for an exclusive holder

The scheme is described in the file header:

The basic idea is to have a single atomic variable ’lockcount’ instead of the formerly separate shared and exclusive counters and to use atomic operations to acquire the lock. That’s fairly easy to do for plain rw-spinlocks, but a lot harder for something like LWLocks that want to wait in the OS.

For lock acquisition we use an atomic compare-and-exchange on the lockcount variable. For exclusive lock we swap in a sentinel value (LW_VAL_EXCLUSIVE), for shared locks we count the number of holders.

To release the lock we use an atomic decrement to release the lock. If the new value is zero (we get that atomically), we know we can/have to release waiters.

LW_VAL_EXCLUSIVE is MAX_BACKENDS + 1, chosen so that it cannot be confused with a shared holder count — there can never be more than MAX_BACKENDS shared holders, so the two ranges do not overlap. The static assertions check exactly that:

 c
 
 
// src/backend/storage/lmgr/lwlock.c
StaticAssertDecl(((MAX_BACKENDS + 1) & MAX_BACKENDS) == 0,
				 "MAX_BACKENDS + 1 needs to be a power of 2");

StaticAssertDecl((MAX_BACKENDS & LW_FLAG_MASK) == 0,
				 "MAX_BACKENDS and LW_FLAG_MASK overlap");

StaticAssertDecl((LW_VAL_EXCLUSIVE & LW_FLAG_MASK) == 0,
				 "LW_VAL_EXCLUSIVE and LW_FLAG_MASK overlap");

The attempt

 c
 
 
// src/backend/storage/lmgr/lwlock.c
/*
 * Internal function that tries to atomically acquire the lwlock in the passed
 * in mode.
 *
 * This function will not block waiting for a lock to become free - that's the
 * caller's job.
 *
 * Returns true if the lock isn't free and we need to wait.
 */
static bool
LWLockAttemptLock(LWLock *lock, LWLockMode mode)
{
	uint32		old_state;

	Assert(mode == LW_EXCLUSIVE || mode == LW_SHARED);

	/*
	 * Read once outside the loop, later iterations will get the newer value
	 * via compare & exchange.
	 */
	old_state = pg_atomic_read_u32(&lock->state);

	/* loop until we've determined whether we could acquire the lock or not */
	while (true)
	{
		uint32		desired_state;
		bool		lock_free;

		desired_state = old_state;

		if (mode == LW_EXCLUSIVE)
		{
			lock_free = (old_state & LW_LOCK_MASK) == 0;
			if (lock_free)
				desired_state += LW_VAL_EXCLUSIVE;
		}
		else
		{
			lock_free = (old_state & LW_VAL_EXCLUSIVE) == 0;
			if (lock_free)
				desired_state += LW_VAL_SHARED;
		}

		/*
		 * Attempt to swap in the state we are expecting. If we didn't see
		 * lock to be free, that's just the old value. If we saw it as free,
		 * we'll attempt to mark it acquired. The reason that we always swap
		 * in the value is that this doubles as a memory barrier. We could try
		 * to be smarter and only swap in values if we saw the lock as free,
		 * but benchmark haven't shown it as beneficial so far.
		 *
		 * Retry if the value changed since we last looked at it.
		 */
		if (pg_atomic_compare_exchange_u32(&lock->state,
										 &old_state, desired_state))
		{
			if (lock_free)
			{
				/* Great! Got the lock. */
#ifdef LOCK_DEBUG
				if (mode == LW_EXCLUSIVE)
					lock->owner = MyProc;
#endif
				return false;
			}
			else
				return true;	/* somebody else has the lock */
		}
	}
	pg_unreachable();
}

The comment about the memory barrier is the interesting part:

The reason that we always swap in the value is that this doubles as a memory barrier. We could try to be smarter and only swap in values if we saw the lock as free, but benchmark haven’t shown it as beneficial so far.

On x86, lock cmpxchg is a full barrier whatever it does. So whether the CAS succeeds or fails, the CPU has executed a full fence. The code takes advantage of that unconditionally: it always issues the CAS, even in the case where it already knows the lock is held and it is going to have to wait anyway. The alternative — checking first and skipping the CAS — would save an atomic operation on the contended path, but benchmarks said it wasn’t worth it. (And on a weak-memory architecture, the unconditional CAS is what gives you the barrier for free; skipping it would require an explicit fence.)

Also note the loop structure: old_state is refreshed by the failed CAS (the “current value of *ptr will always be stored in *expected” contract), so the retry always operates on the freshest value.

The wait list lock is a bit in the same word

The LW_FLAG_LOCKED bit in the state word is a spinlock protecting the waiters list. This is a neat bit of economy: the wait list lock lives in the same cache line as the state word, so taking it does not cost an extra cache miss.

 c
 
 
// src/backend/storage/lmgr/lwlock.c
/*
 * Lock the LWLock's wait list against concurrent activity.
 *
 * NB: even though the wait list is locked, non-conflicting lock operations
 * may still happen concurrently.
 *
 * Time spent holding mutex should be short!
 */
static void
LWLockWaitListLock(LWLock *lock)
{
	uint32		old_state;
#ifdef LWLOCK_STATS
	lwlock_stats *lwstats;
	uint32		delays = 0;

	lwstats = get_lwlock_stats_entry(lock);
#endif

	while (true)
	{
		/*
		 * Always try once to acquire the lock directly, without setting up
		 * the spin-delay infrastructure. The work necessary for that shows up
		 * in profiles and is rarely necessary.
		 */
		old_state = pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_LOCKED);
		if (likely(!(old_state & LW_FLAG_LOCKED)))
			break;				/* got lock */

		/* and then spin without atomic operations until lock is released */
		{
			SpinDelayStatus delayStatus;

			init_local_spin_delay(&delayStatus);

			while (old_state & LW_FLAG_LOCKED)
			{
				perform_spin_delay(&delayStatus);
				old_state = pg_atomic_read_u32(&lock->state);
			}
#ifdef LWLOCK_STATS
			delays += delayStatus.delays;
#endif
			finish_spin_delay(&delayStatus);
		}

		/*
		 * Retry. The lock might obviously already be re-acquired by the time
		 * we're attempting to get it again.
		 */
	}

#ifdef LWLOCK_STATS
	lwstats->spin_delay_count += delays;
#endif
}

Two things to notice:

“Always try once to acquire the lock directly, without setting up the spin-delay infrastructure.” The SpinDelayStatus struct has to be initialised, and init_local_spin_delay() writes five fields. For the overwhelmingly common case where the wait list is not contended, that setup is pure overhead — and the comment says it “shows up in profiles”. So the code tries the atomic fetch-or first and only sets up the spin machinery if that fails.
“spin without atomic operations until lock is released” — inside the spin loop, the state word is read with a plain atomic read (pg_atomic_read_u32), not a CAS loop. There is no attempt to take the lock during the spin; the spinner just watches the bit. Only when the bit clears does it retry the fetch-or. That avoids hammering the cache line with atomic RMWs.

Queue-self-then-retry

Now the interesting algorithmic part. The file header describes the two-phase approach:

The attentive reader might have noticed that naively doing the above has a glaring race condition: We try to lock using the atomic operations and notice that we have to wait. Unfortunately by the time we have finished queuing, the former locker very well might have already finished its work. That’s problematic because we’re now stuck waiting inside the OS.

To mitigate those races we use a two phased attempt at locking: Phase 1: Try to do it atomically, if we succeed, nice Phase 2: Add ourselves to the waitqueue of the lock Phase 3: Try to grab the lock again, if we succeed, remove ourselves from the queue Phase 4: Sleep till wake-up, goto Phase 1

And the implementation:

 c
 
 
// src/backend/storage/lmgr/lwlock.c
	for (;;)
	{
		bool		mustwait;

		/*
		 * Try to grab the lock the first time, we're not in the waitqueue
		 * yet/anymore.
		 */
		mustwait = LWLockAttemptLock(lock, mode);

		if (!mustwait)
		{
			LOG_LWDEBUG("LWLockAcquire", lock, "immediately acquired lock");
			break;				/* got the lock */
		}

		/*
		 * Ok, at this point we couldn't grab the lock on the first try. We
		 * cannot simply queue ourselves to the end of the list and wait to be
		 * woken up because by now the lock could long have been released.
		 * Instead add us to the queue and try to grab the lock again. If we
		 * succeed we need to revert the queuing and be happy, otherwise we
		 * recheck the lock. If we still couldn't grab it, we know that the
		 * other locker will see our queue entries when releasing since they
		 * existed before we checked for the lock.
		 */

		/* add to the queue */
		LWLockQueueSelf(lock, mode);

		/* we're now guaranteed to be woken up if necessary */
		mustwait = LWLockAttemptLock(lock, mode);

		/* ok, grabbed the lock the second time round, need to undo queueing */
		if (!mustwait)
		{
			LOG_LWDEBUG("LWLockAcquire", lock, "acquired, undoing queue");

			LWLockDequeueSelf(lock);
			break;
		}

		/*
		 * Wait until awakened.
		 *
		 * It is possible that we get awakened for a reason other than being
		 * signaled by LWLockRelease. If so, loop back and wait again. Once
		 * we've gotten the LWLock, re-increment the sema by the number of
		 * additional signals received.
		 */
		LOG_LWDEBUG("LWLockAcquire", lock, "waiting");

		LWLockReportWaitStart(lock);
		...
		for (;;)
		{
			PGSemaphoreLock(proc->sem);
			if (proc->lwWaiting == LW_WS_NOT_WAITING)
				break;
			extraWaits++;
		}

		/* Retrying, allow LWLockRelease to release waiters again. */
		pg_atomic_fetch_and_u32(&lock->state, ~LW_FLAG_WAKE_IN_PROGRESS);

		...
		/* Now loop back and try to acquire lock again. */
		result = false;
	}

The critical insight is in the comment: queue first, then try again. If you tried the lock, failed, and then queued, the holder could have released the lock in the window between your failed attempt and your queueing — and nobody would ever wake you up, because the release had already happened. By queueing before the second attempt, you guarantee that any release that happens after your first failed attempt will see you on the wait list and wake you.

The queueing itself:

 c
 
 
// src/backend/storage/lmgr/lwlock.c
static void
LWLockQueueSelf(LWLock *lock, LWLockMode mode)
{
	/*
	 * If we don't have a PGPROC structure, there's no way to wait. This
	 * should never occur, since MyProc should only be null during shared
	 * memory initialization.
	 */
	if (MyProc == NULL)
		elog(PANIC, "cannot wait without a PGPROC structure");

	if (MyProc->lwWaiting != LW_WS_NOT_WAITING)
		elog(PANIC, "queueing for lock while waiting on another one");

	LWLockWaitListLock(lock);

	/* setting the flag is protected by the spinlock */
	pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_HAS_WAITERS);

	MyProc->lwWaiting = LW_WS_WAITING;
	MyProc->lwWaitMode = mode;

	/* LW_WAIT_UNTIL_FREE waiters are always at the front of the queue */
	if (mode == LW_WAIT_UNTIL_FREE)
		proclist_push_head(&lock->waiters, MyProcNumber, lwWaitLink);
	else
		proclist_push_tail(&lock->waiters, MyProcNumber, lwWaitLink);

	/* Can release the mutex now */
	LWLockWaitListUnlock(lock);
	...
}

Note the panic on double-queueing: elog(PANIC, "queueing for lock while waiting on another one"). An LWLock is a leaf lock — you may hold at most one while waiting for another.

Sleeping: a POSIX semaphore, not a futex

The actual blocking is PGSemaphoreLock(proc->sem) — a POSIX semaphore stored in shared memory, one per PGPROC:

 c
 
 
// src/backend/port/posix_sema.c
/*
 * PGSemaphoreLock
 *
 * Lock a semaphore (decrement count), blocking if count would be < 0
 */
void
PGSemaphoreLock(PGSemaphore sema)
{
	int			errStatus;

	/* See notes in sysv_sema.c's implementation of PGSemaphoreLock. */
	do
	{
		errStatus = sem_wait(PG_SEM_REF(sema));
	} while (errStatus < 0 && errno == EINTR);

	if (errStatus < 0)
		elog(FATAL, "sem_wait failed: %m");
}

The semaphore is padded to a cache line to avoid false sharing:

 c
 
 
// src/backend/port/posix_sema.c
typedef union SemTPadded
{
	sem_t		pgsem;
	char		pad[PG_CACHE_LINE_SIZE];
} SemTPadded;

There is no futex here. PostgreSQL does not implement the Linux futex fast path, so every LWLock wait that reaches the kernel costs a real sem_wait syscall, and every wakeup costs a sem_post. The semaphore implementations are chosen at configure time:

unnamed POSIX semaphores (sem_init/sem_wait), the default where available;
named POSIX semaphores (sem_open);
SysV semaphores (semop) as a fallback.

Releasing and waking

The release path wakes waiters, and it uses the LW_FLAG_WAKE_IN_PROGRESS bit to make sure only one waker is in flight at a time:

 c
 
 
// src/backend/storage/lmgr/lwlock.c
	/* unset required flags, and release lock, in one fell swoop */
	{
		uint32		old_state;
		uint32		desired_state;

		old_state = pg_atomic_read_u32(&lock->state);
		while (true)
		{
			desired_state = old_state;

			/* compute desired flags */

			if (new_wake_in_progress)
				desired_state |= LW_FLAG_WAKE_IN_PROGRESS;
			else
				desired_state &= ~LW_FLAG_WAKE_IN_PROGRESS;

			if (proclist_is_empty(&lock->waiters))
				desired_state &= ~LW_FLAG_HAS_WAITERS;

			desired_state &= ~LW_FLAG_LOCKED;	/* release lock */

			if (pg_atomic_compare_exchange_u32(&lock->state, &old_state,
											 desired_state))
				break;
		}
	}

	/* Awaken any waiters I removed from the queue. */
	proclist_foreach_modify(iter, &wakeup, lwWaitLink)
	{
		PGPROC	 *waiter = GetPGProcByNumber(iter.cur);

		LOG_LWDEBUG("LWLockRelease", lock, "release waiter");
		proclist_delete(&wakeup, iter.cur, lwWaitLink);

		/*
		 * Guarantee that lwWaiting being unset only becomes visible once the
		 * unlink from the link has completed. Otherwise the target backend
		 * could be woken up for other reason and enqueue for a new lock - if
		 * that happens before the list unlink happens, the list would end up
		 * being corrupted.
		 *
		 * The barrier pairs with the LWLockWaitListLock() when enqueuing for
		 * another lock.
		 */
		pg_write_barrier();
		waiter->lwWaiting = LW_WS_NOT_WAITING;
		PGSemaphoreUnlock(waiter->sem);
	}

Two details: the flags are updated with a single CAS so that the wait-list lock is released and the flags are cleared atomically together (otherwise a racing LWLockQueueSelf() could set LW_FLAG_HAS_WAITERS after the releaser checked the list and before it released the lock, and the flag would be stuck on forever); and the write barrier before clearing lwWaiting ensures that the unlink from the wait list is visible before the waiter can wake up and try to queue for another lock.

Deadlock detection: not here

LWLocks have no deadlock detection, and this is not an oversight — it is a design decision. An LWLock is a leaf lock: it protects a small piece of shared state, and the code that holds it does not wait on other locks while holding it. There is no wait-for graph to inspect.

Deadlock detection lives in the heavyweight lock manager instead, in src/backend/storage/lmgr/deadlock.c, which maintains a wait-for graph over the heavyweight locks:

DeadLockCheck – Checks for deadlocks for a given process

This code looks for deadlocks involving the given process. If any are found, we try to resolve them by rearranging the wait queues. If resolution is impossible, return DS_HARD_DEADLOCK — the process must then throw an error.

The practical consequence: if an LWLock deadlock happens, PostgreSQL does not detect it. The backends sit there until deadlock_timeout fires — but that timeout applies to lock waits that the lock manager knows about, not to arbitrary LWLock waits.

Layer 4: latches

A latch is not a lock. It is a flag in shared memory plus a way to wake up whoever is waiting on it. MyLatch is the one every backend has, and it is what WaitLatch() waits on.

SetLatch

 c
 
 
// src/backend/storage/ipc/latch.c
/*
 * Sets a latch and wakes up anyone waiting on it.
 *
 * This is cheap if the latch is already set, otherwise not so much.
 *
 * NB: when calling this in a signal handler, be sure to save and restore
 * errno around it. (That's standard practice in most signal handlers, of
 * course, but we used to omit it in handlers that only set a flag.)
 *
 * NB: this function is called from critical sections and signal handlers so
 * throwing an error is not a good idea.
 */
void
SetLatch(Latch *latch)
{
	...
	/*
	 * The memory barrier has to be placed here to ensure that any flag
	 * variables possibly changed by this process have been flushed to main
	 * memory, before we check/set is_set.
	 */
	pg_memory_barrier();

	/* Quick exit if already set */
	if (latch->is_set)
		return;

	latch->is_set = true;

	pg_memory_barrier();
	if (!latch->maybe_sleeping)
		return;

	...
	owner_pid = latch->owner_pid;
	if (owner_pid == 0)
		return;
	else if (owner_pid == MyProcPid)
		WakeupMyProc();
	else
		WakeupOtherProc(owner_pid);
	...
}

Four things, in order:

A full memory barrier before the test. Any flag variables the process has changed must be visible in main memory before we publish is_set. Otherwise a waiter could see is_set == true, wake up, and look at flags that still have stale values. This is the classic Dekker-style store-load ordering problem, and the barrier is what makes it safe.
“Quick exit if already set.” Setting an already-set latch is a no-op. This is the common case in a signal handler that fires repeatedly, and it means SetLatch() is cheap enough to call from anywhere.
A second memory barrier after setting is_set. This makes the store to is_set visible before we look at maybe_sleeping.
The maybe_sleeping test. This is the optimisation that matters: if nobody is waiting on the latch, do not signal anybody. Signalling is expensive (a syscall, or a write to the self-pipe), and the overwhelmingly common case for SetLatch() is “set the flag and carry on”. The maybe_sleeping flag is set by the waiter just before it blocks:
c
```
		// src/backend/storage/ipc/waiteventset.c
		if (set->latch && !set->latch->is_set)
		{
			/* about to sleep on a latch */
			set->latch->maybe_sleeping = true;
			pg_memory_barrier();
			/* and recheck */
		}

		if (set->latch && set->latch->is_set)
		{
			...
		}
```
The waiter sets maybe_sleeping, issues a barrier, and then re-checks is_set. That re-check is the other half of the protocol: if the latch was set in the window between the waiter’s check and its setting of maybe_sleeping, the re-check catches it and the waiter never sleeps at all. If the latch is set after the re-check, the setter will see maybe_sleeping == true and will signal. Between the two, no wakeup is lost.

Note the ordering of the two flags: the waiter publishes maybe_sleeping before the final check of is_set, and the setter publishes is_set before the final check of maybe_sleeping. That is the standard “store both, then check both” pattern, and it is why there are two barriers in SetLatch().

ResetLatch

 c
 
 
// src/backend/storage/ipc/latch.c
/*
 * Clear the latch. Calling WaitLatch after this will sleep, unless
 * the latch is set again before the WaitLatch call.
 */
void
ResetLatch(Latch *latch)
{
	/* Only the owner should reset the latch */
	Assert(latch->owner_pid == MyProcPid);
	Assert(latch->maybe_sleeping == false);

	latch->is_set = false;

	/*
	 * Ensure that the write to is_set gets flushed to main memory before we
	 * examine any flag variables. Otherwise a concurrent SetLatch might
	 * falsely conclude that it needn't signal us, even though we have missed
	 * seeing some flag updates that SetLatch was supposed to inform us of.
	 */
	pg_memory_barrier();
}

Symmetrically, the barrier after clearing is_set ensures that the clear is visible before the owner examines any flags it might use to decide what to do next. Without it, a concurrent SetLatch() could see the stale is_set == true, decide “already set, nothing to do”, and skip the signal — while the owner has already decided to go to sleep.

WaitLatch and the long-lived WaitEventSet

 c
 
 
// src/backend/storage/ipc/latch.c
/* A common WaitEventSet used to implement WaitLatch() */
static WaitEventSet *LatchWaitSet;

/* The positions of the latch and PM death events in LatchWaitSet */
#define LatchWaitSetLatchPos 0
#define LatchWaitSetPostmasterDeathPos 1

void
InitializeLatchWaitSet(void)
{
	...
	/* Set up the WaitEventSet used by WaitLatch(). */
	LatchWaitSet = CreateWaitEventSet(NULL, 2);
	latch_pos = AddWaitEventToSet(LatchWaitSet, WL_LATCH_SET, PGINVALID_SOCKET,
								 MyLatch, NULL);
	Assert(latch_pos == LatchWaitSetLatchPos);

	/*
	 * WaitLatch will modify this to WL_EXIT_ON_PM_DEATH or
	 * WL_POSTMASTER_DEATH on each call.
	 */
	if (IsUnderPostmaster)
	{
		latch_pos = AddWaitEventToSet(LatchWaitSet, WL_EXIT_ON_PM_DEATH,
									 PGINVALID_SOCKET, NULL, NULL);
		Assert(latch_pos == LatchWaitSetPostmasterDeathPos);
	}
}

One WaitEventSet, created once per backend, reused forever. Every WaitLatch() call does not create a new epoll set; it just modifies the existing one:

 c
 
 
// src/backend/storage/ipc/latch.c
int
WaitLatch(Latch *latch, int wakeEvents, long timeout,
		 uint32 wait_event_info)
{
	WaitEvent	event;

	/* Postmaster-managed callers must handle postmaster death somehow. */
	Assert(!IsUnderPostmaster ||
		 (wakeEvents & WL_EXIT_ON_PM_DEATH) ||
		 (wakeEvents & WL_POSTMASTER_DEATH));

	/*
	 * Some callers may have a latch other than MyLatch, or no latch at all,
	 * or want to handle postmaster death differently. It's cheap to assign
	 * those, so just do it every time.
	 */
	if (!(wakeEvents & WL_LATCH_SET))
		latch = NULL;
	ModifyWaitEvent(LatchWaitSet, LatchWaitSetLatchPos, WL_LATCH_SET, latch);

	if (IsUnderPostmaster)
		ModifyWaitEvent(LatchWaitSet, LatchWaitSetPostmasterDeathPos,
						(wakeEvents & (WL_EXIT_ON_PM_DEATH | WL_POSTMASTER_DEATH)),
						NULL);

	if (WaitEventSetWait(LatchWaitSet,
						 (wakeEvents & WL_TIMEOUT) ? timeout : -1,
						 &event, 1,
						 wait_event_info) == 0)
		return WL_TIMEOUT;
	else
		return event.events;
}

The comment “Some callers may have a latch other than MyLatch, or no latch at all, or want to handle postmaster death differently. It’s cheap to assign those, so just do it every time” is a nice micro-optimization philosophy: rather than maintaining several WaitEventSets and picking one, keep one and mutate it.

epoll and signalfd versus the classic self-pipe

waiteventset.c has three implementations of the same idea, and the file header explains the race each one has to solve:

 c
 
 
// src/backend/storage/ipc/waiteventset.c
/*
 * The poll() implementation uses the so-called self-pipe trick to overcome the
 * race condition involved with poll() and setting a global flag in the signal
 * handler. When a latch is set and the current process is waiting for it, the
 * signal handler wakes up the poll() in WaitLatch by writing a byte to a pipe.
 * A signal by itself doesn't interrupt poll() on all platforms, and even on
 * platforms where it does, a signal that arrives just before the poll() call
 * does not prevent poll() from entering sleep. An incoming byte on a pipe
 * however reliably interrupts the sleep, and causes poll() to return
 * immediately even if the signal arrives before poll() begins.
 *
 * The epoll() implementation overcomes the race with a different technique: it
 * keeps SIGURG blocked and consumes from a signalfd() descriptor instead. We
 * don't need to register a signal handler or create our own self-pipe. We
 * assume that any system that has Linux epoll() also has Linux signalfd().
 *
 * The kqueue() implementation waits for SIGURG with EVFILT_SIGNAL.
 *
 * The Windows implementation uses Windows events that are inherited by all
 * postmaster child processes. There's no need for the self-pipe trick there.
 */

The choice is made at compile time:

 c
 
 
// src/backend/storage/ipc/waiteventset.c
#if defined(WAIT_USE_EPOLL) || defined(WAIT_USE_POLL) || \
	defined(WAIT_USE_KQUEUE) || defined(WAIT_USE_WIN32)
/* don't overwrite manual choice */
#elif defined(HAVE_SYS_EPOLL_H)
#define WAIT_USE_EPOLL
#elif defined(HAVE_KQUEUE)
#define WAIT_USE_KQUEUE
#elif defined(HAVE_POLL)
#define WAIT_USE_POLL
#elif WIN32
#define WAIT_USE_WIN32
#else
#error "no wait set implementation available"
#endif

/*
 * By default, we use a self-pipe with poll() and a signalfd with epoll(), if
 * available. For testing the choice can also be manually specified.
 */
#if defined(WAIT_USE_POLL) || defined(WAIT_USE_EPOLL)
#if defined(WAIT_USE_SELF_PIPE) || defined(WAIT_USE_SIGNALFD)
/* don't overwrite manual choice */
#elif defined(WAIT_USE_EPOLL) && defined(HAVE_SYS_SIGNALFD_H)
#define WAIT_USE_SIGNALFD
#else
#define WAIT_USE_SELF_PIPE
#endif
#endif

So on Linux: epoll + signalfd. Elsewhere with poll(): poll + self-pipe.

The signalfd variant

 c
 
 
// src/backend/storage/ipc/waiteventset.c
#ifdef WAIT_USE_SIGNALFD
	sigset_t	signalfd_mask;

	...
	/* Block SIGURG, because we'll receive it through a signalfd. */
	sigaddset(&UnBlockSig, SIGURG);

	/* Set up the signalfd to receive SIGURG notifications. */
	sigemptyset(&signalfd_mask);
	sigaddset(&signalfd_mask, SIGURG);
	signal_fd = signalfd(-1, &signalfd_mask, SFD_NONBLOCK | SFD_CLOEXEC);
	if (signal_fd < 0)
		elog(FATAL, "signalfd() failed");
	ReserveExternalFD();
#endif

SIGURG is blocked in the process, so it never runs a handler; instead it becomes readable on the signalfd, and the fd is registered with epoll. When the latch is set, SetLatch() sends SIGURG to itself, the kernel marks the signalfd readable, and epoll_wait() returns.

 c
 
 
// src/backend/storage/ipc/waiteventset.c
	/* Sleep */
	rc = epoll_wait(set->epoll_fd, set->epoll_ret_events,
					Min(nevents, set->nevents_space), cur_timeout);

	/* Check return code */
	if (rc < 0)
	{
		/* EINTR is okay, otherwise complain */
		if (errno != EINTR)
		{
			waiting = false;
			ereport(ERROR,
					(errcode_for_socket_access(),
					 errmsg("%s() failed: %m",
							"epoll_wait")));
		}
		return 0;
	}

Note that epoll_wait is a level-triggered wait on the signalfd: the fd stays readable until the pending signals are consumed, which is what the drain() call does.

The self-pipe variant

The classic trick. The pipe is created with both ends non-blocking and close-on-exec:

 c
 
 
// src/backend/storage/ipc/waiteventset.c
	/*
	 * Set up the self-pipe that allows a signal handler to wake up the
	 * poll()/epoll_wait() in WaitLatch. Make the write-end non-blocking, so
	 * that SetLatch won't block if the event has already been set many times
	 * filling the kernel buffer. Make the read-end non-blocking too, so that
	 * we can easily clear the pipe by reading until EAGAIN or EWOULDBLOCK.
	 * Also, make both FDs close-on-exec, since we surely do not want any
	 * child processes messing with them.
	 */
	if (pipe(pipefd) < 0)
		elog(FATAL, "pipe() failed: %m");
	if (fcntl(pipefd[0], F_SETFL, O_NONBLOCK) == -1)
		elog(FATAL, "fcntl(F_SETFL) failed on read-end of self-pipe: %m");
	if (fcntl(pipefd[1], F_SETFL, O_NONBLOCK) == -1)
		elog(FATAL, "fcntl(F_SETFL) failed on write-end of self-pipe: %m");
	if (fcntl(pipefd[0], F_SETFD, FD_CLOEXEC) == -1)
		elog(FATAL, "fcntl(F_SETFD) failed on read-end of self-pipe: %m");
	if (fcntl(pipefd[1], F_SETFD, FD_CLOEXEC) == -1)
		elog(FATAL, "fcntl(F_SETFD) failed on write-end of self-pipe: %m");

	selfpipe_readfd = pipefd[0];
	selfpipe_writefd = pipefd[1];
	selfpipe_owner_pid = MyProcPid;

And the signal handler:

 c
 
 
// src/backend/storage/ipc/waiteventset.c
/*
 * SetLatch uses SIGURG to wake up the process waiting on the latch.
 *
 * Wake up WaitLatch, if we're waiting.
 */
static void
latch_sigurg_handler(SIGNAL_ARGS)
{
	if (waiting)
		sendSelfPipeByte();
}

/* Send one byte to the self-pipe, to wake up WaitLatch */
static void
sendSelfPipeByte(void)
{
	ssize_t		rc;
	char		dummy = 0;

retry:
	rc = write(selfpipe_writefd, &dummy, 1);
	if (rc < 0)
	{
		/* If interrupted by signal, just retry */
		if (errno == EINTR)
			goto retry;

		/*
		 * If the pipe is full, we don't need to retry, the data that's there
		 * already is enough to wake up WaitLatch.
		 */
		if (errno == EAGAIN || errno == EWOULDBLOCK)
			return;

		/*
		 * Oops, the write() failed for some other reason. We might be in a
		 * signal handler, so it's not safe to elog(). We have no choice but
		 * silently ignore the error.
		 */
		return;
	}
}

Every detail here is load-bearing:

One byte, not a message. The byte’s value is irrelevant; only its presence matters.
The write end is non-blocking, and EAGAIN is treated as success. If the pipe’s kernel buffer is full, there is already a byte pending, and a pending byte is exactly as good as a new one: the reader will wake up anyway. Retrying would be pointless (the buffer is full!) and blocking would be fatal (we might be in a signal handler, where blocking is not allowed).
EINTR is retried, because a signal can interrupt the write.
Any other error is silently ignored, because we may be in a signal handler and elog() is not safe there.

The waiting flag is what keeps the handler cheap when nobody is waiting:

 c
 
 
// src/backend/storage/ipc/waiteventset.c
#ifndef WIN32
/* Are we currently in WaitLatch? The signal handler would like to know. */
static volatile sig_atomic_t waiting = false;
#endif

volatile sig_atomic_t is the only kind of variable a signal handler may safely touch.

And the drain:

 c
 
 
// src/backend/storage/ipc/waiteventset.c
/*
 * Read all available data from self-pipe or signalfd.
 * ...
 */
static void
drain(void)
{
	char		buf[1024];
	ssize_t		rc;
	int			fd;

#ifdef WAIT_USE_SELF_PIPE
	fd = selfpipe_readfd;
#else
	fd = signal_fd;
#endif

	for (;;)
	{
		rc = read(fd, buf, sizeof(buf));
		if (rc < 0)
		{
			if (errno == EAGAIN || errno == EWOULDBLOCK)
				break;			/* the descriptor is empty */
			else if (errno == EINTR)
				continue;		/* retry */
			else
			{
				waiting = false;
				...
				elog(ERROR, "read() on self-pipe failed: %m");
			}
		}
		else if (rc == 0)
		{
			...
		}
		else if (rc < sizeof(buf))
		{
			/* we successfully drained the pipe; no need to read() again */
			break;
		}
		/* else buffer wasn't big enough, so read again */
	}
}

The same function drains a signalfd or a self-pipe, which is why the code is shared between the two variants.

The whole picture

text

contended lock
 │
 ├─ spinlock ──── TAS_SPIN (free test + lock xchgb)
 │ └── SPIN_DELAY(): rep; nop / isb
 │ └── pg_usleep(cur_delay), cur_delay *= rand(1,2)
 │ └── after NUM_DELAYS: PANIC "stuck spinlock"
 │
 ├─ LWLock ────── CAS on the state word
 │ └── queue self on the wait list, try again
 │ └── PGSemaphoreLock (sem_wait) — kernel
 │
 └─ Latch ─────── maybe_sleeping test
 └── epoll_wait (signalfd) or poll (self-pipe) — kernel

What it all means in practice

Spinning burns CPU, and it is visible. If a backend is spinning, pg_stat_activity will show it with no wait event, and top will show it at 100% of a core. If it has fallen asleep in the spin delay, the wait event is SpinDelay. Both are symptoms of the same thing: somebody is holding a lock that a hot path needs.

The adaptive spin count is deliberately coarse, and that is fine. The estimate moves by +100 or −1 per acquisition locally, and by 1/16 of the difference per backend exit globally. It is not trying to be precise; it is trying to be stable. A backend that spins 100 times when 10 would do wastes 90 cache-coherence round trips; a backend that spins 10 times when 100 would do pays a pg_usleep() syscall. The asymmetry of the local adjustment (+100/−1) says that erring on the side of spinning is preferred, because spinning does not involve the kernel.

The global estimate only updates when a backend exits. It lives in ProcGlobal->spins_per_delay and is folded in at ProcKill() time. A long-lived backend with a short-lived one can therefore be running with a stale estimate for a long time. This is deliberate: the adjustment is cheap and the noise would be worse than the staleness.

PostgreSQL does not use futexes. Every LWLock wait that reaches the kernel costs a sem_wait syscall, and every wakeup costs a sem_post. On a machine with high LWLock contention, that syscall traffic shows up in strace counts and in perf samples of sem_wait. If you are chasing microseconds of lock overhead, this is where they go.

LWLocks have no deadlock detection. If two LWLocks could deadlock each other, PostgreSQL would not notice and would not report it. The heavyweight lock manager (deadlock.c) does the deadlock detection, and it only knows about heavyweight locks. The practical rule: never acquire an LWLock while holding another one in a way that could cycle. The elog(PANIC, "queueing for lock while waiting on another one") in LWLockQueueSelf() enforces the weaker property that you cannot wait for an LWLock while already waiting for one.

The wait event tells you which rung you are on. pg_stat_activity distinguishes LWLock (waiting for an LWLock), SpinDelay (sleeping inside a spinlock wait), and the latch/socket wait events. If a query is “stuck” and you see LWLock: BufferMapping for a long time, you have a buffer contention problem; if you see SpinDelay, you have a spinlock problem, which usually means something else is wrong.

SetLatch() is cheap on purpose. The maybe_sleeping test means that setting a latch nobody is waiting on costs one memory barrier and a predictable branch. This matters because latches are set on hot paths — every CHECK_FOR_INTERRUPTS(), every lock acquisition that might need to be interrupted — and a syscall there would be unacceptable.

The self-pipe trick exists because signals are unreliable as wakeups. A signal that arrives just before poll() is entered does not prevent poll() from sleeping. A byte in a pipe does. This is why the pipe write end is non-blocking and why EAGAIN is a success: the wakeup mechanism has to be safe to call from a signal handler, where blocking is forbidden and where throwing an error would be fatal.

The 64-bit atomic fallback is a spinlock. On a 32-bit platform without 64-bit CAS, pg_atomic_uint64 carries an embedded spinlock and every 64-bit atomic operation is a spinlock acquisition. That is “not unusably slow” per the header comment, but it does mean that 64-bit atomics are a scalability cliff on such platforms.

Two orders of magnitude separate “spin” from “sleep”. A spinlock acquisition that succeeds costs tens of nanoseconds; one that has to sleep costs at least a microsecond, usually much more. Most performance problems attributed to “lock contention” are really problems with the duration the lock is held, not with the waiting mechanism. Shortening the critical section is worth far more than tuning the spin count.

PostgreSQL's timestamp and timestamptz Implementation in C

Jay Hubbert — Tue, 06 Oct 2026 06:17:33 +0000

Here is the punchline up front: in PostgreSQL, timestamp and timestamptz are the same value. Both are an 8-byte signed integer counting microseconds since 2000-01-01. The type names suggest two different things — one naive, one zone-aware — but the storage layer cannot tell them apart. The entire distinction lives in the input and output functions, which consult the session TimeZone setting.

That single decision explains why timestamptz sorts with a plain int64 compare, why the binary wire format is identical for both types, and why comparing a timestamp to a timestamptz can flip from true to false just because someone ran SET TIME ZONE.

This is a walk through the C implementation, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entries

From src/include/catalog/pg_type.dat, timestamp (lines 299–304) and timestamptz (lines 305–311):

 text
 
 
{ oid => '1114', array_type_oid => '1115', descr => 'date and time',
 typname => 'timestamp', typlen => '8', typbyval => 't', typcategory => 'D',
 typinput => 'timestamp_in', typoutput => 'timestamp_out',
 typreceive => 'timestamp_recv', typsend => 'timestamp_send',
 typmodin => 'timestamptypmodin', typmodout => 'timestamptypmodout',
 typalign => 'd' },
{ oid => '1184', array_type_oid => '1185',
 descr => 'date and time with time zone',
 typname => 'timestamptz', typlen => '8', typbyval => 't', typcategory => 'D',
 typispreferred => 't', typinput => 'timestamptz_in',
 typoutput => 'timestamptz_out', typreceive => 'timestamptz_recv',
 typsend => 'timestamptz_send', typmodin => 'timestamptztypmodin',
 typmodout => 'timestamptztypmodout', typalign => 'd' },

The two rows are nearly identical:

Field	`timestamp`	`timestamptz`
oid	`1114`	`1184`
array oid	`1115`	`1185`
`typlen`	`8`	`8`
`typbyval`	`t`	`t`
`typalign`	`d`	`d`
`typispreferred`	(absent)	`t`

Same width, same pass-by-value, same 8-byte alignment, no typstorage (so it defaults to p, PLAIN — nothing to compress or TOAST). The one asymmetry is typispreferred: an untyped string literal such as 'now' resolves to timestamptz, not timestamp.

The C representation

Both types are one-line typedefs in src/include/datatype/timestamp.h:

 c
 
 
typedef int64 Timestamp;
typedef int64 TimestampTz;
typedef int64 TimeOffset;
typedef int32 fsec_t;			/* fractional seconds (in microseconds) */

src/include/utils/timestamp.h turns that into Datum conversions, with a comment that is the whole story:

 c
 
 
/*
 * For Timestamp, we make use of the same support routines as for int64.
 * Therefore Timestamp is pass-by-reference if and only if int64 is!
 */
static inline Timestamp
DatumGetTimestamp(Datum X)
{
	return (Timestamp) DatumGetInt64(X);
}

static inline Datum
TimestampGetDatum(Timestamp X)
{
	return Int64GetDatum(X);
}

On a 64-bit machine int64 is passed by value, so a timestamp is a Datum holding the integer itself. In a heap tuple it is stored inline like any other by-value fixed-length attribute — no pointer, no header, no alignment padding beyond typalign = 'd'.

text

+---------------------------------------+
| Timestamp / TimestampTz (int64) |
| microseconds since 2000-01-01 00:00 |
+---------------------------------------+
 0 7

Microseconds since 2000, via Julian day

The unit is microseconds; the origin is the Julian day constant POSTGRES_EPOCH_JDATE (src/include/datatype/timestamp.h):

 c
 
 
#define UNIX_EPOCH_JDATE		2440588 /* == date2j(1970, 1, 1) */
#define POSTGRES_EPOCH_JDATE	2451545 /* == date2j(2000, 1, 1) */
#define USECS_PER_DAY	INT64CONST(86400000000)
#define USECS_PER_HOUR	INT64CONST(3600000000)
#define USECS_PER_SEC	INT64CONST(1000000)

Conversion from a broken-down pg_tm is tm2timestamp() (src/backend/utils/adt/timestamp.c), which is pure integer arithmetic with overflow guards:

 c
 
 
	date = date2j(tm->tm_year, tm->tm_mon, tm->tm_mday) - POSTGRES_EPOCH_JDATE;
	time = time2t(tm->tm_hour, tm->tm_min, tm->tm_sec, fsec);

	if (unlikely(pg_mul_s64_overflow(date, USECS_PER_DAY, result) ||
				 pg_add_s64_overflow(*result, time, result)))
	{
		*result = 0;			/* keep compiler quiet */
		return -1;
	}
	if (tzp != NULL)
		*result = dt2local(*result, -(*tzp));

Note the tzp parameter. timestamp_in() passes NULL; timestamptz_in() passes the parsed zone offset. dt2local() is just dt -= timezone * USECS_PER_SEC. That is the entire runtime difference between the two types at input time: one applies an offset, the other does not.

The range is deliberately narrower than int64 allows:

 c
 
#define MIN_TIMESTAMP	INT64CONST(-211813488000000000)
#define END_TIMESTAMP	INT64CONST(9223371331200000000)

#define IS_VALID_TIMESTAMP(t) (MIN_TIMESTAMP <= (t) && (t) < END_TIMESTAMP)

That is 4714-11-24 BC through 294276-12-31 AD, leaving slop below the point where int64 microseconds would overflow.

The extremes of the range are reserved for infinity, exactly as with date:

 c
 
 
#define TIMESTAMP_MINUS_INFINITY	PG_INT64_MIN
#define TIMESTAMP_INFINITY	PG_INT64_MAX

#define TIMESTAMP_NOBEGIN(j)	\
	do {(j) = DT_NOBEGIN;} while (0)
#define TIMESTAMP_NOT_FINITE(j) (TIMESTAMP_IS_NOBEGIN(j) || TIMESTAMP_IS_NOEND(j))

So '-infinity'::timestamptz is literally INT64_MIN, and every arithmetic path checks TIMESTAMP_NOT_FINITE before touching the bits.

Output, and where the timezone comes back

timestamp_out() and timestamptz_out() both call timestamp2tm() and then EncodeDateTime(). The difference is again one argument — timestamp_out passes NULL for the zone pointer, timestamptz_out passes &tz:

 c
 
 
	else if (timestamp2tm(dt, &tz, tm, &fsec, &tzn, NULL) == 0)
		EncodeDateTime(tm, fsec, true, tz, tzn, DateStyle, buf);

timestamp2tm() splits the value into a day count and a within-day microsecond count, then rotates J2000 back to the Julian epoch and calls j2date()/dt2time():

 c
 
 
	time = dt;
	TMODULO(time, date, USECS_PER_DAY);

	if (time < INT64CONST(0))
	{
		time += USECS_PER_DAY;
		date -= 1;
	}

	/* add offset to go from J2000 back to standard Julian date */
	date += POSTGRES_EPOCH_JDATE;
	...
	j2date((int) date, &tm->tm_year, &tm->tm_mon, &tm->tm_mday);
	dt2time(time, &tm->tm_hour, &tm->tm_min, &tm->tm_sec, fsec);

Only when tzp != NULL does it call pg_localtime() to rotate the instant into the session zone, filling tm_gmtoff and setting *tzp = -tm->tm_gmtoff. For dates outside pg_time_t’s range it falls back to treating the value as GMT. A naive timestamp never enters that branch.

Casting, and cross-type comparison

The cast functions make the reinterpretation explicit. timestamp2timestamptz_safe() reads the naive value, looks up the offset for that local time in the session zone, and shifts it:

 c
 
 
		tz = DetermineTimeZoneOffset(tm, session_timezone);
		result = dt2local(timestamp, -tz);

timestamptz2timestamp_safe() goes the other way through timestamp2tm/tm2timestamp. Neither changes the instant; they change which wall-clock instant the integer is said to denote.

Cross-type comparison inherits the same rule (timestamp_cmp_timestamptz_internal() in timestamp.c): it calls timestamp2timestamptz_safe() and then compares as int64. So timestamp '2024-01-01 12:00' equals timestamptz '2024-01-01 12:00+00' only if your session zone is UTC.

Typmod rounds, it does not truncate

timestamp(3) keeps three fractional-second digits. The typmod is just the digit count (0–6, capped by MAX_TIMESTAMP_PRECISION), and AdjustTimestampForTypmod() rounds half-up using a scale and offset table:

 c
 
 
	static const int64 TimestampScales[MAX_TIMESTAMP_PRECISION + 1] = {
		INT64CONST(1000000), INT64CONST(100000), INT64CONST(10000),
		INT64CONST(1000), INT64CONST(100), INT64CONST(10), INT64CONST(1)
	};
	static const int64 TimestampOffsets[MAX_TIMESTAMP_PRECISION + 1] = {
		INT64CONST(500000), INT64CONST(50000), INT64CONST(5000),
		INT64CONST(500), INT64CONST(50), INT64CONST(5), INT64CONST(0)
	};

The offset is half a unit of the target precision, so .1235 rounds up and .1234 does not. The negative branch negates, rounds, and re-negates, so rounding is symmetric around the epoch.

Wire format, comparison, hashing

The binary representation is the bare integer, identical for both types:

 c
 
 
Datum
timestamp_send(PG_FUNCTION_ARGS)
{
	Timestamp	timestamp = PG_GETARG_TIMESTAMP(0);
	StringInfoData buf;

	pq_begintypsend(&buf);
	pq_sendint64(&buf, timestamp);
	PG_RETURN_BYTEA_P(pq_endtypsend(&buf));
}

timestamp_recv() reads it with pq_getmsgint64(), then range-checks by calling timestamp2tm() and IS_VALID_TIMESTAMP(). Comparison is a three-way int64 compare (timestamp_cmp_internal()), sort support is wired straight to ssup_datum_int64_cmp, and hashing is delegated to hashint8 — the same routine int8 uses:

 c
 
 
Datum
timestamptz_hash(PG_FUNCTION_ARGS)
{
	return hashint8(fcinfo);
}

There is no normalization, no calendar math, and no timezone lookup anywhere in the ordering hot path. That work is paid once, at input.

Look at it yourself

The catalog rows confirm the shared layout, and the epoch arithmetic is directly observable:

 sql
 
 
SELECT typname, typlen, typbyval, typalign, typarray, typispreferred
FROM pg_type
WHERE typname IN ('timestamp', 'timestamptz');

-- the internal origin is 2000-01-01, not 1970-01-01
SELECT '2000-01-01 00:00:00'::timestamp - '1970-01-01 00:00:00'::timestamp; -- 946684800
SELECT 'epoch'::timestamptz; -- 1970-01-01 00:00:00+00

SELECT pg_column_size('2024-01-01 00:00:00'::timestamp), -- 8
 pg_column_size('2024-01-01 00:00:00+00'::timestamptz); -- 8

And the timezone dependency of cross-type equality is easy to reproduce:

 sql
 
 
SET TIME ZONE 'UTC';
SELECT '2024-01-01 12:00:00'::timestamp = '2024-01-01 12:00:00+00'::timestamptz; -- true
SET TIME ZONE 'America/New_York';
SELECT '2024-01-01 12:00:00'::timestamp = '2024-01-01 12:00:00+00'::timestamptz; -- false

Same two literals, same two integers on disk, different answer — because the comparison converted the naive value using the current session zone.

What it all means

timestamp and timestamptz are both int64 microseconds since 2000-01-01; typlen = 8, typbyval = t, typalign = d, PLAIN storage.
The only difference is I/O: timestamptz_in/out consult the session TimeZone via DetermineTimeZoneOffset/pg_localtime, timestamp does not.
Casting between them is a reinterpretation of the same instant, not a conversion; cross-type comparison does the same thing, so it is session-zone-dependent.
Sorting and hashing are plain int64 operations (hashint8, ssup_datum_int64_cmp) — calendar and zone logic live only at the edges.
Infinity is INT64_MIN/INT64_MAX, and every arithmetic path short-circuits on TIMESTAMP_NOT_FINITE.
Typmod is a 0–6 digit precision that rounds half-up, symmetrically for negative values.

PostgreSQL's bool Implementation in C

Jay Hubbert — Sun, 04 Oct 2026 04:06:31 +0000

bool looks like the easiest type in PostgreSQL. One bit of meaning, one byte of storage, done. It is not. The value has two different text spellings depending on which code path you go through, the input parser accepts eight different words plus unique prefixes, the binary wire format is not the text format, and the aggregate machinery behind bool_and/bool_or is a small state machine with its own struct. All of it lives in src/backend/utils/adt/bool.c, a little over 400 lines.

This is based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entry

The type’s identity is one line in src/include/catalog/pg_type.dat:

 text
 
 
{ oid => '16', array_type_oid => '1000',
 descr => 'boolean, format \'t\'/\'f\'',
 typname => 'bool', typlen => '1', typbyval => 't', typcategory => 'B',
 typispreferred => 't', typinput => 'boolin', typoutput => 'boolout',
 typreceive => 'boolrecv', typsend => 'boolsend', typalign => 'c' },

Five of those fields matter here:

Field	Value	Meaning
`typlen`	1	one byte on disk
`typbyval`	`t`	passed by value, not by reference
`typalign`	`c`	`TYPALIGN_CHAR`: no alignment padding
`typcategory`	`B`	the Boolean category (sorts last in the built-in order)
`array_type_oid`	1000	`_bool` is generated automatically from this entry

typalign => 'c' is TYPALIGN_CHAR, documented in src/include/catalog/pg_type.h as “char alignment (ie no alignment needed)”. So a bool column never forces a padding byte before it. typstorage is not listed, so it takes the default 'p' (TYPSTORAGE_PLAIN): a bool is never TOASTed.

The C representation

In memory, a bool is a C99 bool (from <stdbool.h>, pulled in by src/include/c.h), which is one byte holding 0 or 1. But the function-call convention never hands you a bare bool. Every value crosses a function boundary as a Datum — a pointer-sized integer — and the conversion is a pair of trivial inline functions in src/include/postgres.h:

 c
 
 
static inline bool
DatumGetBool(Datum X)
{
	return (X != 0);
}

static inline Datum
BoolGetDatum(bool X)
{
	return (Datum) (X ? 1 : 0);
}

Notice the asymmetry in the doc comments: “any nonzero value will be considered true.” The representation is 8 bytes in memory but 1 byte on disk. PG_GETARG_BOOL(n) and PG_RETURN_BOOL(x) in src/include/fmgr.h are just these two functions.

Two text spellings

boolout is the type’s official output function, and it emits a single character:

 c
 
 
Datum
boolout(PG_FUNCTION_ARGS)
{
	bool		b = PG_GETARG_BOOL(0);
	char	 *result = (char *) palloc(2);

	result[0] = (b) ? 't' : 'f';
	result[1] = '\0';
	PG_RETURN_CSTRING(result);
}

That is why SELECT true; prints t and SELECT false; prints f — the type’s own output function is what a bare SELECT invokes. But the cast to text uses a different function, booltext, which follows the SQL standard instead:

 c
 
 
Datum
booltext(PG_FUNCTION_ARGS)
{
	bool		arg1 = PG_GETARG_BOOL(0);
	const char *str;

	if (arg1)
		str = "true";
	else
		str = "false";

	PG_RETURN_TEXT_P(cstring_to_text(str));
}

src/include/catalog/pg_cast.dat wires that function up for all three string targets:

 text
 
 
{ castsource => 'bool', casttarget => 'text', castfunc => 'text(bool)', ... }
{ castsource => 'bool', casttarget => 'varchar', castfunc => 'text(bool)', ... }
{ castsource => 'bool', casttarget => 'bpchar', castfunc => 'text(bool)', ... }

So true::text is true, and so is true::varchar and true::char(n). One value, two spellings, chosen by whether you go through boolout or through a cast to a string type.

The permissive input parser

boolin trims leading and trailing whitespace, then calls parse_bool_with_len. That function is a switch on the first character (src/backend/utils/adt/bool.c):

 c
 
 
bool
parse_bool_with_len(const char *value, size_t len, bool *result)
{
	/* Check the most-used possibilities first. */
	switch (*value)
	{
		case 't':
		case 'T':
			if (pg_strncasecmp(value, "true", len) == 0)
			{
				if (result)
					*result = true;
				return true;
			}
			break;
		...
		case 'o':
		case 'O':
			/* 'o' is not unique enough */
			if (pg_strncasecmp(value, "on", (len > 2 ? len : 2)) == 0)
			...

The comment above the function claims it accepts “unique prefixes thereof” — and the implementation really does, because pg_strncasecmp only compares len characters. 't', 'tr', 'tru', and 'true' all parse as true; the same for f/fa/fal/fals/false, y/ye/yes, and n/no. The one exception is o, which the code calls out as “not unique enough”: it clamps the comparison length to at least 2, so 'o' alone is rejected while 'on' and 'of' are accepted.

The full set of accepted inputs is true, false, yes, no, on, off, 1, 0, each case-insensitive and optionally prefixed (except o, which needs two characters).

Binary format

boolrecv and boolsend handle the binary protocol (COPY ... WITH (FORMAT binary)). They do not reuse the text representation:

 c
 
 
Datum
boolrecv(PG_FUNCTION_ARGS)
{
	StringInfo	buf = (StringInfo) PG_GETARG_POINTER(0);
	int			ext;

	ext = pq_getmsgbyte(buf);
	PG_RETURN_BOOL(ext != 0);
}

Datum
boolsend(PG_FUNCTION_ARGS)
{
	bool		arg1 = PG_GETARG_BOOL(0);
	StringInfoData buf;

	pq_begintypsend(&buf);
	pq_sendbyte(&buf, arg1 ? 1 : 0);
	PG_RETURN_BYTEA_P(pq_endtypsend(&buf));
}

One byte on the wire, 0 or 1 on the way out, and any nonzero byte counts as true on the way in. That is the same “any nonzero is true” rule as DatumGetBool, which is what lets a boolean survive a round trip even if a client sends 0x02.

Hashing, comparison, and aggregates

Equality is trivial (arg1 == arg2), and hashing widens the byte to an int32 and reuses the integer hash:

 c
 
 
Datum
hashbool(PG_FUNCTION_ARGS)
{
	return hash_uint32((int32) PG_GETARG_BOOL(0));
}

The btree comparison is a plain subtraction (src/backend/access/nbtree/nbtcompare.c):

 c
 
 
Datum
btboolcmp(PG_FUNCTION_ARGS)
{
	bool		a = PG_GETARG_BOOL(0);
	bool		b = PG_GETARG_BOOL(1);

	PG_RETURN_INT32((int32) a - (int32) b);
}

For int4 and wider, btint4cmp cannot subtract (it could overflow) and switches to > / <. For one-byte bools the subtraction is safe.

bool_and and bool_or are where the interesting struct appears. They are aggregates, so they need per-group state. BoolAggState keeps two counters:

 c
 
 
typedef struct BoolAggState
{
	int64		aggcount;		/* number of non-null values aggregated */
	int64		aggtrue;		/* number of values aggregated that are true */
} BoolAggState;

The transition function bool_accum increments both counters, skipping NULLs, and bool_alltrue / bool_anytrue answer from the counters at the end. That is why bool_and of an all-NULL group returns NULL rather than an arbitrary default. The whole thing is one more example of the general PostgreSQL aggregate pattern: a state struct, a transition function, an inverse (bool_accum_inv, for moving-aggregate mode), and a final function.

Look at it yourself

 sql
 
 
SELECT 't'::bool, 'tru'::bool, 'o'::bool; -- t, t, ERROR
SELECT true::text, true::varchar; -- true, true
SELECT true; -- t
SELECT bool_and(x) FROM (VALUES (true),(false)) v(x); -- false
SELECT bool_and(x) FROM (VALUES (NULL),(NULL)) v(x); -- NULL

The parser’s prefix behavior is easy to confirm:

 sql
 
 
SELECT 'ye'::bool, 'n'::bool, 'off'::bool, 'of'::bool;
-- t, f, f, f

And the catalog entry is visible directly:

 sql
 
 
SELECT typname, typlen, typbyval, typalign, typcategory
FROM pg_type WHERE oid = 16;
-- bool, 1, t, c, B

What it all means

A bool is one byte of storage and eight bytes of in-memory Datum, with typbyval meaning no pointer chasing.
typalign => 'c' means it never forces padding, so a bool column can sit at any byte offset in a tuple.
boolout says t/f; booltext says true/false. Which one you see depends on the cast.
The input parser accepts case-insensitive prefixes of eight words (true/false/yes/no/on/off/1/0), with o as the only minimum-two-character special case.
The binary protocol uses one byte and treats any nonzero value as true.

PostgreSQL Locks Buffers Without LWLocks

Jay Hubbert — Fri, 02 Oct 2026 09:00:00 +0000

Every time PostgreSQL reads a page it does two things: it pins the shared buffer that holds the page, and it takes a content lock on that buffer so that nobody rewrites the bytes underneath it while it is being read.

The second of those used to be an LWLock — a general-purpose, one-word-at-a-time lightweight lock living in a global array in shared memory. PostgreSQL 20 changed that. Buffer content locks are now implemented entirely inside a single 64-bit atomic word that already existed in the buffer descriptor, with its own three-level lock hierarchy, its own wait queue, and its own wakeup logic.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

What a content lock protects

Let’s be precise about what is being locked, because PostgreSQL has two different “buffer locks” and they are easy to confuse.

The buffer header lock protects the descriptor — the metadata in BufferDesc: the tag, the validity flags, the I/O state. It is a very short critical section, taken for a handful of instructions at a time.

The buffer content lock protects the page data — the 8 kB of bytes in the buffer. It is held for as long as the caller is looking at the page. This is the one that matters for correctness: two backends cannot be reading the same page while one of them is rewriting it.

The header comment in src/include/storage/buf_internals.h is explicit about the division:

The content of buffers is protected via the buffer content lock, implemented as part of the buffer state. Note that the buffer header lock is not used to control access to the data in the buffer! We used to use an LWLock to implement the content lock, but having a dedicated implementation of content locks allows us to implement some otherwise hard things (e.g. race-freely checking if AIO is in progress before locking a buffer exclusively) and enables otherwise impossible optimizations (e.g. unlocking and unpinning a buffer in one atomic operation).

That comment is the entire design rationale in one paragraph, and it is worth reading twice. The dedicated implementation exists to make two specific things possible: racing safely against asynchronous I/O, and fusing lock release with unpinning.

The old world: an LWLock per buffer

Before the change, BufferDesc had a lock in it:

 c
 
 
	/*
	 * Backend of pin-count waiter. The buffer header spinlock needs to be
	 * held to modify this field.
	 */
	int			wait_backend_pgprocno;

	PgAioWaitRef io_wref;		/* set iff AIO is in progress */
	LWLock		content_lock;	/* to lock access to buffer contents */
} BufferDesc;

and the accessor was a one-liner:

 c
 
 
static inline LWLock *
BufferDescriptorGetContentLock(const BufferDesc *bdesc)
{
	return (LWLock *) (&bdesc->content_lock);
}

An LWLock is a 16-byte structure with a uint32 state, a pg_atomic_uint32 nwaiters, a proclist_head waiters, and a bool is_shared. Taking one is a CAS on the state word; if that fails, you get pushed onto the wait queue in the same structure. Releasing it is another CAS plus, possibly, a walk of the wait queue.

The problem is not the algorithm — the LWLock algorithm is good. The problem is that there are now two independent atomic words per buffer: the LWLock’s state, and the BufferDesc.state word that holds the refcount and the flags. Anything that needs to reason about both at once has to do two separate atomic operations, with a window in between.

Concretely, the old code path for a shared content lock was:

 c
 
	LWLockAcquire(BufferDescriptorGetContentLock(buf), LW_SHARED);

and for an exclusive one:

 c
 
	LWLockAcquire(BufferDescriptorGetContentLock(desc), LW_EXCLUSIVE);

That is a full generic lock acquisition: read the lock’s state, decide whether it conflicts, CAS, and — if it conflicts — walk into lwlock.c’s queueing machinery.

The new world: one 64-bit word

The change is commit fcb9c977aa5, “bufmgr: Implement buffer content locks independently of lwlocks”, by Andres Freund, January 2026. The commit message lays out the four reasons:

Hint bits are currently set while holding only a share lock. This leads to having to copy pages while they are being written out if checksums are enabled, which is not cheap. We would like to add AIO writes, however once many buffers can be written out at the same time, it gets a lot more expensive to copy them, particularly because that copy needs to reside in shared buffers (for worker mode to have access to the buffer).

In addition, modifying buffers while they are being written out can cause issues with unbuffered/direct-IO, as some filesystems (like btrfs) do not like that, due to filesystem internal checksums getting corrupted.

The solution to this is to require a new share-exclusive lock-level to set hint bits and to write out buffers, making those operations mutually exclusive. We could introduce such a lock-level into the generic lwlock implementation, however it does not look like there would be other users, and it does add some overhead into important code paths.

For AIO writes we need to be able to race-freely check whether a buffer is undergoing IO and whether an exclusive lock on the page can be acquired. That is rather hard to do efficiently when the buffer state and the lock state are separate atomic variables. This is a major hindrance to allowing writes to be done asynchronously.

Buffer locks are by far the most frequently taken locks. Optimizing them specifically for their use case is worth the effort. E.g. by merging content locks into buffer locks we will be able to release a buffer lock and pin in one atomic operation.

There are more complicated optimizations, like long-lived “super pinned & locked” pages, that cannot realistically be implemented with the generic lwlock implementation.

Point 3 is the headline: buffer content locks are, by a wide margin, the most frequently taken lock in the system. Every ReadBuffer() pins and locks; every index scan does it once per page; every page inspection does it again.

The layout

The state word was already 64 bits wide before this change, but only half of it was used. The old header comment read:

 c
 
 
/*
 * Buffer state is a single 64-bit variable where following data is combined.
 *
 * State of the buffer itself (in order):
 * - 18 bits refcount
 * - 4 bits usage count
 * - 10 bits of flags
 *
 * Combining these values allows to perform some operations without locking
 * the buffer header, by modifying them together with a CAS loop.
 *
 * NB: A future commit will use a significant portion of the remaining bits to
 * implement buffer locking as part of the state variable.
 *
 * The definition of buffer state components is below.
 */
#define BUF_REFCOUNT_BITS 18
#define BUF_USAGECOUNT_BITS 4
#define BUF_FLAG_BITS 10

StaticAssertDecl(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS + BUF_FLAG_BITS == 32,
				 "parts of buffer state space need to equal 32");

The new one (src/include/storage/buf_internals.h):

 c
 
 
/*
 * Buffer state is a single 64-bit variable where following data is combined.
 *
 * State of the buffer itself (in order):
 * - 18 bits refcount
 * - 4 bits usage count
 * - 12 bits of flags
 * - 18 bits share-lock count
 * - 1 bit share-exclusive locked
 * - 1 bit exclusive locked
 *
 * Combining these values allows to perform some operations without locking
 * the buffer header, by modifying them together with a CAS loop.
 *
 * The definition of buffer state components is below.
 */
#define BUF_REFCOUNT_BITS 18
#define BUF_USAGECOUNT_BITS 4
#define BUF_FLAG_BITS 12
#define BUF_LOCK_BITS (18+2)

StaticAssertDecl(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS + BUF_FLAG_BITS + BUF_LOCK_BITS <= 64,
				 "parts of buffer state space need to be <= 64");

So the word went from 32 bits used to 54 bits used, and the invariant changed from “exactly 32” to “at most 64”. The two extra flag bits are the two new lock-related flags; the 20 new lock bits are the lock state.

Here is the whole layout, on a 64-bit machine, with MAX_BACKENDS_BITS == 18:

Bits	Width	Field	Meaning
0–17	18	refcount	number of pins on this buffer
18–21	4	usage count	clock-sweep second-chance counter
22–33	12	flags	`BM_*` bits, see below
34–51	18	share-lock count	number of backends holding a share lock
52	1	share-exclusive	one backend holds a share-exclusive lock
53	1	exclusive	one backend holds an exclusive lock
54–63	10	unused

The shift constants:

 c
 
 
/* refcount related definitions */
#define BUF_REFCOUNT_ONE 1
#define BUF_REFCOUNT_MASK \
	((UINT64CONST(1) << BUF_REFCOUNT_BITS) - 1)

/* usage count related definitions */
#define BUF_USAGECOUNT_SHIFT \
	BUF_REFCOUNT_BITS
#define BUF_USAGECOUNT_MASK \
	(((UINT64CONST(1) << BUF_USAGECOUNT_BITS) - 1) << (BUF_USAGECOUNT_SHIFT))
#define BUF_USAGECOUNT_ONE \
	(UINT64CONST(1) << BUF_REFCOUNT_BITS)

/* flags related definitions */
#define BUF_FLAG_SHIFT \
	(BUF_REFCOUNT_BITS + BUF_USAGECOUNT_BITS)
#define BUF_FLAG_MASK \
	(((UINT64CONST(1) << BUF_FLAG_BITS) - 1) << BUF_FLAG_SHIFT)

and the lock bits:

 c
 
 
/* lock state related definitions */
#define BM_LOCK_SHIFT \
	(BUF_FLAG_SHIFT + BUF_FLAG_BITS)
#define BM_LOCK_VAL_SHARED \
	(UINT64CONST(1) << (BM_LOCK_SHIFT))
#define BM_LOCK_VAL_SHARE_EXCLUSIVE \
	(UINT64CONST(1) << (BM_LOCK_SHIFT + MAX_BACKENDS_BITS))
#define BM_LOCK_VAL_EXCLUSIVE \
	(UINT64CONST(1) << (BM_LOCK_SHIFT + MAX_BACKENDS_BITS + 1))
#define BM_LOCK_MASK \
	((((uint64) MAX_BACKENDS) << BM_LOCK_SHIFT) | BM_LOCK_VAL_SHARE_EXCLUSIVE | BM_LOCK_VAL_EXCLUSIVE)

MAX_BACKENDS_BITS comes from src/include/storage/procnumber.h:

 c
 
 
#define MAX_BACKENDS_BITS		18
#define MAX_BACKENDS			((1U << MAX_BACKENDS_BITS)-1)

Eighteen bits, which is exactly the number of lock bits available for the share-lock count. That is not a coincidence: it means the share-lock count can never exceed the number of possible backends, and the BM_LOCK_MASK expression (((uint64) MAX_BACKENDS) << BM_LOCK_SHIFT) covers exactly bits 34–51 — the whole share-lock field.

Two static assertions tie this together:

 c
 
 
StaticAssertDecl(MAX_BACKENDS_BITS <= BUF_REFCOUNT_BITS,
				 "MAX_BACKENDS_BITS needs to be <= BUF_REFCOUNT_BITS");
StaticAssertDecl(MAX_BACKENDS_BITS <= (BUF_LOCK_BITS - 2),
				 "MAX_BACKENDS_BITS needs to be <= BUF_LOCK_BITS - 2");

The subtle part is that the “share lock” field is not a single bit — it is a count, stored in the low bits of the lock field. BM_LOCK_VAL_SHARED is the value 1 << 34, and acquiring a share lock adds it to the state word:

 c
 
 
		else
		{
			lock_free = (old_state & BM_LOCK_VAL_EXCLUSIVE) == 0;
			if (lock_free)
				desired_state += BM_LOCK_VAL_SHARED;
		}

So desired_state += BM_LOCK_VAL_SHARED is really desired_state += 1 in the count field. The BM_LOCK_MASK then detects “is anybody holding any share lock” by testing all 18 count bits at once.

This is why the exclusive acquisition test is a mask against the whole lock field rather than a single bit:

 c
 
 
		if (mode == BUFFER_LOCK_EXCLUSIVE)
		{
			lock_free = (old_state & BM_LOCK_MASK) == 0;
			if (lock_free)
				desired_state += BM_LOCK_VAL_EXCLUSIVE;
		}

BM_LOCK_MASK covers the share-lock count and both exclusive bits, so “the lock is free” means “no share lockers, no share-exclusive holder, no exclusive holder”.

The flag bits

The flags live in the middle of the word, and two of them are new:

 c
 
 
/* buffer header is locked */
#define BM_LOCKED					BUF_DEFINE_FLAG( 0)
/* data needs writing */
#define BM_DIRTY					BUF_DEFINE_FLAG( 1)
/* data is valid */
#define BM_VALID					BUF_DEFINE_FLAG( 2)
/* tag is assigned */
#define BM_TAG_VALID				BUF_DEFINE_FLAG( 3)
/* read or write in progress */
#define BM_IO_IN_PROGRESS			BUF_DEFINE_FLAG( 4)
/* previous I/O failed */
#define BM_IO_ERROR					BUF_DEFINE_FLAG( 5)
/* flag bit 6 is not used anymore */
/* have waiter for sole pin */
#define BM_PIN_COUNT_WAITER			BUF_DEFINE_FLAG( 7)
/* must write for checkpoint */
#define BM_CHECKPOINT_NEEDED		BUF_DEFINE_FLAG( 8)
/* permanent buffer (not unlogged, or init fork) */
#define BM_PERMANENT				BUF_DEFINE_FLAG( 9)
/* content lock has waiters */
#define BM_LOCK_HAS_WAITERS			BUF_DEFINE_FLAG(10)
/* waiter for content lock has been signalled but not yet run */
#define BM_LOCK_WAKE_IN_PROGRESS	BUF_DEFINE_FLAG(11)

BM_LOCK_HAS_WAITERS and BM_LOCK_WAKE_IN_PROGRESS are the content-lock equivalents of LW_FLAG_HAS_WAITERS and LW_FLAG_WAKE_IN_PROGRESS. They are in the flag field rather than the lock field because they are advisory hints, not part of the lock’s conflict state — they must not affect whether a lock can be acquired.

The descriptor

The BufferDesc itself changed shape. The embedded LWLock content_lock is gone; in its place is a wait list:

 c
 
 
typedef struct BufferDesc
{
	/*
	 * ID of page contained in buffer. The buffer header spinlock needs to be
	 * held to modify this field.
	 */
	BufferTag	tag;

	/*
	 * Buffer's index number (from 0). The field never changes after
	 * initialization, so does not need locking.
	 */
	int			buf_id;

	/*
	 * State of the buffer, containing flags, refcount and usagecount. See
	 * BUF_* and BM_* defines at the top of this file.
	 */
	pg_atomic_uint64 state;

	/*
	 * Backend of pin-count waiter. The buffer header spinlock needs to be
	 * held to modify this field.
	 */
	int			wait_backend_pgprocno;

	PgAioWaitRef io_wref;		/* set iff AIO is in progress */

	/*
	 * List of PGPROCs waiting for the buffer content lock. Protected by the
	 * buffer header spinlock.
	 */
	proclist_head lock_waiters;
} BufferDesc;

Note the proclist_head lock_waiters — a per-buffer intrusive wait list, analogous to the one inside an LWLock, but only 8 bytes instead of a whole LWLock’s worth of state.

Acquiring a content lock

The heart of it is BufferLockAttempt() in src/backend/storage/buffer/bufmgr.c. This is the analogue of LWLockAttemptLock():

 c
 
 
static inline bool
BufferLockAttempt(BufferDesc *buf_hdr, BufferLockMode mode)
{
	uint64		old_state;

	/*
	 * Read once outside the loop, later iterations will get the newer value
	 * via compare & exchange.
	 */
	old_state = pg_atomic_read_u64(&buf_hdr->state);

	/* loop until we've determined whether we could acquire the lock or not */
	while (true)
	{
		uint64		desired_state;
		bool		lock_free;

		desired_state = old_state;

		if (mode == BUFFER_LOCK_EXCLUSIVE)
		{
			lock_free = (old_state & BM_LOCK_MASK) == 0;
			if (lock_free)
				desired_state += BM_LOCK_VAL_EXCLUSIVE;
		}
		else if (mode == BUFFER_LOCK_SHARE_EXCLUSIVE)
		{
			lock_free = (old_state & (BM_LOCK_VAL_EXCLUSIVE | BM_LOCK_VAL_SHARE_EXCLUSIVE)) == 0;
			if (lock_free)
				desired_state += BM_LOCK_VAL_SHARE_EXCLUSIVE;
		}
		else
		{
			lock_free = (old_state & BM_LOCK_VAL_EXCLUSIVE) == 0;
			if (lock_free)
				desired_state += BM_LOCK_VAL_SHARED;
		}

		/*
		 * Attempt to swap in the state we are expecting. If we didn't see
		 * lock to be free, that's just the old value. If we saw it as free,
		 * we'll attempt to mark it acquired. The reason that we always swap
		 * in the value is that this doubles as a memory barrier. We could try
		 * to be smarter and only swap in values if we saw the lock as free,
		 * but benchmark haven't shown it as beneficial so far.
		 *
		 * Retry if the value changed since we last looked at it.
		 */
		if (likely(pg_atomic_compare_exchange_u64(&buf_hdr->state,
												 &old_state, desired_state)))
		{
			if (lock_free)
			{
				/* Great! Got the lock. */
				return false;
			}
			else
				return true;	/* somebody else has the lock */
		}
	}

	pg_unreachable();
}

Read the three conflict rules off that:

Mode	Acquired if	Adds
`BUFFER_LOCK_SHARE`	no exclusive holder	`+1` to the share count
`BUFFER_LOCK_SHARE_EXCLUSIVE`	no exclusive and no share-exclusive holder	sets bit 52
`BUFFER_LOCK_EXCLUSIVE`	nothing held at all	sets bit 53

That is exactly the semantics the commit message describes: “An additional lock-level share-exclusive has been added. This lock-level conflicts with exclusive locks and itself, but not share locks.”

Two things are worth noting about the implementation.

First, the CAS always happens, even when the lock is known to be held. The comment explains why: the CAS doubles as a memory barrier, and the benchmark showed that conditionally skipping it did not help.

Second, the conflict test is a read of a shared word, not a read of a private lock structure. Because the lock state lives in the same word as the refcount and the flags, a concurrent pin or unpin is visible to the lock decision for free.

The queue-self-then-retry algorithm

If the first attempt fails, we cannot simply go to sleep: the lock might have been released in the meantime, and we would sleep on a lock nobody is going to signal. The classic solution — and the one lwlock.c uses — is to enqueue yourself and then try again. BufferLockAcquire() does exactly that:

 c
 
 
static inline void
BufferLockAcquire(Buffer buffer, BufferDesc *buf_hdr, BufferLockMode mode)
{
	PrivateRefCountEntry *entry;
	int			extraWaits = 0;

	/*
	 * Get reference to the refcount entry before we hold the lock, it seems
	 * better to do before holding the lock.
	 */
	entry = GetPrivateRefCountEntry(buffer, true);

	/*
	 * We better not already hold a lock on the buffer.
	 */
	Assert(entry->data.lockmode == BUFFER_LOCK_UNLOCK);

	/*
	 * Lock out cancel/die interrupts until we exit the code section protected
	 * by the content lock. This ensures that interrupts will not interfere
	 * with manipulations of data structures in shared memory.
	 */
	HOLD_INTERRUPTS();

	for (;;)
	{
		uint32		wait_event = 0; /* initialized to avoid compiler warning */
		bool		mustwait;

		/*
		 * Try to grab the lock the first time, we're not in the waitqueue
		 * yet/anymore.
		 */
		mustwait = BufferLockAttempt(buf_hdr, mode);

		if (likely(!mustwait))
		{
			break;
		}

		/*
		 * Ok, at this point we couldn't grab the lock on the first try. We
		 * cannot simply queue ourselves to the end of the list and wait to be
		 * woken up because by now the lock could long have been released.
		 * Instead add us to the queue and try to grab the lock again. If we
		 * succeed we need to revert the queuing and be happy, otherwise we
		 * recheck the lock. If we still couldn't grab it, we know that the
		 * other locker will see our queue entries when releasing since they
		 * existed before we checked for the lock.
		 */

		/* add to the queue */
		BufferLockQueueSelf(buf_hdr, mode);

		/* we're now guaranteed to be woken up if necessary */
		mustwait = BufferLockAttempt(buf_hdr, mode);

		/* ok, grabbed the lock the second time round, need to undo queueing */
		if (!mustwait)
		{
			BufferLockDequeueSelf(buf_hdr);
			break;
		}

		switch (mode)
		{
			case BUFFER_LOCK_EXCLUSIVE:
				wait_event = WAIT_EVENT_BUFFER_EXCLUSIVE;
				break;
			case BUFFER_LOCK_SHARE_EXCLUSIVE:
				wait_event = WAIT_EVENT_BUFFER_SHARE_EXCLUSIVE;
				break;
			case BUFFER_LOCK_SHARE:
				wait_event = WAIT_EVENT_BUFFER_SHARED;
				break;
			case BUFFER_LOCK_UNLOCK:
				pg_unreachable();

		}
		pgstat_report_wait_start(wait_event);

		/*
		 * Wait until awakened.
		 *
		 * It is possible that we get awakened for a reason other than being
		 * signaled by BufferLockWakeup(). If so, loop back and wait again.
		 * Once we've gotten the lock, re-increment the sema by the number of
		 * additional signals received.
		 */
		for (;;)
		{
			PGSemaphoreLock(MyProc->sem);
			if (MyProc->lwWaiting == LW_WS_NOT_WAITING)
				break;
			extraWaits++;
		}

		pgstat_report_wait_end();

		/* Retrying, allow BufferLockReleaseSub to release waiters again. */
		pg_atomic_fetch_and_u64(&buf_hdr->state, ~BM_LOCK_WAKE_IN_PROGRESS);
	}

	/* Remember that we now hold this lock */
	entry->data.lockmode = mode;

	/*
	 * Fix the process wait semaphore's count for any absorbed wakeups.
	 */
	while (unlikely(extraWaits-- > 0))
		PGSemaphoreUnlock(MyProc->sem);
}

Compare this to LWLockAcquire() in src/backend/storage/lmgr/lwlock.c — the structure is identical, down to the comments:

 c
 
 
		/*
		 * Try to grab the lock the first time, we're not in the waitqueue
		 * yet/anymore.
		 */
		mustwait = LWLockAttemptLock(lock, mode);

		if (!mustwait)
		{
			LOG_LWDEBUG("LWLockAcquire", lock, "immediately acquired lock");
			break;				/* got the lock */
		}

		/*
		 * Ok, at this point we couldn't grab the lock on the first try. We
		 * cannot simply queue ourselves to the end of the list and wait to be
		 * woken up because by now the lock could long have been released.
		 * Instead add us to the queue and try to grab the lock again. If we
		 * succeed we need to revert the queuing and be happy, otherwise we
		 * recheck the lock. If we still couldn't grab it, we know that the
		 * other locker will see our queue entries when releasing since they
		 * existed before we checked for the lock.
		 */

		/* add to the queue */
		LWLockQueueSelf(lock, mode);

		/* we're now guaranteed to be woken up if necessary */
		mustwait = LWLockAttemptLock(lock, mode);

		/* ok, grabbed the lock the second time round, need to undo queueing */
		if (!mustwait)
		{
			LOG_LWDEBUG("LWLockAcquire", lock, "acquired, undoing queue");

			LWLockDequeueSelf(lock);
			break;
		}

The commit message is frank about this: “This commit unfortunately introduces some code that is very similar to the code in lwlock.c, however the code is not equivalent enough to easily merge it.”

One wart the commit message calls out explicitly: the new code still uses PGPROC->lwWaiting and PGPROC->lwWaitMode, rather than inventing a parallel set of fields:

 c
 
 
	/*
	 * These are currently used both for lwlocks and buffer content locks,
	 * which is acceptable, although not pretty, because a backend can't wait
	 * for both types of locks at the same time.
	 */
	MyProc->lwWaiting = LW_WS_WAITING;
	MyProc->lwWaitMode = mode;

	proclist_push_tail(&buf_hdr->lock_waiters, MyProcNumber, lwWaitLink);

A backend can only be blocked on one lock at a time, so reusing the fields is safe — but the fields are now lying about their name.

The public API

The lock modes are an enum in src/include/storage/bufmgr.h:

 c
 
 
/*
 * Buffer content lock modes (mode argument for LockBuffer())
 */
typedef enum BufferLockMode
{
	BUFFER_LOCK_UNLOCK,

	/*
	 * A share lock conflicts with exclusive locks.
	 */
	BUFFER_LOCK_SHARE,

	/*
	 * A share-exclusive lock conflicts with itself and exclusive locks.
	 */
	BUFFER_LOCK_SHARE_EXCLUSIVE,

	/*
	 * An exclusive lock conflicts with every other lock type.
	 */
	BUFFER_LOCK_EXCLUSIVE,
} BufferLockMode;

and there is a wrapper that keeps the mode argument a compile-time constant at the call site, because a runtime switch generated worse code:

 c
 
 
/*
 * Handling BUFFER_LOCK_UNLOCK in bufmgr.c leads to sufficiently worse branch
 * prediction to impact performance. Therefore handle that switch here, where
 * most of the time `mode` will be a constant and thus can be optimized out by
 * the compiler.
 */
static inline void
LockBuffer(Buffer buffer, BufferLockMode mode)
{
	if (mode == BUFFER_LOCK_UNLOCK)
		UnlockBuffer(buffer);
	else
		LockBufferInternal(buffer, mode);
}

LockBufferInternal() does the same trick one level down:

 c
 
 
	/*
	 * Test the most frequent lock modes first. While a switch (mode) would be
	 * nice, at least gcc generates considerably worse code for it.
	 *
	 * Call BufferLockAcquire() with a constant argument for mode, to generate
	 * more efficient code for the different lock modes.
	 */
	if (mode == BUFFER_LOCK_SHARE)
		BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_SHARE);
	else if (mode == BUFFER_LOCK_EXCLUSIVE)
		BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_EXCLUSIVE);
	else if (mode == BUFFER_LOCK_SHARE_EXCLUSIVE)
		BufferLockAcquire(buffer, buf_hdr, BUFFER_LOCK_SHARE_EXCLUSIVE);
	else
		elog(ERROR, "unrecognized buffer lock mode: %d", mode);

Because BufferLockAcquire() is static inline and is called with a constant mode three different times, the compiler generates three specialised copies, each with the conflict test folded to a constant. A share lock’s conflict test becomes “is the exclusive bit set?” and nothing else.

Lock ownership tracking

Where lwlock.c keeps a global array of held locks per backend, the buffer content lock reuses the buffer manager’s existing private refcount hash:

 c
 
 
/*
 * This is separated out from PrivateRefCountEntry to allow for copying all
 * the data members via struct assignment.
 */
typedef struct PrivateRefCountData
{
	/*
	 * How many times has the buffer been pinned by this backend.
	 */
	int32		refcount;

	/*
	 * Is the buffer locked by this backend? BUFFER_LOCK_UNLOCK indicates that
	 * the buffer is not locked.
	 */
	BufferLockMode lockmode;
} PrivateRefCountData;

This is elegant: one hash lookup answers both “how many pins does this backend hold on this buffer?” and “in what mode is it locked?”. And it means error recovery needs no new machinery — the resource owner’s buffer cleanup already releases the content locks, so there is no LWLockReleaseAll() equivalent to write.

Releasing a content lock

The release path is split into three pieces, so that the “subtract the lock from the state word” step can be fused with other atomic operations:

 c
 
 
/*
 * Release a previously acquired buffer content lock.
 */
static void
BufferLockUnlock(Buffer buffer, BufferDesc *buf_hdr)
{
	BufferLockMode mode;
	uint64		oldstate;
	uint64		sub;

	mode = BufferLockDisownInternal(buffer, buf_hdr);

	/*
	 * Release my hold on lock, after that it can immediately be acquired by
	 * others, even if we still have to wakeup other waiters.
	 */
	sub = BufferLockReleaseSub(mode);

	oldstate = pg_atomic_sub_fetch_u64(&buf_hdr->state, sub);

	BufferLockProcessRelease(buf_hdr, mode, oldstate);

	/*
	 * Now okay to allow cancel/die interrupts.
	 */
	RESUME_INTERRUPTS();
}

BufferLockReleaseSub() computes what to subtract:

 c
 
 
static inline uint64
BufferLockReleaseSub(BufferLockMode mode)
{
	/*
	 * Turns out that a switch() leads to gcc to generate sufficiently worse code
	 * for this to show up in profiles...
	 */
	if (mode == BUFFER_LOCK_EXCLUSIVE)
		return BM_LOCK_VAL_EXCLUSIVE;
	else if (mode == BUFFER_LOCK_SHARE_EXCLUSIVE)
		return BM_LOCK_VAL_SHARE_EXCLUSIVE;
	else
	{
		Assert(mode == BUFFER_LOCK_SHARE);
		return BM_LOCK_VAL_SHARED;
	}

	return 0;					/* keep compiler quiet */
}

Note that releasing a share lock subtracts BM_LOCK_VAL_SHARED — the count goes down, which is why it is a counter and not a bit.

The wakeup decision:

 c
 
 
static void
BufferLockProcessRelease(BufferDesc *buf_hdr, BufferLockMode mode, uint64 lockstate)
{
	bool		check_waiters = false;
	bool		wake_exclusive = false;

	/* nobody else can have that kind of lock */
	Assert(!(lockstate & BM_LOCK_VAL_EXCLUSIVE));

	/*
	 * If we're still waiting for backends to get scheduled, don't wake them
	 * up again. Otherwise check if we need to look through the waitqueue to
	 * wake other backends.
	 */
	if ((lockstate & BM_LOCK_HAS_WAITERS) &&
		!(lockstate & BM_LOCK_WAKE_IN_PROGRESS))
	{
		if ((lockstate & BM_LOCK_MASK) == 0)
		{
			/*
			 * We released a lock and the lock was, in that moment, free. We
			 * therefore can wake waiters for any kind of lock.
			 */
			check_waiters = true;
			wake_exclusive = true;
		}
		else if (mode == BUFFER_LOCK_SHARE_EXCLUSIVE)
		{
			/*
			 * We released the lock, but another backend still holds a lock. We
			 * can't have released an exclusive lock, as there couldn't
			 * have been other lock holders. If we released a share lock, no
			 * waiters need to be woken up, as there must be other share
			 * lockers. However, if we held a share-exclusive lock, another
			 * backend now could acquire a share-exclusive lock.
			 */
			check_waiters = true;
			wake_exclusive = false;
		}
	}

	/*
	 * As waking up waiters requires the spinlock to be acquired, only do so
	 * if necessary.
	 */
	if (check_waiters)
		BufferLockWakeup(buf_hdr, wake_exclusive);
}

The three cases:

The lock is now completely free (lockstate & BM_LOCK_MASK == 0). Wake everybody — share, share-exclusive and exclusive waiters.
We released a share lock and others still hold locks. Nothing to do: there must still be other share lockers, so a share waiter can proceed.
We released a share-exclusive lock and others still hold locks. A share-exclusive waiter can now proceed (share-exclusive does not conflict with share), but an exclusive waiter cannot.

BufferLockWakeup() then walks the per-buffer wait list. This is where the three-level conflict matrix shows up:

 c
 
 
	proclist_foreach_modify(iter, &buf_hdr->lock_waiters, lwWaitLink)
	{
		PGPROC	 *waiter = GetPGProcByNumber(iter.cur);

		/*
		 * Already woke up a conflicting lock, so skip over this wait list
		 * entry.
		 */
		if (!wake_exclusive && waiter->lwWaitMode == BUFFER_LOCK_EXCLUSIVE)
			continue;
		if (!wake_share_exclusive && waiter->lwWaitMode == BUFFER_LOCK_SHARE_EXCLUSIVE)
			continue;

		proclist_delete(&buf_hdr->lock_waiters, iter.cur, lwWaitLink);
		proclist_push_tail(&wakeup, iter.cur, lwWaitLink);

		/*
		 * Prevent additional wakeups until retryer gets to run. Backends that
		 * are just waiting for the lock to become free don't retry
		 * automatically.
		 */
		new_wake_in_progress = true;

		/*
		 * Signal that the process isn't on the wait list anymore. This allows
		 * BufferLockDequeueSelf() to remove itself from the waitlist with a
		 * proclist_delete(), rather than having to check if it has been
		 * removed from the list.
		 */
		Assert(waiter->lwWaiting == LW_WS_WAITING);
		waiter->lwWaiting = LW_WS_PENDING_WAKEUP;

		/*
		 * Don't wakeup further waiters after waking a conflicting waiter.
		 */
		if (waiter->lwWaitMode == BUFFER_LOCK_SHARE)
		{
			/*
			 * Share locks conflict with exclusive locks.
			 */
			wake_exclusive = false;
		}
		else if (waiter->lwWaitMode == BUFFER_LOCK_SHARE_EXCLUSIVE)
		{
			/*
			 * Share-exclusive locks conflict with share-exclusive and
			 * exclusive locks.
			 */
			wake_exclusive = false;
			wake_share_exclusive = false;
		}
		else if (waiter->lwWaitMode == BUFFER_LOCK_EXCLUSIVE)
		{
			/*
			 * Exclusive locks conflict with all other locks, there's no point
			 * in waking up anybody else.
			 */
			break;
		}
	}

The classic FIFO-ish behaviour: wake share waiters, and stop at the first exclusive waiter, because everybody behind it conflicts with it anyway.

The wait list itself is protected by the buffer header spinlock, which is what BufferLockQueueSelf() does:

 c
 
 
static void
BufferLockQueueSelf(BufferDesc *buf_hdr, BufferLockMode mode)
{
	/*
	 * If we don't have a PGPROC structure, there's no way to wait. This
	 * should never occur, since MyProc should only be null during shared
	 * memory initialization.
	 */
	if (MyProc == NULL)
		elog(PANIC, "cannot wait without a PGPROC structure");

	if (MyProc->lwWaiting != LW_WS_NOT_WAITING)
		elog(PANIC, "queueing for lock while waiting on another one");

	LockBufHdr(buf_hdr);

	/* setting the flag is protected by the spinlock */
	pg_atomic_fetch_or_u64(&buf_hdr->state, BM_LOCK_HAS_WAITERS);
	...
	proclist_push_tail(&buf_hdr->lock_waiters, MyProcNumber, lwWaitLink);

	/* Can release the mutex now */
	UnlockBufHdr(buf_hdr);
}

The header lock is not an LWLock either

A pleasant side effect of the change is that the buffer header lock is now also not a real spinlock with a slock_t. It is a bit in the state word, and LockBufHdr() is a single fetch_or:

 c
 
 
/*
 * Lock buffer header - set BM_LOCKED in buffer state.
 */
uint64
LockBufHdr(BufferDesc *desc)
{
	uint64		old_buf_state;

	Assert(!BufferIsLocal(BufferDescriptorGetBuffer(desc)));

	while (true)
	{
		/*
		 * Always try once to acquire the lock directly, without setting up
		 * the spin-delay infrastructure. The work necessary for that shows up
		 * in profiles and is rarely necessary.
		 */
		old_buf_state = pg_atomic_fetch_or_u64(&desc->state, BM_LOCKED);
		if (likely(!(old_buf_state & BM_LOCKED)))
			break;				/* got lock */

		/* and then spin without atomic operations until lock is released */
		{
			SpinDelayStatus delayStatus;

			init_local_spin_delay(&delayStatus);

			while (old_buf_state & BM_LOCKED)
			{
				perform_spin_delay(&delayStatus);
				old_buf_state = pg_atomic_read_u64(&desc->state);
			}
			finish_spin_delay(&delayStatus);
		}

		/*
		 * Retry. The lock might obviously already be re-acquired by the time
		 * we're attempting to get it again.
		 */
	}

	return old_buf_state | BM_LOCKED;
}

That comment — “Always try once to acquire the lock directly, without setting up the spin-delay infrastructure. The work necessary for that shows up in profiles and is rarely necessary.” — is a micro-optimisation with a real history behind it. Setting up SpinDelayStatus costs enough that it showed up in profiles when the lock was uncontended (which it almost always is).

The uncontended path is now: one lock or on a cache line that is already hot, one branch. The contended path spins on a plain load, no atomic instructions at all, until the bit clears.

Releasing is symmetric and equally cheap:

 c
 
 
/*
 * Unlock the buffer header.
 *
 * This can only be used if the caller did not modify BufferDesc.state. To
 * set/unset flag bits or change the refcount use UnlockBufHdrExt().
 */
static inline void
UnlockBufHdr(BufferDesc *desc)
{
	Assert(pg_atomic_read_u64(&desc->state) & BM_LOCKED);

	pg_atomic_fetch_sub_u64(&desc->state, BM_LOCKED);
}

And the more general version, which sets flags and adjusts the refcount in the same CAS — this is the “release a buffer lock and pin in one atomic operation” that the commit message promised, in its simpler form:

 c
 
 
/*
 * Unlock the buffer header, while atomically unsetting the ones in
 * unset_bits, adding the flags in set_bits, and changing the refcount by
 * refcount_change. If a flag is both cleared and added, it will end up being
 * set.
 *
 * Note that this approach would not trivially work for usagecount, since we
 * need to cap the usagecount at BM_MAX_USAGE_COUNT.
 */
static inline uint64
UnlockBufHdrExt(BufferDesc *desc, uint64 old_buf_state,
				uint64 set_bits, uint64 unset_bits,
				int refcount_change)
{
	for (;;)
	{
		uint64		buf_state = old_buf_state;

		Assert(buf_state & BM_LOCKED);

		/*
		 * Set bits after clearing bits, so that a cleared and re-added flag
		 * survives.
		 */
		buf_state &= ~unset_bits;
		buf_state |= set_bits;
		buf_state &= ~BM_LOCKED;

		if (refcount_change != 0)
			buf_state += BUF_REFCOUNT_ONE * refcount_change;

		if (pg_atomic_compare_exchange_u64(&desc->state, &old_buf_state,
										 buf_state))
		{
			return old_buf_state;
		}
	}
}

Note the comment about the usage count: it cannot be handled this way, because the usage count is capped at BM_MAX_USAGE_COUNT and a blind addition would overflow the cap.

The counterpart — waiting for the header lock to clear — is a pg_noinline spin loop:

 c
 
 
/*
 * Wait until the BM_LOCKED flag isn't set anymore and return the buffer's
 * state at that point.
 *
 * Obviously the buffer could be locked by the time the value is returned, so
 * this is primarily useful in CAS style loops.
 */
pg_noinline uint64
WaitBufHdrUnlocked(BufferDesc *buf)
{
	SpinDelayStatus delayStatus;
	uint64		buf_state;

	init_local_spin_delay(&delayStatus);

	buf_state = pg_atomic_read_u64(&buf->state);

	while (buf_state & BM_LOCKED)
	{
		perform_spin_delay(&delayStatus);
		buf_state = pg_atomic_read_u64(&buf->state);
	}

	finish_spin_delay(&delayStatus);

	return buf_state;
}

pg_noinline is deliberate: this is the slow path, and inlining it into every caller would bloat the instruction cache for no benefit.

The reason the third lock level exists at all is a correctness issue with hint bits.

PostgreSQL caches visibility facts (Xmin committed, Xmax invalid, …) in the tuple’s t_infomask. Those are called hint bits. Setting them modifies the page. If the page is being written out at the same time, the page checksum computed over the bytes that go to disk does not match the bytes that were in memory when the checksum was computed — so the buffer manager had to copy the page before writing it, which is expensive.

With a third lock level, hint-bit setting and write-out can be made mutually exclusive. Both need share-exclusive, and share-exclusive conflicts with itself.

Writing the buffer takes share-exclusive:

 c
 
 
/*
 * Convenience wrapper around FlushBuffer() that locks/unlocks the buffer
 * before/after calling FlushBuffer().
 */
static void
FlushUnlockedBuffer(BufferDesc *buf, SMgrRelation reln,
					IOObject io_object, IOContext io_context)
{
	Buffer		buffer = BufferDescriptorGetBuffer(buf);

	BufferLockAcquire(buffer, buf, BUFFER_LOCK_SHARE_EXCLUSIVE);
	FlushBuffer(buf, reln, io_object, io_context);
	BufferLockUnlock(buffer, buf);
}

Setting hint bits takes share-exclusive too — but if the backend only holds a share lock, it has to upgrade first, and the upgrade can fail:

 c
 
 
/*
 * Helper for BufferBeginSetHintBits() and BufferSetHintBits16().
 *
 * This checks if the current lock mode already suffices to allow hint bits
 * being set and, if not, whether the current lock can be upgraded.
 *
 * Updates *lockstate when returning true.
 */
static inline bool
SharedBufferBeginSetHintBits(Buffer buffer, BufferDesc *buf_hdr, uint64 *lockstate)
{
	uint64		old_state;
	PrivateRefCountEntry *ref;
	BufferLockMode mode;

	ref = GetPrivateRefCountEntry(buffer, true);

	if (ref == NULL)
		elog(ERROR, "buffer is not pinned");

	mode = ref->data.lockmode;
	if (mode == BUFFER_LOCK_UNLOCK)
		elog(ERROR, "buffer is not locked");

	/* we're done if we are already holding a sufficient lock level */
	if (mode == BUFFER_LOCK_EXCLUSIVE || mode == BUFFER_LOCK_SHARE_EXCLUSIVE)
	{
		*lockstate = pg_atomic_read_u64(&buf_hdr->state);
		return true;
	}

	/*
	 * We are only holding a share lock right now, try to upgrade it to
	 * SHARE_EXCLUSIVE.
	 */
	Assert(mode == BUFFER_LOCK_SHARE);

	old_state = pg_atomic_read_u64(&buf_hdr->state);
	while (true)
	{
		uint64		desired_state;

		desired_state = old_state;

		/*
		 * Can't upgrade if somebody else holds the lock in exclusive or
		 * share-exclusive mode.
		 */
		if (unlikely((old_state & (BM_LOCK_VAL_EXCLUSIVE | BM_LOCK_VAL_SHARE_EXCLUSIVE)) != 0))
		{
			return false;
		}

		/* currently held lock state */
		desired_state -= BM_LOCK_VAL_SHARED;

		/* new lock level */
		desired_state += BM_LOCK_VAL_SHARE_EXCLUSIVE;

		if (likely(pg_atomic_compare_exchange_u64(&buf_hdr->state,
												 &old_state, desired_state)))
		{
			ref->data.lockmode = BUFFER_LOCK_SHARE_EXCLUSIVE;
			*lockstate = desired_state;

			return true;
		}
	}
}

Note that the upgrade is a non-blocking CAS. If it fails — because somebody else holds share-exclusive or exclusive — the backend simply does not set the hint bits this time round. That is fine: hint bits are an optimisation, and the whole point is to avoid blocking a reader on a writer.

The comment on BufferBeginSetHintBits() spells out the trade-off:

 c
 
 
/*
 * Try to acquire the right to set hint bits on the buffer.
 *
 * To be allowed to set hint bits, this backend needs to hold either a
 * share-exclusive or an exclusive lock. In case this backend only holds a
 * share lock, this function will try to upgrade the lock to
 * share-exclusive. The caller is only allowed to set hint bits if true is
 * returned.
 *
 * Once BufferBeginSetHintBits() has returned true, hint bits may be set
 * without further calls to BufferBeginSetHintBits(), until the buffer is
 * unlocked.
 *
 *
 * Requiring a share-exclusive lock to set hint bits prevents setting hint
 * bits on buffers that are currently being written out, which could corrupt
 * the checksum on the page. Flushing buffers also requires a share-exclusive
 * lock.
 *
 * Due to a lock >= share-exclusive being required to set hint bits, only one
 * backend can set hint bits at a time. Allowing multiple backends to set
 * hint bits would require more complicated locking: For setting hint bits
 * we'd need to store the count of backends currently setting hint bits, for
 * I/O we would need another lock-level conflicting with the hint-setting
 * lock-level. Given that the share-exclusive lock for setting hint bits is
 * only held for a short time, that backends often would just set the same
 * hint bits and that the cost of occasionally not setting hint bits in hotly
 * accessed pages is fairly low, this seems like an acceptable tradeoff.
 */

The “only one backend at a time” is a direct consequence of the share-lock field being 18 bits wide but the share-exclusive level being a single bit. There is room in the word for a second share-exclusive counter; there just is not one.

The lock-free clock sweep

The other place the new atomicity pays off is the buffer replacement strategy. src/backend/storage/buffer/freelist.c implements the clock sweep, and the clock hand is a plain atomic counter:

 c
 
 
typedef struct
{
	/* Spinlock: protects the values below */
	slock_t		buffer_strategy_lock;

	/*
	 * clock-sweep hand: index of next buffer to consider grabbing. Note that
	 * this isn't a concrete buffer - we only ever increase the value. So, to
	 * get an actual buffer, it needs to be used modulo NBuffers.
	 */
	pg_atomic_uint32 nextVictimBuffer;

	/*
	 * Statistics. These counters should be wide enough that they can't
	 * overflow during a single bgwriter cycle.
	 */
	uint32		completePasses; /* Complete cycles of the clock-sweep */
	pg_atomic_uint32 numBufferAllocs;	/* Buffers allocated since last reset */

	/*
	 * Bgworker process to be notified upon activity or -1 if none. See
	 * StrategyNotifyBgWriter.
	 */
	int			bgwprocno;
} BufferStrategyControl;

The hand moves with a single fetch_add:

 c
 
 
static inline uint32
ClockSweepTick(void)
{
	uint32		victim;

	/*
	 * Atomically move hand ahead one buffer - if there's several processes
	 * doing this, this can lead to buffers being returned slightly out of
	 * apparent order.
	 */
	victim =
		pg_atomic_fetch_add_u32(&StrategyControl->nextVictimBuffer, 1);

	if (victim >= NBuffers)
	{
		uint32		originalVictim = victim;

		/* always wrap what we look up in BufferDescriptors */
		victim = victim % NBuffers;

		/*
		 * If we're the one that just caused a wraparound, force
		 * completePasses to be incremented while holding the spinlock. We
		 * need the spinlock so StrategySyncStart() can return a consistent
		 * value consisting of nextVictimBuffer and completePasses.
		 */
		if (victim == 0)
		{
			uint32		expected;
			uint32		wrapped;
			bool		success = false;

			expected = originalVictim + 1;

			while (!success)
			{
				/*
				 * Acquire the spinlock while increasing completePasses. That
				 * allows other readers to read nextVictimBuffer and
				 * completePasses in a consistent manner which is required for
				 * StrategySyncStart(). In theory delaying the increment
				 * could lead to an overflow of nextVictimBuffers, but that's
				 * highly unlikely and wouldn't be particularly harmful.
				 */
				SpinLockAcquire(&StrategyControl->buffer_strategy_lock);

				wrapped = expected % NBuffers;

				success = pg_atomic_compare_exchange_u32(&StrategyControl->nextVictimBuffer,
														 &expected, wrapped);
				if (success)
					StrategyControl->completePasses++;
				SpinLockRelease(&StrategyControl->buffer_strategy_lock);
			}
		}
	}
	return victim;
}

There is exactly one spinlock left in the whole file, and it is held only for the wraparound bookkeeping. Advancing the hand is a single atomic add — no lock, ever.

The interesting part is StrategyGetBuffer(), which selects a victim and pins it without ever taking the buffer header spinlock:

 c
 
 
	/* Use the "clock sweep" algorithm to find a free buffer */
	trycounter = NBuffers;
	for (;;)
	{
		uint64		old_buf_state;
		uint64		local_buf_state;

		buf = GetBufferDescriptor(ClockSweepTick());

		/*
		 * Check whether the buffer can be used and pin it if so. Do this
		 * using a CAS loop, to avoid having to lock the buffer header.
		 */
		old_buf_state = pg_atomic_read_u64(&buf->state);
		for (;;)
		{
			local_buf_state = old_buf_state;

			/*
			 * If the buffer is pinned or has a nonzero usage_count, we cannot
			 * use it; decrement the usage_count (unless pinned) and keep
			 * scanning.
			 */

			if (BUF_STATE_GET_REFCOUNT(local_buf_state) != 0)
			{
				if (--trycounter == 0)
				{
					/*
					 * We've scanned all the buffers without making any state
					 * changes, so all the buffers are pinned (or were when we
					 * looked at them). We could hope that someone will free
					 * one eventually, but it's probably better to fail than
					 * to risk getting stuck in an infinite loop.
					 */
					elog(ERROR, "no unpinned buffers available");
				}
				break;
			}

			/* See equivalent code in PinBuffer() */
			if (unlikely(local_buf_state & BM_LOCKED))
			{
				old_buf_state = WaitBufHdrUnlocked(buf);
				continue;
			}

			if (BUF_STATE_GET_USAGECOUNT(local_buf_state) != 0)
			{
				local_buf_state -= BUF_USAGECOUNT_ONE;

				if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
												 local_buf_state))
				{
					trycounter = NBuffers;
					break;
				}
			}
			else
			{
				/* pin the buffer if the CAS succeeds */
				local_buf_state += BUF_REFCOUNT_ONE;

				if (pg_atomic_compare_exchange_u64(&buf->state, &old_buf_state,
												 local_buf_state))
				{
					/* Found a usable buffer */
					if (strategy != NULL)
						AddBufferToRing(strategy, buf);
					*buf_state = local_buf_state;

					TrackNewBufferPin(BufferDescriptorGetBuffer(buf));

					return buf;
				}
			}
		}
	}

Read the two CAS cases:

Usage count is non-zero. The buffer is not pinned, but it has been used recently, so it gets a second chance: decrement the usage count and move on. The CAS is needed because the usage count and the refcount live in the same word, and we must not clobber a concurrent refcount change.
Usage count is zero and refcount is zero. The buffer is free. Add one to the refcount — that is the pin — and take it.

The pin and the “is this buffer still free?” decision are the same CAS. Compare that with the old code, which had to take the buffer header spinlock to increment the refcount and then release it. Here, the only time the header lock is involved at all is when BM_LOCKED happens to be set, in which case the code waits for it to clear and retries:

 c
 
 
			/* See equivalent code in PinBuffer() */
			if (unlikely(local_buf_state & BM_LOCKED))
			{
				old_buf_state = WaitBufHdrUnlocked(buf);
				continue;
			}

The second chance, quantified

The usage count is four bits wide, capped at 5:

 c
 
 
/*
 * The maximum allowed value of usage_count represents a tradeoff between
 * accuracy and speed of the clock-sweep buffer management algorithm. A
 * large value (comparable to NBuffers) would approximate LRU semantics.
 * But it can take as many as BM_MAX_USAGE_COUNT+1 complete cycles of the
 * clock-sweep hand to find a free buffer, so in practice we don't want the
 * value to be very large.
 */
#define BM_MAX_USAGE_COUNT	5

StaticAssertDecl(BM_MAX_USAGE_COUNT < (UINT64CONST(1) << BUF_USAGECOUNT_BITS),
				 "BM_MAX_USAGE_COUNT doesn't fit in BUF_USAGECOUNT_BITS bits");

Five is a compromise, and the comment says so: a large value would approximate LRU, but the clock hand might need BM_MAX_USAGE_COUNT + 1 full passes over the buffer array to find a free buffer. With shared_buffers = 128 GB and 8 kB pages that is 16 million buffers, and a value of 5 means at most 6 passes.

The value is bumped on pin, capped at the maximum:

 c
 
 
			/* increase refcount */
			buf_state += BUF_REFCOUNT_ONE;

			if (strategy == NULL)
			{
				/* Default case: increase usagecount unless already max. */
				if (BUF_STATE_GET_USAGECOUNT(buf_state) < BM_MAX_USAGE_COUNT)
					buf_state += BUF_USAGECOUNT_ONE;
			}

This is CLOCK in the textbook sense: the hand gives each buffer a bounded number of second chances, which makes the eviction policy an approximation of LRU rather than a strict LRU.

Caveats

This is a big, invasive change

The commit touches the buffer manager, the buffer replacement strategy, the resource owner machinery, the wait-event taxonomy, and the index access methods. The diff is large, and the new code deliberately duplicates a meaningful chunk of lwlock.c.

The commit message is explicit about the cost:

This commit unfortunately introduces some code that is very similar to the code in lwlock.c, however the code is not equivalent enough to easily merge it. The future wins that this commit makes possible seem worth the cost.

And about the wart:

It’s worth calling out one wart in this commit: Despite content locks not being lwlocks anymore, they continue to use PGPROC->lw* - that seemed better than duplicating the relevant infrastructure.

The wait event taxonomy changed

Content locks are no longer reported as LWLock waits. There are three new wait events in the Buffer class:

text

BUFFER_SHARED	"Waiting to acquire a shared lock on a buffer."
BUFFER_SHARE_EXCLUSIVE	"Waiting to acquire a share exclusive lock on a buffer."
BUFFER_EXCLUSIVE	"Waiting to acquire an exclusive lock on a buffer."

The old BufferContent LWLock tranche is gone, and the BUFFERPIN wait event class was renamed to BUFFER in commit 6c5c393b740 to make room for them. Any monitoring that watched for LWLock waits on the buffer content tranche needs updating.

A backend can only hold one lock at a time

The implementation does not track multiple lock ownerships of the same buffer within one backend. BufferLockConditional() documents the consequence:

 c
 
 
/*
 * Acquire the content lock for the buffer, but only if we don't have to wait.
 *
 * It is allowed to try to conditionally acquire a lock on a buffer that this
 * backend has already locked, but the lock acquisition will always fail, even
 * if the new lock acquisition does not conflict with an already held lock
 * (e.g. two share locks). This is because we currently do not have space to
 * track multiple lock ownerships of the same buffer within one backend. That
 * is ok for the current uses of BufferLockConditional().
 */

So a backend that already holds a share lock on a buffer cannot conditionally upgrade it; BufferLockConditional() returns false. The non-conditional upgrade path (SharedBufferBeginSetHintBits()) does its own CAS and works fine.

The 18-bit share-lock field holds at most 262 143 concurrent share lockers. That is exactly MAX_BACKENDS, so it cannot overflow in practice — but it also means the design assumes at most one share lock per backend per buffer, which is what the private refcount hash enforces.

Deadlock semantics are essentially unchanged

The queueing algorithm is the LWLock algorithm, so the deadlock properties are the same: the wait list is FIFO-ish, exclusive waiters block later waiters, and there is no deadlock detector — PostgreSQL relies on the lock ordering conventions of the code that takes the locks.

The one structural difference is that content locks and LWLocks are now different lock types. A backend that holds a buffer content lock and then tries to take an LWLock (or vice versa) could in principle deadlock in a way the old design could not, since the old content lock was an LWLock and the global lock-ordering discipline applied uniformly. The commit’s answer to this is the PGPROC->lwWaiting check:

 c
 
 
	if (MyProc->lwWaiting != LW_WS_NOT_WAITING)
		elog(PANIC, "queueing for lock while waiting on another one");

A backend cannot block on a buffer content lock while already blocked on something else — that case is a PANIC, not a deadlock. Which is a stronger guarantee than a deadlock detector, but only because it is enforced by the PGPROC state.

Platforms without 64-bit atomics

The whole design rests on pg_atomic_uint64. On platforms that lack native 64-bit atomic support, PostgreSQL emulates them with a spinlock (src/include/port/atomics/fallback.h):

 c
 
 
#if !defined(PG_HAVE_ATOMIC_U64_SUPPORT)

#define PG_HAVE_ATOMIC_U64_SIMULATION

#define PG_HAVE_ATOMIC_U64_SUPPORT
typedef struct pg_atomic_uint64
{
	int			sema;
	volatile uint64 value;
} pg_atomic_uint64;

#define PG_HAVE_ATOMIC_INIT_U64
extern void pg_atomic_init_u64_impl(volatile pg_atomic_uint64 *ptr, uint64 val_);

#define PG_HAVE_ATOMIC_COMPARE_EXCHANGE_U64
extern bool pg_atomic_compare_exchange_u64_impl(volatile pg_atomic_uint64 *ptr,
												uint64 *expected, uint64 newval);

#define PG_HAVE_ATOMIC_FETCH_ADD_U64
extern uint64 pg_atomic_fetch_add_u64_impl(volatile pg_atomic_uint64 *ptr, int64 add_);

#endif /* PG_HAVE_ATOMIC_U64_SUPPORT */

and the implementation in src/backend/port/atomics.c:

 c
 
 
bool
pg_atomic_compare_exchange_u64_impl(volatile pg_atomic_uint64 *ptr,
									uint64 *expected, uint64 newval)
{
	bool		ret;

	/*
	 * Do atomic op under a spinlock. It might look like we could just skip
	 * the cmpxchg if the lock isn't available, but that'd just emulate a
	 * 'weak' compare and swap. I.e. one that allows spurious failures. Since
	 * several algorithms rely on a strong variant and that is efficiently
	 * implementable on most major architectures let's emulate it here as
	 * well.
	 */
	SpinLockAcquire((slock_t *) &ptr->sema);

	/* perform compare/exchange logic */
	ret = ptr->value == *expected;
	*expected = ptr->value;
	if (ret)
		ptr->value = newval;

	/* and release lock */
	SpinLockRelease((slock_t *) &ptr->sema);

	return ret;
}

The other operations are built on top of the CAS, in src/include/port/atomics/generic.h:

 c
 
 
#if !defined(PG_HAVE_ATOMIC_FETCH_OR_U64) && defined(PG_HAVE_ATOMIC_COMPARE_EXCHANGE_U64)
#define PG_HAVE_ATOMIC_FETCH_OR_U64
static inline uint64
pg_atomic_fetch_or_u64_impl(volatile pg_atomic_uint64 *ptr, uint64 or_)
{
	uint64 old;
	old = ptr->value;			/* ok if read is not atomic */
	while (!pg_atomic_compare_exchange_u64_impl(ptr, &old, old | or_))
		/* skip */;
	return old;
}
#endif

So on such a platform, LockBufHdr() — a single fetch_or — becomes a spinlock acquire and a CAS loop. The logical result is identical; the performance is not. The whole point of the change is to make the common path cheap, and on a fallback platform it is not cheap at all.

This matters more for the content lock than for the header lock. The header lock was already a BM_LOCKED bit in the state word before this change — commit dac328c8a68, “bufmgr: Change BufferDesc.state to be a 64-bit atomic”, had just widened that word from 32 to 64 bits — but it was a bit on 32-bit atomics, which have native support on every platform PostgreSQL builds on. The content lock, likewise, was an LWLock, whose own state word is a uint32. Moving both into a 64-bit word turns a native lock acquisition into a spinlock-guarded CAS on a platform without 64-bit atomics.

On every mainstream platform — x86-64, ARM64, POWER, s390x — 64-bit atomics are native and the fallback is not used.

The state word is now a contention point

Before the change, the refcount lived in the state word and the lock lived in the LWLock. Now they live together. That is the whole point — it lets the lock decision see the refcount — but it also means that unrelated operations on the same buffer now touch the same cache line. A buffer that is being pinned and unpinned rapidly is a buffer whose state word is hot.

The BufferDesc is padded to a cache line to keep it from sharing with its neighbours:

 c
 
 
/*
 * Concurrent access to buffer headers has proven to be more efficient if
 * they're cache line aligned. So we force the start of the BufferDescriptors
 * array to be on a cache line boundary and force the elements to be cache
 * line sized.
 *
 * XXX: As this is primarily matters in highly concurrent workloads which
 * probably all are 64bit these days, and the space wastage would be a bit
 * more noticeable on 32bit systems, we don't force the stride to be cache
 * line sized on those. If somebody does actual performance testing, we can
 * reevaluate.
 *
 * Note that local buffer descriptors aren't forced to be aligned - as there's
 * no concurrent access to those it's unlikely to be beneficial.
 *
 * We use a 64-byte cache line size here, because that's the most common
 * size. Making it bigger would be a waste of memory. Even if running on a
 * platform with either 32 or 128 byte line sizes, it's good to align to
 * boundaries and avoid false sharing.
 */
#define BUFFERDESC_PAD_TO_SIZE	(SIZEOF_VOID_P == 8 ? 64 : 1)

typedef union BufferDescPadded
{
	BufferDesc	bufferdesc;
	char		pad[BUFFERDESC_PAD_TO_SIZE];
} BufferDescPadded;

Note the XXX comment: on 32-bit systems the padding is disabled, because the waste would be noticeable. The header comment in the struct is explicit about the constraint:

Be careful to avoid increasing the size of the struct when adding or reordering members. Keeping it below 64 bytes (the most common CPU cache line size) is fairly important for performance.

Removing the embedded LWLock (16 bytes) and replacing it with an 8-byte proclist_head is what keeps the struct within the budget.

Local buffers are a different story

Local buffers (temp tables) use a completely separate mechanism, and the content lock is a no-op for them:

 c
 
 
	Assert(BufferIsPinned(buffer));
	if (BufferIsLocal(buffer))
		return;					/* local buffers need no lock */

Because a local buffer belongs to exactly one backend, there is nothing to lock against.

What it all means in practice

The buffer content lock is now the cheapest lock in PostgreSQL. In the uncontended case it is a single compare-and-swap on a word that is already in the cache. There is no lock structure to walk, no separate state word to coherency-check, no LWLock array to index.

The lock state and the pin state are now the same word. This is the structural change. It means:

Deciding whether a buffer is free to evict and pinning it are one atomic operation (StrategyGetBuffer()).
Checking whether I/O is in progress and taking an exclusive lock are atomic with respect to each other — which is what makes asynchronous writes possible.
Releasing a lock and dropping a pin can be fused into one CAS.

Hint-bit setting is now correct with respect to write-out. Before, a reader could set a hint bit on a page while the checkpointer was writing that same page out, forcing an expensive page copy. Now the two are mutually exclusive via the share-exclusive level, and the copy can go away. That matters more as PostgreSQL moves towards AIO, where many buffers are in flight at once.

The lock-free clock sweep is the quiet win. StrategyGetBuffer() no longer takes the buffer header spinlock to pin a victim. On a busy system with a large shared_buffers, the clock hand’s path through the buffer array no longer serialises on a lock.

Watch the wait events. If you have dashboards that count LWLock waits on BufferContent, they are now dead. The replacements are Buffer/Shared, Buffer/ShareExclusive and Buffer/Exclusive.

PostgreSQL's oid Implementation in C

Jay Hubbert — Thu, 01 Oct 2026 08:19:40 +0000

oid is the type of every system identifier in PostgreSQL: tables, indexes, functions, operators, types, namespaces. It looks like a narrower int4, and that is almost right. It is a 4-byte unsigned integer, carried by value inside a Datum, with its own input/output functions and its own operator set. In this source tree there is also a 64-bit sibling, oid8, that is quietly becoming the identity type for out-of-line TOAST values.

This is based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entries

Two entries in src/include/catalog/pg_type.dat — OID 26 and the newer OID 6437:

 text
 
 
{ oid => '26', array_type_oid => '1028',
 descr => 'object identifier(oid), maximum 4 billion',
 typname => 'oid', typlen => '4', typbyval => 't', typcategory => 'N',
 typispreferred => 't', typinput => 'oidin', typoutput => 'oidout',
 typreceive => 'oidrecv', typsend => 'oidsend', typalign => 'i' },
{ oid => '6437', array_type_oid => '6442',
 descr => 'object identifier(oid8), 8 bytes',
 typname => 'oid8', typlen => '8', typbyval => 't', typcategory => 'N',
 typinput => 'oid8in', typoutput => 'oid8out', typreceive => 'oid8recv',
 typsend => 'oid8send', typalign => 'd' },

oid is 4 bytes, typbyval, typalign => 'i' (TYPALIGN_INT). oid8 is 8 bytes, typbyval, typalign => 'd' (TYPALIGN_DOUBLE). Both are plain storage, so neither is ever TOASTed. oid is marked typispreferred => 't' for its category.

The C representation

Oid is defined in the client-visible src/include/postgres_ext.h:

 c
 
typedef unsigned int Oid;

#define InvalidOid		((Oid) 0)
#define OID_MAX UINT_MAX

Oid8 is a newer addition in src/include/c.h:

 c
 
 
/* 8-byte Object ID */
typedef uint64 Oid8;

#define InvalidOid8		((Oid8) 0)
#define OID8_MAX	UINT64_MAX

The Datum conversions are the same cast-to-from-Datum shape as the integers (src/include/postgres.h):

 c
 
 
static inline Oid
DatumGetObjectId(Datum X) { return (Oid) X; }

static inline Oid8
DatumGetObjectId8(Datum X) { return (Oid8) X; }

Because both are typbyval, an Oid or Oid8 lives inside the Datum itself. InvalidOid (zero) is the sentinel meaning “no object”; it is not a valid identifier.

Input: the permissive unsigned parser

oidin and oid8in do not use the pg_strtoint* parser. They use two unsigned helpers in src/backend/utils/adt/numutils.c:

 c
 
 
Datum
oidin(PG_FUNCTION_ARGS)
{
	char	 *s = PG_GETARG_CSTRING(0);
	Oid			result;

	result = uint32in_subr(s, NULL, "oid", fcinfo->context);
	PG_RETURN_OID(result);
}

uint32in_subr wraps strtoul with base 0:

 c
 
 
	errno = 0;
	cvt = strtoul(s, &endptr, 0);

Base 0 is the detail worth noticing: it accepts 0x/0X hexadecimal and a leading 0 octal form, but not the 0o/0b prefixes that the signed integer parser accepts. So SELECT '0x10'::oid is 16 and SELECT '010'::oid is 8 (octal), while 0b101 is not valid oid input. That asymmetry is a consequence of reusing strtoul rather than the custom signed parser.

The same function also accepts a leading minus sign, which the comment says is for backwards compatibility:

 c
 
 
	/*
	 * For backwards compatibility, we want to accept inputs that are given
	 * with a minus sign, so allow the input value if it matches after either
	 * signed or unsigned extension to long.
	 */

So SELECT '-1'::oid yields 4294967295, not an error. oid8in uses uint64in_subr, which goes through the strtou64 macro (src/include/c.h) and has the same behavior at 8 bytes.

Output

oidout is a single snprintf with %u:

 c
 
 
Datum
oidout(PG_FUNCTION_ARGS)
{
	Oid			o = PG_GETARG_OID(0);
	char	 *result = (char *) palloc(12);

	snprintf(result, 12, "%u", o);
	PG_RETURN_CSTRING(result);
}

The 12-byte buffer holds a leading sign, up to 10 digits, and the NUL. oid8out is more careful about avoiding a redundant strlen; it formats into a stack buffer with pg_ulltoa_n (unsigned, since Oid8 is uint64) and then pallocs and memcpys the known length:

 c
 
 
Datum
oid8out(PG_FUNCTION_ARGS)
{
	Oid8		val = PG_GETARG_OID8(0);
	char		buf[MAXOID8LEN + 1];
	char	 *result;
	int			len;

	len = pg_ulltoa_n(val, buf) + 1;
	buf[len - 1] = '\0';

	result = palloc(len);
	memcpy(result, buf, len);
	PG_RETURN_CSTRING(result);
}

Binary format

oidrecv and oidsend use the width-aware pq_getmsgint / pq_sendint32:

 c
 
 
Datum
oidrecv(PG_FUNCTION_ARGS)
{
	StringInfo	buf = (StringInfo) PG_GETARG_POINTER(0);

	PG_RETURN_OID((Oid) pq_getmsgint(buf, sizeof(Oid)));
}

Datum
oidsend(PG_FUNCTION_ARGS)
{
	Oid			arg1 = PG_GETARG_OID(0);
	StringInfoData buf;

	pq_begintypsend(&buf);
	pq_sendint32(&buf, arg1);
	PG_RETURN_BYTEA_P(pq_endtypsend(&buf));
}

The oid8 pair is identical in shape but uses pq_getmsgint64 / pq_sendint64 (src/backend/utils/adt/oid8.c). Four bytes on the wire for oid, eight for oid8.

Operators and the qsort comparator

The comparison operators are direct C comparisons, e.g. oideq:

 c
 
 
Datum
oideq(PG_FUNCTION_ARGS)
{
	Oid			arg1 = PG_GETARG_OID(0);
	Oid			arg2 = PG_GETARG_OID(1);

	PG_RETURN_BOOL(arg1 == arg2);
}

Because Oid is unsigned, the oid_cmp helper used by qsort compares unsigned (src/backend/utils/adt/oid.c):

 c
 
 
int
oid_cmp(const void *p1, const void *p2)
{
	Oid			v1 = *((const Oid *) p1);
	Oid			v2 = *((const Oid *) p2);

	return pg_cmp_u32(v1, v2);
}

pg_cmp_u32 in src/include/common/int.h returns (a > b) - (a < b) rather than a - b, so it cannot overflow. The btree comparator btoidcmp follows the same three-way pattern, and btoid8cmp does the same for the 64-bit type. oid8 also has a sort-support hook that points at ssup_datum_uint64_cmp, the unsigned 64-bit comparator.

The oidvector type

oidvector (OID 30) is the variable-length array of Oid that system catalogs use. It is not just oid[]; it has its own struct and a validator. buildoidvector shows the layout and one deliberate quirk:

 c
 
 
	result = (oidvector *) palloc0(OidVectorSize(n));

	if (n > 0 && oids)
		memcpy(result->values, oids, n * sizeof(Oid));

	/*
	 * Attach standard array header. For historical reasons, we set the index
	 * lower bound to 0 not 1.
	 */
	SET_VARSIZE(result, OidVectorSize(n));
	result->ndim = 1;
	result->dataoffset = 0;		/* never any nulls */
	result->elemtype = OIDOID;
	result->dim1 = n;
	result->lbound1 = 0;

lbound1 = 0 is the historical 0-based indexing. Because a general oid[] can be cast to oidvector, check_valid_oidvector insists on ndim == 1, dataoffset == 0 (no nulls), and elemtype == OIDOID. The btree comparator btoidvectorcmp sorts first by length, then lexicographically.

oid8 in the wild: TOAST chunk IDs

oid8 is not (only) a user-facing curiosity. In src/backend/catalog/toasting.c the TOAST table’s chunk_id column can be either an Oid or an Oid8, chosen by the table’s toast_value_type reloption:

 c
 
 
			case STDRD_OPTION_TOAST_VALUE_TYPE_OID:
				toast_chunkid_typid = OIDOID;
				break;
			case STDRD_OPTION_TOAST_VALUE_TYPE_OID8:
				toast_chunkid_typid = OID8OID;
				break;

And when building the index on that column, the opclass is chosen to match:

 c
 
 
	if (toast_chunkid_typid == OID8OID)
		opclassIds[0] = OID8_BTREE_OPS_OID;
	else
		opclassIds[0] = OID_BTREE_OPS_OID;

The earlier article on row storage noted that TOAST pointers identify a value by (toast table, value id). The value id here is exactly this chunk_id, and oid8 exists to give those identifiers a wider namespace without abandoning the Oid-style by-value, unsigned representation.

Look at it yourself

 sql
 
 
SELECT '0x10'::oid, '010'::oid; -- 16, 8 (hex, octal)
SELECT '-1'::oid; -- 4294967295
SELECT 4294967295::oid8; -- 4294967295
SELECT typname, typlen, typbyval, typalign FROM pg_type
WHERE typname IN ('oid','oid8');

To see the parser reject what the integer parser accepts:

 sql
 
SELECT '0b101'::oid; -- ERROR: invalid input syntax for type oid: "0b101"

What it all means

Oid is unsigned int, Oid8 is uint64; both are typbyval, so the identifier lives inside the Datum.
InvalidOid (0) is the sentinel, not a valid object id.
Input goes through uint32in_subr / uint64in_subr (strtoul/strtou64 with base 0), so hex and octal prefixes work but 0b/0o do not, and a minus sign is accepted for backwards compatibility.
oid8 is the 64-bit sibling, and in this tree it is the identity type for TOAST chunk_id values.

PostgreSQL's float4 and float8 Implementation in C

Jay Hubbert — Tue, 22 Sep 2026 06:53:25 +0000

float4 (SQL real) and float8 (SQL double precision) are the two IEEE-754 binary floating-point types. They are also the least mysterious types in PostgreSQL: a float4 really is the four bytes of a C float, and a float8 really is the eight bytes of a C double, stuffed directly into a Datum. There is no varlena header, no TOAST, no palloc, no pointer.

That simplicity is exactly why the C is interesting. Because the raw bytes are the value, the interesting decisions have nowhere to hide: they live in the inline helpers in src/include/utils/float.h and the I/O functions in src/backend/utils/adt/float.c. This is a walk through both, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entry

Everything about how a value is stored starts in src/include/catalog/pg_type.dat:

 text
 
 
{ oid => '700', array_type_oid => '1021',
 descr => 'single-precision floating point number, 4-byte storage',
 typname => 'float4', typlen => '4', typbyval => 't', typcategory => 'N',
 typinput => 'float4in', typoutput => 'float4out', typreceive => 'float4recv',
 typsend => 'float4send', typalign => 'i' },
{ oid => '701', array_type_oid => '1022',
 descr => 'double-precision floating point number, 8-byte storage',
 typname => 'float8', typlen => '8', typbyval => 't', typcategory => 'N',
 typispreferred => 't', typinput => 'float8in', typoutput => 'float8out',
 typreceive => 'float8recv', typsend => 'float8send', typalign => 'd' },

Read the fields as a contract:

Field	float4	float8	Meaning
`typlen`	4	8	fixed width, no length word
`typbyval`	`t`	`t`	the value is the `Datum`
`typalign`	`i` (4)	`d` (8)	alignment inside a tuple
`typstorage`	`p` (default)	`p` (default)	not toastable
`typcategory`	`N`	`N`	numeric-ish
array oid	1021	1022	`_float4` / `_float8`

typstorage is not written, so it takes the catalog default of p — TYPSTORAGE_PLAIN (src/include/catalog/pg_type.h), which is the only sensible choice for a fixed-width by-value type. A float8 cannot be compressed or moved out of line; it is 8 bytes, forever. Note also that float8 carries typispreferred => 't' in category N (as does oid); that flag matters when the parser has to pick a numeric type for an ambiguous operator or literal.

The value is the Datum

Datum is uint64_t (src/include/postgres.h). Converting between a float and a Datum cannot be a plain cast — C does not guarantee that integer and floating-point values use the same registers or calling convention — so postgres.h uses a union to reinterpret the bits:

 c
 
 
static inline Datum
Float8GetDatum(float8 X)
{
	union
	{
		float8		value;
		int64		retval;
	}			myunion;

	myunion.value = X;
	return Int64GetDatum(myunion.retval);
}

DatumGetFloat4 / Float4GetDatum do the same dance through int32. So PG_RETURN_FLOAT8(x) is a bit-reinterpretation into a word, and PG_GETARG_FLOAT8(0) is the inverse. Nothing is allocated.

Arithmetic is checked, not silent

If the value is just a C float, why not use a + b? Because C’s floating point is quiet: it returns inf on overflow and 0 on underflow, and PostgreSQL wants an error. The wrappers in src/include/utils/float.h add the checks:

 c
 
 
static inline float4
float4_pl(const float4 val1, const float4 val2)
{
	float4		result;

	result = val1 + val2;
	if (unlikely(isinf(result)) && !isinf(val1) && !isinf(val2))
		float_overflow_error();

	return result;
}

The test is deliberately specific: an infinite result is only an error if neither input was already infinite. 'Infinity'::float8 + 1 is legal; the overflow path is for finite inputs that escaped the range. Division is the busiest case:

 c
 
 
static inline float4
float4_div(const float4 val1, const float4 val2)
{
	float4		result;

	if (unlikely(val2 == 0.0f) && !isnan(val1))
		float_zero_divide_error();
	result = val1 / val2;
	if (unlikely(isinf(result)) && !isinf(val1))
		float_overflow_error();
	if (unlikely(result == 0.0f) && val1 != 0.0f && !isinf(val2))
		float_underflow_error();

	return result;
}

float_overflow_error() and friends are pg_noinline out-of-line ereport() calls in float.c; the header comment explains that inlining them at every call site would bloat the binary. The _safe variants (float8_pl_safe, float8_div_safe, …) thread an escontext through so the same logic can return a soft error for ErrorSaveContext.

There is even a build-time guard at the top of float.c:

 c
 
 
#ifdef __FAST_MATH__
#error -ffast-math is known to break this code
#endif

-ffast-math implies -ffinite-math-only, which would make the isinf() checks above unreliable — and PostgreSQL’s float semantics depend on them.

NaN is a first-class value, with an opinion

IEEE-754 says every comparison with NaN is false, and each NaN is unequal to even itself. That is a disaster for ORDER BY, B-tree indexes, and merge joins, which need a total order. PostgreSQL therefore defines its own comparison functions, all in float.h:

 c
 
 
static inline bool
float4_eq(const float4 val1, const float4 val2)
{
	return isnan(val1) ? isnan(val2) : !isnan(val2) && val1 == val2;
}

static inline bool
float4_lt(const float4 val1, const float4 val2)
{
	return !isnan(val1) && (isnan(val2) || val1 < val2);
}

The rule, spelled out in the header comment, is: all NaNs are equal to each other, and every NaN is larger than every non-NaN. So NaN = NaN is true, NaN > 1 is true, and NaN < 1 is false. The B-tree comparator is built on those predicates (src/backend/utils/adt/float.c):

 c
 
 
int
float4_cmp_internal(float4 a, float4 b)
{
	if (float4_gt(a, b))
		return 1;
	if (float4_lt(a, b))
		return -1;
	return 0;
}

btfloat4cmp / btfloat8cmp are the SQL-visible comparators, and btfloat4sortsupport / btfloat8sortsupport install a direct DatumGetFloat4-based fast comparator so a sort avoids the fmgr call.

Input

float4in and float8in are thin wrappers over float4in_internal and float8in_internal. The float4 path is not a copy of the float8 path with a smaller type — it deliberately calls strtof, and the comment explains why:

The motivation for using strtof() is to avoid a double-rounding problem: for certain decimal inputs, if you round the input correctly to a double, and then round the double to a float, the result is incorrect…

The comment works through 7.038531e-26, which lands exactly on the midpoint between two floats when routed through a double, and therefore rounds the wrong way. Using strtof rounds the decimal to float directly.

Both functions handle NaN and the infinities themselves, because the C standard does not require strtod to accept all the spellings consistently. They also distinguish ERANGE on a genuine out-of-range value from ERANGE on a denormalized result (which they accept). A non-empty string with trailing junk is a syntax error.

Output depends on a GUC

Here is the genuinely surprising part. What float8out prints is not a property of the value alone — it depends on the extra_float_digits setting (src/backend/utils/adt/float.c):

 c
 
 
char *
float8out_internal(double num)
{
	char	 *ascii = (char *) palloc(32);
	int			ndig = DBL_DIG + extra_float_digits;

	if (extra_float_digits > 0)
	{
		double_to_shortest_decimal_buf(num, ascii);
		return ascii;
	}

	(void) pg_strfromd(ascii, 32, ndig, num);
	return ascii;
}

When extra_float_digits > 0 (the default is 1, per src/backend/utils/misc/guc_parameters.dat), PostgreSQL uses a Ryu-based shortest-round-trip algorithm, double_to_shortest_decimal_buf from src/common/shortest_dec.c. Otherwise it falls back to a fixed precision of DBL_DIG + extra_float_digits (15 plus the GUC) via pg_strfromd. float4out mirrors this with FLT_DIG and float_to_shortest_decimal_buf.

Because the result depends on a GUC, these functions cannot be IMMUTABLE. They are marked STABLE in src/include/catalog/pg_proc.dat:

 text
 
 
{ oid => '215', descr => 'I/O',
 proname => 'float8out', provolatile => 's', prorettype => 'cstring',
 proargtypes => 'float8', prosrc => 'float8out' },

That provolatile => 's' is, in fact, the subject of this tree’s tip commit — “Mark GUC-dependent type output functions as STABLE”.

Binary I/O bypasses text entirely: float8send/float8recv use pq_sendfloat8/pq_getmsgfloat8 from libpq/pqformat.c, which write and read the raw IEEE bytes in network order.

The float aggregates use Youngs-Cramer

AVG(), VARIANCE() and STDDEV() over floats do not accumulate sum(X) and sum(X^2). The comment above float8_accum in float.c explains that the schoolbook formula (N*sum(X^2) - sum(X)^2) / N^2 subtracts two huge, nearly equal numbers and loses most of its precision. Instead the transition state is a 3-element float8 array [N, Sx, Sxx], where Sx = sum(X) and Sxx = sum((X - Sx/N)^2), updated incrementally. The parallel combine step generalizes cleanly:

 text
 
 
N = N1 + N2
Sx = Sx1 + Sx2
Sxx = Sxx1 + Sxx2 + N1 * N2 * (Sx1/N1 - Sx2/N2)^2 / N;

Note that N is stored as a float8 to avoid defining a bespoke transition type; the comment concedes that accuracy is fine until N exceeds about 2^52.

Look at it yourself

The catalog and the comparison semantics are both directly observable:

 sql
 
 
SELECT typname, typlen, typbyval, typalign, typstorage, typispreferred
FROM pg_type WHERE typname IN ('float4', 'float8');

-- NaN equals itself and sorts last
SELECT 'NaN'::float8 = 'NaN'::float8, 'NaN'::float8 > 1e308;

-- the GUC changes the printed digits, not the stored value
SHOW extra_float_digits;
SET extra_float_digits = 3;
SELECT 0.1::float8; -- more digits, same 8 bytes
SELECT pg_column_size(0.1::float8), pg_column_size(0.1::float4);

pg_column_size reports 8 and 4 (plus, in a real row, whatever alignment padding the tuple layout imposes — float8 needs 8-byte alignment, float4 only 4). There is no \d+ trick to see here, because there is nothing variable-length to inspect.

What it all means

float4/float8 are by-value, fixed-width, and never toasted. The value is the raw IEEE bits reinterpreted into a Datum through a union.
Every arithmetic operator is a checked wrapper, not a bare +. Overflow and underflow on finite inputs raise errors; existing infinities pass through.
NaN has a defined place in the total order: equal to itself and greater than everything. That is what lets B-trees and sorts treat floats as a normal ordered type.
Output is GUC-dependent. extra_float_digits selects Ryu shortest-round-trip output, which is why the output functions are STABLE.
float4 input uses strtof, not strtod, specifically to dodge a double-rounding bug.

PostgreSQL Allocates Memory Fast

Jay Hubbert — Sat, 19 Sep 2026 09:00:00 +0000

Every C program that wants to be fast needs to be fast at allocating memory. PostgreSQL allocates memory in the tightest loop imaginable: a hash join probes its hash table once per outer row, an aggregate re-evaluates its transition function once per input row, and a sort copies tuples into a buffer. In a query that touches ten million rows, that is tens of millions of allocations, and every one of them has to be cheap.

PostgreSQL’s answer is palloc(), and palloc() is emphatically not malloc(). It does not call malloc() per allocation. It does not keep a general-purpose heap. It quantises every request to a power of two, keeps one free list per size class, carves the tail of an exhausted block into free chunks rather than throwing it away, and recycles whole memory context objects so that a per-tuple context never has to go back to malloc() at all.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The idea: lifetimes, not ownership

The foundational idea is not an allocator technique at all. It is that memory has a lifetime, and the lifetime should decide who frees it.

PostgreSQL’s src/backend/utils/mmgr/README states it plainly:

The main advantage of memory contexts over plain use of malloc/free is that the entire contents of a memory context can be freed easily, without having to request freeing of each individual chunk within it. This is both faster and more reliable than per-chunk bookkeeping.

So instead of tracking every individual allocation, PostgreSQL builds a tree of memory contexts, each with its own lifetime, and frees a whole subtree in one operation. A per-tuple context is reset after every tuple. A per-query context is reset after every query. A per-transaction context is reset after every transaction. And the allocator inside each context is free to make whatever local trade-offs it likes, because the context boundary, not the individual free, is what actually matters for memory return.

The tree is a plain first-child/next-sibling structure in src/include/nodes/memnodes.h:

 c
 
 
typedef struct MemoryContextData
{
	pg_node_attr(abstract)		/* there are no nodes of this type */

	NodeTag		type;			/* identifies exact kind of context */
	/* these two fields are placed here to minimize alignment wastage: */
	bool		isReset;		/* T = no space allocated since last reset */
	bool		allowInCritSection; /* allow palloc in critical section */
	Size		mem_allocated;	/* track memory allocated for this context */
	const MemoryContextMethods *methods;	/* virtual function table */
	MemoryContext parent;		/* NULL if no parent (toplevel context) */
	MemoryContext firstchild;	/* head of linked list of children */
	MemoryContext prevchild;	/* previous child of same parent */
	MemoryContext nextchild;	/* next child of same parent */
	const char *name;			/* context name */
	const char *ident;			/* context ID if any */
	MemoryContextCallback *reset_cbs;	/* list of reset/delete callbacks */
} MemoryContextData;

Note the methods pointer: a memory context is a virtual object. The same tree can be rooted at an AllocSetContext, a SlabContext, a BumpContext or a GenerationContext, and each has completely different allocation behaviour while presenting the same API.

One indirection, but a deliberate one

palloc() does not call malloc(). It calls through the context’s method table. From src/backend/utils/mmgr/mcxt.c:

 c
 
 
/*
 * Call the given function in the MemoryContextMethods for the memory context
 * type that 'pointer' belongs to.
 */
#define MCXT_METHOD(pointer, method) \
	mcxt_methods[GetMemoryChunkMethodID(pointer)].method

And palloc() itself:

 c
 
 
void *
palloc(Size size)
{
	/* duplicates MemoryContextAlloc to avoid increased overhead */
	void	 *ret;
	MemoryContext context = CurrentMemoryContext;

	Assert(MemoryContextIsValid(context));
	AssertNotInCriticalSection(context);

	context->isReset = false;

	/*
	 * For efficiency reasons, we purposefully offload the handling of
	 * allocation failures to the MemoryContextMethods implementation as this
	 * allows these checks to be performed only when an actual malloc needs to
	 * be done to request more memory from the OS. Additionally, not having
	 * to execute any instructions after this call allows the compiler to use
	 * the sibling call optimization. If you're considering adding code after
	 * this call, consider making it the responsibility of the 'alloc'
	 * function instead.
	 */
	ret = context->methods->alloc(context, size, 0);
	/* We expect OOM to be handled by the alloc function */
	Assert(ret != NULL);
	VALGRIND_MEMPOOL_ALLOC(context, ret, size);

	return ret;
}

Three things in that comment deserve attention.

First, the out-of-memory check lives in the allocator, not here. The generic wrapper does not test for NULL and call ereport(ERROR). It cannot: an allocation that is satisfied from an existing free list never touches the operating system, so there is no failure to report. Only when the allocator actually has to malloc() more memory from the OS can it fail — and that is where the check belongs:

 c
 
 
void *
MemoryContextAllocationFailure(MemoryContext context, Size size, int flags)
{
	if ((flags & MCXT_ALLOC_NO_OOM) == 0)
	{
		if (TopMemoryContext)
			MemoryContextStats(TopMemoryContext);
		ereport(ERROR,
				(errcode(ERRCODE_OUT_OF_MEMORY),
				 errmsg("out of memory"),
				 errdetail("Failed on request of size %zu in memory context \"%s\".",
						 size, context->name)));
	}
	return NULL;
}

Second, the tail call. Because the comment says “not having to execute any instructions after this call”, palloc() compiles to something close to a jump: load context->methods->alloc, load size, call it, return the result. The compiler is free to turn the final return ret; into a tail jump into AllocSetAlloc(), avoiding a stack frame in the wrapper entirely. Add an instruction after the call and that optimisation evaporates. This is why the comment ends with a warning to future contributors: “If you’re considering adding code after this call, consider making it the responsibility of the ‘alloc’ function instead.”

Third, palloc0 does not call palloc. It is a copy of the same function with the MemSetAligned bolted on, for the same reason — /* duplicates MemoryContextAlloc to avoid increased overhead */. The extra call would cost a stack frame on one of the hottest paths in the server.

The MemoryContextAlloc() variant, which takes an explicit context rather than using CurrentMemoryContext, carries the identical comment and the identical shape:

 c
 
 
void *
MemoryContextAlloc(MemoryContext context, Size size)
{
	void	 *ret;

	Assert(MemoryContextIsValid(context));
	AssertNotInCriticalSection(context);

	context->isReset = false;

	/*
	 * For efficiency reasons, we purposefully offload the handling of
	 * allocation failures to the MemoryContextMethods implementation as this
	 * allows these checks to be performed only when an actual malloc needs to
	 * be done to request more memory from the OS. Additionally, not having
	 * to execute any instructions after this call allows the compiler to use
	 * the sibling call optimization. If you're considering adding code after
	 * this call, consider making it the responsibility of the 'alloc'
	 * function instead.
	 */
	ret = context->methods->alloc(context, size, 0);

	VALGRIND_MEMPOOL_ALLOC(context, ret, size);

	return ret;
}

How `pfree` knows where to send the pointer

The method-table dispatch works in both directions. pfree() does not take a context argument — a chunk can be freed from any context — so it has to find the right methods from the pointer alone. The trick is an 8-byte header stored immediately before every chunk.

From src/include/utils/memutils_memorychunk.h:

 c
 
 
typedef struct MemoryChunk
{
#ifdef MEMORY_CONTEXT_CHECKING
	Size		requested_size;
#endif

	/* bitfield for storing details about the chunk */
	uint64		hdrmask;		/* must be last */
} MemoryChunk;

In a normal build MemoryChunk is exactly 8 bytes. The file’s header comment describes how those 64 bits are carved up:

The MemoryChunk contains a uint64 field named ‘hdrmask’. This field is used to encode 4 separate pieces of information. Starting with the least significant bits of ‘hdrmask’, the bit space is reserved as follows:

4-bits to indicate the MemoryContextMethodID as defined by MEMORY_CONTEXT_METHODID_MASK

1-bit to denote an “external” chunk (see below)

30-bits reserved for the MemoryContext to use for anything it requires. Most MemoryContexts likely want to store the size of the chunk here.

30-bits for the number of bytes that must be subtracted from the chunk to obtain the address of the block that the chunk is stored on.

And the trick that makes it add up to 64 bits:

If you’re paying close attention, you’ll notice this adds up to 65 bits rather than 64 bits. This is because the highest-order bit of #3 is the same bit as the lowest-order bit of #4. We can do this as we insist that the chunk and block pointers are both MAXALIGNed, therefore the relative offset between those will always be a MAXALIGNed value which means the lowest order bit is always 0.

So one 64-bit word carries the method ID (4 bits, to dispatch pfree), an “external” flag, 30 bits of context-specific payload, and a 30-bit offset back to the owning block. Reading it back:

 c
 
 
static inline void
MemoryChunkSetHdrMask(MemoryChunk *chunk, void *block,
					 Size value, MemoryContextMethodID methodid)
{
	Size		blockoffset = (char *) chunk - (char *) block;

	Assert((char *) chunk >= (char *) block);
	Assert((blockoffset & MEMORYCHUNK_BLOCKOFFSET_MASK) == blockoffset);
	Assert(value <= MEMORYCHUNK_MAX_VALUE);
	Assert((int) methodid <= MEMORY_CONTEXT_METHODID_MASK);

	chunk->hdrmask = (((uint64) blockoffset) << MEMORYCHUNK_BLOCKOFFSET_BASEBIT) |
		(((uint64) value) << MEMORYCHUNK_VALUE_BASEBIT) |
		methodid;
}

GetMemoryChunkMethodID() in mcxt.c reads the low 4 bits, and pfree() becomes a one-line indirect call:

 c
 
 
void
pfree(void *pointer)
{
#ifdef USE_VALGRIND
	MemoryContext context = GetMemoryChunkContext(pointer);
#endif

	MCXT_METHOD(pointer, free_p) (pointer);

	VALGRIND_MEMPOOL_FREE(context, pointer);
}

The 8-byte header is the price of admission. Every allocation pays it. For an 8-byte allocation, that means the usable chunk is 8 bytes and the header is 8 bytes — a 100% overhead. This is precisely why the other allocator types exist.

The AllocSet: power-of-two size classes

aset.c is the default allocator, and it is where all the interesting tricks live. The file’s own header comment describes the design:

 c
 
 
/*--------------------
 * Chunk freelist k holds chunks of size 1 << (k + ALLOC_MINBITS),
 * for k = 0 .. ALLOCSET_NUM_FREELISTS-1.
 *
 * Note that all chunks in the freelists have power-of-2 sizes. This
 * improves recyclability: we may waste some space, but the wasted space
 * should stay pretty constant as requests are made and released.
 *
 * A request too large for the last freelist is handled by allocating a
 * dedicated block from malloc(). The block still has a block header and
 * chunk header, but when the chunk is freed we'll return the whole block
 * to malloc(), not put it on our freelists.
 *
 * CAUTION: ALLOC_MINBITS must be large enough so that
 * 1<<ALLOC_MINBITS is at least MAXALIGN,
 * or we may fail to align the smallest chunks adequately.
 * 8-byte alignment is enough on all currently known machines. This 8-byte
 * minimum also allows us to store a pointer to the next freelist item within
 * the chunk of memory itself.
 *
 * With the current parameters, request sizes up to 8K are treated as chunks,
 * larger requests go into dedicated blocks. Change ALLOCSET_NUM_FREELISTS
 * to adjust the boundary point; and adjust ALLOCSET_SEPARATE_THRESHOLD in
 * memutils.h to agree.
 *--------------------
 */

#define ALLOC_MINBITS		3	/* smallest chunk size is 8 bytes */
#define ALLOCSET_NUM_FREELISTS	11
#define ALLOC_CHUNK_LIMIT	(1 << (ALLOCSET_NUM_FREELISTS-1+ALLOC_MINBITS))
/* Size of largest chunk that we use a fixed size for */
#define ALLOC_CHUNK_FRACTION	4
/* We allow chunks to be at most 1/4 of maxBlockSize (less overhead) */

Eleven free lists. The chunk sizes are:

Free list index	Chunk size	Notes
0	8 bytes	`1 << 3`; also the minimum chunk
1	16 bytes
2	32 bytes
3	64 bytes
4	128 bytes
5	256 bytes
6	512 bytes
7	1 kB
8	2 kB
9	4 kB
10	8 kB	`ALLOC_CHUNK_LIMIT`
—	> 8 kB	dedicated `malloc()` block

A request of 100 bytes lands in free list 4 and gets a 128-byte chunk. A request of 129 bytes lands in free list 5 and gets 256 bytes. The waste is bounded by a factor of two and, crucially, constant — which is the whole point. A 100-byte allocation freed and re-requested always gets the same 128-byte chunk, forever, because it goes back on free list 4.

The context itself:

 c
 
 
typedef struct AllocSetContext
{
	MemoryContextData header;	/* Standard memory-context fields */
	/* Info about storage allocated in this context: */
	AllocBlock	blocks;			/* head of list of blocks in this set */
	MemoryChunk *freelist[ALLOCSET_NUM_FREELISTS];	/* free chunk lists */
	/* Allocation parameters for this context: */
	uint32		initBlockSize;	/* initial block size */
	uint32		maxBlockSize;	/* maximum block size */
	uint32		nextBlockSize;	/* next block size to allocate */
	uint32		allocChunkLimit;	/* effective chunk size limit */
	/* freelist this context could be put in, or -1 if not a candidate: */
	int			freeListIndex;	/* index in context_freelists[], or -1 */
} AllocSetContext;

Eleven pointers, plus four sizing parameters, plus one integer for the context recycling pool. That is the entire state of an AllocSet.

And the block:

 c
 
 
typedef struct AllocBlockData
{
	AllocSet	aset;			/* aset that owns this block */
	AllocBlock	prev;			/* prev block in aset's blocks list, if any */
	AllocBlock	next;			/* next block in aset's blocks list, if any */
	char	 *freeptr;		/* start of free space in this block */
	char	 *endptr;			/* end of space in this block */
}			AllocBlockData;

Blocks are obtained from malloc() and carved into chunks. freeptr walks forward as the active block fills up.

Computing the free list index

Given a request size, the allocator has to decide which of the eleven free lists to use. That is AllocSetFreeIndex():

 c
 
 
static inline int
AllocSetFreeIndex(Size size)
{
	int			idx;

	if (size > (1 << ALLOC_MINBITS))
	{
		/*----------
		 * At this point we must compute ceil(log2(size >> ALLOC_MINBITS)).
		 * This is the same as
		 *		pg_leftmost_one_pos32((size - 1) >> ALLOC_MINBITS) + 1
		 * or equivalently
		 *		pg_leftmost_one_pos32(size - 1) - ALLOC_MINBITS + 1
		 *
		 * However, for platforms without intrinsic support, we duplicate the
		 * logic here, allowing an additional optimization. It's reasonable
		 * to assume that ALLOC_CHUNK_LIMIT fits in 16 bits, so we can unroll
		 * the byte-at-a-time loop in pg_leftmost_one_pos32 and just handle
		 * the last two bytes.
		 *
		 * Yes, this function is enough of a hot-spot to make it worth this
		 * much trouble.
		 *----------
		 */
#ifdef HAVE_BITSCAN_REVERSE
		idx = pg_leftmost_one_pos32((uint32) size - 1) - ALLOC_MINBITS + 1;
#else
		uint32		t,
					tsize;

		/* Statically assert that we only have a 16-bit input value. */
		StaticAssertDecl(ALLOC_CHUNK_LIMIT < (1 << 16),
						 "ALLOC_CHUNK_LIMIT must be less than 64kB");

		tsize = size - 1;
		t = tsize >> 8;
		idx = t ? pg_leftmost_one_pos[t] + 8 : pg_leftmost_one_pos[tsize];
		idx -= ALLOC_MINBITS - 1;
#endif

		Assert(idx < ALLOCSET_NUM_FREELISTS);
	}
	else
		idx = 0;

	return idx;
}

“Yes, this function is enough of a hot-spot to make it worth this much trouble.” That is a real PostgreSQL comment, and it explains the shape of the code. pg_leftmost_one_pos32() is the count-leading-zeros primitive:

 c
 
 
static inline int
pg_leftmost_one_pos32(uint32 word)
{
#ifdef HAVE__BUILTIN_CLZ
	Assert(word != 0);

	return 31 - __builtin_clz(word);

On x86 this compiles to a single bsr instruction (or lzcnt). One instruction to convert “how big is this request?” into “which free list do I use?”. The portable fallback uses a 256-entry byte table and only looks at the top two bytes of the value, which is why the code carries the StaticAssertDecl about ALLOC_CHUNK_LIMIT fitting in 16 bits.

The index is stored in the chunk header

Here is the part that makes pfree() cheap. When a chunk is freed, the allocator has to know which free list to put it back on. It could recompute it from the chunk’s size — but it doesn’t need to, because the chunk header has 30 spare bits.

Recall the MemoryChunk layout: bits 5–34 are a 30-bit “value” field that the context may use for whatever it likes. aset.c uses it to store the free list index:

 c
 
 
	/* store the free list index in the value field */
	MemoryChunkSetHdrMask(chunk, block, fidx, MCTX_ASET_ID);

and AllocSetFree() reads it straight back out:

 c
 
 
		fidx = MemoryChunkGetValue(chunk);
		Assert(FreeListIdxIsValid(fidx));
		link = GetFreeListLink(chunk);

No clz, no lookup table, no search. One shift and one mask. The comment in aset.c makes the rationale explicit:

Note that all chunks in the freelists have power-of-2 sizes. This improves recyclability: we may waste some space, but the wasted space should stay pretty constant as requests are made and released.

The free list index is the size class, so storing the index is exactly equivalent to storing the size — but it is 4 bits instead of 32, and it needs no arithmetic to recover.

The free list link lives inside the chunk

A free list needs a next pointer per chunk. Where does that go? PostgreSQL puts it in the chunk’s own user data:

 c
 
 
/*
 * Obtain a AllocFreeListLink for the given chunk. Allocation sizes are
 * always at least sizeof(AllocFreeListLink), so we reuse the pointer's memory
 * itself to store the freelist link.
 */
#define GetFreeListLink(chkptr) \
	(AllocFreeListLink *) ((char *) (chkptr) + ALLOC_CHUNKHDRSZ)

This is exactly why ALLOC_MINBITS is 3 — the smallest chunk is 8 bytes, which is exactly enough to hold a pointer. When the chunk is live, that space is the caller’s data. When the chunk is free, it is the list link. No separate array of freelist nodes, no extra allocation.

The allocation fast path

AllocSetAlloc() is the whole story for a typical small allocation:

 c
 
 
void *
AllocSetAlloc(MemoryContext context, Size size, int flags)
{
	AllocSet	set = (AllocSet) context;
	AllocBlock	block;
	MemoryChunk *chunk;
	int			fidx;
	Size		chunk_size;
	Size		availspace;

	Assert(AllocSetIsValid(set));

	/* due to the keeper block set->blocks should never be NULL */
	Assert(set->blocks != NULL);

	/*
	 * If requested size exceeds maximum for chunks we hand the request off to
	 * AllocSetAllocLarge().
	 */
	if (size > set->allocChunkLimit)
		return AllocSetAllocLarge(context, size, flags);

	/*
	 * Request is small enough to be treated as a chunk. Look in the
	 * corresponding free list to see if there is a free chunk we could reuse.
	 * If one is found, remove it from the free list, make it again a member
	 * of the alloc set and return its data address.
	 *
	 * Note that we don't attempt to ensure there's space for the sentinel
	 * byte here. We expect a large proportion of allocations to be for sizes
	 * which are already a power of 2. If we were to always make space for a
	 * sentinel byte in MEMORY_CONTEXT_CHECKING builds, then we'd end up
	 * doubling the memory requirements for such allocations.
	 */
	fidx = AllocSetFreeIndex(size);
	chunk = set->freelist[fidx];
	if (chunk != NULL)
	{
		AllocFreeListLink *link = GetFreeListLink(chunk);

		/* Allow access to the chunk header. */
		VALGRIND_MAKE_MEM_DEFINED(chunk, ALLOC_CHUNKHDRSZ);

		Assert(fidx == MemoryChunkGetValue(chunk));

		/* pop this chunk off the freelist */
		VALGRIND_MAKE_MEM_DEFINED(link, sizeof(AllocFreeListLink));
		set->freelist[fidx] = link->next;
		VALGRIND_MAKE_MEM_NOACCESS(link, sizeof(AllocFreeListLink));
		...
		return MemoryChunkGetPointer(chunk);
	}

	/*
	 * Choose the actual chunk size to allocate.
	 */
	chunk_size = GetChunkSizeFromFreeListIdx(fidx);
	Assert(chunk_size >= size);

	block = set->blocks;
	availspace = block->endptr - block->freeptr;

	/*
	 * If there is enough room in the active allocation block, we will put the
	 * chunk into that block. Else must start a new one.
	 */
	if (unlikely(availspace < (chunk_size + ALLOC_CHUNKHDRSZ)))
		return AllocSetAllocFromNewBlock(context, size, flags, fidx);

	/* There's enough space on the current block, so allocate from that */
	return AllocSetAllocChunkFromBlock(context, block, size, chunk_size, fidx);
}

The order of operations is the whole design:

Too big? Hand off to AllocSetAllocLarge(), which does a dedicated malloc().
Free list non-empty? Pop it. This is the common case in a long-running loop, and it costs: one clz, one array load, one pointer swap. No malloc, no free, no copying.
Room in the active block? Bump freeptr and write the header.
Otherwise carve the tail (below) and start a new block.

Notice the Assert(fidx == MemoryChunkGetValue(chunk)); — the assertion that the stored index really does match the list the chunk came off. That is the invariant that makes pfree O(1) with respect to size classes.

Carving the tail of a dying block

Step 4 is where the interesting work happens. When the active block cannot satisfy the request, PostgreSQL does not throw away the remaining space in it. It carves the remainder into free chunks, one per size class, and puts them on the free lists.

From AllocSetAllocFromNewBlock():

 c
 
 
	/*
	 * The existing active (top) block does not have enough room for the
	 * requested allocation, but it might still have a useful amount of space
	 * in it. Once we push it down in the block list, we'll never try to
	 * allocate more space from it. So, before we do that, carve up its free
	 * space into chunks that we can put on the set's freelists.
	 *
	 * Because we can only get here when there's less than ALLOC_CHUNK_LIMIT
	 * left in the block, this loop cannot iterate more than
	 * ALLOCSET_NUM_FREELISTS-1 times.
	 */
	while (availspace >= ((1 << ALLOC_MINBITS) + ALLOC_CHUNKHDRSZ))
	{
		AllocFreeListLink *link;
		MemoryChunk *chunk;
		Size		availchunk = availspace - ALLOC_CHUNKHDRSZ;
		int			a_fidx = AllocSetFreeIndex(availchunk);

		/*
		 * In most cases, we'll get back the index of the next larger freelist
		 * than the one we need to put this chunk on. The exception is when
		 * availchunk is exactly a power of 2.
		 */
		if (availchunk != GetChunkSizeFromFreeListIdx(a_fidx))
		{
			a_fidx--;
			Assert(a_fidx >= 0);
			availchunk = GetChunkSizeFromFreeListIdx(a_fidx);
		}

		chunk = (MemoryChunk *) (block->freeptr);

		/* Prepare to initialize the chunk header. */
		VALGRIND_MAKE_MEM_UNDEFINED(chunk, ALLOC_CHUNKHDRSZ);
		block->freeptr += (availchunk + ALLOC_CHUNKHDRSZ);
		availspace -= (availchunk + ALLOC_CHUNKHDRSZ);

		/* store the freelist index in the value field */
		MemoryChunkSetHdrMask(chunk, block, a_fidx, MCTX_ASET_ID);
#ifdef MEMORY_CONTEXT_CHECKING
		chunk->requested_size = InvalidAllocSize;	/* mark it free */
#endif
		/* push this chunk onto the free list */
		link = GetFreeListLink(chunk);

		VALGRIND_MAKE_MEM_DEFINED(link, sizeof(AllocFreeListLink));
		link->next = set->freelist[a_fidx];
		VALGRIND_MAKE_MEM_NOACCESS(link, sizeof(AllocFreeListLink));

		set->freelist[a_fidx] = chunk;
	}

Read that loop carefully. Suppose the block has 5000 bytes left. It takes the largest chunk that fits into 5000 bytes: 4096 bytes, on free list 9, leaving 904. Then 512 bytes (free list 6), leaving 392. Then 256 (free list 5), leaving 136. Then 64 (free list 3), leaving 72. Then 32 (free list 2), leaving 0. Loop ends.

So one dying block yields five reusable chunks in five different size classes, and nothing at all is wasted — not even the 8-byte chunk headers, which are counted against the available space as the loop runs. Compare that to the naive behaviour of throwing the tail away: those 5000 bytes would have been returned to malloc() (or worse, kept and never reused), and the next 4096-byte request would have had to malloc() again.

Then the block size doubles:

 c
 
 
	/*
	 * The first such block has size initBlockSize, and we double the space in
	 * each succeeding block, but not more than maxBlockSize.
	 */
	blksize = set->nextBlockSize;
	set->nextBlockSize <<= 1;
	if (set->nextBlockSize > set->maxBlockSize)
		set->nextBlockSize = set->maxBlockSize;

So a context starts with an 8 kB block (ALLOCSET_DEFAULT_INITSIZE), then 16 kB, then 32 kB, up to maxBlockSize (8 MB by default). The rationale is in the header comment:

The first block allocated for an allocset has size initBlockSize. Each time we have to allocate another block, we double the block size (if possible, and without exceeding maxBlockSize), so as to reduce the bookkeeping load on malloc().

Note also the retry loop for huge blocks:

 c
 
 
	/*
	 * We could be asking for pretty big blocks here, so cope if malloc fails.
	 * But give up if there's less than 1 MB or so available...
	 */
	while (block == NULL && blksize > 1024 * 1024)
	{
		blksize >>= 1;
		if (blksize < required_size)
			break;
		block = (AllocBlock) malloc(blksize);
	}

Asking for 64 MB and getting a failure is survivable; asking for 64 MB when the machine has 40 MB free is not, so PostgreSQL halves the request and tries again down to 1 MB.

Chunk limits are per-context, not global

The 8 kB ALLOC_CHUNK_LIMIT is the global maximum, but each context computes its own effective limit:

 c
 
 
static inline uint32
AllocSetComputeChunkLimit(Size maxBlockSize)
{
	uint32		allocChunkLimit;

	/*
	 * Compute the allocation chunk size limit for this context. It can't be
	 * more than ALLOC_CHUNK_LIMIT because of the fixed number of freelists.
	 * If maxBlockSize is small then requests exceeding the maxBlockSize, or
	 * even a significant fraction of it, should be treated as large chunks
	 * too. For the typical case of maxBlockSize a power of 2, the chunk size
	 * limit will be at most 1/8th maxBlockSize, so that given a stream of
	 * requests that are all the maximum chunk size we will waste at most
	 * 1/8th of the allocated space.
	 ...
	 */
	allocChunkLimit = ALLOC_CHUNK_LIMIT;
	while ((Size) (allocChunkLimit + ALLOC_CHUNKHDRSZ) >
		 (Size) ((maxBlockSize - ALLOC_BLOCKHDRSZ) / ALLOC_CHUNK_FRACTION))
		allocChunkLimit >>= 1;

	return allocChunkLimit;
}

A context with maxBlockSize of 8 kB (the ALLOCSET_SMALL_SIZES preset) gets a chunk limit of 1 kB or so, so that a stream of maximum-size requests cannot waste more than a quarter of the block. Small contexts therefore do not get 8 kB chunks they can never fill.

`repalloc`: sometimes it is free

Growing a chunk usually means copying. But not always. AllocSetRealloc() first checks whether the existing chunk is already big enough:

 c
 
 
	/*
	 * Chunk sizes are aligned to power of 2 in AllocSetAlloc(). Maybe the
	 * allocated area already is >= the new size. (In particular, we will
	 * fall out here if the requested size is a decrease.)
	 */
	if (oldchksize >= size)
	{
		...
		/* Disallow access to the chunk header. */
		VALGRIND_MAKE_MEM_NOACCESS(chunk, ALLOC_CHUNKHDRSZ);

		return pointer;
	}

oldchksize is GetChunkSizeFromFreeListIdx(fidx) — the chunk size, not the requested size. So a 40-byte allocation lives in a 64-byte chunk; repallocing it to 60 bytes finds 64 >= 60 and returns the same pointer, with no copy at all. The extra 24 bytes were already there, reserved by the power-of-two quantisation.

This is a free side effect of the quantisation scheme. The waste is already paid; handing it back to the caller costs nothing.

Growing past the chunk size does require a copy, and the comment is candid about why the obvious optimisation was rejected:

 c
 
 
		/*
		 * Enlarge-a-small-chunk case. We just do this by brute force, ie,
		 * allocate a new chunk and copy the data. Since we know the existing
		 * data isn't huge, this won't involve any great memcpy expense, so
		 * it's not worth being smarter. (At one time we tried to avoid
		 * memcpy when it was possible to enlarge the chunk in-place, but that
		 * turns out to misbehave unpleasantly for repeated cycles of
		 * palloc/repalloc/pfree: the eventually freed chunks go into the
		 * wrong freelist for the next initial palloc request, and so we leak
		 * memory indefinitely. See pgsql-hackers archives for 2007-08-11.)
		 */

“the eventually freed chunks go into the wrong freelist” — enlarging in place would change the chunk’s size class, and when the chunk is later freed it would go onto a different free list than the one it came off, which permanently removes a chunk from the list it was supposed to return to. A slow leak, per cycle, forever. Good lesson.

Internal fragmentation: the trade-off

Quantising to powers of two is not free. A 100-byte request occupies 128 bytes. Here is the full picture, ignoring the 8-byte chunk header:

Request size	Chunk size	Wasted bytes	Overhead
1	8	7	700%
9	16	7	78%
17	32	15	88%
33	64	31	94%
65	128	63	97%
129	256	127	98%
257	512	255	99%
513	1 kB	511	100%
1025	2 kB	1023	100%
2049	4 kB	2047	100%
4097	8 kB	4095	100%
8193	dedicated block	0	header only

The “overhead” column is worst-case (a request one byte above a power of two). Averaged over a uniformly distributed range of request sizes, a request wastes roughly a third of its chunk — and, crucially, the waste is predictable, which matters more than the average, because predictable overhead can be budgeted for.

The counter-argument is in the header comment:

Note that all chunks in the freelists have power-of-2 sizes. This improves recyclability: we may waste some space, but the wasted space should stay pretty constant as requests are made and released.

A general-purpose allocator that returns exactly the requested size wastes nothing on paper, but then a 100-byte free chunk can only satisfy a 100-byte request, and the space in a 700-byte gap can satisfy nothing. Quantisation guarantees that free space is always usable, at the cost of always being a bit larger than needed.

The other side of the ledger: requests above 8 kB bypass the whole scheme and get their own malloc() block, so large allocations (index tuples, TOAST chunks, hash table buckets) pay no quantisation waste at all.

Recycling whole contexts

Now the second half of the story. Contexts themselves are allocated with malloc() and freed with free(). If a query creates a per-tuple context and destroys it per tuple, that is a malloc/free pair per tuple — exactly the thrashing the context system was supposed to avoid.

PostgreSQL’s answer is a pool of dead contexts. The design note is in aset.c:

 c
 
 
/*
 * Rather than repeatedly creating and deleting memory contexts, we keep some
 * freed contexts in freelists so that we can hand them out again with little
 * work. Before putting a context in a freelist, we reset it so that it has
 * only its initial malloc chunk and no others. To be a candidate for a
 * freelist, a context must have the same minContextSize/initBlockSize as
 * other contexts in the list; but its maxBlockSize is irrelevant since that
 * doesn't affect the size of the initial chunk.
 *
 * We currently provide one freelist for ALLOCSET_DEFAULT_SIZES contexts
 * and one for ALLOCSET_SMALL_SIZES contexts; the latter works for
 * ALLOCSET_START_SMALL_SIZES too, since only the maxBlockSize differs.
 *
 * Ordinarily, we re-use freelist contexts in last-in-first-out order, in
 * hopes of improving locality of reference. But if there get to be too
 * many contexts in the list, we'd prefer to drop the most-recently-created
 * contexts in hopes of keeping the process memory map compact.
 * We approximate that by simply deleting all existing entries when the list
 * overflows, on the assumption that queries that allocate a lot of contexts
 * will probably free them in more or less reverse order of allocation.
 *
 * Contexts in a freelist are chained via their nextchild pointers.
 */
#define MAX_FREE_CONTEXTS 100	/* arbitrary limit on freelist length */

“Contexts in a freelist are chained via their nextchild pointers.” That is a wonderful piece of field-hacking: a deleted context is no longer in the context tree, so its nextchild field (which links it to its siblings) is dead weight. It is reused as the free-list link.

The pool itself:

 c
 
 
typedef struct AllocSetFreeList
{
	int			num_free;		/* current list length */
	AllocSetContext *first_free;	/* list header */
} AllocSetFreeList;

/* context_freelists[0] is for default params, [1] for small params */
static AllocSetFreeList context_freelists[2] =
{
	{
		0, NULL
	},
	{
		0, NULL
	}
};

Two pools: one for “ordinary” contexts and one for “small” contexts, matching the two size presets in memutils.h:

 c
 
 
#define ALLOCSET_DEFAULT_MINSIZE 0
#define ALLOCSET_DEFAULT_INITSIZE (8 * 1024)
#define ALLOCSET_DEFAULT_MAXSIZE (8 * 1024 * 1024)
#define ALLOCSET_DEFAULT_SIZES \
	ALLOCSET_DEFAULT_MINSIZE, ALLOCSET_DEFAULT_INITSIZE, ALLOCSET_DEFAULT_MAXSIZE

#define ALLOCSET_SMALL_MINSIZE	 0
#define ALLOCSET_SMALL_INITSIZE (1 * 1024)
#define ALLOCSET_SMALL_MAXSIZE (8 * 1024)
#define ALLOCSET_SMALL_SIZES \
	ALLOCSET_SMALL_MINSIZE, ALLOCSET_SMALL_INITSIZE, ALLOCSET_SMALL_MAXSIZE

Deleting a context puts it on the pool

AllocSetDelete() is where the recycling happens:

 c
 
 
	/*
	 * If the context is a candidate for a freelist, put it into that freelist
	 * instead of destroying it.
	 */
	if (set->freeListIndex >= 0)
	{
		AllocSetFreeList *freelist = &context_freelists[set->freeListIndex];

		/*
		 * Reset the context, if it needs it, so that we aren't hanging on to
		 * more than the initial malloc chunk.
		 */
		if (!context->isReset)
			MemoryContextResetOnly(context);

		/*
		 * If the freelist is full, just discard what's already in it. See
		 * comments with context_freelists[].
		 */
		if (freelist->num_free >= MAX_FREE_CONTEXTS)
		{
			while (freelist->first_free != NULL)
			{
				AllocSetContext *oldset = freelist->first_free;

				freelist->first_free = (AllocSetContext *) oldset->header.nextchild;
				freelist->num_free--;

				/* Destroy the context's vpool --- see notes below */
				VALGRIND_DESTROY_MEMPOOL(oldset);

				/* All that remains is to free the header/initial block */
				free(oldset);
			}
			Assert(freelist->num_free == 0);
		}

		/* Now add the just-deleted context to the freelist. */
		set->header.nextchild = (MemoryContext) freelist->first_free;
		freelist->first_free = set;
		freelist->num_free++;

		return;
	}

Push at the head → LIFO. The comment explains why: “in hopes of improving locality of reference”. The most recently freed context was also the most recently allocated, so it is the one whose memory is hottest in the cache.

The pool is capped at 100 contexts. When it overflows, the entire pool is thrown away rather than just the oldest entry — because, as the comment says, “queries that allocate a lot of contexts will probably free them in more or less reverse order of allocation”, so dropping everything keeps the process memory map compact.

Creating a context takes one from the pool

The other half is AllocSetContextCreateInternal():

 c
 
 
	/*
	 * Check whether the parameters match either available freelist. We do
	 * not need to demand a match of maxBlockSize.
	 */
	if (minContextSize == ALLOCSET_DEFAULT_MINSIZE &&
		initBlockSize == ALLOCSET_DEFAULT_INITSIZE)
		freeListIndex = 0;
	else if (minContextSize == ALLOCSET_SMALL_MINSIZE &&
			 initBlockSize == ALLOCSET_SMALL_INITSIZE)
		freeListIndex = 1;
	else
		freeListIndex = -1;

	/*
	 * If a suitable freelist entry exists, just recycle that context.
	 */
	if (freeListIndex >= 0)
	{
		AllocSetFreeList *freelist = &context_freelists[freeListIndex];

		if (freelist->first_free != NULL)
		{
			/* Remove entry from freelist */
			set = freelist->first_free;
			freelist->first_free = (AllocSet) set->header.nextchild;
			freelist->num_free--;

			/* nextBlockSize should have been reset in AllocSetReset() */
			Assert(set->nextBlockSize == set->initBlockSize);

			if (set->maxBlockSize != maxBlockSize)
			{
				/*
				 * Update its maxBlockSize and allocChunkLimit; everything
				 * else should be OK.
				 */
				set->maxBlockSize = maxBlockSize;
				set->allocChunkLimit = AllocSetComputeChunkLimit(maxBlockSize);
			}

			/* Reinitialize its header, installing correct name and parent */
			MemoryContextCreate((MemoryContext) set,
								T_AllocSetContext,
								MCTX_ASET_ID,
								parent,
								name);

			((MemoryContext) set)->mem_allocated =
				KeeperBlock(set)->endptr - ((char *) set);

			return (MemoryContext) set;
		}
	}

No malloc. The context object is reused verbatim, only its header is re-initialised and its maxBlockSize adjusted (the block size limit is the only parameter that does not affect the initial chunk’s size, so a context can be recycled across different maxBlockSize values).

The result: a per-tuple context costs zero system allocations once the pool is warm. Create, use, destroy, repeat, and the whole cycle is pointer arithmetic.

Reset versus delete

Two operations, and the difference matters:

MemoryContextReset() — free everything allocated in the context (and its children), but keep the context object itself. The context remains usable.
MemoryContextDelete() — free the context object too. The name is gone.

For AllocSet, the two differ in one crucial detail: the keeper block.

 c
 
 
/*
 * AllocSetReset
 *		Frees all memory which is allocated in the given set.
 *
 * Actually, this routine has some discretion about what to do.
 * It should mark all allocated chunks freed, but it need not necessarily
 * give back all the resources the set owns. Our actual implementation is
 * that we give back all but the "keeper" block (which we must keep, since
 * it shares a malloc chunk with the context header). In this way, we don't
 * thrash malloc() when a context is repeatedly reset after small allocations,
 * which is typical behavior for per-tuple contexts.
 */

The “keeper block” is the region of memory immediately following the AllocSetContext struct, in the same malloc() chunk:

 c
 
 
/* Obtain the keeper block for an allocation set */
#define KeeperBlock(set) \
	((AllocBlock) (((char *) set) + MAXALIGN(sizeof(AllocSetContext))))

/* Check if the block is the keeper block of the given allocation set */
#define IsKeeperBlock(set, block) ((block) == (KeeperBlock(set)))

So the very first malloc() a context makes contains both the context header and its first block. AllocSetReset() keeps that block, wipes its contents, and free()s everything else:

 c
 
 
	block = set->blocks;

	/* New blocks list will be just the keeper block */
	set->blocks = KeeperBlock(set);

	while (block != NULL)
	{
		AllocBlock	next = block->next;

		if (IsKeeperBlock(set, block))
		{
			/* Reset the block, but don't return it to malloc */
			char	 *datastart = ((char *) block) + ALLOC_BLOCKHDRSZ;
			...
			block->freeptr = datastart;
			block->prev = NULL;
			block->next = NULL;
		}
		else
		{
			/* Normal case, release the block */
			context->mem_allocated -= block->endptr - ((char *) block);
			...
			free(block);
		}
		block = next;
	}

So a per-tuple context that allocates 200 bytes and is reset per tuple does exactly one malloc() (at creation) and zero free()s (until it is deleted). Compare with malloc/free per tuple, which would be two system calls per tuple.

AllocSetDelete() is the version that gives everything back, and it is also the version that puts the context on the recycling pool.

The other allocators

aset.c is the general-purpose one, but PostgreSQL ships four context types, because different allocation patterns want different trade-offs. The method table in mcxt.c lists them all:

 c
 
 
static const MemoryContextMethods mcxt_methods[] = {
	/* aset.c */
	[MCTX_ASET_ID].alloc = AllocSetAlloc,
	...
	/* generation.c */
	[MCTX_GENERATION_ID].alloc = GenerationAlloc,
	...
	/* slab.c */
	[MCTX_SLAB_ID].alloc = SlabAlloc,
	...
	/* alignedalloc.c */
	[MCTX_ALIGNED_REDIRECT_ID].alloc = NULL,	/* not required */
	...
	/* bump.c */
	[MCTX_BUMP_ID].alloc = BumpAlloc,
	...

Bump: no chunk header at all

The bump.c header comment states the design goal:

Bump is a MemoryContext implementation designed for memory usages which require allocating a large number of chunks, none of which ever need to be pfree’d or realloc’d. Chunks allocated by this context have no chunk header and operations which ordinarily require looking at the chunk header cannot be performed. For example, pfree, realloc, GetMemoryChunkSpace and GetMemoryChunkContext are all not possible with bump allocated chunks. The only way to release memory allocated by this context type is to reset or delete the context.

And the size of the header is literally zero:

 c
 
 
/* No chunk header unless built with MEMORY_CONTEXT_CHECKING */
#ifdef MEMORY_CONTEXT_CHECKING
#define Bump_CHUNKHDRSZ	sizeof(MemoryChunk)
#else
#define Bump_CHUNKHDRSZ	0
#endif

No 8-byte header per allocation. The allocation is just a bump of a pointer:

 c
 
 
	/* point the freeptr beyond this chunk */
	block->freeptr += (Bump_CHUNKHDRSZ + chunk_size);
	Assert(block->freeptr <= block->endptr);

The cost is that the operations that need the header are refused outright:

 c
 
 
/*
 * BumpFree
 *		Unsupported.
 */
void
BumpFree(void *pointer)
{
	elog(ERROR, "%s is not supported by the bump memory allocator", "pfree");
}

Same for BumpRealloc, BumpGetChunkContext and BumpGetChunkSpace. The MEMORY_CONTEXT_CHECKING build adds the header back precisely so that these mistakes are caught rather than silently corrupting memory.

Where is Bump used? Wherever there are “a large number of short-lived chunks where performance matters” — hash aggregation, tuplesort, set operations, nodeSubplan, the TID store. The header comment explains the shape of the win:

Because bump allocated chunks don’t have a chunk header, it can fit more chunks on each block. This means we can do more with less memory and fewer cache lines.

Slab: one size, and the free list lives in the chunk

slab.c is for the opposite extreme: many objects of exactly the same size. The chunk size is fixed at context creation time:

 c
 
 
MemoryContext
SlabContextCreate(MemoryContext parent,
				 const char *name,
				 Size blockSize,
				 Size chunkSize)

and enforced at allocation time:

 c
 
 
	/*
	 * Make sure we only allow correct request size. This doubles as the
	 * MemoryContextCheckSize check.
	 */
	if (unlikely(size != slab->chunkSize))
		SlabAllocInvalidSize(context, size);

The free list is stored inside the free chunk, exactly as in aset.c — but with an important difference. In aset.c the chunk’s user data holds the link; in slab.c the link is written over the start of the chunk’s data area, and the chunk size is forced to be at least a pointer:

 c
 
 
	/*
	 * Ensure there's space to store the pointer to the next free chunk
	 * in the memory of the (otherwise) unused allocation.
	 */
	if (chunkSize < sizeof(MemoryChunk *))
		chunkSize = sizeof(MemoryChunk *);

 c
 
 
	/* push this chunk onto the head of the block's free list */
	*(MemoryChunk **) pointer = block->freehead;
	block->freehead = chunk;

The interesting policy in slab.c is “fullest block first”. The header comment explains the motivation:

Slab can also help reduce memory fragmentation in cases where longer-lived chunks remain stored on blocks while most of the other chunks have already been pfree’d. We give priority to putting new allocations into the “fullest” block. This help avoid having too many sparsely used blocks around and allows blocks to more easily become completely unused which allows them to be eventually free’d.

The implementation is a small array of lists, partitioned by how many free chunks a block has:

 c
 
 
/*
 * The number of partitions to divide the blocklist into based their number
 * of free chunks. There must be at least 2.
 */
#define SLAB_BLOCKLIST_COUNT 3

with a deliberately clever index computation:

 c
 
 
static inline int32
SlabBlocklistIndex(SlabContext *slab, int nfree)
{
	int32		index;
	int32		blocklist_shift = slab->blocklist_shift;

	Assert(nfree >= 0 && nfree <= slab->chunksPerBlock);

	/*
	 * Determine the blocklist index based on the number of free chunks. We
	 * must ensure that 0 free chunks is dedicated to index 0. Everything
	 * else must be >= 1 and < SLAB_BLOCKLIST_COUNT.
	 *
	 * To make this as efficient as possible, we exploit some two's complement
	 * arithmetic where we reverse the sign before bit shifting. This results
	 * in an nfree of 0 using index 0 and anything non-zero staying non-zero.
	 * This is exploiting 0 and -0 being the same in two's complement. When
	 * we're done, we just need to flip the sign back over again for a
	 * positive index.
	 */
	index = -((-nfree) >> blocklist_shift);

	if (nfree == 0)
		Assert(index == 0);
	else
		Assert(index >= 1 && index < SLAB_BLOCKLIST_COUNT);

	return index;
}

-((-nfree) >> shift) — for nfree == 0 this is -0 == 0, and for anything else it is a positive number. Two’s complement does the work.

The high-water-mark trick

slab.c has one more elegant trick. When a fresh block is allocated, all of its chunks are free — but writing out a free list for every chunk would mean touching the entire block at creation time. Instead, slab.c keeps a pointer to the next never-used chunk and a count of how many there are:

 c
 
 
typedef struct SlabBlock
{
	SlabContext *slab;			/* owning context */
	int32		nfree;			/* number of chunks on free + unused chunks */
	int32		nunused;		/* number of unused chunks */
	MemoryChunk *freehead;		/* pointer to the first free chunk */
	MemoryChunk *unused;		/* pointer to the next unused chunk */
	dlist_node	node;			/* doubly-linked list for blocklist[] */
} SlabBlock;

The header comment explains it:

When we allocate a new block, technically all chunks are free, however, to avoid having to write out the entire block to set the linked list for the free chunks for every chunk in the block, we instead store a pointer to the next “unused” chunk on the block and keep track of how many of these unused chunks there are. When a new block is malloc’d, all chunks are unused. The unused pointer starts with the first chunk on the block and as chunks are allocated, the unused pointer is incremented. As chunks are pfree’d, the unused pointer never goes backwards. The unused pointer can be thought of as a high watermark for the maximum number of chunks in the block which have been in use concurrently. When a chunk is pfree’d the chunk is put onto the head of the free list and the unused pointer is not changed. We only consume more unused chunks if we run out of free chunks on the free list. This method effectively gives priority to using previously used chunks over previously unused chunks, which should perform better due to CPU caching effects.

A high-water mark for concurrency. Freed chunks are reused before virgin space is touched, so the hot parts of the block stay hot in cache.

Generation: for FIFO lifetimes

The fourth allocator, generation.c, is for the case where chunks are freed roughly in the order they were allocated:

This memory context is based on the assumption that the chunks are freed roughly in the same order as they were allocated (FIFO), or in groups with similar lifespan (generations - hence the name of the context).

Instead of a global free list, each block tracks allocated and freed chunks, and an empty block is recycled for future allocations before it is handed back to malloc().

The temporary context

There is no src/backend/utils/mmgr/temp.c in this tree — the four allocator types are aset.c, slab.c, generation.c and bump.c, and the file listing in src/backend/utils/mmgr/ confirms it. The “temporary context” of PostgreSQL’s design is instead a policy, described in the README:

When creating a prepared statement, the parse and plan trees will be built in a temporary context that’s a child of MessageContext (so that it will go away automatically upon error). On success, the finished plan is copied to the prepared statement’s private context, and the temp context is released; this allows planner temporary space to be recovered before execution begins.

The word “temporary” here describes the lifetime, not the allocator type. It is an ordinary AllocSetContext created with AllocSetContextCreate() and destroyed with MemoryContextDelete() once the plan has been copied out. What makes it “temporary” is that it is a child of MessageContext and is destroyed — not reset — once the planner is finished with it.

The executor’s per-tuple contexts work the same way, and they are where the context recycling machinery pays for itself most visibly.

Caveats

Memory is not returned to the OS promptly. pfree() on a chunk smaller than 8 kB does not call free(); it pushes the chunk onto a free list and moves on. The memory stays in the process. Only a context reset or delete, or the release of a dedicated large block, actually hands memory back — and even then, whether the OS reclaims it is malloc’s business, not PostgreSQL’s. A backend that allocated 1 GB of short-lived chunks will keep that 1 GB resident until the contexts holding it are reset.

Fragmentation is real, but bounded. Within a context, the free lists keep chunks segregated by size class, so a freed 100-byte chunk cannot satisfy a 1000-byte request. That is deliberate (recyclability over density), but it means a context whose allocation size distribution changes over time will accumulate idle chunks in the wrong classes.

Reset is not delete. MemoryContextReset() keeps the context object and its keeper block; MemoryContextDelete() frees everything and (for an AllocSet with poolable parameters) puts the context object itself on the recycling pool. Choosing reset where you meant delete leaves the context alive; choosing delete where you meant reset churns the pool.

pfree on a bump chunk is an error, not a no-op. The bump allocator has no chunk header, so it cannot find the freelist to return the chunk to. BumpFree() raises elog(ERROR, "pfree is not supported by the bump memory allocator"). If you are using a bump context, you must let the whole context go.

repalloc on a slab chunk is only allowed at the same size. A slab context allocates one fixed size, and SlabRealloc() errors out for any other size:

 c
 
 
	/* can't do actual realloc with slab, but let's try to be gentle */
	if (size == slab->chunkSize)
		return pointer;

	elog(ERROR, "slab allocator does not support realloc()");

The 8-byte header is real overhead. For an AllocSet, every allocation pays 8 bytes of header plus up to 2× quantisation waste. That is why Slab, Bump and Generation exist, and why MemoryContextAllocAligned() has to allocate extra bytes to build a “redirection” chunk for the alignment.

What it all means in practice

palloc is a size-class allocator, not a heap. If you are porting C code that assumes malloc semantics — variable sizes, exact accounting, memory returned promptly — some of that will not hold.

The 8 kB threshold is a cliff, not a slope. Requests of 8192 bytes or less are quantised chunks in a shared block; requests of 8193 bytes or more get their own malloc(). That is why ALLOCSET_SEPARATE_THRESHOLD exists in memutils.h — “tuplesort/tuplestore need to know it” — so they can account for the difference.

repalloc is often free, and that is not a coincidence. The power-of-two quantisation already reserved the space. If you are writing code that grows a buffer, repalloc within the same size class costs nothing.

Free-list churn is the price of the free list. The pfree path reads the index out of the chunk header and pushes the chunk onto a list; it does not coalesce with neighbours and does not return memory. The payoff is that the next palloc of the same size class is a single array load and a pointer swap.

Contexts are the unit of memory lifetime, and recycling them is what makes per-tuple contexts viable. The difference between “allocate a per-tuple context with malloc/free” and “reuse one from the pool” is two system calls per tuple versus none.

Do not allocate in the wrong context. The CurrentMemoryContext pointer is the single biggest source of memory leaks in PostgreSQL extensions. The README is blunt about it: “Only in very circumscribed code should it ever point at a context having greater than transaction lifespan, since doing so risks permanent memory leaks.”

PostgreSQL Groups Commits

Jay Hubbert — Sat, 12 Sep 2026 09:00:00 +0000

Every database has a moment where a transaction becomes durable, and every database has to decide what that moment costs. PostgreSQL’s answer is unusual: committing a transaction does not mean fsyncing anything.

What happens on COMMIT is that a WAL record is copied into shared memory. Somebody — maybe you, maybe another backend, maybe the background WAL writer — eventually writes that memory to the operating system and asks the OS to flush it to stable storage. If you are lucky, other backends were committing at the same time, and their records went out in the same write() and the same fsync(). If you are unlucky, you paid for the whole fsync() yourself.

The machinery that makes the lucky case happen is small and strange: LWLockAcquireOrWait(), an LWLock acquisition function whose contract is that it may return “false” without giving you the lock. Followers wake up when the lock is released, discover that someone else already flushed their record, and return without ever taking the lock themselves.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

Three frontiers

Everything in the WAL machinery is measured by an LSN — a byte position in the WAL stream. And there are three of them, not one, because “the record is in memory”, “the record is in the OS’s buffer cache”, and “the record is on the platter” are three different states with three different costs.

They live in XLogCtlData (src/backend/access/transam/xlog.c):

 c
 
 
	/* These are accessed using atomics -- info_lck not needed */
	pg_atomic_uint64 logInsertResult;	/* last byte + 1 inserted to buffers */
	pg_atomic_uint64 logWriteResult;	/* last byte + 1 written out */
	pg_atomic_uint64 logFlushResult;	/* last byte + 1 flushed */

Frontier	Meaning	Who advances it
`logInsertResult`	WAL up to here has been copied into the shared WAL buffers	the inserting backends
`logWriteResult`	WAL up to here has been `write()`n to the OS	whoever holds `WALWriteLock`
`logFlushResult`	WAL up to here has been `fsync()`ed	whoever holds `WALWriteLock`

The invariant is a strict ordering, and the code asserts it explicitly:

 c
 
 
		/* WAL written to disk is always ahead of WAL flushed */
		Assert(Write >= Flush);

		/* WAL inserted to buffers is always ahead of WAL written */
		Assert(Insert >= Write);

So Insert >= Write >= Flush, always. A record can be in memory but not written; it can be written but not flushed. It can never be flushed before it is written, and it can never be written before it is in memory.

The comment block above the shared structures explains the design:

 c
 
 
/*----------
 * Shared-memory data structures for XLOG control
 *
 * LogwrtRqst indicates a byte position that we need to write and/or fsync
 * the log up to (all records before that point must be written or fsynced).
 * The positions already written/fsynced are maintained in logWriteResult
 * and logFlushResult using atomic access.
 * In addition to the shared variable, each backend has a private copy of
 * both in LogwrtResult, which is updated when convenient.
 *
 * The request bookkeeping is simpler: there is a shared XLogCtl->LogwrtRqst
 * (protected by info_lck), but we don't need to cache any copies of it.
 *
 * info_lck is only held long enough to read/update the protected variables,
 * so it's a plain spinlock. The other locks are held longer (potentially
 * over I/O operations), so we use LWLocks for them. These locks are:
 *
 * WALBufMappingLock: must be held to replace a page in the WAL buffer cache.
 * It is only held while initializing and changing the mapping. If the
 * contents of the buffer being replaced haven't been written yet, the mapping
 * lock is released while the write is done, and reacquired afterwards.
 *
 * WALWriteLock: must be held to write WAL buffers to disk (XLogWrite or
 * XLogFlush).
 *
 * ControlFileLock: must be held to read/update control file or create
 * new log file.
 *
 *----------
 */

Two different kinds of lock, for two different reasons. info_lck is a spinlock held for nanoseconds; WALWriteLock is an LWLock held across write() and fsync() calls, and therefore across disk latency.

The local copy

Reading an atomic from shared memory on every check would be wasteful, so each backend keeps a private copy:

 c
 
 
/*
 * Private, possibly out-of-date copy of shared LogwrtResult.
 * See discussion above.
 */
static XLogwrtResult LogwrtResult = {0, 0};

and refreshes it with a macro:

 c
 
 
/*
 * Update local copy of shared XLogCtl->log{Write,Flush}Result
 *
 * It's critical that Flush always trails Write, so the order of the reads is
 * important, as is the barrier. See also XLogWrite.
 */
#define RefreshXLogWriteResult(_target) \
	do { \
		_target.Flush = pg_atomic_read_u64(&XLogCtl->logFlushResult); \
		pg_read_barrier(); \
		_target.Write = pg_atomic_read_u64(&XLogCtl->logWriteResult); \
	} while (0)

The barrier in the middle is not decoration. The writer publishes Write before Flush, with a write barrier between them:

 c
 
 
	/*
	 * We write Write first, bar, then Flush. When reading, the opposite must
	 * be done (with a matching barrier in between), so that we always see a
	 * Flush value that trails behind the Write value seen.
	 */
	pg_atomic_write_u64(&XLogCtl->logWriteResult, LogwrtResult.Write);
	pg_write_barrier();
	pg_atomic_write_u64(&XLogCtl->logFlushResult, LogwrtResult.Flush);

Read Flush first, then barrier, then read Write. The barrier guarantees that if you see a Write value, you also see the Flush value that was published before it. If you skipped the barrier you could read a newer Write and an older Flush, and conclude that more WAL had been written than had been flushed — a state that does not exist, and a bug waiting to happen.

Note also that the local copy is explicitly allowed to be stale. It is a cache, and code that needs an authoritative value refreshes it first.

Inserting a record: two phases

XLogInsertRecord() is the entry point for putting a record into the WAL. Its header comment sets out the contract:

 c
 
 
/*
 * Insert an XLOG record represented by an already-constructed chain of data
 * chunks. This is a low-level routine; to construct the WAL record header
 * and data, use the higher-level routines in xloginsert.c.
 ...
 * The first XLogRecData in the chain must be for the record header, and its
 * data must be MAXALIGNed. XLogInsertRecord fills in the xl_prev and
 * xl_crc fields in the header, the rest of the header must already be filled
 * by the caller.
 *
 * Returns XLOG pointer to end of record (beginning of next record).
 * This can be used as LSN for data pages affected by the logged action.
 * (LSN is the XLOG point up to which the XLOG must be flushed to disk
 * before the data page can be written out. This implements the basic
 * WAL rule "write the log before the data".)
 */

The body explains the two-phase structure:

 c
 
 
	/*----------
	 *
	 * We have now done all the preparatory work we can without holding a
	 * lock or modifying shared state. From here on, inserting the new WAL
	 * record to the shared WAL buffer cache is a two-step process:
	 *
	 * 1. Reserve the right amount of space from the WAL. The current head of
	 *	 reserved space is kept in Insert->CurrBytePos, and is protected by
	 *	 insertpos_lck.
	 *
	 * 2. Copy the record to the reserved WAL space. This involves finding the
	 *	 correct WAL buffer containing the reserved space, and copying the
	 *	 record in place. This can be done concurrently in multiple processes.
	 *
	 * To keep track of which insertions are still in-progress, each concurrent
	 * inserter acquires an insertion lock. In addition to just indicating that
	 * an insertion is in progress, the lock tells others how far the inserter
	 * has progressed. There is a small fixed number of insertion locks,
	 * determined by NUM_XLOGINSERT_LOCKS. When an inserter crosses a page
	 * boundary, it updates the value stored in the lock to the how far it has
	 * inserted, to allow the previous buffer to be flushed.
	 *
	 * Holding onto an insertion lock also protects RedoRecPtr and
	 * fullPageWrites from changing until the insertion is finished.
	 *
	 * Step 2 can usually be done completely in parallel. If the required WAL
	 * page is not initialized yet, you have to grab WALBufMappingLock to
	 * initialize it, but the WAL writer tries to do that ahead of insertions
	 * to avoid that from happening in the critical path.
	 *
	 *----------
	 */

Phase 1 (reserve) is serialized across all backends. Phase 2 (copy) is parallel. That split is the whole design: the expensive contended part is reduced to a pointer bump, and the bulky part — copying the record bytes — happens concurrently.

Phase 1: reserving space

The reservation is the only place where all backends serialise on a spinlock. Here it is, in full:

 c
 
 
/*
 * Reserves the right amount of space for a record of given size from the WAL.
 * *StartPos is set to the beginning of the reserved section, *EndPos to
 * its end+1. *PrevPtr is set to the beginning of the previous record; it is
 * used to set the xl_prev of this record.
 *
 * This is the performance critical part of XLogInsert that must be serialized
 * across backends. The rest can happen mostly in parallel. Try to keep this
 * section as short as possible, insertpos_lck can be heavily contended on a
 * busy system.
 *
 * NB: The space calculation here must match the code in CopyXLogRecordToWAL,
 * where we actually copy the record to the reserved space.
 *
 * NB: Testing shows that XLogInsertRecord runs faster if this code is inlined;
 * however, because there are two call sites, the compiler is reluctant to
 * inline. We use pg_always_inline here to try to convince it.
 */
static pg_always_inline void
ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos,
						 XLogRecPtr *PrevPtr)
{
	XLogCtlInsert *Insert = &XLogCtl->Insert;
	uint64		startbytepos;
	uint64		endbytepos;
	uint64		prevbytepos;

	size = MAXALIGN(size);

	/* All (non xlog-switch) records should contain data. */
	Assert(size > SizeOfXLogRecord);

	/*
	 * The duration the spinlock needs to be held is minimized by minimizing
	 * the calculations that have to be done while holding the lock. The
	 * current tip of reserved WAL is kept in CurrBytePos, as a byte position
	 * that only counts "usable" bytes in WAL, that is, it excludes all WAL
	 * page headers. The mapping between "usable" byte positions and physical
	 * positions (XLogRecPtrs) can be done outside the locked region, and
	 * because the usable byte position doesn't include any headers, reserving
	 * X bytes from WAL is almost as simple as "CurrBytePos += X".
	 */
	SpinLockAcquire(&Insert->insertpos_lck);

	startbytepos = Insert->CurrBytePos;
	endbytepos = startbytepos + size;
	prevbytepos = Insert->PrevBytePos;
	Insert->CurrBytePos = endbytepos;
	Insert->PrevBytePos = startbytepos;

	SpinLockRelease(&Insert->insertpos_lck);

	*StartPos = XLogBytePosToRecPtr(startbytepos);
	*EndPos = XLogBytePosToEndRecPtr(endbytepos);
	*PrevPtr = XLogBytePosToRecPtr(prevbytepos);

	/*
	 * Check that the conversions between "usable byte positions" and
	 * XLogRecPtrs work consistently in both directions.
	 */
	Assert(XLogRecPtrToBytePos(*StartPos) == startbytepos);
	Assert(XLogRecPtrToBytePos(*EndPos) == endbytepos);
	Assert(XLogRecPtrToBytePos(*PrevPtr) == prevbytepos);
}

The key trick is the “usable byte position”.

An XLogRecPtr is a physical byte position in the WAL file, and it has to account for the WAL page headers — a 24-byte long header at the start of each segment, and a 24-byte short header at the start of every other page. The mapping between “bytes of actual WAL data” and “bytes of file offset” is therefore not a simple addition: it involves a division by the usable bytes per page, a modulo, and a branch on whether you are in the first page of a segment.

Doing that arithmetic while holding a spinlock that every backend on the system contends for would be terrible. So the shared cursor does not count file bytes. It counts usable bytes — bytes of WAL data, with all page headers excluded. Reserving space is then literally:

 c
 
	endbytepos = startbytepos + size;

Three additions and two assignments. The expensive conversion happens after the lock is released.

The conversion function itself:

 c
 
 
/*
 * Converts a "usable byte position" to XLogRecPtr. A usable byte position
 * is the position starting from the beginning of WAL, excluding all WAL
 * page headers.
 */
static XLogRecPtr
XLogBytePosToRecPtr(uint64 bytepos)
{
	uint64		fullsegs;
	uint64		fullpages;
	uint64		bytesleft;
	uint32		seg_offset;
	XLogRecPtr	result;

	fullsegs = bytepos / UsableBytesInSegment;
	bytesleft = bytepos % UsableBytesInSegment;

	if (bytesleft < XLOG_BLCKSZ - SizeOfXLogLongPHD)
	{
		/* fits on first page of segment */
		seg_offset = bytesleft + SizeOfXLogLongPHD;
	}
	else
	{
		/* account for the first page on segment with long header */
		seg_offset = XLOG_BLCKSZ;
		bytesleft -= XLOG_BLCKSZ - SizeOfXLogLongPHD;

		fullpages = bytesleft / UsableBytesInPage;
		bytesleft = bytesleft % UsableBytesInPage;

		seg_offset += fullpages * XLOG_BLCKSZ + bytesleft + SizeOfXLogShortPHD;
	}

	XLogSegNoOffsetToRecPtr(fullsegs, seg_offset, wal_segment_size, result);

	return result;
}

Two divisions and two modulos, done outside the lock. There is also a XLogBytePosToEndRecPtr() variant which differs in one respect: at a page boundary it returns the position before the page header, because it is used to compute the end of a record rather than the start of one.

The shared cursor itself:

 c
 
 
typedef struct XLogCtlInsert
{
	slock_t		insertpos_lck;	/* protects CurrBytePos and PrevBytePos */

	/*
	 * CurrBytePos is the end of reserved WAL. The next record will be
	 * inserted at that position. PrevBytePos is the start position of the
	 * previously inserted (or rather, reserved) record - it is copied to the
	 * prev-link of the next record. These are stored as "usable byte
	 * positions" rather than XLogRecPtrs (see XLogBytePosToRecPtr()).
	 */
	uint64		CurrBytePos;
	uint64		PrevBytePos;

	/*
	 * Make sure the above heavily-contended spinlock and byte positions are
	 * on their own cache line. In particular, the RedoRecPtr and full page
	 * write variables below should be on a different cache line. They are
	 * read on every WAL insertion, but updated rarely, and we don't want
	 * those reads to steal the cache line containing Curr/PrevBytePos.
	 */
	char		pad[PG_CACHE_LINE_SIZE];

The padding is 128 bytes:

#define PG_CACHE_LINE_SIZE		128

The comment explains the reasoning precisely: RedoRecPtr and fullPageWrites are read on every WAL insertion, but changed only at checkpoints. If they shared a cache line with CurrBytePos, every insertion would be reading a line that a checkpoint might be invalidating. The 128-byte pad guarantees that the hot, mutable cursor and the cold, rarely-mutated metadata sit on different lines.

Note also PrevBytePos. It is the start of the previously reserved record, and it is what gets copied into the new record’s xl_prev field — the backward link that lets WAL readers walk the stream in both directions.

Phase 2: the CRC

Back in XLogInsertRecord(), now that xl_prev is known:

 c
 
 
	if (inserted)
	{
		/*
		 * Now that xl_prev has been filled in, calculate CRC of the record
		 * header.
		 */
		rdata_crc = rechdr->xl_crc;
		COMP_CRC32C(rdata_crc, rechdr, offsetof(XLogRecord, xl_crc));
		FIN_CRC32C(rdata_crc);
		rechdr->xl_crc = rdata_crc;

The CRC is computed over the record header up to but not including the xl_crc field itself (which is why it is offsetof(XLogRecord, xl_crc)), and the CRC of the record data was computed earlier by the caller, in xloginsert.c:

 c
 
 
	INIT_CRC32C(rdata_crc);
	COMP_CRC32C(rdata_crc, hdr_scratch + SizeOfXLogRecord, hdr_rdt.len - SizeOfXLogRecord);

The INIT_CRC32C/COMP_CRC32C/FIN_CRC32C macros dispatch to the best available hardware implementation — SSE4.2, ARMv8 CRC extension, or the software table-based one — via a function pointer chosen at startup:

 c
 
 
#define COMP_CRC32C(crc, data, len) \
	((crc) = pg_comp_crc32c_dispatch((crc), (data), (len)))
#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)

extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);

Phase 2: copying the record

 c
 
 
static void
CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata,
					XLogRecPtr StartPos, XLogRecPtr EndPos, TimeLineID tli)
{
	char	 *currpos;
	int			freespace;
	int			written;
	XLogRecPtr	CurrPos;
	XLogPageHeader pagehdr;

	/*
	 * Get a pointer to the right place in the right WAL buffer to start
	 * inserting to.
	 */
	CurrPos = StartPos;
	currpos = GetXLogBuffer(CurrPos, tli);
	freespace = INSERT_FREESPACE(CurrPos);

	/*
	 * there should be enough space for at least the first field (xl_tot_len)
	 * on this page.
	 */
	Assert(freespace >= sizeof(uint32));

	/* Copy record data */
	written = 0;
	while (rdata != NULL)
	{
		const char *rdata_data = rdata->data;
		int			rdata_len = rdata->len;

		while (rdata_len > freespace)
		{
			/*
			 * Write what fits on this page, and continue on the next page.
			 */
			Assert(CurrPos % XLOG_BLCKSZ >= SizeOfXLogShortPHD || freespace == 0);
			memcpy(currpos, rdata_data, freespace);
			rdata_data += freespace;
			rdata_len -= freespace;
			written += freespace;
			CurrPos += freespace;

			/*
			 * Get pointer to beginning of next page, and set the xlp_rem_len
			 * in the page header. Set XLP_FIRST_IS_CONTRECORD.
			 *
			 * It's safe to set the contrecord flag and xlp_rem_len without a
			 * lock on the page. All the other flags were already set when the
			 * page was initialized, in AdvanceXLInsertBuffer, and we're the
			 * only backend that needs to set the contrecord flag.
			 */
			currpos = GetXLogBuffer(CurrPos, tli);
			pagehdr = (XLogPageHeader) currpos;
			pagehdr->xlp_rem_len = write_len - written;
			pagehdr->xlp_info |= XLP_FIRST_IS_CONTRECORD;

			/* skip over the page header */
			if (XLogSegmentOffset(CurrPos, wal_segment_size) == 0)
			{
				CurrPos += SizeOfXLogLongPHD;
				currpos += SizeOfXLogLongPHD;
			}
			else
			{
				CurrPos += SizeOfXLogShortPHD;
				currpos += SizeOfXLogShortPHD;
			}
			freespace = INSERT_FREESPACE(CurrPos);
		}
		...

Continuation records are the interesting part. A WAL record may be larger than a page, or may straddle a page boundary for any other reason. When that happens, the record is split across pages, and each page after the first carries a header flag XLP_FIRST_IS_CONTRECORD and a field xlp_rem_len saying how much of the record is still to come. The WAL reader uses xlp_rem_len to reassemble the record.

And this is all done without holding any page lock, because the inserting backend owns that region of the WAL exclusively — it reserved it in phase 1, and nobody else writes there.

Getting the buffer

GetXLogBuffer() maps an LSN to a pointer into the WAL buffer cache. The mapping is arithmetic, not a lookup table:

 c
 
 
/*
 * XLogRecPtrToBufIdx returns the index of the WAL buffer that holds, or
 * would hold if it was in cache, the page containing 'recptr'.
 */
#define XLogRecPtrToBufIdx(recptr)	\
	(((recptr) / XLOG_BLCKSZ) % (XLogCtl->XLogCacheBlck + 1))

The WAL buffer cache is a ring: page number modulo the number of buffers. Consecutive WAL pages therefore land in consecutive buffer slots, wrapping around at the end. That is what makes the single big write() possible later on.

The WAL insertion locks

Eight locks, in shared memory, padded so they do not share cache lines:

 c
 
 
/*
 * Number of WAL insertion locks to use. A higher value allows more insertions
 * to happen concurrently, but adds some CPU overhead to flushing the WAL,
 * which needs to iterate all the locks.
 */
#define NUM_XLOGINSERT_LOCKS 8

 c
 
 
/*
 * All the WAL insertion locks are allocated as an array in shared memory. We
 * force the array stride to be a power of 2, which saves a few cycles in
 * indexing, but more importantly also ensures that individual slots don't
 * cross cache line boundaries. (Of course, we have to also ensure that the
 * array start address is suitably aligned.)
 */
typedef union WALInsertLockPadded
{
	WALInsertLock l;
	char		pad[PG_CACHE_LINE_SIZE];
} WALInsertLockPadded;

And the lock itself:

 c
 
 
typedef struct
{
	LWLock		lock;
	pg_atomic_uint64 insertingAt;
	XLogRecPtr	lastImportantAt;
} WALInsertLock;

Three fields: the LWLock itself, the insertingAt progress indicator, and lastImportantAt (used to decide whether a checkpoint needs to do anything).

The design rationale for the whole scheme is spelled out at length:

 c
 
 
/*
 * Inserting to WAL is protected by a small fixed number of WAL insertion
 * locks. To insert to the WAL, you must hold one of the locks - it doesn't
 * matter which one. To lock out other concurrent insertions, you must hold
 * of them. Each WAL insertion lock consists of a lightweight lock, plus an
 * indicator of how far the insertion has progressed (insertingAt).
 *
 * The insertingAt values are read when a process wants to flush WAL from
 * the in-memory buffers to disk, to check that all the insertions to the
 * region the process is about to write out have finished. You could simply
 * wait for all currently in-progress insertions to finish, but the
 * insertingAt indicator allows you to ignore insertions to later in the WAL,
 * so that you only wait for the insertions that are modifying the buffers
 * you're about to write out.
 *
 * This isn't just an optimization. If all the WAL buffers are dirty, an
 * inserter that's holding a WAL insert lock might need to evict an old WAL
 * buffer, which requires flushing the WAL. If it's possible for an inserter
 * to block on another inserter unnecessarily, deadlock can arise when two
 * inserters holding a WAL insert lock wait for each other to finish their
 * insertion.
 *
 * Small WAL records that don't cross a page boundary never update the value,
 * the WAL record is just copied to the page and the lock is released. But
 * to avoid the deadlock-scenario explained above, the indicator is always
 * updated before sleeping while holding an insertion lock.
 *
 * lastImportantAt contains the LSN of the last important WAL record inserted
 * using a given lock. This value is used to detect if there has been
 * important WAL activity since the last time some action, like a checkpoint,
 * was performed - allowing to not repeat the action if not. The LSN is
 * updated for all insertions, unless the XLOG_MARK_UNIMPORTANT flag was
 * set. lastImportantAt is never cleared, only overwritten by the LSN of newer
 * records. Tracking the WAL activity directly in WALInsertLock has the
 * advantage of not needing any additional locks to update the value.
 */

That deadlock scenario is worth dwelling on. If the WAL writer had to wait for all in-progress insertions to finish, and an inserter needs to evict a dirty WAL buffer (which requires flushing, which requires the write lock), then inserter A could be waiting for the write lock while the writer waits for inserter A. The insertingAt protocol breaks the cycle: the writer only waits for insertions that are actually modifying the buffers it is about to write out, and an inserter that is blocked publishes its position before blocking.

Sticky affinity

 c
 
 
static void
WALInsertLockAcquire(void)
{
	bool		immed;

	/*
	 * It doesn't matter which of the WAL insertion locks we acquire, so try
	 * the one we used last time. If the system isn't particularly busy, it's
	 * a good bet that it's still available, and it's good to have some
	 * affinity to a particular lock so that you don't unnecessarily bounce
	 * cache lines between processes when there's no contention.
	 *
	 * If this is the first time through in this backend, pick a lock
	 * (semi-)randomly. This allows the locks to be used evenly if you have a
	 * lot of very short connections.
	 */
	static int	lockToTry = -1;

	if (lockToTry == -1)
		lockToTry = MyProcNumber % NUM_XLOGINSERT_LOCKS;
	MyLockNo = lockToTry;

	/*
	 * The insertingAt value is initially set to 0, as we don't know our
	 * insert location yet.
	 */
	immed = LWLockAcquire(&WALInsertLocks[MyLockNo].l.lock, LW_EXCLUSIVE);
	if (!immed)
	{
		/*
		 * If we couldn't get the lock immediately, try another lock next
		 * time. On a system with more insertion locks than concurrent
		 * inserters, this causes all the inserters to eventually migrate to a
		 * lock that no-one else is using. On a system with more inserters
		 * than locks, it still helps to distribute the inserters evenly
		 * across the locks.
		 */
		lockToTry = (lockToTry + 1) % NUM_XLOGINSERT_LOCKS;
	}
}

Two behaviours in ten lines. Affinity: try the same lock you used last time, because then the lock’s cache line is likely still in your cache. Rotation: if you had to wait for the lock, move to the next one next time. The comment explains why rotation is better than contention: with more locks than inserters, everyone migrates to a private lock; with more inserters than locks, at least the contention is spread evenly.

Note that the rotation only happens on failure to acquire immediately. A backend that always gets the lock immediately never rotates and keeps its affinity forever. That is the point.

The insertingAt protocol

insertingAt is an atomic LSN. It is set to the position the inserter has reached, and it is reset to 0 when the insertion lock is released:

 c
 
 
/*
 * Release our insertion lock (or locks, if we're holding them all).
 *
 * NB: Reset all variables to 0, so they cause LWLockWaitForVar to block the
 * next time the lock is acquired.
 */
static void
WALInsertLockRelease(void)
{
	if (holdingAllLocks)
	{
		int			i;

		for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
			LWLockReleaseClearVar(&WALInsertLocks[i].l.lock,
								 &WALInsertLocks[i].l.insertingAt,
								 0);

		holdingAllLocks = false;
	}
	else
	{
		LWLockReleaseClearVar(&WALInsertLocks[MyLockNo].l.lock,
							 &WALInsertLocks[MyLockNo].l.insertingAt,
							 0);
	}
}

And it is updated when the inserter crosses a page boundary:

 c
 
 
/*
 * Update our insertingAt value, to let others know that we've finished
 * inserting up to that point.
 */
static void
WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt)
{
	if (holdingAllLocks)
	{
		/*
		 * We use the last lock to mark our actual position, see comments in
		 * WALInsertLockAcquireExclusive.
		 */
		LWLockUpdateVar(&WALInsertLocks[NUM_XLOGINSERT_LOCKS - 1].l.lock,
						&WALInsertLocks[NUM_XLOGINSERT_LOCKS - 1].l.insertingAt,
						insertingAt);
	}
	else
		LWLockUpdateVar(&WALInsertLocks[MyLockNo].l.lock,
						&WALInsertLocks[MyLockNo].l.insertingAt,
						insertingAt);
}

The “all locks” case is for XLOG_SWITCH and checkpoints, which must exclude every other inserter:

 c
 
 
static void
WALInsertLockAcquireExclusive(void)
{
	int			i;

	/*
	 * When holding all the locks, all but the last lock's insertingAt
	 * indicator is set to 0xFFFFFFFFFFFFFFFF, which is higher than any real
	 * XLogRecPtr value, to make sure that no-one blocks waiting on those.
	 */
	for (i = 0; i < NUM_XLOGINSERT_LOCKS - 1; i++)
	{
		LWLockAcquire(&WALInsertLocks[i].l.lock, LW_EXCLUSIVE);
		LWLockUpdateVar(&WALInsertLocks[i].l.lock,
						&WALInsertLocks[i].l.insertingAt,
						PG_UINT64_MAX);
	}
	/* Variable value reset to 0 at release */
	LWLockAcquire(&WALInsertLocks[i].l.lock, LW_EXCLUSIVE);

	holdingAllLocks = true;
}

PG_UINT64_MAX is “further than any real LSN”, so a waiter sees an insertion position beyond its target and stops waiting.

The interlock: WaitXLogInsertionsToFinish

Before the writer can write out a range of WAL, it must be sure that no backend is still memcpy()ing into that range. This is what WaitXLogInsertionsToFinish() does:

 c
 
 
/*
 * Wait for any WAL insertions < upto to finish.
 *
 * Returns the location of the oldest insertion that is still in-progress.
 * Any WAL prior to that point has been fully copied into WAL buffers, and
 * can be flushed out to disk. Because this waits for any insertions older
 * than 'upto' to finish, the return value is always >= 'upto'.
 *
 * Note: When you are about to write out WAL, you must call this function
 * *before* acquiring WALWriteLock, to avoid deadlocks. This function might
 * need to wait for an insertion to finish (or at least advance to next
 * uninitialized page), and the inserter might need to evict an old WAL buffer
 * to make room for a new one, which in turn requires WALWriteLock.
 */
static XLogRecPtr
WaitXLogInsertionsToFinish(XLogRecPtr upto)

The “call this before acquiring WALWriteLock” note is the deadlock avoidance again: the inserter may need the write lock to evict a buffer, so the writer must not be holding it while waiting.

The body:

 c
 
 
	/*
	 * Check if there's any work to do. Use a barrier to ensure we get the
	 * freshest value.
	 */
	inserted = pg_atomic_read_membarrier_u64(&XLogCtl->logInsertResult);
	if (upto <= inserted)
		return inserted;

	/* Read the current insert position */
	SpinLockAcquire(&Insert->insertpos_lck);
	bytepos = Insert->CurrBytePos;
	SpinLockRelease(&Insert->insertpos_lck);
	reservedUpto = XLogBytePosToEndRecPtr(bytepos);

Then the loop over the insertion locks:

 c
 
 
	finishedUpto = reservedUpto;
	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
	{
		XLogRecPtr	insertingat = InvalidXLogRecPtr;

		do
		{
			/*
			 * See if this insertion is in progress. LWLockWaitForVar will
			 * wait for the lock to be released, or for the 'value' to be set
			 * by a LWLockUpdateVar call. When a lock is initially acquired,
			 * its value is 0 (InvalidXLogRecPtr), which means that we don't
			 * know where it's inserting yet. We will have to wait for it. If
			 * it's a small insertion, the record will most likely fit on the
			 * same page and the inserter will release the lock without ever
			 * calling LWLockUpdateVar. But if it has to sleep, it will
			 * advertise the insertion point with LWLockUpdateVar before
			 * sleeping.
			 *
			 * In this loop we are only waiting for insertions that started
			 * before WaitXLogInsertionsToFinish was called. The lack of
			 * memory barriers in the loop means that we might see locks as
			 * "unused" that have since become used. This is fine because
			 * they only can be used for later insertions that we would not
			 * want to wait on anyway. Not taking a lock to acquire the
			 * current insertingAt value means that we might see older
			 * insertingAt values. This is also fine, because if we read a
			 * value too old, we will add ourselves to the wait queue, which
			 * contains atomic operations.
			 */
			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
								 &WALInsertLocks[i].l.insertingAt,
								 insertingat, &insertingat))
			{
				/* the lock was free, so no insertion in progress */
				insertingat = InvalidXLogRecPtr;
				break;
			}

			/*
			 * This insertion is still in progress. Have to wait, unless the
			 * inserter has proceeded past 'upto'.
			 */
		} while (insertingat < upto);

		if (XLogRecPtrIsValid(insertingat) && insertingat < finishedUpto)
			finishedUpto = insertingat;
	}

The rule: for each insertion lock, wait until either the lock is free (no insertion in progress) or the inserter’s published position is beyond our target. The result is the oldest still-in-progress insertion position, which is the point up to which WAL is safe to write out.

And the final step, which is a neat atomic trick:

 c
 
 
	/*
	 * Advance the limit we know to have been inserted and return the freshest
	 * value we know of, which might be beyond what we requested if somebody
	 * is concurrently doing this with an 'upto' pointer ahead of us.
	 */
	finishedUpto = pg_atomic_monotonic_advance_u64(&XLogCtl->logInsertResult,
												 finishedUpto);

	return finishedUpto;

pg_atomic_monotonic_advance_u64() moves the shared logInsertResult forward, but only forwards:

 c
 
 
pg_atomic_monotonic_advance_u64(volatile pg_atomic_uint64 *ptr, uint64 target)
{
	uint64		currval;

	...
	currval = pg_atomic_read_u64_impl(ptr);
	if (currval >= target)
	{
		pg_memory_barrier();
		return currval;
	}

	while (currval < target)
	{
		if (pg_atomic_compare_exchange_u64(ptr, &currval, target))
			return target;
	}

	return currval;
}

Two concurrent flushers can both be waiting on insertions; whichever one learns about a later position first publishes it, and the other one gets the fresher value back rather than its own older one. That is why the docstring says the return value “might be beyond what we requested if somebody is concurrently doing this with an ‘upto’ pointer ahead of us”.

Writing: one big pwrite()

XLogWrite() is called with WALWriteLock held. Its job is to move bytes from the WAL buffer ring into the WAL file.

 c
 
 
/*
 * Write and/or fsync the log at least as far as WriteRqst indicates.
 *
 * If flexible == true, we don't have to write as far as WriteRqst, but
 * may stop at any convenient boundary (such as a cache or logfile boundary).
 * This option allows us to avoid uselessly issuing multiple writes when a
 * single one would do.
 *
 * Must be called with WALWriteLock held. WaitXLogInsertionsToFinish(WriteRqst)
 * must be called before grabbing the lock, to make sure the data is ready to
 * write.
 */

The gathering logic:

 c
 
 
	/*
	 * Since successive pages in the xlog cache are consecutively allocated,
	 * we can usually gather multiple pages together and issue just one
	 * write() call. npages is the number of pages we have determined can be
	 * written together; startidx is the cache block index of the first one,
	 * and startoffset is the file offset at which it should go. The latter
	 * two variables are only valid when npages > 0, but we must initialize
	 * all of them to keep the compiler quiet.
	 */
	npages = 0;
	startidx = 0;
	startoffset = 0;

“Successive pages in the xlog cache are consecutively allocated” — because the buffer ring maps WAL page number modulo the number of buffers, and the buffers themselves are one contiguous allocation:

 c
 
 
	/* and the buffers themselves */
	size = add_size(size, mul_size(XLOG_BLCKSZ, XLOGbuffers));

 c
 
 
	/*
	 * Align the start of the page buffers to a full xlog block size boundary.
	 * This simplifies some calculations in XLOG insertion. It is also
	 * required for O_DIRECT.
	 */
	allocptr = (char *) TYPEALIGN(XLOG_BLCKSZ, allocptr);
	XLogCtl->pages = allocptr;
	memset(XLogCtl->pages, 0, (Size) XLOG_BLCKSZ * XLOGbuffers);

So consecutive WAL pages are contiguous in memory, and they are contiguous in the file too — unless the ring wraps, or the segment boundary is hit. Both of those cases break the run, and the code handles them:

 c
 
 
		/* Add current page to the set of pending pages-to-dump */
		if (npages == 0)
		{
			/* first of group */
			startidx = curridx;
			startoffset = XLogSegmentOffset(LogwrtResult.Write - XLOG_BLCKSZ,
											wal_segment_size);
		}
		npages++;

		/*
		 * Dump the set if this will be the last loop iteration, or if we are
		 * at the last page of the cache area (since the next page won't be
		 * contiguous in memory), or if we are at the end of the logfile
		 * segment.
		 */
		last_iteration = WriteRqst.Write <= LogwrtResult.Write;

		finishing_seg = !ispartialpage &&
			(startoffset + npages * XLOG_BLCKSZ) >= wal_segment_size;

		if (last_iteration ||
			curridx == XLogCtl->XLogCacheBlck ||
			finishing_seg)
		{
			char	 *from;
			Size		nbytes;
			Size		nleft;
			ssize_t		written;
			instr_time	start;

			/* OK to write the page(s) */
			from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
			nbytes = npages * (Size) XLOG_BLCKSZ;
			nleft = nbytes;
			do
			{
				errno = 0;

				/*
				 * Measure I/O timing to write WAL data, for pg_stat_io.
				 */
				start = pgstat_prepare_io_time(track_wal_io_timing);

				pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
				written = pg_pwrite(openLogFile, from, nleft, startoffset);
				pgstat_report_wait_end();
				...

One pg_pwrite() for the whole run of pages. The three ways a run ends: the write request is satisfied, we hit the end of the buffer ring (the next page is not contiguous in memory), or we hit the end of the WAL segment.

The segment boundary also forces an fsync():

 c
 
 
			/*
			 * If we just wrote the whole last page of a logfile segment,
			 * fsync the segment immediately. This avoids having to go back
			 * and re-open prior segments when an fsync request comes along
			 * later. Doing it here ensures that one and only one backend will
			 * perform this fsync.
			 *
			 * This is also the right place to notify the Archiver that the
			 * segment is ready to copy to archival storage, and to update the
			 * timer for archive_timeout, and to signal for a checkpoint if
			 * too many logfile segments have been used since the last
			 * checkpoint.
			 */
			if (finishing_seg)
			{
				issue_xlog_fsync(openLogFile, openLogSegNo, tli);

				/* signal that we need to wakeup walsenders later */
				WalSndWakeupRequest();

				LogwrtResult.Flush = LogwrtResult.Write;	/* end of page */

“Doing it here ensures that one and only one backend will perform this fsync” — the segment-boundary fsync is a fixed cost paid once per segment, not once per commit.

And the frontier publication, with the barrier discipline:

 c
 
 
	/*
	 * We write Write first, bar, then Flush. When reading, the opposite must
	 * be done (with a matching barrier in between), so that we always see a
	 * Flush value that trails behind the Write value seen.
	 */
	pg_atomic_write_u64(&XLogCtl->logWriteResult, LogwrtResult.Write);
	pg_write_barrier();
	pg_atomic_write_u64(&XLogCtl->logFlushResult, LogwrtResult.Flush);

Flushing: the group commit loop

Here is the heart of the thing. XLogFlush():

 c
 
 
/*
 * Ensure that all XLOG data through the given position is flushed to disk.
 *
 * NOTE: this differs from XLogWrite mainly in that the WALWriteLock is not
 * already held, and we try to avoid acquiring it if possible.
 */
void
XLogFlush(XLogRecPtr record)

The loop:

 c
 
 
	/*
	 * Since fsync is usually a horribly expensive operation, we try to
	 * piggyback as much data as we can on each fsync: if we see any more data
	 * entered into the xlog buffer, we'll write and fsync that too, so that
	 * the final value of LogwrtResult.Flush is as large as possible. This
	 * gives us some chance of avoiding another fsync immediately after.
	 */

	/* initialize to given target; may increase below */
	WriteRqstPtr = record;

	/*
	 * Now wait until we get the write lock, or someone else does the flush
	 * for us.
	 */
	for (;;)
	{
		XLogRecPtr	insertpos;

		/* done already? */
		RefreshXLogWriteResult(LogwrtResult);
		if (record <= LogwrtResult.Flush)
			break;

		/*
		 * Before actually performing the write, wait for all in-flight
		 * insertions to the pages we're about to write to finish.
		 */
		SpinLockAcquire(&XLogCtl->info_lck);
		if (WriteRqstPtr < XLogCtl->LogwrtRqst.Write)
			WriteRqstPtr = XLogCtl->LogwrtRqst.Write;
		SpinLockRelease(&XLogCtl->info_lck);
		insertpos = WaitXLogInsertionsToFinish(WriteRqstPtr);

		/*
		 * Try to get the write lock. If we can't get it immediately, wait
		 * until it's released, and recheck if we still need to do the flush
		 * or if the backend that held the lock did it for us already. This
		 * helps to maintain a good rate of group committing when the system
		 * is bottlenecked by the speed of fsyncing.
		 */
		if (!LWLockAcquireOrWait(WALWriteLock, LW_EXCLUSIVE))
		{
			/*
			 * The lock is now free, but we didn't acquire it yet. Before we
			 * do, loop back to check if someone else flushed the record for
			 * us already.
			 */
			continue;
		}

		/* Got the lock; recheck whether request is satisfied */
		RefreshXLogWriteResult(LogwrtResult);
		if (record <= LogwrtResult.Flush)
		{
			LWLockRelease(WALWriteLock);
			break;
		}

Three phases in each iteration:

Check whether the record is already flushed. If yes, done — no lock taken at all.
Wait for in-flight insertions to finish, and pick up any extra WAL that has been requested since (WriteRqstPtr is bumped to XLogCtl->LogwrtRqst.Write). This is the piggybacking: you flush everything, not just your own record.
Try to take the write lock, but with LWLockAcquireOrWait(). If somebody else holds it, you wait for them to finish and then re-check.

Then the actual write, and the commit_delay sleep:

 c
 
 
		/*
		 * Sleep before flush! By adding a delay here, we may give further
		 * backends the opportunity to join the backlog of group commit
		 * followers; this can significantly improve transaction throughput,
		 * at the risk of increasing transaction latency.
		 *
		 * We do not sleep if enableFsync is not turned on, nor if there are
		 * fewer than CommitSiblings other backends with active transactions.
		 */
		if (CommitDelay > 0 && enableFsync &&
			MinimumActiveBackends(CommitSiblings))
		{
			pgstat_report_wait_start(WAIT_EVENT_COMMIT_DELAY);
			pg_usleep(CommitDelay);
			pgstat_report_wait_end();
			...
		}

		/* try to write/flush later additions to XLOG as well */
		WriteRqst.Write = insertpos;
		WriteRqst.Flush = insertpos;

		XLogWrite(WriteRqst, insertTLI, false);

		LWLockRelease(WALWriteLock);
		/* done */
		break;
	}

commit_delay and commit_siblings are GUCs, defined in src/backend/utils/misc/guc_parameters.dat:

text

# we have no microseconds designation, so can't supply units here
{ name => 'commit_delay', type => 'int', context => 'PGC_SUSET', group => 'WAL_SETTINGS',
 short_desc => 'Sets the delay in microseconds between transaction commit and flushing WAL to disk.',
 variable => 'CommitDelay',
 boot_val => '0',
 min => '0',
 max => '100000',
},

{ name => 'commit_siblings', type => 'int', context => 'PGC_USERSET', group => 'WAL_SETTINGS',
 short_desc => 'Sets the minimum number of concurrent open transactions required before performing "commit_delay".',
 variable => 'CommitSiblings',
 boot_val => '5',
 min => '0',
 max => '1000',
},

The tradeoff they express is stated plainly in the comment: commit_delay trades latency for throughput. Sleeping before the fsync gives other backends time to arrive at the same WALWriteLock and become followers of the same fsync, so one fsync() covers more commits. But the sleeping backend’s own commit is now commit_delay microseconds slower.

commit_siblings is the guard that keeps this from being stupid: don’t sleep if you are the only one committing. MinimumActiveBackends() implements it, and it is deliberately racy:

 c
 
 
/*
 * MinimumActiveBackends --- count backends (other than myself) that are
 *		in active transactions. Return true if the count exceeds the
 *		minimum threshold passed. This is used as a heuristic to decide if
 *		a pre-XLOG-flush delay is worthwhile during commit.
 *
 * Do not count backends that are blocked waiting for locks, since they are
 * not going to get to run until someone else commits.
 */
bool
MinimumActiveBackends(int min)
{
	...
	/*
	 * Note: for speed, we don't acquire ProcArrayLock. This is a little bit
	 * bogus, but since we are only testing fields for zero or nonzero, it
	 * should be OK. The result is only used for heuristic purposes anyway...
	 */

“It is a little bit bogus” — the PostgreSQL source is refreshingly honest about its own heuristics. It is fine, because the decision is only about whether to sleep a few microseconds.

LWLockAcquireOrWait: the funky one

Now the function that makes all of this work (src/backend/storage/lmgr/lwlock.c):

 c
 
 
/*
 * LWLockAcquireOrWait - Acquire lock, or wait until it's free
 *
 * The semantics of this function are a bit funky. If the lock is currently
 * free, it is acquired in the given mode, and the function returns true. If
 * the lock isn't immediately free, the function waits until it is released
 * and returns false, but does not acquire the lock.
 *
 * This is currently used for WALWriteLock: when a backend flushes the WAL,
 * holding WALWriteLock, it can flush the commit records of many other
 * backends as a side-effect. Those other backends need to wait until the
 * flush finishes, but don't need to acquire the lock anymore. They can just
 * wake up, observe that their records have already been flushed, and return.
 */
bool
LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)

“The semantics of this function are a bit funky.” That sentence is the entire explanation of group commit, and it is in the PostgreSQL source.

The function:

 c
 
 
	mustwait = LWLockAttemptLock(lock, mode);

	if (mustwait)
	{
		LWLockQueueSelf(lock, LW_WAIT_UNTIL_FREE);

		mustwait = LWLockAttemptLock(lock, mode);

		if (mustwait)
		{
			/*
			 * Wait until awakened. Like in LWLockAcquire, be prepared for
			 * bogus wakeups.
			 */
			...
			for (;;)
			{
				PGSemaphoreLock(proc->sem);
				if (proc->lwWaiting == LW_WS_NOT_WAITING)
					break;
				extraWaits++;
			}
			...
		}
		else
		{
			LOG_LWDEBUG("LWLockAcquireOrWait", lock, "acquired, undoing queue");

			/*
			 * Got lock in the second attempt, undo queueing. We need to treat
			 * this as having successfully acquired the lock, otherwise we'd
			 * not necessarily wake up people we've prevented from acquiring
			 * the lock.
			 */
			LWLockDequeueSelf(lock);
		}
	}
	...
	if (mustwait)
	{
		/* Failed to get lock, so release interrupt holdoff */
		RESUME_INTERRUPTS();
		LOG_LWDEBUG("LWLockAcquireOrWait", lock, "failed");
		...
	}
	else
	{
		LOG_LWDEBUG("LWLockAcquireOrWait", lock, "succeeded");
		/* Add lock to list of locks held by this backend */
		held_lwlocks[num_held_lwlocks].lock = lock;
		held_lwlocks[num_held_lwlocks++].mode = mode;
		...
	}

	return !mustwait;
}

The flow:

Lock is free. Take it, return true. You are the flusher.
Lock is busy. Queue yourself, wait to be woken. When the holder releases it, you wake up. You do not take the lock. You return false.
Lock became free while you were queueing. The second LWLockAttemptLock() succeeds, you dequeue yourself, and return true — and the comment explains why you have to treat that as a successful acquisition:

Got lock in the second attempt, undo queueing. We need to treat this as having successfully acquired the lock, otherwise we’d not necessarily wake up people we’ve prevented from acquiring the lock.

If you dequeue yourself without holding the lock, you must still perform the “grant to next waiter” step, or the queue would stall.

Back in XLogFlush(), the follower path is the continue:

 c
 
 
		if (!LWLockAcquireOrWait(WALWriteLock, LW_EXCLUSIVE))
		{
			/*
			 * The lock is now free, but we didn't acquire it yet. Before we
			 * do, loop back to check if someone else flushed the record for
			 * us already.
			 */
			continue;
		}

Loop back to the top. RefreshXLogWriteResult(LogwrtResult) re-reads the flush frontier. If the record you needed is now flushed — because the backend that held the lock flushed it as a side-effect of flushing its own much-later record — you break out of the loop and return. You never took the lock. You never issued a syscall. You waited for someone else’s fsync() and rode along.

That is group commit. It is not a queue, it is not a batching layer, it is not a separate “group commit leader” process. It is one lock and a re-read of an atomic counter.

The commit path

Now the other end: what a transaction actually does when it commits (src/backend/access/transam/xact.c).

The commit critical section

 c
 
 
		/*
		 * Mark ourselves as within our "commit critical section". This
		 * forces any concurrent checkpoint to wait until we've updated
		 * pg_xact. Without this, it is possible for the checkpoint to set
		 * REDO after the XLOG record but fail to flush the pg_xact update to
		 * disk, leading to loss of the transaction commit if the system
		 * crashes a little later.
		 *
		 * Note: we could, but don't bother to, set this flag in
		 * RecordTransactionAbort. That's because loss of a transaction abort
		 * is noncritical; the presumption would be that it aborted, anyway.
		 *
		 * It's safe to change the delayChkptFlags flag of our own backend
		 * without holding the ProcArrayLock, since we're the only one
		 * modifying it. This makes checkpoint's determination of which xacts
		 * are delaying the checkpoint a bit fuzzy, but it doesn't matter.
		 *
		 * Note, it is important to get the commit timestamp after marking the
		 * transaction in the commit critical section. See
		 * RecordTransactionCommitPrepared.
		 */
		Assert((MyProc->delayChkptFlags & DELAY_CHKPT_IN_COMMIT) == 0);
		START_CRIT_SECTION();
		MyProc->delayChkptFlags |= DELAY_CHKPT_IN_COMMIT;

		Assert(xactStopTimestamp == 0);

		/*
		 * Ensures the DELAY_CHKPT_IN_COMMIT flag write is globally visible
		 * before commit time is written.
		 */
		pg_write_barrier();

The critical section has two jobs. It stops a checkpoint from advancing the redo point past a commit record whose pg_xact update has not yet reached disk — otherwise a crash could lose a committed transaction. And the write barrier makes sure the flag is globally visible before the commit timestamp is written, so a standby cannot see a commit timestamp for a transaction that is not yet in the commit critical section.

The flush decision

 c
 
 
	/*
	 * Check if we want to commit asynchronously. We can allow the XLOG flush
	 * to happen asynchronously if synchronous_commit=off, or if the current
	 * transaction has not performed any WAL-logged operation or didn't assign
	 * an xid. The transaction can end up not writing any WAL, even if it has
	 * an xid, if it only wrote to temporary and/or unlogged tables. It can
	 * end up having written WAL without an xid if it did HOT pruning. In
	 * case of a crash, the loss of such a transaction will be irrelevant;
	 * temp tables will be lost anyway, unlogged tables will be truncated and
	 * HOT pruning will be done again later. (Given the foregoing, you might
	 * think that it would be unnecessary to emit the XLOG record at all in
	 * this case, but we don't currently try to do that. It would certainly
	 * cause problems at least in Hot Standby mode, where the
	 * KnownAssignedXids machinery requires tracking every XID assignment. It
	 * might be OK to skip it only when wal_level < replica, but for now we
	 * don't.)
	 *
	 * However, if we're doing cleanup of any non-temp rels or committing any
	 * command that wanted to force sync commit, then we must flush XLOG
	 * immediately. (We must not allow asynchronous commit if there are any
	 * non-temp tables to be deleted, because we might delete the files before
	 * the COMMIT record is flushed to disk. We do allow asynchronous commit
	 * if all to-be-deleted tables are temporary though, since they are lost
	 * anyway if we crash.)
	 */
	if ((wrote_xlog && markXidCommitted &&
		 synchronous_commit > SYNCHRONOUS_COMMIT_OFF) ||
		forceSyncCommit || nrels > 0)
	{
		XLogFlush(XactLastRecEnd);
		...
		/*
		 * Now we may update the CLOG, if we wrote a COMMIT record above
		 */
		if (markXidCommitted)
			TransactionIdCommitTree(xid, nchildren, children);
	}
	else
	{
		/*
		 * Asynchronous commit case:
		 *
		 * This enables possible committed transaction loss in the case of a
		 * postmaster crash because WAL buffers are left unwritten. Ideally we
		 * could issue the WAL write without the fsync, but some
		 * wal_sync_methods do not allow separate write/fsync.
		 *
		 * Report the latest async commit LSN, so that the WAL writer knows to
		 * flush this commit.
		 */
		XLogSetAsyncXactLSN(XactLastRecEnd);

		/*
		 * We must not immediately update the CLOG, since we didn't flush the
		 * XLOG. Instead, we store the LSN up to which the XLOG must be
		 * flushed before the CLOG may be updated.
		 */
		if (markXidCommitted)
			TransactionIdAsyncCommitTree(xid, nchildren, children, XactLastRecEnd);
	}

Note the ordering: the CLOG is updated after the WAL flush, in the synchronous case. That ordering is the whole reason a synchronous commit is safe: the commit record is on disk before anyone can see the transaction as committed.

In the asynchronous case, the CLOG update is deferred and carries the LSN that the WAL must reach first. That’s the group LSN mechanism in clog.c, which we come to below.

The synchronous_commit levels

The levels are an enum (src/include/access/xact.h):

 c
 
 
typedef enum
{
	SYNCHRONOUS_COMMIT_OFF,		/* asynchronous commit */
	SYNCHRONOUS_COMMIT_LOCAL_FLUSH, /* wait for local flush only */
	SYNCHRONOUS_COMMIT_REMOTE_WRITE,	/* wait for local flush and remote
										 * write */
	SYNCHRONOUS_COMMIT_REMOTE_FLUSH,	/* wait for local and remote flush */
	SYNCHRONOUS_COMMIT_REMOTE_APPLY,	/* wait for local and remote flush and
										 * remote apply */
}			SyncCommitLevel;

/* Define the default setting for synchronous_commit */
#define SYNCHRONOUS_COMMIT_ON	SYNCHRONOUS_COMMIT_REMOTE_FLUSH

Setting	Enum value	Waits for
`off`	`SYNCHRONOUS_COMMIT_OFF`	nothing
`local`	`SYNCHRONOUS_COMMIT_LOCAL_FLUSH`	local WAL flush
`remote_write`	`SYNCHRONOUS_COMMIT_REMOTE_WRITE`	local flush + remote write
`on`	`SYNCHRONOUS_COMMIT_REMOTE_FLUSH`	local flush + remote flush
`remote_apply`	`SYNCHRONOUS_COMMIT_REMOTE_APPLY`	local flush + remote flush + remote apply

Note the subtlety in the names: the enum value called SYNCHRONOUS_COMMIT_REMOTE_FLUSH is what on maps to. And on is the default:

#define SYNCHRONOUS_COMMIT_ON	SYNCHRONOUS_COMMIT_REMOTE_FLUSH

The accepted spellings are enumerated in src/backend/utils/misc/guc_tables.c:

 c
 
 
/*
 * Although only "on", "off", "remote_apply", "remote_write", and "local" are
 * documented, we accept all the likely variants of "on" and "off".
 */
static const struct config_enum_entry synchronous_commit_options[] = {
	{"local", SYNCHRONOUS_COMMIT_LOCAL_FLUSH, false},
	{"remote_write", SYNCHRONOUS_COMMIT_REMOTE_WRITE, false},
	{"remote_apply", SYNCHRONOUS_COMMIT_REMOTE_APPLY, false},
	{"on", SYNCHRONOUS_COMMIT_ON, false},
	{"off", SYNCHRONOUS_COMMIT_OFF, false},
	{"true", SYNCHRONOUS_COMMIT_ON, true},
	{"false", SYNCHRONOUS_COMMIT_OFF, true},
	{"yes", SYNCHRONOUS_COMMIT_ON, true},
	{"no", SYNCHRONOUS_COMMIT_OFF, true},
	{"1", SYNCHRONOUS_COMMIT_ON, true},
	{"0", SYNCHRONOUS_COMMIT_OFF, true},
	{NULL, 0, false}
};

The check in RecordTransactionCommit() is synchronous_commit > SYNCHRONOUS_COMMIT_OFF, so anything above off takes the flushing path.

The asynchronous committer: a latch, not an I/O

The async committer does no I/O at all. It publishes an LSN and pokes the WAL writer’s latch:

 c
 
 
/*
 * Record the LSN for an asynchronous transaction commit/abort
 * and nudge the WALWriter if there is work for it to do.
 * (This should not be called for synchronous commits.)
 */
void
XLogSetAsyncXactLSN(XLogRecPtr asyncXactLSN)
{
	XLogRecPtr	WriteRqstPtr = asyncXactLSN;
	bool		sleeping;
	bool		wakeup = false;
	XLogRecPtr	prevAsyncXactLSN;

	SpinLockAcquire(&XLogCtl->info_lck);
	sleeping = XLogCtl->WalWriterSleeping;
	prevAsyncXactLSN = XLogCtl->asyncXactLSN;
	if (XLogCtl->asyncXactLSN < asyncXactLSN)
		XLogCtl->asyncXactLSN = asyncXactLSN;
	SpinLockRelease(&XLogCtl->info_lck);

	/*
	 * If somebody else already called this function with a more aggressive
	 * LSN, they will have done what we needed (and perhaps more).
	 */
	if (asyncXactLSN <= prevAsyncXactLSN)
		return;

	/*
	 * If the WALWriter is sleeping, kick it to make it come out of low-power
	 * mode, so that this async commit will reach disk within the expected
	 * amount of time. Otherwise, determine whether it has enough WAL
	 * available to flush, the same way that XLogBackgroundFlush() does.
	 */
	if (sleeping)
		wakeup = true;
	else
	{
		int			flushblocks;

		RefreshXLogWriteResult(LogwrtResult);

		flushblocks =
			WriteRqstPtr / XLOG_BLCKSZ - LogwrtResult.Flush / XLOG_BLCKSZ;

		if (WalWriterFlushAfter == 0 || flushblocks >= WalWriterFlushAfter)
			wakeup = true;
	}

	if (wakeup)
	{
		ProcNumber	walwriterProc = pg_atomic_read_u32(&ProcGlobal->walwriterProc);

		if (walwriterProc != INVALID_PROC_NUMBER)
			SetLatch(&GetPGProcByNumber(walwriterProc)->procLatch);
	}
}

Two things happen: the LSN is recorded so the WAL writer knows how far it must eventually flush, and the WAL writer’s latch is set so it wakes up immediately rather than waiting for its next timeout.

The WAL writer’s main loop is in src/backend/postmaster/walwriter.c:

 c
 
 
		/* Clear any already-pending wakeups */
		ResetLatch(MyLatch);

		/* Process any signals received recently */
		ProcessMainLoopInterrupts();

		INJECTION_POINT("walwriter-before-background-flush", NULL);

		/*
		 * Do what we're here for; then, if XLogBackgroundFlush() found useful
		 * work to do, reset hibernation counter.
		 */
		if (XLogBackgroundFlush())
			left_till_hibernate = LOOPS_UNTIL_HIBERNATE;
		else if (left_till_hibernate > 0)
			left_till_hibernate--;

		/* report pending statistics to the cumulative stats system */
		pgstat_report_wal(false);

		/*
		 * Sleep until we are signaled or WalWriterDelay has elapsed. If we
		 * haven't done anything useful for quite some time, lengthen the
		 * sleep time so as to reduce the server's idle power consumption.
		 */
		if (left_till_hibernate > 0)
			cur_timeout = WalWriterDelay;	/* in ms */
		else
			cur_timeout = WalWriterDelay * HIBERNATE_FACTOR;

		(void) WaitLatch(MyLatch,
						 WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
						 cur_timeout,
						 WAIT_EVENT_WAL_WRITER_MAIN);

And XLogBackgroundFlush() decides how far to flush:

 c
 
 
	/*
	 * Determine how far to flush WAL, based on the wal_writer_delay and
	 * wal_writer_flush_after GUCs.
	 *
	 * Note that XLogSetAsyncXactLSN() performs similar calculation based on
	 * wal_writer_flush_after, to decide when to wake us up. Make sure the
	 * logic is the same in both places if you change this.
	 */
	now = GetCurrentTimestamp();
	flushblocks =
		WriteRqst.Write / XLOG_BLCKSZ - LogwrtResult.Flush / XLOG_BLCKSZ;

	if (WalWriterFlushAfter == 0 || lastflush == 0)
	{
		/* first call, or block based limits disabled */
		WriteRqst.Flush = WriteRqst.Write;
		lastflush = now;
	}
	else if (TimestampDifferenceExceeds(lastflush, now, WalWriterDelay))
	{
		/*
		 * Flush the writes at least every WalWriterDelay ms. This is
		 * important to bound the amount of time it takes for an asynchronous
		 * commit to hit disk.
		 */
		WriteRqst.Flush = WriteRqst.Write;
		lastflush = now;
	}
	else if (flushblocks >= WalWriterFlushAfter)
	{
		/* exceeded wal_writer_flush_after blocks, flush */
		WriteRqst.Flush = WriteRqst.Write;
		lastflush = now;
	}
	else
	{
		/* no flushing, this time round */
		WriteRqst.Flush = InvalidXLogRecPtr;
	}

So an async commit reaches disk within at most a few wal_writer_delay cycles. The WAL writer’s docstring says so:

 c
 
 
 * We can guarantee that async commits reach disk after at most three
 * wal_writer_delay cycles. (When flushing complete blocks, we allow XLogWrite
 * to write "flexibly", meaning it can stop at the end of the buffer ring;
 * this makes a difference only with very high load or long wal_writer_delay,
 * but imposes one extra cycle for the worst case for async commits.)

The CLOG group LSN

An async committer cannot update pg_xact immediately — the commit record might not be on disk yet. So instead it stores the LSN that the WAL must reach before the CLOG update is allowed to become visible. The CLOG stores those LSNs in groups of 32 transactions (src/backend/access/transam/clog.c):

 c
 
 
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP	32	/* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE	(CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)

#define GetLSNIndex(slotno, xid)	((slotno) * CLOG_LSNS_PER_PAGE + \
	((xid) % (TransactionId) CLOG_XACTS_PER_PAGE) / CLOG_XACTS_PER_LSN_GROUP)

And updating it:

 c
 
 
	/*
	 * Update the group LSN if the transaction completion LSN is higher.
	 *
	 * Note: lsn will be invalid when supplied during InRecovery processing,
	 * so we don't need to do anything special to avoid LSN updates during
	 * recovery. After recovery completes the next clog change will set the
	 * LSN correctly.
	 */
	if (XLogRecPtrIsValid(lsn))
	{
		int			lsnindex = GetLSNIndex(slotno, xid);

		if (XactCtl->shared->group_lsn[lsnindex] < lsn)
			XactCtl->shared->group_lsn[lsnindex] = lsn;
	}

Reading it back:

 c
 
 
/*
 * Interrogate the state of a transaction in the commit log.
 *
 * Aside from the actual commit status, this function returns (into *lsn)
 * an LSN that is late enough to be able to guarantee that if we flush up to
 * that LSN then we will have flushed the transaction's commit record to disk.
 * The result is not necessarily the exact LSN of the transaction's commit
 * record!	For example, for long-past transactions (those whose clog pages
 * already migrated to disk), we'll return InvalidXLogRecPtr. Also, because
 * we group transactions on the same clog page to conserve storage, we might
 * return the LSN of a later transaction that falls into the same group.
 *
 * NB: this is a low-level routine and is NOT the preferred entry point
 * for most uses; TransactionLogFetch() in transam.c is the intended caller.
 */
XidStatus
TransactionIdGetStatus(TransactionId xid, XLogRecPtr *lsn)
{
	...
	lsnindex = GetLSNIndex(slotno, xid);
	*lsn = XactCtl->shared->group_lsn[lsnindex];
	...
}

The grouping is a compromise: one LSN is stored per 32 transactions, so the LSN you read back might be later than your own transaction’s commit record. That is conservative in the safe direction — flushing to that LSN definitely flushed yours.

The other half of clog.c’s concurrency story is the group update mechanism, which is about the SLRU bank lock rather than about the WAL:

 c
 
 
/*
 * Subroutine for TransactionIdSetPageStatus, q.v.
 *
 * When we cannot immediately acquire the SLRU bank lock in exclusive mode at
 * commit time, add ourselves to a list of processes that need their XIDs
 * status update. The first process to add itself to the list will acquire
 * the lock in exclusive mode and set transaction status as required on behalf
 * of all group members. This avoids a great deal of contention when many
 * processes are trying to commit at once, since the lock need not be
 * repeatedly handed off from one committing process to the next.
 *
 * Returns true when transaction status has been updated in clog; returns
 ...
 */

Same trick as the WAL write lock: one backend does the work on behalf of many, and the others observe the result and move on.

Caveats

The unlucky committer pays full latency. Group commit helps throughput; it does nothing for the latency of the single backend that arrives just after an fsync started. Its record will be flushed by the next fsync, and it waits for the whole of that one.

The flusher may flush more than it needs to. XLogFlush() deliberately piggybacks: it bumps the write request to XLogCtl->LogwrtRqst.Write and flushes everything up to there. That is good for throughput — every follower gets its record flushed for free — but it means one backend’s commit can force a larger write and a larger fsync than its own record required. Under heavy write load, one unlucky commit can trigger an fsync of the entire accumulated WAL.

commit_delay is a latency-for-throughput trade, and it is only beneficial in a narrow band. Too small and it does not gather enough followers; too large and every commit pays the delay for no benefit. The commit_siblings guard exists so that a lightly loaded server does not sleep at all. In practice, on modern hardware with a decent NVMe device, commit_delay is usually best left at zero: the fsync is fast enough that the group forms naturally.

synchronous_commit = off can lose committed transactions. The source is blunt about it: “This enables possible committed transaction loss in the case of a postmaster crash because WAL buffers are left unwritten.” It is fine for bulk loads into unlogged tables and for many data-warehouse workloads. It is not fine for a ledger.

synchronous_commit = local is not “safe” in the usual sense. It waits for the local WAL flush but not for the standby. If you are not using replication, local and on are equivalent. If you are, local means a failover can lose the last few transactions.

The fsync is the whole latency. Everything above is microseconds. XLogFlush() at the end of the day is an fsync() (or fdatasync(), or O_SYNC write) on a device whose flush latency is measured in milliseconds. Group commit cannot change that number; it only changes how many commits share it.

The WAL writer can flush asynchronously while you are committing, but it cannot help a synchronous commit. The background writer and a committing backend both contend for WALWriteLock. A synchronous committer that finds the lock held by the WAL writer has to wait for it, then do its own fsync.

WaitXLogInsertionsToFinish() must be called before taking WALWriteLock. Getting this backwards is a deadlock, and the source says so twice: in the function’s own header comment, and in XLogFlush(), which does the wait outside the lock acquisition.

What it all means in practice

Commit latency is fsync latency, and nothing else. If your commits are slow, look at the device, not at PostgreSQL. pg_stat_io (with track_wal_io_timing = on) shows the WAL write and flush timings separately, and pg_stat_bgwriter has the WAL counters.

Group commit is automatic and free. You do not configure it. You do not “enable” it. Every synchronous commit already participates: if another backend is in XLogFlush(), you ride along. The only knob is commit_delay, and it is usually best left alone.

The “one backend does the fsync for everyone” design has a consequence you should know about. The flusher flushes everything it can see. If you are debugging a slow commit and see a single fsync() covering a large amount of WAL, that is not a bug — that is one backend doing the work for twenty others.

The three frontiers are the model to keep in your head. “Inserted” means in shared memory and costs nothing. “Written” means the OS has the bytes and costs a write(). “Flushed” means the device has them and costs an fsync(). A synchronous commit waits for the third. An asynchronous commit waits for none of them and hopes the WAL writer gets there.

The reservation trick is why WAL insertion scales. CurrBytePos += size under a spinlock, with the expensive LSN arithmetic moved outside the lock, is what lets a hundred backends insert WAL records at once without the insertion point becoming a bottleneck. The “usable byte position” — a cursor that does not count page headers — is the trick that makes the arithmetic inside the lock trivial.

synchronous_commit is the knob that matters. on (the default) is the durable choice. remote_write and remote_apply trade local durability for a stronger replication guarantee. local is for “I have replication but I accept losing the last few transactions on failover”. off is for bulk loads and nothing else.

The commit critical section is a real cost, and it exists for a reason. It pins checkpoints so that a commit record cannot reach disk without its pg_xact update. If you see checkpoint lag on a write-heavy workload, that is what you are looking at.

If you are reading the source to understand durability, read these four functions in order: RecordTransactionCommit() for the policy, XLogInsertRecord() and ReserveXLogInsertLocation() for how the record gets into memory, XLogFlush() for the group commit loop, and LWLockAcquireOrWait() for the trick that makes the loop work.

PostgreSQL's bit and varbit Implementation in C

Jay Hubbert — Tue, 08 Sep 2026 08:34:05 +0000

In SQL, bit(4) and bit varying(4) look like different animals: one has a fixed width, the other does not. In PostgreSQL they are the same type underneath. The same struct, the same I/O routines, the same comparison functions, the same catalog-provided C symbol for output. The only things that differ are the meaning of the type modifier and one line of arithmetic in the input function.

There is also a storage-layout invariant that is easy to miss but explains a lot of the code: the low-order padding bits of the final byte must be zero, and the whole design leans on that guarantee.

This is a walk through bit and varbit, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The struct

From src/include/utils/varbit.h:

 c
 
 
typedef struct
{
	int32		vl_len_;		/* varlena header (do not touch directly!) */
	int32		bit_len;		/* number of valid bits */
	uint8		bit_dat[FLEXIBLE_ARRAY_MEMBER]; /* bit string, most sig. byte
												 * first */
} VarBit;

A varlena header, then a length in bits, then the bytes. The header comment is worth quoting because it is the invariant the rest of the file depends on:

Caution: if bit_len is not a multiple of BITS_PER_BYTE, the low-order bits of the last byte of bit_dat[] are unused and MUST be zeroes. (This allows bit_cmp() to not bother masking the last byte.) Also, there should not be any excess bytes counted in the header length.

So the datum carries two lengths. VARSIZE gives the number of bytes; bit_len gives the number of valid bits. The difference is the padding:

 c
 
 
#define VARBITHDRSZ			sizeof(int32)
#define VARBITLEN(PTR)		(((VarBit *) (PTR))->bit_len)
#define VARBITS(PTR)		(((VarBit *) (PTR))->bit_dat)
#define VARBITBYTES(PTR)	(VARSIZE(PTR) - VARHDRSZ - VARBITHDRSZ)
#define VARBITPAD(PTR)		(VARBITBYTES(PTR)*BITS_PER_BYTE - VARBITLEN(PTR))
#define VARBITTOTALLEN(BITLEN)	(((BITLEN) + BITS_PER_BYTE-1)/BITS_PER_BYTE + \
								 VARHDRSZ + VARBITHDRSZ)
#define VARBITMAXLEN		(INT_MAX - BITS_PER_BYTE + 1)

VARBITTOTALLEN is the allocation formula: ceil(bits / 8) data bytes plus VARHDRSZ (4) plus VARBITHDRSZ (4). Even a bit(1) has an 8-byte overhead before its single data byte. VARBITMAXLEN is INT_MAX - 7, chosen so that the largest internal computation, bitlen + BITS_PER_BYTE - 1, cannot overflow an int32.

Catalog identity

From src/include/catalog/pg_type.dat:

Type	OID	Array OID	`typlen`	`typbyval`	`typalign`	`typstorage`	`typcategory`
`bit`	1560	1561	-1	f	`i`	`x`	`V`
`varbit`	1562	1563	-1	f	`i`	`x`	`V`

The important row is typlen => '-1'. bit(n) is not a fixed-length type, no matter how fixed its width looks. It is a varlena, and typstorage => 'x' (EXTENDED) means it is fully toastable: a large bit value can be compressed and moved out of line. That is a genuine difference from SQL char(n), which is a fixed-length, inline, space-padded type.

varbit carries typispreferred => 't' in category V; bit does not, so an unadorned literal prefers bit varying. Both take a type modifier, via typmodin => 'bittypmodin' / 'varbittypmodin'.

What the bytes look like

 text
 
 
bit(4) datum '1010', stored short:
+--------+----------------+---------------+--------+
| 1-byte | bit_len = 4 | bit_dat[0] | (pad) |
| header | (int32) | 1010 0000 | |
+--------+----------------+---------------+--------+
 total: 6 bytes (9 bytes before the short-header conversion)

The bits are stored most-significant-bit-first within each byte, and the string runs from the beginning of bit_dat. '1010' becomes the byte 0b10100000: four significant bits, four zero padding bits.

Input and output

bit_in() and varbit_in() are near-duplicates, and the difference between them is the whole semantic distinction between the types. The parsing prefix is shared:

 c
 
 
if (input_string[0] == 'b' || input_string[0] == 'B')
{
	bit_not_hex = true;
	sp = input_string + 1;
}
else if (input_string[0] == 'x' || input_string[0] == 'X')
{
	bit_not_hex = false;
	sp = input_string + 1;
}
else
{
	/* Otherwise it's binary. */
	bit_not_hex = true;
	sp = input_string;
}

So B'1010', X'1a', and a bare '1010' are all accepted; the bare form is binary, which is what makes cast('1001' as bit) work.

The divergence comes next. For bit, the type modifier is an exact length:

 c
 
 
if (atttypmod <= 0)
	atttypmod = bitlen;
else if (bitlen != atttypmod)
	ereturn(escontext, (Datum) 0,
			(errcode(ERRCODE_STRING_DATA_LENGTH_MISMATCH),
			 errmsg("bit string length %d does not match type bit(%d)",
					bitlen, atttypmod)));

For varbit, it is a maximum:

 c
 
 
else if (bitlen > atttypmod)
	ereturn(escontext, (Datum) 0,
			(errcode(ERRCODE_STRING_DATA_RIGHT_TRUNCATION),
			 errmsg("bit string too long for type bit varying(%d)",
					atttypmod)));

Note that bit errors on a mismatch (too short or too long), while varbit errors only on too long. And note the subtlety that an exact-length bit still stores atttypmod as its bit_len even when the input had the same number of bits — the datum’s length is the column’s declared width.

Casting is deliberately more forgiving than input. The bit() function has an isExplicit argument: an implicit cast raises an error on a length mismatch, but an explicit cast “silently truncate[s] or zero-pad[s]” to the requested length. So '1'::bit(1)::bit(4) is 1000, while assigning '1'::bit(1) to a bit(4) column is an error.

Output has a nice detail. bit_out() is not its own implementation:

 c
 
 
Datum
bit_out(PG_FUNCTION_ARGS)
{
#if 1
	/* same as varbit output */
	return varbit_out(fcinfo);
#else
	...
#endif
}

The #else branch holds a complete, working hex-output formatter that is compiled out — kept, per its comment, “in case someone wants to write a formatting function”. varbit_out() itself walks the bytes printing 1 and 0, with a separate loop for the final partial byte so it prints exactly bit_len characters. It also carries an XXX comment noting that varbit_recv() and hex input can load values the bit-oriented output cannot round-trip.

The padding invariant, and who has to maintain it

VARBIT_PAD is the enforcement macro:

 c
 
 
#define VARBIT_PAD(vb) \
	do { \
		int32	pad_ = VARBITPAD(vb); \
		Assert(pad_ >= 0 && pad_ < BITS_PER_BYTE); \
		if (pad_ > 0) \
			*(VARBITS(vb) + VARBITBYTES(vb) - 1) &= BITMASK << pad_; \
	} while (0)

Any function that can produce non-zero padding bits must call it. The clearest example is bitnot():

 c
 
 
for (size_t i = 0; i < nbytes; i++)
	r[i] = ~p[i];

/* Must zero-pad the result, because extra bits are surely 1's here */
VARBIT_PAD(result);

bit_and, bit_or, and bitxor do not need to pad, and each says so: “of 0 pads is 0”. bitfromint4 and bitfromint8 build their results carefully for the same reason, and varbit_recv() pads after copying bytes off the wire because the sender’s padding is not trusted. The invariant is checked under USE_ASSERT_CHECKING by VARBIT_CORRECTLY_PADDED, which is invoked at the top of the output function.

The payoff is in comparison:

 c
 
 
static int32
bit_cmp(VarBit *arg1, VarBit *arg2)
{
	...
	cmp = memcmp(VARBITS(arg1), VARBITS(arg2), Min(bytelen1, bytelen2));
	if (cmp == 0)
	{
		bitlen1 = VARBITLEN(arg1);
		bitlen2 = VARBITLEN(arg2);
		if (bitlen1 != bitlen2)
			cmp = (bitlen1 < bitlen2) ? -1 : 1;
	}
	return cmp;
}

Because padding is guaranteed zero, bit_cmp is a plain memcmp over the common prefix with no masking, and then a tiebreak on the bit length. That tiebreak is what the comment above the comparison operators warns about: the standard “is not unambiguous about the comparison between zero-padded bit strings and varying bitstrings”, and PostgreSQL deliberately treats extra zero bits as making a string larger. '10' and '1000' compare unequal even though their integer values would be equal.

biteq() and bitne() add a fast path that compares bit_len before doing any byte comparison, so equal-length values — the common case in a bit(n) column — short-circuit to bit_cmp.

Operations

The bitwise operators are byte loops over VARBITBYTES, with a length check first (cannot AND bit strings of different sizes). Shifts are more involved: bitshiftleft() and bitshiftright() split the shift into a whole-byte part and a sub-byte part, use memcpy/MemSet when the sub-byte part is zero, and shift bit-by-bit otherwise. A negative shift delegates to the opposite direction, with a guard against INT_MIN negation. Right shift has to re-pad, since it can shift one-bits into the padding (VARBIT_PAD_LAST).

bit_count() (C name bit_bit_count) is a single call:

 c
 
PG_RETURN_INT64(pg_popcount((char *) VARBITS(arg), VARBITBYTES(arg)));

Conveniently, the padding bits are zero, so they contribute nothing to the population count — no masking needed. get_bit/set_bit (C names bitgetbit / bitsetbit) index from the left in zero-based fashion, “inconsistent with the standard substring, position, overlay functions” per the comment.

One thing that is missing

Index support for varbit is narrower than you might expect. In src/include/catalog/pg_opclass.dat the only opclasses are btree/varbit_ops and brin/varbit_minmax_ops. There is no hash opclass, and no GiST or SP-GiST opclass. You can btree-index a bit string and you can BRIN-index it by min/max, but you cannot hash-index it.

Look at it yourself

 sql
 
 
-- bit and varbit are the same representation
SELECT '1010'::bit(4), '1010'::varbit, 'x1a'::varbit; -- 1010 | 1010 | 00011010

-- bit length vs byte length
SELECT length('1010'::varbit), -- 4 (VARBITLEN)
 octet_length('1010'::varbit); -- 1 (VARBITBYTES)

-- explicit casts pad, implicit ones do not
SELECT '1'::bit(1)::bit(4); -- 1000

-- population count ignores the zero padding
SELECT bit_count(B'10101010'); -- 4

-- zero-based, left-to-right bit access
SELECT get_bit('1010'::bit(4), 0), -- 1
 get_bit('1010'::bit(4), 3); -- 0

-- varlena storage: 8 bytes of overhead plus the data, shortened on disk
CREATE TEMP TABLE b (x bit(4), y varbit);
INSERT INTO b VALUES ('1010', '1010');
SELECT pg_column_size(x), pg_column_size(y) FROM b; -- 6, 6

-- btree and BRIN exist; this narrower query returns zero rows
SELECT c.opcname, a.amname
FROM pg_opclass c JOIN pg_am a ON a.oid = c.opcmethod
WHERE c.opcintype = 'varbit'::regtype AND a.amname = 'hash';

Key takeaways

bit and varbit share one VarBit struct: a varlena header, an int32 bit count, and the bytes.
bit(n) is stored as a varlena (typlen = -1) and is fully toastable (typstorage = 'x'); the width is enforced by the type modifier, not the layout.
The only real difference between the two types is that bit’s modifier is an exact length and varbit’s is a maximum — one branch in the input function.
The unused low bits of the last byte are required to be zero; that lets bit_cmp use memcmp without masking, and bit_count use pg_popcount without masking.
bit/varbit support btree and BRIN min/max indexes, but there is no hash opclass.

PostgreSQL's interval Implementation in C

Jay Hubbert — Mon, 07 Sep 2026 01:55:11 +0000

An interval looks like a duration, but PostgreSQL does not store it as one number. It stores three: a month count, a day count, and a microsecond count, packed into a 16-byte struct. They are never summed into a single value at storage time, because they cannot be — a month is not a fixed number of days, and a day is not a fixed number of hours once daylight saving time is involved. The addition only happens when you compare, and the conversion factor PostgreSQL picks for that comparison is the surprising part.

This is a walk through the C implementation, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entry

From src/include/catalog/pg_type.dat (lines 312–318):

 text
 
 
{ oid => '1186', array_type_oid => '1187',
 descr => 'time interval, format \'number units ...\'',
 typname => 'interval', typlen => '16', typbyval => 'f', typcategory => 'T',
 typispreferred => 't', typinput => 'interval_in', typoutput => 'interval_out',
 typreceive => 'interval_recv', typsend => 'interval_send',
 typmodin => 'intervaltypmodin', typmodout => 'intervaltypmodout',
 typalign => 'd' },

Field	Value	Meaning
`typlen`	`16`	a 16-byte fixed-length value
`typbyval`	`f`	passed by reference (a `Datum` holds a pointer)
`typalign`	`d`	aligned to an 8-byte boundary
`array_type_oid`	`1187`	the `interval[]` array type
`typstorage`	(absent)	defaults to `p`, PLAIN — never TOASTed

interval is typispreferred, so a bare literal like '3 days' resolves to interval rather than to another category-T type.

The C representation

The struct is in src/include/datatype/timestamp.h:

 c
 
 
/*
 * Storage format for type interval.
 */
typedef struct
{
	TimeOffset	time;			/* all time units other than days, months and
								 * years */
	int32		day;			/* days, after time for alignment */
	int32		month;			/* months and years, after time for alignment */
} Interval;

TimeOffset is int64, so sizeof(Interval) is 8 + 4 + 4 = 16, exactly typlen. Unlike timetz — whose in-memory struct is 16 bytes but whose typlen is 12 — there is no padding discrepancy here: the wide field is placed first so nothing needs to be aligned around it, and the whole struct is what gets copied into the tuple.

text

+------------------+-----------+-----------+
| time (int64) | day(int32)| month(int32)
| microseconds | days | months
+------------------+-----------+-----------+
 0 8 12 16

The introductory comment in the same header explains why three fields exist at all:

Interval represents delta time. Keep track of months (and years), days, and hours/minutes/seconds separately since the elapsed time spanned is unknown until instantiated relative to an absolute time.

The broken-down helpers are struct pg_itm (for output and arithmetic) and struct pg_itm_in (for decoding input), both in the same header. pg_itm splits the microseconds back into hours, minutes, seconds, and a remainder, and splits the month count into years and months.

Input and output

interval_in() (src/backend/utils/adt/timestamp.c) parses with the shared ParseDateTime/DecodeInterval pair, falling back to DecodeISO8601Interval when the classic parser rejects the string:

 c
 
 
	dterr = ParseDateTime(str, workbuf, sizeof(workbuf), field,
						 ftype, MAXDATEFIELDS, &nf);
	if (dterr == 0)
		dterr = DecodeInterval(field, ftype, nf, range,
							 &dtype, itm_in);

	/* if those functions think it's a bad format, try ISO8601 style */
	if (dterr == DTERR_BAD_FORMAT)
		dterr = DecodeISO8601Interval(str,
									 &dtype, itm_in);

The decoder fills a pg_itm_in, and itmin2interval() folds the year/month parts into a single month count and copies the rest across:

 c
 
 
	int64		total_months = (int64) itm_in->tm_year * MONTHS_PER_YEAR + itm_in->tm_mon;

	if (total_months > INT_MAX || total_months < INT_MIN)
		return -1;
	span->month = (int32) total_months;
	span->day = itm_in->tm_mday;
	span->time = itm_in->tm_usec;

interval_out() goes the other way: interval2itm() explodes the three fields into a pg_itm, and EncodeInterval() renders it according to the IntervalStyle GUC. There are four styles, defined in src/include/miscadmin.h:

 c
 
 
#define INTSTYLE_POSTGRES			0
#define INTSTYLE_POSTGRES_VERBOSE	1
#define INTSTYLE_SQL_STANDARD		2
#define INTSTYLE_ISO_8601			3

The default is INTSTYLE_POSTGRES, which is why '1 day 02:03:04'::interval prints the way it does, while SET intervalstyle = 'iso_8601' makes the same value print as P1DT2H3M4S.

The binary format is the struct field by field, in memory order (interval_send/interval_recv):

 c
 
 
	pq_sendint64(&buf, interval->time);
	pq_sendint32(&buf, interval->day);
	pq_sendint32(&buf, interval->month);

Comparison: a 30-day month

This is the interesting part. interval cannot be compared field by field — '1 month' and '30 days' are different structs but the same span. So interval_cmp_internal() collapses both operands to a single number using interval_cmp_value():

 c
 
 
static inline INT128
interval_cmp_value(const Interval *interval)
{
	INT128		span;
	int64		days;

	/*
	 * Combine the month and day fields into an integral number of days.
	 * Because the inputs are int32, int64 arithmetic suffices here.
	 */
	days = interval->month * INT64CONST(30);
	days += interval->day;

	/* Widen time field to 128 bits */
	span = int64_to_int128(interval->time);

	/* Scale up days to microseconds, forming a 128-bit product */
	int128_add_int64_mul_int64(&span, days, USECS_PER_DAY);

	return span;
}

A month is exactly 30 days, a day is exactly 24 hours, and the result is widened to INT128 because 30 days’ worth of microseconds times an int32 month count overflows int64. This is the same DAYS_PER_MONTH 30 constant used by justify_interval, and it is why:

 sql
 
 
SELECT interval '1 month' = interval '30 days'; -- true
SELECT interval '1 day' = interval '24 hours'; -- true
SELECT interval '1 month' = interval '31 days'; -- false

Hashing has to agree with that equivalence, so interval_hash() hashes the same collapsed span — and deliberately only its low 64 bits, “for compatibility with hashes calculated before use of INT128 was introduced”:

 c
 
	INT128		span = interval_cmp_value(interval);
	int64		span64;

	span64 = int128_to_int64(span);

	return DirectFunctionCall1(hashint8, Int64GetDatumFast(span64));

So two intervals that compare equal always hash equal, even though their underlying three fields may differ.

Typmod: two numbers in one int32

interval’s type modifier is unusually dense. The range (which fields are allowed) lives in the high 16 bits and the sub-second precision in the low 16 bits (src/include/utils/timestamp.h):

 c
 
 
#define INTERVAL_FULL_RANGE (0x7FFF)
#define INTERVAL_FULL_PRECISION (0xFFFF)
#define INTERVAL_TYPMOD(p,r) ((((r) & INTERVAL_RANGE_MASK) << 16) | ((p) & INTERVAL_PRECISION_MASK))
#define INTERVAL_PRECISION(t) ((t) & INTERVAL_PRECISION_MASK)
#define INTERVAL_RANGE(t) (((t) >> 16) & INTERVAL_RANGE_MASK)

The range is a bitmask built from the field codes in src/include/utils/datetime.h (YEAR 2, MONTH 1, DAY 3, HOUR 10, MINUTE 11, SECOND 12), combined by INTERVAL_MASK(b) = (1 << (b)). The header comment there notes the codes “must be in the range 0..14 so that the associated bitmasks can fit into the left half of an INTERVAL’s typmod value”. That is why INTERVAL DAY TO SECOND is representable in a single int32.

AdjustIntervalForTypmod() then truncates or rounds according to that granularity. For INTERVAL YEAR it zeroes the day and time fields; for INTERVAL MINUTE it floors time to a whole minute; the precision half rounds sub-second digits with the same offset trick the timestamp code uses. Note the deliberate rule, spelled out in a comment: fields to the right of the last specified one are zeroed, but fields to the left are left alone, so INTERVAL MONTH and INTERVAL YEAR TO MONTH behave identically.

Normalization is opt-in

Because the three fields are independent, interval '1 day 25 hours' is a legal value: day = 1, time = 25 * USECS_PER_HOUR. The addition operators do not normalize — finite_interval_pl() just adds the fields independently, with overflow checks:

 c
 
 
	if (pg_add_s32_overflow(span1->month, span2->month, &result->month) ||
		pg_add_s32_overflow(span1->day, span2->day, &result->day) ||
		pg_add_s64_overflow(span1->time, span2->time, &result->time) ||
		INTERVAL_NOT_FINITE(result))
		ereport(ERROR, ...);

Normalization is available only if you ask for it. justify_hours() folds whole days out of the time field into day; justify_days() folds every 30 days out of day into month; justify_interval() does both and also makes all three fields share a sign:

 c
 
 
	TMODULO(result->time, wholeday, USECS_PER_DAY);
	result->day += wholeday;

	wholemonth = result->day / DAYS_PER_MONTH;
	result->day -= wholemonth * DAYS_PER_MONTH;

There is a beautiful fossil in timestamp_mi() (which computes timestamp - timestamp). The result is an absolute instant difference, so only the time field is set — yet the function then calls interval_justify_hours() with this comment:

 c
 
 
	/*----------
	 *	This is wrong, but removing it breaks a lot of regression tests.
	 *	For example:
	 *
	 *	test=> SET timezone = 'EST5EDT';
	 *	test=> SELECT
	 *	test-> ('2005-10-30 13:22:00-05'::timestamptz -
	 *	test(>	'2005-10-29 13:22:00-04'::timestamptz);
	 *	 ?column?
	 *	----------------
	 *	 1 day 01:00:00
	 *	 (1 row)
	 *----------
	 */
	result = DatumGetIntervalP(DirectFunctionCall1(interval_justify_hours,
											 IntervalPGetDatum(result)));

The 25-hour wall-clock gap across a DST fall-back is reported as “1 day 01:00:00” rather than “25:00:00”, because the code normalizes it. The comment admits the semantics are debatable and that backward compatibility won.

Adding an interval to a timestamp

Here the three fields finally acquire meaning, and they are applied in order: months first, then days, then the microsecond time (timestamp_pl_interval()). Months are added to the calendar, with an end-of-month clamp:

 c
 
 
		if (pg_add_s32_overflow(tm->tm_mon, span->month, &tm->tm_mon))
			ereport(ERROR, ...);
		if (tm->tm_mon > MONTHS_PER_YEAR)
		{
			tm->tm_year += (tm->tm_mon - 1) / MONTHS_PER_YEAR;
			tm->tm_mon = ((tm->tm_mon - 1) % MONTHS_PER_YEAR) + 1;
		}
		...
		/* adjust for end of month boundary problems... */
		if (tm->tm_mday > day_tab[isleap(tm->tm_year)][tm->tm_mon - 1])
			tm->tm_mday = (day_tab[isleap(tm->tm_year)][tm->tm_mon - 1]);

That clamp is why '2024-01-31'::timestamp + interval '1 month' is 2024-02-29, not an error and not March 2. Days are added through the Julian day number (date2j/j2date), and only the microseconds are a plain int64 addition. For timestamptz, timestamptz_pl_interval_internal() recomputes the zone offset with DetermineTimeZoneOffset() after each calendar step, which is what makes “same wall-clock time tomorrow” work across a DST boundary.

interval_mul() cascades fractional results downward — a fractional month becomes days, a fractional day becomes seconds — using DAYS_PER_MONTH and SECS_PER_DAY and TSROUND(), but it never cascades back up. The comment is explicit: “We do NOT cascade up, since we are not forced to do so by the representation. The user can choose to cascade up later, using justify_hours and/or justify_days.”

Infinity

Like the timestamp types, interval reserves its extremes. But because it has three fields, all three must be at their extreme simultaneously:

 c
 
 
#define INTERVAL_NOBEGIN(i)	\
	do {	\
		(i)->time = PG_INT64_MIN;	\
		(i)->day = PG_INT32_MIN;	\
		(i)->month = PG_INT32_MIN;	\
	} while (0)

#define INTERVAL_IS_NOBEGIN(i)	\
	((i)->month == PG_INT32_MIN && (i)->day == PG_INT32_MIN && (i)->time == PG_INT64_MIN)

Every arithmetic function checks INTERVAL_NOT_FINITE() first, and interval 'infinity' - interval 'infinity' is an error, since the type has nothing equivalent to NaN.

Look at it yourself

The catalog row and the 16-byte size are both directly visible:

 sql
 
 
SELECT typname, typlen, typbyval, typalign, typarray
FROM pg_type
WHERE typname = 'interval';

SELECT pg_column_size(interval '1 month 2 days 3:04:05'); -- 16

The three-field representation shows through the arithmetic and the comparison rules:

 sql
 
 
SELECT interval '1 day 25 hours'; -- 1 day 25:00:00
SELECT justify_hours(interval '1 day 25 hours'); -- 2 days 01:00:00
SELECT justify_days(interval '35 days'); -- 1 mon 5 days
SELECT interval '1 month' = interval '30 days'; -- true

-- months are calendar-aware when added, but 30-day-normalized when compared
SELECT timestamp '2024-01-31' + interval '1 month'; -- 2024-02-29 00:00:00
SELECT extract(epoch FROM interval '1 month'); -- 2592000

That last line is the same 30-day assumption again: extract(epoch ...) in interval_part_common() computes month seconds from DAYS_PER_YEAR/DAYS_PER_MONTH, and DAYS_PER_MONTH is 30.

What it all means

An interval is typlen 16, typbyval f, typalign d, PLAIN storage: a fixed 16-byte struct of time (int64 µs), day (int32), month (int32).
The three fields are never combined at storage time, because the elapsed time depends on the absolute time it is applied to.
Comparison collapses the fields with a 30-day month and a 24-hour day, widened to INT128; hashing uses the same collapsed value (low 64 bits).
Addition and subtraction are field-wise and non-normalizing; justify_hours/justify_days/justify_interval normalize on request.
Adding an interval to a timestamp applies months first (with an end-of-month clamp), then days via Julian conversion, then microseconds.
The typmod packs a field-range bitmask in the high 16 bits and a sub-second precision in the low 16 bits.
Infinity requires all three fields at their extremes simultaneously.

PostgreSQL's bytea Implementation in C

Jay Hubbert — Sat, 05 Sep 2026 17:41:12 +0000

bytea is the one string type in PostgreSQL that is genuinely binary. It has no encoding, no collation, no length limit in the SQL sense, and no blank-padding semantics. That makes its C implementation almost suspiciously small: src/include/c.h gives it the same one-line treatment as text.

 c
 
 
typedef varlena bytea;
typedef varlena text;
typedef varlena BpChar;			/* blank-padded char, ie SQL char(n) */
typedef varlena VarChar;		/* var-length char, ie SQL varchar(n) */

So where does all the binary behavior live? In the I/O functions in src/backend/utils/adt/bytea.c and in one session GUC. This is a walk through the real bytes, based on the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entry

From src/include/catalog/pg_type.dat:

 c
 
 
{ oid => '17', array_type_oid => '1001',
 descr => 'variable-length string, binary values escaped',
 typname => 'bytea', typlen => '-1', typbyval => 'f', typcategory => 'U',
 typinput => 'byteain', typoutput => 'byteaout', typreceive => 'bytearecv',
 typsend => 'byteasend', typalign => 'i', typstorage => 'x' },

Every flag is the same shape as text: varlena, by reference, INT-aligned, EXTENDED storage (toastable). Three things are absent, and each is meaningful:

No typcollation. A bytea value cannot use collations. It is category U (user), not S (string), so COLLATE does not apply.
No typmod. There is no bytea(n).
typstorage => 'x', so it is compressible and out-of-line storable.

The varlena payload

bytea is exactly a varlena: a 4-byte length word (VARHDRSZ) followed by raw bytes. There is no terminator and no interpretation of the payload — a 0x00 is just a byte, not an end marker. src/include/varatt.h supplies the shared formats: the 4-byte aligned header, the 1-byte short header for payloads up to 126 bytes, the compressed-in-line header, and the external TOAST pointer.

The comparison and length functions read the payload with the alignment-oblivious macros, because a short varlena may sit at any byte offset in a tuple:

 c
 
	len1 = VARSIZE_ANY_EXHDR(arg1);
	len2 = VARSIZE_ANY_EXHDR(arg2);

	cmp = memcmp(VARDATA_ANY(arg1), VARDATA_ANY(arg2), Min(len1, len2));

Input: hex or escape

byteain accepts two textual forms and dispatches on the first two characters:

 c
 
 
	/* Recognize hex input */
	if (inputText[0] == '\\' && inputText[1] == 'x')
	{
		bc = (len - 2) / 2 + VARHDRSZ;	/* maximum possible length */
		result = palloc(bc);
		bc = hex_decode_safe(inputText + 2, len - 2, VARDATA(result),
							 escontext);
		SET_VARSIZE(result, bc + VARHDRSZ); /* actual length */

		PG_RETURN_BYTEA_P(result);
	}

The \x form is decoded by hex_decode_safe, which lives in src/backend/utils/adt/encode.c. It is SIMD-accelerated: hex_decode_safe processes two Vector8 blocks at a time and falls back to the scalar hex_decode_safe_scalar for the tail and for any input it could not handle vectorially. The scalar path also tolerates embedded spaces and newlines:

 c
 
 
		if (*s == ' ' || *s == '\n' || *s == '\t' || *s == '\r')
		{
			s++;
			continue;
		}

The other form is the traditional escape syntax. byteain walks the input and turns \ooo (three octal digits) into one byte, \\ into a backslash, and anything else into a literal byte:

 c
 
 
		else if ((tp[1] >= '0' && tp[1] <= '3') &&
				 (tp[2] >= '0' && tp[2] <= '7') &&
				 (tp[3] >= '0' && tp[3] <= '7'))
		{
			int			v;

			v = VAL(tp[1]);
			v <<= 3;
			v += VAL(tp[2]);
			v <<= 3;
			*rp++ = v + VAL(tp[3]);

			tp += 4;
		}
		else if (tp[1] == '\\')
		{
			*rp++ = '\\';
			tp += 2;
		}

Output: the `bytea_output` GUC

byteaout is where bytea becomes surprising. The output format is not a property of the type; it is a session setting, bytea_output, defined in src/backend/utils/misc/guc_parameters.dat with a default of BYTEA_OUTPUT_HEX:

 c
 
 
Datum
byteaout(PG_FUNCTION_ARGS)
{
	bytea	 *vlena = PG_GETARG_BYTEA_PP(0);
	char	 *result;
	char	 *rp;

	if (bytea_output == BYTEA_OUTPUT_HEX)
	{
		/* Print hex format */
		rp = result = palloc(VARSIZE_ANY_EXHDR(vlena) * 2 + 2 + 1);
		*rp++ = '\\';
		*rp++ = 'x';
		rp += hex_encode(VARDATA_ANY(vlena), VARSIZE_ANY_EXHDR(vlena), rp);
	}
	else if (bytea_output == BYTEA_OUTPUT_ESCAPE)
	{

The hex path calls hex_encode, the SIMD counterpart of the decoder in encode.c. The escape path is a hand-rolled state machine: a backslash is doubled, and any byte outside the printable ASCII range (< 0x20 or > 0x7e) becomes \ooo:

 c
 
 
		for (i = VARSIZE_ANY_EXHDR(vlena); i != 0; i--, vp++)
		{
			if (*vp == '\\')
			{
				*rp++ = '\\';
				*rp++ = '\\';
			}
			else if ((unsigned char) *vp < 0x20 || (unsigned char) *vp > 0x7e)
			{
				int			val;	/* holds unprintable chars */

				val = *vp;
				rp[0] = '\\';
				rp[3] = DIG(val & 07);
				val >>= 3;
				rp[2] = DIG(val & 07);
				val >>= 3;
				rp[1] = DIG(val & 03);
				rp += 4;
			}
			else
				*rp++ = *vp;
		}

The output buffer is sized by a first pass over the data, and byteaout rejects a result that would exceed MaxAllocSize. Crucially, this is a printing choice: SET bytea_output = 'escape' changes the text you see, but the stored varlena is byte-for-byte identical.

Binary recv/send: copy the bytes

The wire format is as simple as it gets. bytearecv copies the message payload verbatim, and byteasend is a documented special case:

 c
 
 
Datum
byteasend(PG_FUNCTION_ARGS)
{
	bytea	 *vlena = PG_GETARG_BYTEA_P_COPY(0);

	PG_RETURN_BYTEA_P(vlena);
}

The comment above it says “This is a special case: just copy the input…” — PG_GETARG_BYTEA_P_COPY makes an OK-to-write copy, and that copy is the output. There is no length prefix added beyond the varlena header; the receiving side reads the length from the header.

Comparison: memcmp plus a length tiebreak

Ordering is byte-by-byte unsigned comparison, then length. byteacmp is the clearest statement:

 c
 
 
	cmp = memcmp(VARDATA_ANY(arg1), VARDATA_ANY(arg2), Min(len1, len2));
	if ((cmp == 0) && (len1 != len2))
		cmp = (len1 < len2) ? -1 : 1;

bytealt and friends use the same shape with the length tiebreak folded into the boolean. The equality operators add a cheap fast path: they compare raw datum sizes with toast_raw_datum_size first, so two values of different uncompressed lengths are known unequal without detoasting either one.

Because there is no collation, every comparison is pure memcmp. That is why bytea is the one string type where = means “the bytes are identical” with no locale in the way.

Sort support and the HyperLogLog

The most interesting part of bytea.c is not the I/O; it is the sort support used to make ORDER BY on large bytea columns faster. bytea_sortsupport installs an abbreviated-key comparator, a converter, and an abort callback:

 c
 
 
	ssup->comparator = byteafastcmp;
	...
	ssup->abbrev_converter = bytea_abbrev_convert;
	ssup->abbrev_abort = bytea_abbrev_abort;

bytea_abbrev_convert packs the first sizeof(Datum) — 8 — bytes of the value into a single Datum, byte-swaps it so that unsigned integer comparison orders the byte strings the same way memcmp would, and returns it. Comparing 8-byte integers is far cheaper than memcmp over a large value.

But an 8-byte prefix is a lossy abbreviation: distinct values can collide. PostgreSQL hedges with HyperLogLog cardinality estimators. The ByteaSortSupport struct holds two of them:

 c
 
 
typedef struct
{
	hyperLogLogState abbr_card; /* Abbreviated key cardinality state */
	hyperLogLogState full_card; /* Full key cardinality state */
	double		prop_card;		/* Required cardinality proportion */
} ByteaSortSupport;

The converter hashes both the abbreviated key and (up to a cache line of) the full key into the two estimators. Then bytea_abbrev_abort periodically asks whether the abbreviation is actually helping:

 c
 
 
	abbrev_distinct = estimateHyperLogLog(&bss->abbr_card);
	key_distinct = estimateHyperLogLog(&bss->full_card);

If the abbreviated keys are as numerous as the full keys, the prefix is collapsing too many values together and the optimization is abandoned. It is a runtime cost/benefit decision driven by measured cardinality — a nice piece of engineering hiding in a “binary blob” type.

Look at it yourself

The GUC makes the round-trip visible. With standard_conforming_strings on, '\x...' is a literal backslash-x, which byteain recognizes:

 sql
 
 
-- hex is the default
SELECT '\x414243'::bytea; -- \x414243

SET bytea_output = 'escape';
SELECT '\x414243'::bytea; -- ABC
SELECT '\x0041'::bytea; -- \000A
SET bytea_output = 'hex';

-- length is in bytes, and NULs count
SELECT length('\x0041'::bytea); -- 2
SELECT octet_length('\x0041'::bytea); -- 2

-- no collation, no padding, raw bytes
SELECT '\x00'::bytea = '\x00'::bytea; -- true

pg_type confirms there is no collation to set:

 sql
 
 
SELECT typname, typlen, typbyval, typalign, typstorage, typcollation
FROM pg_type WHERE oid = 17;

What it all means

bytea is a plain varlena; all the binary behavior is in bytea.c.
Input accepts \x hex (SIMD-decoded) or the escape format; output is chosen by the session GUC bytea_output, not by the type.
Comparison is memcmp over the shorter value plus a length tiebreak, with a raw-size fast path for equality.
The wire format is a bare copy of the bytes; byteasend does no encoding.
The sort support uses 8-byte abbreviated keys guarded by HyperLogLog cardinality estimates — an adaptive optimization, not a fixed format.

PostgreSQL Computes Page Checksums Fast

Jay Hubbert — Sat, 05 Sep 2026 09:00:00 +0000

Every page PostgreSQL writes to disk gets a checksum: two bytes in the page header that let the server notice, on the next read, that the page it just read is not the page it wrote.

That sounds trivial. It is not, and the reason is a nice illustration of how a deceptively simple requirement turns into a performance problem. The checksum is computed over 8 kB of memory, on every page write and every page read, and the whole point of the feature is that it must be cheap enough to always leave on. A checksum that costs five percent of your I/O is not a checksum; it is a tax you eventually turn off.

So PostgreSQL does something quite clever: it computes thirty-two checksums at once, over a thirty-two-column view of the page, so that the latency of the one expensive instruction in the algorithm — the multiplication — is hidden behind the others. Thirty-two is not an arbitrary number. It is the number that fits in the register file.

This is the story of that design, in version 20devel (commit e75fcc9867f).

Where the checksum lives

Two bytes, in the page header:

 c
 
 
// src/include/storage/bufpage.h
typedef struct PageHeaderData
{
	/* XXX LSN is member of *any* block, not only page-organized ones */
	PageXLogRecPtr pd_lsn;		/* LSN: next byte after last byte of xlog
								 * record for last change to this page */
	uint16		pd_checksum;	/* checksum */
	uint16		pd_flags;		/* flag bits, see below */
	LocationIndex pd_lower;		/* offset to start of free space */
	LocationIndex pd_upper;		/* offset to end of free space */
	LocationIndex pd_special;	/* offset to start of special space */
	uint16		pd_pagesize_version;
	TransactionId pd_prune_xid; /* oldest prunable XID, or zero if none */
	ItemIdData	pd_linp[FLEXIBLE_ARRAY_MEMBER]; /* line pointer array */
} PageHeaderData;

The header comment above the struct explains the semantics, and they are subtler than they look:

pd_checksum stores the page checksum, if it has been set for this page; zero is a valid value for a checksum. If a checksum is not in use then we leave the field unset. This will typically mean the field is zero though non-zero values may also be present if databases have been pg_upgraded from releases prior to 9.3, when the same byte offset was used to store the current timelineid when the page was last updated. Note that there is no indication on a page as to whether the checksum is valid or not, a deliberate design choice which avoids the problem of relying on the page contents to decide whether to verify it. Hence there are no flag bits relating to checksums.

That last sentence is the design decision that makes the whole feature tractable: a page does not say “I have a checksum”. Whether a checksum is expected is a property of the cluster, not of the page. That means the verification code never has to guess, and it means enabling checksums on an existing cluster is a cluster-wide decision recorded in pg_control.

The states live in src/include/storage/checksum.h:

 c
 
 
// src/include/storage/checksum.h
/*
 * Checksum state 0 is used for when data checksums are disabled (OFF).
 * PG_DATA_CHECKSUM_INPROGRESS_{ON|OFF} defines that data checksums are either
 * currently being enabled or disabled, and PG_DATA_CHECKSUM_VERSION defines
 * that data checksums are enabled. The ChecksumStateType is stored in
 * pg_control so changing requires a catversion bump, and the values cannot
 * be reordered. New states must be added at the end.
 */
typedef enum ChecksumStateType
{
	PG_DATA_CHECKSUM_OFF = 0,
	PG_DATA_CHECKSUM_VERSION = 1,
	PG_DATA_CHECKSUM_INPROGRESS_OFF = 2,
	PG_DATA_CHECKSUM_INPROGRESS_ON = 3,
} ChecksumStateType;

Note the two “in progress” states. pg_checksums can be run offline to turn checksums on or off for a stopped cluster, but turning them on while the server is running has to be WAL-logged and crash-safe, so there is a state where checksums are written but not yet verified, and a state where they are verified but not yet written. Both matter for the “is a checksum expected here?” question.

The two runtime predicates are in src/backend/access/transam/xlog.c:

 c
 
 
// src/backend/access/transam/xlog.c
/*
 * DataChecksumsNeedWrite
 *		Returns whether data checksums must be written or not
 *
 * Returns true if data checksums are enabled, or are in the process of being
 * enabled. During "inprogress-on" and "inprogress-off" states checksums must
 * be written even though they are not verified (see datachecksum_state.c for
 * a longer discussion).
 *
 * This function is intended for callsites which are about to write a data page
 * to storage, and need to know whether to re-calculate the checksum for the
 * page header. Calling this function must be performed as close to the write
 * operation as possible to keep the critical section short.
 */
bool
DataChecksumsNeedWrite(void)
{
	return (LocalDataChecksumState == PG_DATA_CHECKSUM_VERSION ||
			LocalDataChecksumState == PG_DATA_CHECKSUM_INPROGRESS_ON ||
			LocalDataChecksumState == PG_DATA_CHECKSUM_INPROGRESS_OFF);
}

 c
 
 
// src/backend/access/transam/xlog.c
/*
 * DataChecksumsNeedVerify
 *		Returns whether data checksums must be verified or not
 *
 * Data checksums are only verified if they are fully enabled in the cluster.
 * During the "inprogress-on" and "inprogress-off" states they are only
 * updated, not verified (see datachecksum_state.c for a longer discussion).
 *
 * This function is intended for callsites which have read data and are about
 * to perform checksum validation based on the result of this. Calling this
 * function must be performed as close to the validation call as possible to
 * keep the critical section short. This is in order to protect against time of
 * check/time of use situations around data checksum validation.
 */
bool
DataChecksumsNeedVerify(void)
{
	return (LocalDataChecksumState == PG_DATA_CHECKSUM_VERSION);
}

Note the asymmetry: DataChecksumsNeedWrite() is true in three states (the “on” state and both “in progress” states), while DataChecksumsNeedVerify() is true only in the fully-enabled state. You must not verify a checksum that was never written.

The algorithm: FNV-1a, with a twist

The core of the algorithm is documented in one of the best comments in the PostgreSQL tree. From src/include/storage/checksum_impl.h:

 c
 
 
// src/include/storage/checksum_impl.h
/*
 * The algorithm used to checksum pages is chosen for very fast calculation.
 * Workloads where the database working set fits into OS file cache but not
 * into shared buffers can read in pages at a very fast pace and the checksum
 * algorithm itself can become the largest bottleneck.
 *
 * The checksum algorithm itself is based on the FNV-1a hash (FNV is shorthand
 * for Fowler/Noll/Vo). The primitive of a plain FNV-1a hash folds in data 1
 * byte at a time according to the formula:
 *
 *	 hash = (hash ^ value) * FNV_PRIME
 *
 * FNV-1a algorithm is described at http://www.isthe.com/chongo/tech/comp/fnv/
 *
 * PostgreSQL doesn't use FNV-1a hash directly because it has bad mixing of
 * high bits - high order bits in input data only affect high order bits in
 * output data. To resolve this we xor in the value prior to multiplication
 * shifted right by 17 bits. The number 17 was chosen because it doesn't
 * have common denominator with set bit positions in FNV_PRIME and empirically
 * provides the fastest mixing for high order bits of final iterations quickly
 * avalanche into lower positions. For performance reasons we choose to combine
 * 4 bytes at a time. The actual hash formula used as the basis is:
 *
 *	 hash = (hash ^ value) * FNV_PRIME ^ ((hash ^ value) >> 17)
 *
 * The main bottleneck in this calculation is the multiplication latency. To
 * hide the latency and to make use of SIMD parallelism multiple hash values
 * are calculated in parallel. The page is treated as a 32 column two
 * dimensional array of 32 bit values. Each column is aggregated separately
 * into a partial checksum. Each partial checksum uses a different initial
 * value (offset basis in FNV terminology). The initial values actually used
 * were chosen randomly, as the values themselves don't matter as much as that
 * they are different and don't match anything in real data. After initializing
 * partial checksums each value in the column is aggregated according to the
 * above formula. Finally two more iterations of the formula are performed with
 * value 0 to mix the bits of the last value added.
 *
 * The partial checksums are then folded together using xor to form a single
 * 32-bit checksum. The caller can safely reduce the value to 16 bits
 * using modulo 2^16-1. That will cause a very slight bias towards lower
 * values but this is not significant for the performance of the
 * checksum.

The base is FNV-1a: a one-byte-at-a-time hash with the formula

text

 hash = (hash ^ value) * FNV_PRIME

PostgreSQL does not use it unmodified, for the reason given: plain FNV-1a mixes badly — high-order bits of the input only affect high-order bits of the output, which is exactly the kind of weakness a checksum cannot afford when the input is a page full of mostly-zero bytes.

The fix is to fold in the value shifted as well:

text

 hash = (hash ^ value) * FNV_PRIME ^ ((hash ^ value) >> 17)

The shift by 17 is not arbitrary; the comment explains that 17 “doesn’t have common denominator with set bit positions in FNV_PRIME and empirically provides the fastest mixing for high order bits”.

And then the crucial decision:

For performance reasons we choose to combine 4 bytes at a time.

So the unit of work is a 32-bit word, not a byte. The formula is implemented as a macro:

 c
 
 
// src/include/storage/checksum_impl.h
/*
 * Calculate one round of the checksum.
 */
#define CHECKSUM_COMP(checksum, value) \
do { \
	uint32 __tmp = (checksum) ^ (value); \
	(checksum) = __tmp * FNV_PRIME ^ (__tmp >> 17); \
} while (0)

__tmp is the xor of the running hash and the new word. The hash becomes __tmp * FNV_PRIME, xored with __tmp >> 17. Note that the xor with the shifted value happens after the multiplication — the shift of the pre-multiplication value, which is the same as __tmp >> 17 because the multiplication is applied to __tmp in the first term only.

Why thirty-two hashes

Here is the paragraph that explains the whole design:

The main bottleneck in this calculation is the multiplication latency. To hide the latency and to make use of SIMD parallelism multiple hash values are calculated in parallel. The page is treated as a 32 column two dimensional array of 32 bit values. Each column is aggregated separately into a partial checksum. Each partial checksum uses a different initial value (offset basis in FNV terminology).

An integer multiply on a modern x86 has a latency of about three cycles and a throughput of one per cycle. The critical path in the checksum loop is: multiply (3 cycles) → xor → multiply → xor … If you have only one hash in flight, you are latency-bound and the multiply unit is idle most of the time. If you have several independent hashes in flight, they fill the gap: the CPU issues all the multiplies, then all the xors, and the latency of one is covered by the throughput of the others.

So the page is viewed as a matrix, and the columns are the parallel hashes:

text

 page bytes, viewed as uint32 words

 col 0: w[0] w[32] w[64] ... w[2016] ──▶ hash 0
 col 1: w[1] w[33] w[65] ... w[2017] ──▶ hash 1
 col 2: w[2] w[34] w[66] ... w[2018] ──▶ hash 2
 ...
 col 31: w[31] w[63] w[95] ... w[2047] ──▶ hash 31

Each column is a contiguous run of words in the page, strided by 32 words. The 32 hashes march through their columns in lockstep, one word per row, and each row’s 32 multiplies are independent.

The implementation of that view is a union:

 c
 
 
// src/include/storage/checksum_impl.h
#include "storage/bufpage.h"

/* number of checksums to calculate in parallel */
#define N_SUMS 32
/* prime multiplier of FNV-1a hash */
#define FNV_PRIME 16777619

/* Use a union so that this code is valid under strict aliasing */
typedef union
{
	PageHeaderData phdr;
	uint32		data[BLCKSZ / (sizeof(uint32) * N_SUMS)][N_SUMS];
} PGChecksummablePage;

/*
 * Base offsets to initialize each of the parallel FNV hashes into a
 * different initial state.
 */
static const uint32 checksumBaseOffsets[N_SUMS] = {
	0x5B1F36E9, 0xB8525960, 0x02AB50AA, 0x1DE66D2A,
	0x79FF467A, 0x9BB9F8A3, 0x217E7CD2, 0x83E13D2C,
	0xF8D4474F, 0xE39EB970, 0x42C6AE16, 0x993216FA,
	0x7B093B5D, 0x98DAFF3C, 0xF718902A, 0x0B1C9CDB,
	0xE58F764B, 0x187636BC, 0x5D7B3BB1, 0xE73DE7DE,
	0x92BEC979, 0xCCA6C0B2, 0x304A0979, 0x85AA43D4,
	0x783125BB, 0x6CA8EAA2, 0xE407EAC6, 0x4B5CFC3E,
	0x9FBF8C76, 0x15CA20BE, 0xF2CA9FD3, 0x959BD756
};

Two things are hiding in there.

The union is not incidental — it is a strict-aliasing workaround. The comment says so: “Use a union so that this code is valid under strict aliasing”. The page is a char *; the code wants to read it as an array of uint32. If you did that with a cast and a pointer, the compiler would be entitled to assume that the uint32 accesses never alias the char accesses and could reorder or elide them — technically undefined behavior, and the kind that produces a checksum that is correct on one compiler and wrong on another. Putting the two views in a union is the standard C way of saying “these are the same bytes, and I know what I’m doing”. (PostgreSQL also compiles with -fno-strict-aliasing in some configurations, but the union is the portable answer, and it is what the code does.)

The array shape is the algorithm. data[BLCKSZ / (sizeof(uint32) * N_SUMS)][N_SUMS] is a 64-by-32 array of uint32 for an 8 kB page: 8192 / (4 * 32) = 64 rows, 32 columns. The first index is the row (which step along a column) and the second is the column (which hash). That is why page->data[i][j] in the main loop means “row i, column j” and why the inner loop over j is the vectorizable one.

The initial values for the 32 hashes are constants:

 c
 
 
// src/include/storage/checksum_impl.h
/*
 * Base offsets to initialize each of the parallel FNV hashes into a
 * different initial state.
 */
static const uint32 checksumBaseOffsets[N_SUMS] = {
	0x5B1F36E9, 0xB8525960, 0x02AB50AA, 0x1DE66D2A,
	0x79FF467A, 0x9BB9F8A3, 0x217E7CD2, 0x83E13D2C,
	0xF8D4474F, 0xE39EB970, 0x42C6AE16, 0x993216FA,
	0x7B093B5D, 0x98DAFF3C, 0xF718902A, 0x0B1C9CDB,
	0xE58F764B, 0x187636BC, 0x5D7B3BB1, 0xE73DE7DE,
	0x92BEC979, 0xCCA6C0B2, 0x304A0979, 0x85AA43D4,
	0x783125BB, 0x6CA8EAA2, 0xE407EAC6, 0x4B5CFC3E,
	0x9FBF8C76, 0x15CA20BE, 0xF2CA9FD3, 0x959BD756
};

and the comment above them says exactly what they are for:

Base offsets to initialize each of the parallel FNV hashes into a different initial state.

The header comment adds a note that is easy to miss and quite philosophical:

Each partial checksum uses a different initial value (offset basis in FNV terminology). The initial values actually used were chosen randomly, as the values themselves don’t matter as much as that they are different and don’t match anything in real data.

So the constants are not magic numbers with a story; they are just 32 different random 32-bit numbers, chosen so that no two hashes start in the same state.

The core loop

The actual algorithm lives in a separate header, checksum_block_internal.h, which is #included — as a body, not as a function — into whichever implementation is being compiled. That is the trick that lets the same algorithm be compiled four ways (see below) without any duplication.

 c
 
 
// src/include/storage/checksum_block_internal.h
/* there is deliberately not an #ifndef CHECKSUM_BLOCK_INTERNAL_H here */

uint32		sums[N_SUMS];
uint32		result = 0;
uint32		i,
			j;

/* ensure that the size is compatible with the algorithm */
Assert(sizeof(PGChecksummablePage) == BLCKSZ);

/* initialize partial checksums to their corresponding offsets */
memcpy(sums, checksumBaseOffsets, sizeof(checksumBaseOffsets));

/* main checksum calculation */
for (i = 0; i < (uint32) (BLCKSZ / (sizeof(uint32) * N_SUMS)); i++)
	for (j = 0; j < N_SUMS; j++)
		CHECKSUM_COMP(sums[j], page->data[i][j]);

/* finally add in two rounds of zeroes for additional mixing */
for (i = 0; i < 2; i++)
	for (j = 0; j < N_SUMS; j++)
		CHECKSUM_COMP(sums[j], 0);

/* xor fold partial checksums together */
for (i = 0; i < N_SUMS; i++)
	result ^= sums[i];

return result;

Read it as four phases.

Phase 1: initialize. The 32 partial checksums start at their base offsets.

Phase 2: the main loop. for i in 64 rows, for j in 32 columns: CHECKSUM_COMP(sums[j], page->data[i][j]). The inner loop over j is the vectorizable one: 32 independent multiply-xor chains, one per column. The compiler can put eight of them in a 256-bit AVX2 register (or four in a 128-bit SSE2 register) and issue a vector multiply.

Phase 3: two rounds of zeroes. CHECKSUM_COMP(sums[j], 0) twice. The header comment explains why:

Finally two more iterations of the formula are performed with value 0 to mix the bits of the last value added.

Without this, the last word added to each column would have a weaker effect on the final result than earlier words. Two extra rounds of the same mixing operation push the influence of the last real word further down the mix.

Phase 4: the fold. All 32 partial checksums are xored together into a single 32-bit value:

The partial checksums are then folded together using xor to form a single 32-bit checksum.

Note this is an XOR, not an addition. That matters: XOR is associative, commutative, and order-independent, so the fold is exact and there is no carry propagation between columns. It also means the fold is itself embarrassingly parallel — though at 32 elements, that hardly matters.

From 32 bits to 16

The fold produces a 32-bit value. The page header has sixteen bits. The final mapping is in pg_checksum_page():

 c
 
 
// src/include/storage/checksum_impl.h
/*
 * Compute the checksum for a Postgres page.
 *
 * The page must be adequately aligned (at least on a 4-byte boundary).
 * Beware also that the checksum field of the page is transiently zeroed.
 *
 * The checksum includes the block number (to detect the case where a page is
 * somehow moved to a different location), the page header (excluding the
 * checksum itself), and the page data.
 */
uint16
pg_checksum_page(char *page, BlockNumber blkno)
{
	PGChecksummablePage *cpage = (PGChecksummablePage *) page;
	uint16		save_checksum;
	uint32		checksum;

	/* We only calculate the checksum for properly-initialized pages */
	Assert(!PageIsNew((Page) page));

	/*
	 * Save pd_checksum and temporarily set it to zero, so that the checksum
	 * calculation isn't affected by the old checksum stored on the page.
	 * Restore it after, because actually updating the checksum is NOT part of
	 * the API of this function.
	 */
	save_checksum = cpage->phdr.pd_checksum;
	cpage->phdr.pd_checksum = 0;
	checksum = pg_checksum_block(cpage);
	cpage->phdr.pd_checksum = save_checksum;

	/* Mix in the block number to detect transposed pages */
	checksum ^= blkno;

	/*
	 * Reduce to a uint16 (to fit in the pd_checksum field) with an offset of
	 * one. That avoids checksums of zero, which seems like a good idea.
	 */
	return (uint16) ((checksum % 65535) + 1);
}

Several things are going on here.

The checksum is not self-referential. The header comment says it, and the code does it:

 c
 
 
	save_checksum = cpage->phdr.pd_checksum;
	cpage->phdr.pd_checksum = 0;
	checksum = pg_checksum_block(cpage);
	cpage->phdr.pd_checksum = save_checksum;

The old checksum is saved, the field is zeroed, the checksum is computed over the page including the (now zero) checksum field, and the old value is restored — because, as the comment notes, “actually updating the checksum is NOT part of the API of this function”. The caller does that.

Zeroing the field rather than excluding it from the hash is simpler, and it means the checksum covers the whole 8 kB without a special case for two bytes.

The block number is mixed in afterwards:

 c
 
 
	/* Mix in the block number to detect transposed pages */
	checksum ^= blkno;

A page copied to the wrong position in the file — a “transposed page” — would otherwise have a perfectly valid checksum, because its contents are intact. Xoring in the block number means such a page fails verification.

The reduction to sixteen bits avoids zero:

 c
 
 
	/*
	 * Reduce to a uint16 (to fit in the pd_checksum field) with an offset of
	 * one. That avoids checksums of zero, which seems like a good idea.
	 */
	return (uint16) ((checksum % 65535) + 1);

Modulo 65535 rather than 65536 keeps the result in 1..65535, so the stored value is never zero. And the header comment is upfront about the consequence:

The caller can safely reduce the value to 16 bits using modulo 2^16-1. That will cause a very slight bias towards lower values but this is not significant for the performance of the checksum.

The bias is real: 0xFFFF can never occur, and 0x0001 occurs slightly more often than its neighbors. For a corruption-detection checksum, that is irrelevant.

Where it is called from

Two places, and they are not symmetric.

Writing goes through PageSetChecksum():

 c
 
 
// src/backend/storage/page/bufpage.c
 * If the page is in shared buffers, it needs to be locked in at least
 * share-exclusive mode.
 *
 * If checksums are disabled, or if the page is not initialized, just
 * return. Otherwise compute and set the checksum.
 *
 * In the past this needed to be done on a copy of the page, due to the
 * possibility of e.g., hint bits being set concurrently. However, this is not
 * necessary anymore as hint bits won't be set while IO is going on.
 */
void
PageSetChecksum(Page page, BlockNumber blkno)
{
	HOLD_INTERRUPTS();
	/* If we don't need a checksum, just return */
	if (PageIsNew(page) || !DataChecksumsNeedWrite())
	{
		RESUME_INTERRUPTS();
		return;
	}

	((PageHeader) page)->pd_checksum = pg_checksum_page(page, blkno);
	RESUME_INTERRUPTS();
}

Note the HOLD_INTERRUPTS() / RESUME_INTERRUPTS() bracket, and note the comment above the function:

In the past this needed to be done on a copy of the page, due to the possibility of e.g., hint bits being set concurrently. However, this is not necessary anymore as hint bits won’t be set while IO is going on.

Reading goes through PageIsVerified():

 c
 
 
// src/backend/storage/page/bufpage.c
bool
PageIsVerified(PageData *page, BlockNumber blkno, int flags, bool *checksum_failure_p)
{
	const PageHeaderData *p = (const PageHeaderData *) page;
	size_t	 *pagebytes;
	bool		checksum_failure = false;
	bool		header_sane = false;
	uint16		checksum = 0;

	if (checksum_failure_p)
		*checksum_failure_p = false;

	/*
	 * Don't verify page data unless the page passes basic non-zero test
	 */
	if (!PageIsNew(page))
	{
		/*
		 * There shouldn't be any check for interrupt calls happening in this
		 * codepath, but just to be on the safe side we hold interrupts since
		 * if they did happen the data checksum state could change during
		 * verifying checksums, which could lead to incorrect verification
		 * results.
		 */
		HOLD_INTERRUPTS();
		if (DataChecksumsNeedVerify())
		{
			checksum = pg_checksum_page(page, blkno);

			if (checksum != p->pd_checksum)
			{
				checksum_failure = true;
				if (checksum_failure_p)
					*checksum_failure_p = true;
			}
		}
		RESUME_INTERRUPTS();

		/*
		 * The following checks don't prove the header is correct, only that
		 * it looks sane enough to allow into the buffer pool. Later usage of
		 * the block can still reveal problems, which is why we offer the
		 * checksum option.
		 */
		if ((p->pd_flags & ~PD_VALID_FLAG_BITS) == 0 &&
			p->pd_lower <= p->pd_upper &&
			p->pd_upper <= p->pd_special &&
			p->pd_special <= BLCKSZ &&
			p->pd_special == MAXALIGN(p->pd_special))
			header_sane = true;

		if (header_sane && !checksum_failure)
			return true;
	}

Note the order of the checks. The checksum is verified first, and then the header is checked for sanity — but the sanity check is explicitly not treated as proof:

 c
 
 
		/*
		 * The following checks don't prove the header is correct, only that
		 * it looks sane enough to allow into the buffer pool. Later usage of
		 * the block can still reveal problems, which is why we offer the
		 * checksum option.
		 */

And the all-zero page case is handled separately, because a zeroed page is a legitimate thing to find (a backend extended a relation and crashed before writing any WAL about it).

Runtime dispatch: the AVX2 path

Now the part that makes this a performance story rather than an algorithm story.

The checksum algorithm is a few dozen lines, but it is compiled into several implementations, and the right one is chosen at runtime.

 c
 
 
// src/backend/storage/page/checksum.c
#include "postgres.h"

#include "port/pg_cpu.h"
#include "storage/checksum.h"
/*
 * The actual code is in storage/checksum_impl.h. This is done so that
 * external programs can incorporate the checksum code by #include'ing
 * that file from the exported Postgres headers. (Compare our legacy
 * CRC code in pg_crc.h.)
 * The PG_CHECKSUM_INTERNAL symbol allows core to use hardware-specific
 * coding without affecting external programs.
 */
#define PG_CHECKSUM_INTERNAL
#include "storage/checksum_impl.h"	/* IWYU pragma: keep */


static uint32
pg_checksum_block_fallback(const PGChecksummablePage *page)
{
#include "storage/checksum_block_internal.h"
}

/*
 * AVX2-optimized block checksum algorithm.
 */
#ifdef USE_AVX2_WITH_RUNTIME_CHECK
pg_attribute_target("avx2")
static uint32
pg_checksum_block_avx2(const PGChecksummablePage *page)
{
#include "storage/checksum_block_internal.h"
}
#endif							/* USE_AVX2_WITH_RUNTIME_CHECK */

/*
 * Choose the best available checksum implementation.
 */
static uint32
pg_checksum_choose(const PGChecksummablePage *page)
{
	pg_checksum_block = pg_checksum_block_fallback;

#ifdef USE_AVX2_WITH_RUNTIME_CHECK
	if (x86_feature_available(PG_AVX2))
		pg_checksum_block = pg_checksum_block_avx2;
#endif

	return pg_checksum_block(page);
}

static uint32 (*pg_checksum_block) (const PGChecksummablePage *page) = pg_checksum_choose;

Four things are going on.

The function pointer. pg_checksum_block is a function pointer, initialized to pg_checksum_choose. The first call to pg_checksum_page goes through the pointer, and pg_checksum_choose replaces the pointer with the best available implementation before doing the work. Every subsequent call is a direct-ish indirect call to the chosen implementation. The AVX2 commit message explains the tradeoff:

Similar to other hardware-specific paths, we set a function pointer on first use. We don’t bother to avoid this on platforms without AVX2 since the overhead of indirect calls doesn’t matter for multi-kilobyte inputs.

That is the right call. The input is 8 kB; an indirect call is worth about a nanosecond. The AVX2 win is worth hundreds.

The same source, four ways. The pg_checksum_block_* functions do not contain the algorithm; they #include it:

 c
 
#include "storage/checksum_block_internal.h"

So the algorithm is textually included in the fallback and in the AVX2 variant. There is exactly one copy of the algorithm, and it is compiled twice with different compiler flags. That is the whole reason for the awkward “include a body” idiom.

The AVX2 variant uses a function attribute. pg_attribute_target("avx2") expands to __attribute__((target("avx2"))), which tells the compiler “compile this function for AVX2, even though the rest of the program was not compiled for AVX2”. The attribute is applied to a single function, so the rest of the binary remains runnable on a pre-AVX2 CPU. The macro itself:

 c
 
 
// src/include/c.h
 * pg_attribute_target allows specifying different target options that the
 * function should be compiled with (e.g., for using special CPU instructions).
 * Note that there still needs to be a configure-time check to verify that a
 * specific target is understood by the compiler.
 */
#if __has_attribute (target)
#define pg_attribute_target(...) __attribute__((target(__VA_ARGS__)))
#else
#define pg_attribute_target(...)
#endif

Note the comment’s warning: “Note that there still needs to be a configure-time check to verify that a specific target is understood by the compiler.” That is exactly what PGAC_AVX2_SUPPORT() in configure.ac is for.

Whether the compiler supports it is decided at configure time:

 c
 
 
// configure.ac
# Check for AVX2 target support
#
if test x"$host_cpu" = x"x86_64"; then
 PGAC_AVX2_SUPPORT()
 if test x"$pgac_avx2_support" = x"yes"; then
 AC_DEFINE(USE_AVX2_WITH_RUNTIME_CHECK, 1, [Define to 1 to use AVX2 instructions with a runtime check.])
 fi
fi

and the runtime check uses the CPUID-based feature detection in src/include/port/pg_cpu.h:

 c
 
 
// src/include/port/pg_cpu.h
#if defined(__x86_64__) || defined(__i386__)

typedef enum X86FeatureId
{
	/* Have we run feature detection? */
	INIT_PG_X86,

	/* scalar registers and 128-bit XMM registers */
	PG_SSE4_2,
	PG_POPCNT,

	/* 256-bit YMM registers */
	PG_AVX2,

	/* 512-bit ZMM registers */
	PG_AVX512_BW,
	PG_AVX512_VL,
	PG_AVX512_VPCLMULQDQ,
	PG_AVX512_VPOPCNTDQ,

	/* identification */
	PG_HYPERVISOR,

	/* Time-Stamp Counter (TSC) flags */
	PG_RDTSCP,
	PG_TSC_INVARIANT,

x86_feature_available(PG_AVX2) triggers set_x86_features() on first use and then reads a cached array.

The fallback relies on the compiler. For a build without AVX2 — or on a non-x86 platform — the fallback implementation is compiled with -funroll-loops -ftree-vectorize, applied to this one object file:

 make
 
 
# src/backend/storage/page/Makefile
# Provide special optimization flags for checksum.c
checksum.o: CFLAGS += ${CFLAGS_UNROLL_LOOPS} ${CFLAGS_VECTORIZE}

The header comment explains why those two flags are the whole trick:

Vectorization of the algorithm works best with a 32bit x 32bit -> 32bit vector integer multiplication instruction, Examples include x86 AVX2 extensions (vpmulld) and ARM NEON (vmul.i32). Without that, vectorization is still possible if the compiler can turn multiplication by FNV_PRIME into a sequence of vectorized shifts and adds. For simplicity we rely on the compiler to do the vectorization for us. For GCC and clang the flags -funroll-loops -ftree-vectorize are enough to achieve vectorization.

So: on AVX2 hardware you get real vector multiply (vpmulld), eight columns per instruction. Elsewhere you get the compiler’s best attempt at expressing the constant multiply x * 16777619 as shifts and adds, which is what SSE2 has to do — SSE2 has no 32-bit vector multiply at all.

Why thirty-two, exactly?

The header comment is explicit:

The parallelism number 32 was chosen based on the fact that it is the largest state that fits into architecturally visible x86 SSE registers while leaving some free registers for intermediate values. For processors with 256-bit vector registers this leaves some performance on the table.

So: 32 partial hashes = 32 × 32 bits = 1024 bits of state. In 128-bit XMM registers that is eight registers — half of x86-64’s sixteen XMM registers, leaving the other half free for the loop’s intermediate values. With AVX2’s 256-bit YMM registers you could hold twice as much state, but you cannot: the number 32 is baked into the algorithm.

And it cannot be unbaked:

The number chosen is a fixed part of the algorithm because changing the parallelism changes the checksum result.

Caveats

N_SUMS is part of the on-disk format

This is the big one. N_SUMS is 32, the base offsets are a fixed table, and the fold is a fixed XOR. The resulting 16-bit value is stored in pd_checksum on every page of every relation in the cluster. If you changed N_SUMS to 16, or reordered the base offsets, or changed the prime, every existing page in every existing cluster would fail verification. The checksum algorithm is not an implementation detail; it is a storage format.

This is why the algorithm lives in checksum_impl.h with the comment:

This file exists for the benefit of external programs that may wish to check Postgres page checksums. They can #include this to get the code referenced by storage/checksum.h.

External tools — pg_checksums, the pageinspect extension, backup and restore tooling, third-party extensions — need to compute the same value, so the algorithm is published as a header that anyone can include. The PG_CHECKSUM_INTERNAL symbol then splits it:

 c
 
 
// src/include/storage/checksum_impl.h
/*
 * Block checksum algorithm. The page must be adequately aligned
 * (at least on 4-byte boundary).
 */
#ifdef PG_CHECKSUM_INTERNAL
/* definitions in src/backend/storage/page/checksum.c */
static uint32 (*pg_checksum_block) (const PGChecksummablePage *page);

#else
/* static definition for external programs */
static uint32
pg_checksum_block(const PGChecksummablePage *page)
{
#include "storage/checksum_block_internal.h"
}

#endif

With PG_CHECKSUM_INTERNAL defined (as in src/backend/storage/page/checksum.c), the core gets the runtime-dispatched function pointer. Without it, an external program gets a plain static function and never has to know that dispatch exists.

What the checksum does and does not cover

It covers the whole page: the page header (with the checksum field zeroed), the line pointer array, the tuple data, and the special space. It does not cover the block number — the block number is mixed in afterwards, by XOR.

It cannot detect all corruption. Sixteen bits means a false-negative rate of roughly 1 in 65535 for a random corruption; the header comment quotes the measured figure:

Tests of 1-4 uncorrelated bit errors, stray 0 and 0xFF bytes, overwriting page from random position to end with 0 bytes, and overwriting random segments of page with 0x00, 0xFF and random data all show optimal 2e-16 false positive rate within margin of error.

(That “false positive rate” is the rate of undetected corruption — the terminology in the comment is from the perspective of “the checksum says OK when the data is bad”.)

And it is not a substitute for WAL. A page checksum detects that this page is corrupt; it cannot tell you what the page should have contained, and it cannot undo the damage. Recovery from corruption still requires a restore from backup or a pg_checksums-style surgical repair.

The cost is real

Every dirty page written to disk pays for the checksum, and every page read from disk pays for the verification. For a write-heavy workload on slow storage, that is measurable. That is why checksums are opt-in at initdb time (initdb --data-checksums), and why the pg_checksums utility exists to turn them on or off offline for an existing cluster.

The AVX2 commit message names the case where the cost hurts most:

This matters most when using io_uring since in that case the checksum computation is not done in parallel by IO workers.

Which is to say: the checksum is computed on the path to disk, and anything that adds work to that path shows up.

The “in progress” states are subtle

A cluster in the PG_DATA_CHECKSUM_INPROGRESS_ON state writes checksums but does not verify them; a cluster in PG_DATA_CHECKSUM_INPROGRESS_OFF state does neither. Getting this wrong means either failing to notice corruption or failing to read your own data, so the state machine is stored in pg_control and the transitions are WAL-logged.

Sixteen bits is not a lot

The stored value is a 16-bit checksum, and the reduction is not uniform: checksum % 65535 + 1 can produce 1..65535, so 0x0000 is impossible and the distribution is very slightly skewed. It is fine for detecting “this page is garbage”, and it is not fine for anything stronger. Do not confuse it with a cryptographic digest.

Look at it yourself

The pg_checksums utility is the supported way to inspect and repair checksums on a stopped cluster:

pg_checksums --check -D /var/lib/postgresql/16/main
pg_checksums --enable -D /var/lib/postgresql/16/main

pg_controldata shows the cluster’s checksum state:

pg_controldata /var/lib/postgresql/16/main | grep -i checksum

And to watch the algorithm run, look at the assembly of the fallback implementation — the pmulld/vpmulld in the AVX2 object file is the whole point:

objdump -d src/backend/storage/page/checksum.o | less

What it all means in practice

The checksum algorithm is a latency problem, not a throughput problem. The single expensive operation is a 32-bit multiply, and multiply latency — not bandwidth — is what limits a dependent chain of hashes. The fix is not a faster multiply; it is thirty-two independent chains so the CPU’s out-of-order engine can fill the pipeline. That is a general-purpose optimization that applies to any hash-like construction.

The page is treated as a matrix, and the matrix shape is the algorithm. Thirty-two columns, sixty-four rows. The inner loop walks across columns, and that is the loop that vectorizes. It is a staggered layout — column j reads words j, j+32, j+64 — so it is not contiguous, but it is perfectly regular, which is what the compiler and the vectorizer need.

Register pressure is a design constraint, not an afterthought. The number 32 came from “the largest state that fits into architecturally visible x86 SSE registers while leaving some free registers for intermediate values”. AVX2 could double it. It cannot, because the constant is part of the on-disk format. This is the general shape of optimizing a persistent format: the moment you define it, your implementation choices freeze.

Runtime dispatch with a function pointer is the pragmatic way to get AVX2 without breaking old CPUs. Compile one function with __attribute__((target("avx2"))), detect AVX2 with CPUID, and swap the pointer on first use. The overhead is one indirect call per 8 kB page, which is nothing.

The correctness argument rests on the compiler. The fallback implementation is not hand-written SIMD; it is the same C code with -funroll-loops -ftree-vectorize, trusting GCC or clang to vectorize the constant multiplication into shifts and adds. The result must be bit-identical to the AVX2 path, and it is, because both are the same source text.

PostgreSQL's Integer Type Implementation in C (int2, int4, int8)

Jay Hubbert — Thu, 03 Sep 2026 15:13:13 +0000

PostgreSQL has three signed integer widths: int2 (2 bytes), int4 (4 bytes), and int8 (8 bytes). They are not three separate designs. They are one design instantiated three times: a fixed-length, pass-by-value Datum holding a plain C integer, with a text parser that accepts decimal and hex/octal/ binary, a text formatter, a binary send/recv pair, and an entire layer of overflow-checked arithmetic so that SELECT 2147483647 + 1 raises an error instead of wrapping to a negative number.

This is based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entries

Three entries in src/include/catalog/pg_type.dat — OIDs 21, 23, and 20:

 text
 
 
{ oid => '21', array_type_oid => '1005',
 typname => 'int2', typlen => '2', typbyval => 't', typcategory => 'N',
 typinput => 'int2in', typoutput => 'int2out', typreceive => 'int2recv',
 typsend => 'int2send', typalign => 's' },
{ oid => '23', array_type_oid => '1007',
 typname => 'int4', typlen => '4', typbyval => 't', typcategory => 'N',
 typinput => 'int4in', typoutput => 'int4out', typreceive => 'int4recv',
 typsend => 'int4send', typalign => 'i' },
{ oid => '20', array_type_oid => '1016',
 typname => 'int8', typlen => '8', typbyval => 't', typcategory => 'N',
 typinput => 'int8in', typoutput => 'int8out', typreceive => 'int8recv',
 typsend => 'int8send', typalign => 'd' },

The only differences are the widths and the alignment code. typalign is 's' (TYPALIGN_SHORT, 2 bytes), 'i' (TYPALIGN_INT, 4 bytes), and 'd' (TYPALIGN_DOUBLE, 8 bytes). All three are typbyval => 't', so a value is stored inline in the tuple, never as a pointer. All three take the default typstorage => 'p' (TYPSTORAGE_PLAIN): integers are never TOASTed.

There is also int2vector (OID 22), a variable-length array of int2 used inside system catalogs. It is a real type, not just int2[], and it has its own parser (int2vectorin) and its own layout rules.

The C representation

The three types are just C int16, int32, and int64. The Datum conversions in src/include/postgres.h are one-line casts:

 c
 
 
static inline int16
DatumGetInt16(Datum X) { return (int16) X; }

static inline int32
DatumGetInt32(Datum X) { return (int32) X; }

static inline int64
DatumGetInt64(Datum X) { return (int64) X; }

Because all three are typbyval, the value travels inside the Datum itself. An int8 is therefore 8 bytes on disk and 8 bytes in a register, with no indirection. The int2 and int4 forms are promoted to a wider C type whenever they are loaded, so arithmetic rarely operates on a 2-byte value.

Input: one parser, four radixes

int2in, int4in, and int8in are thin wrappers over three sibling functions in src/backend/utils/adt/numutils.c, declared in src/include/utils/builtins.h:

 c
 
 
PG_RETURN_INT16(pg_strtoint16_safe(num, fcinfo->context));
PG_RETURN_INT32(pg_strtoint32_safe(num, fcinfo->context));
PG_RETURN_INT64(pg_strtoint64_safe(num, fcinfo->context));

The parser has two paths. A fast path handles plain decimal; anything else falls through to a slow path that understands 0x/0X hex, 0o/0O octal, 0b/0B binary, and _ digit separators. Here is the fast path’s core, from pg_strtoint64_safe:

 c
 
 
	/* process remaining digits */
	for (;;)
	{
		digit = (*ptr - '0');

		if (digit >= 10)
			break;

		ptr++;

		if (unlikely(tmp > -(PG_INT64_MIN / 10)))
			goto out_of_range;

		tmp = tmp * 10 + digit;
	}

The header comment explains the trick: “Accumulate input as an unsigned number, to deal with two’s complement representation of the most negative number, which can’t be represented as a signed positive number.” So the accumulator is uint64, and the sign is applied at the end via pg_neg_u64_overflow. That is what lets -9223372036854775808 parse without special-casing the minimum value.

So SELECT '0x10'::int4 is 16, SELECT '0b101'::int4 is 5, and SELECT '1_000'::int4 is 1000 — the same parser backs all three widths.

Output: fixed-size buffers

The output functions format into a caller-sized buffer, then hand out a NUL-terminated string. int2out delegates to pg_ltoa through pg_itoa:

 c
 
 
Datum
int2out(PG_FUNCTION_ARGS)
{
	int16		arg1 = PG_GETARG_INT16(0);
	char	 *result = (char *) palloc(7);	/* sign, 5 digits, '\0' */

	pg_itoa(arg1, result);
	PG_RETURN_CSTRING(result);
}

int4out calls pg_ltoa directly, and int8out calls pg_lltoa, which first formats the magnitude as unsigned and then prepends the sign. The buffers are sized for the worst case: 7 bytes for int2, 12 for int4 (MAXINT8LEN is 20, plus a sign and NUL, for int8).

Binary format

The send/recv pair is the same shape for all three. int8recv and int8send (src/backend/utils/adt/int8.c):

 c
 
 
Datum
int8recv(PG_FUNCTION_ARGS)
{
	StringInfo	buf = (StringInfo) PG_GETARG_POINTER(0);

	PG_RETURN_INT64(pq_getmsgint64(buf));
}

Datum
int8send(PG_FUNCTION_ARGS)
{
	int64		arg1 = PG_GETARG_INT64(0);
	StringInfoData buf;

	pq_begintypsend(&buf);
	pq_sendint64(&buf, arg1);
	PG_RETURN_BYTEA_P(pq_endtypsend(&buf));
}

int2recv uses pq_getmsgint(buf, sizeof(int16)) and int4recv uses pq_getmsgint(buf, sizeof(int32)) — the length is passed explicitly so the wire width matches the type. Big-endian integers are byte-swapped by the pq_send* helpers, so the binary format is platform-independent.

Arithmetic: the part that is actually hard

Addition, subtraction, and multiplication do not do raw a + b. They call overflow-checked helpers in src/include/common/int.h, which are explicitly written to be “well defined C, without relying on compiler flags like -fwrapv”:

 c
 
 
Datum
int4pl(PG_FUNCTION_ARGS)
{
	int32		arg1 = PG_GETARG_INT32(0);
	int32		arg2 = PG_GETARG_INT32(1);
	int32		result;

	if (unlikely(pg_add_s32_overflow(arg1, arg2, &result)))
		ereport(ERROR,
				(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
				 errmsg("integer out of range")));
	PG_RETURN_INT32(result);
}

pg_add_s16_overflow, pg_add_s32_overflow, and pg_add_s64_overflow use __builtin_add_overflow when available, widen to a larger type otherwise, and return true on overflow. The same pattern guards pg_sub_*, pg_mul_*, and pg_neg_*, so int2pl, int2mi, int2mul, int4pl, and their int8 equivalents all raise ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE instead of wrapping.

Division has a special case because two’s complement cannot represent the result of INT_MIN / -1:

 c
 
 
	/*
	 * INT64_MIN / -1 is problematic, since the result can't be represented on a
	 * two's-complement machine. Some machines produce INT64_MIN, some produce
	 * zero, some throw an exception. We can dodge the problem by recognizing
	 * that division by -1 is the same as negation.
	 */
	if (arg2 == -1)
	{
		if (pg_neg_s64_overflow(arg1, &result))
			ereport(ERROR,
					(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
					 errmsg("bigint out of range")));
		PG_RETURN_INT64(result);
	}

The same comment appears in int4div and int2div. int8mod returns 0 directly for % -1 (the mathematically correct answer), and int8abs refuses PG_INT64_MIN because its absolute value cannot fit.

Casts and cross-type operators

There are explicit casts between the widths and to bool. i2toi4 widens without checking; i4toi2 narrows with a range check:

 c
 
 
Datum
i4toi2(PG_FUNCTION_ARGS)
{
	int32		arg1 = PG_GETARG_INT32(0);

	if (unlikely(arg1 < SHRT_MIN) || unlikely(arg1 > SHRT_MAX))
		ereturn(fcinfo->context, (Datum) 0,
				(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
				 errmsg("smallint out of range")));

	PG_RETURN_INT16((int16) arg1);
}

Comparisons between different widths live in int8.c (int84eq, int48eq, int82eq, int28eq, …) rather than being generated, and the catalog carries separate operator entries for each pair.

btree ordering

For indexes, the comparison must return an int32 sign. btint2cmp can subtract because an int16 difference fits in an int32:

 c
 
 
Datum
btint2cmp(PG_FUNCTION_ARGS)
{
	int16		a = PG_GETARG_INT16(0);
	int16		b = PG_GETARG_INT16(1);

	PG_RETURN_INT32((int32) a - (int32) b);
}

btint4cmp and btint8cmp cannot subtract safely and use explicit comparisons instead:

 c
 
 
Datum
btint8cmp(PG_FUNCTION_ARGS)
{
	int64		a = PG_GETARG_INT64(0);
	int64		b = PG_GETARG_INT64(1);

	if (a > b)
		PG_RETURN_INT32(A_GREATER_THAN_B);
	else if (a == b)
		PG_RETURN_INT32(0);
	else
		PG_RETURN_INT32(A_LESS_THAN_B);
}

The header of src/backend/access/nbtree/nbtcompare.c spells out the invariant: “it is critical that the comparison function impose a total order on all non-NULL values of the data type, and that the datatype’s boolean comparison operators yield results consistent with the comparison routine.” That is why these small functions exist at all.

Look at it yourself

 sql
 
 
SELECT '0x10'::int4, '0b101'::int4, '1_000'::int4; -- 16, 5, 1000
SELECT 32767::int2 + 1; -- ERROR: smallint out of range
SELECT 2147483647 + 1; -- ERROR: integer out of range
SELECT 9223372036854775807::int8 + 1; -- ERROR: bigint out of range
SELECT int8 '-9223372036854775808' / -1; -- ERROR: bigint out of range
SELECT 2147483647::int4::int2; -- ERROR: smallint out of range

The catalog side is just as easy to inspect:

 sql
 
 
SELECT typname, typlen, typbyval, typalign FROM pg_type
WHERE typname IN ('int2','int4','int8') ORDER BY typlen;

What it all means

int2/int4/int8 are three widths of one design: fixed-length, typbyval, plain-storage Datums.
The alignment codes (s/i/d) control tuple padding, not value semantics.
One parser (pg_strtoint*_safe) backs all three widths and accepts decimal, hex, octal, binary, underscores, and a leading +/-.
Arithmetic is wrapped in pg_*_overflow helpers so overflow raises ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE rather than wrapping.
INT_MIN / -1 and % -1 are special-cased because two’s complement makes the naive result undefined.

PostgreSQL Deduplicates B-Tree Entries

Jay Hubbert — Tue, 01 Sep 2026 09:00:00 +0000

Every index is a lie about multiplicity. A CREATE INDEX on a boolean column produces a structure in which, logically, there are only two distinct keys — but the index has to be able to answer “give me every row where is_active is true”, and there may be nine million of them. The naive answer is one index entry per row: nine million entries, nine million TIDs, nine million line pointers, and a B-tree whose leaves are almost entirely made of redundant copies of the same key.

PostgreSQL refuses to pay that. Instead of storing one entry per row, a B-tree leaf can store one entry with a list of TIDs — a posting list. The key appears once; the TIDs appear in a compact array. When that array gets too long, it is not promoted to some other structure and it does not cause the page to split; it is split in place, and the split is folded into the atomic action that inserted the offending row, so that concurrent inserter never sees a half-split list.

This is a walk through how it works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The logical index versus the physical index

The logical contents of a B-tree index never change. What changes is the physical representation of a leaf page. The nbtree README says it plainly:

Deduplication alters the physical representation of tuples without changing the logical contents of the index, and without adding overhead to read queries.

So there are two representations of “the key true occurs in rows 1, 2, 3, and 7”:

Representation	Physical form
Plain (non-posting) tuple	one index tuple per row, each with its own TID in `t_tid`
Posting list tuple	one index tuple, with the key followed by an array of TIDs

Both represent the same set of (key, TID) pairs. The scan code has to know which one it is looking at, which is why the format has an explicit “is this a posting list?” bit.

The tuple format

An index tuple starts with the standard IndexTupleData header. From src/include/access/itup.h:

 c
 
 
typedef struct IndexTupleData
{
	ItemPointerData t_tid;		/* item pointer to heap tuple */
	unsigned short t_info;		/* flag bits, first 11 bits are size of tuple */
	/* On machines where it might be too large, t_ctid is stored in two
	 * pieces, though it's always 6 bytes. t_info is 16 bits, so the
	 * offset number can only be 14 bits. We use the upper 2 bits of the
	 * offset number for the "has nulls" and "has varwidth" flags.
	 */
	/* Bits 0-10 are size in bytes of the variable-size part of the tuple,
	 * including any null bitmap. Bits 11-15 are the flag bits.
	 */
} IndexTupleData;

The key data follows. But a posting list tuple needs to store two things after the key: how many TIDs there are, and where the TID array starts. There is no room in t_info (its upper bits are already spoken for) and there is no room in the 6-byte t_tid either — unless you stop using t_tid for a TID at all.

And that is exactly what a posting list tuple does. It repurposes t_tid. From src/include/access/nbtree.h:

 c
 
 
// src/include/access/nbtree.h
/*
 * Posting tuple format (alternative non-pivot tuple representation):
 *
 * t_tid | t_info | key values | posting list (TID array)
 *
 * Posting list tuples are recognized as such by having the
 * INDEX_ALT_TID_MASK status bit set in t_info and the BT_IS_POSTING status
 * bit set in t_tid's offset number. These flags redefine the content of
 * the posting tuple's t_tid to store the location of the posting list
 * (instead of a block number), as well as the total number of heap TIDs
 * present in the tuple (instead of a real offset number).
 *
 * The 12 least significant bits from t_tid's offset number are used to
 * represent the number of heap TIDs present in the tuple, leaving 4 status
 * bits (the BT_STATUS_OFFSET_MASK bits). Like any non-pivot tuple, the
 * number of columns stored is always implicitly the total number in the
 * index (in practice there can never be non-key columns stored, since
 * deduplication is not supported with INCLUDE indexes).
 */

So the layout of a posting list tuple’s t_tid is:

Bits	Meaning
Block number	Byte offset of the TID array within the tuple
Offset number, low 12 bits	Number of TIDs in the posting list
Offset number, bit 12	`BT_PIVOT_HEAP_TID_ATTR` (unused here)
Offset number, bit 13	`BT_IS_POSTING` — this is a posting list tuple
Offset number, bits 14-15	unused

The masks:

 c
 
 
// src/include/access/nbtree.h
/* Item pointer offset bit masks */
#define BT_OFFSET_MASK				0x0FFF
#define BT_STATUS_OFFSET_MASK		0xF000
/* BT_STATUS_OFFSET_MASK status bits */
#define BT_PIVOT_HEAP_TID_ATTR		0x1000
#define BT_IS_POSTING				0x2000

And the accessors, which are just bit-fiddling on t_tid:

 c
 
 
// src/include/access/nbtree.h
static inline void
BTreeTupleSetPosting(IndexTuple itup, uint16 nhtids, int postingoffset)
{
	Assert(nhtids > 1);
	Assert((nhtids & BT_STATUS_OFFSET_MASK) == 0);
	Assert((size_t) postingoffset == MAXALIGN(postingoffset));
	Assert(postingoffset < INDEX_SIZE_MASK);
	Assert(!BTreeTupleIsPivot(itup));

	itup->t_info |= INDEX_ALT_TID_MASK;
	ItemPointerSetOffsetNumber(&itup->t_tid, (nhtids | BT_IS_POSTING));
	ItemPointerSetBlockNumber(&itup->t_tid, postingoffset);
}

static inline uint16
BTreeTupleGetNPosting(IndexTuple posting)
{
	OffsetNumber existing;

	Assert(BTreeTupleIsPosting(posting));

	existing = ItemPointerGetOffsetNumberNoCheck(&posting->t_tid);
	return (existing & BT_OFFSET_MASK);
}

static inline uint32
BTreeTupleGetPostingOffset(IndexTuple posting)
{
	Assert(BTreeTupleIsPosting(posting));

	return ItemPointerGetBlockNumberNoCheck(&posting->t_tid);
}

static inline ItemPointer
BTreeTupleGetPosting(IndexTuple posting)
{
	return (ItemPointer) ((char *) posting +
						 BTreeTupleGetPostingOffset(posting));
}

Two consequences fall out of this immediately:

The number of TIDs in a posting list is capped at 4095 (BT_OFFSET_MASK is 12 bits), even though the storage could hold far more.
The TID array is a plain array of ItemPointerData — six bytes each, no compression, no encoding. This is the single biggest difference from GIN, which compresses the same kind of array.

The posting list lives inside the tuple, immediately after the key columns, at a MAXALIGN’d offset. BTreeTupleGetHeapTID() returns the first TID of a posting list, and BTreeTupleGetMaxHeapTID() the last:

 c
 
 
// src/include/access/nbtree.h
static inline ItemPointer
BTreeTupleGetHeapTID(IndexTuple itup)
{
	if (BTreeTupleIsPivot(itup))
	{
		/* Pivot tuple heap TID representation? */
		if ((ItemPointerGetOffsetNumberNoCheck(&itup->t_tid) &
			 BT_PIVOT_HEAP_TID_ATTR) != 0)
			return (ItemPointer) ((char *) itup + IndexTupleSize(itup) -
								 sizeof(ItemPointerData));

		/* Heap TID attribute was truncated */
		return NULL;
	}
	else if (BTreeTupleIsPosting(itup))
		return BTreeTupleGetPosting(itup);

	return &itup->t_tid;
}

/*
 * Get maximum heap TID attribute, which could be the only TID in the case of
 * a non-pivot tuple that does not have a posting list.
 *
 * Works with non-pivot tuples only.
 */
static inline ItemPointer
BTreeTupleGetMaxHeapTID(IndexTuple itup)
{
	Assert(!BTreeTupleIsPivot(itup));

	if (BTreeTupleIsPosting(itup))
	{
		uint16		nposting = BTreeTupleGetNPosting(itup);

		return BTreeTupleGetPostingN(itup, nposting - 1);
	}

	return &itup->t_tid;
}

Building a posting list tuple

The function that builds one is _bt_form_posting(), in src/backend/access/nbtree/nbtdedup.c — a file that exists only in modern PostgreSQL, because the nbtree implementation was split up into fourteen files during the v20 development cycle.

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
/*
 * Build a posting list tuple based on caller's "base" index tuple and list of
 * heap TIDs. When nhtids == 1, builds a standard non-pivot tuple without a
 * posting list. (Posting list tuples can never have a single heap TID, partly
 * because that ensures that deduplication always reduces final MAXALIGN()'d
 * size of entire tuple.)
 *
 * Convention is that posting list starts at a MAXALIGN()'d offset (rather
 * than a SHORTALIGN()'d offset), in line with the approach taken when
 * appending a heap TID to new pivot tuple/high key during suffix truncation.
 * This sometimes wastes a little space that was only needed as alignment
 * padding in the original tuple. Following this convention simplifies the
 * space accounting used when deduplicating a page (the same convention
 * simplifies the accounting for choosing a point to split a page at).
 *
 * Note: Caller's "htids" array must be unique and already in ascending TID
 * order. Any existing heap TIDs from "base" won't automatically appear in
 * returned posting list tuple (they must be included in htids array.)
 */
IndexTuple
_bt_form_posting(IndexTuple base, const ItemPointerData *htids, int nhtids)
{
	uint32		keysize,
				newsize;
	IndexTuple	itup;

	if (BTreeTupleIsPosting(base))
		keysize = BTreeTupleGetPostingOffset(base);
	else
		keysize = IndexTupleSize(base);

	Assert(!BTreeTupleIsPivot(base));
	Assert(nhtids > 0 && nhtids <= PG_UINT16_MAX);
	Assert(keysize == MAXALIGN(keysize));

	/* Determine final size of new tuple */
	if (nhtids > 1)
		newsize = MAXALIGN(keysize +
						 nhtids * sizeof(ItemPointerData));
	else
		newsize = keysize;

	Assert(newsize <= INDEX_SIZE_MASK);
	Assert(newsize == MAXALIGN(newsize));

	/* Allocate memory using palloc0() (matches index_form_tuple()) */
	itup = palloc0(newsize);
	memcpy(itup, base, keysize);
	itup->t_info &= ~INDEX_SIZE_MASK;
	itup->t_info |= newsize;
	if (nhtids > 1)
	{
		/* Form posting list tuple */
		BTreeTupleSetPosting(itup, nhtids, keysize);
		memcpy(BTreeTupleGetPosting(itup), htids,
			 sizeof(ItemPointerData) * nhtids);
		Assert(_bt_posting_valid(itup));
	}
	else
	{
		/* Form standard non-pivot tuple */
		itup->t_info &= ~INDEX_ALT_TID_MASK;
		ItemPointerCopy(htids, &itup->t_tid);
		Assert(ItemPointerIsValid(&itup->t_tid));
	}

	return itup;
}

The size formula is the whole of the space accounting:

text

posting tuple size = MAXALIGN(key bytes + nhtids * 6)

For a boolean column, the key is 8 bytes of IndexTupleData plus one byte of datum, so MAXALIGN’d to 16. A posting list of 200 TIDs is therefore 16 + 1200 = 1216 bytes, versus 200 separate tuples at (16 + 4 bytes of line pointer) each = 4000 bytes. Three times smaller, and — more importantly — 200 line pointers become one.

The nhtids == 1 case deliberately degrades to a plain tuple rather than a one-element posting list. The comment explains why: a posting list tuple can never hold a single TID, “partly because that ensures that deduplication always reduces final MAXALIGN()’d size of entire tuple.” If you merged two 16-byte tuples into a posting list of two TIDs, you would get 16 + 12 = 28 bytes plus a line pointer — bigger than the two separate tuples (32 + 8 = 40 bytes, so actually smaller), but the point stands: there must never be a case where merging makes the page bigger.

When does deduplication happen?

Not eagerly. Deduplication is lazy, and it happens at exactly one moment: when an insert finds that the target leaf page has no room and a page split is about to happen.

The nbtree README:

Deduplication is always applied lazily, at the point where it would otherwise be necessary to perform a page split. It occurs only when LP_DEAD items have been removed, as our last line of defense against splitting a leaf page (bottom-up index deletion may be attempted first, as our second last line of defense).

The chain of decisions lives in _bt_delete_or_dedup_one_page() in src/backend/access/nbtree/nbtinsert.c:

 c
 
 
// src/backend/access/nbtree/nbtinsert.c
	/*
	 * Perform bottom-up index deletion pass when executor hint indicated that
	 * incoming item is logically unchanged, or for a unique index that is
	 * known to have physical duplicates for some other reason. ...
	 *
	 * Note: Bottom-up index deletion uses the same equality/equivalence
	 * routines as deduplication internally. However, it does not merge
	 * together index tuples, so the same correctness considerations do not
	 * apply. We deliberately omit an index-is-allequalimage test here.
	 */
	if ((indexUnchanged || uniquedup) &&
		_bt_bottomupdel_pass(rel, buffer, heapRel, insertstate->itemsz))
		return;

	/* Perform deduplication pass (when enabled and index-is-allequalimage) */
	if (BTGetDeduplicateItems(rel) && itup_key->allequalimage)
		_bt_dedup_pass(rel, buffer, insertstate->itup, insertstate->itemsz,
					 (indexUnchanged || uniquedup));

Two gates there, and they are the two conditions under which deduplication is possible at all:

deduplicate_items — the storage parameter, on by default:

 c
 
 
// src/include/access/nbtree.h
#define BTGetDeduplicateItems(relation) \
	(AssertMacro(relation->rd_rel->relkind == RELKIND_INDEX && \
				 relation->rd_rel->relam == BTREE_AM_OID), \
	((relation)->rd_options ? \
	 ((BTOptions *) (relation)->rd_options)->deduplicate_items : true))

allequalimage — whether the index’s operator class guarantees that “equal” implies “bytewise identical”. This is the interesting one, and it is discussed below.

The deduplication pass itself

_bt_dedup_pass() in nbtdedup.c rewrites the whole page. It copies the page to a temporary buffer, walks the tuples, groups the equal ones, forms posting list tuples for the groups, and then swaps the new page in. The critical part is the size limit:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
	/*
	 * Initialize deduplication state.
	 *
	 * It would be possible for maxpostingsize (limit on posting list tuple
	 * size) to be set to one third of the page. However, it seems like a
	 * good idea to limit the size of posting lists to one sixth of a page.
	 * That ought to leave us with a good split point when pages full of
	 * duplicates can be split several times.
	 */
	state = palloc_object(BTDedupStateData);
	state->deduplicate = true;
	state->nmaxitems = 0;
	state->maxpostingsize = Min(BTMaxItemSize / 2, INDEX_SIZE_MASK);

BTMaxItemSize is the maximum size of a single btree index entry:

 c
 
 
// src/include/access/nbtree.h
/*
 * Maximum size of a btree index entry, including its tuple header.
 *
 * We actually need to be able to fit three items on every page,
 * so restrict any one item to 1/3 the per-page available space.
 * ...
 */
#define BTMaxItemSize \
	(MAXALIGN_DOWN((BLCKSZ - \
					MAXALIGN(SizeOfPageHeaderData + 3*sizeof(ItemIdData)) - \
					MAXALIGN(sizeof(BTPageOpaqueData))) / 3) - \
					MAXALIGN(sizeof(ItemPointerData)))

On an 8 kB page that works out to 2696 bytes, so maxpostingsize is 1348 bytes — a bit under a sixth of the page, as the comment says. And 1348 bytes of 6-byte TIDs is at most 224 TIDs per posting list, less the space taken by the key.

This is the constant that bounds a posting list. There are actually two bounds, and they are different things:

Bound	Value	Where it comes from
`maxpostingsize`	1348 bytes (~1/6 page)	`Min(BTMaxItemSize / 2, INDEX_SIZE_MASK)`
TID count per list	≤ 4095	`BT_OFFSET_MASK`, 12 bits of the offset number
Derived TID count	≤ 224, less the key	1348 bytes / 6 bytes per TID

The 4095 limit is the hard ceiling; the 1348-byte limit is what actually binds in practice, and it is deliberately conservative so that a page full of posting lists can still be split sensibly.

The merge condition

Now the heart of it. While walking the page, the pass keeps a “pending posting list”: a base tuple plus a growing array of TIDs. Each subsequent tuple is either merged into the pending list, or terminates it:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
		else if (state->deduplicate &&
				 _bt_keep_natts_fast(rel, state->base, itup) > nkeyatts &&
				 _bt_dedup_save_htid(state, itup))
		{
			/*
			 * Tuple is equal to base tuple of pending posting list. Heap
			 * TID(s) for itup have been saved in state.
			 */
		}

Read that condition as three separate tests:

state->deduplicate — the pass has not given up (the “single value” strategy can turn it off; see below).
_bt_keep_natts_fast(rel, state->base, itup) > nkeyatts — the keys are equal. _bt_keep_natts_fast() returns the number of leading attributes that must be kept to distinguish two tuples; if it exceeds the number of key attributes, the two tuples have identical keys.
_bt_dedup_save_htid(state, itup) — the merged list still fits. This returns false if appending the TIDs would push the posting list over maxpostingsize.

That third test is the “posting list is not too long” condition, and it is worth reading closely because it is where the size limit is enforced:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
bool
_bt_dedup_save_htid(BTDedupState state, IndexTuple itup)
{
	int			nhtids;
	ItemPointer htids;
	Size		mergedtupsz;

	Assert(!BTreeTupleIsPivot(itup));

	if (!BTreeTupleIsPosting(itup))
	{
		nhtids = 1;
		htids = &itup->t_tid;
	}
	else
	{
		nhtids = BTreeTupleGetNPosting(itup);
		htids = BTreeTupleGetPosting(itup);
	}

	/*
	 * Don't append (have caller finish pending posting list as-is) if
	 * appending heap TID(s) from itup would put us over maxpostingsize limit.
	 *
	 * This calculation needs to match the code used within _bt_form_posting()
	 * for new posting list tuples.
	 */
	mergedtupsz = MAXALIGN(state->basetupsize +
						 (state->nhtids + nhtids) * sizeof(ItemPointerData));

	if (mergedtupsz > state->maxpostingsize)
	{
		/*
		 * Count this as an oversized item for single value strategy, though
		 * only when there are 50 TIDs in the final posting list tuple. This
		 * limit (which is fairly arbitrary) avoids confusion about how many
		 * 1/6 of a page tuples have been encountered/created by the current
		 * deduplication pass.
		 * ...
		 */
		if (state->nhtids > 50)
			state->nmaxitems++;

		return false;
	}

	/*
	 * Save heap TIDs to pending posting list tuple -- itup can be merged into
	 * pending posting list
	 */
	state->nitems++;
	memcpy(state->htids + state->nhtids, htids,
		 sizeof(ItemPointerData) * nhtids);
	state->nhtids += nhtids;
	state->phystupsize += MAXALIGN(IndexTupleSize(itup)) + sizeof(ItemIdData);

	return true;
}

When the limit is hit, the pending list is finalized as-is and a new pending list starts with the tuple that did not fit. That is exactly what produces a page with several posting lists of the same key rather than one enormous one.

Finishing a pending list

When a group ends, _bt_dedup_finish_pending() either writes the base tuple unchanged (if the group had only one member) or forms a posting list tuple:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
Size
_bt_dedup_finish_pending(Page newpage, BTDedupState state)
{
	OffsetNumber tupoff;
	Size		tuplesz;
	Size		spacesaving;

	Assert(state->nitems > 0);
	Assert(state->nitems <= state->nhtids);
	Assert(state->intervals[state->nintervals].baseoff == state->baseoff);

	tupoff = OffsetNumberNext(PageGetMaxOffsetNumber(newpage));
	if (state->nitems == 1)
	{
		/* Use original, unchanged base tuple */
		tuplesz = IndexTupleSize(state->base);
		...
		spacesaving = 0;
	}
	else
	{
		IndexTuple	final;

		/* Form a tuple with a posting list */
		final = _bt_form_posting(state->base, state->htids, state->nhtids);
		tuplesz = IndexTupleSize(final);
		Assert(tuplesz <= state->maxpostingsize);

		/* Save final number of items for posting list */
		state->intervals[state->nintervals].nitems = state->nitems;

		...
		spacesaving = state->phystupsize - (tuplesz + sizeof(ItemIdData));
		/* Increment nintervals, since we wrote a new posting list tuple */
		state->nintervals++;
		Assert(spacesaving > 0 && spacesaving < BLCKSZ);
	}

	/* Reset state for next pending posting list */
	state->nhtids = 0;
	state->nitems = 0;
	state->phystupsize = 0;

	return spacesaving;
}

Note spacesaving > 0: the code asserts that merging always made the page smaller. If it ever did not, merging would be a pessimization and the assert would fire.

The “single value” strategy

A page full of one repeated key is a special case. The dedup pass has a special strategy for it, described in _bt_do_singleval():

The general goal of this strategy is to ensure that nbtsplitloc.c (which uses its own single value strategy) will find a useful split point as further duplicates are inserted, and successive rightmost page splits occur among pages that store the same duplicate value. When the page finally splits, it should end up BTREE_SINGLEVAL_FILLFACTOR% full, just like it would if deduplication were disabled.

The strategy limits the page to at most six maxpostingsize-sized posting lists, and then stops deduplicating entirely. It is triggered when the page’s first and last data items both compare equal to the tuple being inserted — that is, the whole page holds one value:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
	itemid = PageGetItemId(page, minoff);
	itup = (IndexTuple) PageGetItem(page, itemid);

	if (_bt_keep_natts_fast(rel, newitem, itup) > nkeyatts)
	{
		itemid = PageGetItemId(page, PageGetMaxOffsetNumber(page));
		itup = (IndexTuple) PageGetItem(page, itemid);

		if (_bt_keep_natts_fast(rel, newitem, itup) > nkeyatts)
			return true;
	}

	return false;

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
			if (singlevalstrat)
			{
				/*
				 * Single value strategy's extra steps.
				 *
				 * Lower maxpostingsize for sixth and final large posting list
				 * tuple at the point where 5 maxpostingsize-capped tuples
				 * have either been formed or observed.
				 *
				 * When a sixth maxpostingsize-capped item is formed/observed,
				 * stop merging together tuples altogether. ...
				 */
				if (state->nmaxitems == 5)
					_bt_singleval_fillfactor(page, state, newitemsz);
				else if (state->nmaxitems == 6)
				{
					state->deduplicate = false;
					singlevalstrat = false; /* won't be back here */
				}
			}

This is a strange-looking piece of code that exists for a very practical reason: a page full of duplicates must eventually be split, and the split must land in a sensible place. The split-point logic (_bt_findsplitloc(), in nbtsplitloc.c) has its own “single value strategy” that leaves the left page 96% full, and the dedup pass has to cooperate with it.

Splitting a posting list

Now the interesting case: what happens when you insert a row whose TID falls inside the range of TIDs already covered by an existing posting list?

Physically, the posting list is a sorted array of TIDs, and the new row’s TID belongs somewhere in the middle of it. You cannot simply append it — the array must stay sorted, because everything downstream (binary searches, range checks, VACUUM) assumes it is.

PostgreSQL’s answer is _bt_swap_posting().

Finding the insertion point

The binary search over the page finds the tuple; if that tuple is a posting list, a second binary search over the posting list finds the offset at which the new TID belongs. From src/backend/access/nbtree/nbtsearch.c:

 c
 
 
// src/backend/access/nbtree/nbtsearch.c
		/*
		 * If tuple at offset located by binary search is a posting list whose
		 * TID range overlaps with caller's scantid, perform posting list
		 * binary search to set postingoff for caller. Caller must split the
		 * posting list when postingoff is set. This should happen
		 * infrequently.
		 */
		if (unlikely(result == 0 && key->scantid != NULL))
		{
			/*
			 * postingoff should never be set more than once per leaf page
			 * binary search. That would mean that there are duplicate table
			 * TIDs in the index, which is never okay. Check for that here.
			 */
			if (insertstate->postingoff != 0)
				ereport(ERROR,
						(errcode(ERRCODE_INDEX_CORRUPTED),
						 errmsg_internal("table tid from new index tuple (%u,%u) cannot find insert offset between offsets %u and %u of block %u in index \"%s\"",
										 ItemPointerGetBlockNumber(key->scantid),
										 ItemPointerGetOffsetNumber(key->scantid),
										 low, stricthigh,
										 BufferGetBlockNumber(insertstate->buf),
										 RelationGetRelationName(rel))));

			insertstate->postingoff = _bt_binsrch_posting(key, page, mid);
		}

and the helper itself:

 c
 
 
// src/backend/access/nbtree/nbtsearch.c
static int
_bt_binsrch_posting(BTScanInsert key, Page page, OffsetNumber offnum)
{
	IndexTuple	itup;
	ItemId		itemid;
	int			low,
				high,
				mid,
				res;

	/*
	 * If this isn't a posting tuple, then the index must be corrupt (if it is
	 * an ordinary non-pivot tuple then there must be an existing tuple with a
	 * heap TID that equals inserter's new heap TID/scantid). Defensively
	 * check that tuple is a posting list tuple whose posting list range
	 * includes caller's scantid.
	 * ...
	 */
	itemid = PageGetItemId(page, offnum);
	itup = (IndexTuple) PageGetItem(page, itemid);
	if (!BTreeTupleIsPosting(itup))
		return 0;

	Assert(key->heapkeyspace && key->allequalimage);

	/*
	 * In the event that posting list tuple has LP_DEAD bit set, indicate this
	 * to _bt_binsrch_insert() caller by returning -1, a sentinel value. A
	 * second call to _bt_binsrch_insert() can take place when its caller has
	 * removed the dead item.
	 */
	if (ItemIdIsDead(itemid))
		return -1;

	/* "high" is past end of posting list for loop invariant */
	low = 0;
	high = BTreeTupleGetNPosting(itup);
	Assert(high >= 2);

	while (high > low)
	{
		mid = low + ((high - low) / 2);
		res = ItemPointerCompare(key->scantid,
								 BTreeTupleGetPostingN(itup, mid));

		if (res > 0)
			low = mid + 1;
		else if (res < 0)
			high = mid;
		else
			return mid;
	}

	/* Exact match not found */
	return low;
}

The returned postingoff is the offset within the posting list where the new TID belongs.

The swap

Here is the whole of _bt_swap_posting():

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
/*
 * Prepare for a posting list split by swapping heap TID in newitem with heap
 * TID from original posting list (the 'oposting' heap TID located at offset
 * 'postingoff'). Modifies newitem, so caller should pass their own private
 * copy that can safely be modified.
 *
 * Returns new posting list tuple, which is palloc()'d in caller's context.
 * This is guaranteed to be the same size as 'oposting'. Modified newitem is
 * what caller actually inserts. (This happens inside the same critical
 * section that performs an in-place update of old posting list using new
 * posting list returned here.)
 * ...
 * See nbtree/README for details on the design of posting list splits.
 */
IndexTuple
_bt_swap_posting(IndexTuple newitem, IndexTuple oposting, int postingoff)
{
	int			nhtids;
	char	 *replacepos;
	char	 *replaceposright;
	Size		nmovebytes;
	IndexTuple	nposting;

	nhtids = BTreeTupleGetNPosting(oposting);
	Assert(_bt_posting_valid(oposting));

	/*
	 * The postingoff argument originated as a _bt_binsrch_posting() return
	 * value. It will be 0 in the event of corruption that makes a leaf page
	 * contain a non-pivot tuple that's somehow identical to newitem (no two
	 * non-pivot tuples should ever have the same TID). This has been known
	 * to happen in the field from time to time.
	 *
	 * Perform a basic sanity check to catch this case now.
	 */
	if (!(postingoff > 0 && postingoff < nhtids))
		elog(ERROR, "posting list tuple with %d items cannot be split at offset %d",
			 nhtids, postingoff);

	/*
	 * Move item pointers in posting list to make a gap for the new item's
	 * heap TID. We shift TIDs one place to the right, losing original
	 * rightmost TID. (nmovebytes must not include TIDs to the left of
	 * postingoff, nor the existing rightmost/max TID that gets overwritten.)
	 */
	nposting = CopyIndexTuple(oposting);
	replacepos = (char *) BTreeTupleGetPostingN(nposting, postingoff);
	replaceposright = (char *) BTreeTupleGetPostingN(nposting, postingoff + 1);
	nmovebytes = (nhtids - postingoff - 1) * sizeof(ItemPointerData);
	memmove(replaceposright, replacepos, nmovebytes);

	/* Fill the gap at postingoff with TID of new item (original new TID) */
	Assert(!BTreeTupleIsPivot(newitem) && !BTreeTupleIsPosting(newitem));
	ItemPointerCopy(&newitem->t_tid, (ItemPointer) replacepos);

	/* Now copy oposting's rightmost/max TID into new item (final new TID) */
	ItemPointerCopy(BTreeTupleGetMaxHeapTID(oposting), &newitem->t_tid);

	Assert(ItemPointerCompare(BTreeTupleGetMaxHeapTID(nposting),
							 BTreeTupleGetHeapTID(newitem)) < 0);
	Assert(_bt_posting_valid(nposting));

	return nposting;
}

Three steps, and they are worth tracing with concrete numbers. Suppose the posting list is [10, 20, 30, 40] and the new row’s TID is 25.

The binary search returns the offset at which 25 belongs, which is index 2 (between 20 and 30): _bt_binsrch_posting() walks low=0, high=4, probes index 2 (30), sees 25 < 30 and narrows to low=0, high=2, probes index 1 (20), sees 25 > 20 and sets low=2. The loop ends with high == low == 2, and 2 is returned.

Now the three steps, with nhtids = 4 and postingoff = 2:

Shift. nmovebytes = (4 - 2 - 1) * 6 = 6 bytes, so memmove() copies the TID at slot 2 (30) into slot 3. The array is now [10, 20, 30, 30] — slot 2 still holds its old value because it was the source of the copy, and it is about to be overwritten. The old maximum 40 has been pushed out of the array entirely (“losing original rightmost TID”).
Fill the hole. The new item’s original TID (25) is written into slot 2: [10, 20, 25, 30]. Still sorted, because the binary search guaranteed that 25 belongs exactly there.
Take over the maximum. The old maximum (40) is copied into the new item’s t_tid. The new index tuple on the page is now a plain tuple pointing at row 40, and the posting list tuple covers rows 10, 20, 25 and 30.

The union of covered rows is exactly right: {10, 20, 25, 30} from the posting list plus {40} from the new tuple. Nothing is lost, nothing is duplicated, and the posting list stays sorted.

So the general rule is: the posting list keeps its first postingoff TIDs, gains the new TID at position postingoff, and keeps its original TIDs from postingoff onward shifted right by one — with the old maximum pushed out into the newly inserted tuple. The posting list tuple is exactly the same size as it was before; the new tuple is an ordinary, small index tuple that happens to carry the old maximum’s TID. That is the sense in which the README says “the size of the posting list won’t change”.

The assertions in the function are the machine-checked statement of this invariant:

 c
 
 
	Assert(ItemPointerCompare(BTreeTupleGetMaxHeapTID(nposting),
							 BTreeTupleGetHeapTID(newitem)) < 0);
	Assert(_bt_posting_valid(nposting));

The first says the modified list’s maximum (the old second-to-last TID) is strictly less than the new tuple’s TID (the old maximum), and the second says the modified list is still strictly increasing.

There is no posting tree

This is worth stating explicitly, because the phrase “posting tree” shows up in GIN, in mailing-list discussion of btree, and in older blog posts about index deduplication — but there is no such thing in btree.

A “posting tree”, in the GIN sense, is a separate tree of pages whose leaves hold the list of rows matching one key. GIN needs it because a single GIN key can match millions of rows and therefore cannot be stored in one tuple on one page. btree never needed it, because in btree a posting list is bounded by the page: it lives inside a tuple on a leaf page, and the bound (1348 bytes, ~224 TIDs) is small enough that a page full of posting lists can still be split sensibly.

Upstream btree has only ever had posting lists — deduplication arrived in a single commit, 0d861bbb702 (“Add deduplication to nbtree”, February 2020), and it arrived in exactly the form it still has: a posting list tuple on a leaf page, with in-place posting list splits. You can verify the absence of the older design in the current source: grep -r BTP_POSTING over src/ returns nothing, and there is no posting-tree page type, no posting-tree split, and no posting-tree WAL record anywhere in nbtree.

The README explains the design rationale:

Posting list splits conceptually “rewrite” an insert that overlaps with an existing posting list into an insert that adds its final new item just to the right of the posting list instead. The size of the posting list won’t change, and so page space accounting code does not need to care about posting list splits at all.

The “size of the posting list won’t change” phrasing is about the tuple’s size: the posting list tuple keeps exactly the same number of TIDs (it gains one and loses one to the new tuple), so it occupies exactly the same number of bytes on the page. That is what makes the page-split accounting simple.

WAL: the special case

The write-ahead log for a posting list split is unusual, and it is unusual in an instructive way. Normally a WAL record describes a delta — “insert this tuple at this offset”. For a posting list split, the delta would be “insert a TID into the middle of an array inside a tuple that is already on the page”, which is exactly the kind of fiddly, error-prone delta that WAL is supposed to avoid.

PostgreSQL’s solution: log the inputs, not the result. The WAL record contains the postingoff value and the original new tuple; the REDO routine replays the very same _bt_swap_posting() call that the primary performed. The record does not have to describe the whole posting list; it describes the operation on it, and the operation is deterministic.

From src/backend/access/nbtree/nbtinsert.c:

 c
 
 
// src/backend/access/nbtree/nbtinsert.c
			else if (postingoff != 0)
			{
				/*
				 * Leaf insert with posting list split. Must include
				 * postingoff field before newitem/orignewitem.
				 */
				Assert(isleaf);
				xlinfo = XLOG_BTREE_INSERT_POST;
			}
			...
			XLogRegisterBuffer(0, buf, REGBUF_STANDARD);
			if (postingoff == 0)
			{
				/* Just log itup from caller */
				XLogRegisterBufData(0, itup, IndexTupleSize(itup));
			}
			else
			{
				/*
				 * Insert with posting list split (XLOG_BTREE_INSERT_POST
				 * record) case.
				 *
				 * Log postingoff. Also log origitup, not itup. REDO routine
				 * must reconstruct final itup (as well as nposting) using
				 * _bt_swap_posting().
				 */
				upostingoff = postingoff;

				XLogRegisterBufData(0, &upostingoff, sizeof(uint16));
				XLogRegisterBufData(0, origitup,
									IndexTupleSize(origitup));
			}

And the REDO routine, in src/backend/access/nbtree/nbtxlog.c:

 c
 
 
// src/backend/access/nbtree/nbtxlog.c
		else
		{
			ItemId		itemid;
			IndexTuple	oposting,
						newitem,
						nposting;
			uint16		postingoff;

			/*
			 * A posting list split occurred during leaf page insertion. WAL
			 * record data will start with an offset number representing the
			 * point in an existing posting list that a split occurs at.
			 *
			 * Use _bt_swap_posting() to repeat posting list split steps from
			 * primary. Note that newitem from WAL record is 'orignewitem',
			 * not the final version of newitem that is actually inserted on
			 * page.
			 */
			postingoff = *((uint16 *) datapos);
			datapos += sizeof(uint16);
			datalen -= sizeof(uint16);

			itemid = PageGetItemId(page, OffsetNumberPrev(xlrec->offnum));
			oposting = (IndexTuple) PageGetItem(page, itemid);

			/* Use mutable, aligned newitem copy in _bt_swap_posting() */
			Assert(isleaf && postingoff > 0);
			newitem = CopyIndexTuple((IndexTuple) datapos);
			nposting = _bt_swap_posting(newitem, oposting, postingoff);

			/* Replace existing posting list with post-split version */
			memcpy(oposting, nposting, MAXALIGN(IndexTupleSize(nposting)));

			/* Insert "final" new item (not orignewitem from WAL stream) */
			Assert(IndexTupleSize(newitem) == datalen);
			if (PageAddItem(page, newitem, datalen, xlrec->offnum, false, false) == InvalidOffsetNumber)
				elog(PANIC, "failed to add posting split new item");
		}

Note what is not in the WAL record: the modified posting list tuple. The REDO routine recomputes it from the original posting list (which is on the page, unmodified in the WAL stream — the page is replayed from the last full-page image or incremental change) and the postingoff. This is why the record is small: two bytes of offset plus the new tuple.

The same trick is used for page splits that coincide with a posting list split. The split record carries the offset:

 c
 
 
// src/include/access/nbtxlog.h
typedef struct xl_btree_split
{
	uint32		level;			/* tree level of page being split */
	OffsetNumber firstrightoff; /* first origpage item on rightpage */
	OffsetNumber newitemoff;	/* new item's offset */
	uint16		postingoff;		/* offset inside orig posting tuple */
} xl_btree_split;

#define SizeOfBtreeSplit	(offsetof(xl_btree_split, postingoff) + sizeof(uint16))

and the header comment above it explains the subtlety:

_R variant leaf page split records that must deal with a posting list split will include an explicit newitem, though it is never used on the right page – it is actually an orignewitem needed to update existing posting list.

Deduplication itself has its own record type, which is even more compact — it logs the intervals of the page that were deduplicated, not the resulting tuples:

 c
 
 
// src/include/access/nbtxlog.h
/*
 * When page is deduplicated, consecutive groups of tuples with equal keys are
 * merged together into posting list tuples.
 *
 * The WAL record represents a deduplication pass for a leaf page. An array
 * of BTDedupInterval structs follows.
 */
typedef struct xl_btree_dedup
{
	uint16		nintervals;

	/* DEDUPLICATION INTERVALS FOLLOW */
} xl_btree_dedup;

#define SizeOfBtreeDedup 	(offsetof(xl_btree_dedup, nintervals) + sizeof(uint16))

and the corresponding logging code:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
	/* XLOG stuff */
	if (RelationNeedsWAL(rel))
	{
		xl_btree_dedup xlrec_dedup;

		xlrec_dedup.nintervals = state->nintervals;

		XLogBeginInsert();
		XLogRegisterBuffer(0, buf, REGBUF_STANDARD);
		XLogRegisterData(&xlrec_dedup, SizeOfBtreeDedup);

		/*
		 * The intervals array is not in the buffer, but pretend that it is.
		 * When XLogInsert stores the whole buffer, the array need not be
		 * stored too.
		 */
		XLogRegisterBufData(0, state->intervals,
							state->nintervals * sizeof(BTDedupInterval));

		recptr = XLogInsert(RM_BTREE_ID, XLOG_BTREE_DEDUP);
	}

An interval says “offsets baseoff..baseoff+nitems-1 were merged into a posting list tuple”, and the REDO routine rebuilds it. Again: the WAL records the fact of the transformation, not the transformed bytes.

VACUUM and dead TIDs

A posting list can contain dead TIDs, and VACUUM has to remove them. But it cannot simply delete a TID from the middle of the array without breaking the sort order or the tuple’s self-description (the number of TIDs is stored in the tuple header).

The approach: rebuild the tuple without the dead TIDs, and overwrite the old tuple in place.

First, the scan. In src/backend/access/nbtree/nbtree.c, the vacuum scan distinguishes plain tuples from posting list tuples:

 c
 
 
// src/backend/access/nbtree/nbtree.c
				Assert(!BTreeTupleIsPivot(itup));
				if (!BTreeTupleIsPosting(itup))
				{
					/* Regular tuple, standard table TID representation */
					if (callback(&itup->t_tid, callback_state))
					{
						deletable[ndeletable++] = offnum;
						nhtidsdead++;
					}
					else
						nhtidslive++;
				}
				else
				{
					BTVacuumPosting vacposting;
					int			nremaining;

					/* Posting list tuple */
					vacposting = btreevacuumposting(vstate, itup, offnum,
													&nremaining);
					if (vacposting == NULL)
					{
						/*
						 * All table TIDs from the posting tuple remain, so no
						 * delete or update required
						 */
						Assert(nremaining == BTreeTupleGetNPosting(itup));
					}
					else if (nremaining > 0)
					{
						/*
						 * Store metadata about posting list tuple in
						 * updatable array for entire page. Existing tuple
						 * will be updated during the later call to
						 * _bt_delitems_vacuum().
						 */
						Assert(nremaining < BTreeTupleGetNPosting(itup));
						updatable[nupdatable++] = vacposting;
						nhtidsdead += BTreeTupleGetNPosting(itup) - nremaining;
					}
					else
					{
						/*
						 * All table TIDs from the posting list must be
						 * deleted. We'll delete the index tuple completely
						 * (no update required).
						 */
						Assert(nremaining == 0);
						deletable[ndeletable++] = offnum;
						nhtidsdead += BTreeTupleGetNPosting(itup);
						pfree(vacposting);
					}

					nhtidslive += nremaining;
				}

Three outcomes for a posting list tuple:

Live TIDs	Action
All	Nothing
Some	Rewrite the tuple without the dead TIDs
None	Delete the whole tuple

The helper that works out which is which:

 c
 
 
// src/backend/access/nbtree/nbtree.c
static BTVacuumPosting
btreevacuumposting(BTVacState *vstate, IndexTuple posting,
				 OffsetNumber updatedoffset, int *nremaining)
{
	int			live = 0;
	int			nitem = BTreeTupleGetNPosting(posting);
	ItemPointer items = BTreeTupleGetPosting(posting);
	BTVacuumPosting vacposting = NULL;

	for (int i = 0; i < nitem; i++)
	{
		if (!vstate->callback(items + i, vstate->callback_state))
		{
			/* Live table TID */
			live++;
		}
		else if (vacposting == NULL)
		{
			/*
			 * First dead table TID encountered.
			 *
			 * It's now clear that we need to delete one or more dead table
			 * TIDs, so start maintaining metadata describing how to update
			 * existing posting list tuple.
			 */
			vacposting = palloc(offsetof(BTVacuumPostingData, deletetids) +
								nitem * sizeof(uint16));

			vacposting->itup = posting;
			vacposting->updatedoffset = updatedoffset;
			vacposting->ndeletedtids = 0;
			vacposting->deletetids[vacposting->ndeletedtids++] = i;
		}
		else
		{
			/* Second or subsequent dead table TID */
			vacposting->deletetids[vacposting->ndeletedtids++] = i;
		}
	}

	*nremaining = live;
	return vacposting;
}

Note the optimization: nothing is allocated at all unless there is actually a dead TID to remove.

Then _bt_update_posting() in nbtdedup.c builds the new tuple:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
void
_bt_update_posting(BTVacuumPosting vacposting)
{
	IndexTuple	origtuple = vacposting->itup;
	uint32		keysize,
				newsize;
	IndexTuple	itup;
	int			nhtids;
	int			ui,
				d;
	ItemPointer htids;

	nhtids = BTreeTupleGetNPosting(origtuple) - vacposting->ndeletedtids;

	Assert(_bt_posting_valid(origtuple));
	Assert(nhtids > 0 && nhtids < BTreeTupleGetNPosting(origtuple));

	/*
	 * Determine final size of new tuple.
	 *
	 * This calculation needs to match the code used within _bt_form_posting()
	 * for new posting list tuples. We avoid calling _bt_form_posting() here
	 * to save ourselves a second memory allocation for a htids workspace.
	 */
	keysize = BTreeTupleGetPostingOffset(origtuple);
	if (nhtids > 1)
		newsize = MAXALIGN(keysize +
						 nhtids * sizeof(ItemPointerData));
	else
		newsize = keysize;
	...
	ui = 0;
	d = 0;
	for (int i = 0; i < BTreeTupleGetNPosting(origtuple); i++)
	{
		if (d < vacposting->ndeletedtids && vacposting->deletetids[d] == i)
		{
			d++;
			continue;
		}
		htids[ui++] = *BTreeTupleGetPostingN(origtuple, i);
	}
	Assert(ui == nhtids);
	Assert(d == vacposting->ndeletedtids);
	Assert(nhtids == 1 || _bt_posting_valid(itup));
	Assert(nhtids == 1 || ItemPointerIsValid(&itup->t_tid));

	/* vacposting arg's itup will now point to updated version */
	vacposting->itup = itup;
}

The nhtids == 1 case degrades back to a plain tuple, exactly as in _bt_form_posting() — and note the assertion at the end, which says that a single-TID result must have a valid TID in t_tid rather than in a posting list.

The physical write happens in _bt_delitems_vacuum() (src/backend/access/nbtree/nbtpage.c), which overwrites the tuple in place with PageIndexTupleOverwrite():

 c
 
 
// src/backend/access/nbtree/nbtpage.c
	/*
	 * Handle posting tuple updates.
	 *
	 * Deliberately do this before handling simple deletes. If we did it the
	 * other way around (i.e. WAL record order -- simple deletes before
	 * updates) then we'd have to make compensating changes to the 'updatable'
	 * array of offset numbers.
	 *
	 * PageIndexTupleOverwrite() won't unset each item's LP_DEAD bit when it
	 * happens to already be set. It's important that we not interfere with
	 * any future simple index tuple deletion operations.
	 */
	for (int i = 0; i < nupdatable; i++)
	{
		OffsetNumber updatedoffset = updatedoffsets[i];
		IndexTuple	itup;
		Size		itemsz;

		itup = updatable[i]->itup;
		itemsz = MAXALIGN(IndexTupleSize(itup));
		if (!PageIndexTupleOverwrite(page, updatedoffset, itup, itemsz))
			elog(PANIC, "failed to update partially dead item in block %u of index \"%s\"",
				 BufferGetBlockNumber(buf), RelationGetRelationName(rel));
	}

One more subtlety, from the nbtree README:

We can set the LP_DEAD bit with posting list tuples, though only when all TIDs are known dead.

That is, a posting list tuple is either fully live-ish (some TIDs may be dead but the tuple is not marked dead) or fully dead (LP_DEAD set). A posting list is never partially marked dead, because the LP_DEAD bit is per-line-pointer and there is no way to say “this one is dead but that one isn’t” without rewriting the tuple.

Bottom-up deletion

The other way dead index entries get cleaned up is bottom-up index deletion, which is the preferred mechanism in modern PostgreSQL and predates (and complements) deduplication. The pass is _bt_bottomupdel_pass() in nbtdedup.c, and it uses exactly the same equality machinery as deduplication:

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
		else if (_bt_keep_natts_fast(rel, state->base, itup) > nkeyatts &&
				 _bt_dedup_save_htid(state, itup))
		{
			/* Tuple is equal; just added its TIDs to pending interval */
		}

The difference is that it does not merge tuples; it asks the table AM which of the logical rows behind the duplicates are safe to delete, and the table AM deletes the heap tuples, which makes the index entries dead by themselves.

For posting list tuples the heuristics are more conservative, and the comment explains why:

Our approach to marking entries whose TIDs come from posting lists is more complicated. Posting lists can only be formed by a deduplication pass (or during an index build), so recent version churn affecting the pointed-to logical rows is not particularly likely.

 c
 
 
// src/backend/access/nbtree/nbtdedup.c
			/*
			 * Complicated case: A posting list tuple.
			 *
			 * We make the conservative assumption that there can only be at
			 * most one affected logical row per posting list tuple. ...
			 */
			int			nitem = BTreeTupleGetNPosting(itup);
			bool		firstpromising = false;
			bool		lastpromising = false;
			...
			if (dupinterval)
			{
				/*
				 * Complicated rule: either the first or last TID in the
				 * posting list gets marked promising (if any at all)
				 */
				...
				/* Only entry with predominant table block can be promising */
				firstpromising = (minblocklist == midblocklist);
				lastpromising = (!firstpromising &&
								 midblocklist == maxblocklist);
			}

At most one TID per posting list tuple is offered to the table AM as a “promising” candidate, and only if the tuple’s TIDs mostly point at the same heap block.

The counterpoint: how GIN does it

GIN has had posting lists far longer than btree, and it does everything differently. The comparison is instructive because it shows what btree gave up.

GIN compresses the TID array

GIN stores a posting list as a varbyte-encoded delta stream. From src/backend/access/gin/ginpostinglist.c:

 c
 
 
// src/backend/access/gin/ginpostinglist.c
/*
 * For encoding purposes, item pointers are represented as 64-bit unsigned
 * integers. The lowest 11 bits represent the offset number, and the next
 * lowest 32 bits are the block number. That leaves 21 bits unused, i.e.
 * only 43 low bits are used.
 *
 * 11 bits is enough for the offset number, because MaxHeapTuplesPerPage <
 * 2^11 on all supported block sizes. We are frugal with the bits, because
 * smaller integers use fewer bytes in the varbyte encoding, saving disk
 * space. ...
 *
 * These 43-bit integers are encoded using varbyte encoding. In each byte,
 * the 7 low bits contain data, while the highest bit is a continuation bit.
 * When the continuation bit is set, the next byte is part of the same
 * integer, otherwise this is the last byte of this integer. 43 bits need
 * at most 7 bytes in this encoding:
 *
 * 0XXXXXXX
 * 1XXXXXXX 0XXXXYYY
 * ...
 *
 * X = bits used for offset number
 * Y = bits used for block number
 * u = unused bit
 *
 * The bytes are in stored in little-endian order.
 *
 * An important property of this encoding is that removing an item from list
 * never increases the size of the resulting compressed posting list. Proof:
 *
 * Removing number is actually replacement of two numbers with their sum. We
 * have to prove that varbyte encoding of a sum can't be longer than varbyte
 * encoding of its summands. Sum of two numbers is at most one bit wider than
 * the larger of the summands. Widening a number by one bit enlarges its length
 * in varbyte encoding by at most one byte. Therefore, varbyte encoding of sum
 * is at most one byte longer than varbyte encoding of larger summand. Lesser
 * summand is at least one byte, so the sum cannot take more space than the
 * summands, Q.E.D.
 *
 * This property greatly simplifies VACUUM, which can assume that posting
 * lists always fit on the same page after vacuuming. Note that even though
 * that holds for removing items from a posting list, you must also be
 * careful to not cause expansion e.g. when merging uncompressed items on the
 * page into the compressed lists, when vacuuming.
 */

That “removing an item never increases the size” property is a lovely piece of engineering, and btree simply cannot have it: btree’s posting list is a flat array of fixed-size TIDs, and it must be rebuilt to remove an entry.

The encoder itself:

 c
 
 
// src/backend/access/gin/ginpostinglist.c
static void
encode_varbyte(uint64 val, unsigned char **ptr)
{
	unsigned char *p = *ptr;

	while (val > 0x7F)
	{
		*(p++) = 0x80 | (val & 0x7F);
		val >>= 7;
	}
	*(p++) = (unsigned char) val;

	*ptr = p;
}

and the compressed list structure:

 c
 
 
// src/include/access/ginblock.h
typedef struct
{
	ItemPointerData first;		/* first item in this posting list (unpacked) */
	uint16		nbytes;			/* number of bytes that follow */
	unsigned char bytes[FLEXIBLE_ARRAY_MEMBER]; /* varbyte encoded items */
} GinPostingList;

#define SizeOfGinPostingList(plist) (offsetof(GinPostingList, bytes) + SHORTALIGN((plist)->nbytes) )
#define GinNextPostingListSegment(cur) ((GinPostingList *) (((char *) (cur)) + SizeOfGinPostingList((cur))))

The first TID is stored unpacked (so that the decoder knows where the list starts without decoding from the beginning); everything after it is a delta stream.

For a btree-style comparison: btree’s 6-byte-per-TID array versus GIN’s typically 1–2 bytes per TID for dense lists. GIN wins on space by a factor of three or more, and btree does not even try, because — as the README says:

Posting list compression would break the assumptions made by posting list splits about page space accounting … Besides, posting list compression does not offer a compelling trade-off for nbtree, since in general nbtree is optimized for consistent performance with many concurrent readers and writers. Compression would also make the deletion of a subset of TIDs from a posting list slow and complicated, which would be a big problem for workloads that depend heavily on bottom-up index deletion.

GIN has a posting tree; btree does not

The other half of the contrast: GIN does have a posting tree. A GIN index entry for a key points at a tree of pages that hold the list of rows containing that key, because a single key can match millions of rows and cannot fit on one page. The GIN data pages store GinPostingList structures, and the tree has internal pages with PostingItem pairs.

btree never needed that, because in btree the posting list is bounded by the page: it lives inside a tuple on a leaf page, and the bound (1348 bytes, ~224 TIDs) is chosen so that a page full of posting lists can still be split sensibly.

GIN’s key extraction

One more piece of GIN machinery worth comparing, because it is the mirror image of btree’s problem. GIN has to extract multiple keys from one column value (intarray, tsvector, arrays), sort them, and remove duplicates before it can build a posting list. That happens in ginExtractEntries() in src/backend/access/gin/ginutil.c.

The comparator is a hand-written one that also detects duplicates as a side effect, which is a neat trick:

 c
 
 
// src/backend/access/gin/ginutil.c
typedef struct
{
	FmgrInfo *cmpDatumFunc;
	Oid			collation;
	bool		haveDups;
} cmpEntriesArg;

static int
cmpEntries(const void *a, const void *b, void *arg)
{
	const Datum *aa = (const Datum *) a;
	const Datum *bb = (const Datum *) b;
	cmpEntriesArg *data = (cmpEntriesArg *) arg;
	int			res;

	res = DatumGetInt32(FunctionCall2Coll(data->cmpDatumFunc,
										 data->collation,
										 *aa, *bb));

	/*
	 * Detect if we have any duplicates. If there are equal keys, qsort must
	 * compare them at some point, else it wouldn't know whether one should go
	 * before or after the other.
	 */
	if (res == 0)
		data->haveDups = true;

	return res;
}

#define ST_SORT qsort_arg_entries
#define ST_ELEMENT_TYPE Datum
#define ST_COMPARE_ARG_TYPE cmpEntriesArg
#define ST_COMPARE(a, b, arg) cmpEntries(a, b, arg)
#define ST_SCOPE static
#define ST_DEFINE
#define ST_DECLARE
#include "lib/sort_template.h"

The sort_template.h trick generates a specialised sort with a context argument, so GIN does not pay for a qsort callback per comparison. The haveDups flag is the payoff: the sort has to compare equal elements at least once, so it learns for free whether there are duplicates, and only then pays for a uniquifying pass:

 c
 
 
// src/backend/access/gin/ginutil.c
	if (nentries > 1)
	{
		cmpEntriesArg arg;

		arg.cmpDatumFunc = &ginstate->compareFn[attnum - 1];
		arg.collation = ginstate->supportCollation[attnum - 1];
		arg.haveDups = false;

		qsort_arg_entries(entries, nentries, &arg);

		if (arg.haveDups)
			nentries = qunique_arg(entries, nentries, sizeof(Datum), cmpEntries, &arg);
	}

And NULLs are removed before the sort, because NULLs are all equal to each other and the sort would otherwise have to special-case them:

 c
 
 
// src/backend/access/gin/ginutil.c
	/*
	 * Scan the items for any NULLs. All NULLs are considered equal, so we
	 * just need to check and remember if there are any. We remove them from
	 * the array here, and after deduplication, put back one NULL entry to
	 * represent them all.
	 */
	hasNull = false;
	if (nullFlags)
	{
		int32		numNonNulls = 0;

		for (int32 i = 0; i < nentries; i++)
		{
			if (nullFlags[i])
				hasNull = true;
			else
			{
				entries[numNonNulls] = entries[i];
				numNonNulls++;
			}
		}
		nentries = numNonNulls;
	}

The comment “put back one NULL entry to represent them all” is the same idea as btree’s “NULL is just another value from the domain of indexed values” — deduplication works on NULLs just as well as on any other value.

The rules for when deduplication applies

Three independent conditions, all of which must hold.

1. The index must be deduplication-safe (“allequalimage”)

The operator class must guarantee that two equal datums have the same image — the same bytes. This is not the same as “equal”. A bpchar value 'a' and 'a ' compare equal but have different images, and if they were merged into one posting list, an index-only scan would have to return one image for both rows.

The check lives in _bt_allequalimage() in src/backend/access/nbtree/nbtutils.c:

 c
 
 
// src/backend/access/nbtree/nbtutils.c
/*
 * Are all attributes in rel "equality is image equality" attributes?
 *
 * We use each attribute's BTEQUALIMAGE_PROC opclass procedure. If any
 * opclass either lacks a BTEQUALIMAGE_PROC procedure or returns false, we
 * return false; otherwise we return true.
 *
 * Returned boolean value is stored in index metapage during index builds.
 * Deduplication can only be used when we return true.
 */
bool
_bt_allequalimage(Relation rel, bool debugmessage)
{
	bool		allequalimage = true;

	/* INCLUDE indexes can never support deduplication */
	if (IndexRelationGetNumberOfAttributes(rel) !=
		IndexRelationGetNumberOfKeyAttributes(rel))
		return false;

	for (int i = 0; i < IndexRelationGetNumberOfKeyAttributes(rel); i++)
	{
		Oid			opfamily = rel->rd_opfamily[i];
		Oid			opcintype = rel->rd_opcintype[i];
		Oid			collation = rel->rd_indcollation[i];
		Oid			equalimageproc;

		equalimageproc = get_opfamily_proc(opfamily, opcintype, opcintype,
										 BTEQUALIMAGE_PROC);

		/*
		 * If there is no BTEQUALIMAGE_PROC then deduplication is assumed to
		 * be unsafe. Otherwise, actually call proc and see what it says.
		 */
		if (!OidIsValid(equalimageproc) ||
			!DatumGetBool(OidFunctionCall1Coll(equalimageproc, collation,
											 ObjectIdGetDatum(opcintype))))
		{
			allequalimage = false;
			break;
		}
	}

	if (debugmessage)
	{
		if (allequalimage)
			elog(DEBUG1, "index \"%s\" can safely use deduplication",
				 RelationGetRelationName(rel));
		else
			elog(DEBUG1, "index \"%s\" cannot use deduplication",
				 RelationGetRelationName(rel));
	}

	return allequalimage;
}

The result is recorded in the index metapage at build time, so the decision is made once, by CREATE INDEX or REINDEX, and is then consulted on every insert.

The types that fail this test are listed in the documentation (doc/src/sgml/btree.sgml):

Type	Why
`text`, `varchar` with a nondeterministic collation	Case and accent differences must be preserved
`character(n)` / `char` / `bpchar`	Trailing spaces are disregarded when comparing, but must be preserved
`numeric`	Numeric display scale must be preserved
`jsonb`	Uses `numeric` internally
`float4`, `float8`	`-0` and `0` are equal but have distinct representations
Container types (arrays, ranges, composites)	Not supported yet

The bpchar case is not theoretical. In October 2026, PostgreSQL rescinded deduplication support for bpchar_ops, bpchar_pattern_ops and oidvector_ops (commit a32b8dc9865, “Rescind unsafe deduplication support”):

bpchar equality ignores trailing spaces, so equal values in a column with no length specifier can have different images. Similarly, oidvector equality intentionally pays no attention to the array lower bound field. We nevertheless allowed nbtree deduplication to be used with indexes built with bpchar_ops, bpchar_pattern_ops, or oidvector_ops. A posting list split could therefore place a TID in an index tuple whose key was equal to, but not bytewise identical to, the corresponding table value. This allowed index-only scans to return subtly wrong results.

That is the whole failure mode in one sentence: a posting list split moves a TID from one index tuple to another, and if the two tuples' keys are merely equal rather than identical, the row now claims to have a key value it never had.

2. INCLUDE columns make deduplication impossible

The same function returns false for any index with non-key columns:

 c
 
 
	/* INCLUDE indexes can never support deduplication */
	if (IndexRelationGetNumberOfAttributes(rel) !=
		IndexRelationGetNumberOfKeyAttributes(rel))
		return false;

The reason is obvious in hindsight: an INCLUDE column’s value is per-row, not per-key. Two rows with the same key but different INCLUDE values cannot share one index tuple, because an index-only scan has to return the row’s own INCLUDE value.

The nbtree header comment says the same thing about the format:

Like any non-pivot tuple, the number of columns stored is always implicitly the total number in the index (in practice there can never be non-key columns stored, since deduplication is not supported with INCLUDE indexes).

3. The index must not be unique… except when it is

Unique indexes can use deduplication, contrary to what many people assume. The rationale is in the README:

Unique index leaf pages only get a deduplication pass when an insertion (that might have to split the page) observed an existing duplicate on the page in passing. … Our ultimate goal is to delay leaf page splits indefinitely (i.e. to prevent them altogether).

A unique index with version churn (an UPDATE that changes the key) can temporarily hold two entries with the same key. Deduplication lets the page absorb that without splitting, buying time for bottom-up deletion to clean up.

What it all means in practice

A B-tree index over a low-cardinality column can be dramatically smaller than you think. A boolean column with a 50/50 split over ten million rows produces roughly 5000 posting list tuples, not ten million index entries. If you REINDEX such an index and it shrinks by an order of magnitude, that is deduplication doing its job.

Deduplication is lazy, so a freshly created index may not look deduplicated. CREATE INDEX does deduplicate eagerly, using a different strategy — each group of duplicates in the sorted input is merged before being added to the pending page. But an index that grew by incremental inserts will only be deduplicated when a page split was about to happen. If you want the compact form, REINDEX.

deduplicate_items = off is a real option, and rarely a good one. The cost of a deduplication pass on a page with no duplicates is a scan of the page’s tuples comparing keys — not free, but not catastrophic. The documentation’s advice is:

Write-heavy workloads that don’t benefit from deduplication due to having few or no duplicate values in indexes will incur a small, fixed performance penalty (unless deduplication is explicitly disabled).

If you have a high-cardinality index on a write-heavy table and you are chasing insert throughput, turning it off is a legitimate experiment.

A posting list is not a compression scheme. Six bytes per TID, no deltas, no varbyte encoding. If you want compressed posting lists, you want GIN. btree’s posting lists are a deduplication mechanism: they remove the repeated keys, not the repeated structure within the TID array.

Posting list splits are cheap in WAL terms, and that was a deliberate design goal. The WAL record carries two bytes of offset and the new tuple; the modified posting list is recomputed at replay time. A posting list split is not a “rewrite the whole posting list” WAL record.

Dead TIDs in a posting list are cleaned up by rewriting the tuple, not by marking it. The LP_DEAD bit is only set on a posting list tuple when every TID in it is dead. If a posting list has 200 TIDs and 3 are dead, VACUUM rebuilds the tuple with 197 TIDs. On a high-churn table that is a lot of tuple rewriting — which is one reason bottom-up index deletion, which deletes the heap rows and lets the index entries die naturally, is the preferred cleanup path.

Deduplication requires equality to imply identity. If your column compares equal in more ways than it compares identically — trailing spaces, numeric display scale, -0 versus 0, a nondeterministic collation — then PostgreSQL will refuse to deduplicate it, and if it was ever allowed to, that was a bug (as a32b8dc9865 fixed).

The number of TIDs per posting list is bounded twice, and the smaller bound is the interesting one. 4095 is the hard limit from the 12-bit offset number; ~224 is the practical limit from the one-sixth-of-a-page size cap. A key that is 300 bytes long leaves room for only ~174 TIDs per posting list, so a wide key means short lists.

Look at it yourself

pageinspect can show you the raw posting list tuples:

 sql
 
 
CREATE EXTENSION pageinspect;

-- the leaf page's tuples, with the raw t_tid
SELECT itemoffset, ctid, itemlen, data
FROM bt_page_items('myindex', 0);

For a posting list tuple, the ctid column is the repurposed t_tid: the block number is the byte offset of the TID array within the tuple, and the offset number’s low 12 bits are the number of TIDs. itemlen is the whole tuple’s length, key plus TID array. The data column deliberately excludes the posting list, so it shows you just the key columns:

 c
 
 
// contrib/pageinspect/btreefuncs.c
	/*
	 * Make sure that "data" column does not include posting list or pivot
	 * tuple representation of heap TID(s).
	 * ...
	 */
	if (BTreeTupleIsPosting(itup))
		dlen -= IndexTupleSize(itup) - BTreeTupleGetPostingOffset(itup);

Which means that to see the raw TID array you have to go at the raw page yourself.

The amcheck extension’s bt_index_check() and bt_index_parent_check() will verify posting list invariants, and bt_index_check() has specific handling for posting list tuples.

PostgreSQL's Multirange Type Implementation in C

Jay Hubbert — Mon, 31 Aug 2026 11:06:16 +0000

A multirange is what you get when you let a range type hold more than one range: int4multirange can be {[1,5),[10,20)}. The surprising thing is how little C there is to it. All six built-in multirange types — int4multirange, int8multirange, nummultirange, tsmultirange, tstzmultirange, datemultirange — are implemented by one set of functions in src/backend/utils/adt/multirangetypes.c. The only thing that varies is the element type, and even that is not compiled in; it is looked up at runtime.

This is a walk through the stored layout, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog

The six concrete multirange types live in src/include/catalog/pg_type.dat:

oid	type	element type	`typalign`	array oid
4451	`int4multirange`	`int4`	`i`	6150
4532	`nummultirange`	`numeric`	`i`	6151
4533	`tsmultirange`	`timestamp`	`d`	6152
4534	`tstzmultirange`	`timestamptz`	`d`	6153
4535	`datemultirange`	`date`	`i`	6155
4536	`int8multirange`	`int8`	`d`	6157

They are all typlen => -1 (varlena), typbyval => f, typtype => 'm', typcategory => 'R', and typstorage => 'x' (fully toastable). typalign is copied from the element type, which is why the integer and date multiranges are 4-byte aligned while the timestamp and bigint ones are 8-byte aligned. The element mapping is recorded in src/include/catalog/pg_range.dat as rngmultitypid, alongside the range constructor functions (rngmltconstruct0/1/2).

There is no seventh concrete type. The oid 4537 in the neighborhood is anymultirange, a typtype => 'p' pseudo-type used for polymorphic functions — not a value you can store.

The in-memory struct

MultirangeType is declared in src/include/utils/multirangetypes.h:

 c
 
 
typedef struct
{
	int32		vl_len_;		/* varlena header (do not touch directly!) */
	Oid			multirangetypid;	/* multirange type's own OID */
	uint32		rangeCount;		/* the number of ranges */

	/*
	 * Following the count are the range objects themselves, as ShortRangeType
	 * structs. Note that ranges are varlena too, depending on whether they
	 * have lower/upper bounds and because even their base types can be
	 * varlena. So we can't really index into this list.
	 */
} MultirangeType;

Twelve bytes. The “ShortRangeType structs” are not a real struct — they are just the bound values and a flags byte, appended after the header. Because the bounds can be variable length, you cannot index into the array directly; you have to walk it. That constraint shapes the entire format.

The stored format

The file’s header comment states the layout, and it is worth reading verbatim:

 text
 
 
12 bytes: MultirangeType struct including varlena header, multirange
 type's OID and the number of ranges in the multirange.
4 * (rangesCount - 1) bytes: 32-bit items pointing to the each range
 in the multirange starting from the second one.
1 * rangesCount bytes : 8-bit flags for each range in the multirange
The rest of the multirange are range bound values pointed by multirange items.

So after the 12-byte header comes an array of (rangeCount - 1) 32-bit items (the first range starts at offset zero and needs no item), then rangeCount one-byte flags, then the aligned bound values. The clever part is what goes in the items:

 c
 
 
#define MULTIRANGE_ITEM_OFF_BIT 0x80000000
#define MULTIRANGE_ITEM_GET_OFFLEN(item) ((item) & 0x7FFFFFFF)
#define MULTIRANGE_ITEM_HAS_OFF(item) ((item) & MULTIRANGE_ITEM_OFF_BIT)
#define MULTIRANGE_ITEM_OFFSET_STRIDE 4

Most items store a length, which makes them small and therefore compression-friendly. Every fourth item (i % 4 == 0) instead stores an absolute offset with the high bit set. That gives you a cheap “jump point” every four ranges while keeping the common case tiny. write_multirange_data() shows the encoding:

 c
 
 
items[i - 1] = ptr - begin;
if ((i % MULTIRANGE_ITEM_OFFSET_STRIDE) != 0)
	items[i - 1] -= prev_offset;	/* a length */
else
	items[i - 1] |= MULTIRANGE_ITEM_OFF_BIT;	/* an absolute offset */
prev_offset = ptr - begin;

Canonicalization: sort and merge on the way in

make_multirange() is the single entry point every constructor calls. Its first act is multirange_canonicalize(), which sorts the incoming ranges and merges any that touch:

 c
 
 
if (range_adjacent_internal(rangetyp, lastRange, currentRange))
{
	ranges[output_range_count - 1] = lastRange =
		range_union_internal(rangetyp, lastRange, currentRange, false);
}

The result is canonical: ranges are stored in order, non-overlapping, and never adjacent. That is why int4multirange(int4range(1,5), int4range(5,10)) and int4multirange(int4range(1,10)) are the same value — and why equality can be a straight range-by-range comparison. Note that empty ranges are dropped entirely; a multirange can never contain an empty range. It just stores zero ranges and reports MultirangeIsEmpty(mr) as mr->rangeCount == 0.

Reading a range back out

multirange_get_bounds() is the workhorse. It computes the offset by summing lengths until it hits an offset item:

 c
 
 
while (i > 0)
{
	offset += MULTIRANGE_ITEM_GET_OFFLEN(items[i - 1]);
	if (MULTIRANGE_ITEM_HAS_OFF(items[i - 1]))
		break;
	i--;
}

Then it walks the bound values using the element type’s typlen/typalign and returns a RangeBound for each side, reading the flags byte to fill in inclusive/infinite state. multirange_get_range() is the same walk but with the bound bytes copied into a freshly allocated RangeType. The fast path is deliberately cheaper: it avoids constructing a whole range when only the bounds are wanted.

The binary wire format is a different animal. multirange_send() writes an int32 range count, then for each range an int32 length followed by that range’s binary encoding; multirange_recv() mirrors it. It stores neither the multirange OID nor the compact item array.

Comparison and hashing

multirange_cmp() is the btree comparator. It walks the two multiranges range by range and compares lower then upper bounds; a shorter multirange is treated as if padded with empty ranges at the end, so it sorts first:

 c
 
 
if (i >= range_count_1)
{
	cmp = -1;
	break;
}

multirange_eq_internal() is a bound-by-bound loop, and equality of the OIDs is asserted up front. Hashing is hash_multirange(): each range’s flags and bounds are hashed through the element type’s hash function, then folded together:

 c
 
 
range_hash = hash_bytes_uint32((uint32) flags);
range_hash ^= lower_hash;
range_hash = pg_rotate_left32(range_hash, 1);
range_hash ^= upper_hash;
result = (result << 5) - result + range_hash;

That (result << 5) - result trick is the same one hash_array uses to combine elements. There is also a hash_multirange_extended() for 64-bit, seeded hashing.

Look at it yourself

The sizes follow directly from the layout. An empty int4multirange is just the 12-byte header; adding a range adds one flags byte plus the aligned bounds:

 sql
 
 
SELECT octet_length('{}'::int4multirange); -- 12
SELECT octet_length('{[1,5)}'::int4multirange); -- 24
SELECT octet_length('{[1,5),[10,20)}'::int4multirange); -- 36

Canonicalization and unnest are visible too:

 sql
 
 
SELECT '{[1,5),[5,10)}'::int4multirange; -- {[1,10)}
SELECT unnest('{[1,5),[10,20)}'::int4multirange);
-- int4range
-- ----------
-- [1,5)
-- [10,20)

The multirange family is supported by btree/multirange_ops, hash/multirange_ops, and gist/multirange_ops (see pg_amproc.dat), so you can put a multirange in an index or a hash join.

What it all means

A multirange is a varlena with a 12-byte header, a compact item array, a flags array, and aligned bound values.
The items store lengths by default and absolute offsets every fourth entry, which is what makes the format cheap to compress and cheap to seek into.
make_multirange() canonicalizes: sort, drop empties, merge touching ranges. The stored form is canonical, which is why equality is a simple comparison.
All six types share one implementation; only the element type, and therefore the alignment and the element hash/compare functions, differs.
The wire format is not the storage format — send/recv use a flat length-prefixed list of range encodings.

PostgreSQL Does Async I/O with io_uring

Jay Hubbert — Thu, 27 Aug 2026 09:00:00 +0000

Every storage engine eventually runs into the same wall. You ask for a block, the block is not in memory, and the process blocks in the kernel until the disk spins around. It works, but it wastes the machine: one backend sleeps while the storage device is idle, another backend sleeps while the device is busy, and the CPU sits there with nothing to do.

PostgreSQL spent twenty years hiding this from you. The operating system page cache absorbed most reads; effective_io_concurrency and the kernel’s own readahead absorbed most sequential scans. What the operating system could not hide was fdatasync(), and that is exactly the operation WAL writing is made of.

PostgreSQL 20 (version 20devel, commit e75fcc9867f) contains an answer: a pluggable asynchronous I/O subsystem with an io_uring backend, an io_method=worker backend, and — the part I find most interesting — a read-ahead controller that only looks further ahead when the backend actually had to wait for I/O.

That last part is the real story. Async I/O without a feedback loop is just a different way to burn memory. The feedback loop is what makes it worth having.

This is a walk through how it all works, based on reading the PostgreSQL source.

The subsystem lives in one directory

The entire asynchronous I/O subsystem is in src/backend/storage/aio/. The file header of aio.c explains the split:

 c
 
 
 * AIO is a complicated subsystem. To keep things navigable, it is split
 * across a number of files:
 *
 * - method_*.c - different ways of executing AIO (e.g. worker process)
 *
 * - aio_target.c - IO on different kinds of targets
 *
 * - aio_io.c - method-independent code for specific IO ops (e.g. readv)
 *
 * - aio_callback.c - callbacks at IO operation lifecycle events
 *
 * - aio_init.c - per-server and per-backend initialization
 *
 * - aio.c - all other topics
 *
 * - read_stream.c - helper for reading buffered relation data
 *
 * - README.md - higher-level overview over AIO

File	Lines	What it does
`README.md`	425	The design document: motivation, deadlocks, the API walkthrough
`aio.c`	1358	Core logic — handle lifecycle, state machine, wait references, batch mode
`aio_io.c`	240	Associating an operation with a handle; the synchronous fallback
`aio_init.c`	255	Shared memory sizing, per-backend initialization
`aio_target.c`	122	“What am I reading?” — file identity, reopening for workers
`aio_callback.c`	333	Completion callbacks, staged/complete_shared/complete_local
`method_io_uring.c`	812	The io_uring backend
`method_worker.c`	1065	The worker-process backend
`method_sync.c`	47	The “do it inline” backend
`read_stream.c`	1495	The adaptive read-ahead controller

Note what is not in there: bufmgr.c is not in there, smgr.c is not in there, md.c is not in there. The AIO subsystem knows nothing about buffers or relations. It knows about file descriptors, iovecs, and completion. Everything else is layered on top.

The design goals, in the author’s own words

The README.md in that directory is the best documentation PostgreSQL has ever written for a new subsystem, and it is worth quoting.

First, the motivation for asynchronous I/O at all:

Until the introduction of asynchronous IO postgres relied on the operating system to hide the cost of synchronous IO from postgres. While this worked surprisingly well in a lot of workloads, it does not do as good a job on prefetching and controlled writeback as we would like.

There are important expensive operations like fdatasync() where the operating system cannot hide the storage latency. This is particularly important for WAL writes, where the ability to asynchronously issue fdatasync() or O_DSYNC writes can yield significantly higher throughput.

Then, why direct I/O (this is the payoff, and also the trap):

The main reasons to want to use Direct IO are:

Lower CPU usage / higher throughput. Particularly on modern storage buffered writes are bottlenecked by the operating system having to copy data from the kernel’s page cache to postgres buffer pool using the CPU. Whereas direct IO can often move the data directly between the storage devices and postgres' buffer cache, using DMA. While that transfer is ongoing, the CPU is free to perform other work.

Reduced latency - Direct IO can have substantially lower latency than buffered IO, which can be impactful for OLTP workloads bottlenecked by WAL write latency.

Avoiding double buffering between operating system cache and postgres' shared_buffers.

And the catch, which is the sentence that explains why this took twenty years to build:

The main reasons not to use Direct IO are:

Without AIO, Direct IO is unusably slow for most purposes.

Even with AIO, many parts of postgres need to be modified to perform explicit prefetching.

In situations where shared_buffers cannot be set appropriately large, e.g. because there are many different postgres instances hosted on shared hardware, performance will often be worse than when using buffered IO.

“Direct IO is unusably slow without AIO” is not a throwaway line. It is the central constraint. A direct read that misses the page cache costs a full storage round trip and it costs the caller a context switch, because the caller has nowhere else to go while it waits. With a buffered read, the kernel’s readahead covers the next few kilobytes and the syscall returns almost immediately. Take away the kernel’s readahead and you take away the only thing making buffered I/O tolerable.

The third design constraint shaped every decision that follows — the deadlock hazard. From the README:

Consider one backend performing readahead on a table, initiating IO for a number of buffers ahead of the current “scan position”. If that backend then performs some operation that blocks, or even just is slow, the IO completion for the asynchronously initiated read may not be processed.

Completions must be processable even when the issuing backend is stuck. That is why method_io_uring.c puts a completion lock on each backend’s ring, and why method_worker.c exists at all.

AIO handles: the central abstraction

The API’s central object is the AIO handle. You acquire one, define it (attach an operation to it), submit it, and eventually reclaim it. The state machine is in src/include/storage/aio_internal.h, and it moves linearly:

State	Meaning
`PGAIO_HS_IDLE`	not in use
`PGAIO_HS_HANDED_OUT`	returned by `pgaio_io_acquire()`
`PGAIO_HS_DEFINED`	`pgaio_io_start_*()` called, not yet staged
`PGAIO_HS_STAGED`	staged, ready for submission
`PGAIO_HS_SUBMITTED`	handed to the IO method
`PGAIO_HS_COMPLETED_IO`	the IO finished, result not yet processed
`PGAIO_HS_COMPLETED_SHARED`	shared completion callbacks have run
`PGAIO_HS_COMPLETED_LOCAL`	local completion callbacks have run

One handle per in-flight operation, reused as soon as it completes. That reuse is why the API also needs wait references: a handle can be recycled out from under you, so you cannot use the handle itself to wait for completion. A wait reference carries the handle’s generation counter, so a stale reference is detectable:

 c
 
 
	iow->aio_index = ioh - pgaio_ctl->io_handles;
	iow->generation_upper = (uint32) (ioh->generation >> 32);
	iow->generation_lower = (uint32) ioh->generation;

There is also a deliberate rule about how many handles a backend may hold at once:

Because an AIO Handle is not executable just after calling pgaio_io_acquire() and because pgaio_io_acquire() needs to always succeed (absent a PANIC), only a single AIO Handle may be acquired (i.e. returned by pgaio_io_acquire()) without causing the IO to have been defined (by, potentially indirectly, causing pgaio_io_start_*() to have been called). Otherwise a backend could trivially self-deadlock by using up all AIO Handles without the ability to wait for some of the IOs to complete.

And the code enforces it:

 c
 
 
	if (pgaio_my_backend->handed_out_io)
		elog(ERROR, "API violation: Only one IO can be handed out");

Callbacks, not function pointers

Completion handling is done by callbacks, identified by an ID rather than a pointer. src/include/storage/aio.h explains why:

Memory within PgAioHandle is precious, due to the number of PgAioHandle structs in pre-allocated shared memory.

Due to EXEC_BACKEND function pointers are not necessarily stable between different backends, therefore function pointers cannot directly be in shared memory.

There are two flavours of completion callback, and the distinction matters:

Callback	Runs in	May modify
`complete_shared`	whichever backend completed the I/O	shared memory only
`complete_local`	the issuing backend	backend-local state

The complete_shared variant is the one that updates BufferDescs so other backends can use a buffer that has just been read. The complete_local variant exists for temporary tables, whose BufferDesc is only reachable from the owning backend. Both are invoked inside a critical section, because WAL writes need to be able to complete I/O inside one.

Shared memory, and why

aio_init.c sizes five shared memory segments — AioCtl, AioBackend, AioHandle, AioHandleIOV and AioHandleData — and hands out the handle pool:

 c
 
 
	ShmemRequestStruct(.name = "AioHandle",
					 .size = AioHandleShmemSize(),
					 .ptr = (void **) &AioHandleShmemPtr,
		);

	ShmemRequestStruct(.name = "AioHandleIOV",
					 .size = AioHandleIOVShmemSize(),
					 .ptr = (void **) &AioHandleIOVShmemPtr,
		);

The iovecs live in shared memory too, because the worker backend has to be able to execute an I/O that another backend staged. That is also why aio_target.c has the notion of a target — a description of what file is being read, sufficient for another process to reopen it:

 c
 
 
struct PgAioTargetInfo
{
	/*
	 * To support executing using worker processes, the file descriptor for an
	 * IO may need to be reopened in a different process.
	 */
	void		(*reopen) (PgAioHandle *ioh);

	/* describe the target of the IO, used for log messages and views */
	char	 *(*describe_identity) (const PgAioTargetData *sd);

	/* name of the target, used in log messages / views */
	const char *name;
};

How many handles exist per backend? io_max_concurrency, which defaults to -1 meaning “figure it out”:

 c
 
 
static int
AioChooseMaxConcurrency(void)
{
	uint32		max_backends;
	int			max_proportional_pins;

	/* Similar logic to LimitAdditionalPins() */
	max_backends = MaxBackends + NUM_AUXILIARY_PROCS;
	max_proportional_pins = NBuffers / max_backends;

	max_proportional_pins = Max(max_proportional_pins, 1);

	/* apply upper limit */
	return Min(max_proportional_pins, 64);
}

So: one I/O handle per buffer you could plausibly pin, capped at 64.

Three ways to do the same thing

The method is selected by the io_method GUC, which is PGC_POSTMASTER — you have to restart to change it:

 c
 
 
/* Options for io_method. */
const struct config_enum_entry io_method_options[] = {
	{"sync", IOMETHOD_SYNC, false},
	{"worker", IOMETHOD_WORKER, false},
#ifdef IOMETHOD_IO_URING_ENABLED
	{"io_uring", IOMETHOD_IO_URING, false},
#endif
	{NULL, 0, false}
};

Method	Platform	How it works	Default
`sync`	everywhere	Does the I/O inline; the API is still exercised	no
`worker`	everywhere	Hands the I/O to a pool of worker processes	yes
`io_uring`	Linux 5.1+	Submits to the kernel’s io_uring from the backend itself	no

io_uring is compiled in only when liburing is available and the build is not EXEC_BACKEND:

 c
 
 
/* io_uring is incompatible with EXEC_BACKEND */
#if defined(USE_LIBURING) && !defined(EXEC_BACKEND)
#define IOMETHOD_IO_URING_ENABLED
#endif

method_sync.c

The whole of the synchronous method is 47 lines. Every I/O is executed inline:

 c
 
 
static bool
pgaio_sync_needs_synchronous_execution(PgAioHandle *ioh)
{
	return true;
}

static int
pgaio_sync_submit(uint16 num_staged_ios, PgAioHandle **staged_ios)
{
	elog(ERROR, "IO should have been executed synchronously");

	return 0;
}

The actual preadv()/pwritev() live in aio_io.c, because the worker method falls back on them too:

 c
 
 
		case PGAIO_OP_READV:
			pgstat_report_wait_start(WAIT_EVENT_DATA_FILE_READ);
			result = pg_preadv(ioh->op_data.read.fd, iov,
							 ioh->op_data.read.iov_length,
							 ioh->op_data.read.offset);
			pgstat_report_wait_end();
			break;

io_method=sync exists to AIO what io_method=worker is to the whole server: a way to run the same code paths without the asynchronous behaviour, so you can measure what the asynchronous behaviour is worth.

method_worker.c

Worker mode is asynchronous from the issuing backend’s point of view but synchronous underneath: a pool of worker processes consumes IOs from a shared memory queue and runs ordinary blocking syscalls.

The interesting part is the decision of what cannot be handed to a worker:

 c
 
 
static bool
pgaio_worker_needs_synchronous_execution(PgAioHandle *ioh)
{
	return
		!IsUnderPostmaster
		|| ioh->flags & PGAIO_HF_REFERENCES_LOCAL
		|| !pgaio_io_can_reopen(ioh);
}

Three cases: we are not under a postmaster (standalone backend, no workers exist); the I/O references backend-local memory (a temporary table’s buffer); or the target cannot be reopened in another process.

Submission is deliberately best-effort — if the queue is full, or the queue lock cannot be taken, the remaining I/Os just happen inline in the submitting backend. The batch is handed to a single worker, which is then woken:

 c
 
 
	if (LWLockConditionalAcquire(AioWorkerSubmissionQueueLock, LW_EXCLUSIVE))
	{
		for (int i = 0; i < num_staged_ios; ++i)
		{
			Assert(!pgaio_worker_needs_synchronous_execution(staged_ios[i]));
			if (!pgaio_worker_submission_queue_insert(staged_ios[i]))
			{
				/*
				 * Do the rest synchronously. If the queue is full, give up
				 * and do the rest synchronously. We're holding an exclusive
				 * lock on the queue so nothing can consume entries.
				 */
				synchronous_ios = &staged_ios[i];
				nsync = (num_staged_ios - i);

				break;
			}
		}
		/* Choose one worker to wake for this batch. */
		worker = pgaio_worker_choose_idle(-1);
		LWLockRelease(AioWorkerSubmissionQueueLock);

		/* Wake up chosen worker. It will wake peers if necessary. */
		if (worker != -1)
			pgaio_worker_wake(worker);
	}

The worker pool is sized by io_min_workers (2) and io_max_workers (8), and the workers themselves are scaled dynamically based on recent demand.

method_io_uring.c

Now the interesting one. io_uring is the method where the I/O is submitted from inside the backend, so no context switch is needed to hand the work over — which is exactly why the README says it “lowers context switch rate / latency”.

One ring per backend, created by the postmaster

 c
 
 
/*
 * Per-backend state when using io_method=io_uring
 */
typedef struct PgAioUringContext
{
	/*
	 * Align the whole struct to a cacheline boundary, to prevent false
	 * sharing between completion_lock and prior backend's io_uring_ring.
	 */
	alignas(PG_CACHE_LINE_SIZE)

	/*
	 * Multiple backends can process completions for this backend's io_uring
	 * instance (e.g. when the backend issuing IO is busy doing something
	 * else). To make that safe we have to ensure that only a single backend
	 * gets io completions from the io_uring instance at a time.
	 */
	LWLock		completion_lock;

	struct io_uring io_uring_ring;
} PgAioUringContext;

The rings are created in the postmaster, at shared memory init time, one per possible backend. The file header explains why:

For now we create one io_uring instance for each backend. These io_uring instances have to be created in postmaster, during startup, to allow other backends to process IO completions, if the issuing backend is currently busy doing other things. Other backends may not use another backend’s io_uring instance to submit IO, that’d require additional locking that would likely be harmful for performance.

So: backend A owns ring A and submits to it. Backend B may complete A’s I/Os if A is busy — that is the deadlock-avoidance requirement from the README — but B never submits into A’s ring.

The queue depth is io_max_concurrency, the same value that sizes the handle pool:

 c
 
		ret = io_uring_queue_init(io_max_concurrency, &context->io_uring_ring, 0);

There is a newer optimization in here worth noting: on kernels 6.5+ with a recent liburing, the ring’s memory can be placed in PostgreSQL’s own shared memory instead of a separate mmap per backend, because “a large number of memory mappings slows things down, backend exit is particularly affected”.

Submitting: batching

pgaio_uring_submit() turns staged handles into io_uring submission queue entries and submits them in one syscall:

 c
 
 
static int
pgaio_uring_submit(uint16 num_staged_ios, PgAioHandle **staged_ios)
{
	struct io_uring *uring_instance = &pgaio_my_uring_context->io_uring_ring;

	Assert(num_staged_ios <= PGAIO_SUBMIT_BATCH_SIZE);

	for (int i = 0; i < num_staged_ios; i++)
	{
		PgAioHandle *ioh = staged_ios[i];
		struct io_uring_sqe *sqe;

		sqe = io_uring_get_sqe(uring_instance);

		if (!sqe)
			elog(ERROR, "io_uring submission queue is unexpectedly full");

		pgaio_io_prepare_submit(ioh);
		pgaio_uring_sq_from_io(ioh, sqe);
	}

	while (true)
	{
		int			ret;

		pgstat_report_wait_start(WAIT_EVENT_AIO_IO_URING_SUBMIT);
		ret = io_uring_submit(uring_instance);
		pgstat_report_wait_end();

One io_uring_submit() syscall for up to PGAIO_SUBMIT_BATCH_SIZE I/Os, which is 32:

#define PGAIO_SUBMIT_BATCH_SIZE 32

That batching is what pgaio_enter_batchmode() is for. The deadlock hazard is spelled out in aio.c:

Consider backend A staging an IO for buffer 1 and then trying to start IO on buffer 2, while backend B does the inverse. If A submitted the IO before moving on to buffer 2, this works just fine, B will wait for the IO to complete. But if batching were used, each backend will wait for IO that has not yet been submitted to complete, i.e. forever.

So batch mode is opt-in and the caller must promise not to block on another backend while holding unsubmitted I/O.

The error handling on submit is unusually blunt, and the reasoning is in the comment:

 c
 
 
		else if (ret < 0)
		{
			/*
			 * The io_uring_enter() manpage suggests that the appropriate
			 * reaction to EAGAIN is:
			 *
			 * "The application should wait for some completions and try
			 * again"
			 *
			 * However, it seems unlikely that that would help in our case, as
			 * we apply a low limit to the number of outstanding IOs and thus
			 * also outstanding completions, making it unlikely that we'd get
			 * EAGAIN while the OS is in good working order.
			 *
			 * Additionally, it would be problematic to just wait here, our
			 * caller might hold critical locks. It'd possibly lead to
			 * delaying the crash-restart that seems likely to occur when the
			 * kernel is under such heavy memory pressure.
			 *
			 * Update errno to allow %m to work.
			 */
			errno = -ret;
			elog(PANIC, "io_uring submit failed: %m");
		}

Building the SQE

pgaio_uring_sq_from_io() translates an AIO handle into an io_uring submission queue entry:

 c
 
 
static void
pgaio_uring_sq_from_io(PgAioHandle *ioh, struct io_uring_sqe *sqe)
{
	struct iovec *iov;
	size_t		io_size = 0;

	switch ((PgAioOp) ioh->op)
	{
		case PGAIO_OP_READV:
			iov = &pgaio_ctl->iovecs[ioh->iovec_off];
			if (ioh->op_data.read.iov_length == 1)
			{
				io_uring_prep_read(sqe,
								 ioh->op_data.read.fd,
								 iov->iov_base,
								 iov->iov_len,
								 ioh->op_data.read.offset);

				io_size = iov->iov_len;
			}
			else
			{
				io_uring_prep_readv(sqe,
									ioh->op_data.read.fd,
									iov,
									ioh->op_data.read.iov_length,
									ioh->op_data.read.offset);

				for (int i = 0; i < ioh->op_data.read.iov_length; i++, iov++)
					io_size += iov->iov_len;
			}

			if (pgaio_uring_should_use_async(ioh, io_size))
				io_uring_sqe_set_flags(sqe, IOSQE_ASYNC);

			break;

The single-iov case uses io_uring_prep_read, the multi-iov case uses io_uring_prep_readv. IOSQE_ASYNC is the interesting flag: it tells the kernel to hand the request to a worker thread rather than executing it in the submitting process’s context.

Note that writes deliberately do not get IOSQE_ASYNC:

 c
 
 
			/*
			 * For now don't trigger use of IOSQE_ASYNC for writes, it's not
			 * clear there is a performance benefit in doing so.
			 */

When to go asynchronous

pgaio_uring_should_use_async() is the heuristic, and its comment is one of the best explanations of the io_uring-vs-worker tradeoff in the tree:

io_uring executes IO in process context if possible. That’s generally good, as it reduces context switching. When performing a lot of buffered IO that means that copying between page cache and userspace memory happens in the foreground, as it can’t be offloaded to DMA hardware as is possible when using direct IO. When executing a lot of buffered IO this causes io_uring to be slower than worker mode, as worker mode parallelizes the copying. io_uring can be told to offload work to worker threads instead.

If the IOs are small, we only benefit from forcing things into the background if there is a lot of IO, as otherwise the overhead from context switching is higher than the gain.

If IOs are large, there is benefit from asynchronous processing at lower queue depths, as IO latency is less of a crucial factor and parallelizing memory copies is more important. In addition, it is important to trigger asynchronous processing even at low queue depth, as with foreground processing we might never actually reach deep enough IO depths to trigger asynchronous processing, which in turn would deprive readahead control logic of information about whether a deeper look-ahead distance would be advantageous.

And the code:

 c
 
 
static bool
pgaio_uring_should_use_async(PgAioHandle *ioh, size_t io_size)
{
	/*
	 * With DIO there's no benefit from forcing asynchronous processing, as
	 * io_uring will never execute direct IO synchronously during submission.
	 */
	if (!(ioh->flags & PGAIO_HF_BUFFERED))
		return false;

	/*
	 * Once the IO queue depth is not that shallow anymore, the overhead of
	 * dispatching to the background is a less significant factor.
	 */
	if (dclist_count(&pgaio_my_backend->in_flight_ios) > 4)
		return true;

	/*
	 * If the IO is larger, the gains from parallelizing the memory copy are
	 * larger and typically the impact of the latency is smaller.
	 */
	if (io_size >= (BLCKSZ * 4))
		return true;

	return false;
}

Three rules, then:

Condition	Result
Not buffered (i.e. direct I/O)	never `IOSQE_ASYNC` — the kernel never does DIO inline anyway
More than 4 I/Os already in flight	always `IOSQE_ASYNC`
I/O size ≥ 4 × `BLCKSZ` (32 kB)	always `IOSQE_ASYNC`
Otherwise	foreground processing

The PGAIO_HF_BUFFERED flag is set by the storage manager when direct I/O is not in use:

 c
 
 
	if (!(io_direct_flags & IO_DIRECT_DATA))
		pgaio_io_set_flag(ioh, PGAIO_HF_BUFFERED);

Note the third condition — 32 kB, four blocks — is exactly where read-ahead starts combining blocks. The two heuristics are tuned together, which is a nice example of the subsystem being designed as a whole rather than in pieces.

Completing

Completions are drained under the completion lock, at most PGAIO_MAX_LOCAL_COMPLETED_IO (32) at a time, and inside a critical section:

 c
 
 
	orig_ready = ready = io_uring_cq_ready(&context->io_uring_ring);

	while (ready > 0)
	{
		struct io_uring_cqe *cqes[PGAIO_MAX_LOCAL_COMPLETED_IO];
		uint32		ncqes;

		START_CRIT_SECTION();
		ncqes =
			io_uring_peek_batch_cqe(&context->io_uring_ring,
									cqes,
									Min(PGAIO_MAX_LOCAL_COMPLETED_IO, ready));
		Assert(ncqes <= ready);

		ready -= ncqes;

		for (uint32 i = 0; i < ncqes; i++)
		{
			struct io_uring_cqe *cqe = cqes[i];
			PgAioHandle *ioh = io_uring_cqe_get_data(cqe);
			int			result = cqe->res;

			errcallback.arg = ioh;

			io_uring_cqe_seen(&context->io_uring_ring, cqe);

			pgaio_io_process_completion(ioh, result);
			errcallback.arg = NULL;
		}

		END_CRIT_SECTION();
	}

The START_CRIT_SECTION() / END_CRIT_SECTION() pair is not decoration: completions must be processable inside a critical section so that WAL writes can complete I/O there.

pgaio_uring_wait_one() is the blocking wait, and it is where the “other backends may complete my I/O” logic lives:

 c
 
 
static void
pgaio_uring_wait_one(PgAioHandle *ioh, uint64 ref_generation)
{
	PgAioHandleState state;
	ProcNumber	owner_procno = ioh->owner_procno;
	PgAioUringContext *owner_context = &pgaio_uring_contexts[owner_procno];
	bool		expect_cqe;
	int			waited = 0;

	/*
	 * XXX: It would be nice to have a smarter locking scheme, nearly all the
	 * time the backend owning the ring will consume the completions, making
	 * the locking unnecessarily expensive.
	 */
	LWLockAcquire(&owner_context->completion_lock, LW_EXCLUSIVE);

	while (true)
	{
		...
		if (pgaio_io_was_recycled(ioh, ref_generation, &state) ||
			state != PGAIO_HS_SUBMITTED)
		{
			/* the IO was completed by another backend */
			break;
		}
		else if (io_uring_cq_ready(&owner_context->io_uring_ring))
		{
			/* no need to wait in the kernel, io_uring has a completion */
			expect_cqe = true;
		}
		else
		{
			int			ret;
			struct io_uring_cqe *cqes;

			/* need to wait in the kernel */
			pgstat_report_wait_start(WAIT_EVENT_AIO_IO_URING_EXECUTION);
			ret = io_uring_wait_cqes(&owner_context->io_uring_ring, &cqes, 1, NULL, NULL);
			pgstat_report_wait_end();

pgaio_uring_check_one() is the non-blocking variant, used by pgaio_wref_check_done():

 c
 
 
static void
pgaio_uring_check_one(PgAioHandle *ioh, uint64 ref_generation)
{
	ProcNumber	owner_procno = ioh->owner_procno;
	PgAioUringContext *owner_context = &pgaio_uring_contexts[owner_procno];

	/*
	 * This check is not reliable when not holding the completion lock, but
	 * it's a useful cheap pre-check to see if it's worth trying to get the
	 * completion lock.
	 */
	if (!io_uring_cq_ready(&owner_context->io_uring_ring))
		return;

	/*
	 * If the completion lock is currently held, the holder will likely
	 * process any pending completions, give up.
	 */
	if (!LWLockConditionalAcquire(&owner_context->completion_lock, LW_EXCLUSIVE))
		return;

The `IOSQE_ASYNC` / close-FD interaction

One more io_uring-specific hazard, recorded in the ops table: while io_uring is mostly fine with an FD being closed while the I/O is in flight, that is not true for IOs submitted with IOSQE_ASYNC. Hence .wait_on_fd_before_close = true, which makes pgaio_closing_fd() wait for any in-flight I/O referencing that file descriptor before it is closed. This is handled generically in aio.c, which is why the flag exists on the ops structure at all.

The read-ahead controller

Here is where the async I/O subsystem stops being about syscalls and starts being about scheduling.

src/backend/storage/aio/read_stream.c implements a “read stream”: an object that, given a callback producing block numbers, hands the caller one pinned buffer at a time while arranging the underlying I/O to be issued early and in large batches.

The file header explains the algorithm:

The algorithm for controlling the look-ahead distance is based on recent cache / miss history, as well as whether we need to wait for I/O completion after a miss. When no I/O is necessary, there is no benefit in looking ahead more than one block. This is the default initial assumption. When blocks needing I/O are streamed, the combine distance is increased to benefit from I/O combining and the read-ahead distance is increased whenever we need to wait for I/O to try to benefit from increased I/O concurrency. Both are reduced gradually when cached blocks are streamed.

Two distances, and the distinction between them is subtle:

Field	Meaning	Capped by
`readahead_distance`	How far ahead of the current position to look	`max_pinned_buffers`
`combine_distance`	How large a single I/O request may grow	`io_combine_limit`

They were originally the same variable. Commit 8ca147d582a split them:

In a subsequent commit the read-ahead distance will only be increased when waiting for IO. Without further work that would cause a regression: As IO combining and read-ahead are currently controlled by the same mechanism, we would end up not allowing IO combining when never needing to wait for IO (as the distance ends up too small to allow for full sized IOs), which can increase CPU overhead. A typical reason to not have to wait for IO completion at a low look-ahead distance is use of io_uring with the to-be-read data in the page cache.

Two decisions, two functions

read_stream_should_look_ahead() decides whether to pull more block numbers from the callback; read_stream_should_issue_now() decides whether to actually start the pending I/O:

 c
 
 
static inline bool
read_stream_should_look_ahead(ReadStream *stream)
{
	/* If the callback has signaled end-of-stream, we're done */
	if (stream->readahead_distance == 0)
		return false;

	/* never start more IOs than our cap */
	if (stream->ios_in_progress >= stream->max_ios)
		return false;

	/*
	 * Don't start more read-ahead if that'd put us over the distance limit
	 * for doing read-ahead. As stream->readahead_distance is capped by
	 * max_pinned_buffers, this prevents us from looking ahead so far that it
	 * would put us over the pin limit.
	 */
	if (stream->pinned_buffers + stream->pending_read_nblocks >= stream->readahead_distance)
		return false;

	return true;
}

 c
 
 
static inline bool
read_stream_should_issue_now(ReadStream *stream)
{
	int16		pending_read_nblocks = stream->pending_read_nblocks;

	/* there is no pending IO that could be issued */
	if (pending_read_nblocks == 0)
		return false;

	/* never start more IOs than our cap */
	if (stream->ios_in_progress >= stream->max_ios)
		return false;

	/*
	 * If the callback has signaled end-of-stream, start the pending read
	 * immediately. There is no further potential for IO combining.
	 */
	if (stream->readahead_distance == 0)
		return true;

	/*
	 * If we've already reached combine_distance, there's no chance of growing
	 * the read further.
	 */
	if (pending_read_nblocks >= stream->combine_distance)
		return true;

	/*
	 * If we currently have no reads in flight or prepared, issue the IO once
	 * we are not looking ahead further. This ensures there's always at least
	 * one IO prepared.
	 */
	if (stream->pinned_buffers == 0 &&
		!read_stream_should_look_ahead(stream))
		return true;

	return false;
}

The asymmetry between the two is the point. combine_distance decides how big an I/O may get; readahead_distance decides how far ahead we look. A scan that hits everything in cache should issue small I/Os of one block, but it should still combine them into fewer syscalls — which is why the two are separate.

read_stream_look_ahead() is the loop that drives both decisions, asking the callback for more block numbers and starting I/Os when it decides to:

 c
 
 
static void
read_stream_look_ahead(ReadStream *stream)
{
	/*
	 * Allow amortizing the cost of submitting IO over multiple IOs. This
	 * requires that we don't do any operations that could lead to a deadlock
	 * with staged-but-unsubmitted IO. The callback needs to opt-in to being
	 * careful.
	 */
	if (stream->batch_mode)
		pgaio_enter_batchmode();

	while (read_stream_should_look_ahead(stream))
	{
		...
		if (read_stream_should_issue_now(stream))
		{
			read_stream_start_pending_read(stream);
			continue;
		}
		...
		/* Can we merge it with the pending read? */
		if (stream->pending_read_nblocks > 0 &&
			stream->pending_read_blocknum + stream->pending_read_nblocks == blocknum)
		{
			stream->pending_read_nblocks++;
			continue;
		}

The closed loop

Here is the heart of it. When the consumer of the stream pulls a buffer and has to wait for I/O to complete, the read-ahead distance is doubled:

 c
 
 
		/*
		 * Have the read-ahead distance ramp up rapidly after we needed to
		 * wait for IO. We only increase the read-ahead-distance when we
		 * needed to wait, to avoid increasing the distance further than
		 * necessary, as looking ahead too far can be costly, both due to the
		 * cost of unnecessarily pinning many buffers and due to doing IOs
		 * that may never be consumed if the stream is ended/reset before
		 * completion.
		 *
		 * If we did not need to wait, the current distance was evidently
		 * sufficient.
		 *
		 * NB: Must not increase the distance if we already reached the end of
		 * the stream, as stream->readahead_distance == 0 is used to keep
		 * track of having reached the end.
		 */
		if (stream->readahead_distance > 0 && needed_wait)
		{
			/* wider temporary value, due to overflow risk */
			int32		readahead_distance;

			readahead_distance = stream->readahead_distance * 2;
			readahead_distance = Min(readahead_distance, stream->max_pinned_buffers);
			stream->readahead_distance = readahead_distance;
		}

And the decay, which happens when the I/O did not have to wait:

 c
 
 
		if (stream->ios_in_progress == 0)
		{
			if (stream->distance_decay_holdoff > 0)
				stream->distance_decay_holdoff--;
			else
			{
				if (stream->readahead_distance > 1)
					stream->readahead_distance--;
				...
				if (stream->combine_distance > 1)
					stream->combine_distance--;
			}
		}

The decay is guarded by distance_decay_holdoff, which is reset whenever an I/O is issued:

 c
 
 
		/*
		 * As we needed IO, prevent distances from being reduced within our
		 * maximum look-ahead window. This avoids collapsing distances too
		 * quickly in workloads where most of the required blocks are cached,
		 * but where the remaining IOs are a sufficient enough factor to cause
		 * a substantial slowdown if executed synchronously.
		 *
		 * There are valid arguments for preventing decay for max_ios or for
		 * max_pinned_buffers. But the argument for max_pinned_buffers seems
		 * clearer - if we can't see any misses within the maximum look-ahead
		 * distance, we can't do any useful read-ahead.
		 */
		stream->distance_decay_holdoff = stream->max_pinned_buffers;

And the combine distance grows on any I/O, wait or not:

 c
 
 
		/*
		 * Whether we needed to wait or not, allow for more IO combining if we
		 * needed to do IO. The reason to do so independent of needing to wait
		 * is that when the data is resident in the kernel page cache, IO
		 * combining reduces the syscall / dispatch overhead, making it
		 * worthwhile regardless of needing to wait.
		 *
		 * It is also important with io_uring as it will never signal the need
		 * to wait for reads if all the data is in the page cache. There are
		 * heuristics to deal with that in method_io_uring.c, but they only
		 * work when the IO gets large enough.
		 */

This last comment is a direct link between the two halves of this article. io_uring will happily serve a read out of the page cache without ever blocking, so the “did I wait?” signal — the entire basis of the closed loop — goes silent exactly in the case where io_uring is fastest. The combine_distance mechanism is the escape hatch: it grows the I/O size until io_uring’s IOSQE_ASYNC heuristic kicks in and starts paying attention to how deep the queue is.

The commit that introduced the wait-conditional growth is f63ca337902, and its message is explicit about the trade-off:

This avoids increasing the distance to the maximum in cases where the I/O subsystem is already keeping up. This turns out to be important for performance for two reasons:

Pinning a lot of buffers is not cheap. If additional pins allow us to avoid IO waits, it’s definitely worth it, but if we can already do all the necessary readahead at a distance of 16, reading ahead 512 buffers can increase the CPU overhead substantially. This is particularly noticeable when the to-be-read blocks are already in the kernel page cache.

If the read stream is read to completion, reading in data earlier than needed is of limited consequences, leaving aside the CPU costs mentioned above. But if the read stream will not be fully consumed, e.g. because it is on the inner side of a nested loop join, the additional IO can be a serious performance issue.

So the controller is not “look ahead as far as possible”. It is “look ahead exactly as far as needed to avoid waiting, and no further”.

The limits

max_pinned_buffers is derived from the buffer manager’s own limits:

 c
 
 
	max_pinned_buffers = (max_ios + 1) * io_combine_limit;
	max_pinned_buffers = Min(max_pinned_buffers,
							 PG_INT16_MAX - queue_overflow - 1);

	/* Give the strategy a chance to limit the number of buffers we pin. */
	strategy_pin_limit = GetAccessStrategyPinLimit(strategy);
	max_pinned_buffers = Min(strategy_pin_limit, max_pinned_buffers);

max_ios comes from effective_io_concurrency (or the tablespace’s override). So the whole controller is bounded by three knobs a DBA already knows: effective_io_concurrency, io_combine_limit, and the buffer manager’s per-backend pin limit.

The buffer manager’s side

The buffer manager does not know about io_uring. It knows about StartBufferIO() and WaitReadBuffers().

StartSharedBufferIO() in src/backend/storage/buffer/bufmgr.c is where “someone else is already reading this buffer” is handled — and with AIO, “someone else” can be another backend entirely:

 c
 
 
	for (;;)
	{
		buf_state = LockBufHdr(buf);

		if (!(buf_state & BM_IO_IN_PROGRESS))
			break;

		/* Join the existing IO */
		if (io_wref != NULL && pgaio_wref_valid(&buf->io_wref))
		{
			*io_wref = buf->io_wref;
			UnlockBufHdr(buf);

			return BUFFER_IO_IN_PROGRESS;
		}

Joining an in-flight I/O is strictly better than waiting for it: the joining backend attaches its own wait reference to the same I/O and continues doing other work.

Then there is AsyncReadBuffers(), which is the function that actually starts the read:

 c
 
 
	/*
	 * We must get an IO handle before StartBufferIO(), as pgaio_io_acquire()
	 * might block, which we don't want after setting IO_IN_PROGRESS. If we
	 * don't need to do the IO, we'll release the handle.
	 *
	 * If we need to wait for IO before we can get a handle, submit
	 * already-staged IO first, so that other backends don't need to wait.
	 */
	ioh = pgaio_io_acquire_nb(CurrentResourceOwner, &operation->io_return);
	if (unlikely(!ioh))
	{
		pgaio_submit_staged();
		ioh = pgaio_io_acquire(CurrentResourceOwner, &operation->io_return);
	}

Note pgaio_io_acquire_nb() — the non-blocking variant — and the fallback that submits staged I/O before blocking. Blocking while holding staged-but-unsubmitted I/O is exactly the deadlock the batch-mode comment warns about.

Relation extension is not an AIO consumer

The other half of the buffer manager’s I/O surface is relation extension. In older PostgreSQL this was ExtendBufferedIO(); in this tree it is ExtendBufferedRel() / ExtendBufferedRelBy() in src/backend/storage/buffer/bufmgr.c, which does not read anything. It asks the storage manager for the current size, then takes victim buffers and zeroes them:

 c
 
 
	first_block = smgrnblocks(BMR_GET_SMGR(bmr), fork);
	...
	for (uint32 i = 0; i < extend_by; i++)
	{
		Buffer		buf_block;

		buffers[i] = GetVictimBuffer(strategy, io_context);
		buf_block = BufHdrGetBlock(GetBufferDescriptor(buffers[i] - 1));

		/* new buffers are zero-filled */
		MemSet(buf_block, 0, BLCKSZ);
	}

So extending a relation is not an asynchronous-read workload, and it does not participate in the AIO machinery. The pages it creates are written later through the ordinary dirty-buffer flush path.

Direct I/O, for real

Direct I/O is now reachable, and the README’s warning is the reason it was worth building AIO first.

With the debug_io_direct GUC (a developer option, PGC_POSTMASTER) set, mdstartreadv() simply stops setting the buffered flag:

 c
 
 
	if (!(io_direct_flags & IO_DIRECT_DATA))
		pgaio_io_set_flag(ioh, PGAIO_HF_BUFFERED);

and pgaio_uring_should_use_async() then always returns false for those I/Os, because the kernel never performs a direct read inline during submission anyway.

The double-buffering saving is real: with direct I/O, the data goes straight from the device into PostgreSQL’s shared buffers via DMA, with no copy through the kernel page cache. But the price is that PostgreSQL now owns the readahead problem entirely — which is what read_stream.c is for, and why the README says:

Even with AIO, many parts of postgres need to be modified to perform explicit prefetching.

Instrumenting it: EXPLAIN (IO)

The subsystem ships with instrumentation, exposed through EXPLAIN (IO). The option is parsed in src/backend/commands/explain_state.c:

 c
 
 
		else if (strcmp(opt->defname, "io") == 0)
			es->io = defGetBoolean(opt);

and printed by print_io_usage() in src/backend/commands/explain.c:

 c
 
 
			ExplainIndentText(es);
			appendStringInfo(es->str, "Prefetch: avg=%.2f max=%d capacity=%d\n",
							 (stats->distance_sum * 1.0 / stats->prefetch_count),
							 stats->distance_max,
							 stats->distance_capacity);

			/* prefetch I/O info (only if there were actual I/Os) */
			if (stats->io_count > 0)
			{
				ExplainIndentText(es);
				appendStringInfo(es->str, "I/O: count=%" PRIu64 " waits=%" PRIu64
								 " size=%.2f in-progress=%.2f\n",
								 stats->io_count, stats->wait_count,
								 (stats->io_nblocks * 1.0 / stats->io_count),
								 (stats->io_in_progress * 1.0 / stats->io_count));
			}

The counters themselves come from read_stream.c:

 c
 
 
static inline void
read_stream_count_prefetch(ReadStream *stream)
{
	IOStats *stats = stream->stats;

	if (stats == NULL)
		return;

	stats->prefetch_count++;
	stats->distance_sum += stream->pinned_buffers;
	if (stream->pinned_buffers > stats->distance_max)
		stats->distance_max = stream->pinned_buffers;
}

So EXPLAIN (ANALYZE, IO) on a sequential scan will tell you the average look-ahead distance the controller settled on, the maximum it reached, and how many times it actually had to wait. That is the feedback loop made visible — and it is the first thing to look at when tuning effective_io_concurrency on a system that is using io_method=io_uring.

The instrumentation is being added incrementally: currently it is wired up for scan nodes that use read streams (sequential scans, bitmap heap scans, TID range scans), and the IOStats structure (src/include/executor/instrument_node.h) is a new addition rather than a long-standing feature.

Caveats

io_uring is Linux-only, and off by default. io_method=worker is the default; io_method=io_uring requires a build with liburing and a non-EXEC_BACKEND configuration. io_method is PGC_POSTMASTER, so changing it requires a restart.
The kernel version matters. The README says “available on Linux 5.1+”. Newer kernels (6.5+) allow the ring memory to live in PostgreSQL’s shared memory, which avoids one mmap per backend.
Many code paths still need explicit prefetching. The README is explicit: “Even with AIO, many parts of postgres need to be modified to perform explicit prefetching.” Read streams exist for the access patterns that have been converted; others have not been.
The write path is not converted yet. The AIO API has a write operation — PGAIO_OP_WRITEV, pgaio_io_start_writev(), and the PGAIO_OP_WRITEV branch in pgaio_uring_sq_from_io() — but in this tree it has no callers outside the AIO subsystem itself. Ordinary relation writes still go through mdwritev(), which does a plain pwritev(), and the WAL write path has not been converted either. So the README’s claim that asynchronous WAL writes “can yield significantly higher throughput” describes the design goal, not something you can turn on and measure yet.
Direct I/O is still a sharp tool. It requires that shared_buffers be large enough, that the workload tolerate the loss of the kernel’s page cache, and that the access patterns have been converted to read streams.
The io_uring heuristics are empirical. The comment on pgaio_uring_should_use_async() says outright: “We have done some basic benchmarking to validate the thresholds used, but it’s quite plausible that there are better values.” The > 4 in-flight threshold and the BLCKSZ * 4 size threshold are starting points, not laws.
EXPLAIN (IO) coverage is partial. It exists and works for the scan nodes that use read streams, but it is new and being extended.

What it all means in practice

A few consequences worth carrying around:

Async I/O is a scheduling problem, not a syscall problem. The interesting code in this subsystem is read_stream.c, not method_io_uring.c. The io_uring backend is a fairly thin layer; the closed loop that decides when to look ahead is where the value is.
“Did I wait?” is the signal. A read-ahead controller that grows unconditionally is a memory hog; one that never grows is synchronous I/O with extra steps. Doubling the distance on a wait and halving it otherwise is a simple, effective feedback loop.
Batching and deadlock avoidance pull in opposite directions. The batch API gives you one syscall for 32 I/Os, but it also gives you a way to deadlock with yourself. The code is careful about this, and the comments explain why at each site.
The shared-memory constraint shapes everything. Callbacks are IDs, not pointers. Targets exist so a file can be reopened. Completion happens in critical sections. Almost every awkward-looking decision in this subsystem traces back to “the state has to be visible to other processes.”

Internals on Opensource internals

PostgreSQL Stores a Row

A row is not a record

Line pointers: the indirection that makes everything possible

The tuple header: 23 bytes

Five virtual fields in three physical ones

t_infomask

t_infomask2

The null bitmap and the 23/24 trick

The data area

Variable-length values: the four header formats

TOAST: when a row doesn’t fit

The TOAST table

UPDATE: the version chain

Visibility: the rule

HOT: updating without touching the index

Pruning: reclaiming the space

Two side structures

The free space map

The visibility map

Look at it yourself

What it all means in practice

PostgreSQL's time and timetz Implementation in C

The catalog entries

The C representation

Microseconds since midnight

Typmod and rounding

timetz: the 12-byte puzzle

Comparing timetz

Arithmetic

Look at it yourself

What it all means

Further reading

PostgreSQL Speeds Up Hash Joins

Where the time goes

Hashing as an expression

Building the hash expression

The interpreter

Pattern-matching the step list

The specialised functions

The strict variant

The seeded variant

The virtual-slot variant

The payoff

Hardcoding the NULL check

The NULL-key bug

The hash table side

Memoize: a hash join that remembers

The probe slot

The LRU eviction

16 bytes per tuple

The JIT has its own copy

Caveats

What it all means in practice

Further reading

PostgreSQL's numeric Implementation in C

The catalog entry

On disk: two headers over base-10000 digits

In memory: NumericVar

Input and output

make_result: stripping zeros and choosing a header

Two non-obvious decisions

Look at it yourself

Key takeaways

Further reading

PostgreSQL Waits for Locks

The ladder, in one table

Layer 1: the spinlock

The hardware primitives

The free test

The pause instruction

The memory ordering contract

The generic spin loop

perform_spin_delay()

Why the randomness matters

The adaptive spin count

Stuck spinlock detection

Layer 2: the atomics

The x86 implementations

The API contract