Opensource internals

PostgreSQL Defers and Speeds Up Tuple Deformation

36 min read Jay Hubbert

Reading a column out of a heap tuple used to mean walking every attribute before it; PostgreSQL now caches byte offsets in the TupleDesc and skips NULL bitmap bytes, so fetching column 40 can cost one addition.

A heap tuple is a packed byte string with no field boundaries in it. Column 3 might be at byte 27, column 4 at byte 31, column 5 — if it is a text — at byte 35. Getting to column 40 means knowing where all 39 attributes before it ended.

For most of PostgreSQL’s life, that meant walking. The deformer would start at the beginning of the tuple, add up the length of attribute 1, align to attribute 2’s boundary, add its length, check the NULL bitmap, and so on. The cost of reading the last column of a wide table was proportional to the number of columns before it, and the whole tuple was walked again for every single column.

PostgreSQL 20 changes this in two ways: the byte offset of every fixed-width attribute is computed once per TupleDesc and cached, and the NULL bitmap is processed eight bits at a time instead of one.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

Why deformation is worth optimising

Deformation is the step between “a tuple is on disk” and “a query has a value”. It happens for every row, for every column that the query actually wants. For an OLAP query over a wide table, it can easily be the dominant cost of the whole plan.

The commit that started this work is c456e391138, “Optimize tuple deformation”, by David Rowley:

For some workloads, tuple deformation can be the most CPU intensive part of processing the query. Some testing with 16 columns on a table where the first column was variable length showed around a 10% increase in transactions per second for an OLAP type query performing aggregation on the 16th column. However, in certain cases, the increases were much higher, up to ~25% on one AMD Zen4 machine.

(That quote is from the earlier commit 5983a4cffc3, “Introduce CompactAttribute array in TupleDesc, take 2”, which is the foundation this one builds on. More on it below.)

The problem with walking

Here is the classic shape of the problem. A tuple’s data area is laid out in attribute order, with each attribute aligned to its type’s requirement:

  • int4 needs 4-byte alignment and occupies 4 bytes.
  • int8 needs 8-byte alignment and occupies 8 bytes.
  • text is a varlena: either a 1-byte header (payload ≤ 126 bytes, no alignment) or a 4-byte header (aligned).

To find attribute N you need the cumulative length of attributes 1..N-1, including the padding bytes inserted for alignment. And NULL attributes occupy zero bytes, so you have to consult the NULL bitmap to know whether to add a length at all.

So the deformer had to do, for each attribute:

c 
	if (att_isnull(attnum - 1, tup->t_data->t_bits))
	{
		*isnull = true;
		return (Datum) 0;
	}
	else
		return nocachegetattr(tup, attnum, tupleDesc);

One bitmap bit test per attribute, plus a full re-walk of the tuple for every column fetched. (That shape survives today, but only for tuples that have NULLs and whose target attribute has no cached offset — see fastgetattr() below.)

The TupleDesc is where the knowledge lives

The fix is to move the per-tuple work into the per-TupleDesc work. A TupleDesc describes the shape of a tuple — the columns, their types, their alignment, their nullability — and that shape does not change from row to row. So anything that can be computed once per shape should be.

src/include/access/tupdesc.h holds the result:

c 
/*
 * firstNonCachedOffsetAttr stores the index into the compact_attrs array for
 * the first attribute that we don't have a known attcacheoff for.
 *
 * firstNonGuaranteedAttr stores the index to into the compact_attrs array for
 * the first attribute that is either NULLable, missing, or !attbyval.  This
 * can be used in locations as a guarantee that attributes before this will
 * always exist in tuples.  The !attbyval part isn't required for this, but
 * including this allows various tuple deforming routines to forego any checks
 * for !attbyval.
 *
 * Once a TupleDesc has been populated, before it is used for any purpose,
 * TupleDescFinalize() must be called on it.
 */
typedef struct TupleDescData
{
	int			natts;			/* number of attributes in the tuple */
	Oid			tdtypeid;		/* composite type ID for tuple type */
	int32		tdtypmod;		/* typmod for tuple type */
	int			tdrefcount;		/* reference count, or -1 if not counting */
	int			firstNonCachedOffsetAttr;	/* index of first compact_attrs
											 * element without an attcacheoff */
	int			firstNonGuaranteedAttr; /* index of the first nullable,
										 * missing, dropped, or !attbyval
										 * compact_attrs element. */
	TupleConstr *constr;		/* constraints, or NULL if none */
	/* compact_attrs[N] is the compact metadata of Attribute Number N+1 */
	CompactAttribute compact_attrs[FLEXIBLE_ARRAY_MEMBER];
}			TupleDescData;

Two integers and a per-attribute attcacheoff field. That is the entire cache.

CompactAttribute

The per-attribute metadata is stored in a parallel array of CompactAttribute structs, which is a cut-down version of FormData_pg_attribute:

c 
/*
 * CompactAttribute
 *		Cut-down version of FormData_pg_attribute for faster access for tasks
 *		such as tuple deformation.  The fields of this struct are populated
 *		using the populate_compact_attribute() function, which must be called
 *		directly after the FormData_pg_attribute struct is populated or
 *		altered in any way.
 *
 * Currently, this struct is 8 bytes.  Any code changes which enlarge this
 * struct should be considered very carefully.
 *
 * Code which must access a TupleDesc's attribute data should always make use
 * the fields of this struct when required fields are available here.  It's
 * more efficient to access the memory in CompactAttribute due to it being a
 * more compact representation of FormData_pg_attribute and also because
 * accessing the FormData_pg_attribute requires an additional calculations to
 * obtain the base address of the array within the TupleDesc.
 */
typedef struct CompactAttribute
{
	int16		attcacheoff;	/* fixed offset into tuple, if known, or -1 */
	int16		attlen;			/* attr len in bytes or -1 = varlen, -2 =
								 * cstring */
	bool		attbyval;		/* as FormData_pg_attribute.attbyval */
	uint8		attalignby;		/* alignment requirement in bytes */
	bool		attispackable:1;	/* FormData_pg_attribute.attstorage !=
									 * TYPSTORAGE_PLAIN */
	bool		atthasmissing:1;	/* as FormData_pg_attribute.atthasmissing */
	bool		attisdropped:1; /* as FormData_pg_attribute.attisdropped */
	bool		attgenerated:1; /* FormData_pg_attribute.attgenerated != '\0' */
	char		attnullability; /* status of not-null constraint, see below */
} CompactAttribute;

Eight bytes per column. The commit that introduced this array put it at 16 bytes; a later commit (d8a859d22b1, “Reduce size of CompactAttribute struct to 8 bytes”) halved it.

Compare that to FormData_pg_attribute, which is 104 bytes per column. The compact array is 13 times smaller, which means a 40-column table’s descriptor metadata fits in 320 bytes instead of 4160 — and, more importantly, the deformation loop touches one or two cache lines instead of seven.

The attcacheoff field is the cache itself. It is an int16, and -1 means “I don’t know where this attribute is”.

The accessor is trivial, but it has an assert-enabled sanity check:

c 
/*
 * Accessor for the i'th CompactAttribute element of tupdesc.
 */
static inline CompactAttribute *
TupleDescCompactAttr(TupleDesc tupdesc, int i)
{
	CompactAttribute *cattr = &tupdesc->compact_attrs[i];

#ifdef USE_ASSERT_CHECKING

	/* Check that the CompactAttribute is correctly populated */
	verify_compact_attribute(tupdesc, i);
#endif

	return cattr;
}

TupleDescFinalize: computing the cache once

The cache is built by TupleDescFinalize(), in src/backend/access/common/tupdesc.c:

c 
/*
 * TupleDescFinalize
 *		Finalize the given TupleDesc.  This must be called after the
 *		attributes arrays have been populated or adjusted by any code.
 *
 * Must be called after populate_compact_attribute() and before
 * BlessTupleDesc().
 */
void
TupleDescFinalize(TupleDesc tupdesc)
{
	int			firstNonCachedOffsetAttr = 0;
	int			firstNonGuaranteedAttr = tupdesc->natts;
	int			off = 0;

	for (int i = 0; i < tupdesc->natts; i++)
	{
		CompactAttribute *cattr = TupleDescCompactAttr(tupdesc, i);
		Form_pg_attribute attr = TupleDescAttr(tupdesc, i);

		/*
		 * Find the highest attnum which is guaranteed to exist in all tuples
		 * in the table.  We currently only pay attention to byval attributes
		 * to allow additional optimizations during tuple deformation.
		 */
		if (firstNonGuaranteedAttr == tupdesc->natts &&
			(cattr->attnullability != ATTNULLABLE_VALID || !cattr->attbyval ||
			 cattr->atthasmissing || cattr->attisdropped ||
			 cattr->attlen <= 0 ||
			 attr->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL))
			firstNonGuaranteedAttr = i;

		/*
		 * Don't cache offsets beyond fixed-width attributes.  Virtual
		 * generated attributes are stored as NULLs in the tuple, so we don't
		 * cache offsets beyond these.
		 */
		if (cattr->attlen <= 0 ||
			attr->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
			break;

		off = att_nominal_alignby(off, cattr->attalignby);

		/*
		 * attcacheoff is an int16, so don't try to cache any offsets larger
		 * than will fit in that type.  Any attributes which are offset more
		 * than 2^15 are likely due to variable-length attributes.  Since we
		 * don't cache offsets for or beyond variable-length attributes, using
		 * an int16 rather than an int32 here is unlikely to cost us anything.
		 */
		if (off > PG_INT16_MAX)
			break;

		cattr->attcacheoff = (int16) off;

		off += cattr->attlen;
		firstNonCachedOffsetAttr = i + 1;
	}

	tupdesc->firstNonCachedOffsetAttr = firstNonCachedOffsetAttr;
	tupdesc->firstNonGuaranteedAttr = firstNonGuaranteedAttr;
}

This is the whole algorithm. It walks the attributes once, tracking a running offset, and stops at the first attribute it cannot handle.

Why it stops at the first varlena

The loop breaks when cattr->attlen <= 0. A negative attlen means variable-length (-1) or cstring (-2). The reason is obvious once you think about it: a varlena’s size is not known from the descriptor — it is stored in the tuple. So the offset of the attribute after a varlena cannot be computed from the descriptor alone.

That means attcacheoff is only defined for the fixed-width prefix of a tuple’s columns. If your table is id int, name text, age int, then id has attcacheoff == 0 and name and age have attcacheoff == -1. The walk has to start over at the varlena.

firstNonCachedOffsetAttr records where the fixed-width prefix ends, so the deformer knows how far it can trust the cache.

Why the PG_INT16_MAX limit barely matters

c 
		if (off > PG_INT16_MAX)
			break;

attcacheoff is an int16, so the maximum representable offset is 32767. The comment argues that this is nearly free:

Any attributes which are offset more than 2^15 are likely due to variable-length attributes. Since we don’t cache offsets for or beyond variable-length attributes, using an int16 rather than an int32 here is unlikely to cost us anything.

The reasoning: to reach offset 32767 you would need 32 kB of fixed-width attributes. But a tuple’s data area is at most MaxHeapTupleSize = 8160 bytes. So a fixed-width prefix can never get near 32 kB anyway. The limit is belt-and-braces.

The guaranteed-attribute computation

The same loop computes firstNonGuaranteedAttr: the index of the first attribute that is not guaranteed to exist in every tuple of the relation.

An attribute is guaranteed if:

  • it has a valid NOT NULL constraint (ATTNULLABLE_VALID, not ATTNULLABLE_UNKNOWN — see below),
  • it is attbyval (passes its value by value, not by reference),
  • it has no missing value (atthasmissing),
  • it is not dropped,
  • its attlen > 0 (fixed width),
  • it is not a virtual generated column.

The attbyval restriction is not strictly necessary for the guarantee, but the comment explains the extra benefit:

We currently only pay attention to byval attributes to allow additional optimizations during tuple deformation.

and, from the header:

The !attbyval part isn’t required for this, but including this allows various tuple deforming routines to forego any checks for !attbyval.

By-restriction values would require emitting a pointer-fetching branch in the hot loop even though the fast path never needs it. Restricting the guarantee to by-value attributes keeps the guaranteed-attribute loop a straight-line sequence of aligned loads.

The nullability values are:

c 
/* Valid values for CompactAttribute->attnullability */
#define	ATTNULLABLE_UNRESTRICTED 'f'	/* No constraint exists */
#define	ATTNULLABLE_UNKNOWN		'u' /* constraint exists, validity unknown */
#define	ATTNULLABLE_VALID		'v' /* valid constraint exists */
#define	ATTNULLABLE_INVALID		'i' /* constraint exists, marked invalid */

and they are populated in populate_compact_attribute_internal():

c 
	/*
	 * Assign nullability status for this column.  Assuming that a constraint
	 * exists, at this point we don't know if a not-null constraint is valid,
	 * so we assign UNKNOWN unless the table is a catalog, in which case we
	 * know it's valid.
	 */
	dst->attnullability = !src->attnotnull ? ATTNULLABLE_UNRESTRICTED :
		IsCatalogRelationOid(src->attrelid) ? ATTNULLABLE_VALID :
		ATTNULLABLE_UNKNOWN;

Note the UNKNOWN state: a user table can have NOT NULL declared but not yet validated (or validated and then marked invalid), so the descriptor cannot assume the constraint holds until the constraint is confirmed valid. Catalogs, on the other hand, are trusted.

And the default attcacheoff is “unknown”:

c 
static inline void
populate_compact_attribute_internal(Form_pg_attribute src,
									CompactAttribute *dst)
{
	memset(dst, 0, sizeof(CompactAttribute));

	dst->attcacheoff = -1;
	dst->attlen = src->attlen;
	...
}

The fast path: fastgetattr

src/include/access/htup_details.h has the entry point:

c 
/*
 *	fastgetattr
 *		Fetch a user attribute's value as a Datum (might be either a
 *		value, or a pointer into the data area of the tuple).
 *
 *		This must not be used when a system attribute might be requested.
 *		Furthermore, the passed attnum MUST be valid.  Use heap_getattr()
 *		instead, if in doubt.
 *
 *		This gets called many times, so we macro the cacheable and NULL
 *		lookups, and call nocachegetattr() for the rest.
 */
static inline Datum
fastgetattr(HeapTuple tup, int attnum, TupleDesc tupleDesc, bool *isnull)
{
	Assert(attnum > 0);

	*isnull = false;
	if (HeapTupleNoNulls(tup))
	{
		CompactAttribute *att;

		att = TupleDescCompactAttr(tupleDesc, attnum - 1);
		if (att->attcacheoff >= 0)
			return fetchatt(att, (char *) tup->t_data + tup->t_data->t_hoff +
							att->attcacheoff);
		else
			return nocachegetattr(tup, attnum, tupleDesc);
	}
	else
	{
		if (att_isnull(attnum - 1, tup->t_data->t_bits))
		{
			*isnull = true;
			return (Datum) 0;
		}
		else
			return nocachegetattr(tup, attnum, tupleDesc);
	}
}

This is the payoff. The whole function is:

  1. If the tuple has no NULLs at all (HeapTupleNoNulls), skip the bitmap entirely.
  2. Look up attcacheoff for the requested attribute.
  3. If it is non-negative, the value is at t_data + t_hoff + attcacheoff — one addition, one load.

Fetching column 40 of a table whose first 40 columns are fixed-width really does cost one addition.

And the “no NULLs” test is a single bit test on the tuple header:

c 
static inline bool
HeapTupleHasNulls(const HeapTupleData *tuple)
{
	return (tuple->t_data->t_infomask & HEAP_HASNULL) != 0;
}

static inline bool
HeapTupleNoNulls(const HeapTupleData *tuple)
{
	return !HeapTupleHasNulls(tuple);
}

A tuple with no NULL attributes does not even have a NULL bitmap — the HEAP_HASNULL flag in t_infomask is not set, and the space for the bitmap was never reserved in t_hoff. So the whole bitmap is skipped, and there are no per-attribute NULL checks to make.

The value extraction itself:

c 
/*
 * Given an attbyval and an attlen from either a Form_pg_attribute or
 * CompactAttribute and a pointer into a tuple's data area, return the
 * correct value or pointer.
 *
 * We return a Datum value in all cases.  If attbyval is false,  we return
 * the same pointer into the tuple data area that we're passed.  Otherwise, we
 * return the correct number of bytes fetched from the data area and extended
 * to Datum form.
 *
 * Note that T must already be properly aligned for this to work correctly.
 */
#define fetchatt(A,T) fetch_att(T, (A)->attbyval, (A)->attlen)
c 
static inline Datum
fetch_att(const void *T, bool attbyval, int attlen)
{
	if (attbyval)
	{
		switch (attlen)
		{
			case sizeof(char):
				return CharGetDatum(*((const char *) T));
			case sizeof(int16):
				return Int16GetDatum(*((const int16 *) T));
			case sizeof(int32):
				return Int32GetDatum(*((const int32 *) T));
			case sizeof(int64):
				return Int64GetDatum(*((const int64 *) T));
			default:
				elog(ERROR, "unsupported byval length: %d", attlen);
				return 0;
		}
	}
	else
		return PointerGetDatum(T);
}

Because attcacheoff was computed with att_nominal_alignby(), the pointer is correctly aligned for the type — which is why the comment about T being properly aligned is safe to ignore.

The slow path: nocachegetattr

When the attribute is past the fixed-width prefix, or the tuple has NULLs, the work falls back to nocachegetattr() in src/backend/access/common/heaptuple.c:

c 
Datum
nocachegetattr(HeapTuple tup,
			   int attnum,
			   TupleDesc tupleDesc)
{
	CompactAttribute *cattr;
	HeapTupleHeader td = tup->t_data;
	char	   *tp;				/* ptr to data part of tuple */
	uint8	   *bp = td->t_bits;	/* ptr to null bitmap in tuple */
	int			off;			/* current offset within data */
	int			startAttr;
	int			firstNullAttr;
	int			i;
	bool		hasnulls = HeapTupleHasNulls(tup);

	/* Did someone forget to call TupleDescFinalize()? */
	Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);

	attnum--;

	/*
	 * To minimize the number of attributes we need to look at, start walking
	 * the tuple at the attribute with the highest attcacheoff prior to attnum
	 * or the first NULL attribute prior to attnum, whichever comes first.
	 */
	if (hasnulls)
		firstNullAttr = first_null_attr(bp, attnum);
	else
		firstNullAttr = attnum;

	if (tupleDesc->firstNonCachedOffsetAttr > 0 && firstNullAttr > 0)
	{
		/*
		 * Try to start with the highest attribute with an attcacheoff that's
		 * prior to the one we're looking for, or with the attribute prior to
		 * the first NULL attribute, if there is one.
		 */
		startAttr = Min(tupleDesc->firstNonCachedOffsetAttr - 1, firstNullAttr - 1);
		off = TupleDescCompactAttr(tupleDesc, startAttr)->attcacheoff;
	}
	else
	{
		/* Otherwise, start at the beginning... */
		startAttr = 0;
		off = 0;
	}

	tp = (char *) td + td->t_hoff;

	/*
	 * Calculate 'off' up to the first NULL attr.  We use two cheaper loops
	 * when the tuple has no variable-width columns.  When variable-width
	 * columns exists, we use att_addlength_pointer() to move the offset
	 * beyond the current attribute.
	 */
	if (!HeapTupleHasVarWidth(tup))
	{
		for (i = startAttr; i < firstNullAttr; i++)
		{
			cattr = TupleDescCompactAttr(tupleDesc, i);

			off = att_nominal_alignby(off, cattr->attalignby);
			off += cattr->attlen;
		}

		for (; i < attnum; i++)
		{
			if (att_isnull(i, bp))
				continue;

			cattr = TupleDescCompactAttr(tupleDesc, i);

			off = att_nominal_alignby(off, cattr->attalignby);
			off += cattr->attlen;
		}
	}
	else
	{
		for (i = startAttr; i < firstNullAttr; i++)
		{
			int			attlen;

			cattr = TupleDescCompactAttr(tupleDesc, i);
			attlen = cattr->attlen;
			off = att_pointer_alignby(off,
									  cattr->attalignby,
									  attlen,
									  tp + off);
			off = att_addlength_pointer(off, attlen, tp + off);
		}

		for (; i < attnum; i++)
		{
			int			attlen;

			if (att_isnull(i, bp))
				continue;

			cattr = TupleDescCompactAttr(tupleDesc, i);
			attlen = cattr->attlen;
			off = att_pointer_alignby(off, cattr->attalignby, attlen,
									  tp + off);
			off = att_addlength_pointer(off, attlen, tp + off);
		}
	}

	cattr = TupleDescCompactAttr(tupleDesc, attnum);
	off = att_pointer_alignby(off,
							  cattr->attalignby,
							  cattr->attlen,
							  tp + off);

	return fetchatt(cattr, tp + off);
}

Three optimisations live in here.

Restart from the highest cached offset

startAttr is Min(firstNonCachedOffsetAttr - 1, firstNullAttr - 1). The deformer does not start at attribute 0; it starts at the last attribute whose offset is cached and which is before the target, and takes its attcacheoff as the starting point.

So for a table with 10 fixed-width columns followed by a text column, if you ask for column 12, the walk starts at column 9’s cached offset and adds only columns 10, 11 and 12’s alignment and length. The 9 preceding columns are free.

The two-loop split

Each of the two loops above is split into two halves:

  • Before the first NULL: every attribute is non-NULL, so no bitmap check is needed at all.
  • From the first NULL onwards: each attribute is checked with att_isnull(i, bp).

That is the point of first_null_attr(): it tells you where the NULL bitmap stops being all-ones, so the prefix needs no checks.

The two halves are further split by whether the tuple has variable-width columns, because the two cases need different arithmetic:

  • Fixed-width only (!HeapTupleHasVarWidth): the offset advances by att_nominal_alignby(off, attalignby) plus attlen. No memory access needed to know the size.
  • Has varlena: the size is in the tuple, so you must read the tuple data to find it — att_pointer_alignby() (which knows that a short varlena needs no alignment) and att_addlength_pointer().

first_null_attr: finding the first NULL byte-wise

first_null_attr() lives in src/include/access/tupmacs.h:

c 
/*
 * first_null_attr
 *		Inspect a NULL bitmap from a tuple and return the 0-based attnum of the
 *		first NULL attribute.  Returns natts if no NULLs were found.
 *
 * This is coded to expect that 'bits' contains at least one 0 bit somewhere
 * in the array, but not necessarily < natts.  Note that natts may be passed
 * as a value lower than the number of bits physically stored in the tuple's
 * NULL bitmap, in which case we may not find a NULL and return natts.
 *
 * The reason we require at least one 0 bit somewhere in the NULL bitmap is
 * that the for loop that checks 0xFF bytes would loop to the last byte in
 * the array if all bytes were 0xFF, and the subsequent code that finds the
 * right-most 0 bit would access the first byte beyond the bitmap.  Provided
 * we find a 0 bit before then, that won't happen.  Since tuples which have no
 * NULLs don't have a NULL bitmap, this function won't get called for that
 * case.
 */
static inline int
first_null_attr(const uint8 *bits, int natts)
{
	int			nattByte = natts >> 3;
	int			bytenum;
	int			res;

#ifdef USE_ASSERT_CHECKING
	int			firstnull_check = natts;

	/* Do it the slow way and check we get the same answer. */
	for (int i = 0; i < natts; i++)
	{
		if (att_isnull(i, bits))
		{
			firstnull_check = i;
			break;
		}
	}
#endif

	/* Process all bytes up to just before the byte for the natts attribute */
	for (bytenum = 0; bytenum < nattByte; bytenum++)
	{
		/* break if there's any NULL attrs (a 0 bit) */
		if (bits[bytenum] != 0xFF)
			break;
	}

	/*
	 * Look for the highest 0-bit in the 'bytenum' element.  To do this, we
	 * promote the uint8 to uint32 before performing the bitwise NOT and
	 * looking for the first 1-bit.  This works even when the byte is 0xFF, as
	 * the bitwise NOT of 0xFF in 32 bits is 0xFFFFFF00, in which case
	 * pg_rightmost_one_pos32() will return 8.  We may end up with a value
	 * higher than natts here, but we'll fix that with the Min() below.
	 */
	res = bytenum << 3;
	res += pg_rightmost_one_pos32(~((uint32) bits[bytenum]));

	/*
	 * Since we did no masking to mask out bits beyond the natts'th bit, we
	 * may have found a bit higher than natts, so we must cap res to natts
	 */
	res = Min(res, natts);

	/* Ensure we got the same answer as the att_isnull() loop got */
	Assert(res == firstnull_check);

	return res;
}

The bitmap is scanned one byte at a time, looking for the first byte that is not 0xFF. Eight attributes are eliminated per comparison instead of one. Then the answer is extracted from that single byte with pg_rightmost_one_pos32().

Note the USE_ASSERT_CHECKING block: in assert builds, the function also does the slow bit-by-bit scan and asserts the two agree. That is the kind of self-checking that makes a bit-twiddling optimisation safe to merge.

The SWAR trick: populate_isnull_array

The other place the NULL bitmap is processed is where the executor needs the NULL flags as an array of bools rather than as bits. That is populate_isnull_array() in src/include/access/tupmacs.h:

c 
/*
 * populate_isnull_array
 *		Transform a tuple's null bitmap into a boolean array.
 *
 * Caller must ensure that the isnull array is sized so it contains
 * at least as many elements as there are bits in the 'bits' array.
 * Callers should be aware that isnull is populated 8 elements at a time,
 * effectively as if natts is rounded up to the next multiple of 8.
 */
static inline void
populate_isnull_array(const uint8 *bits, int natts, bool *isnull)
{
	int			nbytes = (natts + 7) >> 3;

	/*
	 * Multiplying the inverted NULL bitmap byte by this value results in the
	 * lowest bit in each byte being set the same as each bit of the inverted
	 * byte.  We perform this as 2 32-bit operations rather than a single
	 * 64-bit operation as multiplying by the required value to do this in
	 * 64-bits would result in overflowing a uint64 in some cases.
	 *
	 * XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done
	 * more efficiently on most X86-64 CPUs with the PDEP instruction.  Beware
	 * that some chips (e.g. AMD's Zen2) are horribly inefficient at PDEP.
	 */
#define SPREAD_BITS_MULTIPLIER_32 0x204081U

	for (int i = 0; i < nbytes; i++, isnull += 8)
	{
		uint64		isnull_8;
		uint8		nullbyte = ~bits[i];

		/* Convert the lower 4 bits of NULL bitmap word into a 64 bit int */
		isnull_8 = (nullbyte & 0xf) * SPREAD_BITS_MULTIPLIER_32;

		/*
		 * Convert the upper 4 bits of NULL bitmap word into a 64 bit int,
		 * shift into the upper 32 bit and bitwise-OR with the result of the
		 * lower 4 bits.
		 */
		isnull_8 |= ((uint64) ((nullbyte >> 4) * SPREAD_BITS_MULTIPLIER_32)) << 32;

		/* Mask out all other bits apart from the lowest bit of each byte. */
		isnull_8 &= UINT64CONST(0x0101010101010101);

#ifdef WORDS_BIGENDIAN

		/*
		 * Fix byte order on big-endian machines before copying to the array.
		 */
		isnull_8 = pg_bswap64(isnull_8);
#endif
		memcpy(isnull, &isnull_8, sizeof(uint64));
	}
}

This is a SWAR — SIMD Within A Register — bit-spreading trick. The goal: turn one byte of bitmap (8 bits) into 8 bytes of bool (each 0 or 1), i.e. a 64-bit word where each byte is 0x00 or 0x01.

The magic constant is 0x204081. Multiply a 4-bit value n (0..15) by 0x204081 and the result has n’s bits spread out, one into each of four byte positions, in the low bit of each byte. The mask 0x0101010101010101 then keeps only those low bits.

Because the bitmap is inverted (1 means not-null), the code inverts the byte first:

c 
		uint8		nullbyte = ~bits[i];

So a NULL attribute becomes a 1 bit in nullbyte, and the multiplication turns that into a true in the output array.

The whole operation is: one invert, two 32-bit multiplies, one shift, one OR, one AND, one 64-bit store. Eight attributes at a time, with no branches.

The comment about PDEP is a nice piece of engineering history:

XXX if we ever require BMI2 (-march=x86-64-v3), then this could be done more efficiently on most X86-64 CPUs with the PDEP instruction. Beware that some chips (e.g. AMD’s Zen2) are horribly inefficient at PDEP.

BMI2’s PDEP (“parallel deposit”) does exactly this bit-spreading in one instruction. But on AMD Zen 2 it is microcoded and slow — slower than the multiply trick — so the portable code stays.

Note also the big-endian fixup: pg_bswap64() reverses the byte order so that the array comes out in attribute order on a big-endian machine.

And note the contract: the caller must ensure the isnull array is sized for the bitmap rounded up to a multiple of 8. The executor’s slot allocation says so explicitly:

c 
	slot->tts_values = (Datum *)
		MemoryContextAlloc(slot->tts_mcxt, tupdesc->natts * sizeof(Datum));

	/*
	 * We round the size of tts_isnull up to the next highest multiple of 8.
	 * This is needed as populate_isnull_array() operates on 8 elements at a
	 * time when converting a tuple's NULL bitmap into a boolean array.
	 */
	slot->tts_isnull = (bool *)
		MemoryContextAlloc(slot->tts_mcxt, TYPEALIGN(8, tupdesc->natts * sizeof(bool)));

The formalisation of that requirement was part of the “Optimize tuple deformation” commit:

This results in much more optimal code when compared to determining the NULLness via att_isnull(). 8 isnull elements are processed at once using this method, which means we need to round the tts_isnull array size up to the next 8 bytes. The palloc code does this anyway, but the round-up needed to be formalized so as not to overwrite the sentinel byte in MEMORY_CONTEXT_CHECKING builds.

The executor: slot_deform_heap_tuple

The hot path in the executor is not fastgetattr() but slot_deform_heap_tuple() in src/backend/executor/execTuples.c, which deforms all attributes up to reqnatts in one go, rather than one at a time. It is an incremental version of heap_deform_tuple():

c 
/*
 * slot_deform_heap_tuple
 *		Given a TupleTableSlot, extract data from the slot's physical tuple
 *		into its Datum/isnull arrays.  Data is extracted up through the
 *		reqnatts'th column.  If there are insufficient attributes in the given
 *		tuple, then slot_getmissingattrs() is called to populate the
 *		remainder.  If reqnatts is above the number of attributes in the
 *		slot's TupleDesc, an error is raised.
 *
 *		This is essentially an incremental version of heap_deform_tuple:
 *		on each call we extract attributes up to the one needed, without
 *		re-computing information about previously extracted attributes.
 *		slot->tts_nvalid is the number of attributes already extracted.
 *
 * This is marked as always inline, so the different offp for different types
 * of slots gets optimized away.
 *
 * support_cstring should be passed as a const to allow the compiler only
 * emit code during inlining for cstring deforming when it is required.
 * cstrings can exist in MinimalTuples, but not in HeapTuples.
 */
static pg_always_inline void
slot_deform_heap_tuple(TupleTableSlot *slot, HeapTuple tuple, uint32 *offp,
					   int reqnatts, bool support_cstring)

The structure of the function is a sequence of loops, each handling a different regime of attributes:

RegimeLoopCost per attribute
Guaranteed (NOT NULL, byval, fixed width)1stone load at a cached offset
Cached offset, before first NULL2ndone load at a cached offset
No cached offset, before first NULL3rdalign, fetch, advance
Anything after the first NULL4thcheck isnull[], then align, fetch, advance

Here is the first loop — the guaranteed-attribute fast path:

c 
	firstNonGuaranteedAttr = Min(reqnatts, slot->tts_first_nonguaranteed);

	firstNonCacheOffsetAttr = tupleDesc->firstNonCachedOffsetAttr;

	if (HeapTupleHasNulls(tuple))
	{
		natts = HeapTupleHeaderGetNatts(tup);
		tp = (char *) tup + MAXALIGN(offsetof(HeapTupleHeaderData, t_bits) +
									 BITMAPLEN(natts));

		natts = Min(natts, reqnatts);
		if (natts > firstNonGuaranteedAttr)
		{
			uint8	   *bp = tup->t_bits;

			/* Find the first NULL attr */
			firstNullAttr = first_null_attr(bp, natts);

			/*
			 * And populate the isnull array for all attributes being fetched
			 * from the tuple.
			 */
			populate_isnull_array(bp, natts, isnull);
		}
		else
		{
			/* Otherwise all required columns are guaranteed to exist */
			firstNullAttr = natts;

			/*
			 * Check TupleDescFinalize() didn't get confused when setting
			 * firstNonGuaranteedAttr.  There should never be a NULL in a
			 * guaranteed column.
			 */
			Assert(first_null_attr(tup->t_bits, natts) >= firstNullAttr);
		}
	}

The natts > firstNonGuaranteedAttr test is the interesting one. If every column the query asked for is guaranteed to exist, the code does not even need to read the tuple’s attribute count — which means it does not need to touch the tuple header at all, and it does not need to populate the isnull array. The Assert verifies that a guaranteed column really is not NULL.

The guaranteed-attribute loop:

c 
	if (attnum < firstNonGuaranteedAttr)
	{
		int			attlen;

		do
		{
			isnull[attnum] = false;
			cattr = &cattrs[attnum];
			attlen = cattr->attlen;

			/* We don't expect any non-byval types */
			pg_assume(attlen > 0);
			Assert(cattr->attbyval == true);

			off = cattr->attcacheoff;
			values[attnum] = fetch_att_noerr(tp + off, true, attlen);
			attnum++;
		} while (attnum < firstNonGuaranteedAttr);

		off += attlen;

		if (attnum == reqnatts)
			goto done;
	}

No NULL checks (they are guaranteed non-NULL), no alignment arithmetic (the offset is cached), no attbyval branch (guaranteed by-value). Just a load and a store.

Then the cached-offset loop, which is the same shape but does have to handle by-reference attributes:

c 
	/* We can use attcacheoff up until the first NULL */
	firstNonCacheOffsetAttr = Min(firstNonCacheOffsetAttr, firstNullAttr);

	/*
	 * Handle the portion of the tuple that we have cached the offset for up
	 * to the first NULL attribute.  The offset is effectively fixed for
	 * these, so we can use the CompactAttribute's attcacheoff.
	 */
	if (attnum < firstNonCacheOffsetAttr)
	{
		int			attlen;

		do
		{
			isnull[attnum] = false;
			cattr = &cattrs[attnum];
			attlen = cattr->attlen;
			off = cattr->attcacheoff;
			values[attnum] = fetch_att_noerr(tp + off,
											 cattr->attbyval,
											 attlen);
			attnum++;
		} while (attnum < firstNonCacheOffsetAttr);

		/*
		 * Point the offset after the end of the last attribute with a cached
		 * offset.  We expect the final cached offset attribute to have a
		 * fixed width, so just add the attlen to the attcacheoff
		 */
		Assert(attlen > 0);
		off += attlen;
	}

And then the two variable-width loops:

c 
	/*
	 * Handle any portion of the tuple that doesn't have a fixed offset up
	 * until the first NULL attribute.  This loop only differs from the one
	 * after it by the NULL checks.
	 */
	for (; attnum < firstNullAttr; attnum++)
	{
		int			attlen;

		isnull[attnum] = false;
		cattr = &cattrs[attnum];
		attlen = cattr->attlen;

		/*
		 * Only emit the cstring-related code in align_fetch_then_add() when
		 * cstring support is needed.  We assume support_cstring will be
		 * passed as a const to allow the compiler to eliminate this branch.
		 */
		if (!support_cstring)
			pg_assume(attlen > 0 || attlen == -1);

		/* align 'off', fetch the datum, and increment off beyond the datum */
		values[attnum] = align_fetch_then_add(tp,
											  &off,
											  cattr->attbyval,
											  attlen,
											  cattr->attalignby);
	}

	/*
	 * Now handle any remaining attributes in the tuple up to the requested
	 * attnum.  This time, include NULL checks as we're now at the first NULL
	 * attribute.
	 */
	for (; attnum < natts; attnum++)
	{
		int			attlen;

		if (isnull[attnum])
		{
			values[attnum] = (Datum) 0;
			continue;
		}

		cattr = &cattrs[attnum];
		attlen = cattr->attlen;

		/* As above, only emit cstring code when needed. */
		if (!support_cstring)
			pg_assume(attlen > 0 || attlen == -1);

		/* align 'off', fetch the datum, and increment off beyond the datum */
		values[attnum] = align_fetch_then_add(tp,
											  &off,
											  cattr->attbyval,
											  attlen,
											  cattr->attalignby);
	}

The isnull[] array is consulted in the last loop — but it is a bool array that was populated eight bytes at a time by the SWAR trick, so the check is a byte load rather than a bit extraction.

align_fetch_then_add() is the workhorse that does align, fetch, and advance in one go:

c 
static inline Datum
align_fetch_then_add(const char *tupptr, uint32 *off, bool attbyval, int attlen,
					 uint8 attalignby)
{
	Datum		res;

	if (attlen > 0)
	{
		const char *offset_ptr;

		*off = TYPEALIGN(attalignby, *off);
		offset_ptr = tupptr + *off;
		*off += attlen;
		if (attbyval)
		{
			switch (attlen)
			{
				case sizeof(char):
					return CharGetDatum(*((const char *) offset_ptr));
				case sizeof(int16):
					return Int16GetDatum(*((const int16 *) offset_ptr));
				case sizeof(int32):
					return Int32GetDatum(*((const int32 *) offset_ptr));
				default:

					/*
					 * populate_compact_attribute_internal() should have
					 * checked
					 */
					Assert(attlen == sizeof(int64));
					return Int64GetDatum(*((const int64 *) offset_ptr));
			}
		}
		return PointerGetDatum(offset_ptr);
	}
	else if (attlen == -1)
	{
		if (!VARATT_IS_SHORT(tupptr + *off))
			*off = TYPEALIGN(attalignby, *off);

		res = PointerGetDatum(tupptr + *off);
		*off += VARSIZE_ANY(DatumGetPointer(res));
		return res;
	}
	else
	{
		Assert(attlen == -2);
		*off = TYPEALIGN(attalignby, *off);
		res = PointerGetDatum(tupptr + *off);
		*off += strlen(tupptr + *off) + 1;
		return res;
	}
}

Note the varlena case: VARATT_IS_SHORT() is checked before aligning, because a short varlena (1-byte header) needs no alignment at all. That is the same rule that governs tuple formation, so the deform path mirrors the form path exactly.

The bulk version

heap_deform_tuple() in src/backend/access/common/heaptuple.c is the non-incremental version, used when all attributes are wanted at once. Its header comment explains when it beats a loop over heap_getattr():

c 
 *		Note that for pass-by-reference datatypes, the pointer placed
 *		in the Datum will point into the given tuple.
 *
 *		When all or most of a tuple's fields need to be extracted,
 *		this routine will be significantly quicker than a loop around
 *		heap_getattr; the loop will become O(N^2) as soon as any
 *		noncacheable attribute offsets are involved.

The O(N²) is the point: fetching each attribute separately re-walks the prefix each time.

The opt-in: TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS

The guaranteed-attribute optimisation is not always safe. Some code deforms tuples before the NOT NULL constraints have been checked — for instance, when the executor is building a tuple that will be inserted, or when it is reading a row that might violate the constraint. In those cases “guaranteed to exist” is a lie.

So it is opt-in, via a slot flag:

c 
/*
 * true = slot's formed tuple guaranteed to not have NULLs in NOT NULLable
 * columns.
 */
#define			TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS		(1 << 3)
#define TTS_OBEYS_NOT_NULL_CONSTRAINTS(slot) \
	(((slot)->tts_flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)

and the slot caches the value:

c 
	int			tts_first_nonguaranteed;	/* The value from the TupleDesc's
											 * firstNonGuaranteedAttr, or 0
											 * when tts_flags does not contain
											 * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS */

The flag is only set by the scan nodes, which read tuples that are known to satisfy the relation’s constraints:

  • src/backend/executor/nodeSeqscan.c
  • src/backend/executor/nodeIndexscan.c
  • src/backend/executor/nodeBitmapHeapscan.c
  • src/backend/executor/nodeTidscan.c
  • src/backend/executor/nodeTidrangescan.c
  • src/backend/executor/nodeSamplescan.c

and the slot initialisation decides whether to use it:

c 
		/*
		 * Precalculate the maximum guaranteed attribute that has to exist in
		 * every tuple which gets deformed into this slot.  When the
		 * TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS flag is enabled, we simply take
		 * the pre-calculated value from the tupleDesc, otherwise the
		 * optimization is disabled, and we set the value to 0.
		 */
		if ((flags & TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS) != 0)
			slot->tts_first_nonguaranteed = tupleDesc->firstNonGuaranteedAttr;
		else
			slot->tts_first_nonguaranteed = 0;

The commit message explains the whole design:

Some locations in the code deform tuples that have yet to go through NOT NULL constraint validation. We’re unable to perform the guaranteed attribute optimization when that’s the case. This optimization is opt-in via the TupleTableSlot using the TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS flag.

The JIT path: sibling call optimisation

There is one more piece of the story, and it is about the JIT. The JIT compiler cannot inline arbitrary C, so it calls back into the executor through a small set of exported functions. One of them is slot_getsomeattrs_int():

c 
/*
 * slot_getsomeattrs_int
 *		external function to call getsomeattrs() for use in JIT
 */
void
slot_getsomeattrs_int(TupleTableSlot *slot, int attnum)
{
	/* Check for caller errors */
	Assert(slot->tts_nvalid < attnum);	/* checked in slot_getsomeattrs */
	Assert(attnum > 0);

	/* Fetch as many attributes as possible from the underlying tuple. */
	slot->tts_ops->getsomeattrs(slot, attnum);

	/*
	 * Avoid putting new code here as that would prevent the compiler from
	 * using the sibling call optimization for the above function.
	 */
}

The commit that produced this is 4deecb52aff, “Allow sibling call optimization in slot_getsomeattrs_int()”, by David Rowley:

This changes the TupleTableSlotOps contract to make it so the getsomeattrs() function is in charge of calling slot_getmissingattrs().

Since this removes all code from slot_getsomeattrs_int() aside from the getsomeattrs() call itself, we may as well adjust slot_getsomeattrs() so that it calls getsomeattrs() directly. We leave slot_getsomeattrs_int() intact as this is still called from the JIT code.

A sibling call is a tail call: the callee reuses the caller’s stack frame instead of pushing a new one. It matters here because the function being called is slot->tts_ops->getsomeattrs, which is a large, branch-heavy function. Without a tail call, the JIT’s call site would have to allocate a stack frame proportional to that function’s needs, and the frame could not be reused for the next call.

The whole point is visible in the comment: “Avoid putting new code here as that would prevent the compiler from using the sibling call optimization for the above function.” One statement, one tail call, no frame.

The contract change moved the slot_getmissingattrs() call into the getsomeattrs() implementation, so that the exported wrapper contains nothing but the call.

Caveats

TupleDescFinalize() must be called

This is the big one. The cache is not computed lazily; it is computed by an explicit call, and every code path that mutates a TupleDesc must call it again.

The header is unambiguous:

Once a TupleDesc has been populated, before it is used for any purpose, TupleDescFinalize() must be called on it.

and the code asserts it in several places:

c 
	/* Did someone forget to call TupleDescFinalize()? */
	Assert(tupleDesc->firstNonCachedOffsetAttr >= 0);

The trick is that firstNonCachedOffsetAttr is initialised to -1 in CreateTemplateTupleDesc():

c 
	/* This will be set to the correct value by TupleDescFinalize() */
	desc->firstNonCachedOffsetAttr = -1;
	desc->firstNonGuaranteedAttr = -1;

so “never finalized” is detectable by a negative value. BlessTupleDesc() checks it too:

c 
/*
 * BlessTupleDesc - make a completed tuple descriptor useful for SRFs
 *
 * Rowtype Datums returned by a function must contain valid type information.
 * This happens "for free" if the tupdesc came from a relcache entry, but
 * not if we have manufactured a tupdesc for a transient RECORD datatype.
 * In that case we have to notify typcache.c of the existence of the type.
 *
 * TupleDescFinalize() must be called on the TupleDesc before calling this
 * function.
 */
TupleDesc
BlessTupleDesc(TupleDesc tupdesc)
{
	/* Did someone forget to call TupleDescFinalize()? */
	Assert(tupdesc->firstNonCachedOffsetAttr >= 0);
	...
}

This was made explicit in a separate commit, 503620311e3, “Add all required calls to TupleDescFinalize()”, whose message says:

As of this commit all TupleDescs must have TupleDescFinalize() called on them once the TupleDesc is set up and before BlessTupleDesc() is called. … The follow-up commit which properly implements TupleDescFinalize() will cause any code which forgets to do this to fail in assert-enabled builds in BlessTupleDesc(). It may still be worth mentioning this change in the release notes so that extension authors update their code.

Extensions that build their own TupleDesc must call TupleDescFinalize(). This is an API change.

The cache is invalidated by any change to the descriptor

If anything modifies the FormData_pg_attribute array — adding a column, dropping one, changing a type — the cached offsets are stale and TupleDescFinalize() must be called again. The rule for the compact array is stated in the header:

Any code making changes manually to and fields in the FormData_pg_attribute array must subsequently call populate_compact_attribute() to flush the changes out to the corresponding ‘compact_attrs’ element.

and populate_compact_attribute() resets attcacheoff to -1, so a stale cache is not silently wrong — it becomes “unknown”, and the deformer falls back to walking. But firstNonCachedOffsetAttr and firstNonGuaranteedAttr would also need recomputing, which is TupleDescFinalize()’s job.

For catalogs, the mechanism is the relcache: ALTER TABLE invalidates the relcache entry, the relation’s TupleDesc is rebuilt from pg_attribute, and TupleDescFinalize() is called on it:

c 
	TupleDescFinalize(relation->rd_att);

That appears in src/backend/utils/cache/relcache.c in several places, and in src/backend/commands/tablecmds.c for the descriptor being built during ALTER TABLE.

Virtual generated columns break the offset cache

TupleDescFinalize() stops at a virtual generated column:

c 
		if (cattr->attlen <= 0 ||
			attr->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL)
			break;

because virtual generated columns are not stored in the tuple at all — they are computed on read. The offset cache would be wrong past that point.

There is a whole commit about this: 89eafad297a, “Fix tuple deforming with virtual generated columns”.

The guaranteed-attribute path is opt-in

Already covered. If the slot does not have TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS, tts_first_nonguaranteed is 0 and the whole first loop is skipped. The optimisation is only available on scan nodes that read committed tuples.

The isnull array must be rounded up

populate_isnull_array() writes 8 bytes at a time and the caller must have allocated the array accordingly. Getting this wrong would be a buffer overrun, so the requirement is spelled out in the function’s comment and formalised in the commit message. In MEMORY_CONTEXT_CHECKING builds, the palloc sentinel byte would catch an overflow.

attcacheoff is only a hint about a specific tuple layout

The cached offsets assume the tuple was formed by heap_form_tuple() with the same descriptor. That is true for tuples read from a relation. It is not true for tuples formed some other way — for example, a tuple formed for an index (IndexTuple), which has a different layout, or a tuple deformed from a different relation.

This is why index tuples have their own accessors (src/include/access/itup.h) and their own nocache_index_getattr(), even though the code is nearly identical:

c 
 *		This only gets called from fastgetattr(), in cases where we
 *		can't use the attcacheoff and the value is not null.
 *
 *		NOTE: if you need to change this code, see also heap_deform_tuple.
 *		Also see nocache_index_getattr, which is the same code for index
 *		tuples.

Index tuples store the same attributes but in an index-specific order and with index-specific types, so their descriptors have their own offsets.

The SWAR trick writes past the end of the attribute range

populate_isnull_array() processes whole bytes, so it may write isnull entries for attributes beyond natts. That is why the array must be padded to a multiple of 8 — and why heap_deform_tuple() does not use it:

c 
		/*
		 * XXX: it'd be nice to use populate_isnull_array() here, but that
		 * requires that the isnull array's size is rounded up to the next
		 * multiple of 8.  Doing that would require adjusting many locations
		 * that allocate the array.
		 */

The bulk heap_deform_tuple() still uses per-attribute att_isnull() checks, because its isnull array is sized exactly natts and there are “many locations that allocate the array” to fix.

What it all means in practice

The width of your table’s fixed-width prefix is the thing that matters. attcacheoff is defined for every column up to the first text, bytea, numeric, or other varlena column. A table with 40 columns where the first is text has no cached offsets at all; a table with 40 columns where the first is int has 39 of them.

A table of fixed-width columns is dramatically cheaper to read than it used to be. The cost of reading column N is O(1) instead of O(N).

NULL-heavy tables got a different win. The bitmap is now scanned a byte at a time to find the first NULL, and the isnull array is built with a SWAR multiply rather than 40 bit tests. The prefix before the first NULL is deformed with no NULL checks at all.

NOT NULL constraints are now a performance feature. A column declared NOT NULL (and validated) that is pass-by-value and fixed-width is guaranteed to exist, and the deformer can skip the NULL bitmap, the attribute count, and the by-value/by-reference branch entirely.

Wide tables with a text column early on are still slow. The cache stops at the first varlena, and everything past it is walked. If you have a table with a nullable text column at position 2 and 40 columns after it, the offsets for columns 3..40 are not cached.

Further reading

  • src/include/access/tupdesc.h — CompactAttribute, TupleDescData, firstNonCachedOffsetAttr, firstNonGuaranteedAttr.
  • src/backend/access/common/tupdesc.c — TupleDescFinalize(), populate_compact_attribute().
  • src/include/access/htup_details.h — fastgetattr(), HeapTupleNoNulls().
  • src/backend/access/common/heaptuple.c — nocachegetattr(), heap_deform_tuple().
  • src/include/access/tupmacs.h — populate_isnull_array(), first_null_attr(), align_fetch_then_add(), fetch_att().
  • src/backend/executor/execTuples.c — slot_deform_heap_tuple(), slot_getsomeattrs_int(), BlessTupleDesc().
  • src/include/executor/tuptable.h — TTS_FLAG_OBEYS_NOT_NULL_CONSTRAINTS.
  • The upstream commits: 5983a4cffc3 (“Introduce CompactAttribute array in TupleDesc, take 2”), c456e391138 (“Optimize tuple deformation”), 503620311e3 (“Add all required calls to TupleDescFinalize()”), 4deecb52aff (“Allow sibling call optimization in slot_getsomeattrs_int()”).

PostgresPerformanceStorageInternals