PostgreSQL's text, varchar, and bpchar Implementation in C

Here is a fact that surprises people who assume SQL types map to C structs: text, varchar(n), and char(n) are all the same C type. In src/include/c.h there are three typedefs of varlena and no new struct for any of them:

typedef varlena bytea;
typedef varlena text;
typedef varlena BpChar;			/* blank-padded char, ie SQL char(n) */
typedef varlena VarChar;		/* var-length char, ie SQL varchar(n) */

The differences live entirely in the I/O functions and in a per-column typmod. This article is about how the varlena layout works and where each of the three types quietly diverges, based on the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entries

From src/include/catalog/pg_type.dat:

All three are varlena (typlen == -1), passed by reference, INT-aligned (typalign => 'i'), and fully toastable (typstorage => 'x'). text is the typispreferred type in category S. The only columns that differ are the typmodin/typmodout function names and, for bpchar, the input/output functions.

That typalign => 'i' is worth pausing on: a full 4-byte-header varlena must be 4-byte aligned in a tuple, because its length word is read as an int32. Short varlenas (see below) are the exception.

The varlena: a length word and bytes

The base struct is tiny (src/include/c.h):

typedef struct varlena
{
	char		vl_len_[4];		/* Do not touch this field directly! */
	char		vl_dat[FLEXIBLE_ARRAY_MEMBER];	/* Data content is here */
} varlena;

#define VARHDRSZ		((int32) sizeof(int32))

VARSIZE reads the header; VARDATA points at the payload; SET_VARSIZE writes the header. There is no terminator and no length parameter passed around — the length is part of the value.

src/include/varatt.h defines the on-disk header formats. There are four, and the low bits of the first byte pick one:

00xxxxxx 4-byte length word, aligned, uncompressed data (up to 1G)
01xxxxxx 4-byte length word, aligned, *compressed* data (up to 1G)
10000000 1-byte length word, unaligned, TOAST pointer
1xxxxxxx 1-byte length word, unaligned, uncompressed data (up to 126b)

The 1-byte “short” format exists so a small string costs 1 byte of header instead of 4. The limit is VARATT_SHORT_MAX, which is 0x7F = 127, so a payload of 126 bytes fits (VARSIZE includes the header byte). In heap_form_tuple, a value that can be made short is stored unaligned — no padding is inserted — while a full 4-byte-header varlena is aligned. That is a real space discontinuity at 127 bytes.

Typmod: the length is not in the type

A varchar(5) column has no length information in its varlena; the length lives in the tuple descriptor’s atttypmod. The shared converter is anychar_typmodin in src/backend/utils/adt/varchar.c:

	/*
	 * For largely historical reasons, the typmod is VARHDRSZ plus the number
	 * of characters; there is enough client-side code that knows about that
	 * that we'd better not change it.
	 */
	typmod = VARHDRSZ + *tl;

So varchar(5) is stored as typmod 9, and anychar_typmodout prints (5). The declared length is limited by MaxAttrSize, which is 10 * 1024 * 1024 (10 MB) in src/include/access/htup_details.h. The input functions receive this typmod as their third argument, which is exactly how varcharin/bpcharin know the column width.

Input: where the three types diverge

text is the base case. textin just wraps the C string:

Datum
textin(PG_FUNCTION_ARGS)
{
	char	   *inputText = PG_GETARG_CSTRING(0);

	PG_RETURN_TEXT_P(cstring_to_text(inputText));
}

and cstring_to_text_with_len is the whole implementation:

text *
cstring_to_text_with_len(const char *s, int len)
{
	text	   *result = (text *) palloc(len + VARHDRSZ);

	SET_VARSIZE(result, len + VARHDRSZ);
	memcpy(VARDATA(result), s, len);

	return result;
}

No limit, no validation, no padding. The bytes are the value.

varchar validates. varchar_input checks the byte length against the typmod and, if it is too long, clips on a multibyte boundary and then insists that every clipped byte is a space:

	maxlen = atttypmod - VARHDRSZ;

	if (atttypmod >= (int32) VARHDRSZ && len > maxlen)
	{
		size_t		mbmaxlen = pg_mbcharcliplen(s, len, maxlen);
		size_t		j;

		for (j = mbmaxlen; j < len; j++)
		{
			if (s[j] != ' ')
				ereturn(escontext, NULL,
						(errcode(ERRCODE_STRING_DATA_RIGHT_TRUNCATION),
						 errmsg("value too long for type character varying(%zu)",
								maxlen)));
		}

		len = mbmaxlen;
	}

	result = (VarChar *) cstring_to_text_with_len(s, len);

bpchar pads. bpchar_input computes the maximum byte length from the character limit, copies, and fills the remainder with spaces:

	result = (BpChar *) palloc(maxlen + VARHDRSZ);
	SET_VARSIZE(result, maxlen + VARHDRSZ);
	r = VARDATA(result);
	memcpy(r, s, len);

	/* blank pad the string if necessary */
	if (maxlen > len)
		memset(r + len, ' ', maxlen - len);

That is the entire semantic difference between varchar and char(n): one truncates to the width, the other truncates and pads to the width. The header comment in varchar.c says it plainly: “The difference between these types and ’text’ is that we truncate and possibly blank-pad the string at insertion time.”

Output and send: they all converge

Output ignores the type. varcharout and bpcharout are both TextDatumGetCString, and varcharsend/bpcharsend both call textsend:

Datum
bpcharout(PG_FUNCTION_ARGS)
{
	Datum		txt = PG_GETARG_DATUM(0);

	PG_RETURN_CSTRING(TextDatumGetCString(txt));
}

TextDatumGetCString is defined in src/include/utils/builtins.h as text_to_cstring((text *) DatumGetPointer(d)), and text_to_cstring in src/backend/utils/adt/varlena.c detoasts if needed, copies the payload, and NUL-terminates it. Because it uses VARSIZE_ANY_EXHDR, a bpchar prints with its trailing spaces intact — output does not re-apply the “true length” trimming.

Comparison: bpchar trims, text does not

text_cmp is a direct varstr_cmp over the payloads. bpchar does something extra: it strips trailing spaces before comparing. The helper is bcTruelen, which delegates to bpchartruelen:

int
bpchartruelen(char *s, int len)
{
	int			i;

	/*
	 * Note that we rely on the assumption that ' ' is a singleton unit on
	 * every supported multibyte server encoding.
	 */
	for (i = len - 1; i >= 0; i--)
	{
		if (s[i] != ' ')
			break;
	}
	return i + 1;
}

So 'hi'::char(5) compares equal to 'hi' — the padding is invisible to equality, ordering, and btree indexes, even though it is visible to output. bpcharlen uses the same trimming to report the logical length, and bpcharoctetlen deliberately does not detoast at all: it returns toast_raw_datum_size(arg) - VARHDRSZ.

texteq has its own optimization: before detoasting, it compares raw datum sizes with toast_raw_datum_size. If two text values have different uncompressed lengths, they cannot be equal, so one or both values never need to be detoasted.

On disk and TOAST

All three types are typstorage => 'x' (EXTENDED): the toaster may compress in place and may move the value out of line into the table’s TOAST table. The TOAST pointer is the one-byte-header varattrib_1b_e format from varatt.h, so an out-of-line string is addressed by (toast table, value id) rather than by the row that contains it — which is why TOAST pointers survive row moves. The layout and the four header formats are shared with every other varlena type, including bytea.

Look at it yourself

-- same bytes, three type names
SELECT 'hi'::text, 'hi'::varchar, 'hi'::char(5);

-- char(n) pads on input; the padding survives to output
SELECT 'hi'::char(5);          -- hi___  (5 chars)

-- but comparison ignores trailing spaces
SELECT 'hi'::char(5) = 'hi';   -- true
SELECT 'hi'::char(5) = 'hi  '; -- true

-- varchar and char(n) reject non-space overflow
SELECT 'toolong'::varchar(3);  -- error
SELECT 'toolong'::char(3);     -- error

-- text has no limit
SELECT length(repeat('x', 1000000))::text;  -- 1000000

-- the typmod is VARHDRSZ + declared length
SELECT atttypmod - 4 AS declared_len
FROM pg_attribute
WHERE attrelid = 't'::regclass AND attname = 'c';

What it all means

• There is one C type for all three: varlena. text, VarChar, and BpChar are typedefs, not structs.
• The SQL-level differences are enforced by the input functions at write time — padding for bpchar, length checking for varchar, neither for text.
• The declared length is not part of the datum; it is a typmod encoded as VARHDRSZ + n, and it reaches the input function as a third argument.
• bpchar trims trailing spaces only for comparison and length; the stored and printed value keeps them.
• All three are toastable varlenas with a 4-byte header, a 1-byte short form, and TOAST pointers for anything that does not fit.

Further reading

• src/backend/utils/adt/varchar.c — bpchar and varchar input, output, comparison, and typmod handling.
• src/backend/utils/adt/varlena.c — textin/textout/textsend, text_cmp, text_to_cstring, and the shared comparison machinery.
• src/include/c.h — the varlena, text, BpChar, VarChar typedefs.
• src/include/varatt.h — the varlena header formats and TOAST pointer structs.
• src/include/catalog/pg_type.h — typlen, typbyval, typalign, typstorage.