PostgreSQL's bytea Implementation in C

bytea is the one string type in PostgreSQL that is genuinely binary. It has no encoding, no collation, no length limit in the SQL sense, and no blank-padding semantics. That makes its C implementation almost suspiciously small: src/include/c.h gives it the same one-line treatment as text.

c
typedef varlena bytea;
typedef varlena text;
typedef varlena BpChar;			/* blank-padded char, ie SQL char(n) */
typedef varlena VarChar;		/* var-length char, ie SQL varchar(n) */

So where does all the binary behavior live? In the I/O functions in src/backend/utils/adt/bytea.c and in one session GUC. This is a walk through the real bytes, based on the PostgreSQL source (version 20devel, commit e75fcc9867f).

The catalog entry

From src/include/catalog/pg_type.dat:

c
{ oid => '17', array_type_oid => '1001',
  descr => 'variable-length string, binary values escaped',
  typname => 'bytea', typlen => '-1', typbyval => 'f', typcategory => 'U',
  typinput => 'byteain', typoutput => 'byteaout', typreceive => 'bytearecv',
  typsend => 'byteasend', typalign => 'i', typstorage => 'x' },

Every flag is the same shape as text: varlena, by reference, INT-aligned, EXTENDED storage (toastable). Three things are absent, and each is meaningful:

No typcollation. A bytea value cannot use collations. It is category U (user), not S (string), so COLLATE does not apply.
No typmod. There is no bytea(n).
typstorage => 'x', so it is compressible and out-of-line storable.

The varlena payload

bytea is exactly a varlena: a 4-byte length word (VARHDRSZ) followed by raw bytes. There is no terminator and no interpretation of the payload — a 0x00 is just a byte, not an end marker. src/include/varatt.h supplies the shared formats: the 4-byte aligned header, the 1-byte short header for payloads up to 126 bytes, the compressed-in-line header, and the external TOAST pointer.

The comparison and length functions read the payload with the alignment-oblivious macros, because a short varlena may sit at any byte offset in a tuple:

c
	len1 = VARSIZE_ANY_EXHDR(arg1);
	len2 = VARSIZE_ANY_EXHDR(arg2);

	cmp = memcmp(VARDATA_ANY(arg1), VARDATA_ANY(arg2), Min(len1, len2));

Input: hex or escape

byteain accepts two textual forms and dispatches on the first two characters:

c
	/* Recognize hex input */
	if (inputText[0] == '\\' && inputText[1] == 'x')
	{
		bc = (len - 2) / 2 + VARHDRSZ;	/* maximum possible length */
		result = palloc(bc);
		bc = hex_decode_safe(inputText + 2, len - 2, VARDATA(result),
							 escontext);
		SET_VARSIZE(result, bc + VARHDRSZ); /* actual length */

		PG_RETURN_BYTEA_P(result);
	}

The \x form is decoded by hex_decode_safe, which lives in src/backend/utils/adt/encode.c. It is SIMD-accelerated: hex_decode_safe processes two Vector8 blocks at a time and falls back to the scalar hex_decode_safe_scalar for the tail and for any input it could not handle vectorially. The scalar path also tolerates embedded spaces and newlines:

c
		if (*s == ' ' || *s == '\n' || *s == '\t' || *s == '\r')
		{
			s++;
			continue;
		}

The other form is the traditional escape syntax. byteain walks the input and turns \ooo (three octal digits) into one byte, \\ into a backslash, and anything else into a literal byte:

c
		else if ((tp[1] >= '0' && tp[1] <= '3') &&
				 (tp[2] >= '0' && tp[2] <= '7') &&
				 (tp[3] >= '0' && tp[3] <= '7'))
		{
			int			v;

			v = VAL(tp[1]);
			v <<= 3;
			v += VAL(tp[2]);
			v <<= 3;
			*rp++ = v + VAL(tp[3]);

			tp += 4;
		}
		else if (tp[1] == '\\')
		{
			*rp++ = '\\';
			tp += 2;
		}

Output: the `bytea_output` GUC

byteaout is where bytea becomes surprising. The output format is not a property of the type; it is a session setting, bytea_output, defined in src/backend/utils/misc/guc_parameters.dat with a default of BYTEA_OUTPUT_HEX:

c
Datum
byteaout(PG_FUNCTION_ARGS)
{
	bytea	   *vlena = PG_GETARG_BYTEA_PP(0);
	char	   *result;
	char	   *rp;

	if (bytea_output == BYTEA_OUTPUT_HEX)
	{
		/* Print hex format */
		rp = result = palloc(VARSIZE_ANY_EXHDR(vlena) * 2 + 2 + 1);
		*rp++ = '\\';
		*rp++ = 'x';
		rp += hex_encode(VARDATA_ANY(vlena), VARSIZE_ANY_EXHDR(vlena), rp);
	}
	else if (bytea_output == BYTEA_OUTPUT_ESCAPE)
	{

The hex path calls hex_encode, the SIMD counterpart of the decoder in encode.c. The escape path is a hand-rolled state machine: a backslash is doubled, and any byte outside the printable ASCII range (< 0x20 or > 0x7e) becomes \ooo:

c
		for (i = VARSIZE_ANY_EXHDR(vlena); i != 0; i--, vp++)
		{
			if (*vp == '\\')
			{
				*rp++ = '\\';
				*rp++ = '\\';
			}
			else if ((unsigned char) *vp < 0x20 || (unsigned char) *vp > 0x7e)
			{
				int			val;	/* holds unprintable chars */

				val = *vp;
				rp[0] = '\\';
				rp[3] = DIG(val & 07);
				val >>= 3;
				rp[2] = DIG(val & 07);
				val >>= 3;
				rp[1] = DIG(val & 03);
				rp += 4;
			}
			else
				*rp++ = *vp;
		}

The output buffer is sized by a first pass over the data, and byteaout rejects a result that would exceed MaxAllocSize. Crucially, this is a printing choice: SET bytea_output = 'escape' changes the text you see, but the stored varlena is byte-for-byte identical.

Binary recv/send: copy the bytes

The wire format is as simple as it gets. bytearecv copies the message payload verbatim, and byteasend is a documented special case:

c
Datum
byteasend(PG_FUNCTION_ARGS)
{
	bytea	   *vlena = PG_GETARG_BYTEA_P_COPY(0);

	PG_RETURN_BYTEA_P(vlena);
}

The comment above it says “This is a special case: just copy the input…” — PG_GETARG_BYTEA_P_COPY makes an OK-to-write copy, and that copy is the output. There is no length prefix added beyond the varlena header; the receiving side reads the length from the header.

Comparison: memcmp plus a length tiebreak

Ordering is byte-by-byte unsigned comparison, then length. byteacmp is the clearest statement:

c
	cmp = memcmp(VARDATA_ANY(arg1), VARDATA_ANY(arg2), Min(len1, len2));
	if ((cmp == 0) && (len1 != len2))
		cmp = (len1 < len2) ? -1 : 1;

bytealt and friends use the same shape with the length tiebreak folded into the boolean. The equality operators add a cheap fast path: they compare raw datum sizes with toast_raw_datum_size first, so two values of different uncompressed lengths are known unequal without detoasting either one.

Because there is no collation, every comparison is pure memcmp. That is why bytea is the one string type where = means “the bytes are identical” with no locale in the way.

Sort support and the HyperLogLog

The most interesting part of bytea.c is not the I/O; it is the sort support used to make ORDER BY on large bytea columns faster. bytea_sortsupport installs an abbreviated-key comparator, a converter, and an abort callback:

c
	ssup->comparator = byteafastcmp;
	...
	ssup->abbrev_converter = bytea_abbrev_convert;
	ssup->abbrev_abort = bytea_abbrev_abort;

bytea_abbrev_convert packs the first sizeof(Datum) — 8 — bytes of the value into a single Datum, byte-swaps it so that unsigned integer comparison orders the byte strings the same way memcmp would, and returns it. Comparing 8-byte integers is far cheaper than memcmp over a large value.

But an 8-byte prefix is a lossy abbreviation: distinct values can collide. PostgreSQL hedges with HyperLogLog cardinality estimators. The ByteaSortSupport struct holds two of them:

c
typedef struct
{
	hyperLogLogState abbr_card; /* Abbreviated key cardinality state */
	hyperLogLogState full_card; /* Full key cardinality state */
	double		prop_card;		/* Required cardinality proportion */
} ByteaSortSupport;

The converter hashes both the abbreviated key and (up to a cache line of) the full key into the two estimators. Then bytea_abbrev_abort periodically asks whether the abbreviation is actually helping:

c
	abbrev_distinct = estimateHyperLogLog(&bss->abbr_card);
	key_distinct = estimateHyperLogLog(&bss->full_card);

If the abbreviated keys are as numerous as the full keys, the prefix is collapsing too many values together and the optimization is abandoned. It is a runtime cost/benefit decision driven by measured cardinality — a nice piece of engineering hiding in a “binary blob” type.

Look at it yourself

The GUC makes the round-trip visible. With standard_conforming_strings on, '\x...' is a literal backslash-x, which byteain recognizes:

sql
-- hex is the default
SELECT '\x414243'::bytea;   -- \x414243

SET bytea_output = 'escape';
SELECT '\x414243'::bytea;   -- ABC
SELECT '\x0041'::bytea;     -- \000A
SET bytea_output = 'hex';

-- length is in bytes, and NULs count
SELECT length('\x0041'::bytea);        -- 2
SELECT octet_length('\x0041'::bytea);  -- 2

-- no collation, no padding, raw bytes
SELECT '\x00'::bytea = '\x00'::bytea;  -- true

pg_type confirms there is no collation to set:

sql
SELECT typname, typlen, typbyval, typalign, typstorage, typcollation
FROM pg_type WHERE oid = 17;

What it all means

bytea is a plain varlena; all the binary behavior is in bytea.c.
Input accepts \x hex (SIMD-decoded) or the escape format; output is chosen by the session GUC bytea_output, not by the type.
Comparison is memcmp over the shorter value plus a length tiebreak, with a raw-size fast path for equality.
The wire format is a bare copy of the bytes; byteasend does no encoding.
The sort support uses 8-byte abbreviated keys guarded by HyperLogLog cardinality estimates — an adaptive optimization, not a fixed format.