PostgreSQL's jsonpath Implementation in C

When you write '$.a[*]'::jsonpath, the database does not store '$.a[*]'. It stores a flattened, linked list of compiled path nodes — and if you select the value back out, you get a normalized form, $."a"[*], with the key quoted and the operators re-spaced. The input string is gone.

That is the first clue that jsonpath is a real type with a real compiled representation, not a thin wrapper around text. The second clue is the architecture: a flex scanner, a bison grammar, a serialization pass that turns the parse tree into 4-byte-aligned nodes, and a recursive interpreter that walks those nodes against a jsonb document. This is how it all fits together, based on reading the PostgreSQL source (commit e75fcc9867f).

The catalog row

From src/include/catalog/pg_type.dat:

c
{ oid => '4072', array_type_oid => '4073', descr => 'JSON path',
  typname => 'jsonpath', typlen => '-1', typbyval => 'f', typcategory => 'U',
  typinput => 'jsonpath_in', typoutput => 'jsonpath_out',
  typreceive => 'jsonpath_recv', typsend => 'jsonpath_send', typalign => 'i',
  typstorage => 'x' },

Same shape as json/jsonb: a varlena (typlen = -1, typbyval = f), 4-byte aligned, extended (TOASTable). The difference is entirely in what the varlena contains.

The on-disk value

The datum is JsonPath, declared in src/include/utils/jsonpath.h:

c
typedef struct
{
	int32		vl_len_;		/* varlena header (do not touch directly!) */
	uint32		header;			/* version and flags (see below) */
	char		data[FLEXIBLE_ARRAY_MEMBER];
} JsonPath;

#define JSONPATH_VERSION	(0x01)
#define JSONPATH_LAX		(0x80000000)
#define JSONPATH_HDRSZ		(offsetof(JsonPath, data))

Eight bytes of fixed header (vl_len_ + header), then the compiled payload. The header’s low byte is the version (JSONPATH_VERSION, currently 1); the top bit is JSONPATH_LAX. That single bit decides the semantics of the whole expression — more on that later.

The payload is a sequence of variable-length nodes. Each node begins with a one-byte type and a four-byte nextPos, then whatever payload the type needs. Nodes are linked by relative offsets from the start of the current node, not by absolute pointers, which is what makes the whole thing position-independent and cheap to walk.

In-memory: `JsonPathItem`

The C code does not read the raw bytes directly. It reads one node into a JsonPathItem, which is a typed view over the buffer:

c
typedef struct JsonPathItem
{
	JsonPathItemType type;

	/* position form base to next node */
	int32		nextPos;

	/* pointer into JsonPath value to current node, all positions of current
	 * are relative to this base */
	char	   *base;

	union
	{
		/* classic operator with two operands: and, or etc */
		struct
		{
			int32		left;
			int32		right;
		}			args;

		/* any unary operation */
		int32		arg;

		/* storage for jpiIndexArray: indexes of array */
		struct
		{
			int32		nelems;
			struct
			{
				int32		from;
				int32		to;
			}		   *elems;
		}			array;

		/* jpiAny: levels */
		struct
		{
			uint32		first;
			uint32		last;
		}			anybounds;

		struct
		{
			char	   *data;	/* for bool, numeric and string/key */
			int32		datalen;	/* filled only for string/key */
		}			value;

		struct
		{
			int32		expr;
			char	   *pattern;
			int32		patternlen;
			uint32		flags;
		}			like_regex;
	}			content;
} JsonPathItem;

The type is the big enum JsonPathItemType in jsonpath.h. The header comment warns that these values become part of the on-disk representation, so the order must not change and new values must be added at the end. The enum starts with the JSON scalar literals (jpiNull, jpiString, jpiNumeric, jpiBool) and then grows through operators, accessors, filters, and the many item methods (.type(), .size(), .abs(), .datetime(), …).

There is a design decision hiding in that union. The header comment says:

Unlike many other representation of expression the first/main node is not an operation but left operand of expression. That allows to implement cheap follow-path descending in jsonb structure and then execute operator with right operand.

In other words, a path is a chain of accessors, and only the accessor’s own arguments are subtrees. Walking $.a[*].b is just following nextPos links from node to node. Subtrees only appear where an operator genuinely needs two operands, and those are reached through left/right/arg offsets.

Parsing: flex + bison + a flatten pass

jsonpath_in (src/backend/utils/adt/jsonpath.c) is a thin wrapper:

c
Datum
jsonpath_in(PG_FUNCTION_ARGS)
{
	char	   *in = PG_GETARG_CSTRING(0);
	int			len = strlen(in);
	return jsonPathFromCstring(in, len, fcinfo->context);
}

jsonPathFromCstring runs three stages:

c
static Datum
jsonPathFromCstring(char *in, int len, struct Node *escontext)
{
	JsonPathParseResult *jsonpath = parsejsonpath(in, len, escontext);
	...
	appendStringInfoSpaces(&buf, JSONPATH_HDRSZ);
	if (!flattenJsonPathParseItem(&buf, NULL, escontext,
								  jsonpath->expr, 0, false))
		return (Datum) 0;
	res = (JsonPath *) buf.data;
	SET_VARSIZE(res, buf.len);
	res->header = JSONPATH_VERSION;
	if (jsonpath->lax)
		res->header |= JSONPATH_LAX;
	PG_RETURN_JSONPATH_P(res);
}

Scan. src/backend/utils/adt/jsonpath_scan.l is a flex scanner that turns the text into tokens (TO_P, STRICT_P, LAX_P, ANY_P, …).
Parse. src/backend/utils/adt/jsonpath_gram.y is a bison grammar. Its top rule is mode expr_or_predicate, and mode defaults to lax:
c
```
mode:
	STRICT_P						{ $$ = false; }
	| LAX_P							{ $$ = true; }
	| /* EMPTY */					{ $$ = true; }
	;
```
The grammar builds a JsonPathParseItem tree (the in-memory AST declared in jsonpath.h). Grammar actions call helpers like makeItemBinary(jpiAdd, $1, $3) for expr '+' expr and makeItemUnary(jpiFilter, $3) for '?' '(' predicate ')'. Bison’s allocator is overridden to palloc/pfree, so a parse error does not leak.

Flatten. flattenJsonPathParseItem serializes the tree into the varlena. It writes the type byte, aligns to an int32 boundary, reserves space for the nextPos link, writes the payload, and then back-patches the relative offsets:

c
	appendStringInfoChar(buf, (char) (item->type));
	alignStringInfoInt(buf);
	next = reserveSpaceForItemPointer(buf);
	...
	if (item->next)
	{
		if (!flattenJsonPathParseItem(buf, &chld, escontext,
									  item->next, nestingLevel,
									  insideArraySubscript))
			return false;
		chld -= pos;
		*(int32 *) (buf->data + next) = chld;
	}

That chld -= pos is the relative-offset encoding: the link is stored as (child position) - (current position), and jspInitByBuffer re-adds the base to recover the absolute location.

Reading a node back

jspInitByBuffer is the mirror of the flatten pass. It reads the type byte, aligns to 4 bytes, reads nextPos, then decodes the payload by type:

c
void
jspInitByBuffer(JsonPathItem *v, char *base, int32 pos)
{
	v->base = base + pos;

	read_byte(v->type, base, pos);
	pos = INTALIGN((uintptr_t) (base + pos)) - (uintptr_t) base;
	read_int32(v->nextPos, base, pos);

	switch (v->type)
	{
		...
		case jpiString:
		case jpiKey:
		case jpiVariable:
			read_int32(v->content.value.datalen, base, pos);
			pg_fallthrough;
		case jpiNumeric:
		case jpiBool:
			v->content.value.data = base + pos;
			break;
		...
	}
}

Note the alignment dance: the type byte is read at an arbitrary offset, then pos is snapped to the next 4-byte boundary before the int32 link is read. That is why the flatten pass calls alignStringInfoInt. The in-memory helpers jspGetNext, jspGetArg, jspGetLeftArg, jspGetRightArg, jspGetString, jspGetNumeric, and jspGetBool are all one-liners that call jspInitByBuffer with the right relative offset.

Executing it

The interpreter lives in src/backend/utils/adt/jsonpath_exec.c (about 4,800 lines). The entry point is executeJsonPath:

c
static JsonPathExecResult
executeJsonPath(JsonPath *path, void *vars, JsonPathGetVarCallback getVar,
				JsonPathCountVarsCallback countVars,
				Jsonb *json, bool throwErrors, JsonValueList *result,
				bool useTz)
{
	JsonPathExecContext cxt;
	...
	jspInit(&jsp, path);
	if (!JsonbExtractScalar(&json->root, &jbv))
		JsonbInitBinary(&jbv, json);

	cxt.vars = vars;
	cxt.getVar = getVar;
	cxt.laxMode = (path->header & JSONPATH_LAX) != 0;
	cxt.ignoreStructuralErrors = cxt.laxMode;
	cxt.root = &jbv;
	cxt.current = &jbv;
	...
}

Two things worth noticing. First, the root document is a JsonbValue, so jsonpath never re-parses the JSON — it consumes the same binary format that jsonb stores. Second, the lax bit from the varlena header becomes two context flags: laxMode and ignoreStructuralErrors.

The workhorse is executeItemOptUnwrapTarget, a big switch over JsonPathItemType. It processes one node against the current JsonbValue, produces a sequence of results, and passes that sequence to the next node via executeNextItem:

c
static JsonPathExecResult
executeNextItem(JsonPathExecContext *cxt,
				JsonPathItem *cur, JsonPathItem *next,
				JsonbValue *v, JsonValueList *found)
{
	...
	if (hasNext)
		return executeItem(cxt, next, v, found);

	if (found)
		JsonValueListAppend(found, v);

	return jperOk;
}

That last if (found) is a performance trick with a name in the header comment: when found == NULL the caller is running an exists query (@@ / jsonb_path_exists), and the executor can stop at the first match instead of collecting every result.

The three-valued logic is explicit. JsonPathBool is jpbFalse, jpbTrue, jpbUnknown; JsonPathExecResult is jperOk, jperNotFound, jperError. Predicates are evaluated by executePredicate, which loops over the left and right result sequences and returns jpbUnknown if any comparison errored in lax mode. Filters are handled in the jpiFilter case:

c
		case jpiFilter:
			{
				JsonPathBool st;
				if (unwrap && JsonbType(jb) == jbvArray)
					return executeItemUnwrapTargetArray(cxt, jsp, jb, found,
														false);
				jspGetArg(jsp, &elem);
				st = executeNestedBoolItem(cxt, &elem, jb);
				if (st != jpbTrue)
					res = jperNotFound;
				else
					res = executeNextItem(cxt, jsp, NULL, jb, found);
				break;
			}

Arithmetic is delegated to Postgres’s own numeric routines (numeric_add_safe, numeric_mul_safe, …), so jsonpath arithmetic has the same exactness as numeric. Item methods like .abs() and .size() go through executeNumericItemMethod and friends.

Strict vs. lax

The lax bit is the whole reason JSONPATH_LAX exists in the header. In lax mode (the default), jspAutoUnwrap is true: an accessor applied to an array silently applies to each element, and structural errors (a missing key, a type mismatch) are suppressed. In strict mode both are turned into errors. The executor encodes the four combinations with macros:

c
#define jspStrictAbsenceOfErrors(cxt)	(!(cxt)->laxMode)
#define jspAutoUnwrap(cxt)				((cxt)->laxMode)
#define jspAutoWrap(cxt)				((cxt)->laxMode)
#define jspIgnoreStructuralErrors(cxt)	((cxt)->ignoreStructuralErrors)

The public entry points are JsonPathExists, JsonPathQuery, and JsonPathValue. JsonPathQuery is what jsonb_path_query uses, and it wraps the result list in an array according to the JsonWrapper mode (JSW_NONE, JSW_UNCONDITIONAL, JSW_CONDITIONAL) — the WITH [UNCONDITIONAL|CONDITIONAL] WRAPPER clause, implemented in C.

Output: a canonicalized path

jsonpath_out → jsonPathToCstring walks the compiled nodes with printJsonPathItem and rebuilds the text. Because it walks the compiled form, the output is normalized, not the input:

Keys are always quoted: '$.a'::jsonpath prints $."a".
Operators are re-spaced and parenthesized: '$+1'::jsonpath prints ($ + 1).
Filters are re-spaced: '$.g ? (@ == 1)'::jsonpath prints $."g"?(@ == 1).
Strict mode is remembered: only strict paths print the strict prefix, because jsonPathToCstring checks the JSONPATH_LAX bit:
c
```
	if (!(in->header & JSONPATH_LAX))
		appendStringInfoString(out, "strict ");
```

Look at it yourself

sql
-- default is lax; keys come back quoted
SELECT '$.a[*]'::jsonpath;              -- $."a"[*]

-- strict is remembered in the header bit
SELECT 'strict $'::jsonpath;            -- strict $

-- the compiled form is re-printed, not stored
SELECT '$+1'::jsonpath;                 -- ($ + 1)

-- you can execute it against a jsonb
SELECT jsonb_path_query('{"a":[1,2,3]}'::jsonb, '$.a[*] ? (@ > 1)');

That last line shows the whole pipeline: the jsonpath varlena is decoded node by node, the interpreter descends the jsonb container by JEntry, and the filter keeps the elements greater than 1.

What it all means

jsonpath is a compiled value, not text. The string you type is parsed, flattened into linked nodes, and only the compiled form is stored. Output is a canonicalization of that compiled form.
Nodes are linked by relative offsets. nextPos is (child - current), which keeps the payload position-independent and lets jspInitByBuffer decode any node from a base pointer plus an offset.
The path is a chain, not a tree. Accessors link forward through nextPos; only operators carry subtrees. That is what makes “follow the path” cheap.
Lax vs. strict is one header bit. JSONPATH_LAX flips array auto-unwrapping and error suppression for the entire expression.
Execution is a recursive descent over JsonbValue. The interpreter consumes the same binary jsonb format the storage layer writes, and delegates arithmetic to Postgres numeric.