PostgreSQL Groups Commits

Jay Hubbert — Sat, 12 Sep 2026 09:00:00 +0000

Every database has a moment where a transaction becomes durable, and every database has to decide what that moment costs. PostgreSQL’s answer is unusual: committing a transaction does not mean fsyncing anything.

What happens on COMMIT is that a WAL record is copied into shared memory. Somebody — maybe you, maybe another backend, maybe the background WAL writer — eventually writes that memory to the operating system and asks the OS to flush it to stable storage. If you are lucky, other backends were committing at the same time, and their records went out in the same write() and the same fsync(). If you are unlucky, you paid for the whole fsync() yourself.

The machinery that makes the lucky case happen is small and strange: LWLockAcquireOrWait(), an LWLock acquisition function whose contract is that it may return “false” without giving you the lock. Followers wake up when the lock is released, discover that someone else already flushed their record, and return without ever taking the lock themselves.

This is a walk through how it all works, based on reading the PostgreSQL source (version 20devel, commit e75fcc9867f).

Three frontiers

Everything in the WAL machinery is measured by an LSN — a byte position in the WAL stream. And there are three of them, not one, because “the record is in memory”, “the record is in the OS’s buffer cache”, and “the record is on the platter” are three different states with three different costs.

They live in XLogCtlData (src/backend/access/transam/xlog.c):

 c
 
 
	/* These are accessed using atomics -- info_lck not needed */
	pg_atomic_uint64 logInsertResult;	/* last byte + 1 inserted to buffers */
	pg_atomic_uint64 logWriteResult;	/* last byte + 1 written out */
	pg_atomic_uint64 logFlushResult;	/* last byte + 1 flushed */

Frontier	Meaning	Who advances it
`logInsertResult`	WAL up to here has been copied into the shared WAL buffers	the inserting backends
`logWriteResult`	WAL up to here has been `write()`n to the OS	whoever holds `WALWriteLock`
`logFlushResult`	WAL up to here has been `fsync()`ed	whoever holds `WALWriteLock`

The invariant is a strict ordering, and the code asserts it explicitly:

 c
 
 
		/* WAL written to disk is always ahead of WAL flushed */
		Assert(Write >= Flush);

		/* WAL inserted to buffers is always ahead of WAL written */
		Assert(Insert >= Write);

So Insert >= Write >= Flush, always. A record can be in memory but not written; it can be written but not flushed. It can never be flushed before it is written, and it can never be written before it is in memory.

The comment block above the shared structures explains the design:

 c
 
 
/*----------
 * Shared-memory data structures for XLOG control
 *
 * LogwrtRqst indicates a byte position that we need to write and/or fsync
 * the log up to (all records before that point must be written or fsynced).
 * The positions already written/fsynced are maintained in logWriteResult
 * and logFlushResult using atomic access.
 * In addition to the shared variable, each backend has a private copy of
 * both in LogwrtResult, which is updated when convenient.
 *
 * The request bookkeeping is simpler: there is a shared XLogCtl->LogwrtRqst
 * (protected by info_lck), but we don't need to cache any copies of it.
 *
 * info_lck is only held long enough to read/update the protected variables,
 * so it's a plain spinlock. The other locks are held longer (potentially
 * over I/O operations), so we use LWLocks for them. These locks are:
 *
 * WALBufMappingLock: must be held to replace a page in the WAL buffer cache.
 * It is only held while initializing and changing the mapping. If the
 * contents of the buffer being replaced haven't been written yet, the mapping
 * lock is released while the write is done, and reacquired afterwards.
 *
 * WALWriteLock: must be held to write WAL buffers to disk (XLogWrite or
 * XLogFlush).
 *
 * ControlFileLock: must be held to read/update control file or create
 * new log file.
 *
 *----------
 */

Two different kinds of lock, for two different reasons. info_lck is a spinlock held for nanoseconds; WALWriteLock is an LWLock held across write() and fsync() calls, and therefore across disk latency.

The local copy

Reading an atomic from shared memory on every check would be wasteful, so each backend keeps a private copy:

 c
 
 
/*
 * Private, possibly out-of-date copy of shared LogwrtResult.
 * See discussion above.
 */
static XLogwrtResult LogwrtResult = {0, 0};

and refreshes it with a macro:

 c
 
 
/*
 * Update local copy of shared XLogCtl->log{Write,Flush}Result
 *
 * It's critical that Flush always trails Write, so the order of the reads is
 * important, as is the barrier. See also XLogWrite.
 */
#define RefreshXLogWriteResult(_target) \
	do { \
		_target.Flush = pg_atomic_read_u64(&XLogCtl->logFlushResult); \
		pg_read_barrier(); \
		_target.Write = pg_atomic_read_u64(&XLogCtl->logWriteResult); \
	} while (0)

The barrier in the middle is not decoration. The writer publishes Write before Flush, with a write barrier between them:

 c
 
 
	/*
	 * We write Write first, bar, then Flush. When reading, the opposite must
	 * be done (with a matching barrier in between), so that we always see a
	 * Flush value that trails behind the Write value seen.
	 */
	pg_atomic_write_u64(&XLogCtl->logWriteResult, LogwrtResult.Write);
	pg_write_barrier();
	pg_atomic_write_u64(&XLogCtl->logFlushResult, LogwrtResult.Flush);

Read Flush first, then barrier, then read Write. The barrier guarantees that if you see a Write value, you also see the Flush value that was published before it. If you skipped the barrier you could read a newer Write and an older Flush, and conclude that more WAL had been written than had been flushed — a state that does not exist, and a bug waiting to happen.

Note also that the local copy is explicitly allowed to be stale. It is a cache, and code that needs an authoritative value refreshes it first.

Inserting a record: two phases

XLogInsertRecord() is the entry point for putting a record into the WAL. Its header comment sets out the contract:

 c
 
 
/*
 * Insert an XLOG record represented by an already-constructed chain of data
 * chunks. This is a low-level routine; to construct the WAL record header
 * and data, use the higher-level routines in xloginsert.c.
 ...
 * The first XLogRecData in the chain must be for the record header, and its
 * data must be MAXALIGNed. XLogInsertRecord fills in the xl_prev and
 * xl_crc fields in the header, the rest of the header must already be filled
 * by the caller.
 *
 * Returns XLOG pointer to end of record (beginning of next record).
 * This can be used as LSN for data pages affected by the logged action.
 * (LSN is the XLOG point up to which the XLOG must be flushed to disk
 * before the data page can be written out. This implements the basic
 * WAL rule "write the log before the data".)
 */

The body explains the two-phase structure:

 c
 
 
	/*----------
	 *
	 * We have now done all the preparatory work we can without holding a
	 * lock or modifying shared state. From here on, inserting the new WAL
	 * record to the shared WAL buffer cache is a two-step process:
	 *
	 * 1. Reserve the right amount of space from the WAL. The current head of
	 *	 reserved space is kept in Insert->CurrBytePos, and is protected by
	 *	 insertpos_lck.
	 *
	 * 2. Copy the record to the reserved WAL space. This involves finding the
	 *	 correct WAL buffer containing the reserved space, and copying the
	 *	 record in place. This can be done concurrently in multiple processes.
	 *
	 * To keep track of which insertions are still in-progress, each concurrent
	 * inserter acquires an insertion lock. In addition to just indicating that
	 * an insertion is in progress, the lock tells others how far the inserter
	 * has progressed. There is a small fixed number of insertion locks,
	 * determined by NUM_XLOGINSERT_LOCKS. When an inserter crosses a page
	 * boundary, it updates the value stored in the lock to the how far it has
	 * inserted, to allow the previous buffer to be flushed.
	 *
	 * Holding onto an insertion lock also protects RedoRecPtr and
	 * fullPageWrites from changing until the insertion is finished.
	 *
	 * Step 2 can usually be done completely in parallel. If the required WAL
	 * page is not initialized yet, you have to grab WALBufMappingLock to
	 * initialize it, but the WAL writer tries to do that ahead of insertions
	 * to avoid that from happening in the critical path.
	 *
	 *----------
	 */

Phase 1 (reserve) is serialized across all backends. Phase 2 (copy) is parallel. That split is the whole design: the expensive contended part is reduced to a pointer bump, and the bulky part — copying the record bytes — happens concurrently.

Phase 1: reserving space

The reservation is the only place where all backends serialise on a spinlock. Here it is, in full:

 c
 
 
/*
 * Reserves the right amount of space for a record of given size from the WAL.
 * *StartPos is set to the beginning of the reserved section, *EndPos to
 * its end+1. *PrevPtr is set to the beginning of the previous record; it is
 * used to set the xl_prev of this record.
 *
 * This is the performance critical part of XLogInsert that must be serialized
 * across backends. The rest can happen mostly in parallel. Try to keep this
 * section as short as possible, insertpos_lck can be heavily contended on a
 * busy system.
 *
 * NB: The space calculation here must match the code in CopyXLogRecordToWAL,
 * where we actually copy the record to the reserved space.
 *
 * NB: Testing shows that XLogInsertRecord runs faster if this code is inlined;
 * however, because there are two call sites, the compiler is reluctant to
 * inline. We use pg_always_inline here to try to convince it.
 */
static pg_always_inline void
ReserveXLogInsertLocation(int size, XLogRecPtr *StartPos, XLogRecPtr *EndPos,
						 XLogRecPtr *PrevPtr)
{
	XLogCtlInsert *Insert = &XLogCtl->Insert;
	uint64		startbytepos;
	uint64		endbytepos;
	uint64		prevbytepos;

	size = MAXALIGN(size);

	/* All (non xlog-switch) records should contain data. */
	Assert(size > SizeOfXLogRecord);

	/*
	 * The duration the spinlock needs to be held is minimized by minimizing
	 * the calculations that have to be done while holding the lock. The
	 * current tip of reserved WAL is kept in CurrBytePos, as a byte position
	 * that only counts "usable" bytes in WAL, that is, it excludes all WAL
	 * page headers. The mapping between "usable" byte positions and physical
	 * positions (XLogRecPtrs) can be done outside the locked region, and
	 * because the usable byte position doesn't include any headers, reserving
	 * X bytes from WAL is almost as simple as "CurrBytePos += X".
	 */
	SpinLockAcquire(&Insert->insertpos_lck);

	startbytepos = Insert->CurrBytePos;
	endbytepos = startbytepos + size;
	prevbytepos = Insert->PrevBytePos;
	Insert->CurrBytePos = endbytepos;
	Insert->PrevBytePos = startbytepos;

	SpinLockRelease(&Insert->insertpos_lck);

	*StartPos = XLogBytePosToRecPtr(startbytepos);
	*EndPos = XLogBytePosToEndRecPtr(endbytepos);
	*PrevPtr = XLogBytePosToRecPtr(prevbytepos);

	/*
	 * Check that the conversions between "usable byte positions" and
	 * XLogRecPtrs work consistently in both directions.
	 */
	Assert(XLogRecPtrToBytePos(*StartPos) == startbytepos);
	Assert(XLogRecPtrToBytePos(*EndPos) == endbytepos);
	Assert(XLogRecPtrToBytePos(*PrevPtr) == prevbytepos);
}

The key trick is the “usable byte position”.

An XLogRecPtr is a physical byte position in the WAL file, and it has to account for the WAL page headers — a 24-byte long header at the start of each segment, and a 24-byte short header at the start of every other page. The mapping between “bytes of actual WAL data” and “bytes of file offset” is therefore not a simple addition: it involves a division by the usable bytes per page, a modulo, and a branch on whether you are in the first page of a segment.

Doing that arithmetic while holding a spinlock that every backend on the system contends for would be terrible. So the shared cursor does not count file bytes. It counts usable bytes — bytes of WAL data, with all page headers excluded. Reserving space is then literally:

 c
 
	endbytepos = startbytepos + size;

Three additions and two assignments. The expensive conversion happens after the lock is released.

The conversion function itself:

 c
 
 
/*
 * Converts a "usable byte position" to XLogRecPtr. A usable byte position
 * is the position starting from the beginning of WAL, excluding all WAL
 * page headers.
 */
static XLogRecPtr
XLogBytePosToRecPtr(uint64 bytepos)
{
	uint64		fullsegs;
	uint64		fullpages;
	uint64		bytesleft;
	uint32		seg_offset;
	XLogRecPtr	result;

	fullsegs = bytepos / UsableBytesInSegment;
	bytesleft = bytepos % UsableBytesInSegment;

	if (bytesleft < XLOG_BLCKSZ - SizeOfXLogLongPHD)
	{
		/* fits on first page of segment */
		seg_offset = bytesleft + SizeOfXLogLongPHD;
	}
	else
	{
		/* account for the first page on segment with long header */
		seg_offset = XLOG_BLCKSZ;
		bytesleft -= XLOG_BLCKSZ - SizeOfXLogLongPHD;

		fullpages = bytesleft / UsableBytesInPage;
		bytesleft = bytesleft % UsableBytesInPage;

		seg_offset += fullpages * XLOG_BLCKSZ + bytesleft + SizeOfXLogShortPHD;
	}

	XLogSegNoOffsetToRecPtr(fullsegs, seg_offset, wal_segment_size, result);

	return result;
}

Two divisions and two modulos, done outside the lock. There is also a XLogBytePosToEndRecPtr() variant which differs in one respect: at a page boundary it returns the position before the page header, because it is used to compute the end of a record rather than the start of one.

The shared cursor itself:

 c
 
 
typedef struct XLogCtlInsert
{
	slock_t		insertpos_lck;	/* protects CurrBytePos and PrevBytePos */

	/*
	 * CurrBytePos is the end of reserved WAL. The next record will be
	 * inserted at that position. PrevBytePos is the start position of the
	 * previously inserted (or rather, reserved) record - it is copied to the
	 * prev-link of the next record. These are stored as "usable byte
	 * positions" rather than XLogRecPtrs (see XLogBytePosToRecPtr()).
	 */
	uint64		CurrBytePos;
	uint64		PrevBytePos;

	/*
	 * Make sure the above heavily-contended spinlock and byte positions are
	 * on their own cache line. In particular, the RedoRecPtr and full page
	 * write variables below should be on a different cache line. They are
	 * read on every WAL insertion, but updated rarely, and we don't want
	 * those reads to steal the cache line containing Curr/PrevBytePos.
	 */
	char		pad[PG_CACHE_LINE_SIZE];

The padding is 128 bytes:

#define PG_CACHE_LINE_SIZE		128

The comment explains the reasoning precisely: RedoRecPtr and fullPageWrites are read on every WAL insertion, but changed only at checkpoints. If they shared a cache line with CurrBytePos, every insertion would be reading a line that a checkpoint might be invalidating. The 128-byte pad guarantees that the hot, mutable cursor and the cold, rarely-mutated metadata sit on different lines.

Note also PrevBytePos. It is the start of the previously reserved record, and it is what gets copied into the new record’s xl_prev field — the backward link that lets WAL readers walk the stream in both directions.

Phase 2: the CRC

Back in XLogInsertRecord(), now that xl_prev is known:

 c
 
 
	if (inserted)
	{
		/*
		 * Now that xl_prev has been filled in, calculate CRC of the record
		 * header.
		 */
		rdata_crc = rechdr->xl_crc;
		COMP_CRC32C(rdata_crc, rechdr, offsetof(XLogRecord, xl_crc));
		FIN_CRC32C(rdata_crc);
		rechdr->xl_crc = rdata_crc;

The CRC is computed over the record header up to but not including the xl_crc field itself (which is why it is offsetof(XLogRecord, xl_crc)), and the CRC of the record data was computed earlier by the caller, in xloginsert.c:

 c
 
 
	INIT_CRC32C(rdata_crc);
	COMP_CRC32C(rdata_crc, hdr_scratch + SizeOfXLogRecord, hdr_rdt.len - SizeOfXLogRecord);

The INIT_CRC32C/COMP_CRC32C/FIN_CRC32C macros dispatch to the best available hardware implementation — SSE4.2, ARMv8 CRC extension, or the software table-based one — via a function pointer chosen at startup:

 c
 
 
#define COMP_CRC32C(crc, data, len) \
	((crc) = pg_comp_crc32c_dispatch((crc), (data), (len)))
#define FIN_CRC32C(crc) ((crc) ^= 0xFFFFFFFF)

extern pg_crc32c (*pg_comp_crc32c) (pg_crc32c crc, const void *data, size_t len);
extern pg_crc32c pg_comp_crc32c_sse42(pg_crc32c crc, const void *data, size_t len);

Phase 2: copying the record

 c
 
 
static void
CopyXLogRecordToWAL(int write_len, bool isLogSwitch, XLogRecData *rdata,
					XLogRecPtr StartPos, XLogRecPtr EndPos, TimeLineID tli)
{
	char	 *currpos;
	int			freespace;
	int			written;
	XLogRecPtr	CurrPos;
	XLogPageHeader pagehdr;

	/*
	 * Get a pointer to the right place in the right WAL buffer to start
	 * inserting to.
	 */
	CurrPos = StartPos;
	currpos = GetXLogBuffer(CurrPos, tli);
	freespace = INSERT_FREESPACE(CurrPos);

	/*
	 * there should be enough space for at least the first field (xl_tot_len)
	 * on this page.
	 */
	Assert(freespace >= sizeof(uint32));

	/* Copy record data */
	written = 0;
	while (rdata != NULL)
	{
		const char *rdata_data = rdata->data;
		int			rdata_len = rdata->len;

		while (rdata_len > freespace)
		{
			/*
			 * Write what fits on this page, and continue on the next page.
			 */
			Assert(CurrPos % XLOG_BLCKSZ >= SizeOfXLogShortPHD || freespace == 0);
			memcpy(currpos, rdata_data, freespace);
			rdata_data += freespace;
			rdata_len -= freespace;
			written += freespace;
			CurrPos += freespace;

			/*
			 * Get pointer to beginning of next page, and set the xlp_rem_len
			 * in the page header. Set XLP_FIRST_IS_CONTRECORD.
			 *
			 * It's safe to set the contrecord flag and xlp_rem_len without a
			 * lock on the page. All the other flags were already set when the
			 * page was initialized, in AdvanceXLInsertBuffer, and we're the
			 * only backend that needs to set the contrecord flag.
			 */
			currpos = GetXLogBuffer(CurrPos, tli);
			pagehdr = (XLogPageHeader) currpos;
			pagehdr->xlp_rem_len = write_len - written;
			pagehdr->xlp_info |= XLP_FIRST_IS_CONTRECORD;

			/* skip over the page header */
			if (XLogSegmentOffset(CurrPos, wal_segment_size) == 0)
			{
				CurrPos += SizeOfXLogLongPHD;
				currpos += SizeOfXLogLongPHD;
			}
			else
			{
				CurrPos += SizeOfXLogShortPHD;
				currpos += SizeOfXLogShortPHD;
			}
			freespace = INSERT_FREESPACE(CurrPos);
		}
		...

Continuation records are the interesting part. A WAL record may be larger than a page, or may straddle a page boundary for any other reason. When that happens, the record is split across pages, and each page after the first carries a header flag XLP_FIRST_IS_CONTRECORD and a field xlp_rem_len saying how much of the record is still to come. The WAL reader uses xlp_rem_len to reassemble the record.

And this is all done without holding any page lock, because the inserting backend owns that region of the WAL exclusively — it reserved it in phase 1, and nobody else writes there.

Getting the buffer

GetXLogBuffer() maps an LSN to a pointer into the WAL buffer cache. The mapping is arithmetic, not a lookup table:

 c
 
 
/*
 * XLogRecPtrToBufIdx returns the index of the WAL buffer that holds, or
 * would hold if it was in cache, the page containing 'recptr'.
 */
#define XLogRecPtrToBufIdx(recptr)	\
	(((recptr) / XLOG_BLCKSZ) % (XLogCtl->XLogCacheBlck + 1))

The WAL buffer cache is a ring: page number modulo the number of buffers. Consecutive WAL pages therefore land in consecutive buffer slots, wrapping around at the end. That is what makes the single big write() possible later on.

The WAL insertion locks

Eight locks, in shared memory, padded so they do not share cache lines:

 c
 
 
/*
 * Number of WAL insertion locks to use. A higher value allows more insertions
 * to happen concurrently, but adds some CPU overhead to flushing the WAL,
 * which needs to iterate all the locks.
 */
#define NUM_XLOGINSERT_LOCKS 8

 c
 
 
/*
 * All the WAL insertion locks are allocated as an array in shared memory. We
 * force the array stride to be a power of 2, which saves a few cycles in
 * indexing, but more importantly also ensures that individual slots don't
 * cross cache line boundaries. (Of course, we have to also ensure that the
 * array start address is suitably aligned.)
 */
typedef union WALInsertLockPadded
{
	WALInsertLock l;
	char		pad[PG_CACHE_LINE_SIZE];
} WALInsertLockPadded;

And the lock itself:

 c
 
 
typedef struct
{
	LWLock		lock;
	pg_atomic_uint64 insertingAt;
	XLogRecPtr	lastImportantAt;
} WALInsertLock;

Three fields: the LWLock itself, the insertingAt progress indicator, and lastImportantAt (used to decide whether a checkpoint needs to do anything).

The design rationale for the whole scheme is spelled out at length:

 c
 
 
/*
 * Inserting to WAL is protected by a small fixed number of WAL insertion
 * locks. To insert to the WAL, you must hold one of the locks - it doesn't
 * matter which one. To lock out other concurrent insertions, you must hold
 * of them. Each WAL insertion lock consists of a lightweight lock, plus an
 * indicator of how far the insertion has progressed (insertingAt).
 *
 * The insertingAt values are read when a process wants to flush WAL from
 * the in-memory buffers to disk, to check that all the insertions to the
 * region the process is about to write out have finished. You could simply
 * wait for all currently in-progress insertions to finish, but the
 * insertingAt indicator allows you to ignore insertions to later in the WAL,
 * so that you only wait for the insertions that are modifying the buffers
 * you're about to write out.
 *
 * This isn't just an optimization. If all the WAL buffers are dirty, an
 * inserter that's holding a WAL insert lock might need to evict an old WAL
 * buffer, which requires flushing the WAL. If it's possible for an inserter
 * to block on another inserter unnecessarily, deadlock can arise when two
 * inserters holding a WAL insert lock wait for each other to finish their
 * insertion.
 *
 * Small WAL records that don't cross a page boundary never update the value,
 * the WAL record is just copied to the page and the lock is released. But
 * to avoid the deadlock-scenario explained above, the indicator is always
 * updated before sleeping while holding an insertion lock.
 *
 * lastImportantAt contains the LSN of the last important WAL record inserted
 * using a given lock. This value is used to detect if there has been
 * important WAL activity since the last time some action, like a checkpoint,
 * was performed - allowing to not repeat the action if not. The LSN is
 * updated for all insertions, unless the XLOG_MARK_UNIMPORTANT flag was
 * set. lastImportantAt is never cleared, only overwritten by the LSN of newer
 * records. Tracking the WAL activity directly in WALInsertLock has the
 * advantage of not needing any additional locks to update the value.
 */

That deadlock scenario is worth dwelling on. If the WAL writer had to wait for all in-progress insertions to finish, and an inserter needs to evict a dirty WAL buffer (which requires flushing, which requires the write lock), then inserter A could be waiting for the write lock while the writer waits for inserter A. The insertingAt protocol breaks the cycle: the writer only waits for insertions that are actually modifying the buffers it is about to write out, and an inserter that is blocked publishes its position before blocking.

Sticky affinity

 c
 
 
static void
WALInsertLockAcquire(void)
{
	bool		immed;

	/*
	 * It doesn't matter which of the WAL insertion locks we acquire, so try
	 * the one we used last time. If the system isn't particularly busy, it's
	 * a good bet that it's still available, and it's good to have some
	 * affinity to a particular lock so that you don't unnecessarily bounce
	 * cache lines between processes when there's no contention.
	 *
	 * If this is the first time through in this backend, pick a lock
	 * (semi-)randomly. This allows the locks to be used evenly if you have a
	 * lot of very short connections.
	 */
	static int	lockToTry = -1;

	if (lockToTry == -1)
		lockToTry = MyProcNumber % NUM_XLOGINSERT_LOCKS;
	MyLockNo = lockToTry;

	/*
	 * The insertingAt value is initially set to 0, as we don't know our
	 * insert location yet.
	 */
	immed = LWLockAcquire(&WALInsertLocks[MyLockNo].l.lock, LW_EXCLUSIVE);
	if (!immed)
	{
		/*
		 * If we couldn't get the lock immediately, try another lock next
		 * time. On a system with more insertion locks than concurrent
		 * inserters, this causes all the inserters to eventually migrate to a
		 * lock that no-one else is using. On a system with more inserters
		 * than locks, it still helps to distribute the inserters evenly
		 * across the locks.
		 */
		lockToTry = (lockToTry + 1) % NUM_XLOGINSERT_LOCKS;
	}
}

Two behaviours in ten lines. Affinity: try the same lock you used last time, because then the lock’s cache line is likely still in your cache. Rotation: if you had to wait for the lock, move to the next one next time. The comment explains why rotation is better than contention: with more locks than inserters, everyone migrates to a private lock; with more inserters than locks, at least the contention is spread evenly.

Note that the rotation only happens on failure to acquire immediately. A backend that always gets the lock immediately never rotates and keeps its affinity forever. That is the point.

The insertingAt protocol

insertingAt is an atomic LSN. It is set to the position the inserter has reached, and it is reset to 0 when the insertion lock is released:

 c
 
 
/*
 * Release our insertion lock (or locks, if we're holding them all).
 *
 * NB: Reset all variables to 0, so they cause LWLockWaitForVar to block the
 * next time the lock is acquired.
 */
static void
WALInsertLockRelease(void)
{
	if (holdingAllLocks)
	{
		int			i;

		for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
			LWLockReleaseClearVar(&WALInsertLocks[i].l.lock,
								 &WALInsertLocks[i].l.insertingAt,
								 0);

		holdingAllLocks = false;
	}
	else
	{
		LWLockReleaseClearVar(&WALInsertLocks[MyLockNo].l.lock,
							 &WALInsertLocks[MyLockNo].l.insertingAt,
							 0);
	}
}

And it is updated when the inserter crosses a page boundary:

 c
 
 
/*
 * Update our insertingAt value, to let others know that we've finished
 * inserting up to that point.
 */
static void
WALInsertLockUpdateInsertingAt(XLogRecPtr insertingAt)
{
	if (holdingAllLocks)
	{
		/*
		 * We use the last lock to mark our actual position, see comments in
		 * WALInsertLockAcquireExclusive.
		 */
		LWLockUpdateVar(&WALInsertLocks[NUM_XLOGINSERT_LOCKS - 1].l.lock,
						&WALInsertLocks[NUM_XLOGINSERT_LOCKS - 1].l.insertingAt,
						insertingAt);
	}
	else
		LWLockUpdateVar(&WALInsertLocks[MyLockNo].l.lock,
						&WALInsertLocks[MyLockNo].l.insertingAt,
						insertingAt);
}

The “all locks” case is for XLOG_SWITCH and checkpoints, which must exclude every other inserter:

 c
 
 
static void
WALInsertLockAcquireExclusive(void)
{
	int			i;

	/*
	 * When holding all the locks, all but the last lock's insertingAt
	 * indicator is set to 0xFFFFFFFFFFFFFFFF, which is higher than any real
	 * XLogRecPtr value, to make sure that no-one blocks waiting on those.
	 */
	for (i = 0; i < NUM_XLOGINSERT_LOCKS - 1; i++)
	{
		LWLockAcquire(&WALInsertLocks[i].l.lock, LW_EXCLUSIVE);
		LWLockUpdateVar(&WALInsertLocks[i].l.lock,
						&WALInsertLocks[i].l.insertingAt,
						PG_UINT64_MAX);
	}
	/* Variable value reset to 0 at release */
	LWLockAcquire(&WALInsertLocks[i].l.lock, LW_EXCLUSIVE);

	holdingAllLocks = true;
}

PG_UINT64_MAX is “further than any real LSN”, so a waiter sees an insertion position beyond its target and stops waiting.

The interlock: WaitXLogInsertionsToFinish

Before the writer can write out a range of WAL, it must be sure that no backend is still memcpy()ing into that range. This is what WaitXLogInsertionsToFinish() does:

 c
 
 
/*
 * Wait for any WAL insertions < upto to finish.
 *
 * Returns the location of the oldest insertion that is still in-progress.
 * Any WAL prior to that point has been fully copied into WAL buffers, and
 * can be flushed out to disk. Because this waits for any insertions older
 * than 'upto' to finish, the return value is always >= 'upto'.
 *
 * Note: When you are about to write out WAL, you must call this function
 * *before* acquiring WALWriteLock, to avoid deadlocks. This function might
 * need to wait for an insertion to finish (or at least advance to next
 * uninitialized page), and the inserter might need to evict an old WAL buffer
 * to make room for a new one, which in turn requires WALWriteLock.
 */
static XLogRecPtr
WaitXLogInsertionsToFinish(XLogRecPtr upto)

The “call this before acquiring WALWriteLock” note is the deadlock avoidance again: the inserter may need the write lock to evict a buffer, so the writer must not be holding it while waiting.

The body:

 c
 
 
	/*
	 * Check if there's any work to do. Use a barrier to ensure we get the
	 * freshest value.
	 */
	inserted = pg_atomic_read_membarrier_u64(&XLogCtl->logInsertResult);
	if (upto <= inserted)
		return inserted;

	/* Read the current insert position */
	SpinLockAcquire(&Insert->insertpos_lck);
	bytepos = Insert->CurrBytePos;
	SpinLockRelease(&Insert->insertpos_lck);
	reservedUpto = XLogBytePosToEndRecPtr(bytepos);

Then the loop over the insertion locks:

 c
 
 
	finishedUpto = reservedUpto;
	for (i = 0; i < NUM_XLOGINSERT_LOCKS; i++)
	{
		XLogRecPtr	insertingat = InvalidXLogRecPtr;

		do
		{
			/*
			 * See if this insertion is in progress. LWLockWaitForVar will
			 * wait for the lock to be released, or for the 'value' to be set
			 * by a LWLockUpdateVar call. When a lock is initially acquired,
			 * its value is 0 (InvalidXLogRecPtr), which means that we don't
			 * know where it's inserting yet. We will have to wait for it. If
			 * it's a small insertion, the record will most likely fit on the
			 * same page and the inserter will release the lock without ever
			 * calling LWLockUpdateVar. But if it has to sleep, it will
			 * advertise the insertion point with LWLockUpdateVar before
			 * sleeping.
			 *
			 * In this loop we are only waiting for insertions that started
			 * before WaitXLogInsertionsToFinish was called. The lack of
			 * memory barriers in the loop means that we might see locks as
			 * "unused" that have since become used. This is fine because
			 * they only can be used for later insertions that we would not
			 * want to wait on anyway. Not taking a lock to acquire the
			 * current insertingAt value means that we might see older
			 * insertingAt values. This is also fine, because if we read a
			 * value too old, we will add ourselves to the wait queue, which
			 * contains atomic operations.
			 */
			if (LWLockWaitForVar(&WALInsertLocks[i].l.lock,
								 &WALInsertLocks[i].l.insertingAt,
								 insertingat, &insertingat))
			{
				/* the lock was free, so no insertion in progress */
				insertingat = InvalidXLogRecPtr;
				break;
			}

			/*
			 * This insertion is still in progress. Have to wait, unless the
			 * inserter has proceeded past 'upto'.
			 */
		} while (insertingat < upto);

		if (XLogRecPtrIsValid(insertingat) && insertingat < finishedUpto)
			finishedUpto = insertingat;
	}

The rule: for each insertion lock, wait until either the lock is free (no insertion in progress) or the inserter’s published position is beyond our target. The result is the oldest still-in-progress insertion position, which is the point up to which WAL is safe to write out.

And the final step, which is a neat atomic trick:

 c
 
 
	/*
	 * Advance the limit we know to have been inserted and return the freshest
	 * value we know of, which might be beyond what we requested if somebody
	 * is concurrently doing this with an 'upto' pointer ahead of us.
	 */
	finishedUpto = pg_atomic_monotonic_advance_u64(&XLogCtl->logInsertResult,
												 finishedUpto);

	return finishedUpto;

pg_atomic_monotonic_advance_u64() moves the shared logInsertResult forward, but only forwards:

 c
 
 
pg_atomic_monotonic_advance_u64(volatile pg_atomic_uint64 *ptr, uint64 target)
{
	uint64		currval;

	...
	currval = pg_atomic_read_u64_impl(ptr);
	if (currval >= target)
	{
		pg_memory_barrier();
		return currval;
	}

	while (currval < target)
	{
		if (pg_atomic_compare_exchange_u64(ptr, &currval, target))
			return target;
	}

	return currval;
}

Two concurrent flushers can both be waiting on insertions; whichever one learns about a later position first publishes it, and the other one gets the fresher value back rather than its own older one. That is why the docstring says the return value “might be beyond what we requested if somebody is concurrently doing this with an ‘upto’ pointer ahead of us”.

Writing: one big pwrite()

XLogWrite() is called with WALWriteLock held. Its job is to move bytes from the WAL buffer ring into the WAL file.

 c
 
 
/*
 * Write and/or fsync the log at least as far as WriteRqst indicates.
 *
 * If flexible == true, we don't have to write as far as WriteRqst, but
 * may stop at any convenient boundary (such as a cache or logfile boundary).
 * This option allows us to avoid uselessly issuing multiple writes when a
 * single one would do.
 *
 * Must be called with WALWriteLock held. WaitXLogInsertionsToFinish(WriteRqst)
 * must be called before grabbing the lock, to make sure the data is ready to
 * write.
 */

The gathering logic:

 c
 
 
	/*
	 * Since successive pages in the xlog cache are consecutively allocated,
	 * we can usually gather multiple pages together and issue just one
	 * write() call. npages is the number of pages we have determined can be
	 * written together; startidx is the cache block index of the first one,
	 * and startoffset is the file offset at which it should go. The latter
	 * two variables are only valid when npages > 0, but we must initialize
	 * all of them to keep the compiler quiet.
	 */
	npages = 0;
	startidx = 0;
	startoffset = 0;

“Successive pages in the xlog cache are consecutively allocated” — because the buffer ring maps WAL page number modulo the number of buffers, and the buffers themselves are one contiguous allocation:

 c
 
 
	/* and the buffers themselves */
	size = add_size(size, mul_size(XLOG_BLCKSZ, XLOGbuffers));

 c
 
 
	/*
	 * Align the start of the page buffers to a full xlog block size boundary.
	 * This simplifies some calculations in XLOG insertion. It is also
	 * required for O_DIRECT.
	 */
	allocptr = (char *) TYPEALIGN(XLOG_BLCKSZ, allocptr);
	XLogCtl->pages = allocptr;
	memset(XLogCtl->pages, 0, (Size) XLOG_BLCKSZ * XLOGbuffers);

So consecutive WAL pages are contiguous in memory, and they are contiguous in the file too — unless the ring wraps, or the segment boundary is hit. Both of those cases break the run, and the code handles them:

 c
 
 
		/* Add current page to the set of pending pages-to-dump */
		if (npages == 0)
		{
			/* first of group */
			startidx = curridx;
			startoffset = XLogSegmentOffset(LogwrtResult.Write - XLOG_BLCKSZ,
											wal_segment_size);
		}
		npages++;

		/*
		 * Dump the set if this will be the last loop iteration, or if we are
		 * at the last page of the cache area (since the next page won't be
		 * contiguous in memory), or if we are at the end of the logfile
		 * segment.
		 */
		last_iteration = WriteRqst.Write <= LogwrtResult.Write;

		finishing_seg = !ispartialpage &&
			(startoffset + npages * XLOG_BLCKSZ) >= wal_segment_size;

		if (last_iteration ||
			curridx == XLogCtl->XLogCacheBlck ||
			finishing_seg)
		{
			char	 *from;
			Size		nbytes;
			Size		nleft;
			ssize_t		written;
			instr_time	start;

			/* OK to write the page(s) */
			from = XLogCtl->pages + startidx * (Size) XLOG_BLCKSZ;
			nbytes = npages * (Size) XLOG_BLCKSZ;
			nleft = nbytes;
			do
			{
				errno = 0;

				/*
				 * Measure I/O timing to write WAL data, for pg_stat_io.
				 */
				start = pgstat_prepare_io_time(track_wal_io_timing);

				pgstat_report_wait_start(WAIT_EVENT_WAL_WRITE);
				written = pg_pwrite(openLogFile, from, nleft, startoffset);
				pgstat_report_wait_end();
				...

One pg_pwrite() for the whole run of pages. The three ways a run ends: the write request is satisfied, we hit the end of the buffer ring (the next page is not contiguous in memory), or we hit the end of the WAL segment.

The segment boundary also forces an fsync():

 c
 
 
			/*
			 * If we just wrote the whole last page of a logfile segment,
			 * fsync the segment immediately. This avoids having to go back
			 * and re-open prior segments when an fsync request comes along
			 * later. Doing it here ensures that one and only one backend will
			 * perform this fsync.
			 *
			 * This is also the right place to notify the Archiver that the
			 * segment is ready to copy to archival storage, and to update the
			 * timer for archive_timeout, and to signal for a checkpoint if
			 * too many logfile segments have been used since the last
			 * checkpoint.
			 */
			if (finishing_seg)
			{
				issue_xlog_fsync(openLogFile, openLogSegNo, tli);

				/* signal that we need to wakeup walsenders later */
				WalSndWakeupRequest();

				LogwrtResult.Flush = LogwrtResult.Write;	/* end of page */

“Doing it here ensures that one and only one backend will perform this fsync” — the segment-boundary fsync is a fixed cost paid once per segment, not once per commit.

And the frontier publication, with the barrier discipline:

 c
 
 
	/*
	 * We write Write first, bar, then Flush. When reading, the opposite must
	 * be done (with a matching barrier in between), so that we always see a
	 * Flush value that trails behind the Write value seen.
	 */
	pg_atomic_write_u64(&XLogCtl->logWriteResult, LogwrtResult.Write);
	pg_write_barrier();
	pg_atomic_write_u64(&XLogCtl->logFlushResult, LogwrtResult.Flush);

Flushing: the group commit loop

Here is the heart of the thing. XLogFlush():

 c
 
 
/*
 * Ensure that all XLOG data through the given position is flushed to disk.
 *
 * NOTE: this differs from XLogWrite mainly in that the WALWriteLock is not
 * already held, and we try to avoid acquiring it if possible.
 */
void
XLogFlush(XLogRecPtr record)

The loop:

 c
 
 
	/*
	 * Since fsync is usually a horribly expensive operation, we try to
	 * piggyback as much data as we can on each fsync: if we see any more data
	 * entered into the xlog buffer, we'll write and fsync that too, so that
	 * the final value of LogwrtResult.Flush is as large as possible. This
	 * gives us some chance of avoiding another fsync immediately after.
	 */

	/* initialize to given target; may increase below */
	WriteRqstPtr = record;

	/*
	 * Now wait until we get the write lock, or someone else does the flush
	 * for us.
	 */
	for (;;)
	{
		XLogRecPtr	insertpos;

		/* done already? */
		RefreshXLogWriteResult(LogwrtResult);
		if (record <= LogwrtResult.Flush)
			break;

		/*
		 * Before actually performing the write, wait for all in-flight
		 * insertions to the pages we're about to write to finish.
		 */
		SpinLockAcquire(&XLogCtl->info_lck);
		if (WriteRqstPtr < XLogCtl->LogwrtRqst.Write)
			WriteRqstPtr = XLogCtl->LogwrtRqst.Write;
		SpinLockRelease(&XLogCtl->info_lck);
		insertpos = WaitXLogInsertionsToFinish(WriteRqstPtr);

		/*
		 * Try to get the write lock. If we can't get it immediately, wait
		 * until it's released, and recheck if we still need to do the flush
		 * or if the backend that held the lock did it for us already. This
		 * helps to maintain a good rate of group committing when the system
		 * is bottlenecked by the speed of fsyncing.
		 */
		if (!LWLockAcquireOrWait(WALWriteLock, LW_EXCLUSIVE))
		{
			/*
			 * The lock is now free, but we didn't acquire it yet. Before we
			 * do, loop back to check if someone else flushed the record for
			 * us already.
			 */
			continue;
		}

		/* Got the lock; recheck whether request is satisfied */
		RefreshXLogWriteResult(LogwrtResult);
		if (record <= LogwrtResult.Flush)
		{
			LWLockRelease(WALWriteLock);
			break;
		}

Three phases in each iteration:

Check whether the record is already flushed. If yes, done — no lock taken at all.
Wait for in-flight insertions to finish, and pick up any extra WAL that has been requested since (WriteRqstPtr is bumped to XLogCtl->LogwrtRqst.Write). This is the piggybacking: you flush everything, not just your own record.
Try to take the write lock, but with LWLockAcquireOrWait(). If somebody else holds it, you wait for them to finish and then re-check.

Then the actual write, and the commit_delay sleep:

 c
 
 
		/*
		 * Sleep before flush! By adding a delay here, we may give further
		 * backends the opportunity to join the backlog of group commit
		 * followers; this can significantly improve transaction throughput,
		 * at the risk of increasing transaction latency.
		 *
		 * We do not sleep if enableFsync is not turned on, nor if there are
		 * fewer than CommitSiblings other backends with active transactions.
		 */
		if (CommitDelay > 0 && enableFsync &&
			MinimumActiveBackends(CommitSiblings))
		{
			pgstat_report_wait_start(WAIT_EVENT_COMMIT_DELAY);
			pg_usleep(CommitDelay);
			pgstat_report_wait_end();
			...
		}

		/* try to write/flush later additions to XLOG as well */
		WriteRqst.Write = insertpos;
		WriteRqst.Flush = insertpos;

		XLogWrite(WriteRqst, insertTLI, false);

		LWLockRelease(WALWriteLock);
		/* done */
		break;
	}

commit_delay and commit_siblings are GUCs, defined in src/backend/utils/misc/guc_parameters.dat:

text

# we have no microseconds designation, so can't supply units here
{ name => 'commit_delay', type => 'int', context => 'PGC_SUSET', group => 'WAL_SETTINGS',
 short_desc => 'Sets the delay in microseconds between transaction commit and flushing WAL to disk.',
 variable => 'CommitDelay',
 boot_val => '0',
 min => '0',
 max => '100000',
},

{ name => 'commit_siblings', type => 'int', context => 'PGC_USERSET', group => 'WAL_SETTINGS',
 short_desc => 'Sets the minimum number of concurrent open transactions required before performing "commit_delay".',
 variable => 'CommitSiblings',
 boot_val => '5',
 min => '0',
 max => '1000',
},

The tradeoff they express is stated plainly in the comment: commit_delay trades latency for throughput. Sleeping before the fsync gives other backends time to arrive at the same WALWriteLock and become followers of the same fsync, so one fsync() covers more commits. But the sleeping backend’s own commit is now commit_delay microseconds slower.

commit_siblings is the guard that keeps this from being stupid: don’t sleep if you are the only one committing. MinimumActiveBackends() implements it, and it is deliberately racy:

 c
 
 
/*
 * MinimumActiveBackends --- count backends (other than myself) that are
 *		in active transactions. Return true if the count exceeds the
 *		minimum threshold passed. This is used as a heuristic to decide if
 *		a pre-XLOG-flush delay is worthwhile during commit.
 *
 * Do not count backends that are blocked waiting for locks, since they are
 * not going to get to run until someone else commits.
 */
bool
MinimumActiveBackends(int min)
{
	...
	/*
	 * Note: for speed, we don't acquire ProcArrayLock. This is a little bit
	 * bogus, but since we are only testing fields for zero or nonzero, it
	 * should be OK. The result is only used for heuristic purposes anyway...
	 */

“It is a little bit bogus” — the PostgreSQL source is refreshingly honest about its own heuristics. It is fine, because the decision is only about whether to sleep a few microseconds.

LWLockAcquireOrWait: the funky one

Now the function that makes all of this work (src/backend/storage/lmgr/lwlock.c):

 c
 
 
/*
 * LWLockAcquireOrWait - Acquire lock, or wait until it's free
 *
 * The semantics of this function are a bit funky. If the lock is currently
 * free, it is acquired in the given mode, and the function returns true. If
 * the lock isn't immediately free, the function waits until it is released
 * and returns false, but does not acquire the lock.
 *
 * This is currently used for WALWriteLock: when a backend flushes the WAL,
 * holding WALWriteLock, it can flush the commit records of many other
 * backends as a side-effect. Those other backends need to wait until the
 * flush finishes, but don't need to acquire the lock anymore. They can just
 * wake up, observe that their records have already been flushed, and return.
 */
bool
LWLockAcquireOrWait(LWLock *lock, LWLockMode mode)

“The semantics of this function are a bit funky.” That sentence is the entire explanation of group commit, and it is in the PostgreSQL source.

The function:

 c
 
 
	mustwait = LWLockAttemptLock(lock, mode);

	if (mustwait)
	{
		LWLockQueueSelf(lock, LW_WAIT_UNTIL_FREE);

		mustwait = LWLockAttemptLock(lock, mode);

		if (mustwait)
		{
			/*
			 * Wait until awakened. Like in LWLockAcquire, be prepared for
			 * bogus wakeups.
			 */
			...
			for (;;)
			{
				PGSemaphoreLock(proc->sem);
				if (proc->lwWaiting == LW_WS_NOT_WAITING)
					break;
				extraWaits++;
			}
			...
		}
		else
		{
			LOG_LWDEBUG("LWLockAcquireOrWait", lock, "acquired, undoing queue");

			/*
			 * Got lock in the second attempt, undo queueing. We need to treat
			 * this as having successfully acquired the lock, otherwise we'd
			 * not necessarily wake up people we've prevented from acquiring
			 * the lock.
			 */
			LWLockDequeueSelf(lock);
		}
	}
	...
	if (mustwait)
	{
		/* Failed to get lock, so release interrupt holdoff */
		RESUME_INTERRUPTS();
		LOG_LWDEBUG("LWLockAcquireOrWait", lock, "failed");
		...
	}
	else
	{
		LOG_LWDEBUG("LWLockAcquireOrWait", lock, "succeeded");
		/* Add lock to list of locks held by this backend */
		held_lwlocks[num_held_lwlocks].lock = lock;
		held_lwlocks[num_held_lwlocks++].mode = mode;
		...
	}

	return !mustwait;
}

The flow:

Lock is free. Take it, return true. You are the flusher.
Lock is busy. Queue yourself, wait to be woken. When the holder releases it, you wake up. You do not take the lock. You return false.
Lock became free while you were queueing. The second LWLockAttemptLock() succeeds, you dequeue yourself, and return true — and the comment explains why you have to treat that as a successful acquisition:

Got lock in the second attempt, undo queueing. We need to treat this as having successfully acquired the lock, otherwise we’d not necessarily wake up people we’ve prevented from acquiring the lock.

If you dequeue yourself without holding the lock, you must still perform the “grant to next waiter” step, or the queue would stall.

Back in XLogFlush(), the follower path is the continue:

 c
 
 
		if (!LWLockAcquireOrWait(WALWriteLock, LW_EXCLUSIVE))
		{
			/*
			 * The lock is now free, but we didn't acquire it yet. Before we
			 * do, loop back to check if someone else flushed the record for
			 * us already.
			 */
			continue;
		}

Loop back to the top. RefreshXLogWriteResult(LogwrtResult) re-reads the flush frontier. If the record you needed is now flushed — because the backend that held the lock flushed it as a side-effect of flushing its own much-later record — you break out of the loop and return. You never took the lock. You never issued a syscall. You waited for someone else’s fsync() and rode along.

That is group commit. It is not a queue, it is not a batching layer, it is not a separate “group commit leader” process. It is one lock and a re-read of an atomic counter.

The commit path

Now the other end: what a transaction actually does when it commits (src/backend/access/transam/xact.c).

The commit critical section

 c
 
 
		/*
		 * Mark ourselves as within our "commit critical section". This
		 * forces any concurrent checkpoint to wait until we've updated
		 * pg_xact. Without this, it is possible for the checkpoint to set
		 * REDO after the XLOG record but fail to flush the pg_xact update to
		 * disk, leading to loss of the transaction commit if the system
		 * crashes a little later.
		 *
		 * Note: we could, but don't bother to, set this flag in
		 * RecordTransactionAbort. That's because loss of a transaction abort
		 * is noncritical; the presumption would be that it aborted, anyway.
		 *
		 * It's safe to change the delayChkptFlags flag of our own backend
		 * without holding the ProcArrayLock, since we're the only one
		 * modifying it. This makes checkpoint's determination of which xacts
		 * are delaying the checkpoint a bit fuzzy, but it doesn't matter.
		 *
		 * Note, it is important to get the commit timestamp after marking the
		 * transaction in the commit critical section. See
		 * RecordTransactionCommitPrepared.
		 */
		Assert((MyProc->delayChkptFlags & DELAY_CHKPT_IN_COMMIT) == 0);
		START_CRIT_SECTION();
		MyProc->delayChkptFlags |= DELAY_CHKPT_IN_COMMIT;

		Assert(xactStopTimestamp == 0);

		/*
		 * Ensures the DELAY_CHKPT_IN_COMMIT flag write is globally visible
		 * before commit time is written.
		 */
		pg_write_barrier();

The critical section has two jobs. It stops a checkpoint from advancing the redo point past a commit record whose pg_xact update has not yet reached disk — otherwise a crash could lose a committed transaction. And the write barrier makes sure the flag is globally visible before the commit timestamp is written, so a standby cannot see a commit timestamp for a transaction that is not yet in the commit critical section.

The flush decision

 c
 
 
	/*
	 * Check if we want to commit asynchronously. We can allow the XLOG flush
	 * to happen asynchronously if synchronous_commit=off, or if the current
	 * transaction has not performed any WAL-logged operation or didn't assign
	 * an xid. The transaction can end up not writing any WAL, even if it has
	 * an xid, if it only wrote to temporary and/or unlogged tables. It can
	 * end up having written WAL without an xid if it did HOT pruning. In
	 * case of a crash, the loss of such a transaction will be irrelevant;
	 * temp tables will be lost anyway, unlogged tables will be truncated and
	 * HOT pruning will be done again later. (Given the foregoing, you might
	 * think that it would be unnecessary to emit the XLOG record at all in
	 * this case, but we don't currently try to do that. It would certainly
	 * cause problems at least in Hot Standby mode, where the
	 * KnownAssignedXids machinery requires tracking every XID assignment. It
	 * might be OK to skip it only when wal_level < replica, but for now we
	 * don't.)
	 *
	 * However, if we're doing cleanup of any non-temp rels or committing any
	 * command that wanted to force sync commit, then we must flush XLOG
	 * immediately. (We must not allow asynchronous commit if there are any
	 * non-temp tables to be deleted, because we might delete the files before
	 * the COMMIT record is flushed to disk. We do allow asynchronous commit
	 * if all to-be-deleted tables are temporary though, since they are lost
	 * anyway if we crash.)
	 */
	if ((wrote_xlog && markXidCommitted &&
		 synchronous_commit > SYNCHRONOUS_COMMIT_OFF) ||
		forceSyncCommit || nrels > 0)
	{
		XLogFlush(XactLastRecEnd);
		...
		/*
		 * Now we may update the CLOG, if we wrote a COMMIT record above
		 */
		if (markXidCommitted)
			TransactionIdCommitTree(xid, nchildren, children);
	}
	else
	{
		/*
		 * Asynchronous commit case:
		 *
		 * This enables possible committed transaction loss in the case of a
		 * postmaster crash because WAL buffers are left unwritten. Ideally we
		 * could issue the WAL write without the fsync, but some
		 * wal_sync_methods do not allow separate write/fsync.
		 *
		 * Report the latest async commit LSN, so that the WAL writer knows to
		 * flush this commit.
		 */
		XLogSetAsyncXactLSN(XactLastRecEnd);

		/*
		 * We must not immediately update the CLOG, since we didn't flush the
		 * XLOG. Instead, we store the LSN up to which the XLOG must be
		 * flushed before the CLOG may be updated.
		 */
		if (markXidCommitted)
			TransactionIdAsyncCommitTree(xid, nchildren, children, XactLastRecEnd);
	}

Note the ordering: the CLOG is updated after the WAL flush, in the synchronous case. That ordering is the whole reason a synchronous commit is safe: the commit record is on disk before anyone can see the transaction as committed.

In the asynchronous case, the CLOG update is deferred and carries the LSN that the WAL must reach first. That’s the group LSN mechanism in clog.c, which we come to below.

The synchronous_commit levels

The levels are an enum (src/include/access/xact.h):

 c
 
 
typedef enum
{
	SYNCHRONOUS_COMMIT_OFF,		/* asynchronous commit */
	SYNCHRONOUS_COMMIT_LOCAL_FLUSH, /* wait for local flush only */
	SYNCHRONOUS_COMMIT_REMOTE_WRITE,	/* wait for local flush and remote
										 * write */
	SYNCHRONOUS_COMMIT_REMOTE_FLUSH,	/* wait for local and remote flush */
	SYNCHRONOUS_COMMIT_REMOTE_APPLY,	/* wait for local and remote flush and
										 * remote apply */
}			SyncCommitLevel;

/* Define the default setting for synchronous_commit */
#define SYNCHRONOUS_COMMIT_ON	SYNCHRONOUS_COMMIT_REMOTE_FLUSH

Setting	Enum value	Waits for
`off`	`SYNCHRONOUS_COMMIT_OFF`	nothing
`local`	`SYNCHRONOUS_COMMIT_LOCAL_FLUSH`	local WAL flush
`remote_write`	`SYNCHRONOUS_COMMIT_REMOTE_WRITE`	local flush + remote write
`on`	`SYNCHRONOUS_COMMIT_REMOTE_FLUSH`	local flush + remote flush
`remote_apply`	`SYNCHRONOUS_COMMIT_REMOTE_APPLY`	local flush + remote flush + remote apply

Note the subtlety in the names: the enum value called SYNCHRONOUS_COMMIT_REMOTE_FLUSH is what on maps to. And on is the default:

#define SYNCHRONOUS_COMMIT_ON	SYNCHRONOUS_COMMIT_REMOTE_FLUSH

The accepted spellings are enumerated in src/backend/utils/misc/guc_tables.c:

 c
 
 
/*
 * Although only "on", "off", "remote_apply", "remote_write", and "local" are
 * documented, we accept all the likely variants of "on" and "off".
 */
static const struct config_enum_entry synchronous_commit_options[] = {
	{"local", SYNCHRONOUS_COMMIT_LOCAL_FLUSH, false},
	{"remote_write", SYNCHRONOUS_COMMIT_REMOTE_WRITE, false},
	{"remote_apply", SYNCHRONOUS_COMMIT_REMOTE_APPLY, false},
	{"on", SYNCHRONOUS_COMMIT_ON, false},
	{"off", SYNCHRONOUS_COMMIT_OFF, false},
	{"true", SYNCHRONOUS_COMMIT_ON, true},
	{"false", SYNCHRONOUS_COMMIT_OFF, true},
	{"yes", SYNCHRONOUS_COMMIT_ON, true},
	{"no", SYNCHRONOUS_COMMIT_OFF, true},
	{"1", SYNCHRONOUS_COMMIT_ON, true},
	{"0", SYNCHRONOUS_COMMIT_OFF, true},
	{NULL, 0, false}
};

The check in RecordTransactionCommit() is synchronous_commit > SYNCHRONOUS_COMMIT_OFF, so anything above off takes the flushing path.

The asynchronous committer: a latch, not an I/O

The async committer does no I/O at all. It publishes an LSN and pokes the WAL writer’s latch:

 c
 
 
/*
 * Record the LSN for an asynchronous transaction commit/abort
 * and nudge the WALWriter if there is work for it to do.
 * (This should not be called for synchronous commits.)
 */
void
XLogSetAsyncXactLSN(XLogRecPtr asyncXactLSN)
{
	XLogRecPtr	WriteRqstPtr = asyncXactLSN;
	bool		sleeping;
	bool		wakeup = false;
	XLogRecPtr	prevAsyncXactLSN;

	SpinLockAcquire(&XLogCtl->info_lck);
	sleeping = XLogCtl->WalWriterSleeping;
	prevAsyncXactLSN = XLogCtl->asyncXactLSN;
	if (XLogCtl->asyncXactLSN < asyncXactLSN)
		XLogCtl->asyncXactLSN = asyncXactLSN;
	SpinLockRelease(&XLogCtl->info_lck);

	/*
	 * If somebody else already called this function with a more aggressive
	 * LSN, they will have done what we needed (and perhaps more).
	 */
	if (asyncXactLSN <= prevAsyncXactLSN)
		return;

	/*
	 * If the WALWriter is sleeping, kick it to make it come out of low-power
	 * mode, so that this async commit will reach disk within the expected
	 * amount of time. Otherwise, determine whether it has enough WAL
	 * available to flush, the same way that XLogBackgroundFlush() does.
	 */
	if (sleeping)
		wakeup = true;
	else
	{
		int			flushblocks;

		RefreshXLogWriteResult(LogwrtResult);

		flushblocks =
			WriteRqstPtr / XLOG_BLCKSZ - LogwrtResult.Flush / XLOG_BLCKSZ;

		if (WalWriterFlushAfter == 0 || flushblocks >= WalWriterFlushAfter)
			wakeup = true;
	}

	if (wakeup)
	{
		ProcNumber	walwriterProc = pg_atomic_read_u32(&ProcGlobal->walwriterProc);

		if (walwriterProc != INVALID_PROC_NUMBER)
			SetLatch(&GetPGProcByNumber(walwriterProc)->procLatch);
	}
}

Two things happen: the LSN is recorded so the WAL writer knows how far it must eventually flush, and the WAL writer’s latch is set so it wakes up immediately rather than waiting for its next timeout.

The WAL writer’s main loop is in src/backend/postmaster/walwriter.c:

 c
 
 
		/* Clear any already-pending wakeups */
		ResetLatch(MyLatch);

		/* Process any signals received recently */
		ProcessMainLoopInterrupts();

		INJECTION_POINT("walwriter-before-background-flush", NULL);

		/*
		 * Do what we're here for; then, if XLogBackgroundFlush() found useful
		 * work to do, reset hibernation counter.
		 */
		if (XLogBackgroundFlush())
			left_till_hibernate = LOOPS_UNTIL_HIBERNATE;
		else if (left_till_hibernate > 0)
			left_till_hibernate--;

		/* report pending statistics to the cumulative stats system */
		pgstat_report_wal(false);

		/*
		 * Sleep until we are signaled or WalWriterDelay has elapsed. If we
		 * haven't done anything useful for quite some time, lengthen the
		 * sleep time so as to reduce the server's idle power consumption.
		 */
		if (left_till_hibernate > 0)
			cur_timeout = WalWriterDelay;	/* in ms */
		else
			cur_timeout = WalWriterDelay * HIBERNATE_FACTOR;

		(void) WaitLatch(MyLatch,
						 WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
						 cur_timeout,
						 WAIT_EVENT_WAL_WRITER_MAIN);

And XLogBackgroundFlush() decides how far to flush:

 c
 
 
	/*
	 * Determine how far to flush WAL, based on the wal_writer_delay and
	 * wal_writer_flush_after GUCs.
	 *
	 * Note that XLogSetAsyncXactLSN() performs similar calculation based on
	 * wal_writer_flush_after, to decide when to wake us up. Make sure the
	 * logic is the same in both places if you change this.
	 */
	now = GetCurrentTimestamp();
	flushblocks =
		WriteRqst.Write / XLOG_BLCKSZ - LogwrtResult.Flush / XLOG_BLCKSZ;

	if (WalWriterFlushAfter == 0 || lastflush == 0)
	{
		/* first call, or block based limits disabled */
		WriteRqst.Flush = WriteRqst.Write;
		lastflush = now;
	}
	else if (TimestampDifferenceExceeds(lastflush, now, WalWriterDelay))
	{
		/*
		 * Flush the writes at least every WalWriterDelay ms. This is
		 * important to bound the amount of time it takes for an asynchronous
		 * commit to hit disk.
		 */
		WriteRqst.Flush = WriteRqst.Write;
		lastflush = now;
	}
	else if (flushblocks >= WalWriterFlushAfter)
	{
		/* exceeded wal_writer_flush_after blocks, flush */
		WriteRqst.Flush = WriteRqst.Write;
		lastflush = now;
	}
	else
	{
		/* no flushing, this time round */
		WriteRqst.Flush = InvalidXLogRecPtr;
	}

So an async commit reaches disk within at most a few wal_writer_delay cycles. The WAL writer’s docstring says so:

 c
 
 
 * We can guarantee that async commits reach disk after at most three
 * wal_writer_delay cycles. (When flushing complete blocks, we allow XLogWrite
 * to write "flexibly", meaning it can stop at the end of the buffer ring;
 * this makes a difference only with very high load or long wal_writer_delay,
 * but imposes one extra cycle for the worst case for async commits.)

The CLOG group LSN

An async committer cannot update pg_xact immediately — the commit record might not be on disk yet. So instead it stores the LSN that the WAL must reach before the CLOG update is allowed to become visible. The CLOG stores those LSNs in groups of 32 transactions (src/backend/access/transam/clog.c):

 c
 
 
/* We store the latest async LSN for each group of transactions */
#define CLOG_XACTS_PER_LSN_GROUP	32	/* keep this a power of 2 */
#define CLOG_LSNS_PER_PAGE	(CLOG_XACTS_PER_PAGE / CLOG_XACTS_PER_LSN_GROUP)

#define GetLSNIndex(slotno, xid)	((slotno) * CLOG_LSNS_PER_PAGE + \
	((xid) % (TransactionId) CLOG_XACTS_PER_PAGE) / CLOG_XACTS_PER_LSN_GROUP)

And updating it:

 c
 
 
	/*
	 * Update the group LSN if the transaction completion LSN is higher.
	 *
	 * Note: lsn will be invalid when supplied during InRecovery processing,
	 * so we don't need to do anything special to avoid LSN updates during
	 * recovery. After recovery completes the next clog change will set the
	 * LSN correctly.
	 */
	if (XLogRecPtrIsValid(lsn))
	{
		int			lsnindex = GetLSNIndex(slotno, xid);

		if (XactCtl->shared->group_lsn[lsnindex] < lsn)
			XactCtl->shared->group_lsn[lsnindex] = lsn;
	}

Reading it back:

 c
 
 
/*
 * Interrogate the state of a transaction in the commit log.
 *
 * Aside from the actual commit status, this function returns (into *lsn)
 * an LSN that is late enough to be able to guarantee that if we flush up to
 * that LSN then we will have flushed the transaction's commit record to disk.
 * The result is not necessarily the exact LSN of the transaction's commit
 * record!	For example, for long-past transactions (those whose clog pages
 * already migrated to disk), we'll return InvalidXLogRecPtr. Also, because
 * we group transactions on the same clog page to conserve storage, we might
 * return the LSN of a later transaction that falls into the same group.
 *
 * NB: this is a low-level routine and is NOT the preferred entry point
 * for most uses; TransactionLogFetch() in transam.c is the intended caller.
 */
XidStatus
TransactionIdGetStatus(TransactionId xid, XLogRecPtr *lsn)
{
	...
	lsnindex = GetLSNIndex(slotno, xid);
	*lsn = XactCtl->shared->group_lsn[lsnindex];
	...
}

The grouping is a compromise: one LSN is stored per 32 transactions, so the LSN you read back might be later than your own transaction’s commit record. That is conservative in the safe direction — flushing to that LSN definitely flushed yours.

The other half of clog.c’s concurrency story is the group update mechanism, which is about the SLRU bank lock rather than about the WAL:

 c
 
 
/*
 * Subroutine for TransactionIdSetPageStatus, q.v.
 *
 * When we cannot immediately acquire the SLRU bank lock in exclusive mode at
 * commit time, add ourselves to a list of processes that need their XIDs
 * status update. The first process to add itself to the list will acquire
 * the lock in exclusive mode and set transaction status as required on behalf
 * of all group members. This avoids a great deal of contention when many
 * processes are trying to commit at once, since the lock need not be
 * repeatedly handed off from one committing process to the next.
 *
 * Returns true when transaction status has been updated in clog; returns
 ...
 */

Same trick as the WAL write lock: one backend does the work on behalf of many, and the others observe the result and move on.

Caveats

The unlucky committer pays full latency. Group commit helps throughput; it does nothing for the latency of the single backend that arrives just after an fsync started. Its record will be flushed by the next fsync, and it waits for the whole of that one.

The flusher may flush more than it needs to. XLogFlush() deliberately piggybacks: it bumps the write request to XLogCtl->LogwrtRqst.Write and flushes everything up to there. That is good for throughput — every follower gets its record flushed for free — but it means one backend’s commit can force a larger write and a larger fsync than its own record required. Under heavy write load, one unlucky commit can trigger an fsync of the entire accumulated WAL.

commit_delay is a latency-for-throughput trade, and it is only beneficial in a narrow band. Too small and it does not gather enough followers; too large and every commit pays the delay for no benefit. The commit_siblings guard exists so that a lightly loaded server does not sleep at all. In practice, on modern hardware with a decent NVMe device, commit_delay is usually best left at zero: the fsync is fast enough that the group forms naturally.

synchronous_commit = off can lose committed transactions. The source is blunt about it: “This enables possible committed transaction loss in the case of a postmaster crash because WAL buffers are left unwritten.” It is fine for bulk loads into unlogged tables and for many data-warehouse workloads. It is not fine for a ledger.

synchronous_commit = local is not “safe” in the usual sense. It waits for the local WAL flush but not for the standby. If you are not using replication, local and on are equivalent. If you are, local means a failover can lose the last few transactions.

The fsync is the whole latency. Everything above is microseconds. XLogFlush() at the end of the day is an fsync() (or fdatasync(), or O_SYNC write) on a device whose flush latency is measured in milliseconds. Group commit cannot change that number; it only changes how many commits share it.

The WAL writer can flush asynchronously while you are committing, but it cannot help a synchronous commit. The background writer and a committing backend both contend for WALWriteLock. A synchronous committer that finds the lock held by the WAL writer has to wait for it, then do its own fsync.

WaitXLogInsertionsToFinish() must be called before taking WALWriteLock. Getting this backwards is a deadlock, and the source says so twice: in the function’s own header comment, and in XLogFlush(), which does the wait outside the lock acquisition.

What it all means in practice

Commit latency is fsync latency, and nothing else. If your commits are slow, look at the device, not at PostgreSQL. pg_stat_io (with track_wal_io_timing = on) shows the WAL write and flush timings separately, and pg_stat_bgwriter has the WAL counters.

Group commit is automatic and free. You do not configure it. You do not “enable” it. Every synchronous commit already participates: if another backend is in XLogFlush(), you ride along. The only knob is commit_delay, and it is usually best left alone.

The “one backend does the fsync for everyone” design has a consequence you should know about. The flusher flushes everything it can see. If you are debugging a slow commit and see a single fsync() covering a large amount of WAL, that is not a bug — that is one backend doing the work for twenty others.

The three frontiers are the model to keep in your head. “Inserted” means in shared memory and costs nothing. “Written” means the OS has the bytes and costs a write(). “Flushed” means the device has them and costs an fsync(). A synchronous commit waits for the third. An asynchronous commit waits for none of them and hopes the WAL writer gets there.

The reservation trick is why WAL insertion scales. CurrBytePos += size under a spinlock, with the expensive LSN arithmetic moved outside the lock, is what lets a hundred backends insert WAL records at once without the insertion point becoming a bottleneck. The “usable byte position” — a cursor that does not count page headers — is the trick that makes the arithmetic inside the lock trivial.

synchronous_commit is the knob that matters. on (the default) is the durable choice. remote_write and remote_apply trade local durability for a stronger replication guarantee. local is for “I have replication but I accept losing the last few transactions on failover”. off is for bulk loads and nothing else.

The commit critical section is a real cost, and it exists for a reason. It pins checkpoints so that a commit record cannot reach disk without its pg_xact update. If you see checkpoint lag on a write-heavy workload, that is what you are looking at.

If you are reading the source to understand durability, read these four functions in order: RecordTransactionCommit() for the policy, XLogInsertRecord() and ReserveXLogInsertLocation() for how the record gets into memory, XLogFlush() for the group commit loop, and LWLockAcquireOrWait() for the trick that makes the loop work.

Wal on Opensource internals