tmux Splits the Terminal into Panes: A Tree, Not a Geometry Problem

TL;DR: The hard part of tmux’s pane splitting isn’t the geometry — it’s serialisability. The layout is a tree of three node types, distributed with a divide-and-remainder algorithm, and dumped to JSON so it can be saved, restored, applied elsewhere and shared between servers.

Splitting a terminal into panes sounds like a geometry problem, and it is — but the hard part isn’t the geometry. It’s that the layout has to be serialisable, so it can be saved when you detach, restored when you come back, applied to a different window, and shared between servers.

tmux’s layout system is a tree of cells with three node types, a divide-and-remainder algorithm for distributing space, and a JSON serialisation format.

The tree

tmux.h defines exactly three layout types:

enum layout_type { LAYOUT_LEFTRIGHT, LAYOUT_TOPBOTTOM, LAYOUT_WINDOWPANE };

• LAYOUT_LEFTRIGHT — a node split vertically into children side by side,
• LAYOUT_TOPBOTTOM — a node split horizontally into children stacked,
• LAYOUT_WINDOWPANE — a leaf holding one pane.

Each struct layout_cell has a geometry (sx, sy, xoff, yoff) and a list of children. The tree is a general n-ary tree, not a binary one, so a row of three panes is one node with three children rather than a chain of two.

struct layout_cell {
	struct layout_cell	*parent;

	enum layout_type		 type;
	struct grid_cell		 g;

	TAILQ_HEAD(layout_cells, layout_cell) cells;
	TAILQ_ENTRY(layout_cell)	 entry;

	struct window_pane	*wp;
	int			 flags;
#define LAYOUT_CELL_FLOATING 0x1
};

The LAYOUT_CELL_FLOATING flag is what makes a floating pane work: it marks a cell as not participating in the tiling. layout_cell_is_tiled() explicitly excludes it:

	int	is_floating = lc->flags & LAYOUT_CELL_FLOATING;
	return is_leaf && !is_floating;

So a floating pane is a cell in the same tree, but the tiling algorithms skip it, and it has its own position, size and z-index.

The seven presets

layout-set.c has a table of preset arrangements:

static const struct {
	const char	*name;
	void	      	(*arrange)(struct window *);
} layout_sets[] = {
	{ "even-horizontal", layout_set_even_h },
	{ "even-vertical", layout_set_even_v },
	{ "main-horizontal", layout_set_main_h },
	{ "main-horizontal-mirrored", layout_set_main_h_mirrored },
	{ "main-vertical", layout_set_main_v },
	{ "main-vertical-mirrored", layout_set_main_v_mirrored },
	{ "tiled", layout_set_tiled },
};

The lookup does prefix matching, so select-layout -t even is ambiguous between even-horizontal and even-vertical and is an error, while select-layout -t even-h works.

main-horizontal and friends put one pane in a large region and the rest in a strip. The size of the main region comes from an option specified as a percentage:

	s = options_get_string(w->options, "main-pane-height");
	mainh = args_string_percentage(s, 0, sy, sy, &cause);

so main-pane-height 60% and main-pane-height 20 (absolute cells) both work.

tiled is the interesting one. It’s recursive: it splits the window alternately vertically and horizontally depending on the aspect ratio, and calls layout_resize_adjust() to make sure no pane ends up unusably small. It’s the layout that most resembles a “just divide the screen up” expectation.

Spreading the space

The core algorithm is layout_spread_cell():

/* Spread out cells inside a parent cell. */
int
layout_spread_cell(struct window *w, struct layout_cell *parent)
{
	struct layout_cell	*lc, *root = w->layout_root;
	u_int			 number, each, size, this, remainder;
	int			 change, changed, status;

	number = 0;
	TAILQ_FOREACH (lc, &parent->cells, entry)
		if (layout_cell_is_tiled(lc) || layout_cell_has_tiled_child(lc))
			number++;
	if (number <= 1)
		return (0);
	status = window_get_pane_status(w);

	if (parent->type == LAYOUT_LEFTRIGHT)
		size = parent->g.sx;
	else if (parent->type == LAYOUT_TOPBOTTOM) {
		if (layout_add_horizontal_border(root, parent, status))
			size = parent->g.sy - 1;
		else
			size = parent->g.sy;
	} else
		return (0);
	if (size < number - 1)
		return (0);
	each = (size - (number - 1)) / number;
	if (each == 0)
		return (0);

	/*
	 * Remaining space after assigning that which can be evenly
	 * distributed.
	 */
	remainder = size - (number * (each + 1)) + 1;

The arithmetic is worth unpacking:

• size is the space available, minus one per border if the parent has horizontal borders.
• each = (size - (number - 1)) / number — divide the space, having reserved one cell for each of the number - 1 borders between cells.
• remainder = size - (number * (each + 1)) + 1 — the space left over after giving every cell each + 1 cells (the extra one accounts for the border that was reserved on the right of each cell).

The remainder is then handed out one cell at a time to the first panes, which is why even-horizontal with a window 81 columns wide and three panes gives 27, 27, 26 rather than 27, 27, 27 with a gap.

The border handling is the subtle part. A pane status line can be either above or below the pane, and whether there is a horizontal border depends on the layout, not just on the option. layout_add_horizontal_border() figures that out, and the whole calculation shifts by one depending on the answer. Getting this wrong is how you end up with a one-column gap in a layout.

Serialisation: the JSON layout string

The layout has to survive a round trip through a string, because:

• select-layout saves a layout string and applies it later,
• tmux list-windows shows the layout so you can copy it,
• join-pane and move-pane move panes between windows with layouts,
• and the layout is part of what makes a session portable between servers.

The current format is JSON. The documentation is at the top of layout-custom.c:

 * Layouts can be represented as strings in a JSON format (v2). The version 1
 * format will be removed in the future and should no longer be used.
 *
 * The current (v2) format is JSON. The top level has two keys:
 *    "V": version number, currently 2
 *    "L": root layout cell
 *
 * Each cell is an object with:
 *    "t": cell type:
 *        "h": horizontal
 *        "v": vertical
 *        "p": pane
 *    "w": cell width
 *    "h": cell height
 *    "x": horizontal position
 *    "y": vertical position
 *
 * If the cell is a node cell (with child cells), it additionally has:
 *    "c": array of child cells
 *
 * If the cell is a leaf cell (that is, containing a pane and no child cells),
 * it additionally has:
 *    "I": pane ID as %n (currently ignored on parse)
 *    "l": index into last panes list if visited and not the active pane
 *    "a": true if the active pane
 *    "i": pane index
 *    "z": z-index, if a floating pane
 */

The dump is a direct recursion:

/* Append information for a single cell in a JSON (v2) format. */
static int
layout_append_v2(struct layout_cell *lc, struct layout_string *ls)
{
	...
	layout_string_write(ls, "{\"t\":\"%c\",\"w\":%u,\"h\":%u,\"x\":%d"
	    ",\"y\":%d", c, lc->g.sx, lc->g.sy, lc->g.xoff, lc->g.yoff);
	if (type != LAYOUT_WINDOWPANE) {
		layout_string_write(ls, ",\"c\":[");
		n = 0;
		TAILQ_FOREACH(lcchild, &lc->cells, entry) {
			if (layout_append_v2(lcchild, ls) != 0)
				return (-1);
			layout_string_write(ls, ",");
			n++;
		}
		if (n == 0)
			return (-1);
		ls->dat[--ls->size] = '\0'; /* removing trailing comma */
		layout_string_write(ls, "]");
	} else {
		...
		if (wp == wp->window->active)
			layout_string_write(ls, ",\"a\":true");
		else if (window_pane_last_index(wp, &i) == 0)
			layout_string_write(ls, ",\"l\":%u", i);
		...
		if (lc->flags & LAYOUT_CELL_FLOATING) {
			z = layout_cell_zindex(lc);
			layout_string_write(ls, ",\"z\":%u", z);
		}
		layout_string_write(ls, ",\"I\":\"%%%u\"", wp->id);
	}

Note the manual comma handling — ls->dat[--ls->size] = '\0' removes the trailing comma. tmux isn’t using a JSON library; it’s writing JSON by hand, because it needs to produce it incrementally without allocating a tree of JSON values.

The top-level wrapper is:

	if (~flags & LAYOUT_CUSTOM_OLD_FORMAT)
		xasprintf(&out, "{\"V\":2,\"L\":%s}", layout_string.dat);
	else {
		xasprintf(&out, "%04hx,%s", layout_checksum(layout_string.dat),
		    layout_string.dat);
	}

So a v2 layout looks like:

{"V":2,"L":{"t":"h","w":120,"h":40,"x":0,"y":0,"c":[
  {"t":"p","w":60,"h":40,"x":0,"y":0,"a":true,"i":0,"I":"%1"},
  {"t":"p","w":59,"h":40,"x":61,"y":0,"i":1,"I":"%2"}]}}

The old format

Version 1 was a string with a checksum:

0123,80x24,0,0,1

with {} for a left-right split and [] for a top-bottom split. The checksum is a rotate-and-add:

	xasprintf(&out, "%04hx,%s", layout_checksum(layout_string.dat),
	    layout_string.dat);

and it’s verified on parse:

	if (csum != layout_checksum(input)) {
		*pctx->cause = xstrdup("invalid layout checksum");
		return (-1);
	}

The checksum exists because the v1 format had no way to mark the end of the string, so a truncated or corrupted layout could be parsed as something valid. The v1 parser is also depth-limited:

/* Maximum nesting depth for version 1 layouts. */
#define LAYOUT_V1_MAX_DEPTH 1000

which exists because the format is recursive and the input is untrusted. There’s a dedicated fuzzer for this: fuzz/layout-fuzzer.c.

Parsing is not just parsing

layout_parse() does considerably more than decode the string. After decoding it:

• closes the bottom-right cells until the number of cells matches the number of panes,
• repairs a historically-wrong root size,
• assigns the panes into the cells.

The “closes the bottom-right cells” step is a compatibility hack: older versions of tmux produced layouts with a different cell count than the pane count, and rather than reject them, tmux patches them up.

Resizing

Resizing a window is resize_window() in resize.c:

	/* If the window is zoomed, unzoom. */
	zwp = window_zoomed_pane(w);
	if (zwp != NULL) window_unzoom(w, 1);
	/* Resize the layout first. */
	layout_resize(w, sx, sy);
	/* Resize the window, it can be no smaller than the layout. */
	if (sx < w->layout_root->g.sx) sx = w->layout_root->g.sx;

The order matters: the layout is resized first, and then the window is allowed to grow to match it if the layout asked for more.

layout_resize() walks the tree and adjusts the split points. The interesting constraint is that resizing must not change the proportions the user chose — a user who made a pane 30 columns wide at 100 columns total wants it 36 columns wide at 120, not 30. layout_resize_adjust() implements that by moving the boundary between two cells by the same proportion.

There’s a minimum size check (layout_resize_check()) so a pane can’t be squeezed to nothing.

Window sizing policy

Separately from the layout, there’s the question of how big a window should be when several clients of different sizes are attached. That’s resize.c’s recalculate_size(), and it honours the window-size option:

• latest — use the size of the most recent client,
• largest — use the largest,
• smallest — use the smallest,
• manual — do not resize automatically.

And aggressive-resize, which controls whether a window is resized as soon as one client resizes, or only when the largest client resizes.

This is the part that tmux gets wrong most often in users’ minds. A window is not the size of the terminal. A window is the size that the layout requires, and the layout is sized to fit the window.

Wrapping up

The design here is a small, clean abstraction — a tree of three node types — with a JSON serialisation that makes the whole thing portable. The complexity is concentrated in three places:

1. the border arithmetic, which shifts every calculation by one cell depending on the layout,
2. the compatibility fixes on parse, which exist because old versions produced layouts that are technically wrong but practically usable, and
3. the resize proportions, which must preserve user intent rather than just fill the space.

Each of those is a case where the “obvious” implementation produces a layout that’s subtly off by one. The only defence is a regression test that checks the exact geometry — and there are a lot of those in regress/.