Go's regexp: OnePass, Backtrack and the NFA Fallback

TL;DR: Go’s regexp is not one engine, it is three. A pattern that is anchored and has unambiguous alternations runs on a one-pass engine that never backtracks. A small input runs on a bit-state backtracker with a visited bitmap. Everything else runs on a parallel NFA with sparse queues. The package doc’s promise — linear time in the size of the input — holds for all three, because none of them ever explore the same (position, instruction) pair twice.

Regular expressions are where language runtimes usually lose their minds. Backtracking engines give you the “leftmost-first” semantics that Perl and Python users expect, and then give you catastrophic backtracking, a denial of service in a regular expression.

Go’s answer was to throw away backtracking entirely and simulate an NFA. That is the theoretically right answer and it makes some things slower. What is genuinely interesting is the layer of adaptive strategy Go puts on top: the same pattern, compiled once, runs on a different engine depending on the shape of the pattern and the size of the input.

This is a walk through how it all works, based on reading the Go source tree (Go 1.28 development tree).

The promise

The package doc leads with it:

go
// src/regexp/regexp.go:13
// The regexp implementation provided by this package is
// guaranteed to run in time linear in the size of the input.
// (This is a property not guaranteed by most open source
// implementations of regular expressions.) For more information
// about this property, see https://swtch.com/~rsc/regexp/regexp1.html
// or any book about automata theory.

That guarantee is what allows Go programs to accept a regex from a user and run it over untrusted input without a timeout. Everything below is in service of it.

Compile: one program, three engines’ worth of metadata

regexp.Compile parses, simplifies and compiles the pattern, then decides what it is going to do with it:

go
// src/regexp/regexp.go:166
func compile(expr string, mode syntax.Flags, longest bool) (*Regexp, error) {
	re, err := syntax.Parse(expr, mode)
	if err != nil {
		return nil, err
	}
	maxCap := re.MaxCap()
	capNames := re.CapNames()

	re = re.Simplify()
	prog, err := syntax.Compile(re)
	if err != nil {
		return nil, err
	}
	matchcap := prog.NumCap
	if matchcap < 2 {
		matchcap = 2
	}
	regexp := &Regexp{
		expr:        expr,
		prog:        prog,
		onepass:     compileOnePass(prog),
		numSubexp:   maxCap,
		subexpNames: capNames,
		cond:        prog.StartCond(),
		longest:     longest,
		matchcap:    matchcap,
		minInputLen: minInputLen(re),
	}
	if regexp.onepass == nil {
		regexp.prefix, regexp.prefixComplete = prog.Prefix()
		regexp.maxBitStateLen = maxBitStateLen(prog)
	} else {
		regexp.prefix, regexp.prefixComplete, regexp.prefixEnd = onePassPrefix(prog)
	}
	if regexp.prefix != "" {
		// TODO(rsc): Remove this allocation by adding
		// IndexString to package bytes.
		regexp.prefixBytes = []byte(regexp.prefix)
		regexp.prefixRune, _ = utf8.DecodeRuneInString(regexp.prefix)
	}

	n := len(prog.Inst)
	i := 0
	for matchSize[i] != 0 && matchSize[i] < n {
		i++
	}
	regexp.mpool = i

	return regexp, nil
}

Four things happen here that matter later:

compileOnePass(prog) is attempted. If it succeeds, the regexp has a one-pass program and will use that engine.
If there is no one-pass program, the literal prefix is extracted (prog.Prefix()) and maxBitStateLen is computed — that is the input size below which the backtracker is allowed to run.
The prefix is also kept as bytes and its first rune, so the engine can use strings.Index to skip ahead.
The machine pool index is chosen from the program size, so that the NFA allocates queues of the right size.

The Regexp struct holds the lot:

go
// src/regexp/regexp.go:79
type Regexp struct {
	expr           string       // as passed to Compile
	prog           *syntax.Prog // compiled program
	onepass        *onePassProg // onepass program or nil
	numSubexp      int
	maxBitStateLen int
	subexpNames    []string
	prefix         string         // required prefix in unanchored matches
	prefixBytes    []byte         // prefix, as a []byte
	prefixRune     rune           // first rune in prefix
	prefixEnd      uint32         // pc for last rune in prefix
	mpool          int            // pool for machines
	matchcap       int            // size of recorded match lengths
	prefixComplete bool           // prefix is the entire regexp
	cond           syntax.EmptyOp // empty-width conditions required at start of match
	minInputLen    int            // minimum length of the input in bytes

	// This field can be modified by the Longest method,
	// but it is otherwise read-only.
	longest bool // whether regexp prefers leftmost-longest match
}

The dispatch

This is the whole engine selection, in one function:

go
// src/regexp/exec.go:517
// find finds the leftmost match in the input, appends the position
// of its subexpressions to dstCap and returns dstCap.
//
// nil is returned if no matches are found and non-nil if matches are found.
func (re *Regexp) find(r io.RuneReader, b []byte, s string, pos int, ncap int, dstCap []int) []int {
	if dstCap == nil {
		// Make sure 'return dstCap' is non-nil.
		dstCap = arrayNoInts[:0:0]
	}

	if r == nil && len(b)+len(s) < re.minInputLen {
		return nil
	}

	if re.onepass != nil {
		return re.doOnePass(r, b, s, pos, ncap, dstCap)
	}
	if r == nil && len(b)+len(s) < re.maxBitStateLen {
		return re.backtrack(b, s, pos, ncap, dstCap)
	}

	m := re.get()
	i, _ := m.inputs.init(r, b, s)

	m.init(ncap)
	if !m.match(i, pos) {
		re.put(m)
		return nil
	}

	dstCap = append(dstCap, m.matchcap...)
	re.put(m)
	return dstCap
}

Read the order:

text
   re.find(reader, bytes, str)
              |
              v
   +-----------------------------+
   | len(input) < minInputLen ?  |--yes--> no match (too short)
   +-----------------------------+
              |
              v
   +-----------------------------+
   | re.onepass != nil ?         |--yes--> ONE-PASS ENGINE
   | (anchored + unambiguous)    |          single pc, no state, no alloc
   +-----------------------------+
              |
              v
   +-----------------------------+
   | len(input) < maxBitStateLen |--yes--> BACKTRACK ENGINE
   | ? (and not a RuneReader)    |          job stack + visited bitmap
   +-----------------------------+
              |
              v
            NFA MACHINE
         sparse queues, dedup
         by instruction pointer

Too short to match at all? minInputLen is the minimum number of bytes any match could consume (computed at compile time by walking the syntax tree). A three-byte input is rejected before any engine runs.
One-pass program available? Use it. Always, regardless of input size.
Input small enough for the backtracker? Use the backtracker.
Otherwise: the NFA machine.

The r == nil conditions matter: an io.RuneReader input has unknown length, so the size-based heuristics cannot be applied and the NFA is always used.

Engine three: the parallel NFA

The general engine is a Thompson NFA simulation with a twist: instead of keeping one state per thread, it keeps a set of threads, deduplicated by instruction pointer. The dedup is what makes it linear.

go
// src/regexp/exec.go:13
// A queue is a 'sparse array' holding pending threads of execution.
// See https://research.swtch.com/2008/03/using-uninitialized-memory-for-fun-and.html
type queue struct {
	sparse []uint32
	dense  []entry
}

// An entry is an entry on a queue.
// It holds both the instruction pc and the actual thread.
// Some queue entries are just place holders so that the machine
// knows it has considered that pc. Such entries have t == nil.
type entry struct {
	pc uint32
	t  *thread
}

// A thread is the state of a single path through the machine:
// an instruction and a corresponding capture array.
// See https://swtch.com/~rsc/regexp/regexp2.html
type thread struct {
	inst *syntax.Inst
	cap  []int
}

// A machine holds all the state during an NFA simulation for p.
type machine struct {
	re       *Regexp      // corresponding Regexp
	p        *syntax.Prog // compiled program
	q0, q1   queue        // two queues for runq, nextq
	pool     []*thread    // pool of available threads
	matched  bool         // whether a match was found
	matchcap []int        // capture information for the match

	inputs inputs
}

The sparse/dense pair is a sparse set: sparse[pc] is the index into dense where instruction pc lives (or an old index, meaning “not present”). Adding a thread for an instruction that is already in the queue is a no-op:

go
// src/regexp/exec.go:313
// add adds an entry to q for pc, unless the q already has such an entry.
// It also recursively adds an entry for all instructions reachable from pc by following
// empty-width conditions satisfied by cond.  pos gives the current position
// in the input.
func (m *machine) add(q *queue, pc uint32, pos int, cap []int, cond *lazyFlag, t *thread) *thread {
Again:
	if pc == 0 {
		return t
	}
	if j := q.sparse[pc]; j < uint32(len(q.dense)) && q.dense[j].pc == pc {
		return t
	}

	j := len(q.dense)
	q.dense = q.dense[:j+1]
	d := &q.dense[j]
	d.t = nil
	d.pc = pc
	q.sparse[pc] = uint32(j)

	i := &m.p.Inst[pc]
	switch i.Op {
	default:
		panic("unhandled")
	case syntax.InstFail:
		// nothing
	case syntax.InstAlt, syntax.InstAltMatch:
		t = m.add(q, i.Out, pos, cap, cond, t)
		pc = i.Arg
		goto Again
	case syntax.InstEmptyWidth:
		if cond.match(syntax.EmptyOp(i.Arg)) {
			pc = i.Out
			goto Again
		}
	case syntax.InstNop:
		pc = i.Out
		goto Again
	case syntax.InstCapture:
		if int(i.Arg) < len(cap) {
			opos := cap[i.Arg]
			cap[i.Arg] = pos
			m.add(q, i.Out, pos, cap, cond, nil)
			cap[i.Arg] = opos
		} else {
			pc = i.Out
			goto Again
		}
	case syntax.InstMatch, syntax.InstRune, syntax.InstRune1, syntax.InstRuneAny, syntax.InstRuneAnyNotNL:
		if t == nil {
			t = m.alloc(i)
		} else {
			t.inst = i
		}
		if len(cap) > 0 && &t.cap[0] != &cap[0] {
			copy(t.cap, cap)
		}
		d.t = t
		t = nil
	}
	return t
}

Because a given instruction can only be in the queue once per step, the queue is bounded by the program size. The main loop advances one rune at a time and swaps two queues, so the work per input byte is O(program size), which is O(1) with respect to the input length. That is where the linear time guarantee comes from.

The main loop:

go
// src/regexp/exec.go:197
	for {
		if len(runq.dense) == 0 {
			if startCond&syntax.EmptyBeginText != 0 && pos != 0 {
				// Anchored match, past beginning of text.
				break
			}
			if m.matched {
				// Have match; finished exploring alternatives.
				break
			}
			if len(m.re.prefix) > 0 && r1 != m.re.prefixRune && i.canCheckPrefix() {
				// Match requires literal prefix; fast search for it.
				advance := i.index(m.re, pos)
				if advance < 0 {
					break
				}
				pos += advance
				r, width = i.step(pos)
				r1, width1 = i.step(pos + width)
			}
		}

That block is the literal prefix optimisation. If the pattern requires a fixed literal at the start of a match — foo[0-9]+ requires foo — the engine does not walk the input one rune at a time looking for it. It uses strings.Index (which, as we saw in the previous article, is SIMD assembly) to jump straight to the next place the literal could occur. For a pattern like ^\d{6}-\d{6} searching a 10 MB log file, this is the difference between scanning 10 MB and scanning 10 MB with a vectorised memchr.

i.canCheckPrefix() is false for io.RuneReader inputs, and i.index returns -1 when the literal is not found, ending the search.

Zero-width conditions (^, $, \b) are handled lazily:

go
// src/regexp/exec.go:117
// A lazyFlag is a lazily-evaluated syntax.EmptyOp,
// for checking zero-width flags like ^ $ \A \z \B \b.
// It records the pair of relevant runes and does not
// determine the implied flags until absolutely necessary
// (most of the time, that means never).
type lazyFlag uint64

func newLazyFlag(r1, r2 rune) lazyFlag {
	return lazyFlag(uint64(r1)<<32 | uint64(uint32(r2)))
}

Instead of computing a bitmask of which assertions hold at every position, it stores the two runes around the position and evaluates the assertion only if some instruction actually asks. For most patterns, \b and friends are never asked about.

Engine one: the one-pass engine

The one-pass engine exists for patterns where the engine can always tell which way to go — no backtracking, no state to save, one instruction pointer.

The gate is in compileOnePass:

go
// src/regexp/onepass.go:455
// compileOnePass returns a new *syntax.Prog suitable for onePass execution if the original Prog
// can be recharacterized as a one-pass regexp program, or syntax.nil if the
// Prog cannot be converted. For a one pass prog, the fundamental condition that must
// be true is: at any InstAlt, there must be no ambiguity about what branch to  take.
func compileOnePass(prog *syntax.Prog) (p *onePassProg) {
	if prog.Start == 0 {
		return nil
	}
	// onepass regexp is anchored
	if prog.Inst[prog.Start].Op != syntax.InstEmptyWidth ||
		syntax.EmptyOp(prog.Inst[prog.Start].Arg)&syntax.EmptyBeginText != syntax.EmptyBeginText {
		return nil
	}
	hasAlt := false
	for _, inst := range prog.Inst {
		if inst.Op == syntax.InstAlt || inst.Op == syntax.InstAltMatch {
			hasAlt = true
			break
		}
	}
	// If we have alternates, every instruction leading to InstMatch must be EmptyEndText.
	// Also, any match on empty text must be $.
	for _, inst := range prog.Inst {
		opOut := prog.Inst[inst.Out].Op
		switch inst.Op {
		default:
			if opOut == syntax.InstMatch && hasAlt {
				return nil
			}
		case syntax.InstAlt, syntax.InstAltMatch:
			if opOut == syntax.InstMatch || prog.Inst[inst.Arg].Op == syntax.InstMatch {
				return nil
			}
		case syntax.InstEmptyWidth:
			if opOut == syntax.InstMatch {
				if syntax.EmptyOp(inst.Arg)&syntax.EmptyEndText == syntax.EmptyEndText {
					continue
				}
				return nil
			}
		}
	}
	// Creates a slightly optimized copy of the original Prog
	// that cleans up some Prog idioms that block valid onepass programs
	p = onePassCopy(prog)

	// checkAmbiguity on InstAlts, build onepass Prog if possible
	p = makeOnePass(p)

	if p != nil {
		cleanupOnePass(p, prog)
	}
	return p
}

Three requirements, all of them necessary:

The program must be anchored (\A, i.e. ^ in non-multiline mode). An unanchored pattern has to be tried at every position in the input, so it is not “one pass” over the input at all.
No alternation may lead directly to a match, because then the engine would have to try one branch and, on failure, go back.
No ambiguity at any alternation: the runes accepted by the two branches must be disjoint.

makeOnePass is where the disjointness is actually proven. It walks the program and, at every InstAlt, merges the rune sets of the two branches:

go
// src/regexp/onepass.go:288
// makeOnePass creates a onepass Prog, if possible. It is possible if at any alt,
// the match engine can always tell which branch to take. The routine may modify
// p if it is turned into a onepass Prog. If it isn't possible for this to be a
// onepass Prog, the Prog nil is returned. makeOnePass is recursive
// to the size of the Prog.
func makeOnePass(p *onePassProg) *onePassProg {
	// If the machine is very long, it's not worth the time to check if we can use one pass.
	if len(p.Inst) >= 1000 {
		return nil
	}

go
// src/regexp/onepass.go:316
		case syntax.InstAlt, syntax.InstAltMatch:
			ok = check(inst.Out, m) && check(inst.Arg, m)
			// check no-input paths to InstMatch
			matchOut := m[inst.Out]
			matchArg := m[inst.Arg]
			if matchOut && matchArg {
				ok = false
				break
			}
			// Match on empty goes in inst.Out
			if matchArg {
				inst.Out, inst.Arg = inst.Arg, inst.Out
				matchOut, matchArg = matchArg, matchOut
			}
			if matchOut {
				m[pc] = true
				inst.Op = syntax.InstAltMatch
			}

			// build a dispatch operator from the two legs of the alt.
			onePassRunes[pc], inst.Next = mergeRuneSets(
				&onePassRunes[inst.Out], &onePassRunes[inst.Arg], inst.Out, inst.Arg)
			if len(inst.Next) > 0 && inst.Next[0] == mergeFailed {
				ok = false
				break
			}

mergeRuneSets merges two sorted, non-overlapping rune-pair lists and builds a parallel array of target program counters. If the two lists overlap, it returns mergeFailed and the pattern is rejected:

go
// src/regexp/onepass.go:146
// mergeRuneSets merges two non-intersecting runesets, and returns the merged result,
// and a NextIp array. The idea is that if a rune matches the OnePassRunes at index
// i, NextIp[i/2] is the target. If the input sets intersect, an empty runeset and a
// NextIp array with the single element mergeFailed is returned.
// The code assumes that both inputs contain ordered and non-intersecting rune pairs.
const mergeFailed = uint32(0xffffffff)

The result is a dispatch table. At run time the engine peeks at the next rune and jumps:

go
// src/regexp/onepass.go:71
// onePassNext selects the next actionable state of the prog, based on the input character.
// It should only be called when i.Op == InstAlt or InstAltMatch, and from the one-pass machine.
// One of the alternates may ultimately lead without input to end of line. If the instruction
// is InstAltMatch the path to the InstMatch is in i.Out, the normal node in i.Next.
func onePassNext(i *onePassInst, r rune) uint32 {
	next := i.MatchRunePos(r)
	if next >= 0 {
		return i.Next[next]
	}
	if i.Op == syntax.InstAltMatch {
		return i.Out
	}
	return 0
}

and the engine loop is a plain for with a switch — no queues, no backtracking, no saved state:

go
// src/regexp/exec.go:471
		// peek at the input rune to see which branch of the Alt to take
		case syntax.InstAlt, syntax.InstAltMatch:
			pc = int(onePassNext(inst, r))
			continue

The one-pass engine is by far the fastest of the three. It has no allocation, no thread pool, no queue — just an instruction pointer walking a program and a capture array.

Note also that the one-pass engine gets its own literal prefix handling (onePassPrefix), including the prefixEnd program counter so it can skip the prefix bytes entirely.

Engine two: the bit-state backtracker

The backtracker is a genuine backtracking search — it keeps a stack of jobs — but it is not allowed to revisit a state. That is what keeps it linear.

go
// src/regexp/backtrack.go:5
// backtrack is a regular expression search with submatch
// tracking for small regular expressions and texts. It allocates
// a bit vector with (length of input) * (length of prog) bits,
// to make sure it never explores the same (character position, instruction)
// state multiple times. This limits the search to run in time linear in
// the length of the test.
//
// backtrack is a fast replacement for the NFA code on small
// regexps when onepass cannot be used.

The state is a bitmap:

go
// src/regexp/backtrack.go:30
const (
	visitedBits        = 32
	maxBacktrackProg   = 500        // len(prog.Inst) <= max
	maxBacktrackVector = 256 * 1024 // bit vector size <= max (bits)
)

// bitState holds state for the backtracker.
type bitState struct {
	end      int
	cap      []int
	matchcap []int
	jobs     []job
	visited  []uint32

	inputs inputs
}

and the check is one bit test:

go
// src/regexp/backtrack.go:116
// shouldVisit reports whether the combination of (pc, pos) has not
// been visited yet.
func (b *bitState) shouldVisit(pc uint32, pos int) bool {
	n := uint(int(pc)*(b.end+1) + pos)
	if b.visited[n/visitedBits]&(1<<(n&(visitedBits-1))) != 0 {
		return false
	}
	b.visited[n/visitedBits] |= 1 << (n & (visitedBits - 1))
	return true
}

The index pc*(end+1) + pos packs the (instruction, position) pair into a single integer, and the bit is set when visited. If the pair has been seen before, the search abandons that branch — which is exactly what turns catastrophic backtracking into “we already know this fails”.

The size limits follow from the bitmap size:

go
// src/regexp/backtrack.go:62
// maxBitStateLen returns the maximum length of a string to search with
// the backtracker using prog.
func maxBitStateLen(prog *syntax.Prog) int {
	if !shouldBacktrack(prog) {
		return 0
	}
	return maxBacktrackVector / len(prog.Inst)
}

// shouldBacktrack reports whether the program is too
// long for the backtracker to run.
func shouldBacktrack(prog *syntax.Prog) bool {
	return len(prog.Inst) <= maxBacktrackProg
}

The bitmap needs len(prog.Inst) * (end+1) bits, and the budget is 256 * 1024 bits (32 KiB). So the maximum searchable input length is 256*1024 / len(prog.Inst). A ten-instruction program can backtrack over 26 000 bytes; a 500-instruction program can only manage 524 bytes. The budget is the product of the two, and that is the entire reason there is a size check at all.

The bitmap is allocated once and reused, with the pool keeping it warm:

go
// src/regexp/backtrack.go:77
// reset resets the state of the backtracker.
// end is the end position in the input.
// ncap is the number of captures.
func (b *bitState) reset(prog *syntax.Prog, end int, ncap int) {
	b.end = end

	if cap(b.jobs) == 0 {
		b.jobs = make([]job, 0, 256)
	} else {
		b.jobs = b.jobs[:0]
	}

	visitedSize := (len(prog.Inst)*(end+1) + visitedBits - 1) / visitedBits
	if cap(b.visited) < visitedSize {
		b.visited = make([]uint32, visitedSize, maxBacktrackVector/visitedBits)
	} else {
		b.visited = b.visited[:visitedSize]
		clear(b.visited) // set to 0
	}

The search itself is a job stack. tryBacktrack pops a job, executes one instruction, and pushes the continuations:

go
// src/regexp/backtrack.go:137
// tryBacktrack runs a backtracking search starting at pos.
func (re *Regexp) tryBacktrack(b *bitState, i input, pc uint32, pos int) bool {
	longest := re.longest

	b.push(re, pc, pos, false)
	for len(b.jobs) > 0 {
		l := len(b.jobs) - 1
		// Pop job off the stack.
		pc := b.jobs[l].pc
		pos := b.jobs[l].pos
		arg := b.jobs[l].arg
		b.jobs = b.jobs[:l]

and the arg flag is a neat trick for handling alternation without pushing two jobs:

go
// src/regexp/backtrack.go:171
		case syntax.InstAlt:
			// Cannot just
			//   b.push(inst.Out, pos, false)
			//   b.push(inst.Arg, pos, false)
			// If during the processing of inst.Out, we encounter
			// inst.Arg via another path, we want to process it then.
			// Pushing it here will inhibit that. Instead, re-push
			// inst with arg==true as a reminder to push inst.Arg out
			// later.
			if arg {
				// Finished inst.Out; try inst.Arg.
				arg = false
				pc = inst.Arg
				goto CheckAndLoop
			} else {
				b.push(re, pc, pos, true)
				pc = inst.Out
				goto CheckAndLoop
			}

The unanchored case has an important comment about why it is not actually quadratic:

go
// src/regexp/backtrack.go:328
	} else {

		// Unanchored search, starting from each possible text position.
		// Notice that we have to try the empty string at the end of
		// the text, so the loop condition is pos <= end, not pos < end.
		// This looks like it's quadratic in the size of the text,
		// but we are not clearing visited between calls to TrySearch,
		// so no work is duplicated and it ends up still being linear.
		width := -1
		for ; pos <= end && width != 0; pos += width {
			if len(re.prefix) > 0 {
				// Match requires literal prefix; fast search for it.
				advance := i.index(re, pos)
				if advance < 0 {
					freeBitState(b)
					return nil
				}
				pos += advance
			}

The visited bitmap is not cleared between start positions, so work done at position 5 is not redone at position 6. That is what keeps the outer loop linear despite looking like a nested loop.

The program

All three engines run the same instruction set, defined in regexp/syntax:

go
// src/regexp/syntax/prog.go:114
// An Inst is a single instruction in a regular expression program.
type Inst struct {
	Op   InstOp
	Out  uint32 // all but InstMatch, InstFail
	Arg  uint32 // InstAlt, InstAltMatch, InstCapture, InstEmptyWidth
	Rune []rune
}

go
// src/regexp/syntax/prog.go:27
const (
	InstAlt InstOp = iota
	InstAltMatch
	InstCapture
	InstEmptyWidth
	InstMatch
	InstFail
	InstNop
	InstRune
	InstRune1
	InstRuneAny
	InstRuneAnyNotNL
)

A Prog is a flat slice of instructions with a Start index. The Rune field holds the character class for a rune-matching instruction, and InstRune1/InstRuneAny/InstRuneAnyNotNL are the common special cases that get their own opcodes so the hot loop does not have to look at the Rune slice at all.

The literal prefix comes from the same package:

go
// src/regexp/syntax/prog.go:147
// Prefix returns a literal string that all matches for the
// regexp must start with. Complete is true if the prefix
// is the entire match.
func (p *Prog) Prefix() (prefix string, complete bool) {
	i := p.skipNop(uint32(p.Start))

	// Avoid allocation of buffer if prefix is empty.
	if i.op() != InstRune || len(i.Rune) != 1 {
		return "", i.Op == InstMatch
	}

	// Have prefix; gather characters.
	var buf strings.Builder
	for i.op() == InstRune && len(i.Rune) == 1 && Flags(i.Arg)&FoldCase == 0 && i.Rune[0] != utf8.RuneError {
		buf.WriteRune(i.Rune[0])
		i = p.skipNop(i.Out)
	}
	return buf.String(), i.Op == InstMatch
}

Only single, case-sensitive, non-RuneError runes count, so (?i)hello has no usable prefix — the case folding means the literal in the input could be HELLO.

Trade-offs

What the three-engine design buys: the common cases are fast. An anchored literal pattern runs on the one-pass engine with no allocation at all. A pattern over a small buffer runs on the backtracker, which avoids the NFA’s queue bookkeeping. A pattern over a large buffer runs on the NFA, which never allocates per match.

What it costs:

Compiling is expensive. syntax.Parse, Simplify, Compile and makeOnePass all run at regexp.Compile time. Compiling a regex in a hot loop is a disaster; compile once, reuse the *Regexp.
The one-pass analysis can reject a pattern that looks simple. ^(a|ab)c has overlapping branches and is rejected.
The one-pass engine only handles anchored patterns. ^abc is one-pass; abc is not, because an unanchored pattern has to be tried at every position.
The backtracker is limited by a 32 KiB bitmap. Long programs get a tiny input budget, and past it you drop to the NFA.
The NFA allocates queues sized by the program. A 5000-instruction pattern means big queues, allocated from a pool on first use.

Caveats

Regexp.Longest() changes the semantics and cannot be called concurrently. It flips the engine from leftmost-first to leftmost-longest matching, which changes which match you get and disables the “cut off lower-priority threads” optimisation in the NFA step.

regexp is safe for concurrent use; Regexp.Copy is deprecated. The copy existed to avoid lock contention in Go 1.11 and earlier; since Go 1.12 the machine pool handles that.

The m.matchcap slice is reused between calls. If you hold on to the result of FindStringSubmatchIndex it is fine — the result is copied out — but the internal capture array is pooled.

An io.RuneReader input always uses the NFA. All the size-based heuristics are disabled, because the length is not known in advance.

Leftmost-first, not leftmost-longest, unless you ask. Go’s default semantics match what Perl and Python do. CompilePOSIX gives POSIX leftmost-longest, but as the doc notes, it “chooses the one that a backtracking search would have found first” — it is not the full POSIX rule, which is “computationally prohibitive and not even well-defined”.

The linear-time guarantee is about the input, not the pattern. A pattern with 100 000 instructions is still O(n · 100 000). The guarantee protects you from user-supplied patterns over user-supplied data, which is the case that matters.

Wrapping up

The one thing to remember: regexp.Compile decides once, at compile time, which of three engines will run your pattern — one-pass, bit-state backtracker, or parallel NFA — and the choice is based on the shape of the pattern (anchored? unambiguous alternations? literal prefix?) and the size of the input.

The part people get wrong: assuming Go’s regexp is a backtracking engine and therefore slow. It is not; it is an NFA with a bit-state backtracker bolted on for small inputs. The catastrophic backtracking you get in Python and Ruby does not happen, and the reason is the visited bitmap — every (instruction, position) pair is explored at most once.

What to go do about it: compile your patterns once at startup and keep the *Regexp; anchor patterns with ^ where you can, because anchoring is what unlocks the one-pass engine; and if you are matching a literal prefix followed by structure, write it so the prefix is extractable — that is what turns an O(n) scan into an O(n) SIMD memchr.