range over func: The Compiler Turns a Loop Into a Coroutine

TL;DR: for x := range f never becomes a loop in the machine code. A compiler pass rewrites it into f(func(x T) bool { ... }) – the iterator calls your body, and your body returns true to keep going. There is no coroutine in that path at all. The coroutine shows up in iter.Pull, which inverts the direction of control and needs a way to suspend a caller in the middle of a function. In this tree that coroutine is a goroutine, switched with gogo rather than by copying stacks.

Go 1.23 added range over a function. It reads like a loop, but it is nothing of the kind, and the difference shows up in profiles, in stack traces, and in what a defer inside the body actually does.

This is a walk through how it works, based on reading the Go 1.28 development tree: the compiler pass in src/cmd/compile/internal/rangefunc/rewrite.go, the runtime coroutine machinery in src/runtime/coro.go, and the standard library in src/iter/iter.go.

The shape of the feature

An iterator is a function that takes a yield callback:

go
// src/iter/iter.go:219
// Seq is an iterator over sequences of individual values.
// When called as seq(yield), seq calls yield(v) for each value v in the sequence,
// stopping early if yield returns false.
// See the [iter] package documentation for more details.
type Seq[V any] func(yield func(V) bool)

// Seq2 is an iterator over sequences of pairs of values, most commonly key-value pairs.
// When called as seq(yield), seq calls yield(k, v) for each pair (k, v) in the sequence,
// stopping early if yield returns false.
// See the [iter] package documentation for more details.
type Seq2[K, V any] func(yield func(K, V) bool)

That is the whole interface. The iterator pushes values at the callback; the callback returns false to say “stop”. The iter package documents the contract from the iterator’s side:

Yield panics if called after it returns false.

So an iterator is not allowed to keep calling yield after the loop body has said stop. That is a promise the compiler cannot check statically, so it checks at runtime.

The rewrite pass

The whole feature is implemented by a front-end pass that runs before the compiler’s normal IR generation. Its package comment is the best description of the design in the tree:

go
// src/cmd/compile/internal/rangefunc/rewrite.go:10
// # Theory of Operation
//
// The basic idea is to rewrite
//
// 	for x := range f {
// 		...
// 	}
//
// into
//
// 	f(func(x T) bool {
// 		...
// 	})
//
// But it's not usually that easy.

The “not usually that easy” is the rest of the file, and it is worth reading because every piece of it corresponds to a rule you already know from Go syntax: break becomes return false, continue becomes return true, a return from inside the body has to unwind through the iterator, and a labelled break out of two nested loops needs a numeric code to say which loop it meant.

The generated code uses three hidden variables, all named with a # prefix so they show up as internal in a debugger:

#yield1, #yield2, … – the generated body closures.
#next – an integer code saying what to do when the iterator returns.
#state1, #state2, … – the iterator-misbehaviour state machine.

A plain loop body becomes a closure that returns true at the end:

go
// src/cmd/compile/internal/rangefunc/rewrite.go:176
// 	for x := range f {
// 		...
// 		if ... { break }
// 		...
// 	}
//
// becomes
//
// 		{
// 			var #state1 = abi.RF_READY
// 			f(func(x T1) bool {
// 				if #state1 != abi.RF_READY { #state1 = abi.RF_PANIC; runtime.panicrangestate(#state1) }
// 				#state1 = abi.RF_PANIC
// 				...
// 				if ... { #state1 = abi.RF_DONE ; return false }
// 				...
// 				#state1 = abi.RF_READY
// 				return true
// 			})
// 	        if #state1 == abi.RF_PANIC {
// 	        	// the code for the loop body did not return normally
// 	        	panic(runtime.panicrangestate(abi.RF_MISSING_PANIC))
// 	        }
// 			#state1 = abi.RF_EXHAUSTED
// 		}

Note the shape of the result: the loop is gone. What is left is a call to the iterator with a closure, followed by some checks.

The compiler builds exactly that. endLoop constructs the block, the closure declaration, the call, and the post-checks:

go
// src/cmd/compile/internal/rangefunc/rewrite.go:1127
	// Give the closure generated for the body a name, to help the debugger connect it to its frame, if active.
	r.bodyClosureCount++
	clo := r.bodyFunc(nfor.Body.List, syntax.UnpackListExpr(rclause.Lhs), rclause.Def, ftyp, start, end)
	cloDecl, cloVar := r.declSingleVar(fmt.Sprintf("#yield%d", r.bodyClosureCount), clo.GetTypeInfo().Type, clo)
	setPos(cloDecl, start)

	// Build X(bodyFunc)
	call := &syntax.ExprStmt{
		X: &syntax.CallExpr{
			Fun: rclause.X,
			ArgList: []syntax.Expr{
				r.useObj(cloVar),
			},
		},
	}
	setPos(call, start)

and the closure itself is a plain function literal whose last statement is return true:

go
// src/cmd/compile/internal/rangefunc/rewrite.go:1298
	// Original loop body (already rewritten by editStmt during inspect).
	bodyFunc.Body.List = append(bodyFunc.Body.List, body...)

	// end of loop body, set state to abi.RF_READY and return true to continue iteration
	if r.checkFuncMisuse() {
		bodyFunc.Body.List = append(bodyFunc.Body.List, r.setState(abi.RF_READY, end))
	}
	ret := &syntax.ReturnStmt{Results: r.useObj(r.true)}
	ret.SetPos(end)
	bodyFunc.Body.List = append(bodyFunc.Body.List, ret)

	return bodyFunc
}

So for x := range f compiles to a call, and your loop body compiles to a closure. That is why the loop body shows up in stack traces as a separate frame with a name like f.func1, and why runtime.Caller inside the body reports the iterator’s frame, not the caller’s.

Checking that the iterator behaves

The #stateN variables exist to catch an iterator that calls yield at the wrong time. The states are shared between the compiler and the runtime, and they live in internal/abi:

go
// src/internal/abi/rangefuncconsts.go:7
type RF_State int

// These constants are shared between the compiler, which uses them for state functions
// and panic indicators, and the runtime, which turns them into more meaningful strings
// For best code generation, RF_DONE and RF_READY should be 0 and 1.
const (
	RF_DONE          = RF_State(iota) // body of loop has exited in a non-panic way
	RF_READY                          // body of loop has not exited yet, is not running  -- this is not a panic index
	RF_PANIC                          // body of loop is either currently running, or has panicked
	RF_EXHAUSTED                      // iterator function return, i.e., sequence is "exhausted"
	RF_MISSING_PANIC = 4              // body of loop panicked but iterator function defer-recovered it away
)

The runtime side is a single function that panics with the right message:

go
// src/runtime/panic.go:371
var rangeDoneError = error(errorString("range function continued iteration after function for loop body returned false"))
var rangePanicError = error(errorString("range function continued iteration after loop body panic"))
var rangeExhaustedError = error(errorString("range function continued iteration after whole loop exit"))
var rangeMissingPanicError = error(errorString("range function recovered a loop body panic and did not resume panicking"))

//go:noinline
func panicrangestate(state int) {
	switch abi.RF_State(state) {
	case abi.RF_DONE:
		panic(rangeDoneError)
	case abi.RF_PANIC:
		panic(rangePanicError)
	case abi.RF_EXHAUSTED:
		panic(rangeExhaustedError)
	case abi.RF_MISSING_PANIC:
		panic(rangeMissingPanicError)
	}
	throw("unexpected state passed to panicrangestate")
}

RF_MISSING_PANIC is the interesting one. If the loop body panics and the iterator swallows the panic with its own recover, the body never gets to return normally and the loop never sees RF_PANIC cleared. The compiler checks for that after the iterator returns and reports it, rather than letting a panic vanish.

`defer` inside the loop body

A defer in a range-over-func body is a problem, because the body is a closure: the deferred call would run when the closure returns, i.e. at the end of each iteration, rather than at the end of the enclosing function.

The rewrite pass solves this with a token. The package comment spells out the protocol:

go
// src/cmd/compile/internal/rangefunc/rewrite.go:480
// # Defers
//
// The last wrinkle is handling defer statements. If we have
//
// 	for range f {
// 		defer print("A")
// 	}
//
// we cannot rewrite that into
//
// 	f(func() {
// 		defer print("A")
// 	})
//
// because the deferred code will run at the end of the iteration, not
// the end of the containing function. To fix that, the runtime provides
// a special hook that lets us obtain a defer "token" representing the
// outer function and then use it in a later defer to attach the deferred
// code to that outer function.
//
// Normally,
//
// 	defer print("A")
//
// compiles to
//
// 	runtime.deferproc(func() { print("A") })
//
// This changes in a range-over-func. For example:
//
// 	for range f {
// 		defer print("A")
// 	}
//
// compiles to
//
// 	var #defers = runtime.deferrangefunc()
// 	f(func() {
// 		runtime.deferprocat(func() { print("A") }, #defers)
// 	})

The token is an any holding a pointer to an atomic list head. The runtime side keeps two lists: the goroutine’s ordinary defer chain, and a separate atomic chain for the body closure’s defers, so that the body can push onto it from a different goroutine without disturbing the main chain:

go
// src/runtime/panic.go:405
// The g._defer list is now a linked list of deferred calls,
// but an atomic list hanging off:
//
//		g._defer => d4 -> d3 -> drangefunc -> d2 -> d1 -> nil
//	                             | .head
//	                             |
//	                             +--> dY -> dX -> nil
//
// with each -> indicating a d.link pointer, and where drangefunc
// has the d.rangefunc = true bit set.
// Note that the function being ranged over may have added
// its own defers (d4 and d3), so drangefunc need not be at the
// top of the list when deferprocat is used. This is why we pass
// the atomic head explicitly.

When the runtime reaches the drangefunc marker during unwinding, it calls deferconvert, which atomically steals the extra list and splices it into the main chain in the right order:

go
// src/runtime/panic.go:505
// deferconvert converts the rangefunc defer list of d0 into an ordinary list
// following d0.
// See the doc comment for deferrangefunc for details.
func deferconvert(d0 *_defer) {
	head := d0.head
	if raceenabled {
		racereadpc(unsafe.Pointer(head), sys.GetCallerPC(), abi.FuncPCABIInternal(deferconvert))
	}
	tail := d0.link
	d0.rangefunc = false

	var d *_defer
	for {
		d = head.Load()
		if head.CompareAndSwap(d, badDefer()) {
			break
		}
	}
	if d == nil {
		return
	}
	for d1 := d; ; d1 = d1.link {
		d1.sp = d0.sp
		d1.pc = d0.pc
		if d1.link == nil {
			d1.link = tail
			break
		}
	}
	d0.link = d
	return
}

d1.sp = d0.sp and d1.pc = d0.pc are what make the defers belong to the outer function’s frame: the runtime decides which deferred calls run in which frame by comparing stack pointers.

There is a catch, and the runtime enforces it: the token is poisoned once the loop is over, so a body that calls yield after the iterator returned and then defers something gets a hard error rather than silent corruption:

go
// src/runtime/panic.go:480
// badDefer returns a fixed bad defer pointer for poisoning an atomic defer list head.
func badDefer() *_defer {
	return (*_defer)(unsafe.Pointer(uintptr(1)))
}

`recover` inside the loop body

recover has the same problem for the same reason: it decides whether to recover by looking at the frame it was called from, and the body’s frame is not the frame of the function that wrote the loop.

The rewrite pass handles it with a second token. The compiler rewrites recover() in a body into runtime.gorecoverat(#recover), where #recover was computed once, on entry to the enclosing function:

go
// src/cmd/compile/internal/rangefunc/rewrite.go:544
// Whether such a recover takes effect depends only on the frame the loop is
// in, so the runtime can decide that part on entry to the function, while
// that frame is the current one, and hand back the panic the body would
// recover:
//
// 	var #recover = runtime.recoverrangefunc()
// 	f(func() {
// 		runtime.gorecoverat(#recover)
// 	})
//
// The token is the panic itself, or nil if a recover in the body would not
// take effect. Unlike the frame it stands for, it does not move when the
// stack grows, so the body can hold onto it. gorecoverat recovers that panic
// if it is still the one being processed.

And the runtime side:

go
// src/runtime/panic.go:1094
// recoverrangefunc is called on entry to a range-over-func loop whose body
// calls recover. The body runs in a frame of its own, called from the iterator,
// so a recover there cannot tell that it is, in the source, a call made directly
// in this function. This does the part of that decision that depends on the frame,
// while the frame is the current one, and returns the panic the body may recover,
// or nil if a recover in the body does not take effect.
//
// See also ../cmd/compile/internal/rangefunc/rewrite.go.
func recoverrangefunc() any {
	p := recoverablePanic(3) // skip systemstack_switch, recoverablePanic, recoverrangefunc
	if p == nil {
		// Either nothing is panicking, or this function is not the one the
		// panic deferred, so nothing in it can recover.
		return nil
	}
	return p
}

Note the recover() in the body is not the ordinary recover. The compiler swaps it out at the syntax level:

go
// src/cmd/compile/internal/rangefunc/rewrite.go:835
	// The call is in the body of a range-over-func loop, which runs in a
	// frame of its own. Hand the runtime the token for the frame of the
	// function the loop is written in, so that it can decide whether to
	// recover as if the call were made where it is in the source.
	x.Fun = runtimeSym(r.info, "gorecoverat")
	setPos(x.Fun, x.Pos())
	x.ArgList = []syntax.Expr{r.recoverToken()}
	setPos(x.ArgList[0], x.Pos())

Where the coroutine actually is

So far, nothing needs a coroutine: the iterator calls the body, the body returns a bool, control flows back to the iterator. That is a plain call, and it is cheap.

The coroutine appears when you want to invert the direction of control: pull values out of a push iterator one at a time. That is iter.Pull, and it is the only user of the runtime’s coroutine machinery.

go
// src/iter/iter.go:239
// Pull converts the “push-style” iterator sequence seq
// into a “pull-style” iterator accessed by the two functions
// next and stop.
//
// Next returns the next value in the sequence
// and a boolean indicating whether the value is valid.
// When the sequence is over, next returns the zero V and false.
// It is valid to call next after reaching the end of the sequence
// or after calling stop. These calls will continue
// to return the zero V and false.
//
// Stop ends the iteration. It must be called when the caller is
// no longer interested in next values and next has not yet
// signaled that the sequence is over (with a false boolean return).
// It is valid to call stop multiple times and when next has
// already returned false. Typically, callers should “defer stop()”.
//
// It is an error to call next or stop from multiple goroutines
// simultaneously.
//
// If the iterator panics during a call to next (or stop),
// then next (or stop) itself panics with the same value.
func Pull[V any](seq Seq[V]) (next func() (V, bool), stop func()) {

The trick is that next has to return a value from the middle of an arbitrary iterator function, and then next has to be callable again to resume it. That requires suspending the iterator’s stack mid-function and resuming it later. In this tree, the runtime does it with a goroutine.

The coro type is deliberately tiny:

go
// src/runtime/coro.go:12
// A coro represents extra concurrency without extra parallelism,
// as would be needed for a coroutine implementation.
// The coro does not represent a specific coroutine, only the ability
// to do coroutine-style control transfers.
// It can be thought of as like a special channel that always has
// a goroutine blocked on it. If another goroutine calls coroswitch(c),
// the caller becomes the goroutine blocked in c, and the goroutine
// formerly blocked in c starts running.
// These switches continue until a call to coroexit(c),
// which ends the use of the coro by releasing the blocked
// goroutine in c and exiting the current goroutine.
//
// Coros are heap allocated and garbage collected, so that user code
// can hold a pointer to a coro without causing potential dangling
// pointer errors.
type coro struct {
	gp guintptr
	f  func(*coro)

	// State for validating thread-lock interactions.
	mp        *m
	lockedExt uint32 // mp's external LockOSThread counter at coro creation time.
	lockedInt uint32 // mp's internal lockOSThread counter at coro creation time.
}

“Extra concurrency without extra parallelism” is the key phrase. A coro is a goroutine, but the switch does not go through the scheduler.

Creating one spawns a goroutine blocked in corostart:

go
// src/runtime/coro.go:39
// newcoro creates a new coro containing a
// goroutine blocked waiting to run f
// and returns that coro.
func newcoro(f func(*coro)) *coro {
	c := new(coro)
	c.f = f
	pc := sys.GetCallerPC()
	gp := getg()
	systemstack(func() {
		mp := gp.m
		start := corostart
		startfv := *(**funcval)(unsafe.Pointer(&start))
		gp = newproc1(startfv, gp, pc, true, waitReasonCoroutine)

		// Scribble down locked thread state if needed and/or donate
		// thread-lock state to the new goroutine.
		if mp.lockedExt+mp.lockedInt != 0 {
			c.mp = mp
			c.lockedExt = mp.lockedExt
			c.lockedInt = mp.lockedInt
		}
	})
	gp.coroarg = c
	c.gp.set(gp)
	return c
}

Switching is where the “no scheduler” promise is kept:

go
// src/runtime/coro.go:97
// coroswitch_m is the implementation of coroswitch
// that runs on the m stack.
//
// Note: Coroutine switches are expected to happen at
// an order of magnitude (or more) higher frequency
// than regular goroutine switches, so this path is heavily
// optimized to remove unnecessary work.
// The fast path here is three CAS: the one at the top on gp.atomicstatus,
// the one in the middle to choose the next g,
// and the one at the bottom on gnext.atomicstatus.
// It is important not to add more atomic operations or other
// expensive operations to the fast path.
func coroswitch_m(gp *g) {

and the final hand-off is a gogo, which is the runtime’s longjmp:

go
// src/runtime/coro.go:258
	// Switch to gnext. Does not return.
	gogo(&gnext.sched)
}

gogo is assembly that restores a saved stack pointer, frame pointer and program counter, and jumps:

asm
// src/runtime/asm_amd64.s:397
TEXT gogo<>(SB), NOSPLIT, $0
	get_tls(CX)
	MOVQ	DX, g(CX)
	MOVQ	DX, R14		// set the g register
	MOVQ	gobuf_sp(BX), SP	// restore SP
	MOVQ	gobuf_ctxt(BX), DX
	MOVQ	gobuf_bp(BX), BP
	MOVQ	$0, gobuf_sp(BX)	// clear to help garbage collector
	MOVQ	$0, gobuf_ctxt(BX)
	MOVQ	$0, gobuf_bp(BX)
	MOVQ	gobuf_pc(BX), BX
	JMP	BX

So the coroutine’s “stack” is an ordinary goroutine stack, allocated from the runtime’s stack pool with a minimum of 2 KiB:

go
// src/runtime/stack.go:77
	// The minimum size of stack used by Go code
	stackMin = 2048

and the coroutine’s state lives in the goroutine’s sched field, a gobuf:

go
// src/runtime/runtime2.go:303
type gobuf struct {
	// ctxt is unusual with respect to GC: it may be a
	// heap-allocated funcval, so GC needs to track it, but it
	// needs to be set and cleared from assembly, where it's
	// difficult to have write barriers. However, ctxt is really a
	// saved, live register, and we only ever exchange it between
	// the real register and the gobuf. Hence, we treat it as a
	// root during stack scanning, which means assembly that saves
	// and restores it doesn't need write barriers. It's still
	// typed as a pointer so that any other writes from Go get
	// write barriers.
	sp   uintptr
	pc   uintptr
	g    guintptr
	ctxt unsafe.Pointer
	lr   uintptr
	bp   uintptr // for framepointer-enabled architectures
}

iter.Pull ties it together. The yield closure the iterator calls switches back to the caller, and the caller’s next switches forward:

go
// src/iter/iter.go:271
	c := newcoro(func(c *coro) {
		race.Acquire(unsafe.Pointer(&pull.racer))
		if pull.done {
			race.Release(unsafe.Pointer(&pull.racer))
			return
		}
		yield := func(v1 V) bool {
			if pull.done {
				return false
			}
			if !pull.yieldNext {
				panic("iter.Pull: yield called again before next")
			}
			pull.yieldNext = false
			pull.v, pull.ok = v1, true
			race.Release(unsafe.Pointer(&pull.racer))
			coroswitch(c)
			race.Acquire(unsafe.Pointer(&pull.racer))
			return !pull.done
		}

The pull struct is shared between the two goroutines, and the racer field is how the race detector is told that the handoff is happening.

Trade-offs

What the rewrite buys: range over a function costs one indirect call per element, with no allocation, no scheduler involvement, and no state machine in the runtime. The generated closure is a normal Go function that the compiler can inline and optimise like any other.

What it costs: the loop body is no longer a loop body. It is a closure, so it cannot break out of an enclosing non-range loop without the #next code machinery, it cannot return values without assigning to named results first, and it shows up in stack traces as its own frame.

What the coroutine buys: iter.Pull can turn any push iterator into a pull iterator, including ones written in terms of yield inside a deeply nested function. The suspended iterator keeps its whole stack alive, so Pull works with arbitrary iterators.

What it costs: a goroutine per Pull, with at least a 2 KiB stack, plus the coro object, plus the pull struct. The switch is a hand-rolled gogo rather than a scheduler round trip, so it skips the scheduler’s work-stealing and preemption machinery – which is exactly why the runtime comments warn against adding work to that path.

Caveats

A range-over-func body is a closure, not a loop body. break and continue work, but goto out of it is rewritten into a numeric protocol, and return with values has to write to named results first.

recover inside the body is a different recover. The compiler rewrites it to runtime.gorecoverat, with a token captured when the loop was entered.

A defer inside the body runs at the end of the enclosing function, not at the end of the iteration. That is deliberate and it surprises people.

The body frame is visible in stack traces, and runtime.Caller from inside it reports the iterator’s caller, not your caller.

An iterator that keeps calling yield after the body returns false is a runtime panic, not a hang: panicrangestate turns the state mismatch into range function continued iteration after function for loop body returned false.

An iterator that swallows a panic from the body is reported, via RF_MISSING_PANIC. A recover in the iterator that hides a body panic is almost always a bug, and the compiler makes it loud.

iter.Pull is not free. Each call allocates a goroutine stack, a coro, and the shared state struct. If you are pulling thousands of values out of a sequence, pull the whole sequence into a slice instead.

iter.Pull is not safe for concurrent use. The documentation says it plainly: “It is an error to call next or stop from multiple goroutines simultaneously.”

The iterator must not retain the yield function and call it later. The runtime poisons the token when the loop ends, so a late defer in the body throws rather than corrupting memory.

Wrapping up

The one thing to remember: for x := range f becomes f(func(x T) bool { ... }). Your loop body is a closure that the iterator calls, and it returns true to continue. There is no loop in the machine code and no coroutine in the common path.

The part people get wrong: the coroutine is not part of range over func. It is part of iter.Pull, which inverts control and therefore needs to suspend a function mid-flight. In this tree that suspension is a goroutine plus a gogo, so pulling values costs a goroutine stack.

What to go do about it: if you write iterators, read the “Theory of Operation” comment at the top of src/cmd/compile/internal/rangefunc/rewrite.go – it is the closest thing to a specification of what the compiler will do with your code. And if you are pulling values one at a time out of a hot path, measure it: iter.Pull is convenient, not free.