How Go Copies Strings: slicebytetostring, concatstrings and the Builder Pattern

TL;DR: string(b) is not a language operation, it is a compiler intrinsic that calls runtime.slicebytetostring — which copies, unless the compiler can prove the bytes are temporary, in which case it calls slicebytetostringtmp and copies nothing. Concatenation goes through concatstrings, which allocates exactly once, and if the result does not escape it uses a 64-byte buffer on the stack. strings.Builder exists for the case where you are building a string in a loop and neither is fast enough.

A Go string is an immutable, two-word value: a pointer and a length. That is all a string is, and it is why strings are cheap to pass around and why the runtime has to be careful about when it copies.

This is a walk through the copying machinery, based on reading the Go 1.28 development tree.

The stack buffer

Two constants near the top of src/runtime/string.go set the scale for everything that follows:

go
// src/runtime/string.go:18
// These constants are known to the compiler (see cmd/compile/internal/walk).
// tmpStringBufSize bounds the stack buffer for string results (concatenation
// and []byte/[]rune->string). tmpRuneBufSize bounds the string->[]rune buffer;
// it is kept smaller because rune buffers cost 4 bytes per element while
// exceeding 32 runes is rare, so a larger size would grow frames for no gain.
const (
	tmpStringBufSize = 64
	tmpRuneBufSize   = 32
)

type tmpBuf [tmpStringBufSize]byte

Sixty-four bytes on the stack. If a string result fits in that, and the compiler can prove the result does not escape the current function, then no heap allocation happens at all. The compiler knows about the constant because it is the one emitting the call.

`string(b)` is a compiler intrinsic

The conversion string(b) where b is a []byte does not call a Go function you can find in the source. The compiler rewrites it:

go
// src/cmd/compile/internal/walk/convert.go:315
// walkBytesRunesToString walks an OBYTES2STR or ORUNES2STR node.
func (w *walkState) walkBytesRunesToString(n *ir.ConvExpr, init *ir.Nodes) ir.Node {
	a := typecheck.NodNil()
	if n.Esc() == ir.EscNone {
		// Create temporary buffer for string on stack.
		a = w.stackBufAddr(tmpstringbufsize, types.Types[types.TUINT8])
	}
	if n.Op() == ir.ORUNES2STR {
		// slicerunetostring(*[64]byte, []rune) string
		return w.mkcall("slicerunetostring", n.Type(), init, a, n.X)
	}
	// slicebytetostring(*[64]byte, ptr *byte, n int) string
	n.X = w.cheapExpr(n.X, init)
	ptr, len := w.backingArrayPtrLen(n.X)
	return w.mkcall("slicebytetostring", n.Type(), init, a, ptr, len)
}

Three things in there. The escape analysis result decides whether a stack buffer is passed at all (n.Esc() == ir.EscNone means “does not escape”). The bytes are addressed by pointer and length rather than by slice header, because the runtime function does not need the capacity. And the call target is slicebytetostring.

Here is that runtime function:

go
// src/runtime/string.go:140
// slicebytetostring converts a byte slice to a string.
// It is inserted by the compiler into generated code.
// ptr is a pointer to the first element of the slice;
// n is the length of the slice.
// Buf is a fixed-size buffer for the result,
// it is not nil if the result does not escape.
func slicebytetostring(buf *tmpBuf, ptr *byte, n int) string {
	if n == 0 {
		// Turns out to be a relatively common case.
		// Consider that you want to parse out data between parens in "foo()bar",
		// you find the indices and convert the subslice to string.
		return ""
	}
	if raceenabled {
		racereadrangepc(unsafe.Pointer(ptr),
			uintptr(n),
			sys.GetCallerPC(),
			abi.FuncPCABIInternal(slicebytetostring))
	}
	if msanenabled {
		msanread(unsafe.Pointer(ptr), uintptr(n))
	}
	if asanenabled {
		asanread(unsafe.Pointer(ptr), uintptr(n))
	}
	if n == 1 {
		p := unsafe.Pointer(&staticuint64s[*ptr])
		if goarch.BigEndian {
			p = add(p, 7)
		}
		return unsafe.String((*byte)(p), 1)
	}

	var p unsafe.Pointer
	if buf != nil && n <= len(buf) {
		p = unsafe.Pointer(buf)
	} else {
		p = mallocgc(uintptr(n), nil, false)
	}
	memmove(p, unsafe.Pointer(ptr), uintptr(n))
	return unsafe.String((*byte)(p), n)
}

The path is: empty string for zero length, a pointer into the runtime’s static byte table for a single byte (no allocation at all, ever), the stack buffer if the result fits and does not escape, and otherwise a heap allocation. Either way there is exactly one memmove — the copy is unavoidable, because strings are immutable and the caller still owns the byte slice.

The single-byte special case is a nice touch: string(b) where b has one byte returns a string pointing into staticuint64s, a global [256]uint64 where staticuint64s[i] == i. The first byte of entry b is therefore the byte you wanted, and the string can point straight at it. Zero allocations, ever.

The zero-copy path

Sometimes the copy is not needed at all. The runtime has a second entry point that simply reinterprets the bytes:

go
// src/runtime/string.go:201
// slicebytetostringtmp returns a "string" referring to the actual []byte bytes.
//
// Callers need to ensure that the returned string will not be used after
// the calling goroutine modifies the original slice or synchronizes with
// another goroutine.
//
// The function is only called when instrumenting
// and otherwise intrinsified by the compiler.
//
// Some internal compiler optimizations use this function.
//   - Used for m[T1{... Tn{..., string(k), ...} ...}] and m[string(k)]
//     where k is []byte, T1 to Tn is a nesting of struct and array literals.
//   - Used for "<"+string(b)+">" concatenation where b is []byte.
//   - Used for string(b)=="foo" comparison where b is []byte.
func slicebytetostringtmp(ptr *byte, n int) string {
	if raceenabled && n > 0 {
		racereadrangepc(unsafe.Pointer(ptr),
			uintptr(n),
			sys.GetCallerPC(),
			abi.FuncPCABIInternal(slicebytetostringtmp))
	}
	if msanenabled && n > 0 {
		msanread(unsafe.Pointer(ptr), uintptr(n))
	}
	if asanenabled && n > 0 {
		asanread(unsafe.Pointer(ptr), uintptr(n))
	}
	return unsafe.String(ptr, n)
}

The name says “tmp”: the resulting string is only valid while the underlying bytes are valid and unchanged. The compiler emits it in exactly the places where that is true.

The most visible of those places is a map lookup with a byte-slice key:

go
// src/cmd/compile/internal/walk/order.go:349
// mapKeyReplaceStrConv replaces OBYTES2STR by OBYTES2STRTMP
// in n to avoid string allocations for keys in map lookups.
// Returns a bool that signals if a modification was made.
//
// For:
//
//	x = m[string(k)]
//	x = m[T1{... Tn{..., string(k), ...}}]
//
// where k is []byte, T1 to Tn is a nesting of struct and array literals,
// the allocation of backing bytes for the string can be avoided
// by reusing the []byte backing array. These are special cases
// for avoiding allocations when converting byte slices to strings.
// It would be nice to handle these generally, but because
// []byte keys are not allowed in maps, the use of string(k)
// comes up in important cases in practice. See issue 3512.
//
// Note that this code does not handle the case:
//
//	s := string(k)
//	x = m[s]
//
// Cases like this are handled during SSA, search for slicebytetostring
// in ../ssa/_gen/generic.rules.
func mapKeyReplaceStrConv(n ir.Node) bool {

So m[string(k)] where k is a []byte does not allocate a string at all. It hashes the bytes directly and compares them directly. If you assign the string to a variable first, the allocation comes back — which is exactly the distinction that comment draws.

Concatenation

a + b + c becomes a call to concatstrings, or to one of the sized variants:

go
// src/runtime/string.go:69
// concatstring2 helps make the callsite smaller (compared to concatstrings),
// and we think this is currently more valuable than omitting one call in the
// chain, the same goes for concatstring{3,4,5}.
func concatstring2(buf *tmpBuf, a0, a1 string) string {
	return concatstrings(buf, []string{a0, a1})
}

func concatstring3(buf *tmpBuf, a0, a1, a2 string) string {
	return concatstrings(buf, []string{a0, a1, a2})
}

func concatstring4(buf *tmpBuf, a0, a1, a2, a3 string) string {
	return concatstrings(buf, []string{a0, a1, a2, a3})
}

func concatstring5(buf *tmpBuf, a0, a1, a2, a3, a4 string) string {
	return concatstrings(buf, []string{a0, a1, a2, a3, a4})
}

The sized variants exist to keep the call site small — passing five strings in registers beats building a slice.

The real work:

go
// src/runtime/string.go:30
// concatstrings implements a Go string concatenation x+y+z+...
// The operands are passed in the slice a.
// If buf != nil, the compiler has determined that the result does not
// escape the calling function, so the string data can be stored in buf
// if small enough.
func concatstrings(buf *tmpBuf, a []string) string {
	idx := 0
	l := 0
	count := 0
	for i, x := range a {
		n := len(x)
		if n == 0 {
			continue
		}
		if l+n < l {
			throw("string concatenation too long")
		}
		l += n
		count++
		idx = i
	}
	if count == 0 {
		return ""
	}

	// If there is just one string and either it is not on the stack
	// or our result does not escape the calling frame (buf != nil),
	// then we can return that string directly.
	if count == 1 && (buf != nil || !stringDataOnStack(a[idx])) {
		return a[idx]
	}
	s, b := rawstringtmp(buf, l)
	for _, x := range a {
		n := copy(b, x)
		b = b[n:]
	}
	return s
}

Three optimisations in one function:

Empty operands are skipped, so "a" + "" + "b" does not copy the empty string.
A single operand is returned as-is, no copy at all — as long as it is not a stack-allocated string that would die with the frame.
One allocation for the whole result, then the operands are copied into it in order.

That last point is the important one. a + b + c is not three concatenations. The compiler flattens it into one call, and the runtime allocates once and copies each piece in. If you write it as s := a + b; s = s + c, you get two allocations and two copies.

The throw("string concatenation too long") is an overflow check on the total length, and it is a throw, not a panic, because there is no sensible way to recover.

The buffer choice happens here:

go
// src/runtime/string.go:191
func rawstringtmp(buf *tmpBuf, l int) (s string, b []byte) {
	if buf != nil && l <= len(buf) {
		b = buf[:l]
		s = slicebytetostringtmp(&b[0], len(b))
	} else {
		s, b = rawstring(l)
	}
	return
}

Small enough and non-escaping: build it in the stack buffer, in place, with no copy at all (the operands are copied into the buffer). Too big or escaping: allocate on the heap.

The allocation

rawstring and rawbyteslice are the two allocation primitives:

go
// src/runtime/string.go:327
// rawstring allocates storage for a new string. The returned
// string and byte slice both refer to the same storage.
// The storage is not zeroed. Callers should use
// b to set the string contents and then drop b.
func rawstring(size int) (s string, b []byte) {
	p := mallocgc(uintptr(size), nil, false)
	return unsafe.String((*byte)(p), size), unsafe.Slice((*byte)(p), size)
}

// rawbyteslice allocates a new byte slice. The byte slice is not zeroed.
func rawbyteslice(size int) (b []byte) {
	cap := roundupsize(uintptr(size), true)
	p := mallocgc(cap, nil, false)
	if cap != uintptr(size) {
		memclrNoHeapPointers(add(p, uintptr(size)), cap-uintptr(size))
	}

	*(*slice)(unsafe.Pointer(&b)) = slice{p, size, int(cap)}
	return
}

rawstring is deliberately not zeroed — the caller is about to overwrite every byte, and zeroing a megabyte you are about to fill is wasted work. rawbyteslice does clear the tail between len and cap, because the caller can observe it through append or copy.

mallocgc(uintptr(size), nil, false) with a nil type means “no pointers in here”, so the garbage collector never scans the memory. A string’s bytes are immutable and can never contain a pointer, so this is safe and it makes strings cheap for the GC.

`strings.Builder`

The problem strings.Builder solves: building a string in a loop with + is quadratic. Each s += x allocates a new string and copies everything accumulated so far.

go
// src/strings/builder.go:14
// A Builder is used to efficiently build a string using [Builder.Write] methods.
// It minimizes memory copying. The zero value is ready to use.
// Do not copy a non-zero Builder.
type Builder struct {
	addr *Builder // of receiver, to detect copies by value

	// External users should never get direct access to this buffer, since
	// the slice at some point will be converted to a string using unsafe, also
	// data between len(buf) and cap(buf) might be uninitialized.
	buf []byte
}

// copyCheck implements a dynamic check to prevent modification after
// copying a non-zero Builder, which would be unsafe (see #25907, #47276).
//
// We cannot add a noCopy field to Builder, to cause vet's copylocks
// check to report copying, because copylocks cannot reliably
// discriminate the zero and nonzero cases.
func (b *Builder) copyCheck() {
	if b.addr == nil {
		// This hack works around a failing of Go's escape analysis
		// that was causing b to escape and be heap allocated.
		// See issue 23382.
		// TODO: once issue 7921 is fixed, this should be reverted to
		// just "b.addr = b".
		b.addr = (*Builder)(abi.NoEscape(unsafe.Pointer(b)))
	} else if b.addr != b {
		panic("strings: illegal use of non-zero Builder copied by value")
	}
}

// String returns the accumulated string.
func (b *Builder) String() string {
	return unsafe.String(unsafe.SliceData(b.buf), len(b.buf))
}

The copyCheck trick is the interesting part. A Builder holds a byte slice; String() converts that slice to a string without copying, using unsafe.String. If the Builder were copied by value, the copy would share the same backing array, and writes through one copy would corrupt the string already handed out by the other — a data race with no compiler support.

copyCheck solves that with a self-pointer. The first use records the address of the receiver; every later use compares. If they differ, the Builder was copied and the program panics with “strings: illegal use of non-zero Builder copied by value”.

The abi.NoEscape call is a workaround for an escape-analysis bug: without it, taking the address of the receiver would force the Builder onto the heap.

Growth is just an append with a doubling policy:

go
// src/strings/builder.go:64
// grow copies the buffer to a new, larger buffer so that there are at least n
// bytes of capacity beyond len(b.buf).
func (b *Builder) grow(n int) {
	buf := bytealg.MakeNoZero(2*cap(b.buf) + n)[:len(b.buf)]
	copy(buf, b.buf)
	b.buf = buf
}

bytealg.MakeNoZero allocates without zeroing, and the copy is memmove. One allocation per growth, doubling each time: building a 1 MB string with WriteString in a loop costs about 2 MB of copying in total, not 500 MB.

`strings.Clone`

Sometimes you want the opposite of a builder: you have a string and you want to make sure a small piece of it does not keep the whole thing alive.

go
// src/strings/clone.go:11
// Clone returns a fresh copy of s.
// It guarantees to make a copy of s into a new allocation,
// which can be important when retaining only a small substring
// of a much larger string. Using Clone can help such programs
// use less memory. Of course, since using Clone makes a copy,
// overuse of Clone can make programs use more memory.
// Clone should typically be used only rarely, and only when
// profiling indicates that it is needed.
// For strings of length zero the string "" will be returned
// and no allocation is made.
func Clone(s string) string {
	return stringslite.Clone(s)
}

The implementation is a copy into a fresh byte slice:

go
// src/internal/stringslite/strings.go:144
func Clone(s string) string {
	if len(s) == 0 {
		return ""
	}
	b := make([]byte, len(s))
	copy(b, s)
	return unsafe.String(&b[0], len(b))
}

Note that s[1:5] does not copy — it is a view onto the same backing array, and the whole original string stays alive as long as the sub-slice does. Clone is the escape hatch, and the doc comment is careful to say it should be used rarely.

The SIMD helpers

Copying is only half of the story; searching and comparing matter just as much, and those live in internal/bytealg with hand-written assembly per architecture. The generic implementations are simple:

go
// src/internal/bytealg/equal_generic.go:14
func Equal(a, b []byte) bool {
	// Neither cmd/compile nor gccgo allocates for these string conversions.
	// There is a test for this in package bytes.
	return string(a) == string(b)
}

Note the comment: converting a byte slice to a string here does not allocate, because the compiler recognises the pattern and turns it into a direct comparison.

On amd64 there is a SIMD Index for short needles, and a heuristic for when to stop using it:

go
// src/internal/bytealg/index_amd64.go:9
const MaxBruteForce = 64

func init() {
	if cpu.X86.HasAVX2 {
		MaxLen = 63
	} else {
		MaxLen = 31
	}
}

// Cutover reports the number of failures of IndexByte we should tolerate
// before switching over to Index.
// n is the number of bytes processed so far.
// See the bytes.Index implementation for details.
func Cutover(n int) int {
	// 1 error per 8 characters, plus a few slop to start.
	return (n + 16) / 8
}

The idea is that the vectorised search is fast but has a setup cost; for short inputs, or inputs where the byte is rare, a simple IndexByte scan wins. The cutover function decides when to switch, based on how many failures the simple scan has already had.

Trade-offs

What the copying design buys:

Immutability for free. A string cannot change under you, so it is safe to share across goroutines, put in a map key, or use as a map key at all.
Cheap headers. A string is two words. Passing one to a function costs two register moves.
One allocation per concatenation. concatstrings allocates the exact final size and copies each piece in once.
Escape analysis can eliminate the allocation entirely. A short, non-escaping string result lives in a stack buffer.

What it costs:

string(b) always copies when the result lives. Converting a large buffer to a string and keeping it means two copies of the bytes in memory.
Building a string with + is quadratic. This is the whole reason strings.Builder exists.
The zero-copy paths are subtle. slicebytetostringtmp produces a string that is only valid while the bytes are unchanged, and the compiler has to be sure of that. A string variable holding such a string and a later mutation of the buffer would be a bug — which is why the compiler only does it in narrow, provable cases.
strings.Builder cannot be copied. A Builder passed by value to a function is a panic waiting to happen.

Caveats

String concatenation is associative at the language level but not at the allocation level. a + b + c is one allocation; (a + b) + c is two. The compiler flattens the first form.

A string and a []byte conversion in a comparison does not allocate. string(b) == "foo" is special-cased, but string(b) == someString where someString is a variable also goes through the temporary path.

Converting a string to a []byte copies. []byte(s) is a fresh allocation; there is no zero-copy version in the language, because a byte slice is mutable and the string is not.

Slicing a string does not copy. s[10:20] shares the backing array with s and keeps the whole string alive. Use strings.Clone if you need to break that link.

strings.Builder’s String() result aliases the builder’s buffer. The string is valid forever, but it is not a copy, so the builder must not be written to afterwards in a way that would matter — and in fact copyCheck panics if the builder has been copied by value, which is the common way this goes wrong.

unsafe.String and unsafe.SliceData are the load-bearing parts of Builder.String(). They are the reason the builder can hand you a string without copying, and the reason the copy check has to exist.

Concatenation length overflow throws, not panics. throw cannot be recovered, so a program that concatenates strings until the length overflows dies immediately.

Wrapping up

The one thing to remember: every byte that becomes a long-lived string is copied exactly once, at the moment of conversion or concatenation, and the compiler’s job is to make sure that copy is the only one. slicebytetostring copies, slicebytetostringtmp does not, and the difference between them is whether the bytes are temporary.

The part people get wrong: thinking string(b) is free, or that s += x in a loop is merely “a bit wasteful”. The first is a full copy of your buffer; the second is quadratic in the length of the result. Use strings.Builder for the second, and expect a copy for the first.

What to go do about it: profile before you optimise. If a hot path is converting byte slices to strings, check whether the result escapes — if it does not, the compiler is already using the stack buffer. If you are building strings in a loop, switch to strings.Builder and measure. And if you are keeping a small substring of a huge string, use strings.Clone and watch the heap profile change.