Go's Experimental SIMD Package: Vector Registers Without Assembly

TL;DR: Go now has a SIMD package. simd.Int8s and friends are portable vector types that the compiler lowers to real VPADDD instructions in the SSA backend. A source-to-source rewrite pass called “midway” specialises your code per vector width and picks one at runtime.

Writing SIMD in Go used to mean one of three things: assembly files, cgo, or a //go:linkname into a C library. All three are unpleasant in different ways, and all three put a wall between you and the compiler’s optimiser.

The Go team’s answer has been in development for a while, and in the Go 1.28 development tree it has taken shape as a real package with real compiler support. This is a walk through how it works, based on reading that tree.

Where the code lives now

The project has moved a couple of times, and if you have been following it you may remember internal/simd with types like simd.Int8x16. In the current tree the layout is:

Path	What it is
`src/simd/`	The portable package. Length-agnostic vector types (`Int8s`, `Float32s`, `Mask8s`, …). Works on every platform.
`src/simd/archsimd/`	The platform package. Fixed-width types (`Int8x16` … `Float64x8`) plus scalable types on SVE. Contents vary by GOARCH.
`src/simd/internal/spec/`	The executable specification. Buildable Go, but not meant to be called directly.
`src/simd/internal/simdref/`	Reference implementation of the API, generated from spec.
`src/simd/internal/bridge/`	Glue between `simd` and `archsimd`. Generated, not user-visible.
`src/internal/simd/variants/`	The variant table, shared between the generator and the compiler.

Everything is behind a build tag:

go
// src/simd/doc.go:7
/*
Package simd implements portable and vector-size-agnostic SIMD types,
and functions and methods for working with these types.  SIMD types
are either implemented in hardware (for example, arm64 "Neon" or
amd64 "AVX/AVX2/AVX512") using the corresponding types in the
[simd/archsimd] package, or emulated in pure Go.  In all cases, the
vector length is at least 128 bits, and within a given program
execution, all vectors have the same length.

and behind a toolchain experiment:

go
// src/internal/goexperiment/exp_simd_on.go:7
const SIMD = true
const SIMDInt = 1

So today you would build a toolchain with GOEXPERIMENT=simd and then import "simd". The package is at the top level of the standard library rather than under internal/, which is the visible shape of the plan to make it public. The api/ directory in the tree has no simd entries yet, so it is not part of the compatibility promise.

Two layers of types

The first thing to understand is that there are two different kinds of vector type, and they exist for different reasons.

simd/archsimd types are fixed width. Int8x16 is always sixteen int8s, Float64x4 is always four float64s, and they map one-to-one onto what the hardware can do:

go
// src/simd/archsimd/types_amd64.go:92
// Int8x16 is a 128-bit SIMD vector of 16 int8s.
//
// CPU Feature: AVX
type Int8x16 struct {
	int8x16 v128
	vals    [16]int8
}

// Len returns the number of elements in an Int8x16.
func (x Int8x16) Len() int { return 16 }

// LoadInt8x16Array loads an array into a vector.
//
//go:noescape
func LoadInt8x16Array(x *[16]int8) (z Int8x16)

// StoreArray stores the elements of x to an array.
//
//go:noescape
func (x Int8x16) StoreArray(y *[16]int8)

Note the structure. v128 is a zero-sized marker whose only job is to make the struct uncomparable:

go
// src/simd/archsimd/types_amd64.go:8
type v128 struct {
	_128 [0]func() // uncomparable
}

The real payload is vals [16]int8. The compiler’s intrinsics treat these structs as vector-register values directly; there is no copying of the array into and out of a register, because the SSA backend has a register class for them.

simd types are scalable. Int8s is “a vector of int8s, of whatever width this machine is running”. The API is the intersection of everything the architectures can do:

go
// src/simd/simd_stubs.go:9
// LoadInt8s loads a slice into a vector. If len(s) < z.Len(), it panics.
func LoadInt8s(s []int8) (z Int8s)

// LoadInt8sPart loads n=min(len(s), z.Len()) elements from slice s as a
// vector and returns the vector and n. If len(s) < z.Len(), the
// remaining vector elements will be zero.
func LoadInt8sPart(s []int8) (z Int8s, n int)

// BroadcastInt8s returns a vector with the input x assigned to all elements of the
// result.
func BroadcastInt8s(x int8) (z Int8s)

The scalable type is a plain struct of two words:

go
// src/simd/simd_types.go:14
// Int8s represents a vector of 8-bit signed integers.
type Int8s struct {
	_    _simd
	a, b uint64 // the actual vector size may be larger.
}

A vector is at least 128 bits, so two uint64s is the floor. On a machine with 512-bit vectors the same type carries four words’ worth.

The “one meaning” rule

The generators enforce a rule that is worth quoting, because it explains why the API looks the way it does. From the generator README:

One name means one thing. A name need not exist on every platform or in both packages, but wherever it does exist it must denote the same operation, with the same signature and the same semantics.

And the consequence:

The simd package API is the intersection of the archsimd package APIs across all architectures, after generalizing fixed-width to scalable width, plus a few methods for querying emulation behavior.

So simd.Float32s.Add and archsimd.Float32x4.Add are the same operation. The simd package is not a hand-written convenience layer over archsimd; it is computed from the intersection of the per-architecture APIs, which is why src/simd/simd_stubs.go is a generated file with no logic in it at all.

How the compiler knows an operation is a vector op

Here is the part that makes it “not assembly”. A method on a vector type is declared with no body at all:

go
// src/simd/archsimd/ops_amd64.go:283
// Add adds x and y elementwise.
//
//	z[i] = x[i] + y[i]
//
// Asm: VPADDD, CPU Feature: AVX
func (x Int32x4) Add(y Int32x4) (z Int32x4)

The // Asm: and // CPU Feature: comments are consumed by the generator, not by the compiler. The compiler learns about the method from a table of intrinsics:

go
// src/cmd/compile/internal/ssagen/simdAMD64intrinsics.go:40
	addF(simdPackage, "Float32x4.Add", opLen2(ssaop.OpAddFloat32x4, types.TypeVec128), sys.AMD64)

simdPackage is literally the import path:

go
// src/cmd/compile/internal/ssagen/intrinsics.go:1875
const simdPackage = "simd/archsimd"

So the compiler hard-codes the knowledge that simd/archsimd.Float32x4.Add should become the SSA opcode AddFloat32x4, whose result type is TypeVec128.

The vector “types” in the SSA backend are new:

go
// src/cmd/compile/internal/types/type.go:1615
	TypeVec128    = newSIMD("vec128")
	TypeVec256    = newSIMD("vec256")
	TypeVec512    = newSIMD("vec512")
	TypeMask      = newSIMD("mask") // not a vector, not 100% sure what this should be.

with widths and alignments assigned at init time:

go
// src/cmd/compile/internal/types/type.go:1626
	TypeVec128.width = 16
	TypeVec128.align = 8
	TypeVec256.width = 32
	TypeVec256.align = 8
	TypeVec512.width = 64
	TypeVec512.align = 8

And then there is a machine opcode per width, generated from the same source as the intrinsics table:

go
// src/cmd/compile/internal/ssa/_gen/simdAMD64ops.go:315
		{name: "VPADDD128", argLength: 2, reg: v21, asm: "VPADDD", commutative: true, typ: "Vec128"},
		{name: "VPADDD256", argLength: 2, reg: v21, asm: "VPADDD", commutative: true, typ: "Vec256"},
		{name: "VPADDD512", argLength: 2, reg: w21, asm: "VPADDD", commutative: true, typ: "Vec512"},

v21 and w21 are register classes: v21 means “two vector inputs, one vector output”, w21 is the same but for the wider register file. The asm: field is the actual instruction mnemonic the assembler will emit.

The lowering rules

The link between the abstract operation and the machine instruction is a rewrite rule, and the SIMD rules are in their own file:

go
// src/cmd/compile/internal/ssa/_gen/simdAMD64.rules:41
(AddInt32x4 ...) => (VPADDD128 ...) // pureVreg

Read that as: “an AddInt32x4 operation becomes a VPADDD128 operation, and the result lives purely in a vector register”. The ... means “all arguments pass through unchanged”. There are three thousand six hundred lines of these, one per (operation, width) pair.

The rules also encode the clever bits — load folding, mask merging, and feature gating:

go
// src/cmd/compile/internal/ssa/_gen/simdAMD64.rules:1477
(VMOVDQU32Masked128 (VPADDD128 x y) mask) => (VPADDDMasked128 x y mask)

go
// src/cmd/compile/internal/ssa/_gen/simdAMD64.rules:2212
(VPBLENDVB128 dst (VPADDD128 x y) mask) && v.Block.CPUfeatures.HasFeature(ssa.CPUavx512) => (VPADDDMasked128Merging dst x y (VPMOVVec32x4ToM <types.TypeMask> mask))

Note the v.Block.CPUfeatures.HasFeature(...) guard: the same Go source compiles to different instructions depending on the CPU features of the machine being compiled for. This is how Go gets AVX-512 masked operations on a machine that has AVX-512, and a different sequence on one that does not, without any assembly and without any runtime dispatch in the common case.

Now the hard part: the width is not known at compile time

A portable vector type has no fixed width. simd.Int8s might be 16 bytes on one machine and 64 bytes on another. That is fine as long as the compiler knows which one to emit — but it does not, until runtime.

Go’s solution is a source-to-source rewrite pass that runs before type checking, in the compiler’s “midway” phase. From the rewriteer’s own documentation:

Go attempts to provide a package similar to the “Highway” library for C++ (https://google.github.io/highway). The library package is “simd” and defines vector types with unspecified widths that are bound to particular machine dependent types as late as program execution. This is accomplished by rewriting code that depends on these types into code that references architecture-specific types, perhaps more than once, and if necessary dynamically choosing which version to execute based on hardware attributes.

So the pipeline is:

text
   your source:  func f(x simd.Int8s) simd.Int8s { return x.Add(y) }
                            |
                            v
        midway rewrite:   f is specialised per width
                            |
              +-------------+-------------+
              |             |             |
           f@simd128    f@simd256    f@simd512     (archsimd.Int8x16, etc.)
              |             |             |
              +-------------+-------------+
                            |
                            v
         f: switch simd.VectorSize() { case 128: ... case 256: ... }
                            |
                            v
              compiler intrinsics: f@simd128 calls archsimd.Int8x16.Add
                            |
                            v
                    SSA: AddInt8x16 => VPADDB128

The rewrite happens after type checking but before conversion to unified IR, and then the type checker is re-run. The rewriter’s comment says why that is the chosen point:

The rewriting takes place early in the compiler, after type checking but before conversion to “unified” IR. To ensure that types are correctly set on the modified version of the code, type checking information is reset and the type checking phase is re-run. This places some limits on the shape of the rewrites, but it also ensures that the rewritten code is well-formed.

The rewriter also has to avoid rewriting the archsimd package itself, and it introduces a bridge package so that the rewritten code does not reference archsimd types directly — the compiler’s export-data naming rules would otherwise reject it:

Rewritten code does not reference “archsimd” types directly, but instead references types in a “bridge” package that filters the available methods and adds a few more.

Variants: the CPU that has most of an instruction set

Now the interesting wrinkle. Suppose your CPU has AVX2 but not AVX-512. Or, on arm64, has NEON but not the carryless-multiply instruction — which is exactly the case for Raspberry Pi.

The naive answer is “fall back to emulation”. The better answer, which Go implements, is a variant: a second implementation of the same operations that emulates only the missing instructions, keeping everything else in hardware.

The variant table is tiny and lives in a package shared between the code generator and the compiler:

go
// src/internal/simd/variants/variants.go:28
type Key struct {
	Arch string
	Size int
}

type Variant struct {
	Suffix          string          // the name (suffix) for this variant.
	Emulated        map[string]bool // Methods that are emulated for this variant.
	DefaultRequires string          // default requires this function return true, false means variant.
}

// Name returns the variant name of type t.
// If receiver v is nil, returns t.
func (v *Variant) Name(t string) string {
	if v == nil {
		return t
	}
	return t + v.Suffix
}

var Variants map[Key]*Variant

func init() {
	Variants = make(map[Key]*Variant)
	Variants[Key{"arm64", 128}] = &Variant{
		Suffix: "nclm",
		Emulated: map[string]bool{
			"CarrylessMultiplyEven": true,
			"CarrylessMultiplyOdd":  true,
		},
		DefaultRequires: "HasHardwareCarrylessMultiply",
	}

}

One variant today: arm64 at 128 bits, suffix nclm (no carryless multiply), which emulates exactly two methods, and which is used when HasHardwareCarrylessMultiply() returns false.

The rewriter uses this table to build the dispatch function. Here is the code that decides what the case 128: arm of the generated switch looks like:

go
// src/cmd/compile/internal/midway/rewrite.go:446
		// if this architecture and size has a variant, then guard
		// the regular case call with variant.require
		// and then follow it with the variant case.
		// `if simd.<require>() { callReturnStmt }`
		if v := variants.Variants[variants.Key{Arch: buildcfg.GOARCH, Size: k}]; v != nil {
			callReturnStmt = guardCallWithCondition(v.DefaultRequires, callReturnStmt)
			caseBody = append(caseBody, callReturnStmt)
			callReturnStmt = makeCallReturnStmt(k, v.Suffix)
		}

		caseBody = append(caseBody, callReturnStmt)

So the generated dispatcher for a function on arm64 ends up roughly like:

go
switch simd.VectorSize() {
case 128:
    if simd.HasHardwareCarrylessMultiply() {
        return f@simd128(args)
    }
    return f@simd128nclm(args)
case 256:
    return f@simd256(args)
}

The HasHardwareCarrylessMultiply predicate is a runtime value:

go
// src/simd/midway_common.go:178
// HasHardwareCarrylessMultiply returns whether this platform
// has a hardware-implemented version of carryless multiply.
// With default GODEBUG=simd settings, if this is false,
// it is emulated and merely slow, but with non-default settings
// this can indicate the possibility of a missing instruction
// that will fail ("SIGILL") if it is executed.
func HasHardwareCarrylessMultiply() bool {
	return hwClmul && archHasHwClmul
}

Choosing the width at runtime

The width decision happens once, at package init time, based on CPU feature detection:

go
// src/simd/midway_amd64.go:16
func archMaxVectorSize() (size, allFeatureSize int, arch string) {
	arch = "amd64"
	if archsimd.X86.AVX() {
		size = 128
		allFeatureSize = 128
	}
	if archsimd.X86.AVX2() {
		size = 256
		if cpu.X86.HasVPCLMULQDQ {
			allFeatureSize = 256
		}
	}
	if archsimd.X86.AVX512() {
		size = 512
		if cpu.X86.HasAVX512VPCLMULQDQ {
			allFeatureSize = 512
		}
	}
	return
}

Note the two returned numbers. size is the widest vector the CPU can do. allFeatureSize is the widest width at which the CPU has every feature the SIMD API needs — here, the carryless-multiply instruction that allFeatureSize keeps bumping down. If the CPU has AVX-512 but not VPCLMULQDQ, you get 512-bit vectors but you also get a variant, because some operations have to be emulated.

The feature detection itself is a plain wrapper over internal/cpu:

go
// src/simd/archsimd/cpu.go:31
// AVX512 returns whether the CPU supports the AVX512F+CD+BW+DQ+VL features.
//
// These five CPU features are bundled together, and no use of AVX-512
// is allowed unless all of these features are supported together.
// Nearly every CPU that has shipped with any support for AVX-512 has
// supported all five of these features.
//
// If it returns true, then the CPU also supports AVX and AVX2.
//
// AVX512 is defined on all GOARCHes, but will only return true on
// GOARCH amd64.
func (X86Features) AVX512() bool {
	return cpu.X86.HasAVX512
}

The GODEBUG knob

Because the width and the emulation decision are made at runtime, they need a knob for testing. GODEBUG=simd=N requests a specific width (128, 256 or 512), GODEBUG=simd=0 requests pure emulation even on hardware that can do better, and a + prefix skips the feature check:

go
// src/simd/midway_common.go:90
	} else if len(gosimd) > 0 && gosimd[0] == '@' {
		// For testing, allow specification of a desired variant.
		// If there is one for this architecture and size, it should work.
		// This is not, currently, intended for everyday use.
		key := variants.Key{Arch: arch, Size: maxVectorSize}
		if v, ok := variants.Variants[key]; ok {
			// there is a variant at this size, just reset hwClmul
			hwClmul = false
			if v.Name("") != gosimd[1:] {
				panic(fmt.Errorf("requested variant %s but obtained variant %s", gosimd[1:], v.Name("")))
			}
			return maxVectorSize, emulated, hwClmul
		}
		panic(fmt.Errorf("requested variant %s but no variant was available", gosimd[1:]))

GODEBUG=simd=@nclm asks for the Raspberry Pi variant explicitly. That is the kind of escape hatch you want when a new CPU turns out to be missing something the feature detection did not anticipate.

Trade-offs

What you get:

Real vector instructions from Go source. No assembly files, no cgo, no build tags per architecture.
Portable code that adapts. The same source runs on AVX, AVX2, AVX-512, NEON, SVE and wasm, and picks the best width at runtime.
Full participation in the compiler. Vector values are SSA values, so they go through the same register allocation, spilling and inlining as everything else.
Graceful degradation. Missing instructions become emulated operations, not a crash.

What it costs:

A source-to-source rewrite pass. Every function that mentions a SIMD type gets duplicated per width. Compile time and binary size both go up.
A runtime dispatch. The generated switch on the vector size is a branch on every call to a boundary function, unless the compiler can prove the size is constant.
A fragile experimental surface. The package is behind GOEXPERIMENT=simd and is not in the API promise files.
A lot of generated code. Three thousand six hundred lines of rewrite rules for amd64 alone, plus the arm64, arm64-SVE and wasm equivalents.

Caveats

This is an experiment, not a release. GOEXPERIMENT=simd is required. Without it, the package does not build. The api/ files list nothing about simd, so nothing about its shape is frozen.

There are open bugs, listed in the package doc itself:

go
// src/simd/doc.go:89
// BUG(reflection): Calls won't work, and there may be other bugs.
// BUG(global initialization): SIMD-dependent var initializers don't work.
// BUG(modified names): Modified names may appear in stack traces and debugging.

That last one is worth understanding: because the rewriter renames functions by appending @simd128 and friends, the names in your stack traces and debug information are not the names you wrote.

Emulation is not always fast. A 512-bit vector emulated in pure Go is slower than a scalar loop written by a human, because the emulation still goes through the generic vector machinery.

The width is fixed for the lifetime of the process. The decision is made at init time. You cannot change it later, and GODEBUG=simd=256 on a machine that only has AVX will panic with a clear message rather than silently downgrading.

The simd and archsimd types are not interchangeable. ToArch() and Int8sFromArch exist for converting between them, and the latter panics if the correspondence is wrong.

The API is the intersection, which means it is the weakest common denominator. An operation that is cheap on one architecture and awkward on another may simply not be in the portable package at all.

Wrapping up

The one thing to remember: Go’s SIMD is two packages and a compiler pass. simd gives you length-agnostic types, simd/archsimd gives you the fixed-width ones that map onto real instructions, and the “midway” rewrite turns your portable code into a set of width-specialised functions plus a runtime dispatcher.

The part people get wrong: the width is not chosen by the compiler and it is not chosen by your build flags. It is chosen at program startup from CPU feature detection, and it can change between two runs of the same binary on two different machines. Code that assumes a particular vector width is wrong.

What to go do about it: build a toolchain with GOEXPERIMENT=simd and write a loop that processes eight int32s at a time, then check the generated assembly. If the numbers are not better than the scalar version, the loop was probably not the bottleneck — and you have learned that for free, without writing a single line of assembly.