Kubernetes Forked encoding/json Because the Standard One Loses Integers

TL;DR: Go’s encoding/json matches struct fields case-insensitively and decodes every number into an interface{} as float64. Both are wrong for Kubernetes: float64 silently corrupts large int64 values (resourceVersion, generation), and case-insensitivity means {"Kind": ...} is accepted where the contract says kind. The standard decoder is not extensible at the field-matching level, so the fix had to be a fork of the stdlib: sigs.k8s.io/json.

Kubernetes has a lot of JSON, and a lot of it is decoded into map[string]interface{} — by the unstructured client, by the kubectl printer, by every controller that handles objects it does not have a Go type for. That path has two behaviours that are actively harmful, and they are inherited from Go’s standard library rather than chosen by Kubernetes.

The two inherited bugs

Case-insensitive field matching. Go’s decoder looks up a JSON key in a map of folded (upper-cased) field names, so {"Kind": "Pod"} unmarshals into a struct with a Kind string \json:“kind”`field just fine. Kubernetes' API contract says the key iskind, lowercase. A client that sends Kind` is sending a malformed object, and the apiserver should reject it. But a decoder that accepts it means the malformation is discovered late, at the apiserver, instead of early, in the client.

Everything is a float64. Go’s decoder has no integer type in its JSON model. When the target is an interface{}, every number becomes a float64. float64 has 53 bits of mantissa, so it can represent every integer exactly up to 2^53 - 1, which is about 9.007 quadrillion. The Kubernetes resourceVersion is a 64-bit integer that grows by one on every write to the etcd cluster. A cluster that has done 2^53 writes does not exist yet, but generation, observedGeneration, resourceVersion values in unstructured form, and any user-supplied 64-bit field (spec.priority, metadata.generation on a high-churn object) routinely exceed 2^53 in long-lived or synthetic setups.

The failure is silent. float64(9007199254740993) is 9007199254740992, and there is no error, no warning, no way to tell from the Go value that a bit was lost.

The public surface is one function

Kubernetes does not expose the fork’s full API. It wraps it in apimachinery/pkg/util/json, and the entire exported surface for decoding is one function:

go
// staging/src/k8s.io/apimachinery/pkg/util/json/json.go:39
// limit recursive depth to prevent stack overflow errors
const maxDepth = 10000

// Unmarshal unmarshals the given data.
// Object keys are case-sensitive.
// Numbers decoded into interface{} fields are converted to int64 or float64.
func Unmarshal(data []byte, v interface{}) error {
	return kjson.UnmarshalCaseSensitivePreserveInts(data, v)
}

Two sentences of documentation, and they are the whole contract: “Object keys are case-sensitive” and “Numbers decoded into interface{} fields are converted to int64 or float64”.

The fork itself exposes exactly that behaviour, and no more:

go
// vendor/sigs.k8s.io/json/json.go:51
// UnmarshalCaseSensitivePreserveInts parses the JSON-encoded data and stores the result in the value pointed to by v.
//
// UnmarshalCaseSensitivePreserveInts matches the behavior of encoding/json#Unmarshal, with the following changes:
//   - When unmarshaling into a struct, JSON keys must case-sensitively match `json` tag names (for tagged struct fields)
//     or struct field names (for untagged struct fields), or they are treated as unknown fields and discarded.
//   - When unmarshaling a number into an interface value, it is unmarshaled as an int64 if
//     the JSON data does not contain a "." character and parses as an integer successfully and
//     does not overflow int64. Otherwise, the number is unmarshaled as a float64.
//   - If a syntax error is returned, it will not be of type encoding/json#SyntaxError,
//     but will be recognizeable by this package's IsSyntaxError() function.
func UnmarshalCaseSensitivePreserveInts(data []byte, v interface{}) error {
	return internaljson.Unmarshal(
		data,
		v,
		internaljson.CaseSensitive,
		internaljson.PreserveInts,
	)
}

Note the wording: “matches the behavior of encoding/json#Unmarshal, with the following changes”. Everything else — the scanner, the tokeniser, the Unmarshaler interface handling, the ,string tag option — is identical, because it is the same code.

Why a fork and not a wrapper

You cannot fix this with a wrapper. The case-insensitive matching happens deep inside the stdlib decoder, in the struct-field lookup, and the fields structure it consults is an unexported type:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/encode.go:680
type structFields struct {
	list         []field
	byExactName  map[string]*field
	byFoldedName map[string]*field
}

The stdlib already builds both maps — byExactName for the fast path and byFoldedName for the case-insensitive fallback — and the decoder decides which one to consult. There is no option, no interface, no hook. The fork’s fix is to simply not consult byFoldedName:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/decode.go:778
			f := fields.byExactName[string(key)]
			if f == nil && !d.caseSensitive {
				f = fields.byFoldedName[string(foldName(key))]
			}
			if f != nil {

d.caseSensitive is set by the CaseSensitive option, which UnmarshalCaseSensitivePreserveInts passes in. One extra boolean comparison on the hot path, and the folded lookup is skipped entirely.

For reference, the folding itself is Go’s unicode.SimpleFold walk — full Unicode case folding, not ASCII lowercasing:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/fold.go:12
// foldName returns a folded string such that foldName(x) == foldName(y)
// is identical to bytes.EqualFold(x, y).
func foldName(in []byte) []byte {
	// This is inlinable to take advantage of "function outlining".
	var arr [32]byte // large enough for most JSON names
	return appendFoldedName(arr[:0], in)
}

So in the stdlib, {"KIND": ...}, {"kind": ...} and {"Kind": ...} are all the same key. In the fork, with caseSensitive set, only kind matches.

The integer preservation

The second change is in convertNumber, the function that turns a number literal into a Go value:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/decode.go:921
// convertNumber converts the number literal s to a float64 or a Number
// depending on the setting of d.useNumber.
func (d *decodeState) convertNumber(s string) (any, error) {
	if d.useNumber {
		return Number(s), nil
	}

	// if the string contains no floating point, return it as an int64 if it decodes successfully and does not overflow.
	// otherwise, fall back to float64 behavior.
	if d.preserveInts && !strings.Contains(s, ".") {
		if i, err := strconv.ParseInt(s, 10, 64); err == nil {
			return i, nil
		}
	}

	f, err := strconv.ParseFloat(s, 64)
	if err != nil {
		return nil, &UnmarshalTypeError{Value: "number " + s, Type: reflect.TypeFor[float64](), Offset: int64(d.off)}
	}
	return f, nil
}

Three conditions have to hold for an int64 to come out:

d.preserveInts is set — i.e. the PreserveInts option was passed.
The literal contains no . — so 1.0 is a float, 1 is an int, and 1e3 is… also a float, because 1e3 contains no . but does contain an exponent. Wait: strconv.ParseInt("1e3", 10, 64) fails, so 1e3 falls through to ParseFloat and becomes 1000.0. That is the desired behaviour.
strconv.ParseInt succeeds and does not overflow int64 — the 64-bit bound is the ParseInt width argument, so an int64-overflowing literal falls through to float64 rather than wrapping.

The overflow fall-through is important. A literal like 18446744073709551615 (2^64 - 1) does not fit in an int64, so it becomes a float64 — which is lossy, but lossy in a way the caller can detect, because the Go type is float64 and not int64. The type of the decoded value is the signal.

This is also why the option is called preserve ints and not use ints: it does not change the wire format, and it does not change how numbers decode into typed fields. A struct { N int64 \json:“n”` }decodes the same way it always did. Only theinterface{}` case changes.

The strict decoder: accumulate, don’t fail fast

The fork also adds a strict mode, used by kubectl apply and the validation path to report every problem in a document rather than only the first one. The accumulation logic is worth reading because of two details:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/kubernetes_patch.go:87
// saveStrictError saves a strict decoding error,
// for reporting at the end of the unmarshal if no other errors occurred.
func (d *decodeState) saveStrictError(err *strictError) {
	// prevent excessive numbers of accumulated errors
	if len(d.savedStrictErrors) >= 100 {
		return
	}
	// dedupe accumulated strict errors
	if d.seenStrictErrors == nil {
		d.seenStrictErrors = map[strictError]struct{}{}
	}
	if _, seen := d.seenStrictErrors[*err]; seen {
		return
	}

	// accumulate the error
	d.seenStrictErrors[*err] = struct{}{}
	d.savedStrictErrors = append(d.savedStrictErrors, err)
}

The cap is 100. A document with ten thousand unknown fields produces 100 errors, not ten thousand. This is a deliberate bound on the error report, because the report is rendered into an API error message that has to fit in an HTTP response.

Deduplication is by exact error identity. The map key is the strictError value, which is {ErrType, Path}:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/kubernetes_patch.go:125
type strictErrType string

const (
	unknownStrictErrType   strictErrType = "unknown field"
	duplicateStrictErrType strictErrType = "duplicate field"
)

// strictError is a strict decoding error
// It has an ErrType (either unknown or duplicate)
// and a path to the erroneous field
type strictError struct {
	ErrType strictErrType
	Path    string
}

So the same unknown field appearing twice is one error, but the same unknown field under two different parents is two errors — the path is part of the identity. And the path is built as a dotted stack as the decoder descends:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/kubernetes_patch.go:107
func (d *decodeState) appendStrictFieldStackKey(key string) {
	if !d.disallowDuplicateFields && !d.disallowUnknownFields {
		return
	}
	if len(d.strictFieldStack) > 0 {
		d.strictFieldStack = append(d.strictFieldStack, ".", key)
	} else {
		d.strictFieldStack = append(d.strictFieldStack, key)
	}
}

Array indices use a different separator:

go
// vendor/sigs.k8s.io/json/internal/golang/encoding/json/kubernetes_patch.go:118
func (d *decodeState) appendStrictFieldStackIndex(i int) {
	if !d.disallowDuplicateFields && !d.disallowUnknownFields {
		return
	}
	d.strictFieldStack = append(d.strictFieldStack, "[", strconv.Itoa(i), "]")
}

And critically: the strict checks do not change what is stored. The object is parsed and returned even when strict errors exist:

go
// vendor/sigs.k8s.io/json/json.go:95
// Note that the strict checks do not change what is stored in v.
// For example, if duplicate fields are present, they will be parsed and stored in v,
// and errors about the duplicate fields will be returned in the strict error list.

That is the whole point of “accumulate instead of fail fast”. kubectl apply needs to tell you about all five problems in your manifest, not just the first one, and it needs the partially-decoded object either way.

The YAML path has to match

YAML is a superset of JSON, and Kubernetes accepts YAML everywhere it accepts JSON — kubectl apply -f, the API server’s content negotiation, every CRD manifest. If YAML and JSON produced different Go types for the same document, then the same manifest would behave differently depending on its file extension. So the YAML decoder has to produce int64 too, and it gets there by a different route: decode with UseNumber(), then walk the tree converting json.Number into int64 or float64.

go
// staging/src/k8s.io/apimachinery/pkg/util/yaml/decoder.go:38
func Unmarshal(data []byte, v interface{}) error {
	preserveIntFloat := func(d *json.Decoder) *json.Decoder {
		d.UseNumber()
		return d
	}
	switch v := v.(type) {
	case *map[string]interface{}:
		if err := yaml.Unmarshal(data, v, preserveIntFloat); err != nil {
			return err
		}
		return jsonutil.ConvertMapNumbers(*v, 0)
	case *[]interface{}:
		if err := yaml.Unmarshal(data, v, preserveIntFloat); err != nil {
			return err
		}
		return jsonutil.ConvertSliceNumbers(*v, 0)
	case *interface{}:
		if err := yaml.Unmarshal(data, v, preserveIntFloat); err != nil {
			return err
		}
		return jsonutil.ConvertInterfaceNumbers(v, 0)
	default:
		return yaml.Unmarshal(data, v)
	}
}

Note that the preserveIntFloat hook only applies to the three unstructured cases. A typed target (*v1.Pod) goes down the plain yaml.Unmarshal path, because the YAML library converts YAML to JSON and then decodes it into the struct — and there the struct’s field types decide everything.

The post-walk conversion is where the actual int64 decision is made:

go
// staging/src/k8s.io/apimachinery/pkg/util/json/json.go:112
// convertNumber converts a json.Number to an int64 or float64, or returns an error
func convertNumber(n json.Number) (interface{}, error) {
	// Attempt to convert to an int64 first
	if i, err := n.Int64(); err == nil {
		return i, nil
	}
	// Return a float64 (default json.Decode() behavior)
	// An overflow will return an error
	return n.Float64()
}

Same rule as the fork, expressed differently: try int64 first, fall back to float64. json.Number.Int64() fails on overflow and on a fractional value, so 1.5 becomes float64(1.5) and 18446744073709551615 becomes a float64.

The walk is bounded by maxDepth = 10000, because a deeply nested document would otherwise be able to blow the stack during the recursive walk:

go
// staging/src/k8s.io/apimachinery/pkg/util/json/json.go:39
// limit recursive depth to prevent stack overflow errors
const maxDepth = 10000

go
// staging/src/k8s.io/apimachinery/pkg/util/json/json.go:64
// ConvertMapNumbers traverses the map, converting any json.Number values to int64 or float64.
// values which are map[string]interface{} or []interface{} are recursively visited
func ConvertMapNumbers(m map[string]interface{}, depth int) error {
	if depth > maxDepth {
		return fmt.Errorf("exceeded max depth of %d", maxDepth)
	}

The trade-off

What the fork buys: the API contract is enforced at the edge instead of at the server. A malformed Kind is a decode error in the client, not a 410 Gone in production. And 64-bit integers survive the round trip through interface{} without losing a bit.

What it costs:

A vendored fork of the Go standard library. The file headers still say “Copyright 2010 The Go Authors”, BSD licensed, but every Go release that changes encoding/json requires a manual merge. The vendored tree at vendor/sigs.k8s.io/json/internal/golang/encoding/json/ is the stdlib’s decoder with surgical edits.
Two different decoding paths to keep in sync. The JSON path uses PreserveInts inside the decoder; the YAML path uses UseNumber plus a post-walk. They must agree, and the only thing enforcing that is a shared test suite.
Ints are not ints everywhere. 1.0 in JSON is a float64. 1e3 is a float64. A literal that overflows int64 is a float64. If your code does obj["count"].(int64) you will panic on any of those. The safe form is obj["count"].(int64) guarded by a type switch.
Case sensitivity is a breaking change for sloppy clients. Any code that was relying on {"Kind": ...} working is now broken. That is the point, but it is still a break.
The strict error cap of 100 silently truncates. If a document has 1500 unknown fields, you are told about 100 of them and the rest are dropped without a “and N more” marker.

Wrapping up

The one thing to remember: sigs.k8s.io/json is the Go stdlib decoder with two switches flipped — caseSensitive skips the folded-name lookup at vendor/sigs.k8s.io/json/internal/golang/encoding/json/decode.go:779, and preserveInts prefers int64 over float64 at decode.go:930. Kubernetes reaches it through exactly one function: apimachinery/pkg/util/json.Unmarshal.

The part people get wrong: assuming int64 is guaranteed for integers. It is guaranteed only for literals with no . and no exponent that fit in 64 bits. 1.0 is a float64, 1e3 is a float64, and anything above 2^63-1 is a float64. Type-switch on the value; do not assert.

What to go do about it: when you decode into map[string]interface{} or unstructured.Unstructured, always use k8s.io/apimachinery/pkg/util/json.Unmarshal (or k8s.io/apimachinery/pkg/util/yaml.Unmarshal) rather than encoding/json.Unmarshal, and expect int64. If you want the strict behaviour — unknown fields and duplicate fields reported rather than silently dropped — use sigs.k8s.io/json.UnmarshalStrict, and remember it returns the parsed object and a list of errors, capped at 100 entries. If you are writing a controller that must not lose precision, decode resourceVersion and generation into typed int64 fields rather than relying on the unstructured path at all.