How CPython Implements bytes in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s bytes is an immutable sequence of bytes. In CPython it is implemented by PyBytesObject in Objects/bytesobject.c, a variable-size object whose payload is a single trailing char array (ob_sval). This article traces the representation and the hot paths through the 3.15 source.

The object struct

Every bytes object is a PyVarObject: the size lives in ob_size (the number of bytes, not counting the trailing NUL).

 c
 
 
// Include/cpython/bytesobject.h
typedef struct {
 PyObject_VAR_HEAD
 Py_DEPRECATED(3.11) Py_hash_t ob_shash;
 char ob_sval[1];

 /* Invariants:
 * ob_sval contains space for 'ob_size+1' elements.
 * ob_sval[ob_size] == 0.
 * ob_shash is the hash of the byte string or -1 if not computed yet.
 */
} PyBytesObject;

ob_sval is declared with size 1 and the allocator over-allocates the trailing storage, exactly like PyLongObject’s digit array. The ob_shash field caches the hash; -1 means “not computed yet” (and a real hash of -1 is remapped to -2 by Py_HashBuffer).

Memory layout and `_PyBytesObject_SIZE`

Because ob_sval[1] already accounts for the first byte, the basic size used for allocation is one byte less than sizeof(PyBytesObject):

 c
 
 
// Include/internal/pycore_bytesobject.h
#define _PyBytesObject_SIZE (offsetof(PyBytesObject, ob_sval) + 1)

That saves 3 or 7 bytes per object on typical systems. bytesobject.c aliases it:

 c
 
 
// Objects/bytesobject.c
#define PyBytesObject_SIZE _PyBytesObject_SIZE

So an object of length n is allocated as PyBytesObject_SIZE + n bytes, with the final ob_sval[n] byte reserved for the terminating '\0'.

text

+---------------------------+ <- PyObject_Malloc result
| ob_refcnt |
+---------------------------+
| *ob_type (&PyBytes_Type) |
+---------------------------+
| ob_size (length n) |
+---------------------------+
| ob_shash (cached hash) |
+---------------------------+
| ob_sval[0] |
| ob_sval[1] |
| ... |
| ob_sval[n-1] |
| ob_sval[n] == '\0' |
+---------------------------+

The trailing NUL lets bytes objects be passed directly to C APIs expecting null-terminated strings, while ob_size keeps the true (possibly embedded-NUL) length.

Singletons: empty and single characters

There is no general interning of bytes, but two families of immortal singletons are created at runtime init:

 c
 
 
// Objects/bytesobject.c
#define CHARACTERS _Py_SINGLETON(bytes_characters)
#define CHARACTER(ch) \
 ((PyBytesObject *)&(CHARACTERS[ch]))
#define EMPTY (&_Py_SINGLETON(bytes_empty))

bytes_empty is the immortal empty bytes object (b""); PyBytes_FromStringAndSize returns it for size 0.
bytes_characters[256] are immortal single-byte objects for every byte value 0..255. PyBytes_FromStringAndSize and PyBytes_FromString return CHARACTER(*str & 255) when size == 1.

 c
 
 
// Objects/bytesobject.c
if (size == 1 && str != NULL) {
 op = CHARACTER(*str & 255);
 assert(_Py_IsImmortal(op));
 return (PyObject *)op;
}

These are immortal (PEP 683), so they are never freed and their refcounts never hit zero. This is why b"a" is b"a" is always true for a single byte, but b"ab" is b"ab" is generally false.

Allocation

The single allocation routine is _PyBytes_FromSize:

 c
 
 
// Objects/bytesobject.c
static PyObject *
_PyBytes_FromSize(Py_ssize_t size, int use_calloc)
{
 PyBytesObject *op;
 assert(size >= 0);

 if (size == 0) {
 return bytes_get_empty();
 }

 if ((size_t)size > (size_t)PY_SSIZE_T_MAX - PyBytesObject_SIZE) {
 PyErr_SetString(PyExc_OverflowError,
 "byte string is too large");
 return NULL;
 }

 /* Inline PyObject_NewVar */
 if (use_calloc)
 op = (PyBytesObject *)PyObject_Calloc(1, PyBytesObject_SIZE + size);
 else
 op = (PyBytesObject *)PyObject_Malloc(PyBytesObject_SIZE + size);
 if (op == NULL) {
 return PyErr_NoMemory();
 }
 _PyObject_InitVar((PyVarObject*)op, &PyBytes_Type, size);
 set_ob_shash(op, -1);
 if (!use_calloc) {
 op->ob_sval[size] = '\0';
 }
 return (PyObject *) op;
}

Points worth noting:

size 0 collapses to the empty singleton.
use_calloc zeroes the whole allocation (so the trailing NUL and data are already zero); otherwise only ob_sval[size] is written.
There is no freelist for bytes objects (unlike int’s compact freelist). Allocation always goes through PyObject_Malloc/Calloc.
PyBytes_FromString skips _PyBytes_FromSize and inlines the PyObject_Malloc + memcpy(str, size+1) for the common null-terminated case.

PyBytes_FromStringAndSize adds the single-byte shortcut and copies str into ob_sval when str != NULL.

Hashing

bytes_hash caches its result in ob_shash:

 c
 
 
// Objects/bytesobject.c
static Py_hash_t
bytes_hash(PyObject *self)
{
 PyBytesObject *a = _PyBytes_CAST(self);
 Py_hash_t hash = get_ob_shash(a);
 if (hash == -1) {
 /* Can't fail */
 hash = Py_HashBuffer(a->ob_sval, Py_SIZE(a));
 set_ob_shash(a, hash);
 }
 return hash;
}

get_ob_shash/set_ob_shash use relaxed atomics on the free-threaded build (Py_GIL_DISABLED), plain field access otherwise. Py_HashBuffer (in Python/pyhash.c) uses a fast inline DJBX33A path for very short inputs (Py_HASH_CUTOFF) and the configured hash function (SipHash by default) otherwise, and remaps a result of -1 to -2.

Because the hash is cached, _PyBytes_Resize refuses to mutate an object whose hash has already been computed (see below).

Resize (the one mutation escape hatch)

Bytes are immutable from Python, but the C API allows in-place mutation while an object is being built, provided it is uniquely referenced and has not been hashed:

 c
 
 
// Objects/bytesobject.c
int
_PyBytes_ResizeKeepOnError(PyObject **pv, Py_ssize_t newsize)
{
 PyObject *v = *pv;
 ...
 Py_ssize_t oldsize = PyBytes_GET_SIZE(v);
 if (oldsize == newsize) {
 return 0;
 }

 if (oldsize == 0) { /* was the empty singleton */
 result = _PyBytes_FromSize(newsize, 0);
 ...
 }
 if (newsize == 0) { /* becoming empty */
 *pv = bytes_get_empty();
 Py_DECREF(v);
 return 0;
 }

 if (!_PyObject_IsUniquelyReferenced(v)
 || get_ob_shash((PyBytesObject *)v) != -1)
 {
 /* Allocate and then copy so we don't get a shared immortal
 one-character singleton! */
 result = _PyBytes_FromSize(newsize, 0);
 ...
 memcpy(PyBytes_AS_STRING(result), PyBytes_AS_STRING(v),
 Py_MIN(oldsize, newsize));
 *pv = result;
 Py_DECREF(v);
 return 0;
 }

 return bytes_resize_inplace(pv, newsize);
}

bytes_resize_inplace does the actual PyObject_Realloc and re-initializes the object header:

 c
 
 
// Objects/bytesobject.c
static inline int
bytes_resize_inplace(PyObject **pv, Py_ssize_t newsize)
{
 assert(Py_REFCNT(v) == 1);
 assert(PyBytes_GET_SIZE(v) >= 1);
 assert(newsize >= 1);
 assert(_PyBytes_IsMutable(v));

 ...
 PyObject *result = PyObject_Realloc(v, PyBytesObject_SIZE + newsize);
 ...
 *pv = result;
 v = result;
 _Py_NewReferenceNoTotal(v);
 PyBytesObject *sv = (PyBytesObject *)v;
 Py_SET_SIZE(sv, newsize);
 sv->ob_sval[newsize] = '\0';
 ...
}

_PyBytes_Resize (the older variant) additionally sets *pv = NULL and decrefs on failure. _PyBytes_IsMutable is a debug-only check that the object is uniquely referenced, not immortal, and not one of the empty/single-byte singletons.

PyBytesWriter (in the same file) builds on this: it starts with a 256-byte stack buffer (small_buffer) and promotes to a heap bytes object via the same resize machinery, which is how bytes formatting avoids many small allocations.

Operations

A few representative hot paths:

Concatenation (_PyBytes_Concat) gets buffers from both operands, handles empty inputs by returning the other operand, and otherwise allocates a new bytes object and memcpys both parts.
Repeat (bytes_repeat) returns the same object when n == 1 or the size is 0, otherwise allocates and uses _PyBytes_Repeat (a doubling copy) to fill the result.
Slicing (bytes_subscript) returns the empty singleton for empty results, returns Py_NewRef(self) when the slice is the whole object, and uses PyBytes_FromStringAndSize(ptr, slicelength) otherwise.
Indexing returns a cached single-byte singleton via _PyLong_FromUnsignedChar.
Buffer protocol (bytes_buffer_getbuffer) exposes ob_sval with PyBuffer_FillInfo, so bytes participates in the buffer API with no copy.

Key takeaways

bytes is a PyVarObject whose ob_size is the length and whose payload is ob_sval, a char array of size + 1 bytes ending in a NUL.
Allocation size is _PyBytesObject_SIZE + n, saving a few bytes over sizeof(PyBytesObject).
There is no freelist; allocation always uses PyObject_Malloc/Calloc.
The empty bytes and all 256 single-byte objects are immortal singletons, returned by the construction functions.
The hash is lazily computed and cached in ob_shash.
In-place resize is allowed only for uniquely referenced, unhashed, non-singleton objects; otherwise a copy is made.

Bytes on Opensource internals

How CPython Implements bytes in C

The object struct

Memory layout and `_PyBytesObject_SIZE`

Singletons: empty and single characters

Allocation

Hashing

Resize (the one mutation escape hatch)

Operations

Key takeaways

Further reading

Bytes on Opensource internals

How CPython Implements bytes in C

The object struct

Memory layout and _PyBytesObject_SIZE

Singletons: empty and single characters

Allocation

Hashing

Resize (the one mutation escape hatch)

Operations

Key takeaways

Further reading

Memory layout and `_PyBytesObject_SIZE`