How CPython Implements bytearray in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s bytearray is the mutable cousin of bytes: a sequence of raw bytes you can grow, shrink and overwrite in place. In CPython that is implemented by PyByteArrayObject in Objects/bytearrayobject.c, and the clever part is that its backing store is not a plain malloc’d block — it is a real PyBytesObject, reused so that bytes and bytearray share one allocation scheme. This article traces the representation, the resize strategy and the free-threaded locking through the 3.15 source.

The object struct

Every bytearray is a PyObject_VAR_HEAD (refcount, type, length) plus the fields that describe its buffer:

 c
 
 
// Include/cpython/bytearrayobject.h
typedef struct {
 PyObject_VAR_HEAD
 /* How many bytes allocated in ob_bytes. Loaded/stored atomically. */
 Py_ssize_t ob_alloc;
 char *ob_bytes; /* Physical backing buffer */
 char *ob_start; /* Logical start inside ob_bytes */
 Py_ssize_t ob_exports; /* How many buffer exports */
 PyObject *ob_bytes_object; /* PyBytes used as zero-copy storage */
} PyByteArrayObject;

ob_size (from PyObject_VAR_HEAD) is the logical length. ob_bytes is the physical allocation base, ob_start is where the live data begins, and ob_alloc is how many bytes are allocated from ob_start’s point of view… in practice, after the compaction step in resize, ob_start == ob_bytes and ob_alloc equals the size of the backing PyBytes.

The trick that makes all of this work is ob_bytes_object: the actual memory lives inside a PyBytesObject. ob_bytes and ob_start are just pointers into PyBytes_AS_STRING(ob_bytes_object). This means a bytearray and a bytes share the same allocator, the same over-allocation heuristics, and the same trailing-NUL guarantee, and bytes(ba) can in some paths hand over the same buffer.

 c
 
 
// Objects/bytearrayobject.c
static void
bytearray_reinit_from_bytes(PyByteArrayObject *self, Py_ssize_t size)
{
 Py_ssize_t alloc = PyBytes_GET_SIZE(self->ob_bytes_object);
 assert(0 <= size && size <= alloc);

 self->ob_bytes = self->ob_start = PyBytes_AS_STRING(self->ob_bytes_object);
 Py_SET_SIZE(self, size);
 FT_ATOMIC_STORE_SSIZE_RELAXED(self->ob_alloc, alloc);
}

So construction is: allocate a PyBytesObject of the right size, point the bytearray at it, and set the logical length. PyByteArray_FromStringAndSize does exactly that:

 c
 
 
// Objects/bytearrayobject.c
PyObject *
PyByteArray_FromStringAndSize(const char *bytes, Py_ssize_t size)
{
 PyByteArrayObject *new = PyObject_New(PyByteArrayObject, &PyByteArray_Type);
 ...
 new->ob_exports = 0;
 /* size=0 uses the immortal empty bytes, so no allocation. */
 new->ob_bytes_object = PyBytes_FromStringAndSize(NULL, size);
 if (new->ob_bytes_object == NULL) { Py_DECREF(new); return NULL; }
 bytearray_reinit_from_bytes(new, size);
 if (bytes != NULL && size > 0) {
 memcpy(new->ob_bytes, bytes, size);
 }
 return (PyObject *)new;
}

When size == 0, PyBytes_FromStringAndSize(NULL, 0) returns the immortal empty-bytes singleton, so an empty bytearray costs only the object header. bytearray_new (used by bytearray()) does the same with Py_GetConstant(Py_CONSTANT_EMPTY_BYTES).

Access macros

The public macros all go through ob_start / ob_size:

 c
 
 
// Include/cpython/bytearrayobject.h
#define PyByteArray_AS_STRING(self) PyByteArray_AS_STRING(_PyObject_CAST(self))
static inline char* PyByteArray_AS_STRING(PyObject *op) {
 return _PyByteArray_CAST(op)->ob_start;
}
static inline Py_ssize_t PyByteArray_GET_SIZE(PyObject *op) {
 ... /* atomic load under free-threaded build, else Py_SIZE(self) */
}

PyByteArray_GET_SIZE is atomic on Py_GIL_DISABLED builds because ob_size is mutated while other threads may be reading it through the buffer protocol.

Buffer exports and `_canresize`

A bytearray can hand out its memory to C code via the buffer protocol. While such a view is alive, the object must not move or shrink its backing store, otherwise the C pointer would dangle. This is tracked by ob_exports:

 c
 
 
// Objects/bytearrayobject.c
static int
bytearray_getbuffer_lock_held(PyObject *self, Py_buffer *view, int flags)
{
 ...
 void *ptr = (void *) PyByteArray_AS_STRING(obj);
 if (PyBuffer_FillInfo(view, (PyObject*)obj, ptr, Py_SIZE(obj), 0, flags) < 0)
 return -1;
 obj->ob_exports++;
 return 0;
}

static void
bytearray_releasebuffer(PyObject *self, Py_buffer *view)
{
 Py_BEGIN_CRITICAL_SECTION(self);
 PyByteArrayObject *obj = _PyByteArray_CAST(self);
 obj->ob_exports--;
 assert(obj->ob_exports >= 0);
 Py_END_CRITICAL_SECTION();
}

_canresize then simply refuses any resize (and __init__) while exports are outstanding:

 c
 
 
// Objects/bytearrayobject.c
static int
_canresize(PyByteArrayObject *self)
{
 if (self->ob_exports > 0) {
 PyErr_SetString(PyExc_BufferError,
 "Existing exports of data: object cannot be re-sized");
 return 0;
 }
 return 1;
}

bytearray_dealloc even prints a SystemError if an object is freed while exports remain — that is a programming error in whoever held the buffer.

A neat trick: methods that read the buffer while calling back into Python (such as join or the bytes-search helpers) bump ob_exports themselves to pin the storage for the duration, even without a real external export:

 c
 
 
// Objects/bytearrayobject.c (bytearray_join_impl)
self->ob_exports++; // protects `self` from being cleared/resized
ret = stringlib_bytes_join((PyObject*)self, iterable_of_bytes);
self->ob_exports--; // unexport `self`

Resize: the heart of the type

PyByteArray_Resize is the public entry point and it is the only path that changes the size. It takes the object lock, then runs the strategy in bytearray_resize_lock_held:

 c
 
 
// Objects/bytearrayobject.c
int
PyByteArray_Resize(PyObject *self, Py_ssize_t requested_size)
{
 int ret;
 Py_BEGIN_CRITICAL_SECTION(self);
 ret = bytearray_resize_lock_held(self, requested_size);
 Py_END_CRITICAL_SECTION();
 return ret;
}

The strategy has several cases:

Same size — nothing to do.
Resize to 0 — reset to the immortal empty bytes.
Current buffer big enough (size + offset <= alloc):
- major downsize (size < alloc/2): shrink the backing PyBytes to exactly size;
- minor downsize: just update ob_size and write the trailing NUL.
Need to grow:
- moderate upsize (size <= alloc * 1.125): over-allocate like list_resize, i.e. alloc = size + (size >> 3) + (size < 9 ? 3 : 6);
- major upsize: allocate exactly size.

Then alloc is clamped to PyByteArray_SIZE_MAX and bytearray_resize_storage does the actual work.

There is also a subtlety: because ob_start may not equal ob_bytes (the data can be shifted inside the buffer after slices/take_bytes), resize first compacts by memmove-ing the live bytes to the base before resizing the PyBytes. On failure it undoes the partial move.

 c
 
 
// Objects/bytearrayobject.c
static int
bytearray_resize_storage(PyByteArrayObject *self,
 Py_ssize_t new_size, Py_ssize_t alloc)
{
 _Py_CRITICAL_SECTION_ASSERT_OBJECT_LOCKED(self);
 ...
 char *old_start = self->ob_start;
 if (self->ob_start != self->ob_bytes) {
 Py_ssize_t move = Py_MIN(new_size, size);
 memmove(self->ob_bytes, self->ob_start, move);
 self->ob_start = self->ob_bytes;
 }
 if (_PyBytes_ResizeKeepOnError(&self->ob_bytes_object, alloc) < 0) {
 if (old_start != self->ob_bytes && new_size < size) {
 /* undo the compaction */
 Py_ssize_t moved = new_size;
 Py_ssize_t remaining = size - moved;
 memmove(self->ob_bytes + moved, old_start + moved, remaining);
 }
 bytearray_write_trailing_null_byte(self);
 return -1;
 }
 return 0;
}

After storage resize, bytearray_reinit_from_bytes repoints ob_bytes / ob_start at the (possibly new) buffer and refreshes ob_alloc. The trailing NUL is written whenever the allocation differs from the logical size, keeping PyByteArray_AS_STRING C-string safe.

`take_bytes`: zero-copy shrink

bytearray even has a __alloc__-style fast path for splitting off the tail: instead of copying, it can hand the front PyBytes to the caller and re-point the bytearray at a freshly allocated tail. This is the _bytearray_take_bytes logic around line 1640 — it swaps ob_bytes_object so the returned bytes and the surviving bytearray no longer share storage. Worth knowing it exists: it is why repeated pop/take patterns are cheaper than they look.

Locking and free-threading

CPython 3.15 ships the free-threaded (no-GIL) build, and bytearray is a shared, mutable buffer, so almost every method is wrapped in a critical section. The pattern is mechanical:

 c
 
 
// Objects/bytearrayobject.c
static PyObject *
bytearray_some_op_impl(PyByteArrayObject *self, ...)
{
 _Py_CRITICAL_SECTION_ASSERT_OBJECT_LOCKED(self);
 ...
}

// wrapper
PyObject *bytearray_some_op(...) {
 ...
 Py_BEGIN_CRITICAL_SECTION(self);
 ret = bytearray_some_op_impl(self, ...);
 Py_END_CRITICAL_SECTION();
 return ret;
}

Clinic-generated code carries @critical_section annotations that produce these wrappers automatically. Inside the lock, ob_alloc is read with FT_ATOMIC_LOAD_SSIZE_RELAXED and written with FT_ATOMIC_STORE_SSIZE_RELAXED so that even lock-free readers (like __alloc__, which only reads ob_alloc) observe a consistent value:

 c
 
 
// Objects/bytearrayobject.c (bytearray_alloc)
Py_ssize_t alloc = FT_ATOMIC_LOAD_SSIZE_RELAXED(self->ob_alloc);
if (alloc > 0) {
 alloc += _PyBytesObject_SIZE;
}
return PyLong_FromSsize_t(alloc);

ob_size itself is loaded atomically in PyByteArray_GET_SIZE on free-threaded builds. The net effect: a bytearray is safe to read and resize concurrently with other threads, with resizes serialized per-object and buffer exports acting as a read-lock against mutation.

Type object

 c
 
 
// Objects/bytearrayobject.c
PyTypeObject PyByteArray_Type = {
 ...
 sizeof(PyByteArrayObject),
 bytearray_dealloc,
 ...
 &bytearray_as_buffer, /* tp_as_buffer */
 ...
 PyType_GenericAlloc, /* tp_alloc */
 bytearray_new, /* tp_new */
 PyObject_Free, /* tp_free */
};

tp_as_buffer points at bytearray_getbuffer / bytearray_releasebuffer, which is what makes bytearray support the buffer protocol (and therefore memoryview, bytes(ba), file writes, etc.). tp_hash is 0, so bytearray is unhashable, consistent with mutability.

Memory layout summary

text

+----------------------------------+ <- PyObject_New result (PyObject_Free)
| ob_refcnt |
+----------------------------------+
| *ob_type (&PyByteArray_Type) |
+----------------------------------+
| ob_size (logical length) |
+----------------------------------+
| ob_alloc (atomic; backing size) |
+----------------------------------+
| *ob_bytes (base of buffer) |
+----------------------------------+
| *ob_start (live data start) |
+----------------------------------+
| ob_exports (buffer view count) |
+----------------------------------+
| *ob_bytes_object (PyBytes) |---+
+----------------------------------+ |
 v
 +-------------------+
 | PyBytesObject |
 | ... ob_sval[] | <- the actual bytes
 +-------------------+

The bytearray object is small and fixed-size; all variable storage lives in the referenced PyBytesObject, which is resized in place (or replaced) during PyByteArray_Resize.

Key takeaways

bytearray is a PyObject_VAR_HEAD plus ob_alloc, ob_bytes, ob_start, ob_exports and ob_bytes_object.
The backing store is a PyBytesObject; ob_bytes/ob_start are pointers into it, so bytearray reuses bytes’ allocator and NUL-termination.
Empty bytearrays reuse the immortal empty bytes, costing only the header.
ob_exports pins the buffer against resize/dealloc while views are live.
Resize uses a list-like over-allocation strategy and compacts ob_start back to ob_bytes first.
Every mutating path is a critical section, with ob_alloc/ob_size read atomically for the free-threaded build.

Bytearray on Opensource internals

How CPython Implements bytearray in C

The object struct

Access macros

Buffer exports and `_canresize`

Resize: the heart of the type

`take_bytes`: zero-copy shrink

Locking and free-threading

Type object

Memory layout summary

Key takeaways

Further reading

Bytearray on Opensource internals

How CPython Implements bytearray in C

The object struct

Access macros

Buffer exports and _canresize

Resize: the heart of the type

take_bytes: zero-copy shrink

Locking and free-threading

Type object

Memory layout summary

Key takeaways

Further reading

Buffer exports and `_canresize`

`take_bytes`: zero-copy shrink