Opensource internals

CPython's memoryview Implementation in C

9 min read Jay Hubbert

memoryview is two objects: a shared managed-buffer snapshot and a per-view copy of shape/strides/suboffsets. Here's how slicing stays zero-copy.

TL;DR: memoryview is really two objects: a shared _PyManagedBufferObject snapshot of the exporter’s buffer, and a PyMemoryViewObject with its own shape/strides/suboffsets. Slicing creates a new view and nudges the pointer — no data is copied.

Python’s memoryview is a zero-copy view over an object that implements the buffer protocol. In CPython it is implemented in Objects/memoryobject.c by two cooperating objects: _PyManagedBufferObject (a shared snapshot of the exporter’s buffer) and PyMemoryViewObject (a per-view private copy of that buffer with its own shape/strides/suboffsets). This article traces the representation and the hot paths through the 3.15 source.

The object structs

PyMemoryViewObject is a variable-size object. Its ob_size (via PyObject_VAR_HEAD) is 3 * ndim, and the trailing ob_array holds three parallel arrays: shape, strides, and suboffsets.

c 
// Include/cpython/memoryobject.h
typedef struct {
    PyObject_VAR_HEAD
    _PyManagedBufferObject *mbuf; /* managed buffer */
    Py_hash_t hash;               /* hash value for read-only views */
    int flags;                    /* state flags */
    Py_ssize_t exports;           /* number of buffer re-exports */
    Py_buffer view;               /* private copy of the exporter's view */
    PyObject *weakreflist;
    Py_ssize_t ob_array[1];       /* shape, strides, suboffsets */
} PyMemoryViewObject;

The managed buffer is a separate GC object that owns the snapshot taken from the original exporter:

c 
// Include/cpython/memoryobject.h
typedef struct {
    PyObject_HEAD
    int flags;          /* state flags */
    Py_ssize_t exports; /* number of direct memoryview exports */
    Py_buffer master;   /* snapshot buffer obtained from the original exporter */
} _PyManagedBufferObject;

The type objects reflect this layout. PyMemoryView_Type has tp_basicsize = offsetof(PyMemoryViewObject, ob_array) and tp_itemsize = sizeof(Py_ssize_t), so ndim dimensions allocate 3 * ndim extra Py_ssize_t slots:

c 
// Objects/memoryobject.c
PyTypeObject PyMemoryView_Type = {
    ...
    offsetof(PyMemoryViewObject, ob_array),   /* tp_basicsize */
    sizeof(Py_ssize_t),                       /* tp_itemsize */
    ...
};

Allocation is therefore:

c 
// Objects/memoryobject.c
static inline PyMemoryViewObject *
memory_alloc(int ndim)
{
    PyMemoryViewObject *mv;
    mv = (PyMemoryViewObject *)
        PyObject_GC_NewVar(PyMemoryViewObject, &PyMemoryView_Type, 3*ndim);
    ...
    mv->view.shape = mv->ob_array;
    mv->view.strides = mv->ob_array + ndim;
    mv->view.suboffsets = mv->ob_array + 2 * ndim;
    ...
}

For ndim == 0 (scalar views) the three arrays are NULL and the shape is stored nowhere; the view is treated as a scalar.

Here’s the trick: shape, strides, and suboffsets aren’t separate allocations. They’re three slices of one trailing array, carved up by index. One malloc, three pointers.

The managed buffer idea

The reason two objects exist is ownership. PEP 3118 lets an underlying object change while a view is exported, which would make chained views (a memoryview built from another memoryview) surprising. Instead of repeatedly re-querying the original exporter, CPython takes a single snapshot into a _PyManagedBufferObject:

c 
// Objects/memoryobject.c
static PyObject *
_PyManagedBuffer_FromObject(PyObject *base, int flags)
{
    _PyManagedBufferObject *mbuf;
    mbuf = mbuf_alloc();
    ...
    if (PyObject_GetBuffer(base, &mbuf->master, flags) < 0) {
        ...
    }
    return (PyObject *)mbuf;
}

mbuf->master is filled once by the exporter’s getbufferproc. Every memoryview that shares the same underlying data is registered with the same managed buffer (mv->mbuf). Shape, strides, and suboffsets of the managed buffer’s master are read-only for all consumers; each memoryview gets its own writable copy in mv->view.

Think of the managed buffer as a deed to the property and the memoryview as a surveyor’s drawing of one corner of it. Multiple surveys, one deed.

Building a view

mbuf_add_view is the single constructor used for “complete” views. It allocates the memoryview, then copies/initializes the buffer fields:

c 
// Objects/memoryobject.c
static PyObject *
mbuf_add_view(_PyManagedBufferObject *mbuf, const Py_buffer *src)
{
    ...
    mv = memory_alloc(src->ndim);
    ...
    dest = &mv->view;
    init_shared_values(dest, src);   /* obj, buf, len, itemsize, ... */
    init_shape_strides(dest, src);   /* copy or reconstruct shape/strides */
    init_suboffsets(dest, src);     /* copy suboffsets if present */
    init_flags(mv);                 /* C/F/PIL flags */
    mv->mbuf = (_PyManagedBufferObject*)Py_NewRef(mbuf);
    mbuf->exports++;
    return (PyObject *)mv;
}

init_shared_values copies the immutable-ish fields (including buf, len, itemsize, readonly, format, internal):

c 
// Objects/memoryobject.c
static inline void
init_shared_values(Py_buffer *dest, const Py_buffer *src)
{
    dest->obj = src->obj;
    dest->buf = src->buf;
    dest->len = src->len;
    dest->itemsize = src->itemsize;
    dest->readonly = src->readonly;
    dest->format = src->format ? src->format : "B";
    dest->internal = src->internal;
}

init_shape_strides copies shape and strides, reconstructing missing ones (a 1-D view with shape == NULL derives shape[0] = len / itemsize and strides[0] = itemsize):

c 
// Objects/memoryobject.c
static void
init_shape_strides(Py_buffer *dest, const Py_buffer *src)
{
    ...
    if (src->ndim == 1) {
        dest->shape[0] = src->shape ? src->shape[0] : src->len / src->itemsize;
        dest->strides[0] = src->strides ? src->strides[0] : src->itemsize;
        return;
    }
    for (i = 0; i < src->ndim; i++)
        dest->shape[i] = src->shape[i];
    if (src->strides) { ... copy ... }
    else { init_strides_from_shape(dest); }
}

init_len recomputes len = product(shape) * itemsize after slicing, and init_flags computes the C/F/PIL contiguity flags stored in mv->flags.

Entry points

Creating a memoryview from Python calls memoryview_impl, which delegates to PyMemoryView_FromObject (PyBUF_FULL_RO flags). There are several C entry points:

c 
// Objects/memoryobject.c
PyObject *PyMemoryView_FromObject(PyObject *v);            /* FULL_RO */
PyObject *PyMemoryView_FromMemory(char *mem, Py_ssize_t size, int flags);
PyObject *PyMemoryView_FromBuffer(const Py_buffer *info);
PyObject *_PyMemoryView_FromBufferProc(PyObject *v, int flags, getbufferproc bufferproc);

PyMemoryView_FromMemory exposes a raw byte buffer; it fills a managed buffer with PyBuffer_FillInfo(..., PyBUF_FULL_RO) and then builds a view.

PyMemoryView_FromObject handles the chained case: if v is already a memoryview, it registers a new view with the same managed buffer instead of re-exporting:

c 
// Objects/memoryobject.c
static PyObject *
PyMemoryView_FromObjectAndFlags(PyObject *v, int flags)
{
    if (PyMemoryView_Check(v)) {
        PyMemoryViewObject *mv = (PyMemoryViewObject *)v;
        CHECK_RELEASED(mv);
        CHECK_RESTRICTED(mv);
        return mbuf_add_view(mv->mbuf, &mv->view);
    }
    else if (PyObject_CheckBuffer(v)) {
        ...
        mbuf = (_PyManagedBufferObject *)_PyManagedBuffer_FromObject(v, flags);
        ret = mbuf_add_view(mbuf, NULL);
        Py_DECREF(mbuf);
        return ret;
    }
    ...
}

Buffer protocol export

A memoryview itself implements the buffer protocol via memory_getbuf / memory_releasebuf. On export it copies its private view into the caller’s Py_buffer, then adjusts the requested level of information (format, strides, shape, contiguity) and increments self->exports:

c 
// Objects/memoryobject.c
static int
memory_getbuf(PyObject *_self, Py_buffer *view, int flags)
{
    PyMemoryViewObject *self = (PyMemoryViewObject *)_self;
    Py_buffer *base = &self->view;
    ...
    *view = *base;
    view->obj = NULL;
    ...
    if (!REQ_FORMAT(flags)) { view->format = NULL; }
    if (!REQ_STRIDES(flags)) { ... view->strides = NULL; }
    if (!REQ_SHAPE(flags)) { view->ndim = 1; view->shape = NULL; }
    ...
    view->obj = Py_NewRef(self);
    FT_ATOMIC_ADD_SSIZE(self->exports, 1);
    return 0;
}

memory_releasebuf only decrements self->exports; PyBuffer_Release decrements view->obj afterwards:

c 
// Objects/memoryobject.c
static void
memory_releasebuf(PyObject *_self, Py_buffer *view)
{
    PyMemoryViewObject *self = (PyMemoryViewObject *)_self;
    FT_ATOMIC_ADD_SSIZE(self->exports, -1);
    return;
    /* PyBuffer_Release() decrements view->obj after this function returns. */
}

Gotcha: the export counter on the view and the one on the managed buffer are different counters. A view that has exported itself to NumPy still holds the managed buffer alive; only the last view releases the underlying exporter.

Release and lifetime

_memory_release is called when a view is no longer used (dealloc, release(), or GC clear). It marks the view released and, if it was the last view registered with the managed buffer, releases the underlying buffer:

c 
// Objects/memoryobject.c
static void
_memory_release(PyMemoryViewObject *self)
{
    if (self->flags & _Py_MEMORYVIEW_RELEASED)
        return;
    self->flags |= _Py_MEMORYVIEW_RELEASED;
    assert(self->mbuf->exports > 0);
    if (--self->mbuf->exports == 0) {
        mbuf_release(self->mbuf);
    }
}

mbuf_release calls PyBuffer_Release(&mbuf->master), which decrements the original exporter’s obj. Deallocation then clears mbuf and deletes the object:

c 
// Objects/memoryobject.c
static void
memory_dealloc(PyObject *_self)
{
    PyMemoryViewObject *self = (PyMemoryViewObject *)_self;
    assert(get_exports(self) == 0);
    _PyObject_GC_UNTRACK(self);
    _memory_release(self);
    Py_CLEAR(self->mbuf);
    if (self->weakreflist != NULL)
        PyObject_ClearWeakRefs((PyObject *) self);
    PyObject_GC_Del(self);
}

The get_exports(self) == 0 assertion ensures no outstanding buffers were exported from this view when it dies.

Hashing

Only read-only byte-like views are hashable. memory_hash caches the result in self->hash and, for non-contiguous views, materializes a contiguous copy before hashing:

c 
// Objects/memoryobject.c
memory_hash(PyObject *_self)
{
    PyMemoryViewObject *self = (PyMemoryViewObject *)_self;
    if (self->hash == -1) {
        ...
        if (!view->readonly) { error... }
        ...
        if (view->obj != NULL) {
            FT_ATOMIC_ADD_SSIZE(self->exports, 1);
            Py_hash_t h = PyObject_Hash(view->obj);
            FT_ATOMIC_ADD_SSIZE(self->exports, -1);
            if (h == -1) return -1;
        }
        if (!MV_C_CONTIGUOUS(self->flags)) {
            mem = PyMem_Malloc(view->len);
            ...
            buffer_to_contiguous(mem, view, 'C');
        }
        self->hash = Py_HashBuffer(mem, view->len);
        if (mem != view->buf) PyMem_Free(mem);
    }
    return self->hash;
}

Slicing

Slicing a memoryview does not copy data. memory_subscript allocates a new view registered with the same managed buffer, then adjusts buf, shape, and strides in place via init_slice:

c 
// Objects/memoryobject.c
else if (PySlice_Check(key)) {
    CHECK_RESTRICTED(self);
    PyMemoryViewObject *sliced;
    sliced = (PyMemoryViewObject *)mbuf_add_view(self->mbuf, view);
    if (sliced == NULL) return NULL;
    if (init_slice(&sliced->view, key, 0) < 0) {
        Py_DECREF(sliced);
        return NULL;
    }
    init_len(&sliced->view);
    init_flags(sliced);
    return (PyObject *)sliced;
}

init_slice uses PySlice_Unpack/PySlice_AdjustIndices to compute start/stop/step/slicelength, advances buf by strides[dim] * start, sets shape[dim] = slicelength, and scales strides[dim] *= step. For PIL-style (suboffsets) views it adjusts the relevant suboffset instead:

c 
// Objects/memoryobject.c
static inline int
init_slice(Py_buffer *base, PyObject *key, int dim)
{
    ...
    slicelength = PySlice_AdjustIndices(base->shape[dim], &start, &stop, step);
    if (base->suboffsets == NULL || dim == 0) {
    adjust_buf:
        base->buf = (char *)base->buf + base->strides[dim] * start;
    }
    else {
        ...
        base->suboffsets[n] = base->suboffsets[n] + base->strides[dim] * start;
    }
    base->shape[dim] = slicelength;
    base->strides[dim] = base->strides[dim] * step;
    return 0;
}

Indexing a 1-D view with an integer returns a packed scalar via unpack_single; multi-dimensional sub-views are not implemented.

This is the part people get wrong. Slicing a memoryview does not copy bytes. It moves the base pointer and scales the stride. That’s the whole point — and also why a slice of a memoryview keeps the original exporter alive.

Casting

memoryview.cast reinterprets the bytes with a new format/shape. It requires a C-contiguous view, allocates an incomplete view (shape/strides not yet set), then fills them in:

c 
// Objects/memoryobject.c
mv = (PyMemoryViewObject *)
    mbuf_add_incomplete_view(self->mbuf, &self->view, ndim==0 ? 1 : (int)ndim);
...
if (cast_to_1D(mv, format) < 0) goto error;
if (shape && cast_to_ND(mv, shape, (int)ndim) < 0) goto error;

mbuf_add_incomplete_view only copies shared values; shape/strides/suboffsets are left for cast_to_1D/cast_to_ND to populate. Casts are restricted to 1-D <-> N-D.

State flags

mv->flags packs several predicates used throughout:

c 
// Include/cpython/memoryobject.h
#define _Py_MEMORYVIEW_RELEASED    0x001  /* access to master buffer blocked */
#define _Py_MEMORYVIEW_C           0x002  /* C-contiguous layout */
#define _Py_MEMORYVIEW_FORTRAN     0x004  /* Fortran contiguous layout */
#define _Py_MEMORYVIEW_SCALAR      0x008  /* scalar: ndim = 0 */
#define _Py_MEMORYVIEW_PIL         0x010  /* PIL-style layout */
#define _Py_MEMORYVIEW_RESTRICTED  0x020  /* Disallow new references to the buffer */

CHECK_RELEASED/CHECK_RESTRICTED macros guard entry points against released/restricted views.

Key takeaways

  • memoryview is split into _PyManagedBufferObject (shared snapshot) and PyMemoryViewObject (per-view private copy).
  • PyMemoryViewObject is variable-sized: tp_itemsize = sizeof(Py_ssize_t) and ob_array holds 3 * ndim slots (shape, strides, suboffsets).
  • Views are registered with a managed buffer; chained memoryviews reuse the same managed buffer.
  • Slicing allocates a new view and adjusts buf/shape/strides without copying data.
  • Exporting a memoryview copies its private view and adjusts the requested detail level; release just decrements an export counter.
  • Release of the last view frees the underlying exporter buffer.
  • Hashing is restricted to read-only byte formats and materializes a contiguous copy when needed.

Further reading

  • Objects/memoryobject.c – all of the above.
  • Include/cpython/memoryobject.h – PyMemoryViewObject, _PyManagedBufferObject, flags.
  • Include/pybuffer.h – Py_buffer struct and PyBUF_* flags.
  • Doc/c-api/buffer.rst – buffer protocol documentation.
  • PEP 3118 – revising the buffer protocol.

PythonCpythonInternalsMemoryviewBuffer protocol