Python on Opensource internals

CPython's Weak Reference Implementation in C (weakref)

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s weak references – weakref.ref, weakref.proxy, the machinery behind weakref.WeakValueDictionary and __weakref__ – are implemented in CPython in Objects/weakrefobject.c as a single C struct, PyWeakReference, that is reused for three Python types: ReferenceType, ProxyType, and CallableProxyType. A weak reference is a “stealth” pointer: it points at an object without incrementing that object’s reference count, and when the object dies every weak reference to it is cleared (and any callbacks run). This article traces that implementation through the 3.15 source.

The object: `PyWeakReference`

The layout is defined in Include/cpython/weakrefobject.h:

 c
 
 
// Include/cpython/weakrefobject.h
struct _PyWeakReference {
 PyObject_HEAD

 /* The object to which this is a weak reference, or Py_None if none.
 * Note that this is a stealth reference: wr_object's refcount is
 * not incremented to reflect this pointer. */
 PyObject *wr_object;

 /* A callable to invoke when wr_object dies, or NULL if none. */
 PyObject *wr_callback;

 /* A cache for wr_object's hash code. -1 if unknown. */
 Py_hash_t hash;

 /* Doubly-linked list pointers into the referenced object's weakref list. */
 PyWeakReference *wr_prev;
 PyWeakReference *wr_next;
 vectorcallfunc vectorcall;

#ifdef Py_GIL_DISABLED
 PyMutex *weakrefs_lock;
#endif
};

Invariants:

wr_object is either a live object or Py_None. Py_None is the sentinel for “dead”; it is not a real reference, so no Py_DECREF is owed for it.
wr_object is a stealth reference: its refcount is never incremented to account for this pointer. This is the entire point of a weak reference.
wr_callback is an owned (refcounted) reference, or NULL.
The object is GC-tracked (Py_TPFLAGS_HAVE_GC) because it owns wr_callback (and, transitively, may keep a dead ref’s callback alive).
All three Python types (ReferenceType, ProxyType, CallableProxyType) share this same struct; only their tp_* slots differ.
PyWeakref_CheckRef, PyWeakref_CheckProxy, PyWeakref_Check are the type tests. PyWeakref_CheckRefExact is an exact-type test used for the “basic ref” fast paths.

The per-object weakref list

Every object that supports weak references carries a doubly-linked, NULL-terminated list of the weak references pointing at it. The head of that list is stored in the object itself, at an offset given by the type’s tp_weaklistoffset:

 c
 
 
// Include/internal/pycore_object.h
static inline PyObject **
_PyObject_GET_WEAKREFS_LISTPTR(PyObject *op)
{
 Py_ssize_t offset = Py_TYPE(op)->tp_weaklistoffset;
 return (PyObject **)((char *)op + offset);
}

Support is gated by _PyType_SUPPORTS_WEAKREFS(type) which simply checks type->tp_weaklistoffset != 0. Concrete examples in the source:

 c
 
 
// Objects/funcobject.c
.tp_weaklistoffset = offsetof(PyFunctionObject, func_weakreflist),
// Objects/genobject.c
.tp_weaklistoffset = offsetof(PyGenObject, gi_weakreflist),
// Objects/moduleobject.c
.tp_weaklistoffset = offsetof(PyModuleObject, md_weaklist),
// Objects/setobject.c
.tp_weaklistoffset = offsetof(PySetObject, weakreflist),

For heap (user-defined) classes, __weakref__ is added automatically unless __slots__ forbids it. In typeobject.c this is recorded as a managed weakref slot:

 c
 
 
// Objects/typeobject.c
if (ctx->add_weak) {
 type_add_flags(type, Py_TPFLAGS_MANAGED_WEAKREF);
 type->tp_weaklistoffset = MANAGED_WEAKREF_OFFSET; // negative sentinel
}

MANAGED_WEAKREF_OFFSET is a negative offset (-2 or -4 sizeof(PyObject*), depending on the build) into the managed pointer area just before the instance dict, so the head pointer lives in the object’s pre-header region rather than in a named field. subtype_getweakref exposes this as the __weakref__ attribute and returns the head weakref (or None):

 c
 
 
// Objects/typeobject.c
static PyObject *
subtype_getweakref(PyObject *obj, void *context)
{
 ...
 weaklistptr = (PyObject **)((char *)obj + type->tp_weaklistoffset);
 if (*weaklistptr == NULL) {
 result = Py_NewRef(Py_None);
 }
 else if (_Py_TryIncref(*weaklistptr)) {
 result = *weaklistptr;
 }
 else {
 result = Py_NewRef(Py_None);
 }
 ...
}

Construction: `get_or_create_weakref`

The public entry points are PyWeakref_NewRef and PyWeakref_NewProxy:

 c
 
 
// Objects/weakrefobject.c
PyObject *
PyWeakref_NewRef(PyObject *ob, PyObject *callback)
{
 return (PyObject *)get_or_create_weakref(&_PyWeakref_RefType, ob, callback);
}

PyObject *
PyWeakref_NewProxy(PyObject *ob, PyObject *callback)
{
 PyTypeObject *type = &_PyWeakref_ProxyType;
 if (PyCallable_Check(ob)) {
 type = &_PyWeakref_CallableProxyType;
 }
 return (PyObject *)get_or_create_weakref(type, ob, callback);
}

get_or_create_weakref does the real work. First it validates that the target supports weak references, normalizes callback == Py_None to NULL, then:

 c
 
 
// Objects/weakrefobject.c
PyWeakReference **list = GET_WEAKREFS_LISTPTR(obj);

if ((type == &_PyWeakref_RefType) ||
 (type == &_PyWeakref_ProxyType) ||
 (type == &_PyWeakref_CallableProxyType))
{
 LOCK_WEAKREFS(obj);
 PyWeakReference *basic_ref = try_reuse_basic_ref(*list, type, callback);
 if (basic_ref != NULL) {
 UNLOCK_WEAKREFS(obj);
 return basic_ref; // reused an existing callback-less ref
 }
 PyWeakReference *newref = allocate_weakref(type, obj, callback);
 if (newref == NULL) { UNLOCK_WEAKREFS(obj); return NULL; }
 insert_weakref(newref, list);
 UNLOCK_WEAKREFS(obj);
 return newref;
}
else {
 // Subtypes / custom types: allocate outside the lock
 PyWeakReference *newref = allocate_weakref(type, obj, callback);
 if (newref == NULL) return NULL;
 LOCK_WEAKREFS(obj);
 insert_weakref(newref, list);
 UNLOCK_WEAKREFS(obj);
 return newref;
}

Two important optimizations:

Basic-ref reuse. A weak reference with no callback of the exact basic type (weakref.ref / weakref.proxy) is “canonical”: at most one of each kind (ref and proxy) is kept at the head of the list. try_reuse_basic_ref returns an existing one instead of allocating a new object, incrementing its refcount. This is why weakref.ref(obj) is weakref.ref(obj) can be true for callback-less refs.
Lock discipline. For the three builtin types allocation happens inside the lock (cheap, tp_alloc); for subtypes, allocation happens outside the lock because a subtype’s tp_alloc may run arbitrary Python code.

init_weakref stores the stealth pointer and owns the callback:

 c
 
 
// Objects/weakrefobject.c
static void
init_weakref(PyWeakReference *self, PyObject *ob, PyObject *callback)
{
 self->hash = -1;
 self->wr_object = ob; // stealth: no incref
 self->wr_prev = NULL;
 self->wr_next = NULL;
 self->wr_callback = Py_XNewRef(callback);
 self->vectorcall = weakref_vectorcall;
#ifdef Py_GIL_DISABLED
 self->weakrefs_lock = &WEAKREF_LIST_LOCK(ob);
 _PyObject_SetMaybeWeakref(ob);
 _PyObject_SetMaybeWeakref((PyObject *)self);
#endif
}

allocate_weakref uses type->tp_alloc(type, 0) (so subtypes allocate correctly) and then init_weakref.

Insertion ordering

The list is kept in a canonical order so the basic (callback-less) refs sit at the head:

 c
 
 
// Objects/weakrefobject.c
static void
insert_weakref(PyWeakReference *newref, PyWeakReference **list)
{
 PyWeakReference *ref, *proxy;
 get_basic_refs(*list, &ref, &proxy);

 PyWeakReference *prev;
 if (is_basic_ref(newref)) {
 prev = NULL; // head
 }
 else if (is_basic_proxy(newref)) {
 prev = ref; // after the basic ref
 }
 else {
 prev = (proxy == NULL) ? ref : proxy; // after both basic refs
 }

 if (prev == NULL) insert_head(newref, list);
 else insert_after(newref, prev);
}

insert_head/insert_after splice the node into the doubly-linked list and update wr_prev/wr_next. The first node’s wr_prev stays NULL and acts as the list head stored in the object.

Reading a weak reference

The core accessor is _PyWeakref_GET_REF (inlined in Include/internal/pycore_weakref.h):

 c
 
 
// Include/internal/pycore_weakref.h
static inline PyObject* _PyWeakref_GET_REF(PyObject *ref_obj)
{
 PyWeakReference *ref = _Py_CAST(PyWeakReference*, ref_obj);
 PyObject *obj = FT_ATOMIC_LOAD_PTR(ref->wr_object);
 if (obj == Py_None) {
 return NULL; // cleared
 }

 LOCK_WEAKREFS(obj);
#ifdef Py_GIL_DISABLED
 if (ref->wr_object == Py_None) { // re-check under lock
 UNLOCK_WEAKREFS(obj);
 return NULL;
 }
#endif
 if (_Py_TryIncref(obj)) { // stealth -> real ref, if alive
 UNLOCK_WEAKREFS(obj);
 return obj; // new reference (caller owns)
 }
 UNLOCK_WEAKREFS(obj);
 return NULL;
}

Because wr_object is stealth, reading it requires trying to incref the target under the lock: if the target is already dead (refcount == 0), the incref fails and NULL is returned. On success the caller receives a new owned reference.

The public API wraps this:

 c
 
 
// Objects/weakrefobject.c
int
PyWeakref_GetRef(PyObject *ref, PyObject **pobj)
{
 ...
 *pobj = _PyWeakref_GET_REF(ref);
 return (*pobj != NULL); // 1 if live, 0 if dead
}

PyWeakref_IsDead / _PyWeakref_IS_DEAD additionally handle the GC/trashcan delay case: a target whose refcount hit zero but whose weakrefs have not yet been cleared is still reported as dead via _is_dead().

The ref object itself is callable (via vectorcall), so ref() returns the target or None:

 c
 
 
// Objects/weakrefobject.c
static PyObject *
weakref_vectorcall(PyObject *self, ...)
{
 PyObject *obj = _PyWeakref_GET_REF(self);
 if (obj == NULL) {
 Py_RETURN_NONE;
 }
 return obj; // new reference
}

Hashing and equality

weakref_hash caches the target’s hash in wr_hash under the weakref’s own critical section; a dead ref raises TypeError: weak object has gone away.
weakref_richcompare compares targets: if either is dead, equality reduces to identity of the two ref objects; otherwise it delegates to PyObject_RichCompare on the targets.

Clearing: when the target dies

The central routine is clear_weakref_lock_held, which removes a node from the target’s list and flips wr_object to Py_None:

 c
 
 
// Objects/weakrefobject.c
static void
clear_weakref_lock_held(PyWeakReference *self, PyObject **callback)
{
 if (self->wr_object != Py_None) {
 PyWeakReference **list = GET_WEAKREFS_LISTPTR(self->wr_object);
 if (*list == self) {
 FT_ATOMIC_STORE_PTR(*list, self->wr_next);
 }
 FT_ATOMIC_STORE_PTR(self->wr_object, Py_None);
 if (self->wr_prev != NULL) self->wr_prev->wr_next = self->wr_next;
 if (self->wr_next != NULL) self->wr_next->wr_prev = self->wr_prev;
 self->wr_prev = NULL;
 self->wr_next = NULL;
 }
 if (callback != NULL) {
 *callback = self->wr_callback; // steal
 self->wr_callback = NULL;
 }
}

Note that the callback is stolen, not invoked, here. Clearing happens under the weakref list lock, and invoking arbitrary Python code under that lock is forbidden (especially in free-threaded builds where the lock is _Py_LOCK_DONT_DETACH). So clear_weakref_lock_held only detaches the callback; the caller decides when to run it.

There are three callers:

PyObject_ClearWeakRefs(object) – called from tp_dealloc when the target’s refcount hits zero. It clears every weakref and runs callbacks in two phases:
- First, the callback-less basic ref/proxy at the head of the list are cleared under the lock (no callbacks to run).
- Then each remaining ref is cleared under the lock, its callback stolen, and the (ref, callback) pair stored in a tuple. After the lock is released, callbacks are invoked via handle_callback:
  c
  static void handle_callback(PyWeakReference *ref, PyObject *callback) { PyObject *cbresult = PyObject_CallOneArg(callback, (PyObject *)ref); if (cbresult == NULL) { PyErr_FormatUnraisable("Exception ignored while " "calling weakref callback %R", callback); } else { Py_DECREF(cbresult); } }
- The original exception (if any) is saved and restored around this, so a failing callback does not clobber an in-flight exception.
GC (gc_clear in weakrefobject.c and the inlined copy in gcmodule.c’s handle_weakrefs). During collection, weakrefs are cleared but callbacks are deliberately left intact and uncalled, because running Python code mid-GC is unsafe. _PyWeakref_ClearRef (note: capital R) is the GC-facing variant that clears the ref while leaving the callback for tp_dealloc to decref later.
Deallocation of the weakref itself (weakref_dealloc / proxy_dealloc) calls clear_weakref (which locks and then Py_XDECREFs the stolen callback) and then tp_free.

_PyWeakref_ClearWeakRefsNoCallbacks and _PyStaticType_ClearWeakRefs are further variants used for bulk clearing without callbacks (e.g. finalization of static types).

The callback member

wr_callback is exposed read-only to Python as __callback__:

 c
 
 
// Objects/weakrefobject.c
static PyMemberDef weakref_members[] = {
 {"__callback__", _Py_T_OBJECT,
 offsetof(PyWeakReference, wr_callback), Py_READONLY},
 {NULL}
};

weakref.ref.__init__ is a no-op (the real construction happens in __new__), so ref(obj, callback) creates a new weakref with a callback that cannot be reused via the basic-ref fast path (callbacks force a fresh node).

Proxy objects

ProxyType and CallableProxyType are the same PyWeakReference struct, but instead of being callable to retrieve the target, they forward every operation to the target. The implementation is a set of macros that unwrap the proxy into the real object, call the generic C API, and decref:

 c
 
 
// Objects/weakrefobject.c
#define UNWRAP(o) \
 if (PyWeakref_CheckProxy(o)) { \
 o = _PyWeakref_GET_REF(o); \
 if (!proxy_check_ref(o)) { return NULL; } // ReferenceError if dead \
 } else { \
 Py_INCREF(o); \
 }

#define WRAP_UNARY(method, generic) \
 static PyObject * method(PyObject *proxy) { \
 UNWRAP(proxy); \
 PyObject* res = generic(proxy); \
 Py_DECREF(proxy); \
 return res; \
 }

#define WRAP_BINARY(method, generic) \
 static PyObject * method(PyObject *x, PyObject *y) { \
 UNWRAP(x); UNWRAP(y); \
 PyObject* res = generic(x, y); \
 Py_DECREF(x); Py_DECREF(y); \
 return res; \
 }

proxy_check_ref raises ReferenceError: weakly-referenced object no longer exists when the target is gone. These macros generate the number slots (proxy_add, proxy_sub, …), sequence/mapping slots (proxy_length, proxy_getitem, proxy_setitem, proxy_contains), iterator slots (proxy_iter, proxy_iternext), and attribute access (proxy_getattr, proxy_setattr). CallableProxyType additionally sets tp_call = proxy_call, which unwraps and calls the target.

Notable details:

Proxies have no tp_hash (intentionally omitted) because they are “mutable” once the reference dies.
proxy_bool, proxy_repr, etc. fetch the target via _PyWeakref_GET_REF directly rather than through UNWRAP.
proxy_dealloc is the same as weakref_dealloc (clear_weakref + PyObject_GC_Del).

Free-threading

In the no-GIL build, weakref state is protected by striped locks keyed on the target’s address (WEAKREF_LIST_LOCK(obj)), acquired with _Py_LOCK_DONT_DETACH so that weakref clearing can run during operations like WeakValueDictionary deletion that must not suspend. The weakref’s own weakrefs_lock pointer is captured at creation (so a ref can be locked even after its target has been set to Py_None). _PyWeakref_GET_REF and _PyWeakref_IS_DEAD take and release these locks; _PyObject_SetMaybeWeakref marks both the target and the ref itself so the runtime knows a weakref may exist. The GC is exempt from locking because the world is stopped during collection.

Summary of the data flow

text

weakref.ref(obj, callback)
 -> get_or_create_weakref(RefType, obj, callback)
 -> supports weakrefs? (tp_weaklistoffset != 0)
 -> list = GET_WEAKREFS_LISTPTR(obj) // head pointer in obj
 -> reuse basic ref if callback is None and one exists
 -> allocate_weakref -> tp_alloc + init_weakref (stealth wr_object,
 owned wr_callback)
 -> insert_weakref into doubly-linked list (basic refs at head)
 -> return ref

ref() -> _PyWeakref_GET_REF -> try incref target under lock -> new ref or None

target dies (refcount 0) -> tp_dealloc -> PyObject_ClearWeakRefs(target)
 -> clear each node (wr_object = Py_None, unlink)
 -> steal callbacks, invoke them outside the lock
 -> ref() now returns None; ref.__callback__ is None

Key takeaways

weakref.ReferenceType, ProxyType, and CallableProxyType are all PyWeakReference; the only differences are their tp_* slots.
wr_object is a stealth (non-owning) pointer; Py_None marks death. Reading it requires _Py_TryIncref under the target’s lock.
Each weakref-able object stores the head of a doubly-linked list of weakrefs at tp_weaklistoffset (a real offset for builtin types, a managed negative sentinel for heap classes with __weakref__).
Callback-less basic refs (one ref + one proxy) are reused instead of reallocated; refs with callbacks are always new nodes.
Clearing unlinks the node and steals the callback; callbacks run outside the lock (and are deferred entirely during GC).
Proxies forward every operation by unwrapping to the target via UNWRAP, raising ReferenceError when the target is gone.
Free-threaded builds protect the list with striped, non-detaching locks keyed on the target’s address.

CPython's Union Type Implementation in C (X | Y)

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s X | Y syntax (and typing.Union[X, Y]) produces an object of type types.UnionType. Unlike Ellipsis or None, a union is a real allocated object: it owns references to its arguments, participates in garbage collection, and can be weakly referenced. The whole implementation lives in Objects/unionobject.c. This article traces it through the 3.15 source.

The struct: `unionobject`

 c
 
 
// Objects/unionobject.c
typedef struct {
 PyObject_HEAD
 PyObject *args; // all args (tuple)
 PyObject *hashable_args; // frozenset or NULL
 PyObject *unhashable_args; // tuple or NULL
 PyObject *parameters;
 PyObject *weakreflist;
} unionobject;

args is the ordered, deduplicated tuple of members, in the order they were first seen. This is what __args__ exposes and what repr iterates.
hashable_args is a frozenset of the members that hash successfully. It exists so that two unions can be compared/hashed cheaply.
unhashable_args is a tuple of members that do not hash (e.g. lists, or types containing unhashable parameters). It may be NULL.
parameters is lazily computed (type variables found in args), guarded by a critical section.
weakreflist supports weakref.ref(union).

The type: `_PyUnion_Type`

 c
 
 
// Objects/unionobject.c
PyTypeObject _PyUnion_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 .tp_name = "typing.Union",
 .tp_doc = PyDoc_STR("Represent a union type\n\nE.g. for int | str"),
 .tp_basicsize = sizeof(unionobject),
 .tp_dealloc = unionobject_dealloc,
 .tp_alloc = PyType_GenericAlloc,
 .tp_free = PyObject_GC_Del,
 .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
 .tp_traverse = union_traverse,
 .tp_hash = union_hash,
 .tp_getattro = union_getattro,
 .tp_members = union_members, // __args__
 .tp_methods = union_methods, // __mro_entries__, __class_getitem__
 .tp_richcompare = union_richcompare,
 .tp_as_mapping = &union_as_mapping, // __getitem__
 .tp_as_number = &union_as_number, // __or__
 .tp_repr = union_repr,
 .tp_getset = union_properties, // __name__, __qualname__, __origin__, __parameters__
 .tp_weaklistoffset = offsetof(unionobject, weakreflist),
};

Key points:

tp_name is "typing.Union" even though Python exposes it as types.UnionType. types.UnionType = type(int | str) (see Lib/types.py), and _typesmodule.c exports _PyUnion_Type under the name "UnionType".
It is a GC type (Py_TPFLAGS_HAVE_GC, tp_traverse, tp_free = PyObject_GC_Del), because it owns references to arbitrary objects.
It is not a heap type and is statically allocated, so it is immortal and safe to export with EXPORT_STATIC_TYPE.
tp_weaklistoffset enables weak references.

It is exported here:

 c
 
 
// Modules/_typesmodule.c
EXPORT_STATIC_TYPE("UnionType", _PyUnion_Type);

Deallocation and GC

 c
 
 
// Objects/unionobject.c
static void
unionobject_dealloc(PyObject *self)
{
 unionobject *alias = (unionobject *)self;

 _PyObject_GC_UNTRACK(self);
 FT_CLEAR_WEAKREFS(self, alias->weakreflist);

 Py_XDECREF(alias->args);
 Py_XDECREF(alias->hashable_args);
 Py_XDECREF(alias->unhashable_args);
 Py_XDECREF(alias->parameters);
 Py_TYPE(self)->tp_free(self);
}

static int
union_traverse(PyObject *self, visitproc visit, void *arg)
{
 unionobject *alias = (unionobject *)self;
 Py_VISIT(alias->args);
 Py_VISIT(alias->hashable_args);
 Py_VISIT(alias->unhashable_args);
 Py_VISIT(alias->parameters);
 return 0;
}

FT_CLEAR_WEAKREFS clears weak references before freeing. tp_free is PyObject_GC_Del, matching GC allocation.

Equality and hashing

A union is hashable only if all of its members are hashable. The split into hashable_args (a frozenset) and unhashable_args (a tuple) lets equality and hashing be computed without re-hashing members on every call:

 c
 
 
// Objects/unionobject.c
static Py_hash_t
union_hash(PyObject *self)
{
 unionobject *alias = (unionobject *)self;
 if (alias->unhashable_args) {
 // Attempt to get an error from one of the values.
 assert(PyTuple_CheckExact(alias->unhashable_args));
 Py_ssize_t n = PyTuple_GET_SIZE(alias->unhashable_args);
 for (Py_ssize_t i = 0; i < n; i++) {
 PyObject *arg = PyTuple_GET_ITEM(alias->unhashable_args, i);
 Py_hash_t hash = PyObject_Hash(arg);
 if (hash == -1) {
 return -1;
 }
 }
 // The unhashable values somehow became hashable again. Still raise
 // an error.
 PyErr_Format(PyExc_TypeError, "union contains %zd unhashable elements", n);
 return -1;
 }
 return PyObject_Hash(alias->hashable_args);
}

unions_equal first compares the frozensets, then (if both sides have unhashable members) checks mutual containment of the unhashable tuples. This makes int | str == str | int true while still distinguishing unions that differ only in unhashable members.

union_richcompare only handles ==/!= between two unions; anything else returns NotImplemented.

Building a union: `unionbuilder`

Construction is not done by directly filling the struct. Instead a small builder accumulates members, deduplicates them, flattens nested unions, and collapses single-member results:

 c
 
 
// Objects/unionobject.c
typedef struct {
 PyObject *args; // list
 PyObject *hashable_args; // set
 PyObject *unhashable_args; // list or NULL
 bool is_checked; // whether to call type_check()
} unionbuilder;

unionbuilder_init creates an empty args list and an empty hashable set. unionbuilder_add_single is the core:

 c
 
 
// Objects/unionobject.c
static bool
unionbuilder_add_single(unionbuilder *ub, PyObject *arg)
{
 if (Py_IsNone(arg)) {
 arg = (PyObject *)&_PyNone_Type; // immortal, so no refcounting needed
 }
 else if (_PyUnion_Check(arg)) {
 PyObject *args = ((unionobject *)arg)->args;
 return unionbuilder_add_tuple(ub, args); // flatten nested unions
 }
 if (ub->is_checked) {
 PyObject *type = type_check(arg, "Union[arg, ...]: each arg must be a type.");
 if (type == NULL) {
 return false;
 }
 bool result = unionbuilder_add_single_unchecked(ub, type);
 Py_DECREF(type);
 return result;
 }
 else {
 return unionbuilder_add_single_unchecked(ub, arg);
 }
}

Two important behaviors:

None is normalized to NoneType (_PyNone_Type), so int | None becomes int | NoneType.
Nested unions are flattened: adding an existing UnionType splices its args tuple into the new builder, so (int | str) | float is the same as int | str | float.

unionbuilder_add_single_unchecked does the dedup:

 c
 
 
// Objects/unionobject.c
static bool
unionbuilder_add_single_unchecked(unionbuilder *ub, PyObject *arg)
{
 Py_hash_t hash = PyObject_Hash(arg);
 if (hash == -1) {
 PyErr_Clear();
 if (ub->unhashable_args == NULL) {
 ub->unhashable_args = PyList_New(0);
 ...
 }
 else {
 int contains = PySequence_Contains(ub->unhashable_args, arg);
 if (contains < 0) return false;
 if (contains == 1) return true; // duplicate, skip
 }
 if (PyList_Append(ub->unhashable_args, arg) < 0) return false;
 }
 else {
 int contains = PySet_Contains(ub->hashable_args, arg);
 if (contains < 0) return false;
 if (contains == 1) return true; // duplicate, skip
 if (PySet_Add(ub->hashable_args, arg) < 0) return false;
 }
 return PyList_Append(ub->args, arg) == 0;
}

Hashable members go into a set for O(1) dedup; unhashable members fall back to linear PySequence_Contains. Either way, args (the ordered list) only gets the member if it is new.

make_union finalizes:

 c
 
 
// Objects/unionobject.c
static PyObject *
make_union(unionbuilder *ub)
{
 Py_ssize_t n = PyList_GET_SIZE(ub->args);
 if (n == 0) {
 PyErr_SetString(PyExc_TypeError, "Cannot take a Union of no types.");
 ...
 return NULL;
 }
 if (n == 1) {
 PyObject *result = PyList_GET_ITEM(ub->args, 0);
 Py_INCREF(result);
 unionbuilder_finalize(ub);
 return result; // collapse single-member "union" to the member
 }
 ... // build args tuple, hashable frozenset, unhashable tuple
 unionobject *result = PyObject_GC_New(unionobject, &_PyUnion_Type);
 ...
 _PyObject_GC_TRACK(result);
 return (PyObject*)result;
}

So int | int evaluates to int, and int | with no arguments is an error. The single-member collapse is what makes int | int is int true.

Entry points

`X | Y` (binary `|`)

type and GenericAlias both install _Py_union_type_or as their nb_or:

 c
 
 
// Objects/typeobject.c
static PyNumberMethods type_as_number = {
 .nb_or = _Py_union_type_or,
};

// Objects/genericaliasobject.c
... .nb_or = _Py_union_type_or ...

_Py_union_type_or first checks is_unionable:

 c
 
 
// Objects/unionobject.c
static int
is_unionable(PyObject *obj)
{
 if (obj == Py_None ||
 PyType_Check(obj) ||
 PySentinel_Check(obj) ||
 _PyGenericAlias_Check(obj) ||
 _PyUnion_Check(obj) ||
 Py_IS_TYPE(obj, &_PyTypeAlias_Type)) {
 return 1;
 }
 return 0;
}

PyObject *
_Py_union_type_or(PyObject* self, PyObject* other)
{
 if (!is_unionable(self) || !is_unionable(other)) {
 Py_RETURN_NOTIMPLEMENTED; // lets normal __or__ / TypeError handling proceed
 }
 unionbuilder ub;
 // unchecked because we already checked is_unionable()
 if (!unionbuilder_init(&ub, false)) return NULL;
 if (!unionbuilder_add_single(&ub, self) ||
 !unionbuilder_add_single(&ub, other)) {
 unionbuilder_finalize(&ub);
 return NULL;
 }
 PyObject *new_union = make_union(&ub);
 return new_union;
}

Because it returns NotImplemented for non-unionable operands, int | 5 falls through to the default int.__or__, which raises TypeError. The builder is unchecked here (is_checked = false), so members are stored as-is rather than being run through type_check.

`typing.Union[X, Y]` and `UnionType[X, Y]`

 c
 
 
// Objects/unionobject.c
static PyObject *
union_class_getitem(PyObject *cls, PyObject *args)
{
 return _Py_union_from_tuple(args);
}

PyObject *
_Py_union_from_tuple(PyObject *args)
{
 unionbuilder ub;
 if (!unionbuilder_init(&ub, true)) return NULL; // checked
 if (PyTuple_CheckExact(args)) {
 if (!unionbuilder_add_tuple(&ub, args)) { ... }
 }
 else {
 if (!unionbuilder_add_single(&ub, args)) { ... }
 }
 return make_union(&ub);
}

__class_getitem__ is registered as a classmethod, so types.UnionType[int, str] works. The builder is checked (is_checked = true), meaning each member passes through type_check:

 c
 
 
// Objects/unionobject.c
static PyObject *
type_check(PyObject *arg, const char *msg)
{
 if (Py_IsNone(arg)) {
 return (PyObject *)Py_TYPE(arg); // None -> NoneType
 }
 if (is_unionable(arg)) {
 return Py_NewRef(arg);
 }
 PyObject *message_str = PyUnicode_FromString(msg);
 PyObject *args[2] = {arg, message_str};
 PyObject *result = call_typing_func_object("_type_check", args, 2);
 Py_DECREF(message_str);
 return result;
}

Fast path for already-unionable objects; otherwise it calls typing._type_check (which may raise TypeError for invalid members). This is why Union[int, "str"] (string forward reference) can resolve, while Union[int, 42] raises.

Substitution: `union.getitem`

Unions support U[int] to substitute type variables:

 c
 
 
// Objects/unionobject.c
static PyObject *
union_getitem(PyObject *self, PyObject *item)
{
 unionobject *alias = (unionobject *)self;
 if (union_init_parameters(alias) < 0) return NULL;

 PyObject *newargs = _Py_subs_parameters(self, alias->args,
 alias->parameters, item);
 if (newargs == NULL) return NULL;

 PyObject *res = _Py_union_from_tuple(newargs);
 Py_DECREF(newargs);
 return res;
}

Parameters are populated lazily and cached under a critical section:

 c
 
 
// Objects/unionobject.c
static int
union_init_parameters(unionobject *alias)
{
 int result = 0;
 Py_BEGIN_CRITICAL_SECTION(alias);
 if (alias->parameters == NULL) {
 alias->parameters = _Py_make_parameters(alias->args);
 if (alias->parameters == NULL) result = -1;
 }
 Py_END_CRITICAL_SECTION();
 return result;
}

So U[T][int] rebuilds the union from substituted args via _Py_union_from_tuple.

Repr

 c
 
 
// Objects/unionobject.c
static PyObject *
union_repr(PyObject *self)
{
 unionobject *alias = (unionobject *)self;
 Py_ssize_t len = PyTuple_GET_SIZE(alias->args);
 Py_ssize_t estimate = (len <= PY_SSIZE_T_MAX / 6) ? len * 6 : len;
 PyUnicodeWriter *writer = PyUnicodeWriter_Create(estimate);
 ...
 for (Py_ssize_t i = 0; i < len; i++) {
 if (i > 0 && PyUnicodeWriter_WriteASCII(writer, " | ", 3) < 0) goto error;
 PyObject *p = PyTuple_GET_ITEM(alias->args, i);
 if (_Py_typing_type_repr(writer, p) < 0) goto error;
 }
 return PyUnicodeWriter_Finish(writer);
 ...
}

Members are rendered with _Py_typing_type_repr, joined by " | ". That helper is what turns _PyNone_Type back into NoneType and handles generics.

Attributes and properties

 c
 
 
// Objects/unionobject.c
static PyMemberDef union_members[] = {
 {"__args__", _Py_T_OBJECT, offsetof(unionobject, args), Py_READONLY},
 {0}
};

static PyGetSetDef union_properties[] = {
 {"__name__", union_name, ...}, // "Union"
 {"__qualname__", union_name, ...}, // "Union"
 {"__origin__", union_origin, ...}, // always _PyUnion_Type
 {"__parameters__", union_parameters, ...},
 {0}
};

__args__ is a read-only member pointing directly at args. __parameters__ is lazily initialized (see above). __mro_entries__ raises TypeError, so unions cannot be subclassed.

union_getattro special-cases __module__, forwarding it to the type object, for compatibility with the typing module.

Comparison with `typing.Union`

types.UnionType is the C implementation; typing.Union is the legacy pure-Python form. Modern code paths (int | str, Union[int, str], types.UnionType[...]) all funnel through _PyUnion_Type. The typing module still provides _type_check and _type_repr-adjacent helpers, but the object itself, its hashing, equality, substitution, and GC are all in C.

Memory layout

text

unionobject (GC object)
+----------------------------+
| ob_refcnt / ob_type ... | PyObject_HEAD
+----------------------------+
| args -> tuple |
| hashable_args -> frozenset (or NULL) |
| unhashable_args -> tuple (or NULL) |
| parameters -> tuple (lazy, or NULL) |
| weakreflist -> weakref list (or NULL) |
+----------------------------+

Allocated with PyObject_GC_New and freed with PyObject_GC_Del; tracked by the GC from creation (_PyObject_GC_TRACK).

Key takeaways

types.UnionType is _PyUnion_Type, defined in Objects/unionobject.c and exported by Modules/_typesmodule.c.
A union is a GC-tracked struct holding an ordered args tuple plus a hashable frozenset and an optional unhashable tuple, plus lazy parameters and a weakref list.
unionbuilder flattens nested unions, normalizes None to NoneType, deduplicates members (set for hashable, linear scan for unhashable), and collapses single-member results.
X | Y routes through type.__or__ / GenericAlias.__or__ = _Py_union_type_or (unchecked builder); Union[X, Y] routes through __class_getitem__ = _Py_union_from_tuple (checked builder, calls typing._type_check).
U[item] substitutes parameters and rebuilds the union.
Equality compares frozensets plus mutual containment of unhashable members; hashing fails if any member is unhashable.
Unions cannot be subclassed (__mro_entries__ raises).

CPython's TypeVar, TypeVarTuple, and ParamSpec Implementation in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s type variables – typing.TypeVar, typing.TypeVarTuple, typing.ParamSpec (plus typing.TypeAliasType) – used to live entirely in Lib/typing.py. In CPython 3.15 they are implemented in C in Objects/typevarobject.c and exposed through the _typing extension module. This article traces that implementation.

Overview

typevarobject.c defines several related types in one file:

typevarobject -> typing.TypeVar
typevartupleobject -> typing.TypeVarTuple
paramspecobject -> typing.ParamSpec
paramspecattrobject -> typing.ParamSpecArgs / ParamSpecKwargs
typealiasobject -> typing.TypeAliasType
_ConstEvaluator and NoDefaultType (internal helpers)

All of these are GC-tracked heap types created from PyType_Spec/PyType_Slot tables and cached in interp->cached_objects during interpreter initialization (_Py_initialize_generic).

Struct layouts

 c
 
 
// Objects/typevarobject.c
typedef struct {
 PyObject_HEAD
 PyObject *name;
 PyObject *bound;
 PyObject *evaluate_bound;
 PyObject *constraints;
 PyObject *evaluate_constraints;
 PyObject *default_value;
 PyObject *evaluate_default;
 bool covariant;
 bool contravariant;
 bool infer_variance;
} typevarobject;

typedef struct {
 PyObject_HEAD
 PyObject *name;
 PyObject *bound;
 PyObject *default_value;
 PyObject *evaluate_default;
 bool covariant;
 bool contravariant;
 bool infer_variance;
} typevartupleobject;

typedef struct {
 PyObject_HEAD
 PyObject *name;
 PyObject *bound;
 PyObject *default_value;
 PyObject *evaluate_default;
 bool covariant;
 bool contravariant;
 bool infer_variance;
} paramspecobject;

Key differences:

TypeVar is the only one that supports constraints (a tuple of types) in addition to bound; TypeVarTuple and ParamSpec only have bound.
All three share variance flags (covariant, contravariant, infer_variance) and an optional default_value.
ParamSpecArgs/ParamSpecKwargs are a separate, smaller struct (paramspecattrobject) holding a single __origin__ pointer back to the ParamSpec they came from.

Lazy evaluation: `evaluate_*` fields

A modern feature (PEP 695 / deferred evaluation) is that bounds and defaults can be stored as evaluator functions rather than concrete values. Each struct carries both a resolved field (bound, default_value, constraints) and an evaluate_* callable. The public getters resolve lazily:

 c
 
 
// Objects/typevarobject.c
static PyObject *
typevar_bound(PyObject *op, void *Py_UNUSED(closure))
{
 typevarobject *self = typevarobject_CAST(op);
 if (self->bound != NULL) {
 return Py_NewRef(self->bound);
 }
 if (self->evaluate_bound == NULL) {
 Py_RETURN_NONE;
 }
 PyObject *bound = PyObject_CallNoArgs(self->evaluate_bound);
 self->bound = Py_XNewRef(bound); // cache the result
 return bound;
}

The same pattern is used for __default__ and __constraints__. The evaluate_bound/evaluate_default/evaluate_constraints accessors expose the raw evaluator (or, if a value was given directly, wrap it in a _ConstEvaluator):

 c
 
 
static PyObject *
typevar_evaluate_bound(PyObject *op, void *Py_UNUSED(closure))
{
 typevarobject *self = typevarobject_CAST(op);
 if (self->evaluate_bound != NULL) {
 return Py_NewRef(self->evaluate_bound);
 }
 if (self->bound != NULL) {
 return constevaluator_alloc(self->bound);
 }
 Py_RETURN_NONE;
}

_ConstEvaluator is an internal, non-instantiable, GC-tracked type whose __call__ returns the stored value (or renders it as a string for _Py_ANNOTATE_FORMAT_STRING). This lets the typing machinery treat eagerly- and lazily-provided values uniformly.

The `NoDefault` marker

Defaults use a sentinel to distinguish “no default” from “default is None”:

 c
 
 
// Objects/typevarobject.c
static PyObject *
NoDefault_repr(PyObject *op) { return PyUnicode_FromString("typing.NoDefault"); }

PyTypeObject _PyNoDefault_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "NoDefaultType",
 .tp_dealloc = nodefault_dealloc, // re-immortalizes if decref'd
 ...
};

PyObject _Py_NoDefaultStruct = _PyObject_HEAD_INIT(&_PyNoDefault_Type);

nodefault_dealloc re-sets the reference count to immortal rather than freeing, so the singleton can never be destroyed. typevar_alloc stores default_value via Py_XNewRef, and typevar_has_default checks both evaluate_default != NULL and that default_value is neither NULL nor the _Py_NoDefaultStruct singleton.

Construction

 c
 
 
static typevarobject *
typevar_alloc(PyObject *name, PyObject *bound, PyObject *evaluate_bound,
 PyObject *constraints, PyObject *evaluate_constraints,
 PyObject *default_value,
 bool covariant, bool contravariant, bool infer_variance,
 PyObject *module)
{
 PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.typevar_type;
 typevarobject *tv = PyObject_GC_New(typevarobject, tp);
 ...
 tv->name = Py_NewRef(name);
 tv->bound = Py_XNewRef(bound);
 ...
 tv->default_value = Py_XNewRef(default_value);
 tv->evaluate_default = NULL;
 _PyObject_GC_TRACK(tv);
 if (module != NULL) {
 PyObject_SetAttrString((PyObject *)tv, "__module__", module);
 }
 return tv;
}

The public TypeVar.__new__ (clinic-generated typevar_new) validates arguments:

covariant and contravariant cannot both be true.
infer_variance cannot be combined with explicit variance.
bound is passed through type_check (which calls typing._type_check).
A single constraint is forbidden; zero constraints becomes NULL; constraints and bound are mutually exclusive.

The caller’s module is captured from the current frame (caller()), so __module__ is set automatically – this is what makes pickling work for globals.

Variance and repr

 c
 
 
static PyObject *
typevar_repr(PyObject *self)
{
 typevarobject *tv = typevarobject_CAST(self);
 if (tv->infer_variance) {
 return Py_NewRef(tv->name); // variance inferred: plain name
 }
 char variance = tv->covariant ? '+' : tv->contravariant ? '-' : '~';
 return PyUnicode_FromFormat("%c%U", variance, tv->name);
}

Manual TypeVars are invariant by default (~), covariant (+), or contravariant (-). When infer_variance=True, the repr drops the prefix because the variance is determined by the type checker from usage.

GC and deallocation

Every struct is GC-tracked. Dealloc untracks, clears all owned references (including the managed dict and weakrefs), then frees with PyObject_GC_Del:

 c
 
 
static void
typevar_dealloc(PyObject *self)
{
 PyTypeObject *tp = Py_TYPE(self);
 typevarobject *tv = typevarobject_CAST(self);
 _PyObject_GC_UNTRACK(self);
 Py_XDECREF(tv->name);
 Py_XDECREF(tv->bound);
 Py_XDECREF(tv->evaluate_bound);
 ...
 PyObject_ClearManagedDict(self);
 PyObject_ClearWeakRefs(self);
 Py_TYPE(self)->tp_free(self);
 Py_DECREF(tp);
}

traverse visits Py_TYPE(self) plus all owned PyObject fields, and clear does Py_CLEAR on them. The same pattern repeats for typevartupleobject, paramspecobject, and paramspecattrobject.

ParamSpecArgs / ParamSpecKwargs

These are lightweight GC objects referencing their origin ParamSpec:

 c
 
 
typedef struct {
 PyObject_HEAD
 PyObject *__origin__;
} paramspecattrobject;

ParamSpec.args and ParamSpec.kwargs (getset descriptors) construct them on demand:

 c
 
 
static PyObject *
paramspec_args(PyObject *self, void *Py_UNUSED(closure))
{
 PyTypeObject *tp = _PyInterpreterState_GET()->cached_objects.paramspecargs_type;
 return (PyObject *)paramspecattr_new(tp, self);
}

They compare by origin and render as P.args / P.kwargs when the origin is a real ParamSpec, otherwise as repr(origin).args.

`|` (union) operator

TypeVar and ParamSpec support | via nb_or = make_union, which builds a typing.Union (not types.UnionType) to preserve legacy forward-reference string behavior:

 c
 
 
static PyObject *
make_union(PyObject *self, PyObject *other)
{
 PyObject *args = _PyTuple_FromPair(self, other);
 PyObject *u = _Py_union_from_tuple(args);
 Py_DECREF(args);
 return u;
}

Initialization and exposure

 c
 
 
int _Py_initialize_generic(PyInterpreterState *interp)
{
 MAKE_TYPE(generic);
 MAKE_TYPE(typevar);
 MAKE_TYPE(typevartuple);
 MAKE_TYPE(paramspec);
 MAKE_TYPE(paramspecargs);
 MAKE_TYPE(paramspeckwargs);
 MAKE_TYPE(constevaluator);
 return 0;
}

MAKE_TYPE calls PyType_FromSpec and stores the result in interp->cached_objects. These types are then imported into typing from _typing:

 python
 
 
from _typing import (
 TypeVar, ParamSpec, TypeVarTuple,
 ParamSpecArgs, ParamSpecKwargs, TypeAliasType, ...
)

Substitution protocol

Each type implements __typing_subst__ and __typing_prepare_subst__, which the typing machinery calls when applying a generic alias to concrete arguments. For TypeVar, __typing_prepare_subst__ looks up the TypeVar’s index among the alias’s __parameters__ and, if a default exists and no argument was supplied, appends the default value to the args tuple. Bare TypeVarTuple substitution raises TypeError (“Substitution of bare TypeVarTuple is not supported”).

TypeAliasType

typealiasobject is a separate struct with name, qualname, type_params, compute_value, value, and module. It supports lazy evaluation of its value (via compute_value), mirroring the constevaluator pattern, and is created by _Py_make_typealias.

Key takeaways

TypeVar, TypeVarTuple, and ParamSpec are GC-tracked C structs in Objects/typevarobject.c, cached per-interpreter in cached_objects, created in _Py_initialize_generic.
TypeVar uniquely supports constraints; all three share bound, variance flags, and optional defaults.
Defaults/bounds can be provided eagerly or as evaluate_* callables, with results cached on first access; eager values are wrapped in _ConstEvaluator when exposed.
NoDefault is an immortal singleton used as the “no default” marker.
ParamSpec.args/.kwargs return on-demand ParamSpecArgs/ParamSpecKwargs objects pointing back at the origin.
TypeVar and ParamSpec support | via make_union, producing typing.Union.
The types are exposed to Python through the _typing extension module and re-exported by Lib/typing.py.

CPython's types.GenericAlias Implementation in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: list[int] is not a class and not a type variable — it’s a tiny C object holding an origin and a tuple of args, plus a lazily computed __parameters__ cache and a starred flag for *Ts unpacking.

Python’s types.GenericAlias is the object you get from list[int] or dict[str, int]. It is not a class and not a type variable; it is a small wrapper that remembers which generic origin (list) was parameterized and with what arguments ((int,)), and knows how to substitute type variables when it is further subscripted. In CPython 3.15 it is implemented by gaobject in Objects/genericaliasobject.c. This article traces that implementation.

The object struct

 c
 
 
// Objects/genericaliasobject.c
typedef struct {
 PyObject_HEAD
 PyObject *origin; // the unparameterized object, e.g. list
 PyObject *args; // tuple of type arguments, e.g. (int,)
 PyObject *parameters; // lazily computed __parameters__, or NULL
 PyObject *weakreflist;
 // Whether we're a starred type, e.g. *tuple[int].
 bool starred;
 vectorcallfunc vectorcall;
} gaobject;

There is also a companion iterator type, gaiterobject, used to implement starred unpacking (*tuple[int]); it holds a single PyObject *obj.

The main object is GC-tracked (it can hold arbitrary objects in args and origin) and supports weak references through weakreflist. Deallocation is straightforward: untrack, clear weakrefs, decref the three owned objects, then free:

 c
 
 
// Objects/genericaliasobject.c
static void
ga_dealloc(PyObject *self)
{
 gaobject *alias = (gaobject *)self;

 _PyObject_GC_UNTRACK(self);
 FT_CLEAR_WEAKREFS(self, alias->weakreflist);
 Py_XDECREF(alias->origin);
 Py_XDECREF(alias->args);
 Py_XDECREF(alias->parameters);
 Py_TYPE(self)->tp_free(self);
}

ga_traverse visits origin, args, and parameters so the collector can find cycles.

Bottom line: six fields, three of which hold real objects. This is about as small as an object can be and still carry full generic-alias semantics.

`origin` and `args`

Two fields carry almost all of the meaning:

origin is the thing being parameterized: a builtin like list, a collections.abc generic, a user class, or even another GenericAlias.
args is always a tuple internally, even if the caller passed a single value. setup_ga normalizes this:

 c
 
 
// Objects/genericaliasobject.c
static inline int
setup_ga(gaobject *alias, PyObject *origin, PyObject *args) {
 if (!PyTuple_Check(args)) {
 args = PyTuple_Pack(1, args);
 if (args == NULL) {
 return 0;
 }
 }
 else {
 Py_INCREF(args);
 }

 alias->origin = Py_NewRef(origin);
 alias->args = args;
 alias->parameters = NULL;
 alias->weakreflist = NULL;

 if (PyVectorcall_Function(origin) != NULL) {
 alias->vectorcall = ga_vectorcall;
 }
 else {
 alias->vectorcall = NULL;
 }

 return 1;
}

Both construction entry points – Py_GenericAlias(origin, args) and GenericAlias.__new__ (ga_new) – go through setup_ga, so the invariant “args is a tuple” holds everywhere. ga_new requires exactly two positional arguments and no keywords, mirroring the documented GenericAlias(origin, args, /).

The two fields are exposed read-only through ga_members:

 c
 
 
// Objects/genericaliasobject.c
static PyMemberDef ga_members[] = {
 {"__origin__", _Py_T_OBJECT, offsetof(gaobject, origin), Py_READONLY},
 {"__args__", _Py_T_OBJECT, offsetof(gaobject, args), Py_READONLY},
 {"__unpacked__", Py_T_BOOL, offsetof(gaobject, starred), Py_READONLY},
 {0}
};

__origin__ and __args__ are therefore just direct field reads – there is no copying or recomputation on access.

A small but important optimization: if origin supports vectorcall, the alias stores ga_vectorcall in its own vectorcall slot (offset tp_vectorcall_offset). When you call the alias, e.g. list[int]() or list[int](iterable), CPython calls the origin directly and then stamps the result with __orig_class__:

 c
 
 
// Objects/genericaliasobject.c
static PyObject *
ga_call(PyObject *self, PyObject *args, PyObject *kwds)
{
 gaobject *alias = (gaobject *)self;
 PyObject *obj = PyObject_Call(alias->origin, args, kwds);
 return set_orig_class(obj, self);
}

static PyObject *
ga_vectorcall(PyObject *self, PyObject *const *args,
 size_t nargsf, PyObject *kwnames)
{
 gaobject *alias = (gaobject *) self;
 PyObject *obj = PyObject_Vectorcall(alias->origin, args, nargsf, kwnames);
 return set_orig_class(obj, self);
}

set_orig_class sets __orig_class__ on the constructed instance when the origin allows it, swallowing AttributeError/TypeError (so origins that reject arbitrary attributes do not fail the call).

`parameters` (type variables)

__parameters__ is not stored eagerly. It is computed on first access and cached in parameters under a per-object critical section (the 3.15 build makes this thread-safe on free-threaded builds):

 c
 
 
// Objects/genericaliasobject.c
static PyObject *
ga_parameters_lock_held(PyObject *self)
{
 _Py_CRITICAL_SECTION_ASSERT_OBJECT_LOCKED(self);
 gaobject *alias = (gaobject *)self;
 if (alias->parameters == NULL) {
 alias->parameters = _Py_make_parameters(alias->args);
 if (alias->parameters == NULL) {
 return NULL;
 }
 }
 return Py_NewRef(alias->parameters);
}

static PyObject *
ga_parameters(PyObject *self, void *unused)
{
 PyObject *result;
 Py_BEGIN_CRITICAL_SECTION(self);
 result = ga_parameters_lock_held(self);
 Py_END_CRITICAL_SECTION();
 return result;
}

_Py_make_parameters walks the args tuple and collects every type variable it finds. The rules, in order:

A plain type (PyType_Check) contributes nothing – list[int] has no parameters because int is concrete.
An object with __typing_subst__ (a TypeVar/TypeVarTuple/ParamSpec) is added, deduplicated via tuple_add.
Otherwise it looks for __parameters__ on the object. If that is missing and the object is a tuple or list, it recurses into _Py_make_parameters, flattening nested parameters.
A __parameters__ tuple is flattened, with the outer tuple resized as needed (_PyTuple_Resize) to make room.

 c
 
 
// Objects/genericaliasobject.c
PyObject *
_Py_make_parameters(PyObject *args)
{
 assert(PyTuple_Check(args) || PyList_Check(args));
 if (Py_EnterRecursiveCall(" in __parameter__ calculation")) {
 return NULL;
 }
 ...
 Py_ssize_t len = nargs;
 PyObject *parameters = PyTuple_New(len);
 ...
 for (Py_ssize_t iarg = 0; iarg < nargs; iarg++) {
 PyObject *t = PyTuple_GET_ITEM(args, iarg);
 if (PyType_Check(t)) {
 continue; // concrete type
 }
 int rc = PyObject_HasAttrWithError(t, &_Py_ID(__typing_subst__));
 ...
 if (rc) {
 iparam += tuple_add(parameters, iparam, t); // TypeVar-like
 }
 else {
 ... // __parameters__ lookup, recursion for tuple/list
 }
 }
 if (iparam < len) {
 if (_PyTuple_Resize(&parameters, iparam) < 0) { ... }
 }
 ...
}

tuple_add only appends when the item is not already present, so dict[T, T] has parameters (T,), not (T, T). The recursion plus resize is what makes list[list[T]] resolve to parameters (T,).

Because parameters is cached, the (possibly expensive) walk happens at most once per alias. Recursive construction is guarded by Py_EnterRecursiveCall, and a recent fix ensures cleanup runs correctly on deeply nested input.

This is the part people get wrong. list[int] has no parameters. Parameters only appear when a type variable is actually in play, and duplicates collapse.

ParamSpec and list-shaped arguments

ParamSpec captures a list of parameter types, and that shows up in __args__ as a list rather than a tuple. For example Callable[[int, str], bool] stores something like (([int, str], bool)) internally, with the ParamSpec argument represented as a list [int, str]. The code handles this in two places.

In _Py_make_parameters, lists are converted to tuples up front (PySequence_Tuple) so the same recursive walk applies. And in ga_repr, a list element of args is rendered by ga_repr_items_list, which prints it with square brackets:

 c
 
 
// Objects/genericaliasobject.c
static int
ga_repr_items_list(PyUnicodeWriter *writer, PyObject *p)
{
 assert(PyList_CheckExact(p));
 ...
 if (PyUnicodeWriter_WriteChar(writer, '[') < 0) { return -1; }
 for (Py_ssize_t i = 0; i < len; i++) {
 ...
 if (_Py_typing_type_repr(writer, item) < 0) { ... }
 }
 if (PyUnicodeWriter_WriteChar(writer, ']') < 0) { return -1; }
 return 0;
}

and in ga_repr:

 c
 
 
// Objects/genericaliasobject.c
for (Py_ssize_t i = 0; i < len; i++) {
 PyObject *p = PyTuple_GET_ITEM(alias->args, i);
 if (PyList_CheckExact(p)) {
 // Looks like we are working with ParamSpec's list of type args
 if (ga_repr_items_list(writer, p) < 0) { goto error; }
 }
 else if (_Py_typing_type_repr(writer, p) < 0) {
 goto error;
 }
}

So the list-vs-tuple distinction in args is precisely what distinguishes a ParamSpec argument list from an ordinary tuple of type arguments, and it is preserved (not normalized) by setup_ga because only a non-tuple scalar is packed; a list passed as an element stays a list.

Gotcha: the top-level args is always a tuple, but an element of it can be a list. That asymmetry is deliberate, and it is what keeps Callable[[int, str], bool] printable as Callable[[int, str], bool].

Substitution: `alias[item]`

Subscripting a GenericAlias is where type variables get replaced. ga_getitem locks the object, materializes __parameters__, then calls _Py_subs_parameters:

 c
 
 
// Objects/genericaliasobject.c
static PyObject *
ga_getitem(PyObject *self, PyObject *item)
{
 gaobject *alias = (gaobject *)self;
 PyObject *parameters;
 Py_BEGIN_CRITICAL_SECTION(self);
 parameters = ga_parameters_lock_held(self);
 Py_END_CRITICAL_SECTION();
 if (parameters == NULL) {
 return NULL;
 }

 PyObject *newargs = _Py_subs_parameters(self, alias->args, parameters, item);
 Py_DECREF(parameters);
 if (newargs == NULL) {
 return NULL;
 }

 PyObject *res = Py_GenericAlias(alias->origin, newargs);
 if (res == NULL) { ... }
 ((gaobject *)res)->starred = alias->starred;

 Py_DECREF(newargs);
 return res;
}

The result is a new GenericAlias with the same origin and the same starred flag, but with substituted args. The origin itself is reused, so list[int][str] becomes list[str].

_Py_subs_parameters does the heavy lifting:

_unpack_args(item) normalizes the subscript into a tuple, flattening starred unpacked TypeVarTuples (*Ts) into the argument list.
It validates the argument count against nparams.
For each original arg:
- plain types are copied verbatim;
- nested tuples/lists are substituted recursively;
- objects with __typing_subst__ have that method called with the matching type argument;
- otherwise subs_tvars handles nested generics by looking up their own __parameters__ and re-subscripting them.

subs_tvars is the recursive case: if obj has a non-empty __parameters__, it builds a fresh subargs tuple by mapping each of its parameters to the corresponding substitution (with special handling for TypeVarTuple, where a tuple argument is spliced in via tuple_extend), then calls PyObject_GetItem(obj, subargs) to produce obj[subargs]. Otherwise it returns obj unchanged:

 c
 
 
// Objects/genericaliasobject.c
static PyObject *
subs_tvars(PyObject *obj, PyObject *params,
 PyObject **argitems, Py_ssize_t nargs)
{
 PyObject *subparams;
 if (PyObject_GetOptionalAttr(obj, &_Py_ID(__parameters__), &subparams) < 0) {
 return NULL;
 }
 if (subparams && PyTuple_Check(subparams) && PyTuple_GET_SIZE(subparams)) {
 ...
 PyObject *subargs = PyTuple_New(nsubargs);
 ...
 for (Py_ssize_t i = 0; i < nsubargs; ++i) {
 PyObject *arg = PyTuple_GET_ITEM(subparams, i);
 Py_ssize_t iparam = tuple_index(params, nparams, arg);
 if (iparam >= 0) {
 PyObject *param = PyTuple_GET_ITEM(params, iparam);
 arg = argitems[iparam];
 if (Py_TYPE(param)->tp_iter && PyTuple_Check(arg)) {
 // TypeVarTuple: splice the tuple in
 j = tuple_extend(&subargs, j, ...);
 continue;
 }
 }
 PyTuple_SET_ITEM(subargs, j, Py_NewRef(arg));
 j++;
 }
 obj = PyObject_GetItem(obj, subargs);
 Py_DECREF(subargs);
 }
 else {
 Py_INCREF(obj);
 }
 Py_XDECREF(subparams);
 return obj;
}

This is why list[T][int] works even though T only appears inside list’s args: the outer alias’s parameters drive substitution into the nested generic.

Starred unpacking (`*tuple[int]`)

A GenericAlias can be starred for use in *Ts unpacking contexts. The starred bool is set on the object and exposed as __unpacked__. Iterating a starred *tuple[...] alias yields one alias per element type, each itself marked starred:

 c
 
 
// Objects/genericaliasobject.c
static PyObject *
ga_iternext(PyObject *op)
{
 gaiterobject *gi = (gaiterobject*)op;
 ...
 gaobject *alias = (gaobject *)obj;
 PyObject *starred_alias = Py_GenericAlias(alias->origin, alias->args);
 ...
 ((gaobject *)starred_alias)->starred = true;
 return starred_alias;
}

ga_iter creates a gaiterobject holding a reference to the original alias, and ga_unpacked_tuple_args fast-paths a starred tuple alias by returning its args directly. Starred state also participates in equality (ga_richcompare compares starred first) and in __reduce__ (a starred alias pickles via next(iter(...))).

Attribute access

ga_getattro implements a hybrid: most attribute names are proxied to origin, but a fixed list of “own” attributes (attr_exceptions) and a blocked list (attr_blocked) are handled by the generic object machinery:

 c
 
 
// Objects/genericaliasobject.c
static const char* const attr_exceptions[] = {
 "__class__", "__origin__", "__args__", "__unpacked__",
 "__parameters__", "__typing_unpacked_tuple_args__",
 "__mro_entries__", "__reduce_ex__", "__reduce__", NULL,
};
static const char* const attr_blocked[] = {
 "__bases__", "__copy__", "__deepcopy__", NULL,
};

Blocked attributes are never proxied (they fall through to generic_getattr, which usually raises AttributeError), preventing a generic alias from inheriting dangerous or meaningless operations from its origin. Everything else – append, __name__, etc. – is forwarded to origin, so list[int].append resolves to list.append.

Hashing and equality

Equality compares starred, then origin, then args. Hashing combines the hashes of origin and args with XOR:

 c
 
 
// Objects/genericaliasobject.c
static Py_hash_t
ga_hash(PyObject *self)
{
 gaobject *alias = (gaobject *)self;
 Py_hash_t h0 = PyObject_Hash(alias->origin);
 if (h0 == -1) return -1;
 Py_hash_t h1 = PyObject_Hash(alias->args);
 if (h1 == -1) return -1;
 return h0 ^ h1;
}

Note the origin’s hash is currently not folded in a more sophisticated way (there is a // TODO: Hash in the hash for the origin comment), but the result is still a valid hash consistent with equality.

Calling, `__mro_entries__`, and isinstance

Calling delegates to origin and sets __orig_class__ (above).
__mro_entries__ returns (origin,), so a generic alias can be used as a base class in class Foo(list[int]): ... (PEP 560).
__instancecheck__ and __subclasscheck__ raise TypeError: you cannot use a parameterized generic with isinstance/issubclass.

Gotcha: isinstance([], list[int]) is a TypeError, not False. If you need a runtime check, use the origin.

Key takeaways

types.GenericAlias is a GC-tracked gaobject with five fields: origin, args (always a tuple), lazily cached parameters, weakreflist, and a starred flag, plus an optional vectorcall pointer.
origin is the unparameterized generic; args holds the type arguments. A non-tuple scalar argument is packed into a one-tuple by setup_ga, but a list element is preserved (it denotes a ParamSpec argument list).
__parameters__ is computed lazily and cached under a critical section; _Py_make_parameters recursively collects and deduplicates type variables, flattening nested generics and (via list-to-tuple conversion) ParamSpecs.
Subscripting builds a new alias with substituted args via _Py_subs_parameters, which handles count validation, nested substitution, __typing_subst__, and TypeVarTuple splicing.
Starred aliases support *Ts unpacking and iterate into per-element starred aliases.
Attribute access proxies to origin except for a fixed allow/block list, so methods resolve while __bases__/copy operations are blocked.
Equality and hashing are derived from starred, origin, and args.

CPython's type Implementation in C (the Metaclass)

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

type is the metaclass of every Python object: calling type(obj) returns the class of obj, and class Foo: ultimately calls type.__new__. In CPython the same object that represents type in Python is PyType_Type, and every class is a PyTypeObject (or, for classes created at runtime, a PyHeapTypeObject). This article traces how that machinery is implemented in Objects/typeobject.c and Include/cpython/object.h in CPython 3.15.

The object struct

Every Python object begins with PyObject_VAR_HEAD. A type object adds dozens of slots that describe how instances behave, plus the things that make a type a first-class object: its tp_dict, bases, and MRO. The 3.15 layout is:

 c
 
 
// Include/cpython/object.h
struct _typeobject {
 PyObject_VAR_HEAD
 const char *tp_name;
 Py_ssize_t tp_basicsize, tp_itemsize;

 destructor tp_dealloc;
 Py_ssize_t tp_vectorcall_offset;
 getattrfunc tp_getattr;
 setattrfunc tp_setattr;
 PyAsyncMethods *tp_as_async;
 reprfunc tp_repr;

 PyNumberMethods *tp_as_number;
 PySequenceMethods *tp_as_sequence;
 PyMappingMethods *tp_as_mapping;

 hashfunc tp_hash;
 ternaryfunc tp_call;
 reprfunc tp_str;
 getattrofunc tp_getattro;
 setattrofunc tp_setattro;

 PyBufferProcs *tp_as_buffer;
 unsigned long tp_flags;
 const char *tp_doc;

 traverseproc tp_traverse;
 inquiry tp_clear;
 richcmpfunc tp_richcompare;

 Py_ssize_t tp_weaklistoffset;
 getiterfunc tp_iter;
 iternextfunc tp_iternext;

 PyMethodDef *tp_methods;
 PyMemberDef *tp_members;
 PyGetSetDef *tp_getset;
 PyTypeObject *tp_base; /* borrowed on static, strong on heap */
 PyObject *tp_dict; /* the class __dict__ */
 descrgetfunc tp_descr_get;
 descrsetfunc tp_descr_set;
 Py_ssize_t tp_dictoffset;
 initproc tp_init;
 allocfunc tp_alloc;
 newfunc tp_new;
 freefunc tp_free;
 inquiry tp_is_gc;
 PyObject *tp_bases; /* tuple of direct bases */
 PyObject *tp_mro; /* method resolution order tuple */
 PyObject *tp_cache; /* unused */
 void *tp_subclasses; /* dict, or index for static builtins */
 PyObject *tp_weaklist;
 destructor tp_del;

 unsigned int tp_version_tag;
 destructor tp_finalize;
 vectorcallfunc tp_vectorcall;

 unsigned char tp_watched;
 uint16_t tp_versions_used;
 _Py_iteritemfunc _tp_iteritem;
};

The slot fields (tp_methods, tp_members, tp_getset) are arrays that PyType_Ready turns into real descriptors in tp_dict. tp_as_number and friends are method suites: pointers to structs of function pointers that implement +, len(), [], etc. They are populated either statically (for built-in types) or by add_operators() during readying.

The real layout: `PyHeapTypeObject`

A type created at runtime (class Foo:) is not just a PyTypeObject. It is a PyHeapTypeObject, whose first member is the PyTypeObject (so it can be cast back and forth), followed by the method-suite structs and heap-only metadata:

 c
 
 
// Include/cpython/object.h
typedef struct _heaptypeobject {
 PyTypeObject ht_type;
 PyAsyncMethods as_async;
 PyNumberMethods as_number;
 PyMappingMethods as_mapping;
 PySequenceMethods as_sequence;
 PyBufferProcs as_buffer;
 PyObject *ht_name, *ht_slots, *ht_qualname;
 struct _dictkeysobject *ht_cached_keys;
 PyObject *ht_module;
 char *_ht_tpname;
 void *ht_token;
 struct _specialization_cache _spec_cache;
#ifdef Py_GIL_DISABLED
 Py_ssize_t unique_id;
#endif
} PyHeapTypeObject;

The distinction is exposed by the Py_TPFLAGS_HEAPTYPE flag. Static types (int, list, str, …) are immortal, singletons stored in the binary; heap types are allocated with PyObject_GC_New and reference-counted.

`tp_dict`: the class dictionary

tp_dict is the __dict__ of the class. For heap types it is a real dict created during type creation. For static built-in types CPython stores tp_dict per-interpreter in managed_static_type_state (so each interpreter gets its own dict), which is why lookup_tp_dict branches on _Py_TPFLAGS_STATIC_BUILTIN:

 c
 
 
// Objects/typeobject.c
static inline PyObject *lookup_tp_dict(PyTypeObject *self) {
 if (self->tp_flags & _Py_TPFLAGS_STATIC_BUILTIN) {
 PyInterpreterState *interp = _PyInterpreterState_GET();
 managed_static_type_state *state = _PyStaticType_GetState(interp, self);
 return state->tp_dict;
 }
 return self->tp_dict;
}

Attribute lookup (type.__getattribute__, i.e. _PyType_Lookup) searches tp_dict of the type and then walks tp_mro, reading each base’s tp_dict. When a class is modified, PyType_Modified invalidates the version tag and notifies watchers.

MRO

The MRO is stored as a tuple in tp_mro, with the type itself first and object last. It is computed by mro_internal, which calls the type’s (or metaclass’s) mro() and falls back to C3 linearization:

 c
 
 
// Objects/typeobject.c
static int mro_internal(PyTypeObject *type, int initial, PyObject **p_old_mro) {
 ...
 old_mro = Py_XNewRef(lookup_tp_mro(type));
 new_mro = mro_invoke(type); /* may re-enter */
 reent = (lookup_tp_mro(type) != old_mro);
 ...
 set_tp_mro(type, new_mro, initial);
 type_mro_modified(type, new_mro);
 ...
}

mro_internal also records the old MRO so that, when bases change, subclasses can be updated. type_mro_modified checks whether the type is eligible for the attribute cache (version tag): a custom mro() or a non-subtype base clears tp_versions_used to _Py_ATTR_CACHE_UNUSED, disabling caching for that type and its subclasses.

Type creation

Creating a class (class Foo(Bar): ...) calls type.__new__, which in C is type_new -> ultimately type_from_slots_or_spec for the C API path. The public entry points are:

 c
 
 
// Objects/typeobject.c
PyObject *PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module,
 PyType_Spec *spec, PyObject *bases);
PyObject *PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases);
PyObject *PyType_FromSpec(PyType_Spec *spec);

A PyType_Spec describes name, basicsize, itemsize, flags and a list of PyType_Slot entries mapping slot IDs (Py_tp_init, Py_tp_methods, …) to function pointers. type_from_slots_or_spec allocates a PyHeapTypeObject, copies the spec into the slots, sets tp_base/tp_bases, creates tp_dict, and finally calls PyType_Ready.

PyType_Ready is the choke point that finishes initialization:

 c
 
 
// Objects/typeobject.c
int PyType_Ready(PyTypeObject *type) {
 if (type->tp_flags & Py_TPFLAGS_READY) return 0;
 assert(!(type->tp_flags & _Py_TPFLAGS_STATIC_BUILTIN));

 if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) {
 type_add_flags(type, Py_TPFLAGS_IMMUTABLETYPE);
 _Py_SetImmortalUntracked((PyObject *)type);
 }

 int res;
 BEGIN_TYPE_LOCK();
 if (!(type->tp_flags & Py_TPFLAGS_READY)) {
 res = type_ready(type, 1, 1);
 }
 END_TYPE_LOCK();
 return res;
}

type_ready (called under the type lock) fills in inherited slots from the base, calls add_operators to turn tp_methods/tp_members/tp_getset into descriptors in tp_dict, computes tp_mro via mro_internal, sets tp_flags |= Py_TPFLAGS_READY, and assigns a version tag. After this the type is usable.

Heap vs static types

Static types (PyLong_Type, PyList_Type, …) are compile-time PyTypeObject constants. They carry _Py_TPFLAGS_STATIC_BUILTIN (for the core builtins) or are plain static types in extension modules. They are immortal and, for builtins, have per-interpreter tp_dict/tp_subclasses/ tp_weaklist stored in managed_static_type_state. init_static_type registers this state and calls type_ready:

 c
 
 
// Objects/typeobject.c
static int init_static_type(PyInterpreterState *interp, PyTypeObject *self,
 int isbuiltin, int initial) {
 ...
 type_add_flags(self, _Py_TPFLAGS_STATIC_BUILTIN);
 type_add_flags(self, Py_TPFLAGS_IMMUTABLETYPE);
 managed_static_type_state_init(interp, self, isbuiltin, initial);
 ...
 res = type_ready(self, initial, 1);
 ...
}

Heap types differ in several ways:

Allocated on the heap and reference-counted (Py_TPFLAGS_HAVE_GC usually).
Py_TPFLAGS_HEAPTYPE is set; tp_name/ht_name live in the PyHeapTypeObject.
Mutable by default: attributes can be added, __bases__ can change, MRO recomputed.
tp_base is a strong reference; for static types it is a borrowed reference.
Version tags come from a per-interpreter counter (NEXT_VERSION_TAG) rather than the global counter used for static types.

The Py_TPFLAGS_IMMUTABLETYPE flag (set by PyType_Ready for any non-heap type, and available to heap types via Py_TPFLAGS_IMMUTABLETYPE) controls whether __setattr__/__delattr__ on the type itself are allowed, which is what makes a class “frozen”.

Type lookup and the version tag

_PyType_Lookup walks the MRO and reads tp_dict. To make attribute access fast, CPython caches results keyed by tp_version_tag:

 c
 
 
// Objects/typeobject.c (conceptually)
PyObject *_PyType_Lookup(PyTypeObject *type, PyObject *name) {
 ...
 /* version tag fast path */
}

assign_version_tag (called lazily) assigns a tag to a type only after all its bases have tags, so a cached lookup is valid for the whole hierarchy. PyType_Modified clears the tag (setting it to 0) and recursively clears subclasses, forcing re-lookup. This is why mutating a base class invalidates caches everywhere below it.

`type.call` and instance creation

When you write Foo(x), Python calls type.__call__(Foo, x). In C that is type_call, which runs tp_new to allocate and initialize the instance, then tp_init to initialize it, and finally returns the object (or propagates an error). The slots are inherited from the base during type_ready, so a type that does not define tp_new gets object.__new__ (i.e. slot_tp_new):

 c
 
 
// Objects/typeobject.c
static PyObject *type_call(PyTypeObject *type, PyObject *args, PyObject *kwds) {
 PyObject *obj;
 ...
 obj = type->tp_new(type, args, kwds);
 if (obj == NULL) return NULL;
 if (type->tp_init != NULL && type->tp_init(obj, args, kwds) < 0) {
 Py_DECREF(obj);
 return NULL;
 }
 return obj;
}

tp_alloc (default PyType_GenericAlloc) allocates the instance using tp_basicsize/tp_itemsize and sets ob_type. tp_free is the matching deallocator. For GC-tracked types tp_alloc calls PyObject_GC_New and tp_free calls PyObject_GC_Del.

Slots and inheritance

Slots are not inherited field-by-field naively. type_ready calls inherit_slots (and add_operators), which copies function pointers from the base when the derived type leaves a slot NULL, and merges the method suites (tp_as_number, etc.) by copying individual function pointers. This is why a subclass of int automatically gets nb_add unless it overrides __add__.

The tp_methods/tp_members/tp_getset arrays are processed by add_methods/add_members/add_getset, which build PyCFunction/member/ getset descriptors and insert them into tp_dict. After that the arrays are no longer needed for lookup (the dict holds the descriptors).

Subclassing rules

PyType_Ready and type_new enforce that a base type must have Py_TPFLAGS_BASETYPE set; otherwise type(...) raises “is not an acceptable base type”. object, type and most builtins set this flag. Some types (e.g. list historically, though now subclassable) clear it to forbid subclassing in C.

type_is_subtype and is_subtype_with_mro are used to validate MROs. A type cannot be its own base, and the MRO must be consistent (C3 merge). If no linearization exists, type() raises TypeError: Cannot create a consistent method resolution order.

`type_new` in detail

type_new (the C implementation of type.__new__) does the heavy lifting for class statements:

Validate name, bases, dict.
Choose the metaclass: if bases is non-empty, use the most derived metaclass among the bases’ metaclasses; otherwise use the provided metaclass or type.
Allocate a PyHeapTypeObject sized for tp_basicsize + nslots * sizeof(void*) when __slots__ is present.
Copy tp_name (stored in _ht_tpname), ht_name, ht_qualname.
Set tp_base to the best base (or object), tp_bases to the tuple.
Call PyType_Ready.

For __slots__, type_new creates a member_descriptor for each slot and stores the slot index in the instance dict layout (tp_dictoffset, tp_weaklistoffset).

Managed dicts and `dict`

Heap types may have a managed __dict__ (Py_TPFLAGS_MANAGED_DICT) instead of a fixed tp_dictoffset. In 3.15 managed dicts are the default for heap types, storing the instance dict in a side table indexed by the object. This saves memory when most instances have no extra attributes. tp_dictoffset is set to 0 in that case and PyObject_GenericGetAttr uses _PyObject_GetDictPtr.

Garbage collection

Heap types set Py_TPFLAGS_HAVE_GC and provide tp_traverse/tp_clear (generated by the GC machinery for the type’s slots and tp_dict). Static types that contain references also set these flags. tp_is_gc answers whether a particular instance is tracked. _PyObject_GC_TRACK is called in tp_alloc for GC types.

Free-threading notes

In Py_GIL_DISABLED builds, several fields become per-interpreter or atomically accessed: tp_dict, tp_mro, tp_bases for static builtins live in managed_static_type_state; tp_version_tag uses atomic loads/stores; tp_subclasses is per-interpreter. Type mutation takes the type lock (TYPE_LOCK) and may stop the world. New types are marked “revealed” via _Py_TYPE_REVEALED_FLAG only after they are fully initialized, so other threads never observe a half-ready type.

`type` itself

PyType_Type is defined statically in Objects/typeobject.c with tp_name = "type", tp_basicsize = sizeof(PyHeapTypeObject), tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_BASETYPE, and slots pointing at type_* functions (type_repr, type_call, type_getattro, etc.). Its metaclass is itself (PyType_Type.tp_type = &PyType_Type), which is the fixed point that makes type(type) is type.

Key takeaways

type is PyType_Type; every class is a PyTypeObject, and runtime classes are PyHeapTypeObjects (heap-allocated, GC-tracked).
Slots (tp_*) describe instance behavior; method suites (tp_as_number, …) group operator implementations.
tp_dict is the class __dict__; static builtins store theirs per-interpreter.
The MRO is a tuple in tp_mro, computed by mro_internal (C3 by default, custom mro() allowed).
PyType_Ready finishes a type: inheritance, operators, MRO, flags, version tag.
Static types are immortal and immutable by default; heap types are mutable and reference-counted.
Version tags power the attribute-lookup cache; PyType_Modified invalidates them.

Type watchers and the attribute cache

CPython 3.11+ added PyType_AddWatcher/PyType_Watch so that tools (jit/optimizer, debuggers) can be notified when a type changes. A watcher bit is stored in tp_watched; when PyType_Modified runs it invokes every registered callback for that type. PyType_Modified also clears the version tag and walks subclasses, so watchers see a consistent invalidation.

tp_cache is a legacy field (always NULL now); caching moved to the per-interpreter type_cache hash table keyed by (version, name).

CPython's tuple Implementation in C: Immutable, Inline, and Recycled

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: A CPython tuple stores its items inside the object’s own allocation — no backing array, no indirection. Its length is also its capacity, its hash is cached in ob_hash, and sizes 1..20 are recycled through per-size freelists.

Python’s tuple is an immutable sequence of object pointers. In CPython it’s PyTupleObject in Objects/tupleobject.c. Unlike list, a tuple never over-allocates: its length and its capacity are the same. Let’s trace the representation and the hot paths through the 3.15 source.

The object struct

Every Python object starts with PyObject_VAR_HEAD (ob_refcnt, ob_type, and ob_size). For tuples the payload adds a cached hash and the item vector:

 c
 
 
// Include/cpython/tupleobject.h
typedef struct {
 PyObject_VAR_HEAD
 /* Cached hash. Initially set to -1. */
 Py_hash_t ob_hash;
 /* ob_item contains space for 'ob_size' elements.
 Items must normally not be NULL, except during construction when
 the tuple is not yet visible outside the function that builds it. */
 PyObject *ob_item[1];
} PyTupleObject;

The trailing ob_item[1] is the classic C “struct hack”: the real object is allocated with ob_size slots, and ob_item is treated as a flexible array. The type object declares this with tp_basicsize and tp_itemsize:

 c
 
 
// Objects/tupleobject.c
PyTypeObject PyTuple_Type = {
 ...
 sizeof(PyTupleObject) - sizeof(PyObject *), /* tp_basicsize */
 sizeof(PyObject *), /* tp_itemsize */
 ...
};

tp_basicsize is the size of the header minus one slot, and tp_itemsize is the size of one slot, so PyObject_GC_NewVar(..., size) allocates exactly sizeof(PyTupleObject) - sizeof(PyObject *) + size * sizeof(PyObject *) bytes.

The items therefore live in the same allocation as the object header – no separate backing array and no indirection.

Memory layout

text

+---------------------------+ <- PyObject_GC_NewVar result
| ob_refcnt |
+---------------------------+
| *ob_type (&PyTuple_Type) |
+---------------------------+
| ob_size (len) |
+---------------------------+
| ob_hash (cached hash) |
+---------------------------+
| ob_item[0] (PyObject *) |
| ob_item[1] (PyObject *) |
| ... |
| ob_item[ob_size - 1] |
+---------------------------+

Because there’s no allocated field, a tuple’s memory is exactly its length. Shrinking or growing means reallocating the whole object (see _PyTuple_Resize).

The empty-tuple singleton

The empty tuple is a statically allocated singleton, not GC-tracked, and its hash is precomputed at compile time:

 c
 
 
// Include/internal/pycore_runtime_structs.h
struct {
 _PyGC_Head_UNUSED _tuple_empty_gc_not_used;
 PyTupleObject tuple_empty;
 ...
} singletons;

// Include/internal/pycore_runtime_init.h
.tuple_empty = {
 .ob_base = _PyVarObject_HEAD_INIT(&PyTuple_Type, 0),
 .ob_hash = _PyTuple_HASH_EMPTY,
},

Every call asking for an empty tuple returns the same object:

 c
 
 
// Objects/tupleobject.c
static inline PyObject *
tuple_get_empty(void)
{
 return (PyObject *)&_Py_SINGLETON(tuple_empty);
}

PyObject *
PyTuple_New(Py_ssize_t size)
{
 PyTupleObject *op;
 if (size == 0) {
 return tuple_get_empty();
 }
 ...
}

tuple_dealloc guards the singleton against being freed:

 c
 
 
// Objects/tupleobject.c
if (Py_SIZE(op) == 0) {
 if (op == &_Py_SINGLETON(tuple_empty)) {
#ifdef Py_DEBUG
 _Py_FatalRefcountError("deallocating the empty tuple singleton");
#else
 return;
#endif
 }
}

Subclasses don’t share the singleton; tuple_subtype_new may allocate its own empty instance.

Free list (tuple reuse)

Exact PyTupleObjects of sizes 1..20 are recycled through a per-size free list, each holding up to 2000 entries:

 c
 
 
// Include/internal/pycore_freelist_state.h
# define PyTuple_MAXSAVESIZE 20 // Largest tuple to save on freelist
# define Py_tuple_MAXFREELIST 2000 // Maximum number of tuples of each size

tuple_alloc pops from the matching bucket before allocating:

 c
 
 
// Objects/tupleobject.c
static PyTupleObject *
tuple_alloc(Py_ssize_t size)
{
 if (size < 0) { ... }
 assert(size != 0); // The empty tuple is statically allocated.
 Py_ssize_t index = size - 1;
 if (index < PyTuple_MAXSAVESIZE) {
 PyTupleObject *op = _Py_FREELIST_POP(PyTupleObject, tuples[index]);
 if (op != NULL) {
 _PyTuple_RESET_HASH_CACHE(op);
 return op;
 }
 }
 ...
 PyTupleObject *result = PyObject_GC_NewVar(PyTupleObject, &PyTuple_Type, size);
 if (result != NULL) {
 _PyTuple_RESET_HASH_CACHE(result);
 }
 return result;
}

On deallocation, maybe_freelist_push returns the object to its bucket instead of freeing it:

 c
 
 
// Objects/tupleobject.c
static inline int
maybe_freelist_push(PyTupleObject *op)
{
 if (!Py_IS_TYPE(op, &PyTuple_Type)) {
 return 0; // subclasses are not recycled
 }
 Py_ssize_t index = Py_SIZE(op) - 1;
 if (index < PyTuple_MAXSAVESIZE) {
 return _Py_FREELIST_PUSH(tuples[index], op, Py_tuple_MAXFREELIST);
 }
 return 0;
}

Because a recycled object may have been tracked by the GC before, _PyTuple_Recycle re-tracks it and resets the hash cache when it’s reused:

 c
 
 
// Include/internal/pycore_tuple.h
static inline void
_PyTuple_Recycle(PyObject *op)
{
 _PyTuple_RESET_HASH_CACHE(op);
 if (!_PyObject_GC_IS_TRACKED(op)) {
 _PyObject_GC_TRACK(op);
 }
}

Creation

PyTuple_New allocates and zeroes the slots, then tracks the object:

 c
 
 
// Objects/tupleobject.c
PyObject *
PyTuple_New(Py_ssize_t size)
{
 PyTupleObject *op;
 if (size == 0) {
 return tuple_get_empty();
 }
 op = tuple_alloc(size);
 if (op == NULL) {
 return NULL;
 }
 for (Py_ssize_t i = 0; i < size; i++) {
 op->ob_item[i] = NULL;
 }
 _PyObject_GC_TRACK(op);
 return (PyObject *) op;
}

PyTuple_Pack builds a tuple from varargs and only enables GC tracking if at least one item may itself be tracked:

 c
 
 
// Objects/tupleobject.c
PyObject *
PyTuple_Pack(Py_ssize_t n, ...)
{
 ...
 PyTupleObject *result = tuple_alloc(n);
 ...
 for (i = 0; i < n; i++) {
 o = va_arg(vargs, PyObject *);
 if (!track && maybe_tracked(o)) {
 track = true;
 }
 items[i] = Py_NewRef(o);
 }
 if (track) {
 _PyObject_GC_TRACK(result);
 }
 return (PyObject *)result;
}

maybe_tracked is a conservative check: any GC-capable type makes the tuple tracked. _PyTuple_MaybeUntrack later untracks a tuple once all of its items are non-GC objects, which keeps short-lived tuples of plain values off the collector’s radar:

 c
 
 
// Objects/tupleobject.c
void
_PyTuple_MaybeUntrack(PyObject *op)
{
 if (!PyTuple_CheckExact(op) || !_PyObject_GC_IS_TRACKED(op))
 return;
 ...
 for (i = 0; i < n; i++) {
 PyObject *elt = PyTuple_GET_ITEM(t, i);
 if (!elt || _PyObject_GC_MAY_BE_TRACKED(elt))
 return; // still potentially cyclic
 }
 _PyObject_GC_UNTRACK(op);
}

PySequence_Tuple special-cases exact tuples (returns the same object) and exact lists (PyList_AsTuple), then falls back to iterating, buffering the first 8 items on the stack and spilling into a list.

tuple.__new__ returns the empty singleton for no argument, and otherwise delegates to PySequence_Tuple. For a tuple argument this returns the same object – tuples are immutable, so sharing is safe.

Hashing

The hash is computed lazily and cached in ob_hash. -1 is the “not yet computed” sentinel, so a real hash of -1 is remapped to 1546275796. The algorithm is a single-accumulator simplification of xxHash:

 c
 
 
// Objects/tupleobject.c
static Py_hash_t
tuple_hash(PyObject *op)
{
 PyTupleObject *v = _PyTuple_CAST(op);

 Py_uhash_t acc = FT_ATOMIC_LOAD_SSIZE_RELAXED(v->ob_hash);
 if (acc != (Py_uhash_t)-1) {
 return acc; // cached
 }

 Py_ssize_t len = Py_SIZE(v);
 PyObject **item = v->ob_item;
 acc = _PyTuple_HASH_XXPRIME_5;
 for (Py_ssize_t i = 0; i < len; i++) {
 Py_uhash_t lane = PyObject_Hash(item[i]);
 if (lane == (Py_uhash_t)-1) {
 return -1; // propagate error
 }
 acc += lane * _PyTuple_HASH_XXPRIME_2;
 acc = _PyTuple_HASH_XXROTATE(acc);
 acc *= _PyTuple_HASH_XXPRIME_1;
 }

 /* Add input length, mangled to keep the historical value of hash(()). */
 acc += len ^ (_PyTuple_HASH_XXPRIME_5 ^ 3527539UL);

 if (acc == (Py_uhash_t)-1) {
 acc = 1546275796;
 }

 FT_ATOMIC_STORE_SSIZE_RELAXED(v->ob_hash, acc);
 return acc;
}

The cache is reset to -1 whenever a tuple is recycled (_PyTuple_RESET_HASH_CACHE), and tuple_richcompare relies on immutability to reuse vlen/wlen across comparison calls.

Immutability

Immutability is enforced at the type level: tp_as_sequence has no sq_ass_item or sq_ass_slice slots, so tuple[i] = x raises TypeError.

The only mutation paths are internal, and they require a uniquely referenced, exact PyTuple_Type instance:

PyTuple_SetItem stores a pointer without increfing the new item (but decrefing the replaced one), and only succeeds when _PyObject_IsUniquelyReferenced(op) holds:

 c
 
 
// Objects/tupleobject.c
int
PyTuple_SetItem(PyObject *op, Py_ssize_t i, PyObject *newitem)
{
 ...
 if (!PyTuple_Check(op) || !_PyObject_IsUniquelyReferenced(op)) {
 Py_XDECREF(newitem);
 PyErr_BadInternalCall();
 return -1;
 }
 ...
 p = ((PyTupleObject *)op) -> ob_item + i;
 Py_XSETREF(*p, newitem);
 return 0;
}

_PyTuple_Resize can grow or shrink a uniquely referenced exact tuple by reallocating the object in place. It decrefs deleted items on shrink and zeroes new slots on grow; the empty tuple is never resized in place (it’s the singleton):

 c
 
 
// Objects/tupleobject.c
int
_PyTuple_Resize(PyObject **pv, Py_ssize_t newsize)
{
 ...
 if (newsize == 0) {
 Py_DECREF(v);
 *pv = tuple_get_empty();
 return 0;
 }
 if (oldsize == 0) {
 /* The empty tuple is statically allocated so we never
 resize it in-place. */
 Py_DECREF(v);
 *pv = PyTuple_New(newsize);
 return *pv == NULL ? -1 : 0;
 }
 ...
 sv = PyObject_GC_Resize(PyTupleObject, v, newsize);
 ...
}

Because tuples are immutable, operations like slicing, concatenation, and repetition can share or reuse objects: a full slice of an exact tuple returns the same object, and tuple * 1 / tuple + () return the original.

The real win: immutability is what lets CPython hand out the same tuple object in a dozen places instead of copying it. tuple(a) is a is True for a tuple.

Key takeaways

tuple is a PyObject_VAR_HEAD plus a cached ob_hash and an embedded ob_item array; ob_size is both the length and the capacity.
Items are stored in the same allocation as the header (flexible array via tp_basicsize/tp_itemsize); there’s no separate buffer.
The empty tuple is a statically allocated, non-GC singleton with a precomputed hash.
Exact tuples of sizes 1..20 are recycled through per-size freelists of up to 2000 entries; recycled objects are re-tracked and have their hash reset.
GC tracking is conservative: a tuple is only tracked if at least one item may be a GC object, and it can be untracked later via _PyTuple_MaybeUntrack.
The hash is a single-accumulator xxHash variant, computed once and cached (with -1 remapped to avoid clashing with the sentinel).
Immutability is enforced by the type (no sq_ass_item); internal mutation (PyTuple_SetItem, _PyTuple_Resize) is allowed only for uniquely referenced exact tuples.

CPython's Template String Implementation in C (t-strings): Interpolation as Data

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: Python 3.14+ template strings (t"...") don’t produce a str. They produce a string.templatelib.Template carrying two parallel sequences: the literal string pieces (strings) and the evaluated Interpolation objects (interpolations). Both types are C types, GC-tracked, built by two dedicated bytecodes.

Template holds the literal pieces and the interpolations side by side. Each Interpolation holds the value, the source expression text, an optional conversion flag, and an optional format spec. The implementation lives in Objects/templateobject.c and Objects/interpolationobject.c. Let’s trace it.

The `Interpolation` struct

 c
 
 
// Objects/interpolationobject.c
typedef struct {
 PyObject_HEAD
 PyObject *value;
 PyObject *expression;
 PyObject *conversion;
 PyObject *format_spec;
} interpolationobject;

The four fields are all owned references:

value – the evaluated result of the interpolated expression.
expression – the source text of the expression (a str, default "").
conversion – None or a one-character str ('s', 'r', 'a').
format_spec – the format specifier string (default "").

The type itself is a plain GC type:

 c
 
 
// Objects/interpolationobject.c
PyTypeObject _PyInterpolation_Type = {
 PyVarObject_HEAD_INIT(NULL, 0)
 .tp_name = "string.templatelib.Interpolation",
 .tp_basicsize = sizeof(interpolationobject),
 .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
 .tp_new = interpolation_new,
 .tp_dealloc = interpolation_dealloc,
 .tp_clear = interpolation_clear,
 .tp_free = PyObject_GC_Del,
 .tp_repr = interpolation_repr,
 .tp_members = interpolation_members,
 .tp_methods = interpolation_methods,
 .tp_traverse = interpolation_traverse,
};

tp_members exposes all four fields as read-only attributes, which is why interp.value, interp.expression, interp.conversion, and interp.format_spec work directly from Python:

 c
 
 
// Objects/interpolationobject.c
static PyMemberDef interpolation_members[] = {
 {"value", Py_T_OBJECT_EX, offsetof(interpolationobject, value), Py_READONLY, "Value"},
 {"expression", Py_T_OBJECT_EX, offsetof(interpolationobject, expression), Py_READONLY, "Expression"},
 {"conversion", Py_T_OBJECT_EX, offsetof(interpolationobject, conversion), Py_READONLY, "Conversion"},
 {"format_spec", Py_T_OBJECT_EX, offsetof(interpolationobject, format_spec), Py_READONLY, "Format specifier"},
 {NULL}
};

Constructing an `Interpolation`

The public constructor is generated by Argument Clinic from:

 c
 
 
// Objects/interpolationobject.c (clinic input)
@classmethod
Interpolation.__new__ as interpolation_new

 value: object
 expression: object(subclass_of='&PyUnicode_Type', c_default='&_Py_STR(empty)') = ""
 conversion: object(converter='_conversion_converter') = None
 format_spec: object(subclass_of='&PyUnicode_Type', c_default='&_Py_STR(empty)') = ""

The converter _conversion_converter validates that conversion is either None or a single-character string among 's', 'r', 'a':

 c
 
 
// Objects/interpolationobject.c
static int
_conversion_converter(PyObject *arg, PyObject **conversion)
{
 if (arg == Py_None) {
 return 1;
 }
 if (!PyUnicode_Check(arg)) {
 ...
 return 0;
 }
 Py_ssize_t len;
 const char *conv_str = PyUnicode_AsUTF8AndSize(arg, &len);
 if (len != 1 || !(conv_str[0] == 'a' || conv_str[0] == 'r' || conv_str[0] == 's')) {
 PyErr_SetString(PyExc_ValueError,
 "Interpolation() argument 'conversion' must be one of 's', 'a' or 'r'");
 return 0;
 }
 *conversion = arg;
 return 1;
}

The implementation allocates a GC object and takes new references to all four arguments:

 c
 
 
// Objects/interpolationobject.c
static PyObject *
interpolation_new_impl(PyTypeObject *type, PyObject *value,
 PyObject *expression, PyObject *conversion,
 PyObject *format_spec)
{
 interpolationobject *self = PyObject_GC_New(interpolationobject, type);
 if (!self) {
 return NULL;
 }

 self->value = Py_NewRef(value);
 self->expression = Py_NewRef(expression);
 self->conversion = Py_NewRef(conversion);
 self->format_spec = Py_NewRef(format_spec);
 PyObject_GC_Track(self);
 return (PyObject *) self;
}

There’s also an internal fast builder, _PyInterpolation_Build, used by the bytecode. It takes a C int conversion (one of FVC_NONE, FVC_STR, FVC_REPR, FVC_ASCII) rather than a Python object and maps it to the stored conversion value:

 c
 
 
// Objects/interpolationobject.c
PyObject *
_PyInterpolation_Build(PyObject *value, PyObject *str, int conversion, PyObject *format_spec)
{
 interpolationobject *interpolation = PyObject_GC_New(interpolationobject, &_PyInterpolation_Type);
 ...
 interpolation->value = Py_NewRef(value);
 interpolation->expression = Py_NewRef(str);
 interpolation->format_spec = Py_NewRef(format_spec);
 interpolation->conversion = NULL;

 if (conversion == 0) { // FVC_NONE
 interpolation->conversion = Py_None;
 }
 else {
 switch (conversion) {
 case FVC_ASCII: // 3
 interpolation->conversion = _Py_LATIN1_CHR('a');
 break;
 case FVC_REPR: // 2
 interpolation->conversion = _Py_LATIN1_CHR('r');
 break;
 case FVC_STR: // 1
 interpolation->conversion = _Py_LATIN1_CHR('s');
 break;
 ...
 }
 }
 PyObject_GC_Track(interpolation);
 return (PyObject *) interpolation;
}

Note that FVC_NONE is 0, which is why conversion == 0 is tested directly.

This builder is the hot path: it avoids going through __new__ and the converter on every interpolated value. Instead of allocating a fresh one-character string for 'a'/'r'/'s', it reaches for the immortal Latin-1 singleton.

_PyInterpolation_InitTypes sets __match_args__ to the four field names so that structural pattern matching works:

 c
 
 
// Objects/interpolationobject.c
PyStatus
_PyInterpolation_InitTypes(PyInterpreterState *interp)
{
 PyObject *tuple = Py_BuildValue("(ssss)", "value", "expression", "conversion", "format_spec");
 ...
 int status = PyDict_SetItemString(dict, "__match_args__", tuple);
 ...
}

Pickling is supported through __reduce__, which returns the class and a 4-tuple of fields:

 c
 
 
// Objects/interpolationobject.c
static PyObject*
interpolation_reduce(PyObject *op, PyObject *Py_UNUSED(dummy))
{
 interpolationobject *self = interpolationobject_CAST(op);
 return Py_BuildValue("(O(OOOO))", (PyObject *)Py_TYPE(op),
 self->value, self->expression,
 self->conversion, self->format_spec);
}

The `Template` struct

 c
 
 
// Objects/templateobject.c
typedef struct {
 PyObject_HEAD
 PyObject *strings;
 PyObject *interpolations;
} templateobject;

strings is a tuple of str whose length is len(interpolations) + 1; interpolations is a tuple of Interpolation. Both are owned references, and the object is GC-tracked:

 c
 
 
// Objects/templateobject.c
PyTypeObject _PyTemplate_Type = {
 PyVarObject_HEAD_INIT(NULL, 0)
 .tp_name = "string.templatelib.Template",
 .tp_basicsize = sizeof(templateobject),
 .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
 .tp_as_sequence = &template_as_sequence,
 .tp_new = template_new,
 .tp_dealloc = template_dealloc,
 .tp_clear = template_clear,
 .tp_free = PyObject_GC_Del,
 .tp_repr = template_repr,
 .tp_members = template_members,
 .tp_methods = template_methods,
 .tp_getset = template_getset,
 .tp_iter = template_iter,
 .tp_traverse = template_traverse,
};

The two tuple fields are exposed read-only:

 c
 
 
// Objects/templateobject.c
static PyMemberDef template_members[] = {
 {"strings", Py_T_OBJECT_EX, offsetof(templateobject, strings), Py_READONLY, "Strings"},
 {"interpolations", Py_T_OBJECT_EX, offsetof(templateobject, interpolations), Py_READONLY, "Interpolations"},
 {NULL},
};

A computed values property returns a tuple of the interpolation values, mirroring Template.values in the Python-level API:

 c
 
 
// Objects/templateobject.c
static PyObject *
template_values_get(PyObject *op, void *Py_UNUSED(data))
{
 templateobject *self = templateobject_CAST(op);
 Py_ssize_t len = PyTuple_GET_SIZE(self->interpolations);
 PyObject *values = PyTuple_New(len);
 if (values == NULL) {
 return NULL;
 }
 for (Py_ssize_t i = 0; i < len; i++) {
 PyObject *item = PyTuple_GET_ITEM(self->interpolations, i);
 PyTuple_SET_ITEM(values, i, _PyInterpolation_GetValueRef(item));
 }
 return values;
}

_PyInterpolation_GetValueRef simply returns a new reference to the interpolation’s value field.

`Template.new`: normalizing the argument list

The constructor accepts a flat sequence of str and Interpolation objects and normalizes them into the two parallel tuples. Consecutive strings are concatenated; an Interpolation with no preceding string gets an empty string inserted before it; a trailing interpolation gets an empty string appended:

 c
 
 
// Objects/templateobject.c
static PyObject *
template_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
 if (kwds != NULL) {
 PyErr_SetString(PyExc_TypeError, "Template.__new__ only accepts *args arguments");
 return NULL;
 }

 Py_ssize_t argslen = PyTuple_GET_SIZE(args);
 Py_ssize_t stringslen = 0;
 Py_ssize_t interpolationslen = 0;
 int last_was_str = 0;

 for (Py_ssize_t i = 0; i < argslen; i++) {
 PyObject *item = PyTuple_GET_ITEM(args, i);
 if (PyUnicode_Check(item)) {
 if (!last_was_str) {
 stringslen++;
 }
 last_was_str = 1;
 }
 else if (_PyInterpolation_CheckExact(item)) {
 if (!last_was_str) {
 stringslen++;
 }
 interpolationslen++;
 last_was_str = 0;
 }
 else {
 PyErr_Format(PyExc_TypeError,
 "Template.__new__ *args need to be of type 'str' or 'Interpolation', got %T",
 item);
 return NULL;
 }
 }
 if (!last_was_str) {
 stringslen++;
 }
 ...
}

The second pass performs the concatenation and empty-string insertion, then delegates to _PyTemplate_Build:

 c
 
 
// Objects/templateobject.c (second pass, abbreviated)
for (Py_ssize_t i = 0; i < argslen; i++) {
 PyObject *item = PyTuple_GET_ITEM(args, i);
 if (PyUnicode_Check(item)) {
 if (last_was_str) {
 PyObject *laststring = PyTuple_GET_ITEM(strings, stringsidx - 1);
 PyObject *concat = PyUnicode_Concat(laststring, item);
 ...
 PyTuple_SET_ITEM(strings, stringsidx - 1, concat);
 Py_DECREF(laststring);
 }
 else {
 PyTuple_SET_ITEM(strings, stringsidx++, Py_NewRef(item));
 }
 last_was_str = 1;
 }
 else if (_PyInterpolation_CheckExact(item)) {
 if (!last_was_str) {
 _Py_DECLARE_STR(empty, "");
 PyTuple_SET_ITEM(strings, stringsidx++, &_Py_STR(empty));
 }
 PyTuple_SET_ITEM(interpolations, interpolationsidx++, Py_NewRef(item));
 last_was_str = 0;
 }
}
if (!last_was_str) {
 _Py_DECLARE_STR(empty, "");
 PyTuple_SET_ITEM(strings, stringsidx++, &_Py_STR(empty));
}

PyObject *template = _PyTemplate_Build(strings, interpolations);
Py_DECREF(strings);
Py_DECREF(interpolations);
return template;

This normalization is what lets Template("a", interp, "b") and Template("a", interp, "b", interp2) both satisfy the invariant that len(strings) == len(interpolations) + 1.

`_PyTemplate_Build`: the factory

 c
 
 
// Objects/templateobject.c
PyObject *
_PyTemplate_Build(PyObject *strings, PyObject *interpolations)
{
 templateobject *template = PyObject_GC_New(templateobject, &_PyTemplate_Type);
 if (template == NULL) {
 return NULL;
 }

 template->strings = Py_NewRef(strings);
 template->interpolations = Py_NewRef(interpolations);
 PyObject_GC_Track(template);
 return (PyObject *) template;
}

Like _PyInterpolation_Build, this is the internal fast path used by the bytecode and by template_new itself.

Iteration: `TemplateIter`

Template.__iter__ returns a GC-tracked templateiterobject that alternates between the strings and interpolations iterators, skipping empty strings that result from adjacent interpolations:

 c
 
 
// Objects/templateobject.c
typedef struct {
 PyObject_HEAD
 PyObject *stringsiter;
 PyObject *interpolationsiter;
 int from_strings;
} templateiterobject;

 c
 
 
// Objects/templateobject.c
static PyObject *
templateiter_next(PyObject *op)
{
 templateiterobject *self = templateiterobject_CAST(op);
 PyObject *item;
 if (self->from_strings) {
 item = PyIter_Next(self->stringsiter);
 self->from_strings = 0;
 if (item == NULL) {
 return NULL;
 }
 if (PyUnicode_GET_LENGTH(item) == 0) {
 Py_SETREF(item, PyIter_Next(self->interpolationsiter));
 self->from_strings = 1;
 }
 }
 else {
 item = PyIter_Next(self->interpolationsiter);
 self->from_strings = 1;
 }
 return item;
}

The from_strings flag starts at 1, so iteration begins with a string. When that string is empty (which happens between two interpolations), it’s replaced by the next interpolation and the flag is reset so the following string is consumed next. That yields the familiar str/Interpolation alternation you get from for part in template:.

template_iter constructs the iterator and obtains iterators over both tuples:

 c
 
 
// Objects/templateobject.c
static PyObject *
template_iter(PyObject *op)
{
 templateobject *self = templateobject_CAST(op);
 templateiterobject *iter = PyObject_GC_New(templateiterobject, &_PyTemplateIter_Type);
 ...
 PyObject *stringsiter = PyObject_GetIter(self->strings);
 ...
 PyObject *interpolationsiter = PyObject_GetIter(self->interpolations);
 ...
 iter->stringsiter = stringsiter;
 iter->interpolationsiter = interpolationsiter;
 iter->from_strings = 1;
 PyObject_GC_Track(iter);
 return (PyObject *)iter;
}

Concatenation

Template supports + via tp_as_sequence.sq_concat. When both operands are exact Template instances, their strings and interpolations are concatenated, merging the boundary strings:

 c
 
 
// Objects/templateobject.c
static PyObject *
template_strings_concat(PyObject *left, PyObject *right)
{
 Py_ssize_t left_stringslen = PyTuple_GET_SIZE(left);
 PyObject *left_laststring = PyTuple_GET_ITEM(left, left_stringslen - 1);
 Py_ssize_t right_stringslen = PyTuple_GET_SIZE(right);
 PyObject *right_firststring = PyTuple_GET_ITEM(right, 0);

 PyObject *concat = PyUnicode_Concat(left_laststring, right_firststring);
 ...
 PyObject *newstrings = PyTuple_New(left_stringslen + right_stringslen - 1);
 ...
 // copy left[0..n-2], then concat, then right[1..]
}

 c
 
 
// Objects/templateobject.c
static PyObject *
template_concat_templates(templateobject *self, templateobject *other)
{
 PyObject *newstrings = template_strings_concat(self->strings, other->strings);
 ...
 PyObject *newinterpolations = PySequence_Concat(self->interpolations, other->interpolations);
 ...
 PyObject *newtemplate = _PyTemplate_Build(newstrings, newinterpolations);
 ...
}

_PyTemplate_Concat is the public entry point and rejects non-Template operands with a TypeError.

Pickling

Template.__reduce__ delegates to string.templatelib._template_unpickle, passing the two tuples:

 c
 
 
// Objects/templateobject.c
static PyObject*
template_reduce(PyObject *op, PyObject *Py_UNUSED(dummy))
{
 PyObject *mod = PyImport_ImportModule("string.templatelib");
 ...
 PyObject *func = PyObject_GetAttrString(mod, "_template_unpickle");
 ...
 templateobject *self = templateobject_CAST(op);
 PyObject *result = Py_BuildValue("O(OO)", func, self->strings, self->interpolations);
 ...
}

The Python helper _template_unpickle in Lib/string/templatelib.py zips the strings and interpolations back into the flat str/Interpolation argument list and calls Template(*parts), re-running the normalization in template_new.

From source to object: the compiler and bytecode

The parser produces a TemplateStr expression node whose values sequence contains Constant (string) and Interpolation nodes. The compiler’s codegen_template (in Python/codegen.c) emits the strings as a tuple, the interpolations as a tuple, and then BUILD_TEMPLATE:

 c
 
 
// Python/codegen.c
Py_ssize_t stringslen = 0;
for (Py_ssize_t i = 0; i < value_count; i++) {
 value = asdl_seq_GET(e->v.TemplateStr.values, i);
 if (value->kind == Interpolation_kind) {
 if (last_was_interpolation) {
 ADDOP_LOAD_CONST(c, loc, Py_NewRef(&_Py_STR(empty)));
 stringslen++;
 }
 last_was_interpolation = 1;
 }
 else {
 VISIT(c, expr, value);
 stringslen++;
 last_was_interpolation = 0;
 }
}
if (last_was_interpolation) {
 ADDOP_LOAD_CONST(c, loc, Py_NewRef(&_Py_STR(empty)));
 stringslen++;
}
ADDOP_I(c, loc, BUILD_TUPLE, stringslen);

Py_ssize_t interpolationslen = 0;
for (Py_ssize_t i = 0; i < value_count; i++) {
 value = asdl_seq_GET(e->v.TemplateStr.values, i);
 if (value->kind == Interpolation_kind) {
 VISIT(c, expr, value);
 interpolationslen++;
 }
}
ADDOP_I(c, loc, BUILD_TUPLE, interpolationslen);
ADDOP(c, loc, BUILD_TEMPLATE);

Each Interpolation node is compiled by codegen_interpolation, which visits the value, loads the expression string constant, optionally visits the format spec, encodes the conversion into the oparg, and emits BUILD_INTERPOLATION:

 c
 
 
// Python/codegen.c
VISIT(c, expr, e->v.Interpolation.value);
ADDOP_LOAD_CONST(c, loc, e->v.Interpolation.str);

int oparg = 2;
if (e->v.Interpolation.format_spec) {
 oparg++;
 VISIT(c, expr, e->v.Interpolation.format_spec);
}

int conversion = e->v.Interpolation.conversion;
if (conversion != -1) {
 switch (conversion) {
 case 's': oparg |= FVC_STR << 2; break;
 case 'r': oparg |= FVC_REPR << 2; break;
 case 'a': oparg |= FVC_ASCII << 2; break;
 ...
 }
}

ADDOP_I(c, loc, BUILD_INTERPOLATION, oparg);

The bytecode implementation consumes the stack and calls the C builders:

 c
 
 
// Python/bytecodes.c
inst(BUILD_INTERPOLATION, (value, str, format[oparg & 1] -- interpolation)) {
 PyObject *value_o = PyStackRef_AsPyObjectBorrow(value);
 PyObject *str_o = PyStackRef_AsPyObjectBorrow(str);
 int conversion = oparg >> 2;
 PyObject *format_o;
 if (oparg & 1) {
 format_o = PyStackRef_AsPyObjectBorrow(format[0]);
 }
 else {
 format_o = &_Py_STR(empty);
 }
 PyObject *interpolation_o = _PyInterpolation_Build(value_o, str_o, conversion, format_o);
 ...
 interpolation = PyStackRef_FromPyObjectSteal(interpolation_o);
}

inst(BUILD_TEMPLATE, (strings, interpolations -- template)) {
 PyObject *strings_o = PyStackRef_AsPyObjectBorrow(strings);
 PyObject *interpolations_o = PyStackRef_AsPyObjectBorrow(interpolations);
 PyObject *template_o = _PyTemplate_Build(strings_o, interpolations_o);
 PyStackRef_CLOSE(interpolations);
 PyStackRef_CLOSE(strings);
 ERROR_IF(template_o == NULL);
 template = PyStackRef_FromPyObjectSteal(template_o);
}

The oparg & 1 bit of BUILD_INTERPOLATION indicates whether a format spec was supplied; the conversion is carried in the upper bits (oparg >> 2).

Memory layout and GC

Both types are GC objects with tp_traverse/tp_clear visiting every owned reference:

 c
 
 
// Objects/interpolationobject.c
static int
interpolation_traverse(PyObject *op, visitproc visit, void *arg)
{
 interpolationobject *self = interpolationobject_CAST(op);
 Py_VISIT(self->value);
 Py_VISIT(self->expression);
 Py_VISIT(self->conversion);
 Py_VISIT(self->format_spec);
 return 0;
}

 c
 
 
// Objects/templateobject.c
static int
template_traverse(PyObject *op, visitproc visit, void *arg)
{
 templateobject *self = templateobject_CAST(op);
 Py_VISIT(self->strings);
 Py_VISIT(self->interpolations);
 return 0;
}

The iterator is also GC-tracked because it holds two iterators. All three use PyObject_GC_Del as tp_free and untrack themselves in tp_dealloc before clearing.

The on-disk sizes are small: an Interpolation is a PyObject header plus four pointers, and a Template is a header plus two pointers. Both are allocated through PyObject_GC_New, so they participate in cyclic garbage collection like tuples and lists.

Key takeaways

Template and Interpolation are GC-tracked C types defined in Objects/templateobject.c and Objects/interpolationobject.c, exported as string.templatelib.Template and string.templatelib.Interpolation.
An Interpolation stores value, expression, conversion (None or a one-char str), and format_spec; conversion is validated by _conversion_converter in the public constructor and synthesized from FVC_* constants by _PyInterpolation_Build on the bytecode path.
A Template stores strings and interpolations tuples satisfying len(strings) == len(interpolations) + 1; template_new normalizes a flat argument list into that invariant.
Iteration alternates strings and interpolations, skipping empty boundary strings; concatenation merges the boundary strings.
The compiler emits BUILD_INTERPOLATION and BUILD_TEMPLATE, which call _PyInterpolation_Build and _PyTemplate_Build.
Both objects are GC-tracked with full traverse/clear support and support pickling via __reduce__.

CPython's super() Implementation in C: Cooperative Multiple Inheritance

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: super() isn’t a special syntax node. It’s an ordinary C type, PySuper_Type, holding three pointers: the class that called it, the object being searched, and the type whose MRO gets walked. And super().attr doesn’t even build one — LOAD_SUPER_ATTR calls straight into _PySuper_Lookup.

super() is the cooperative multiple-inheritance helper. Almost everything interesting — MRO traversal, descriptor binding, zero-argument super() — lives in Objects/typeobject.c.

The `superobject` struct

 c
 
 
// Objects/typeobject.c
typedef struct {
 PyObject_HEAD
 PyTypeObject *type; /* class invoking super() (__thisclass__) */
 PyObject *obj; /* instance or class (__self__) */
 PyTypeObject *obj_type; /* type of obj (__self_class__) */
} superobject;

The three fields are exposed read-only through tp_members:

 c
 
 
static PyMemberDef super_members[] = {
 {"__thisclass__", _Py_T_OBJECT, offsetof(superobject, type), Py_READONLY, ...},
 {"__self__", _Py_T_OBJECT, offsetof(superobject, obj), Py_READONLY, ...},
 {"__self_class__", _Py_T_OBJECT, offsetof(superobject, obj_type), Py_READONLY, ...},
 {0}
};

type is always a PyTypeObject *. obj may be NULL (unbound super(type) or class-level access) and obj_type then is NULL too.

And obj_type is the MRO we actually walk: for an instance it’s type(obj) (or obj.__class__); for a class argument it’s the class itself.

Lifecycle is ordinary GC-tracked:

 c
 
 
static void super_dealloc(PyObject *self) {
 superobject *su = superobject_CAST(self);
 _PyObject_GC_UNTRACK(self);
 Py_XDECREF(su->obj);
 Py_XDECREF(su->type);
 Py_XDECREF(su->obj_type);
 Py_TYPE(self)->tp_free(self);
}

static int super_traverse(PyObject *self, visitproc visit, void *arg) {
 superobject *su = superobject_CAST(self);
 Py_VISIT(su->obj);
 Py_VISIT(su->type);
 Py_VISIT(su->obj_type);
 return 0;
}

Construction: `super_init_impl`

PySuper_Type.tp_new is PyType_GenericNew and tp_init is super_init, which delegates to super_init_impl. The argument forms are super(), super(type), and super(type, obj):

 c
 
 
static inline int
super_init_impl(PyObject *self, PyTypeObject *type, PyObject *obj) {
 superobject *su = superobject_CAST(self);
 PyTypeObject *obj_type = NULL;
 if (type == NULL) {
 /* super() with no args: fill from __class__ and first local */
 PyThreadState *tstate = _PyThreadState_GET();
 _PyInterpreterFrame *frame = _PyThreadState_GetFrame(tstate);
 ...
 int res = super_init_without_args(frame, &type, &obj);
 ...
 }
 else {
 Py_INCREF(type);
 Py_XINCREF(obj);
 }
 if (obj == Py_None) { Py_DECREF(obj); obj = NULL; }
 if (obj != NULL) {
 obj_type = supercheck(type, obj); /* validates + returns obj_type */
 ...
 }
 Py_XSETREF(su->type, (PyTypeObject*)type);
 Py_XSETREF(su->obj, obj);
 Py_XSETREF(su->obj_type, obj_type);
 return 0;
}

supercheck is the gatekeeper. It returns the type whose MRO we walk:

 c
 
 
static PyTypeObject *
supercheck(PyTypeObject *type, PyObject *obj)
{
 /* class argument: super(type, cls) */
 if (PyType_Check(obj) && PyType_IsSubtype((PyTypeObject *)obj, type)) {
 return (PyTypeObject *)Py_NewRef(obj);
 }
 /* instance argument: super(type, instance) */
 if (PyType_IsSubtype(Py_TYPE(obj), type)) {
 return (PyTypeObject*)Py_NewRef(Py_TYPE(obj));
 }
 else {
 /* proxy case: obj.__class__ may differ from Py_TYPE(obj) */
 PyObject *class_attr;
 if (PyObject_GetOptionalAttr(obj, &_Py_ID(__class__), &class_attr) < 0)
 return NULL;
 if (class_attr != NULL && PyType_Check(class_attr) &&
 (PyTypeObject *)class_attr != Py_TYPE(obj)) {
 int ok = PyType_IsSubtype((PyTypeObject *)class_attr, type);
 if (ok) return (PyTypeObject *)class_attr; /* transfers the GetOptionalAttr ref */
 }
 Py_XDECREF(class_attr);
 }
 ... raise TypeError ...
}

The proxy branch is what lets super() work with objects whose __class__ is overridden.

Zero-argument `super()`

When called as super() inside a method, CPython needs __class__ and the first positional argument. super_init_without_args pulls them out of the current frame: the first local (possibly a cell) is the instance, and the __class__ free variable is the class.

This is the machinery behind the implicit __class__ cell the compiler creates for methods:

 c
 
 
static int super_init_without_args(_PyInterpreterFrame *cframe, ...) {
 PyObject *firstarg = ... locals array[0] ...; /* instance or class */
 ...
 /* find the __class__ free var */
 for (; i < co->co_nlocalsplus; i++) {
 if (_PyUnicode_Equal(name, &_Py_ID(__class__))) {
 type = (PyTypeObject *) PyCell_GetRef(cell);
 ...
 }
 }
 *type_p = type;
 *obj_p = firstarg;
 return 0;
}

Worth internalizing: the compiler smuggles __class__ into the method’s closure, and super() reads it back out of the frame at runtime. That’s why zero-arg super() only works inside a method body, and why it breaks inside a nested function that isn’t a method.

`super` as a descriptor: `super_descr_get`

PySuper_Type.tp_descr_get = super_descr_get. So when super is stored in a class body and accessed on an instance, it binds to that instance:

 c
 
 
static PyObject *
super_descr_get(PyObject *self, PyObject *obj, PyObject *type)
{
 superobject *su = superobject_CAST(self);
 if (obj == NULL || obj == Py_None || su->obj != NULL) {
 /* class-level access or already bound */
 return Py_NewRef(self);
 }
 if (!Py_IS_TYPE(su, &PySuper_Type)) /* subclass: delegate */
 return PyObject_CallFunctionObjArgs((PyObject *)Py_TYPE(su), su->type, obj, NULL);
 else {
 PyTypeObject *obj_type = supercheck(su->type, obj);
 ...
 newobj = (superobject *)PySuper_Type.tp_new(&PySuper_Type, NULL, NULL);
 newobj->type = (PyTypeObject*)Py_NewRef(su->type);
 newobj->obj = Py_NewRef(obj);
 newobj->obj_type = obj_type;
 return (PyObject *)newobj;
 }
}

That’s why super() inside a method can be used unbound at class scope and bound on instance access.

MRO traversal: `_PySuper_LookupDescr`

The core lookup skips everything up to and including su->type in the MRO of su->obj_type, then searches each following class’s __dict__:

 c
 
 
PyObject *
_PySuper_LookupDescr(PyTypeObject *su_type, PyTypeObject *su_obj_type, PyObject *name)
{
 PyObject *mro = lookup_tp_mro(su_obj_type); /* PyTuple of types */
 ...
 n = PyTuple_GET_SIZE(mro);
 for (i = 0; i+1 < n; i++) {
 if ((PyObject *)su_type == PyTuple_GET_ITEM(mro, i))
 break;
 }
 i++; /* skip su->type */
 if (i >= n) return NULL;

 do {
 PyObject *obj = PyTuple_GET_ITEM(mro, i);
 PyObject *dict = lookup_tp_dict(_PyType_CAST(obj));
 if (PyDict_GetItemRef(dict, name, &res) != 0) {
 /* found or error */
 return res;
 }
 i++;
 } while (i < n);
 return NULL;
}

Four details worth remembering:

It looks in tp_dict of each class, not via tp_getattro, so it does not re-trigger descriptors on the super object or __getattr__.
It uses PyDict_GetItemRef, which returns a strong reference and distinguishes “not found” (0) from “error” (-1).
The MRO is pinned with a C-stack reference because tp_mro can be replaced mid-lookup.
It returns the raw dict value. Descriptor binding happens in the caller.

Attribute lookup: `super_getattro` and `do_super_lookup`

super_getattro is PySuper_Type.tp_getattro. It special-cases __class__ (so super().__class__ returns super, not the instance’s class) and otherwise calls do_super_lookup:

 c
 
 
static PyObject *
super_getattro(PyObject *self, PyObject *name)
{
 superobject *su = superobject_CAST(self);
 if (PyUnicode_Check(name) && PyUnicode_GET_LENGTH(name) == 9 &&
 _PyUnicode_Equal(name, &_Py_ID(__class__)))
 return PyObject_GenericGetAttr(self, name);

 return do_super_lookup(su, su->type, su->obj, su->obj_type, name, NULL);
}

do_super_lookup is where the descriptor binding happens:

 c
 
 
static PyObject *
do_super_lookup(superobject *su, PyTypeObject *su_type, PyObject *su_obj,
 PyTypeObject *su_obj_type, PyObject *name, int *method)
{
 PyObject *res;
 int temp_su = 0;

 if (su_obj_type == NULL) goto skip;

 res = _PySuper_LookupDescr(su_type, su_obj_type, name);
 if (res != NULL) {
 if (method && _PyType_HasFeature(Py_TYPE(res), Py_TPFLAGS_METHOD_DESCRIPTOR)) {
 *method = 1; /* method descriptor: caller handles binding */
 }
 else {
 descrgetfunc f = Py_TYPE(res)->tp_descr_get;
 if (f != NULL) {
 PyObject *res2 = f(res,
 /* pass obj only in instance mode */
 (su_obj == (PyObject *)su_obj_type) ? NULL : su_obj,
 (PyObject *)su_obj_type);
 Py_SETREF(res, res2);
 }
 }
 return res;
 }
 else if (PyErr_Occurred()) return NULL;

skip:
 /* Fallback: build a temporary super() and use generic getattr */
 if (su == NULL) { ... su = (superobject *)PyObject_Vectorcall(...); temp_su=1; }
 res = PyObject_GenericGetAttr((PyObject *)su, name);
 if (temp_su) Py_DECREF(su);
 return res;
}

Two things to notice:

When the found object’s type has tp_descr_get, it’s invoked with (descriptor, instance_or_NULL, su_obj_type). For class-mode super (super(type, cls)), su_obj == su_obj_type, so obj is passed as NULL — the descriptor runs in class-level mode. For instance mode, the instance is passed, so functions become bound methods.
Method descriptors (Py_TPFLAGS_METHOD_DESCRIPTOR) are returned raw and flagged via *method, letting the bytecode layer avoid a second descriptor call and push self alongside the attribute.

The skip fallback covers the unbound case (super(type) with no object) and any path where no dict entry is found, by constructing a temporary super object and running generic attribute lookup (which can hit __getattr__).

The C API and bytecode fast path

_PySuper_Lookup wraps the whole thing for callers that don’t hold a superobject:

 c
 
 
PyObject *
_PySuper_Lookup(PyTypeObject *su_type, PyObject *su_obj, PyObject *name, int *method)
{
 PyTypeObject *su_obj_type = supercheck(su_type, su_obj);
 if (su_obj_type == NULL) return NULL;
 PyObject *res = do_super_lookup(NULL, su_type, su_obj, su_obj_type, name, method);
 Py_DECREF(su_obj_type);
 return res;
}

The compiler lowers super().attr / super().meth(...) to LOAD_SUPER_ATTR. The specialized LOAD_SUPER_ATTR_ATTR and LOAD_SUPER_ATTR_METHOD forms call _PySuper_Lookup directly with the class and self on the stack, skipping construction of a super object entirely:

 c
 
 
// Python/bytecodes.c
inst(LOAD_SUPER_ATTR_ATTR, ...) {
 ...
 PyObject *attr = _PySuper_Lookup((PyTypeObject *)class, self, name, NULL);
 ...
}
op(_LOAD_SUPER_ATTR_METHOD, ...) {
 ...
 attr_o = _PySuper_Lookup(cls, self, name,
 Py_TYPE(self)->tp_getattro == PyObject_GenericGetAttr ? method_found_ptr : NULL);
 if (method_found) {
 self_or_null = self_st; /* push self so caller can call attr(self, ...) */
 }
 ...
}

The real win: for the common super().method() call, CPython allocates no super object at all. It goes straight from the class and self on the stack to the bound descriptor.

The JIT optimizer also uses _PySuper_LookupDescr directly (via lookup_super_attr) to cache the found descriptor when it’s a method descriptor, which is what makes super().method() in hot loops fast.

Key takeaways

super is PySuper_Type, a GC-tracked type with superobject (type, obj, obj_type) and read-only __thisclass__/__self__/__self_class__.
supercheck validates the object and returns the MRO-bearing type, including a proxy-friendly __class__ fallback.
Zero-arg super() extracts __class__ and the first local from the current frame.
super_descr_get binds an unbound super to an instance by cloning the struct.
MRO traversal in _PySuper_LookupDescr starts after su->type and scans each later class’s tp_dict with PyDict_GetItemRef.
do_super_lookup invokes tp_descr_get on the found object, passing the instance only in instance mode, and flags method descriptors.
LOAD_SUPER_ATTR bypasses super object construction entirely via _PySuper_Lookup.

CPython's structseq Implementation in C (PyStructSequence): Tuples With Names

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: Python’s C-level “named tuples” aren’t collections.namedtuple — they’re struct sequences, tuple subclasses with named fields. PyStructSequence is literally an alias for PyTupleObject, and hidden fields are smuggled into the same ob_item array past ob_size.

sys.version_info, os.stat_result, time.struct_time, and os.uname_result are all struct sequences. In CPython the machinery lives in Objects/structseq.c and is built around PyStructSequence, which is literally an alias for PyTupleObject:

 c
 
 
// Include/cpython/structseq.h
typedef PyTupleObject PyStructSequence;
#define PyStructSequence_SET_ITEM PyStructSequence_SetItem
#define PyStructSequence_GET_ITEM PyStructSequence_GetItem

Let’s trace that implementation through the 3.15 source.

The idea: a tuple with named, sometimes hidden, fields

A struct sequence type is described by a PyStructSequence_Desc:

 c
 
 
// Include/structseq.h
typedef struct PyStructSequence_Field {
 const char *name;
 const char *doc;
} PyStructSequence_Field;

typedef struct PyStructSequence_Desc {
 const char *name;
 const char *doc;
 PyStructSequence_Field *fields;
 int n_in_sequence; /* how many fields are visible as tuple items */
} PyStructSequence_Desc;

The fields array is NULL-terminated. n_in_sequence says how many of those fields participate in the sequence part (tuple length, indexing, unpacking); the rest are “hidden” fields reachable only as attributes.

os.stat_result is the canonical example: its first 10 fields form the tuple, while st_atime, st_mtime, st_ctime, st_atime_ns, … are extra attributes stored in the same object but not counted in len().

 c
 
 
// Modules/posixmodule.c
static PyStructSequence_Desc stat_result_desc = {
 "os.stat_result",
 stat_result__doc__,
 stat_result_fields,
 10 /* n_in_sequence */
};

Fields can also be unnamed. PyStructSequence_UnnamedField (the string "unnamed field") marks a slot that has an index but no attribute name; the first three time fields of stat_result (st_atime’s integer predecessor) use this. Unnamed fields count toward the sequence length but get no tp_members entry.

Visible size vs. real size

Because hidden fields must not show up to Python code (len(), indexing, iteration, unpacking), the object’s ob_size is hacked to the visible count, while the allocation still holds all fields. Two macros capture this:

 c
 
 
// Objects/structseq.c
#define VISIBLE_SIZE(op) Py_SIZE(op)
#define REAL_SIZE(op) get_real_size((PyObject *)op)

static Py_ssize_t
get_real_size(PyObject *op)
{
 // Compute the real size from the visible size (i.e., Py_SIZE()) and the
 // number of non-sequence fields accounted for in tp_basicsize.
 Py_ssize_t hidden = Py_TYPE(op)->tp_basicsize
 - offsetof(PyStructSequence, ob_item);
 return Py_SIZE(op) + hidden / sizeof(PyObject *);
}

The type’s tp_basicsize is deliberately inflated to cover the hidden slots, so they live in the same contiguous ob_item array but beyond ob_size:

 c
 
 
// Objects/structseq.c
Py_ssize_t n_hidden = n_members - desc->n_in_sequence;
type->tp_basicsize = sizeof(PyStructSequence) + (n_hidden - 1) * sizeof(PyObject *);
type->tp_itemsize = sizeof(PyObject *);

This mirrors the tuple layout: tp_basicsize is the header plus the hidden slots (minus one, because PyStructSequence already contains ob_item[1]), and tp_itemsize is one slot. PyObject_GC_NewVar(type, size) with size = REAL_SIZE therefore allocates visible + hidden slots — and then the size is rewritten to the visible count:

 c
 
 
// Objects/structseq.c
PyObject *
PyStructSequence_New(PyTypeObject *type)
{
 PyStructSequence *obj;
 Py_ssize_t size = REAL_SIZE_TP(type), i;
 ...
 Py_ssize_t vsize = VISIBLE_SIZE_TP(type);
 ...

 obj = PyObject_GC_NewVar(PyStructSequence, type, size);
 if (obj == NULL)
 return NULL;
 _PyTuple_RESET_HASH_CACHE(obj);
 /* Hack the size of the variable object, so invisible fields don't appear
 to Python code. */
 Py_SET_SIZE(obj, vsize);
 for (i = 0; i < size; i++)
 obj->ob_item[i] = NULL;

 return (PyObject*)obj;
}

So len(os.stat_result(...)) == 10 even though the object holds ~17 slots, and tuple(os.stat_result(...)) only yields the 10 visible ones. Indexing beyond the visible range isn’t exposed; hidden fields are attributes.

Here’s the trick: the hidden data isn’t in a side table. It’s the same array, just past the end of what ob_size admits.

Memory layout

For a type with V visible and H hidden fields:

text

+----------------------------------+ <- PyObject_GC_NewVar result
| ob_refcnt |
+----------------------------------+
| *ob_type (&TheStructSeqType) |
+----------------------------------+
| ob_size (= V, "visible") |
+----------------------------------+
| ob_hash (cached, from tuple) |
+----------------------------------+
| ob_item[0] (visible field 0) |
| ... |
| ob_item[V-1] (visible field V-1) |
| ob_item[V] (hidden field 0) |
| ... |
| ob_item[V+H-1] (hidden field H-1)|
+----------------------------------+

H hidden slots are absorbed into tp_basicsize, so ob_size stays V. The hash cache (ob_hash) is inherited from PyTupleObject and reset on creation via _PyTuple_RESET_HASH_CACHE, which is why struct sequences hash like tuples over their visible fields.

Field access: attributes via `tp_members`

Named (non-unnamed) fields become PyMemberDef entries whose offset points directly into ob_item:

 c
 
 
// Objects/structseq.c
members[k].name = desc->fields[i].name;
members[k].type = _Py_T_OBJECT;
members[k].offset = offsetof(PyStructSequence, ob_item)
 + i * sizeof(PyObject*);
members[k].flags = Py_READONLY;

_Py_T_OBJECT (read-only object member) gives obj.field returning a borrowed reference without calling a getter, and Py_READONLY makes assignment raise AttributeError.

Unnamed fields are skipped, so they have no attribute. Because the offset is computed with the real index i (including unnamed fields before it), the member table stays aligned with ob_item.

PyStructSequence_SetItem / GetItem are thin wrappers that cast to PyTupleObject and index ob_item directly, with no refcount changes on set:

 c
 
 
// Objects/structseq.c
void
PyStructSequence_SetItem(PyObject *op, Py_ssize_t index, PyObject *value)
{
 PyTupleObject *tuple = _PyTuple_CAST(op);
 ...
 tuple->ob_item[index] = value; // no incref; caller owns reference
}

PyObject *
PyStructSequence_GetItem(PyObject *op, Py_ssize_t index)
{
 ...
 return PyTuple_GET_ITEM(op, index);
}

This matches the tuple convention: PyTuple_SetItem steals a reference and is only safe on a uniquely referenced, not-yet-published object. Typical C usage is exactly that – PyStructSequence_New then SET_ITEM for each slot, then _PyObject_GC_TRACK.

Type creation

There are two paths. Builtin static types (version_info, stat_result, struct_time) use _PyStructSequence_InitBuiltinWithFlags, which fills in a pre-declared PyTypeObject:

 c
 
 
// Objects/structseq.c
int
_PyStructSequence_InitBuiltinWithFlags(PyInterpreterState *interp,
 PyTypeObject *type,
 PyStructSequence_Desc *desc,
 unsigned long tp_flags)
{
 ...
 members = initialize_members(desc, n_members, n_unnamed_members);
 initialize_static_fields(type, desc, members, n_members, tp_flags);
 _Py_SetImmortal((PyObject *)type);
 ...
 _PyStaticType_InitBuiltin(interp, type);
 initialize_structseq_dict(desc, _PyType_GetDict(type), n_members, n_unnamed_members);
 ...
}

initialize_static_fields wires everything up:

 c
 
 
// Objects/structseq.c
static void
initialize_static_fields(PyTypeObject *type, PyStructSequence_Desc *desc,
 PyMemberDef *tp_members, Py_ssize_t n_members,
 unsigned long tp_flags)
{
 type->tp_name = desc->name;
 ...
 type->tp_dealloc = structseq_dealloc;
 type->tp_repr = structseq_repr;
 type->tp_doc = desc->doc;
 type->tp_base = &PyTuple_Type; // <-- tuple subclass
 type->tp_methods = structseq_methods; // __reduce__, __replace__
 type->tp_new = structseq_new;
 type->tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | tp_flags;
 type->tp_traverse = structseq_traverse;
 type->tp_members = tp_members;
}

Setting tp_base = &PyTuple_Type is what makes a struct sequence a tuple: it inherits tuple’s sequence protocol, hashing, comparison, __getitem__, iteration – and isinstance(x, tuple) is true. The type is also made immortal for builtins.

For dynamically created types (e.g. via PyStructSequence_NewType, used by extensions), _PyStructSequence_NewType builds a heap type from a PyType_Spec with PyType_FromSpecWithBases(..., (PyObject *)&PyTuple_Type).

The type dictionary: sizes and `__match_args__`

initialize_structseq_dict stores three integer entries used at runtime by the VISIBLE_SIZE_TP / REAL_SIZE_TP / UNNAMED_FIELDS_TP macros, plus __match_args__:

 c
 
 
// Objects/structseq.c
SET_DICT_FROM_SIZE(visible_length_key, desc->n_in_sequence); // "n_sequence_fields"
SET_DICT_FROM_SIZE(real_length_key, n_members); // "n_fields"
SET_DICT_FROM_SIZE(unnamed_fields_key, n_unnamed_members); // "n_unnamed_fields"

__match_args__ is built from the visible field names (skipping unnamed ones), resized with _PyTuple_Resize, and stored under "__match_args__". That’s what enables match obj: case sys.version_info(major=3, ...) style patterns and positional match args.

Construction from Python: `structseq_new`

__new__ accepts a sequence (and optional dict of field-name overrides):

 c
 
 
// Objects/structseq.c
static PyObject *
structseq_new_impl(PyTypeObject *type, PyObject *arg, PyObject *dict)

It validates that len(arg) is between min_len (visible) and max_len (real), copies the first len items, then fills the remaining slots from dict (by field name) or None.

This allows partial construction such as os.stat_result((mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime), {'st_atime_ns': ...}). After filling, it calls _PyObject_GC_TRACK.

GC: traverse and dealloc

Because hidden fields are in ob_item too, both must iterate over REAL_SIZE:

 c
 
 
// Objects/structseq.c
static int
structseq_traverse(PyObject *op, visitproc visit, void *arg)
{
 ...
 size = REAL_SIZE(obj);
 for (i = 0; i < size; ++i) {
 Py_VISIT(obj->ob_item[i]);
 }
 return 0;
}

static void
structseq_dealloc(PyObject *op)
{
 ...
 size = REAL_SIZE(obj); // cannot use type dict: GC may have cleared it
 for (i = 0; i < size; ++i) {
 Py_XDECREF(obj->ob_item[i]);
 }
 PyObject_GC_Del(obj);
 if (_PyType_HasFeature(tp, Py_TPFLAGS_HEAPTYPE)) {
 Py_DECREF(tp);
 }
}

Note the comment in structseq_dealloc: REAL_SIZE_TP() can’t be used during dealloc because the GC may have already cleared the type’s __dict__; instead REAL_SIZE(op) recomputes the hidden count from tp_basicsize, which survives.

repr, reduce, and `replace`

structseq_repr writes typename(field=value, ...) using only the visible fields:

 c
 
 
// Objects/structseq.c
for (Py_ssize_t i=0; i < VISIBLE_SIZE(obj); i++) {
 ...
 const char *name_utf8 = typ->tp_members[i].name;
 ...
 PyUnicodeWriter_WriteUTF8(writer, name_utf8, -1);
 ...
 PyObject *value = PyStructSequence_GetItem((PyObject*)obj, i);
 PyUnicodeWriter_WriteRepr(writer, value);
}

structseq_reduce returns (type, (visible_tuple, hidden_dict)), so pickling/reconstruction round-trips through __new__:

 c
 
 
// Objects/structseq.c
tup = PyTuple_FromArray(self->ob_item, n_visible_fields);
dict = PyDict_New();
for (i = n_visible_fields; i < n_fields; i++) {
 const char *n = Py_TYPE(self)->tp_members[i - n_unnamed_fields].name;
 PyDict_SetItemString(dict, n, self->ob_item[i]);
}
result = Py_BuildValue("(O(OO))", Py_TYPE(self), tup, dict);

__replace__ (PEP 73? functional update) creates a copy and applies keyword overrides; it’s disabled for types with unnamed fields:

 c
 
 
// Objects/structseq.c
if (n_unnamed_fields > 0) {
 PyErr_Format(PyExc_TypeError,
 "__replace__() is not supported for %.500s "
 "because it has unnamed field(s)", ...);
 return NULL;
}

Why it is “namedtuple-like”

From Python’s perspective a struct sequence is a tuple subclass instance:

isinstance(sys.version_info, tuple) is True.
len(), indexing, slicing, iteration, unpacking, comparison, and hashing all come from PyTuple_Type.
Elements are also attributes: sys.version_info.major, stat.st_mode.
repr is typename(field=value, ...).
It supports __reduce__ and __replace__.

The difference from collections.namedtuple is that struct sequences are defined in C via a descriptor, store hidden (non-sequence) fields, and are immutable tuple subclasses whose slots are raw PyObject * in one allocation. Named tuples created in Python are also tuple subclasses, but generated dynamically with _make, _asdict, etc., and have no hidden fields.

Example: `sys.version_info`

 c
 
 
// Python/sysmodule.c
static PyStructSequence_Field version_info_fields[] = {
 {"major", "Major release number"},
 {"minor", "Minor release number"},
 {"micro", "Patch release number"},
 {"releaselevel", "'alpha', 'beta', 'candidate', or 'final'"},
 {"serial", "Serial release number"},
 {0}
};

static PyStructSequence_Desc version_info_desc = {
 "sys.version_info",
 version_info__doc__,
 version_info_fields,
 5 /* n_in_sequence: all 5 are visible */
};

static PyObject *
make_version_info(PyThreadState *tstate)
{
 PyObject *version_info;
 version_info = PyStructSequence_New(&VersionInfoType);
 ...
 PyStructSequence_SET_ITEM(version_info, pos++, PyLong_FromLong(PY_MAJOR_VERSION));
 ...
}

Here n_in_sequence == 5 == n_members, so there are no hidden fields; the type is a plain named 5-tuple. The type is initialized at interpreter startup with ENSURE_INFO_TYPE(VersionInfoType, version_info_desc).

Key takeaways

PyStructSequence is just PyTupleObject; struct sequences are tuple subclasses (tp_base = &PyTuple_Type), gaining all tuple behavior.
Fields are stored in the same embedded ob_item array as a tuple.
n_in_sequence visible fields are exposed via ob_size; hidden fields are absorbed into tp_basicsize, so they live in ob_item beyond ob_size and are invisible to len/indexing.
Named fields become read-only PyMemberDef entries pointing into ob_item; unnamed fields (PyStructSequence_UnnamedField) have no attribute.
Types are described by PyStructSequence_Desc and initialized either as static builtins (_PyStructSequence_InitBuiltinWithFlags) or heap types (_PyStructSequence_NewType).
The type dict stores n_sequence_fields, n_fields, n_unnamed_fields, and __match_args__.
PyStructSequence_SetItem stores without incref (like tuple); creation is New + SET_ITEM + _PyObject_GC_TRACK.
GC traverse/dealloc use REAL_SIZE (not the visible size) to visit all slots.
repr, __reduce__, and __replace__ operate over visible (and hidden, for reduce) fields.

CPython's str Implementation in C (Unicode): One Byte per Character When You Can

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: A CPython str doesn’t store characters as 4-byte code points. PEP 393’s flexible representation picks the narrowest width that fits the string’s largest code point — 1, 2, or 4 bytes — so ASCII strings are single-byte and single-allocation.

Python’s str is a sequence of Unicode code points. In CPython that’s PyUnicodeObject in Objects/unicodeobject.c, using the “flexible string representation” from PEP 393: each string stores its characters in the smallest width that can hold its largest code point, and the struct itself changes shape depending on whether the string is ASCII and whether it’s “compact”. Let’s trace the representation and the hot paths through the 3.15 source.

The object structs

Every string is at least a PyASCIIObject. Non-ASCII compact strings add a UTF-8 cache header (PyCompactUnicodeObject), and subclasses add a data pointer (PyUnicodeObject):

 c
 
 
// Include/cpython/unicodeobject.h
typedef struct {
 PyObject_HEAD
 Py_ssize_t length; /* Number of code points */
 Py_hash_t hash; /* Hash value; -1 if not set */
 _Py_ALIGNED_DEF(4, struct _PyUnicodeObject_state) state;
} PyASCIIObject;

typedef struct {
 PyASCIIObject _base;
 Py_ssize_t utf8_length; /* bytes in utf8, excluding NUL */
 char *utf8; /* cached UTF-8 (null-terminated) */
} PyCompactUnicodeObject;

typedef struct {
 PyCompactUnicodeObject _base;
 union {
 void *any;
 Py_UCS1 *latin1;
 Py_UCS2 *ucs2;
 Py_UCS4 *ucs4;
 } data; /* Canonical buffer (legacy only) */
} PyUnicodeObject;

The state bitfield carries everything CPython needs to know about the string without inspecting the data:

 c
 
 
// Include/cpython/unicodeobject.h
struct _PyUnicodeObject_state {
 unsigned int interned:2; /* 0 not interned .. 3 static */
 unsigned int kind:3; /* 1, 2, or 4 bytes per char */
 unsigned int compact:1; /* one block vs. two */
 unsigned int ascii:1; /* all chars < 128 */
 unsigned int statically_allocated:1;
 ...
};

kind is one of PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND, PyUnicode_4BYTE_KIND. And interned distinguishes not-interned, mortal interned, immortal interned, and immortal+statically-allocated interned.

Flexible representations (PEP 393)

Instead of always storing 4 bytes per character (the old Py_UNICODE representation), CPython picks the narrowest width that fits the string’s maximum code point:

maxchar < 128 -> 1 byte, ASCII
maxchar < 256 -> 1 byte (latin1)
maxchar < 65536 -> 2 bytes (BMP)
otherwise -> 4 bytes (full range up to U+10FFFF)

That’s decided once, at allocation time, by PyUnicode_New:

 c
 
 
// Objects/unicodeobject.c
PyObject *
PyUnicode_New(Py_ssize_t size, Py_UCS4 maxchar)
{
 ...
 is_ascii = 0;
 struct_size = sizeof(PyCompactUnicodeObject);
 if (maxchar < 128) {
 kind = PyUnicode_1BYTE_KIND;
 char_size = 1;
 is_ascii = 1;
 struct_size = sizeof(PyASCIIObject);
 }
 else if (maxchar < 256) {
 kind = PyUnicode_1BYTE_KIND;
 char_size = 1;
 }
 else if (maxchar < 65536) {
 kind = PyUnicode_2BYTE_KIND;
 char_size = 2;
 }
 else {
 kind = PyUnicode_4BYTE_KIND;
 char_size = 4;
 }
 ...
 obj = (PyObject *) PyObject_Malloc(struct_size + (size + 1) * char_size);
 ...
 _PyUnicode_LENGTH(unicode) = size;
 _PyUnicode_HASH(unicode) = -1;
 _PyUnicode_STATE(unicode).interned = 0;
 _PyUnicode_STATE(unicode).kind = kind;
 _PyUnicode_STATE(unicode).compact = 1;
 _PyUnicode_STATE(unicode).ascii = is_ascii;
 ...
}

The +1 in the allocation leaves room for a trailing NUL, so the data can be handed to C APIs as a null-terminated buffer. PyUnicode_DATA(op) returns a pointer just past the struct, and PyUnicode_READ(kind, data, i) / PyUnicode_WRITE(kind, data, i, ch) index into it. Because kind lives in the object, code can switch on width without changing the API.

The real win: a Python program’s identifiers, log messages, and JSON payloads are overwhelmingly ASCII. Those strings cost one byte per character and one allocation.

Compact vs. legacy

A string is compact when the character buffer lives in the same allocation as the struct (compact == 1). PyUnicode_New always produces compact strings, and the data pointer is computed by walking past the struct:

 c
 
 
// Include/cpython/unicodeobject.h
static inline void* _PyUnicode_COMPACT_DATA(PyObject *op) {
 if (PyUnicode_IS_ASCII(op)) {
 return _Py_STATIC_CAST(void*, (_PyASCIIObject_CAST(op) + 1));
 }
 return _Py_STATIC_CAST(void*, (_PyCompactUnicodeObject_CAST(op) + 1));
}

Legacy strings (compact == 0) are used by subclasses of str. They keep a separate data.any pointer (PyUnicodeObject.data) allocated with PyMem_Malloc, because the subclass may have a different layout or size. unicode_dealloc frees that second block when present:

 c
 
 
// Objects/unicodeobject.c
if (_PyUnicode_HAS_UTF8_MEMORY(unicode)) {
 PyMem_Free(_PyUnicode_UTF8(unicode));
}
if (!PyUnicode_IS_COMPACT(unicode) && _PyUnicode_DATA_ANY(unicode)) {
 PyMem_Free(_PyUnicode_DATA_ANY(unicode));
}
Py_TYPE(unicode)->tp_free(unicode);

So compact strings are one PyObject_Malloc (and one free) for both header and data; legacy strings are two.

ASCII optimization

ASCII strings are the overwhelmingly common case, so CPython gives them a dedicated, smaller struct (PyASCIIObject instead of PyCompactUnicodeObject). No utf8/utf8_length fields are needed, because for ASCII the canonical 1-byte data is the UTF-8 representation. The ascii flag records this, and _PyUnicode_COMPACT_DATA uses the smaller offset.

Latin-1 characters (U+0080..U+00FF) still use 1 byte per character, but they aren’t ASCII, so they carry the UTF-8 cache fields.

There’s also a singleton for every Latin-1 code point, used as a fast path for single-character strings:

 c
 
 
// Include/internal/pycore_global_strings.h
#define _Py_LATIN1_CHR(CH) \
 ((CH) < 128 \
 ? (PyObject*)&_Py_SINGLETON(strings).ascii[(CH)] \
 : (PyObject*)&_Py_SINGLETON(strings).latin1[(CH) - 128])

intern_common (below) collapses any length-1 1-byte string to LATIN1(ch), so "a" and chr(97) are the same object.

UTF-8 caching

For non-ASCII strings, the UTF-8 representation is computed lazily and cached in utf8/utf8_length (inside PyCompactUnicodeObject). ASCII strings don’t need a cache because their 1-byte data is already valid UTF-8, so PyUnicode_AsUTF8 returns the data pointer directly.

_PyUnicode_HAS_UTF8_MEMORY tracks whether the cache owns a separate allocation that must be freed on dealloc (see above).

Hashing

The hash is cached in ob_hash (-1 means “not computed yet”). unicode_hash first checks the cache, then hashes the raw character buffer:

 c
 
 
// Objects/unicodeobject.c
static Py_hash_t
unicode_hash(PyObject *self)
{
 Py_uhash_t x;
 Py_hash_t hash = PyUnicode_HASH(self);
 if (hash != -1) {
 return hash;
 }
 x = Py_HashBuffer(PyUnicode_DATA(self),
 PyUnicode_GET_LENGTH(self) * PyUnicode_KIND(self));

 PyUnicode_SET_HASH(self, x);
 return x;
}

Because the buffer is contiguous bytes in canonical form, hashing is just hashing length * kind bytes — the same bytes as the UTF-8 for ASCII, which is why an ASCII str hashes the same as the equivalent bytes. The result is stored back, so subsequent lookups are O(1).

Interning

Interning deduplicates strings and is central to attribute/identifier lookup. CPython keeps two caches: a process-wide hashtable (INTERNED_STRINGS) for statically allocated strings, and a per-interpreter dict for runtime ones. intern_common is the shared implementation:

 c
 
 
// Objects/unicodeobject.c
static PyObject *
intern_common(PyInterpreterState *interp, PyObject *s, bool immortalize)
{
 ...
 /* Is it already interned? */
 switch (PyUnicode_CHECK_INTERNED(s)) { ... }

 /* if it's a short string, get the singleton */
 if (PyUnicode_GET_LENGTH(s) == 1 &&
 PyUnicode_KIND(s) == PyUnicode_1BYTE_KIND) {
 PyObject *r = LATIN1(*(unsigned char*)PyUnicode_DATA(s));
 Py_DECREF(s);
 return r;
 }

 /* Look in the global cache now. */
 {
 PyObject *r = (PyObject *)_Py_hashtable_get(INTERNED_STRINGS, s);
 if (r != NULL) {
 Py_DECREF(s);
 return Py_NewRef(r);
 }
 }

 /* Do a setdefault on the per-interpreter cache. */
 PyObject *interned = get_interned_dict(interp);
 ...
 int res = PyDict_SetDefaultRef(interned, s, s, &t);
 ...
 /* NOT_INTERNED -> INTERNED_MORTAL: the two dict references
 (key and value) are not counted in ob_refcnt. */
 if (!_Py_IsImmortal(s)) {
 Py_DECREF(s);
 Py_DECREF(s);
 }
 _PyUnicode_STATE(s).interned = SSTATE_INTERNED_MORTAL;
 if (immortalize) {
 immortalize_interned(s);
 }
 return s;
}

Two details to internalize: the interned dict holds two references (key and value) that are not counted in ob_refcnt, so unicode_dealloc restores the count before popping. And immortal interned strings are promoted via immortalize_interned, which sets the immortal flag and the SSTATE_INTERNED_IMMORTAL state. Empty strings come from the immortal _Py_STR(empty) singleton.

Building strings: PyUnicodeWriter

Because str is immutable, constructing one incrementally is awkward. CPython provides PyUnicodeWriter, which keeps a mutable buffer and grows it:

 c
 
 
// Include/cpython/unicodeobject.h
typedef struct PyUnicodeWriter PyUnicodeWriter;
PyAPI_FUNC(PyUnicodeWriter*) PyUnicodeWriter_Create(Py_ssize_t length);
PyAPI_FUNC(int) PyUnicodeWriter_WriteChar(PyUnicodeWriter *writer, Py_UCS4 ch);
PyAPI_FUNC(int) PyUnicodeWriter_WriteStr(PyUnicodeWriter *writer, PyObject *obj);
PyAPI_FUNC(PyObject*) PyUnicodeWriter_Finish(PyUnicodeWriter *writer);

Internally this is backed by a _PyUnicodeWriter struct holding buffer, data, kind, maxchar, size, pos, plus min_length, min_char and an overallocate flag.

_PyUnicodeWriter_Prepare grows the buffer only when needed, and upgrades kind (1 -> 2 -> 4 bytes) if a larger character is written — reallocating the whole buffer. Finish returns the resulting immutable str and clears the writer. This is the path used by str.join, formatting, and decoding.

Conversion to and from UTF-8

PyUnicode_FromString / PyUnicode_FromStringAndSize decode UTF-8 into the narrowest kind; PyUnicode_AsUTF8AndSize encodes (and caches) UTF-8.

For ASCII inputs the kind stays 1 byte and no transcoding beyond validation is needed. For higher code points the encoder chooses 2/4 bytes and the decoder validates UTF-8 sequences.

Surrogate pairs are not combined. Python strings are sequences of code points, so lone surrogates remain individual characters — useful for round-tripping Windows file names.

Operations

Reading is a typed, unchecked index into the canonical buffer:

 c
 
 
// Include/cpython/unicodeobject.h
static inline Py_UCS4 PyUnicode_READ_CHAR(PyObject *unicode, Py_ssize_t index)
{
 int kind = PyUnicode_KIND(unicode);
 if (kind == PyUnicode_1BYTE_KIND)
 return PyUnicode_1BYTE_DATA(unicode)[index];
 if (kind == PyUnicode_2BYTE_KIND)
 return PyUnicode_2BYTE_DATA(unicode)[index];
 return PyUnicode_4BYTE_DATA(unicode)[index];
}

Mutation is gated. _PyUnicode_IsModifiable checks that the object is compact, an exact str, not interned, and has refcount 1; otherwise PyUnicode_CopyCharacters allocates a fresh destination.

PyUnicode_CopyCharacters also performs kind conversion when the destination is wider than the source, and falls back to memcpy when both sides share the same kind. Most “operations” on strings (slicing, concatenation, case conversion, replace) therefore allocate a new PyUnicode_New and copy converted code points into it, choosing the smallest kind that fits the result’s maximum character.

That’s why str is immutable from Python’s point of view even though the C buffer is writable during construction.

Concatenation and repetition

PyUnicode_Concat handles the empty-string fast paths (_PyUnicode_GetEmpty), then allocates a result of len(a)+len(b) and copies both halves with PyUnicode_CopyCharacters, upgrading kind to fit the wider of the two inputs. PyUnicode_Repeat does the same with a loop, overallocating slightly and trimming. Neither ever mutates its inputs.

Comparison

unicode_compare_eq first compares length and (for ASCII) pointer equality, then does a memcmp-style byte comparison over the canonical buffers when kinds match; otherwise it falls back to code-point comparison.

Because the canonical representation is unique for a given string, two equal strings of the same kind compare in O(n) bytes with no per-character branching. Ordering (str.__lt__) compares code points lexicographically after the same kind normalization.

Identifiers and the global string table

CPython pre-creates a large table of interned strings (identifiers, literals, single characters) in Include/internal/pycore_global_strings.h, exposed via _Py_STR(name) and _Py_ID(name). These are statically allocated and marked SSTATE_INTERNED_IMMORTAL_STATIC, so they’re shared across interpreters and never freed.

Attribute and variable name lookups hit this table first (via _Py_hashtable_get(INTERNED_STRINGS, ...) in intern_common), which is why obj.attr and globals["name"] are fast: the name is already an interned singleton.

Memory layout

A compact ASCII string — single allocation, smallest case:

text

+-------------------------------+ <- PyObject_Malloc
| ob_refcnt |
+-------------------------------+
| *ob_type (&PyUnicode_Type) |
+-------------------------------+
| length |
+-------------------------------+
| hash (-1 until computed) |
+-------------------------------+
| state (interned/kind/compact/ |
| ascii/statically_alloc) |
+-------------------------------+
| data[0] (Py_UCS1, ASCII) |
| data[1] |
| ... |
| data[size] = '\0' |
+-------------------------------+

Compact non-ASCII adds utf8_length and utf8 between the header and data; legacy (subclass) strings instead store data.any pointing at a separately allocated buffer.

For an ASCII string there’s no second block and no separate UTF-8 cache — the data is the UTF-8.

Key takeaways

str is a PyObject whose layout is PyASCIIObject / PyCompactUnicodeObject / PyUnicodeObject depending on ASCII-ness and compactness.
PEP 393 stores characters in the narrowest of 1/2/4 bytes per code point, chosen at allocation from maxchar.
Compact strings put header and data in one PyObject_Malloc block; legacy (subclass) strings use a second block for data.
ASCII strings get the smallest struct and share data with their UTF-8 form; single-character Latin-1 strings are immortal singletons.
Hashing caches in ob_hash and hashes the raw canonical buffer; interning deduplicates via a global hashtable plus a per-interpreter dict, with immortal promotion.

CPython's slice Implementation in C: Three Objects and a Bunch of Clamping

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: slice is a tiny GC-tracked object holding three arbitrary Python objects — start, stop, step — with None standing for “omitted”. Turning those into real indices happens later, in PySlice_Unpack (objects → Py_ssize_t) and PySlice_AdjustIndices (clamping to the sequence length).

Python’s slice is the object produced by a[0:10:2] and by slice(0, 10, 2). In CPython it lives in Objects/sliceobject.c as a small GC-tracked struct holding three arbitrary Python objects, where None (Py_None) stands for “omitted”. The same file also owns the immortal Ellipsis singleton. Let’s trace the implementation through the 3.15 source.

The object: `PySliceObject`

The public (limited-API) layout is defined in Include/cpython/sliceobject.h:

 c
 
 
// Include/cpython/sliceobject.h
typedef struct {
 PyObject_HEAD
 PyObject *start, *stop, *step; /* not NULL */
} PySliceObject;

Three invariants follow from this:

The three fields are never NULL. An omitted bound is stored as Py_None, which is why slice(5) produces slice(None, 5, None).
They are arbitrary Python objects. They need not be integers; anything with __index__ is accepted later, when the slice is actually applied.
PySlice_Check(op) is an exact type check (Py_IS_TYPE((op), &PySlice_Type)), so a subclass of slice is not a slice for C-level fast paths.

PySlice_Type.tp_basicsize is sizeof(PySliceObject) and tp_itemsize is 0, so the object is fixed-size. It’s GC-tracked (Py_TPFLAGS_HAVE_GC) because it owns three references.

Construction and the freelist

slice_new (the tp_new) accepts 1 to 3 positional arguments and swaps start/stop to mirror range()’s calling convention:

 c
 
 
// Objects/sliceobject.c
static PyObject *
slice_new(PyTypeObject *type, PyObject *args, PyObject *kw)
{
 PyObject *start, *stop, *step;
 start = stop = step = NULL;
 if (!_PyArg_NoKeywords("slice", kw))
 return NULL;
 if (!PyArg_UnpackTuple(args, "slice", 1, 3, &start, &stop, &step))
 return NULL;
 /* This swapping of stop and start is to maintain similarity with range(). */
 if (stop == NULL) {
 stop = start;
 start = NULL;
 }
 return PySlice_New(start, stop, step);
}

PySlice_New normalizes NULLs to Py_None and delegates to _PyBuildSlice_Consume2, which pops from a tiny per-interpreter freelist before falling back to PyObject_GC_New:

 c
 
 
// Objects/sliceobject.c
static PySliceObject *
_PyBuildSlice_Consume2(PyObject *start, PyObject *stop, PyObject *step)
{
 assert(start != NULL && stop != NULL && step != NULL);
 PySliceObject *obj = _Py_FREELIST_POP(PySliceObject, slices);
 if (obj == NULL) {
 obj = PyObject_GC_New(PySliceObject, &PySlice_Type);
 if (obj == NULL) {
 goto error;
 }
 }
 obj->start = start;
 obj->stop = stop;
 obj->step = Py_NewRef(step);
 _PyObject_GC_TRACK(obj);
 return obj;
error:
 Py_DECREF(start);
 Py_DECREF(stop);
 return NULL;
}

Two things worth noting:

start and stop are consumed (their refcounts are taken over); step is re-referenced via Py_NewRef. That asymmetry is why the bytecode BUILD_SLICE calls PySlice_New(start_o, stop_o, step_o) with borrowed references, while _PyBuildSlice_ConsumeRefs (used by STORE_SLICE and _STORE_SLICE) steals start/stop to avoid extra refcount churn.
The freelist holds at most Py_slices_MAXFREELIST (1) entry, so it’s essentially a single-slot cache. slice_dealloc returns the object to it via _Py_FREELIST_FREE(slices, r, PyObject_GC_Del).

_PySlice_FromIndices is a convenience constructor used when C code already has Py_ssize_t bounds and wants a slice object:

 c
 
 
// Objects/sliceobject.c
PyObject *
_PySlice_FromIndices(Py_ssize_t istart, Py_ssize_t istop)
{
 PyObject *start, *end, *slice;
 start = PyLong_FromSsize_t(istart);
 ...
 slice = PySlice_New(start, end, NULL); /* step defaults to None -> 1 */
 ...
}

`BUILD_SLICE` in the bytecode

The compiler emits BUILD_SLICE for subscript expressions like a[i:j:k]:

 c
 
 
// Python/bytecodes.c
inst(BUILD_SLICE, (args[oparg] -- slice)) {
 PyObject *start_o = PyStackRef_AsPyObjectBorrow(args[0]);
 PyObject *stop_o = PyStackRef_AsPyObjectBorrow(args[1]);
 PyObject *step_o = oparg == 3 ? PyStackRef_AsPyObjectBorrow(args[2]) : NULL;
 PyObject *slice_o = PySlice_New(start_o, stop_o, step_o);
 DECREF_INPUTS();
 ERROR_IF(slice_o == NULL);
 slice = PyStackRef_FromPyObjectStealMortal(slice_o);
}

oparg is 2 or 3 depending on whether a step was given, and None is pushed for omitted bounds, so PySlice_New always receives non-NULL values.

Unpacking a slice: `PySlice_Unpack`

Before a slice can be applied, its three PyObject * fields must become Py_ssize_t values. PySlice_Unpack does that, treating None as a sentinel and clamping/boosting to PY_SSIZE_T_MIN/PY_SSIZE_T_MAX:

 c
 
 
// Objects/sliceobject.c
int
PySlice_Unpack(PyObject *_r,
 Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step)
{
 PySliceObject *r = (PySliceObject*)_r;
 if (r->step == Py_None) {
 *step = 1;
 } else {
 if (!_PyEval_SliceIndex(r->step, step)) return -1;
 if (*step == 0) {
 PyErr_SetString(PyExc_ValueError, "slice step cannot be zero");
 return -1;
 }
 /* Replace -PY_SSIZE_T_MAX-1 with -PY_SSIZE_T_MAX to guard against
 later undefined behaviour from "step = -step" reversal. */
 if (*step < -PY_SSIZE_T_MAX)
 *step = -PY_SSIZE_T_MAX;
 }
 if (r->start == Py_None) {
 *start = *step < 0 ? PY_SSIZE_T_MAX : 0;
 } else {
 if (!_PyEval_SliceIndex(r->start, start)) return -1;
 }
 if (r->stop == Py_None) {
 *stop = *step < 0 ? PY_SSIZE_T_MIN : PY_SSIZE_T_MAX;
 } else {
 if (!_PyEval_SliceIndex(r->stop, stop)) return -1;
 }
 return 0;
}

_PyEval_SliceIndex (in Python/ceval.c) is the bridge from “Python object” to “C integer”:

 c
 
 
// Python/ceval.c
int
_PyEval_SliceIndex(PyObject *v, Py_ssize_t *pi)
{
 if (Py_IsNone(v)) {
 return 1; /* leaves *pi untouched */
 }
 return _PyEval_SliceIndexNotNone(v, pi);
}

int
_PyEval_SliceIndexNotNone(PyObject *v, Py_ssize_t *pi)
{
 PyThreadState *tstate = _PyThreadState_GET();
 Py_ssize_t x;
 if (PyLong_CheckExact(v) && _PyLong_IsCompact((PyLongObject *)v)) {
 *pi = _PyLong_CompactValue((PyLongObject *)v);
 return 1;
 }
 if (_PyIndex_Check(v)) {
 x = PyNumber_AsSsize_t(v, NULL); /* calls __index__ */
 if (x == -1 && _PyErr_Occurred(tstate))
 return 0;
 } else {
 _PyErr_SetString(tstate, PyExc_TypeError,
 "slice indices must be integers or have an __index__ method");
 return 0;
 }
 *pi = x;
 return 1;
}

So an omitted bound becomes the extreme sentinel (PY_SSIZE_T_MAX / PY_SSIZE_T_MIN), __index__ is honored for non-None values, and huge values are silently clamped to the Py_ssize_t range. None-handling for step defaults to 1.

The older PySlice_GetIndices (deprecated in 3.7) is a simpler variant that requires PyLong bounds and returns -1 on out-of-range. New code should use PySlice_Unpack + PySlice_AdjustIndices.

Adjusting indices: `PySlice_AdjustIndices`

After unpacking, the raw start/stop (which may be negative, or the extreme sentinels) must be clamped to [0, length] — or the reverse range for a negative step. PySlice_AdjustIndices does this and returns the slice length:

 c
 
 
// Objects/sliceobject.c
Py_ssize_t
PySlice_AdjustIndices(Py_ssize_t length,
 Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t step)
{
 assert(step != 0);
 assert(step >= -PY_SSIZE_T_MAX);

 if (*start < 0) {
 *start += length;
 if (*start < 0) {
 *start = (step < 0) ? -1 : 0;
 }
 } else if (*start >= length) {
 *start = (step < 0) ? length - 1 : length;
 }

 if (*stop < 0) {
 *stop += length;
 if (*stop < 0) {
 *stop = (step < 0) ? -1 : 0;
 }
 } else if (*stop >= length) {
 *stop = (step < 0) ? length - 1 : length;
 }

 if (step < 0) {
 if (*stop < *start) {
 return (*start - *stop - 1) / (-step) + 1;
 }
 } else {
 if (*start < *stop) {
 return (*stop - *start - 1) / step + 1;
 }
 }
 return 0;
}

The logic, in three steps:

Negative start/stop are taken relative to length (i += length), then clamped to the appropriate boundary (0/length for forward steps, -1/length-1 for backward steps).
Out-of-range positive values are clamped to the same boundaries.
The returned count is the number of elements the slice covers, computed with integer division so a partial final step isn’t counted.

PySlice_GetIndicesEx is just the two steps glued together:

 c
 
 
// Objects/sliceobject.c
int
PySlice_GetIndicesEx(PyObject *_r, Py_ssize_t length,
 Py_ssize_t *start, Py_ssize_t *stop, Py_ssize_t *step,
 Py_ssize_t *slicelength)
{
 if (PySlice_Unpack(_r, start, stop, step) < 0)
 return -1;
 *slicelength = PySlice_AdjustIndices(length, start, stop, *step);
 return 0;
}

`slice.indices(len)`: the Python-facing version

slice.indices(length) returns a tuple of fully-adjusted (start, stop, step) integers. It’s implemented by _PySlice_GetLongIndices, which works in Python int objects (not Py_ssize_t) so huge values never overflow and out-of-bounds indices are clipped consistently:

 c
 
 
// Objects/sliceobject.c
int
_PySlice_GetLongIndices(PySliceObject *self, PyObject *length,
 PyObject **start_ptr, PyObject **stop_ptr,
 PyObject **step_ptr)
{
 PyObject *start=NULL, *stop=NULL, *step=NULL;
 PyObject *upper=NULL, *lower=NULL;
 int step_is_negative, cmp_result;

 /* Convert step to an integer; raise for zero step. */
 if (self->step == Py_None) {
 step = _PyLong_GetOne();
 step_is_negative = 0;
 } else {
 step = evaluate_slice_index(self->step); /* calls __index__ */
 ...
 step_is_negative = step_sign < 0;
 }

 /* Bounds depend on step direction. */
 if (step_is_negative) {
 lower = PyLong_FromLong(-1L);
 upper = PyNumber_Add(length, lower); /* length - 1 */
 } else {
 lower = _PyLong_GetZero(); /* 0 */
 upper = Py_NewRef(length); /* length */
 }

 /* Compute start: None -> upper/lower; else clamp to [lower, upper]. */
 if (self->start == Py_None) {
 start = Py_NewRef(step_is_negative ? upper : lower);
 } else {
 start = evaluate_slice_index(self->start);
 if (_PyLong_IsNegative((PyLongObject *)start)) {
 PyObject *tmp = PyNumber_Add(start, length); /* start += length */
 Py_SETREF(start, tmp);
 cmp_result = PyObject_RichCompareBool(start, lower, Py_LT);
 if (cmp_result) Py_SETREF(start, Py_NewRef(lower));
 } else {
 cmp_result = PyObject_RichCompareBool(start, upper, Py_GT);
 if (cmp_result) Py_SETREF(start, Py_NewRef(upper));
 }
 }
 /* ... analogous block for stop ... */

 *start_ptr = start; *stop_ptr = stop; *step_ptr = step;
 ...
}

The returned start/stop/step are new references that the caller owns; slice.indices packs them into a tuple via Py_BuildValue("(NNN)", ...).

Worth internalizing: the Py_ssize_t path and the Python-int path implement the same semantics with different trade-offs. The C path is fast but clamps; the Python path is exact and overflow-free.

Equality, hashing, and repr

slice_repr produces slice(%R, %R, %R) from the three fields.
slice_richcompare short-circuits identity (v == w) to True for ==/<=/>= and False otherwise; otherwise it packs both triples into tuples and delegates to PyObject_RichCompare.
slice_hash mixes the three field hashes with the same xxhash-style mixer used for tuples.
slice_traverse visits all three fields for the GC.

`Ellipsis`

The same file defines the immortal Ellipsis singleton used for ...:

 c
 
 
// Objects/sliceobject.c
PyObject _Py_EllipsisObject = _PyObject_HEAD_INIT(&PyEllipsis_Type);

ellipsis_new rejects any arguments and returns Py_Ellipsis, and ellipsis_dealloc re-immortalizes the object as a guard. Py_Ellipsis resolves to this struct (or to the immortal constant in the limited API).

Memory layout summary

text

PySliceObject
+---------------------------+
| ob_refcnt / ob_tid |
+---------------------------+
| *ob_type (&PySlice_Type) |
+---------------------------+
| *start (never NULL) | -> Py_None or any PyObject
+---------------------------+
| *stop (never NULL) |
+---------------------------+
| *step (never NULL) |
+---------------------------+

Fixed size (sizeof(PySliceObject)), GC-tracked, with a 1-slot freelist. The three fields are the only payload, which is why a slice is cheap to create and why slice equality reduces to tuple equality of (start, stop, step).

Key takeaways

PySliceObject holds three non-NULL PyObject * fields (start, stop, step); omitted bounds are Py_None.
Construction goes through _PyBuildSlice_Consume2, which reuses a 1-slot freelist and tracks the object with the GC.
PySlice_Unpack converts the three fields to Py_ssize_t, treating None as extreme sentinels and honoring __index__ via _PyEval_SliceIndex.
PySlice_AdjustIndices clamps negative/out-of-range bounds and returns the number of elements covered.
_PySlice_GetLongIndices is the Python-int (overflow-free) variant used by slice.indices() and range slicing.
The Ellipsis singleton lives in the same translation unit.

CPython's SimpleNamespace Implementation in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s types.SimpleNamespace looks like a tiny “bag of attributes” class, but in CPython 3.15 it is a real C type, _PyNamespace_Type, defined in Objects/namespaceobject.c. There is no separate attribute storage: every attribute lives in an ordinary dict that is also the object’s __dict__. This article traces that implementation.

The struct

The whole object is a PyObject_HEAD plus one pointer:

 c
 
 
// Objects/namespaceobject.c
typedef struct {
 PyObject_HEAD
 PyObject *ns_dict;
} _PyNamespaceObject;

That single ns_dict is the entire state. __dict__ is not a separate property layered on top; it is ns_dict. This is exposed twice in the type table: once as a read-only member and once as tp_dictoffset:

 c
 
 
// Objects/namespaceobject.c
static PyMemberDef namespace_members[] = {
 {"__dict__", _Py_T_OBJECT, offsetof(_PyNamespaceObject, ns_dict), Py_READONLY},
 {NULL}
};

 c
 
 
// Objects/namespaceobject.c (in _PyNamespace_Type)
 offsetof(_PyNamespaceObject, ns_dict), /* tp_dictoffset */

Because tp_dictoffset points at ns_dict, CPython’s generic machinery treats that slot as the instance dictionary. __dict__ access through the member definition and through tp_dictoffset therefore return the same object, which keeps obj.__dict__, vars(obj), attribute access, and pickling all consistent.

The type table

_PyNamespace_Type is the C object behind types.SimpleNamespace. Its important slots are:

 c
 
 
// Objects/namespaceobject.c
PyTypeObject _PyNamespace_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "types.SimpleNamespace", /* tp_name */
 sizeof(_PyNamespaceObject), /* tp_basicsize */
 0, /* tp_itemsize */
 namespace_dealloc, /* tp_dealloc */
 ...
 namespace_repr, /* tp_repr */
 ...
 PyObject_GenericGetAttr, /* tp_getattro */
 PyObject_GenericSetAttr, /* tp_setattro */
 ...
 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC |
 Py_TPFLAGS_BASETYPE, /* tp_flags */
 ...
 namespace_traverse, /* tp_traverse */
 namespace_clear, /* tp_clear */
 namespace_richcompare, /* tp_richcompare */
 ...
 namespace_methods, /* tp_methods */
 namespace_members, /* tp_members */
 ...
 offsetof(_PyNamespaceObject, ns_dict), /* tp_dictoffset */
 namespace_init, /* tp_init */
 PyType_GenericAlloc, /* tp_alloc */
 namespace_new, /* tp_new */
 PyObject_GC_Del, /* tp_free */
};

Three choices matter. Py_TPFLAGS_BASETYPE is set, so SimpleNamespace can be subclassed (the doc example class AdvancedNamespace(types.SimpleNamespace) works). Py_TPFLAGS_HAVE_GC is set because the instance holds references to arbitrary objects in its dict. Attribute access is delegated to PyObject_GenericGetAttr and PyObject_GenericSetAttr, which look things up in __dict__; there is no custom tp_getattr/tp_setattr, so the dict is the single source of truth.

The type is exported to Python from Modules/_typesmodule.c:

 c
 
 
// Modules/_typesmodule.c
EXPORT_STATIC_TYPE("SimpleNamespace", _PyNamespace_Type);

and, in a build without the C extension, Lib/types.py falls back to SimpleNamespace = type(sys.implementation), since sys.implementation is itself a SimpleNamespace.

Allocation and initialization

namespace_new allocates the object and immediately creates an empty dict:

 c
 
 
// Objects/namespaceobject.c
static PyObject *
namespace_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
 PyObject *self;

 assert(type != NULL && type->tp_alloc != NULL);
 self = type->tp_alloc(type, 0);
 if (self != NULL) {
 _PyNamespaceObject *ns = (_PyNamespaceObject *)self;
 ns->ns_dict = PyDict_New();
 if (ns->ns_dict == NULL) {
 Py_DECREF(ns);
 return NULL;
 }
 }
 return self;
}

namespace_init implements the documented constructor signature SimpleNamespace(mapping_or_iterable=(), /, **kwargs):

 c
 
 
// Objects/namespaceobject.c
static int
namespace_init(PyObject *op, PyObject *args, PyObject *kwds)
{
 _PyNamespaceObject *ns = _PyNamespace_CAST(op);
 PyObject *arg = NULL;
 if (!PyArg_UnpackTuple(args, _PyType_Name(Py_TYPE(ns)), 0, 1, &arg)) {
 return -1;
 }
 if (arg != NULL) {
 PyObject *dict;
 if (PyDict_CheckExact(arg)) {
 dict = Py_NewRef(arg);
 }
 else {
 dict = PyObject_CallOneArg((PyObject *)&PyDict_Type, arg);
 if (dict == NULL) {
 return -1;
 }
 }
 int err = (!PyArg_ValidateKeywordArguments(dict) ||
 PyDict_Update(ns->ns_dict, dict) < 0);
 Py_DECREF(dict);
 if (err) {
 return -1;
 }
 }
 if (kwds == NULL) {
 return 0;
 }
 if (!PyArg_ValidateKeywordArguments(kwds)) {
 return -1;
 }
 return PyDict_Update(ns->ns_dict, kwds);
}

A positional argument is either used directly (if it is exactly a dict) or converted with dict(arg); every key must be a string, enforced by PyArg_ValidateKeywordArguments. Then PyDict_Update merges it into ns_dict, and keyword arguments are merged the same way. Because the backing store is a normal dict, insertion order is preserved and drives the repr order.

Repr

The repr walks the dict’s keys in insertion order and formats each as key=value!r:

 c
 
 
// Objects/namespaceobject.c
static PyObject *
namespace_repr(PyObject *ns)
{
 ...
 keys = PyDict_Keys(d);
 ...
 while ((key = PyIter_Next(keys_iter)) != NULL) {
 if (PyUnicode_Check(key) && PyUnicode_GET_LENGTH(key) > 0) {
 PyObject *value, *item;

 int has_key = PyDict_GetItemRef(d, key, &value);
 if (has_key == 1) {
 item = PyUnicode_FromFormat("%U=%R", key, value);
 Py_DECREF(value);
 ...
 }
 else if (has_key < 0) {
 loop_error = 1;
 }
 }
 ...
 }
 ...
 repr = PyUnicode_FromFormat("%s(%S)", name, pairsrepr);
}

The name is "namespace" for an exact _PyNamespace_Type instance and Py_TYPE(ns)->tp_name for subclasses, so a subclass repr uses its own class name. PyDict_GetItemRef (rather than the older PyDict_GetItemWithError) is used so the borrowed reference is safely converted to an owned one before formatting, which is what fixed a crash in 3.15 when the dict mutated during repr.

Equality

Equality compares the backing dicts directly, but only for ==/!= and only when both operands are namespace objects:

 c
 
 
// Objects/namespaceobject.c
static PyObject *
namespace_richcompare(PyObject *self, PyObject *other, int op)
{
 if (
 (op == Py_EQ || op == Py_NE) &&
 PyObject_TypeCheck(self, &_PyNamespace_Type) &&
 PyObject_TypeCheck(other, &_PyNamespace_Type)
 ) {
 return PyObject_RichCompare(((_PyNamespaceObject *)self)->ns_dict,
 ((_PyNamespaceObject *)other)->ns_dict, op);
 }
 Py_RETURN_NOTIMPLEMENTED;
}

For any other comparison operator (or non-namespace operand) it returns NotImplemented, letting Python fall back to identity comparison or raise TypeError. The op == Py_EQ || op == Py_NE guard avoids passing an ordering operator to dict.__eq__.

Pickling and copying

__reduce__ returns a three-tuple (type, (), ns_dict), which makes unpickling call type() and then update its dict from the third element:

 c
 
 
// Objects/namespaceobject.c
static PyObject *
namespace_reduce(PyObject *op, PyObject *Py_UNUSED(ignored))
{
 _PyNamespaceObject *ns = (_PyNamespaceObject*)op;
 PyObject *result, *args = PyTuple_New(0);

 if (!args)
 return NULL;

 result = PyTuple_Pack(3, (PyObject *)Py_TYPE(ns), args, ns->ns_dict);
 Py_DECREF(args);
 return result;
}

__replace__, used by copy.replace, allocates a new instance of the same type, copies the old dict in, then applies the keyword overrides:

 c
 
 
// Objects/namespaceobject.c
static PyObject *
namespace_replace(PyObject *self, PyObject *args, PyObject *kwargs)
{
 if (!_PyArg_NoPositional("__replace__", args)) {
 return NULL;
 }

 PyObject *result = PyObject_CallNoArgs((PyObject *)Py_TYPE(self));
 if (!result) {
 return NULL;
 }
 if (!_PyNamespace_Check(result)) {
 PyErr_Format(PyExc_TypeError,
 "expect %N type, but %T() returned '%T' object",
 &_PyNamespace_Type, self, result);
 Py_DECREF(result);
 return NULL;
 }

 if (PyDict_Update(((_PyNamespaceObject*)result)->ns_dict,
 ((_PyNamespaceObject*)self)->ns_dict) < 0)
 {
 Py_DECREF(result);
 return NULL;
 }
 if (kwargs) {
 if (PyDict_Update(((_PyNamespaceObject*)result)->ns_dict, kwargs) < 0) {
 Py_DECREF(result);
 return NULL;
 }
 }
 return result;
}

The _PyNamespace_Check(result) guard matters because Py_TYPE(self) could be a subclass whose __new__/__init__ returns something that is not a namespace; without it the subsequent cast of result would be undefined behavior. This guard was added in 3.15.

GC, clear, and dealloc

Because ns_dict can reference arbitrary objects (including cycles), the type participates in the cyclic GC:

 c
 
 
// Objects/namespaceobject.c
static int
namespace_traverse(PyObject *op, visitproc visit, void *arg)
{
 _PyNamespaceObject *ns = _PyNamespace_CAST(op);
 Py_VISIT(ns->ns_dict);
 return 0;
}

static int
namespace_clear(PyObject *op)
{
 _PyNamespaceObject *ns = _PyNamespace_CAST(op);
 Py_CLEAR(ns->ns_dict);
 return 0;
}

static void
namespace_dealloc(PyObject *op)
{
 _PyNamespaceObject *ns = _PyNamespace_CAST(op);
 PyObject_GC_UnTrack(ns);
 Py_CLEAR(ns->ns_dict);
 Py_TYPE(ns)->tp_free((PyObject *)ns);
}

tp_free is PyObject_GC_Del, matching tp_alloc = PyType_GenericAlloc for a GC-tracked type.

The internal helper

CPython itself creates SimpleNamespaces in many places (module specs, sys.implementation, interpreter configs, instrumentation events, etc.) through a small helper that skips __init__:

 c
 
 
// Objects/namespaceobject.c
PyObject *
_PyNamespace_New(PyObject *kwds)
{
 PyObject *ns = namespace_new(&_PyNamespace_Type, NULL, NULL);
 if (ns == NULL)
 return NULL;

 if (kwds == NULL)
 return ns;
 if (PyDict_Update(((_PyNamespaceObject *)ns)->ns_dict, kwds) != 0) {
 Py_DECREF(ns);
 return NULL;
 }

 return (PyObject *)ns;
}

It calls namespace_new directly (allocating and creating the empty dict) and then merges kwds. Passing NULL yields an empty namespace. This is the function exported by Include/internal/pycore_namespace.h and used by import.c, sysmodule.c, timemodule.c, _interpretersmodule.c, and others.

Summary

types.SimpleNamespace is deliberately minimal: a PyObject_HEAD plus a dict. Attribute access, __dict__, GC, and instance dictionary handling are all inherited from CPython’s generic object machinery, while the type supplies allocation, initialization from a mapping/iterable plus kwargs, repr, equality, pickling, and __replace__. Every attribute is just a key in ns_dict, which is why SimpleNamespace behaves like a mutable, attribute-accessible dict.

CPython's set and frozenset Implementation in C: One Hash Table, Two Flavors

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: set and frozenset share one implementation: PySetObject in Objects/setobject.c, an open-addressing hash table keyed only by element. Deletion doesn’t clear a slot — it turns it into a dummy, so other keys’ probe chains stay intact.

Python’s set and frozenset share a single implementation: an open-addressing hash table keyed only by element. In CPython that’s PySetObject in Objects/setobject.c. Let’s trace the representation and the hot paths through the 3.15 source.

The object struct

Both types use the same struct (there’s no ob_size field; the logical size is used):

 c
 
 
// Include/cpython/setobject.h
typedef struct {
 PyObject *key;
 Py_hash_t hash; /* Cached hash code of the key */
} setentry;

typedef struct {
 PyObject_HEAD

 Py_ssize_t fill; /* Number active and dummy entries */
 Py_ssize_t used; /* Number active entries */
 Py_ssize_t mask; /* table size - 1 (power of two) */
 setentry *table; /* never NULL */
 Py_hash_t hash; /* Only used by frozenset objects */
 Py_ssize_t finger; /* Search finger for pop() */
 setentry smalltable[PySet_MINSIZE]; /* PySet_MINSIZE == 8 */
 PyObject *weakreflist;
} PySetObject;

fill counts active plus deleted (“dummy”) slots; used is len(set). mask is size - 1, so the table has mask + 1 slots and is always a power of two.

And table is never NULL: small sets live in the inline smalltable, large sets in a PyMem_NEW’d array. That kills a null test on every single lookup.

Three entry states

Every slot is in one of three states, distinguished by key and hash:

 c
 
 
// Include/cpython/setobject.h
/* 1. Unused: key == NULL and hash == 0
 2. Dummy: key == dummy and hash == -1
 3. Active: key != NULL and key != dummy and hash != -1 */

dummy is a single process-wide sentinel object (&_dummy_struct), so a deleted slot can be told apart from an unused one without consuming a real key.

Unused slots terminate searches. Dummies are “holes” that an insertion may reuse but that a search must keep walking past.

Probing: linear then randomized

Lookups use open addressing with a hybrid probe sequence (Knuth Algorithm D, as in dictobject.c): LINEAR_PROBES (9) consecutive slots first for cache locality, then a pseudo-random jump:

 c
 
 
// Objects/setobject.c
#define LINEAR_PROBES 9
#define PERTURB_SHIFT 5

static int set_do_lookup(PySetObject *so, setentry *table, size_t mask,
 PyObject *key, Py_hash_t hash, setentry **epp,
 compare_func compare_entry) {
 setentry *entry;
 size_t perturb = hash;
 size_t i = (size_t)hash & mask;
 int probes;
 ...
 while (1) {
 entry = &table[i];
 probes = (i + LINEAR_PROBES <= mask) ? LINEAR_PROBES : 0;
 do {
 status = compare_entry(so, table, entry, key, hash);
 if (status != SET_LOOKKEY_NO_MATCH) {
 if (status == SET_LOOKKEY_EMPTY) /* unused slot */
 return SET_LOOKKEY_NO_MATCH; /* not found */
 *epp = entry;
 return status; /* found / error */
 }
 entry++;
 } while (probes--);
 perturb >>= PERTURB_SHIFT;
 i = (i * 5 + 1 + perturb) & mask;
 }
}

The initial index is hash & mask. Consecutive probes walk memory linearly — cheap, and cache-friendly — then the congruential step scatters to break collision chains. Unused slots (key == NULL && hash == 0) stop a failing search; dummies do not.

Adding an element

set_add_entry_takeref walks the probe sequence looking for either an active match (return without taking ownership again) or a slot to fill:

 c
 
 
// Objects/setobject.c
static int set_add_entry_takeref(PySetObject *so, PyObject *key,
 Py_hash_t hash) {
 ...
 while (1) {
 entry = &so->table[i];
 probes = (i + LINEAR_PROBES <= mask) ? LINEAR_PROBES : 0;
 do {
 if (entry->hash == 0 && entry->key == NULL)
 goto found_unused_or_dummy; /* unused */
 if (entry->hash == hash) {
 ... /* compare; on match goto found_active */
 }
 else if (entry->hash == -1) { /* dummy slot */
 assert(entry->key == dummy);
 freeslot = entry; /* remember first dummy */
 }
 entry++;
 } while (probes--);
 perturb >>= PERTURB_SHIFT;
 i = (i * 5 + 1 + perturb) & mask;
 }

found_unused_or_dummy:
 if (freeslot == NULL)
 goto found_unused;
 /* reuse the dummy slot */
 FT_ATOMIC_STORE_SSIZE_RELAXED(so->used, so->used + 1);
 FT_ATOMIC_STORE_SSIZE_RELAXED(freeslot->hash, hash);
 FT_ATOMIC_STORE_PTR_RELEASE(freeslot->key, key);
 return 0;

found_unused:
 so->fill++;
 FT_ATOMIC_STORE_SSIZE_RELAXED(so->used, so->used + 1);
 FT_ATOMIC_STORE_SSIZE_RELAXED(entry->hash, hash);
 FT_ATOMIC_STORE_PTR_RELEASE(entry->key, key);
 if ((size_t)so->fill * 5 < mask * 3)
 return 0;
 return set_table_resize(so, so->used > 50000 ? so->used * 2
 : so->used * 4);
}

Two details matter here. First, a dummy slot encountered during the walk is remembered as freeslot and reused in preference to extending fill — that’s how deletions get recycled.

Second, the load-factor check fill*5 < mask*3 is roughly a 60% load factor (fill < 0.6 * size). Exceed it and the table grows: 4x for small sets (used <= 50000), 2x for large ones. That growth factor is an amortization choice.

Deleting: turning active into dummy

Discard doesn’t clear the slot. It converts it to a dummy so that probe chains for other keys remain intact:

 c
 
 
// Objects/setobject.c
static int set_discard_entry(PySetObject *so, PyObject *key, Py_hash_t hash) {
 setentry *entry;
 PyObject *old_key;
 int status = set_lookkey(so, key, hash, &entry);
 if (status < 0) return -1;
 if (status == SET_LOOKKEY_NO_MATCH) return DISCARD_NOTFOUND;
 old_key = entry->key;
 FT_ATOMIC_STORE_SSIZE_RELAXED(entry->hash, -1);
 FT_ATOMIC_STORE_SSIZE_RELAXED(so->used, so->used - 1);
 FT_ATOMIC_STORE_PTR_RELEASE(entry->key, dummy);
 Py_DECREF(old_key);
 return DISCARD_FOUND;
}

fill stays unchanged (the slot is still occupied, now by a dummy), so only used drops.

Here’s the thing: over many deletions fill can creep up to size. At that point even failed searches need a resize to recreate a virgin slot. Delete-heavy workloads therefore resize more often than you’d expect.

Resizing and rehashing

set_table_resize allocates a new table and reinserts every active key via set_insert_clean (which assumes absence and never calls back into Python):

 c
 
 
// Objects/setobject.c
static int set_table_resize(PySetObject *so, Py_ssize_t minused) {
 ...
 size_t newsize = PySet_MINSIZE;
 while (newsize <= (size_t)minused) newsize <<= 1;

 oldtable = so->table;
 is_oldtable_malloced = oldtable != so->smalltable;

 if (newsize == PySet_MINSIZE) {
 newtable = so->smalltable;
 if (newtable == oldtable) {
 if (so->fill == so->used) {
 /* No dummies: nothing to do. */
 return 0;
 }
 /* rebuild in place to purge dummies */
 memcpy(small_copy, oldtable, sizeof(small_copy));
 oldtable = small_copy;
 }
 } else {
 newtable = PyMem_NEW(setentry, newsize);
 ...
 }

 set_zero_table(newtable, newsize);
 FT_ATOMIC_STORE_PTR_RELEASE(so->table, NULL);
 FT_ATOMIC_STORE_SSIZE_RELEASE(so->mask, newsize - 1);

 newmask = (size_t)so->mask;
 if (so->fill == so->used) {
 for (entry = oldtable; entry <= oldtable + oldmask; entry++)
 if (entry->key != NULL)
 set_insert_clean(newtable, newmask, entry->key, entry->hash);
 } else {
 so->fill = so->used;
 for (entry = oldtable; entry <= oldtable + oldmask; entry++)
 if (entry->key != NULL && entry->key != dummy)
 set_insert_clean(newtable, newmask, entry->key, entry->hash);
 }

 FT_ATOMIC_STORE_PTR_RELEASE(so->table, newtable);
 if (is_oldtable_malloced)
 free_entries(oldtable, oldsize, SET_IS_SHARED(so));
 return 0;
}

Three things to note. The new size is the smallest power of two strictly greater than minused. Shrinking to PySet_MINSIZE with no dummies is a no-op. And dummies are dropped during reinsertion, so fill becomes used again.

The old array is freed with QSBR-delayed free on free-threaded builds when the set is shared, so concurrent readers don’t use-after-free.

Clearing empties the set before decref’ing keys, to survive re-entrancy from __del__:

 c
 
 
// Objects/setobject.c
static int set_clear_internal(PyObject *self) {
 ...
 if (table_is_malloced)
 set_empty_to_minsize(so); /* swap to smalltable first */
 else if (fill > 0) {
 memcpy(small_copy, table, sizeof(small_copy));
 table = small_copy;
 set_empty_to_minsize(so);
 }
 for (entry = table; used > 0; entry++) {
 if (entry->key && entry->key != dummy) {
 used--;
 Py_DECREF(entry->key);
 }
 }
 if (table_is_malloced)
 free_entries(table, oldsize, SET_IS_SHARED(so));
 return 0;
}

Construction

New sets start with the inline small table and no allocation:

 c
 
 
// Objects/setobject.c
static PyObject *make_new_set_untracked(PyTypeObject *type, PyObject *iterable) {
 PySetObject *so = (PySetObject *)_PyType_AllocNoTrack(type, 0);
 ...
 so->fill = 0;
 so->used = 0;
 so->mask = PySet_MINSIZE - 1; /* 7 */
 so->table = so->smalltable;
 so->hash = -1;
 so->finger = 0;
 so->weakreflist = NULL;
 if (iterable != NULL)
 set_update_local(so, iterable); /* fills via set_add_entry */
 return (PyObject *)so;
}

PySet_New/PyFrozenSet_New wrap this with GC tracking. And frozenset(f) on an exact frozenset is idempotent — it returns the same object.

frozenset: immutability and cached hash

A frozenset uses the identical table. Immutability is enforced at the API level (PySet_Type has mutable methods, PyFrozenSet_Type does not), with one deliberate exception: PySet_Add may mutate a frozenset that is uniquely referenced, so you can fill freshly-built frozensets before they escape:

 c
 
 
// Objects/setobject.c
int PySet_Add(PyObject *anyset, PyObject *key) {
 if (PySet_Check(anyset)) { ... }
 if (PyFrozenSet_Check(anyset) &&
 _PyObject_IsUniquelyReferenced(anyset)) {
 if (PyFrozenSet_CheckExact(anyset) &&
 !PyObject_GC_IsTracked(anyset) &&
 PyObject_GC_IsTracked(key))
 _PyObject_GC_TRACK(anyset);
 return set_add_key((PySetObject *)anyset, key);
 }
 PyErr_BadInternalCall();
 return -1;
}

PySet_Type.tp_hash is PyObject_HashNotImplemented (sets are unhashable), while PyFrozenSet_Type.tp_hash is frozenset_hash, which caches its result in so->hash (initialized to -1):

 c
 
 
// Objects/setobject.c
static Py_hash_t frozenset_hash(PyObject *self) {
 PySetObject *so = _PySet_CAST(self);
 if (FT_ATOMIC_LOAD_SSIZE_RELAXED(so->hash) != -1)
 return FT_ATOMIC_LOAD_SSIZE_ACQUIRE(so->hash);
 Py_uhash_t hash = frozenset_hash_impl(self);
 FT_ATOMIC_STORE_SSIZE_RELEASE(so->hash, hash);
 return hash;
}

The hash is order-independent: it XORs a shuffled version of every slot’s cached hash (including null and dummy slots), then corrects for their counts, mixes in the active count, and finalizes.

Because frozensets are immutable, the cached value never goes stale. That’s the whole reason caching is safe here and nowhere else in the set code.

Frozensets also try to drop GC tracking when none of their elements are GC-tracked (_PyFrozenSet_MaybeUntrack), saving traversal cost.

pop() and the finger

set.pop() removes an arbitrary element. To avoid always scanning from slot 0, it keeps a finger index and searches forward, wrapping around:

 c
 
 
// Objects/setobject.c (abridged)
PyObject *PySet_Pop(PyObject *set) {
 ...
 /* find next active entry starting at finger */
 while (entry->key == NULL || entry->key == dummy) { ... advance ... }
 /* convert to dummy, decrement used, return key */
}

(Exact implementation advances so->finger and wraps at mask.)

pop/iteration summary

Iteration (set_iter) walks the table slots in order, skipping null and dummy entries. The iterator holds a reference to the set and checks for mutation (SET_LOOKKEY_CHANGED).

Key takeaways

set and frozenset share PySetObject: fill/used/mask, a table pointer, an inline 8-slot smalltable, and (for frozensets) a cached hash.
Storage is an open-addressing hash table with three slot states; deletes become dummies to preserve probe chains.
Probing does 9 linear probes then a pseudo-random jump; load factor ~60%; growth is 4x then 2x.
Resizing reinserts active keys and drops dummies; clearing empties the set first.
Frozensets are immutable by API, but PySet_Add may fill a uniquely referenced one; their hash is computed once and cached.

CPython's Sentinel Implementation in C: Unique Marker Objects for Defaults

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: Python 3.15 adds a sentinel builtin (PEP 661). Every call to sentinel("MISSING") allocates a new, distinct object — not a process-wide singleton — carrying a name, an auto-detected module, and an optional repr, all in Objects/sentinelobject.c.

None, Ellipsis, and NotImplemented are singletons: one object per process. But sometimes you want a marker value that’s unique to your library — MISSING, UNSET, NOT_PROVIDED.

Python 3.15’s sentinel builtin gives you exactly that. Unlike the classic singletons, these are not process-wide: every call allocates a fresh, distinct object. Each one carries a name, a module, and an optional repr, and it’s tracked by the garbage collector. The whole implementation lives in Objects/sentinelobject.c. Let’s trace it through the 3.15 source.

The struct: `sentinelobject`

A sentinel is a small GC object with three owned references:

 c
 
 
// Objects/sentinelobject.c
typedef struct {
 PyObject_HEAD
 PyObject *name;
 PyObject *module;
 PyObject *repr;
} sentinelobject;

name is the __name__ (a str, or a str subclass).
module is the __module__, usually inferred from the caller.
repr is an optional custom repr string. When it’s NULL, repr() falls back to name.

The cast macro is exact, because the type doesn’t allow subclasses:

 c
 
 
// Objects/sentinelobject.c
#define sentinelobject_CAST(op) \
 (assert(PySentinel_Check(op)), _Py_CAST(sentinelobject *, (op)))

The type: `PySentinel_Type`

PySentinel_Type is a GC, immutable type. It’s not a base type, so subclassing is rejected:

 c
 
 
// Objects/sentinelobject.c
PyTypeObject PySentinel_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 .tp_name = "sentinel",
 .tp_basicsize = sizeof(sentinelobject),
 .tp_dealloc = sentinel_dealloc,
 .tp_repr = sentinel_repr,
 .tp_as_number = &sentinel_as_number,
 .tp_hash = PyObject_GenericHash,
 .tp_getattro = PyObject_GenericGetAttr,
 .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_IMMUTABLETYPE
 | Py_TPFLAGS_HAVE_GC,
 .tp_doc = sentinel_doc,
 .tp_traverse = sentinel_traverse,
 .tp_clear = sentinel_clear,
 .tp_richcompare = _Py_BaseObject_RichCompare,
 .tp_methods = sentinel_methods,
 .tp_members = sentinel_members,
 .tp_new = sentinel_new,
 .tp_free = PyObject_GC_Del,
};

The flags and slots worth calling out:

Py_TPFLAGS_HAVE_GC is set because name, module, and repr can participate in reference cycles — for example, a str subclass that stores the sentinel.
Py_TPFLAGS_IMMUTABLETYPE makes the type object itself immutable.
Py_TPFLAGS_BASETYPE is not set, so class Sub(sentinel) raises TypeError.
tp_richcompare is _Py_BaseObject_RichCompare, so equality is identity: a sentinel equals itself and nothing else.
tp_hash is PyObject_GenericHash, i.e. identity-based hashing.
tp_as_number.nb_or is _Py_union_type_or, which is what makes sentinel | int (and int | sentinel) work in type expressions.

The type is registered in the interpreter’s type table (Objects/object.c) and exposed as a builtin:

 c
 
 
// Python/bltinmodule.c
SETBUILTIN("sentinel", &PySentinel_Type);

The header exposes the check macros, which are exact checks because subclasses are impossible:

 c
 
 
// Include/cpython/sentinelobject.h
#define PySentinel_CheckExact(op) Py_IS_TYPE((op), &PySentinel_Type)
#define PySentinel_Check(op) PySentinel_CheckExact(op)

Construction: `sentinel_new`

The clinic-generated sentinel_new accepts a positional-only name (which must be a str or subclass) and a keyword-only repr (default None):

 c
 
 
// Objects/sentinelobject.c
static PyObject *
sentinel_new_impl(PyTypeObject *type, PyObject *name, PyObject *repr)
{
 if (repr == Py_None) {
 repr = NULL;
 }
 else if (!PyUnicode_Check(repr)) {
 _PyArg_BadArgument("sentinel", "argument 'repr'", "str or None", repr);
 return NULL;
 }
 PyObject *module = caller();
 PyObject *self = sentinel_new_with_module(type, name, module, repr);
 Py_DECREF(module);
 return self;
}

The module isn’t passed by the user. Instead, caller() inspects the current execution frame and returns the module of the function that’s currently running:

 c
 
 
// Objects/sentinelobject.c
static PyObject *
caller(void)
{
 _PyInterpreterFrame *f = _PyThreadState_GET()->current_frame;
 if (f == NULL || PyStackRef_IsNull(f->f_funcobj)) {
 assert(!PyErr_Occurred());
 Py_RETURN_NONE;
 }
 PyFunctionObject *func = _PyFrame_GetFunction(f);
 assert(PyFunction_Check(func));
 PyObject *r = PyFunction_GetModule((PyObject *)func);
 if (!r) {
 assert(!PyErr_Occurred());
 Py_RETURN_NONE;
 }
 return Py_NewRef(r);
}

This is why a module-level MISSING = sentinel("MISSING") automatically gets __module__ set to that module’s name — and that’s exactly what makes the value picklable later. At top level with no current function, caller() returns None.

Allocation itself is ordinary GC allocation, followed by tracking:

 c
 
 
// Objects/sentinelobject.c
static PyObject *
sentinel_new_with_module(PyTypeObject *type, PyObject *name, PyObject *module, PyObject *repr)
{
 assert(PyUnicode_Check(name));

 sentinelobject *self = PyObject_GC_New(sentinelobject, type);
 if (self == NULL) {
 return NULL;
 }
 self->name = Py_NewRef(name);
 self->module = Py_NewRef(module);
 self->repr = Py_XNewRef(repr);
 _PyObject_GC_TRACK(self);
 return (PyObject *)self;
}

Note that module is always stored (even when it’s None), while repr is stored only when provided.

The C API: `PySentinel_New`

For C extensions, PySentinel_New builds a sentinel without going through the Python-level caller machinery. The caller supplies the module name (or NULL for None) and the repr (or NULL for “use the name”):

 c
 
 
// Objects/sentinelobject.c
PyObject *
PySentinel_New(const char *name, const char *module_name, const char *repr)
{
 PyObject *name_obj = PyUnicode_FromString(name);
 if (name_obj == NULL) {
 return NULL;
 }
 PyObject *repr_obj = NULL;
 if (repr != NULL) {
 repr_obj = PyUnicode_FromString(repr);
 if (repr_obj == NULL) {
 Py_DECREF(name_obj);
 return NULL;
 }
 }
 PyObject *module_obj = module_name == NULL
 ? Py_None
 : PyUnicode_FromString(module_name);
 if (module_obj == NULL) {
 Py_DECREF(name_obj);
 Py_XDECREF(repr_obj);
 return NULL;
 }

 PyObject *sentinel = sentinel_new_with_module(
 &PySentinel_Type, name_obj, module_obj, repr_obj);
 Py_DECREF(module_obj);
 Py_DECREF(name_obj);
 Py_XDECREF(repr_obj);
 return sentinel;
}

The returned object is a new reference owned by the caller.

Garbage collection lifecycle

Because a sentinel owns references, it participates in the cyclic GC:

 c
 
 
// Objects/sentinelobject.c
static int
sentinel_traverse(PyObject *op, visitproc visit, void *arg)
{
 sentinelobject *self = sentinelobject_CAST(op);
 Py_VISIT(self->name);
 Py_VISIT(self->module);
 Py_VISIT(self->repr);
 return 0;
}

static int
sentinel_clear(PyObject *op)
{
 sentinelobject *self = sentinelobject_CAST(op);
 Py_CLEAR(self->name);
 Py_CLEAR(self->module);
 Py_CLEAR(self->repr);
 return 0;
}

static void
sentinel_dealloc(PyObject *op)
{
 _PyObject_GC_UNTRACK(op);
 (void)sentinel_clear(op);
 Py_TYPE(op)->tp_free(op);
}

tp_free is PyObject_GC_Del, matching the PyObject_GC_New allocation. The traverse/clear pair is what lets the test suite create a str subclass that points back at the sentinel and still collect both objects once they become unreachable.

Attributes: `name` and `module`

The two attributes are exposed as members rather than computed properties:

 c
 
 
// Objects/sentinelobject.c
static PyMemberDef sentinel_members[] = {
 {"__name__", Py_T_OBJECT_EX, offsetof(sentinelobject, name), Py_READONLY},
 {"__module__", Py_T_OBJECT_EX, offsetof(sentinelobject, module), 0},
 {NULL}
};

__name__ is read-only: assigning or deleting it raises AttributeError.
__module__ is read/write and deletable. Deleting it sets the underlying pointer to NULL; a subsequent read raises AttributeError rather than returning None.
The type itself (sentinel) rejects attribute assignment entirely, because it’s immutable (Py_TPFLAGS_IMMUTABLETYPE and no tp_setattro override that allows it). Hence sentinel.attribute = ... raises TypeError.

tp_getattro is PyObject_GenericGetAttr, so normal attribute lookup (including the methods below) works.

Repr

 c
 
 
// Objects/sentinelobject.c
static PyObject *
sentinel_repr(PyObject *op)
{
 sentinelobject *self = sentinelobject_CAST(op);
 if (self->repr != NULL) {
 return Py_NewRef(self->repr);
 }
 return Py_NewRef(self->name);
}

tp_str is 0, so str() falls back to repr(). A custom repr="custom" therefore affects both repr() and str().

Copy, deepcopy, and equality

Sentinels are identity objects: copying returns the same instance, and equality is identity:

 c
 
 
// Objects/sentinelobject.c
static PyObject *
sentinel_copy(PyObject *self, PyObject *Py_UNUSED(ignored))
{
 return Py_NewRef(self);
}

static PyObject *
sentinel_deepcopy(PyObject *self, PyObject *Py_UNUSED(memo))
{
 return Py_NewRef(self);
}

 c
 
 
// Objects/sentinelobject.c (methods table)
static PyMethodDef sentinel_methods[] = {
 {"__copy__", sentinel_copy, METH_NOARGS, NULL},
 {"__deepcopy__", sentinel_deepcopy, METH_O, NULL},
 {"__reduce__", sentinel_reduce, METH_NOARGS, NULL},
 {NULL, NULL}
};

Because tp_richcompare is _Py_BaseObject_RichCompare, s == s is True and s == other is False. And copy.copy / copy.deepcopy hand back the same object (assertIs(copy.copy(s), s)).

Pickling: `reduce`

 c
 
 
// Objects/sentinelobject.c
static PyObject *
sentinel_reduce(PyObject *op, PyObject *Py_UNUSED(ignored))
{
 sentinelobject *self = sentinelobject_CAST(op);
 return Py_NewRef(self->name);
}

__reduce__ returns the name string. Pickle interprets a string result as a global name lookup: it imports __module__ and fetches the attribute named name.

So a sentinel pickles successfully only when it’s reachable as an attribute of an importable module under exactly that name. A dynamically created sentinel("MISSING") that isn’t stored anywhere is not picklable and raises PicklingError. That matches the C API documentation: for pickling to work, module_name must name an importable module and the sentinel must be accessible from it.

Gotcha: this is the whole reason the module-level pattern matters. Define your sentinel at module scope, not inside a function.

Union types: `sentinel | X`

sentinel overrides nb_or so it can appear in X | Y type expressions:

 c
 
 
// Objects/sentinelobject.c
static PyNumberMethods sentinel_as_number = {
 .nb_or = _Py_union_type_or,
};

_Py_union_type_or (in Objects/unionobject.c) only proceeds when both operands are “unionable”, and a sentinel is explicitly in that set:

 c
 
 
// Objects/unionobject.c
static int
is_unionable(PyObject *obj)
{
 if (obj == Py_None ||
 PyType_Check(obj) ||
 PySentinel_Check(obj) ||
 _PyGenericAlias_Check(obj) ||
 _PyUnion_Check(obj) ||
 Py_IS_TYPE(obj, &_PyTypeAlias_Type)) {
 return 1;
 }
 return 0;
}

So missing | int, int | missing, missing | None, and missing | list[int] all build a types.UnionType. A union of a sentinel with itself collapses to the sentinel itself (missing | missing is missing), because the union builder deduplicates equal — here identical — members.

Memory layout

A sentinel is a normal heap-allocated GC object:

text

sentinelobject (heap, GC-tracked)
+----------------------------+
| ob_refcnt |
+----------------------------+
| ob_flags (tracked bits) |
+----------------------------+
| *ob_type (&PySentinel_Type)|
+----------------------------+
| *name (owned str) |
+----------------------------+
| *module (owned or NULL) |
+----------------------------+
| *repr (owned or NULL) |
+----------------------------+

There’s no freelist and no static instance: every call allocates. The object is collected by the cyclic GC once it becomes unreachable, and its three owned references are cleared in sentinel_clear.

Key takeaways

sentinel is a builtin, GC-tracked, immutable, non-subclassable type whose instances are heap-allocated in Objects/sentinelobject.c.
Each instance stores name (read-only __name__), module (mutable, deletable __module__), and an optional repr.
The module is inferred from the calling function’s frame by caller(); the C API PySentinel_New takes it explicitly.
Equality and hashing are identity-based; __copy__/__deepcopy__ return self.
__reduce__ returns name, so pickling works only for module-level sentinels reachable by that name.
nb_or delegates to _Py_union_type_or, and sentinels are explicitly unionable, enabling sentinel | X in type expressions.
The type and its instances are tracked by the GC, so reference cycles involving the stored strings get collected.

CPython's range Implementation in C: Lazy Sequences That Never Materialize

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: range isn’t a list of integers. It’s four PyObject * fields — start, stop, step, and a cached length — and every value is computed on demand. len() is O(1), indexing is arithmetic, and x in range(...) is O(1) when x is an int.

Python’s range is not a list of integers. It’s a tiny immutable object that remembers only its start, stop, and step, then computes everything else on demand. range(0, 10**18) takes exactly the same memory as range(3).

In CPython that’s rangeobject in Objects/rangeobject.c, with a cached length and two iterator types. Let’s walk the representation and the hot paths through the 3.15 source.

The object struct

 c
 
 
// Objects/rangeobject.c
typedef struct {
 PyObject_HEAD
 PyObject *start;
 PyObject *stop;
 PyObject *step;
 PyObject *length;
} rangeobject;

Four PyObject * fields, all owned references. start, stop, and step are the user-supplied bounds, normalized to PyLong objects via PyNumber_Index in range_from_array. length is the precomputed number of items, also a PyLong.

There’s no array of integers anywhere. The sequence is fully lazy.

rangeobject is a fixed-size PyObject (sizeof(rangeobject)), so its allocation cost is constant no matter how many values the range would yield. Deallocation just decrements the four fields and returns the object to a small freelist (Py_ranges_MAXFREELIST == 6):

 c
 
 
// Objects/rangeobject.c
static void range_dealloc(PyObject *op) {
 rangeobject *r = (rangeobject*)op;
 Py_DECREF(r->start);
 Py_DECREF(r->stop);
 Py_DECREF(r->step);
 Py_DECREF(r->length);
 _Py_FREELIST_FREE(ranges, r, PyObject_Free);
}

Construction and the cached length

range_new / range_vectorcall funnel into range_from_array, which converts arguments and calls make_range_object:

 c
 
 
// Objects/rangeobject.c
static rangeobject *make_range_object(PyTypeObject *type, PyObject *start,
 PyObject *stop, PyObject *step) {
 PyObject *length = compute_range_length(start, stop, step);
 if (length == NULL) return NULL;
 rangeobject *obj = _Py_FREELIST_POP(rangeobject, ranges);
 if (obj == NULL) {
 obj = PyObject_New(rangeobject, type);
 ...
 }
 obj->start = start;
 obj->stop = stop;
 obj->step = step;
 obj->length = length;
 return obj;
}

compute_range_length first tries a fast path where all three values fit in a C long (compute_range_length_long), using the same formula as get_len_of_range:

 c
 
 
// Objects/rangeobject.c
static unsigned long get_len_of_range(long lo, long hi, long step) {
 assert(step != 0);
 if (step > 0 && lo < hi)
 return 1UL + (hi - 1UL - lo) / step;
 else if (step < 0 && lo > hi)
 return 1UL + (lo - 1UL - hi) / (0UL - step);
 else
 return 0UL;
}

If any value overflows long (or the result doesn’t fit), it falls back to PyObject arithmetic: take the positive-step case, compute (hi - lo - 1) // step + 1, clamping to zero when lo >= hi.

The result is stored once in r->length, so len(r) is just:

 c
 
 
// Objects/rangeobject.c
static Py_ssize_t range_length(PyObject *op) {
 rangeobject *r = (rangeobject*)op;
 return PyLong_AsSsize_t(r->length);
}

That’s why len(range(...)) is O(1) — and why a range whose length exceeds PY_SSIZE_T_MAX (possible on Win64 with huge ints) still works through the length field even though sq_length can’t represent it.

Indexing: computing, not looking up

range_item / range_subscript never touch a buffer. For an integer index, compute_range_item normalizes negative indices against length, bounds-checks, then computes the value:

 c
 
 
// Objects/rangeobject.c
static PyObject *compute_item(rangeobject *r, PyObject *i) {
 /* PyLong equivalent to: return r->start + (i * r->step) */
 if (r->step == _PyLong_GetOne()) {
 return PyNumber_Add(r->start, i);
 } else {
 PyObject *incr = PyNumber_Multiply(i, r->step);
 if (!incr) return NULL;
 PyObject *result = PyNumber_Add(r->start, incr);
 Py_DECREF(incr);
 return result;
 }
}

Notice the fast path when step == 1: it skips the multiply entirely. Negative indices are handled by adding length to the index first, exactly like standard sequence semantics. Out-of-range indices raise IndexError.

Slicing returns a new range, not a list. compute_slice uses _PySlice_GetLongIndices against r->length, then builds substart = compute_item(r, start), substop = compute_item(r, stop) and substep = r->step * step, and constructs a new range via make_range_object. So range(10)[2:8:2] is range(2, 8, 4) and costs O(1).

`contains` is O(1)

For an exact int (or bool), range_contains dispatches to range_contains_long, which avoids iteration entirely:

 c
 
 
// Objects/rangeobject.c
static int range_contains_long(rangeobject *r, PyObject *ob) {
 /* Check if the value can possibly be in the range. */
 cmp1 = PyObject_RichCompareBool(r->step, zero, Py_GT);
 if (cmp1 == 1) { /* positive steps: start <= ob < stop */
 cmp2 = PyObject_RichCompareBool(r->start, ob, Py_LE);
 cmp3 = PyObject_RichCompareBool(ob, r->stop, Py_LT);
 } else { /* negative steps: stop < ob <= start */
 cmp2 = PyObject_RichCompareBool(ob, r->start, Py_LE);
 cmp3 = PyObject_RichCompareBool(r->stop, ob, Py_LT);
 }
 if (cmp2 == 0 || cmp3 == 0) { /* ob outside of range */
 result = 0; goto end;
 }
 /* Check that the stride does not invalidate membership. */
 tmp1 = PyNumber_Subtract(ob, r->start);
 tmp2 = PyNumber_Remainder(tmp1, r->step);
 result = PyObject_RichCompareBool(tmp2, zero, Py_EQ);
end:
 ...
}

The algorithm is two steps: bound check (start <= ob < stop for a positive step, reversed for a negative one), then a divisibility check (ob - start) % step == 0. Both are O(1) big-int operations.

Gotcha: for any other object type, range_contains falls back to _PySequence_IterSearch — a linear scan. So x in range(...) is only guaranteed O(1) when x is an int.

range_index reuses the same membership test and then computes the index as (ob - start) // step (or just ob - start when step is 1).

Iteration: two tiers

range_iter picks between a fast C-long iterator and a generic PyObject-based one, depending on whether everything fits in a long:

 c
 
 
// Objects/rangeobject.c
static PyObject *range_iter(PyObject *seq) {
 rangeobject *r = (rangeobject *)seq;
 long lstart = PyLong_AsLong(r->start);
 if (lstart == -1 && PyErr_Occurred()) { PyErr_Clear(); goto long_range; }
 ... /* lstop, lstep similarly */
 ulen = get_len_of_range(lstart, lstop, lstep);
 if (ulen > (unsigned long)LONG_MAX) goto long_range;
 /* overflow check for lstart + ulen * lstep */
 ...
 return fast_range_iter(lstart, lstop, lstep, (long)ulen);
long_range:
 it = PyObject_New(longrangeiterobject, &PyLongRangeIter_Type);
 it->start = Py_NewRef(r->start);
 it->step = Py_NewRef(r->step);
 it->len = Py_NewRef(r->length);
 return (PyObject *)it;
}

The fast tier uses _PyRangeIterObject, whose entire state is three C longs:

 c
 
 
// Include/internal/pycore_range.h
typedef struct {
 PyObject_HEAD
 long start;
 long step;
 long len;
} _PyRangeIterObject;

Its tp_iternext is trivial, and allocation-free beyond the first PyLong_FromLong:

 c
 
 
// Objects/rangeobject.c
static PyObject *rangeiter_next(PyObject *op) {
 _PyRangeIterObject *r = (_PyRangeIterObject*)op;
 if (r->len > 0) {
 long result = r->start;
 r->start = result + r->step;
 r->len--;
 return PyLong_FromLong(result);
 }
 return NULL;
}

The generic tier (longrangeiterobject) stores PyObject *start/step/len and does PyNumber_Add / PyNumber_Subtract per step, so it handles arbitrary-precision bounds at the cost of per-iteration allocation. Both iterators are recycled through freelists of size 6.

__reversed__ (range_reverse) re-expresses the reversed sequence as a new range range(start+(n-1)*step, start-step, -step) and again prefers the fast C-long iterator when all values fit.

Memory efficiency summary

A range is sizeof(rangeobject) (≈ 40 bytes on 64-bit) plus the four referenced PyLong objects — independent of its length.
The length is computed once at construction (O(1) for small ints, big-int arithmetic otherwise) and cached.
Indexing, slicing, containment, and len() are all O(1) for ints. Only non-int containment and iteration over huge values degrade.
Iteration has a zero-allocation-per-step fast path for the common case where bounds fit in a C long.

Key takeaways

range stores only start, stop, step, and a cached length. The sequence is never materialized.
Membership of an int is O(1) via a bounds check plus a modulo divisibility test; non-ints fall back to a linear search.
Indexing computes start + i * step; slicing produces another range.
There are two iterator tiers: a C-long fast path (_PyRangeIterObject) and a generic PyObject path for arbitrary precision.
Both range and its iterators are recycled through 6-deep freelists.

CPython's PyCapsule Implementation in C: A void* Wrapper for Extension Modules

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: A PyCapsule is a tiny Python object whose only job is to carry a C void * through the interpreter. It adds a name (so you can’t grab the wrong pointer), a destructor (so the C side can clean up), and optional GC hooks. The whole thing lives in Objects/capsule.c.

Extension modules use capsules to export a C API to other extension modules. The exporting module stores a pointer — often to a struct of function pointers — in a capsule and attaches it to a module attribute. The importing module calls PyCapsule_Import to retrieve it.

The implementation lives entirely in Objects/capsule.c.

The struct: `PyCapsule`

Internally a capsule is a plain struct: an object header followed by six fields.

 c
 
 
// Objects/capsule.c
typedef struct {
 PyObject_HEAD
 void *pointer;
 const char *name;
 void *context;
 PyCapsule_Destructor destructor;
 traverseproc traverse_func;
 inquiry clear_func;
} PyCapsule;

What each field is for:

pointer is the wrapped void * (never NULL for a legal capsule).
name is an optional, conventionally dotted attribute path such as "socket.CAPI" (or NULL).
context is an arbitrary void * for the owner’s use.
destructor is called exactly once when the capsule is deallocated.
traverse_func / clear_func are only set by _PyCapsule_SetTraverse and enable GC participation (see below).

The public type is declared in Include/pycapsule.h:

 c
 
 
// Include/pycapsule.h
PyAPI_DATA(PyTypeObject) PyCapsule_Type;
typedef void (*PyCapsule_Destructor)(PyObject *);
#define PyCapsule_CheckExact(op) Py_IS_TYPE((op), &PyCapsule_Type)

The type: `PyCapsule_Type`

PyCapsule_Type is a GC-capable type with tp_basicsize = sizeof(PyCapsule):

 c
 
 
// Objects/capsule.c
PyTypeObject PyCapsule_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 .tp_name = "PyCapsule",
 .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
 .tp_basicsize = sizeof(PyCapsule),
 .tp_dealloc = capsule_dealloc,
 .tp_repr = capsule_repr,
 .tp_doc = PyCapsule_Type__doc__,
 .tp_traverse = capsule_traverse,
 .tp_clear = capsule_clear,
};

The key points:

Py_TPFLAGS_HAVE_GC is set because a capsule may own references to Python objects through its context/pointer (via the optional traverse callbacks).
However, a freshly created capsule is not GC-tracked. Tracking is opt-in through _PyCapsule_SetTraverse. That keeps the common case — no owned Python objects — cheap and skips unnecessary GC work.
tp_clear is capsule_clear, which only runs when traverse was registered.

Construction: `PyCapsule_New`

 c
 
 
// Objects/capsule.c
PyObject *
PyCapsule_New(void *pointer, const char *name, PyCapsule_Destructor destructor)
{
 PyCapsule *capsule;

 if (!pointer) {
 PyErr_SetString(PyExc_ValueError, "PyCapsule_New called with null pointer");
 return NULL;
 }

 capsule = PyObject_GC_New(PyCapsule, &PyCapsule_Type);
 if (capsule == NULL) {
 return NULL;
 }

 capsule->pointer = pointer;
 capsule->name = name;
 capsule->context = NULL;
 capsule->destructor = destructor;
 capsule->traverse_func = NULL;
 capsule->clear_func = NULL;
 // Only track the object by the GC when _PyCapsule_SetTraverse() is called

 return (PyObject *)capsule;
}

Things to note:

A NULL pointer is rejected up front with ValueError.
The name is stored by pointer, not copied. The caller owns that string memory for the capsule’s lifetime — which is why module-level capsule names are typically string literals.
The destructor may be NULL, in which case nothing runs at dealloc.
context starts as NULL.

Legality and name matching

Most accessors begin with the is_legal_capsule guard:

 c
 
 
// Objects/capsule.c
static int
_is_legal_capsule(PyObject *op, const char *invalid_capsule)
{
 if (!op || !PyCapsule_CheckExact(op)) {
 goto error;
 }
 PyCapsule *capsule = (PyCapsule *)op;

 if (capsule->pointer == NULL) {
 goto error;
 }
 return 1;

error:
 PyErr_SetString(PyExc_ValueError, invalid_capsule);
 return 0;
}

A capsule becomes “illegal” once its pointer is cleared (set to NULL), which lets a destructor mark it as consumed.

Name comparison is done by name_matches, which treats two NULL names as equal and otherwise uses strcmp:

 c
 
 
// Objects/capsule.c
static int
name_matches(const char *name1, const char *name2) {
 if (!name1 || !name2) {
 return name1 == name2;
 }
 return !strcmp(name1, name2);
}

PyCapsule_IsValid combines the legality and name checks:

 c
 
 
// Objects/capsule.c
int
PyCapsule_IsValid(PyObject *op, const char *name)
{
 PyCapsule *capsule = (PyCapsule *)op;

 return (capsule != NULL &&
 PyCapsule_CheckExact(capsule) &&
 capsule->pointer != NULL &&
 name_matches(capsule->name, name));
}

The name is a safety tag

The name is a const char *, and it’s compared (not copied) by every getter that takes a name argument.

PyCapsule_GetPointer requires the supplied name to match the stored one:

 c
 
 
// Objects/capsule.c
void *
PyCapsule_GetPointer(PyObject *op, const char *name)
{
 if (!is_legal_capsule(op, "PyCapsule_GetPointer")) {
 return NULL;
 }
 PyCapsule *capsule = (PyCapsule *)op;

 if (!name_matches(name, capsule->name)) {
 PyErr_SetString(PyExc_ValueError, "PyCapsule_GetPointer called with incorrect name");
 return NULL;
 }

 return capsule->pointer;
}

PyCapsule_GetName / PyCapsule_SetName read and overwrite the pointer (again, no copy):

 c
 
 
// Objects/capsule.c
const char *
PyCapsule_GetName(PyObject *op)
{
 if (!is_legal_capsule(op, "PyCapsule_GetName")) {
 return NULL;
 }
 PyCapsule *capsule = (PyCapsule *)op;
 return capsule->name;
}

int
PyCapsule_SetName(PyObject *op, const char *name)
{
 if (!is_legal_capsule(op, "PyCapsule_SetName")) {
 return -1;
 }
 PyCapsule *capsule = (PyCapsule *)op;
 capsule->name = name;
 return 0;
}

Here’s the thing: the name’s purpose is purely defensive. It guards against accidentally retrieving a pointer from the wrong capsule, and it’s the key used by PyCapsule_Import.

The destructor gets one shot

The destructor is a PyCapsule_Destructor (void (*)(PyObject *)). It’s invoked from capsule_dealloc before the object memory is freed:

 c
 
 
// Objects/capsule.c
static void
capsule_dealloc(PyObject *op)
{
 PyCapsule *capsule = _PyCapsule_CAST(op);
 PyObject_GC_UnTrack(op);
 if (capsule->destructor) {
 capsule->destructor(op);
 }
 PyObject_GC_Del(op);
}

Important details:

PyObject_GC_UnTrack runs first, so the object is out of the GC list before the destructor (which may manipulate other objects) runs.
The destructor receives the capsule object itself, so it can call PyCapsule_GetPointer/GetContext to recover its payload.
The destructor is responsible for releasing whatever the pointer owns (e.g. PyMem_Free, Py_DECREF on referenced types). capsule_dealloc then frees the capsule struct via PyObject_GC_Del.
PyCapsule_GetDestructor / PyCapsule_SetDestructor expose the field:

 c
 
 
// Objects/capsule.c
PyCapsule_Destructor
PyCapsule_GetDestructor(PyObject *op)
{
 if (!is_legal_capsule(op, "PyCapsule_GetDestructor")) {
 return NULL;
 }
 PyCapsule *capsule = (PyCapsule *)op;
 return capsule->destructor;
}

int
PyCapsule_SetDestructor(PyObject *op, PyCapsule_Destructor destructor)
{
 if (!is_legal_capsule(op, "PyCapsule_SetDestructor")) {
 return -1;
 }
 PyCapsule *capsule = (PyCapsule *)op;
 capsule->destructor = destructor;
 return 0;
}

A common pattern is to clear the pointer inside the destructor, marking the capsule consumed:

 c
 
 
static void my_destructor(PyObject *capsule) {
 void *p = PyCapsule_GetPointer(capsule, "my.module.API");
 /* release p ... */
 PyCapsule_SetPointer(capsule, NULL); /* mark consumed */
}

Context: the owner’s scratchpad

context is a free-form void * slot, accessed via:

 c
 
 
// Objects/capsule.c
void *
PyCapsule_GetContext(PyObject *op)
{
 if (!is_legal_capsule(op, "PyCapsule_GetContext")) {
 return NULL;
 }
 PyCapsule *capsule = (PyCapsule *)op;
 return capsule->context;
}

int
PyCapsule_SetContext(PyObject *op, void *context)
{
 if (!is_legal_capsule(op, "PyCapsule_SetContext")) {
 return -1;
 }
 PyCapsule *capsule = (PyCapsule *)op;
 capsule->context = context;
 return 0;
}

Unlike pointer, context may be NULL and isn’t subject to the legality check’s non-null requirement.

It’s typically used to stash auxiliary data the destructor or traverse callback needs (for example, a reference to an owning object), since the destructor only receives the capsule.

Optional GC traversal

By default a capsule owns no Python references as far as the GC is concerned.

When the wrapped pointer does refer to Python objects (say, a C API struct holding PyObject * fields), the owner can register traverse/clear callbacks:

 c
 
 
// Objects/capsule.c
int
_PyCapsule_SetTraverse(PyObject *op, traverseproc traverse_func, inquiry clear_func)
{
 if (!is_legal_capsule(op, "_PyCapsule_SetTraverse")) {
 return -1;
 }
 PyCapsule *capsule = (PyCapsule *)op;

 if (traverse_func == NULL || clear_func == NULL) {
 PyErr_SetString(PyExc_ValueError,
 "_PyCapsule_SetTraverse() called with NULL callback");
 return -1;
 }

 if (!_PyObject_GC_IS_TRACKED(op)) {
 _PyObject_GC_TRACK(op);
 }

 capsule->traverse_func = traverse_func;
 capsule->clear_func = clear_func;
 return 0;
}

This is an internal API (declared in Include/internal/pycore_capsule.h), used by modules such as socket and _curses:

 c
 
 
// Modules/socketmodule.c
PyObject *capsule = PyCapsule_New(capi, PySocket_CAPSULE_NAME, sock_capi_destroy);
if (_PyCapsule_SetTraverse(capsule,
 sock_capi_traverse, sock_capi_clear) < 0) { ... }

The callbacks receive the capsule and use PyCapsule_GetPointer to reach the wrapped struct, then Py_VISIT/Py_CLEAR its Python members:

 c
 
 
// Modules/socketmodule.c
static int
sock_capi_traverse(PyObject *capsule, visitproc visit, void *arg)
{
 PySocketModule_APIObject *capi = PyCapsule_GetPointer(capsule, PySocket_CAPSULE_NAME);
 assert(capi != NULL);
 Py_VISIT(capi->Sock_Type);
 return 0;
}

static int
sock_capi_clear(PyObject *capsule)
{
 PySocketModule_APIObject *capi = PyCapsule_GetPointer(capsule, PySocket_CAPSULE_NAME);
 assert(capi != NULL);
 Py_CLEAR(capi->Sock_Type);
 return 0;
}

The generic traverse/clear in capsule.c just dispatch to the registered callbacks (and capsule_clear asserts that a clear callback exists):

 c
 
 
// Objects/capsule.c
static int
capsule_traverse(PyObject *self, visitproc visit, void *arg)
{
 PyCapsule *capsule = _PyCapsule_CAST(self);
 if (capsule->traverse_func != NULL) {
 return capsule->traverse_func(self, visit, arg);
 }
 return 0;
}

static int
capsule_clear(PyObject *self)
{
 PyCapsule *capsule = _PyCapsule_CAST(self);
 assert(capsule->clear_func != NULL);
 return capsule->clear_func(self);
}

`PyCapsule_Import`: the headline use case

This is how one extension module imports a capsule from another module’s attribute:

 c
 
 
// Objects/capsule.c
void *
PyCapsule_Import(const char *name, int no_block)
{
 PyObject *object = NULL;
 void *return_value = NULL;
 char *trace;
 size_t name_length = (strlen(name) + 1) * sizeof(char);
 char *name_dup = (char *)PyMem_Malloc(name_length);
 ...
 memcpy(name_dup, name, name_length);

 trace = name_dup;
 while (trace) {
 char *dot = strchr(trace, '.');
 if (dot) {
 *dot++ = '\0';
 }

 if (object == NULL) {
 object = PyImport_ImportModule(trace);
 ...
 } else {
 PyObject *object2 = PyObject_GetAttrString(object, trace);
 Py_SETREF(object, object2);
 }
 ...
 trace = dot;
 }

 if (PyCapsule_IsValid(object, name)) {
 PyCapsule *capsule = (PyCapsule *)object;
 return_value = capsule->pointer;
 } else {
 PyErr_Format(PyExc_AttributeError,
 "PyCapsule_Import \"%s\" is not valid", name);
 }

EXIT:
 Py_XDECREF(object);
 if (name_dup) {
 PyMem_Free(name_dup);
 }
 return return_value;
}

It walks the dotted name ("module.sub.attr"), importing the first segment and then getattr-ing the rest, and finally validates that the resolved object is a capsule whose name matches the full name.

The no_block parameter is accepted for API compatibility, but the implementation imports normally.

On success it returns the borrowed pointer — the capsule (and thus the pointer) stays alive as long as the module holds it.

Repr

 c
 
 
// Objects/capsule.c
static PyObject *
capsule_repr(PyObject *o)
{
 PyCapsule *capsule = _PyCapsule_CAST(o);
 const char *name;
 const char *quote;

 if (capsule->name) {
 quote = "\"";
 name = capsule->name;
 } else {
 quote = "";
 name = "NULL";
 }

 return PyUnicode_FromFormat("<capsule object %s%s%s at %p>",
 quote, name, quote, capsule);
}

So repr(capsule) looks like <capsule object "socket.CAPI" at 0x...> or <capsule object NULL at 0x...>.

The memory layout

text

PyCapsule (sizeof = 7 words on 64-bit)
+----------------------------+
| ob_refcnt |
+----------------------------+
| ob_flags |
+----------------------------+
| ob_type -> &PyCapsule_Type|
+----------------------------+
| pointer (void *) |
+----------------------------+
| name (const char *) |
+----------------------------+
| context (void *) |
+----------------------------+
| destructor |
+----------------------------+
| traverse_func |
+----------------------------+
| clear_func |
+----------------------------+

Key takeaways

A PyCapsule is a GC-capable object wrapping a void *, plus a name, context, destructor, and optional traverse/clear callbacks.
PyCapsule_New rejects NULL pointers; the name is stored by pointer (not copied) and context starts NULL.
Legality requires a non-null pointer; setting it to NULL (often in the destructor) marks the capsule consumed.
Names are compared with strcmp (with NULL == NULL), and getters like PyCapsule_GetPointer require a matching name.
The destructor runs during dealloc, before PyObject_GC_Del, and receives the capsule so it can recover its payload.
context is a free-form slot not subject to the non-null legality check.
GC tracking is opt-in via the internal _PyCapsule_SetTraverse; otherwise capsules are untracked.
PyCapsule_Import walks a dotted path, validates the final capsule by name, and returns its pointer.

The one thing to remember: a capsule is a typed handle, not a container. The name is a cheap tag that catches “wrong pointer”, the destructor is your one guaranteed cleanup hook, and everything else is convention.

CPython's pickle.PickleBuffer Implementation in C: Out-of-Band Data Without Copies

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: pickle.PickleBuffer is a fixed-size C object that holds exactly one Py_buffer view, taken with PyBUF_FULL_RO, and never copies a byte. Its buffer-protocol implementation redirects to the original exporter, so whatever flags the consumer asks for are honored end to end.

Python’s pickle.PickleBuffer wraps a buffer-providing object so pickle can move its data out-of-band (protocol 5+). It isn’t implemented in Lib/pickle.py — CPython puts it in Objects/picklebufobject.c as a small GC object that stores a single Py_buffer view taken with PyBUF_FULL_RO. Let’s trace it through the 3.15 source.

The object struct

PyPickleBufferObject is fixed-size. It keeps the view exported from the original object plus a weakref list:

 c
 
 
// Objects/picklebufobject.c
typedef struct {
 PyObject_HEAD
 /* The view exported by the original object */
 Py_buffer view;
 PyObject *weakreflist;
} PyPickleBufferObject;

The type object registers it as a GC object with weakref support and buffer procs:

 c
 
 
// Objects/picklebufobject.c
PyTypeObject PyPickleBuffer_Type = {
 PyVarObject_HEAD_INIT(NULL, 0)
 .tp_name = "pickle.PickleBuffer",
 .tp_doc = PyDoc_STR("Wrapper for potentially out-of-band buffers"),
 .tp_basicsize = sizeof(PyPickleBufferObject),
 .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
 .tp_new = picklebuf_new,
 .tp_dealloc = picklebuf_dealloc,
 .tp_traverse = picklebuf_traverse,
 .tp_clear = picklebuf_clear,
 .tp_weaklistoffset = offsetof(PyPickleBufferObject, weakreflist),
 .tp_as_buffer = &picklebuf_as_buffer,
 .tp_methods = picklebuf_methods,
};

Here’s what’s interesting: unlike memoryview, there’s no separate managed-buffer object and no private copy of shape/strides/suboffsets. The PickleBuffer simply holds the exporter’s Py_buffer directly.

Creating a PickleBuffer

Both the C API and the Python constructor do the same thing: allocate the object, then obtain a full read-only view from base:

 c
 
 
// Objects/picklebufobject.c
PyObject *
PyPickleBuffer_FromObject(PyObject *base)
{
 PyTypeObject *type = &PyPickleBuffer_Type;
 PyPickleBufferObject *self;

 self = (PyPickleBufferObject *) type->tp_alloc(type, 0);
 if (self == NULL) {
 return NULL;
 }
 self->view.obj = NULL;
 self->weakreflist = NULL;
 if (PyObject_GetBuffer(base, &self->view, PyBUF_FULL_RO) < 0) {
 Py_DECREF(self);
 return NULL;
 }
 return (PyObject *) self;
}

picklebuf_new is identical in shape, parsing a single positional argument named "":

 c
 
 
// Objects/picklebufobject.c
static PyObject *
picklebuf_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
 ...
 if (!PyArg_ParseTupleAndKeywords(args, kwds, "O:PickleBuffer",
 keywords, &base)) {
 return NULL;
 }
 self = (PyPickleBufferObject *) type->tp_alloc(type, 0);
 ...
 self->view.obj = NULL;
 self->weakreflist = NULL;
 if (PyObject_GetBuffer(base, &self->view, PyBUF_FULL_RO) < 0) {
 Py_DECREF(self);
 return NULL;
 }
 return (PyObject *) self;
}

Notice that self->view.obj is set to NULL first. That’s deliberate: if PyObject_GetBuffer fails (or the allocation is interrupted), you can’t leave a dangling pointer that dealloc/clear would later try to release. On success, view.obj points at base with its reference count held by the Py_buffer, so the original exporter stays alive for as long as the PickleBuffer does.

The released state

A PickleBuffer can be released, after which its view is no longer usable. Because PyBuffer_Release sets view->obj = NULL, the code uses exactly that as the released marker:

 c
 
 
// Objects/picklebufobject.c
const Py_buffer *
PyPickleBuffer_GetBuffer(PyObject *obj)
{
 PyPickleBufferObject *self = (PyPickleBufferObject *) obj;

 if (!PyPickleBuffer_Check(obj)) { ... TypeError ... }
 if (self->view.obj == NULL) {
 PyErr_SetString(PyExc_ValueError,
 "operation forbidden on released PickleBuffer object");
 return NULL;
 }
 return &self->view;
}

The same guard appears in picklebuf_raw and picklebuf_getbuf. Releasing is exposed both as the C API function and as a method:

 c
 
 
// Objects/picklebufobject.c
int
PyPickleBuffer_Release(PyObject *obj)
{
 ...
 PyBuffer_Release(&self->view);
 return 0;
}

static PyObject *
picklebuf_release(PyObject *op, PyObject *Py_UNUSED(ignored))
{
 PyPickleBufferObject *self = (PyPickleBufferObject*)op;
 PyBuffer_Release(&self->view);
 Py_RETURN_NONE;
}

PyBuffer_Release is idempotent here: once view.obj is NULL it does nothing, so calling release() twice is safe.

Buffer protocol redirection

A PickleBuffer itself implements the buffer protocol. But instead of copying its stored view, it re-exports from the original object with whatever flags the caller requests:

 c
 
 
// Objects/picklebufobject.c
static int
picklebuf_getbuf(PyObject *op, Py_buffer *view, int flags)
{
 PyPickleBufferObject *self = (PyPickleBufferObject*)op;
 if (self->view.obj == NULL) {
 PyErr_SetString(PyExc_ValueError,
 "operation forbidden on released PickleBuffer object");
 return -1;
 }
 return PyObject_GetBuffer(self->view.obj, view, flags);
}

This is an important design choice. The stored view is always PyBUF_FULL_RO, but a consumer can ask for a different level of detail — format, strides, shape, writability — and the PickleBuffer forwards that request straight to the original exporter. The release callback is a no-op that exists only as a sentinel:

 c
 
 
// Objects/picklebufobject.c
static void
picklebuf_releasebuf(PyObject *self, Py_buffer *view)
{
 /* Since our bf_getbuffer redirects to the original object, this
 * implementation is never called. It only exists to signal that
 * buffers exported by PickleBuffer have non-trivial releasing
 * behaviour (see check in Python/getargs.c).
 */
}

Because bf_getbuffer redirects, the real release happens against the original object’s buffer — not against the PickleBuffer’s own view.

`raw()`: reinterpreting the buffer as raw bytes

PickleBuffer.raw() returns a memoryview over the underlying memory, but only when the wrapped buffer is contiguous. The implementation builds an ordinary memoryview from self, then mutates it in place to describe the data as a 1-D array of bytes:

 c
 
 
// Objects/picklebufobject.c
static PyObject *
picklebuf_raw(PyObject *op, PyObject *Py_UNUSED(ignored))
{
 PyPickleBufferObject *self = (PyPickleBufferObject*)op;
 if (self->view.obj == NULL) { ... released error ... }
 if (self->view.suboffsets != NULL
 || !PyBuffer_IsContiguous(&self->view, 'A')) {
 PyErr_SetString(PyExc_BufferError,
 "cannot extract raw buffer from non-contiguous buffer");
 return NULL;
 }
 PyObject *m = PyMemoryView_FromObject((PyObject *) self);
 if (m == NULL) {
 return NULL;
 }
 PyMemoryViewObject *mv = (PyMemoryViewObject *) m;
 assert(mv->view.suboffsets == NULL);
 /* Mutate memoryview instance to make it a "raw" memoryview */
 mv->view.format = "B";
 mv->view.ndim = 1;
 mv->view.itemsize = 1;
 /* shape = (length,) */
 mv->view.shape = &mv->view.len;
 /* strides = (1,) */
 mv->view.strides = &mv->view.itemsize;
 /* Fix memoryview state flags */
 mv->flags = _Py_MEMORYVIEW_C | _Py_MEMORYVIEW_FORTRAN;
 return m;
}

Here’s the trick: the shape and strides pointers are pointed at the len and itemsize fields of the same Py_buffer. So the memoryview describes exactly len bytes with stride 1, and no arrays are allocated at all. That’s what pickle uses to obtain a contiguous byte view for in-band serialization, and to hand to a buffer callback for out-of-band transfer.

Lifetime, GC, and weakrefs

The PickleBuffer is GC-tracked because its view.obj can participate in reference cycles — the wrapped object may itself refer back to the PickleBuffer. Traversal visits that reference:

 c
 
 
// Objects/picklebufobject.c
static int
picklebuf_traverse(PyObject *op, visitproc visit, void *arg)
{
 PyPickleBufferObject *self = (PyPickleBufferObject*)op;
 Py_VISIT(self->view.obj);
 return 0;
}

Clearing releases the underlying buffer, and deallocation does the same:

 c
 
 
// Objects/picklebufobject.c
static int
picklebuf_clear(PyObject *op)
{
 PyPickleBufferObject *self = (PyPickleBufferObject*)op;
 PyBuffer_Release(&self->view);
 return 0;
}

static void
picklebuf_dealloc(PyObject *op)
{
 PyPickleBufferObject *self = (PyPickleBufferObject*)op;
 PyObject_GC_UnTrack(self);
 FT_CLEAR_WEAKREFS(op, self->weakreflist);
 PyBuffer_Release(&self->view);
 Py_TYPE(self)->tp_free((PyObject *) self);
}

FT_CLEAR_WEAKREFS clears any weak references before the object is freed, and PyBuffer_Release both calls the exporter’s release callback (if any) and decrements view.obj. After that, the slot is freed with tp_free.

Use in the pickle protocol

In Modules/_pickle.c, a PickleBuffer is detected by type and routed to save_picklebuffer, which obtains a full view, checks contiguity, then either writes the data in-band (as bytes/bytearray) or writes NEXT_BUFFER/READONLY_BUFFER opcodes for out-of-band transfer and hands the buffer to the user’s buffer callback.

On unpickling, NEXT_BUFFER pops the next buffer from the provided buffers iterable, and READONLY_BUFFER wraps it in a read-only memoryview when needed.

Key takeaways

PickleBuffer is a fixed-size GC object holding one Py_buffer view taken with PyBUF_FULL_RO.
It neither copies nor manages shape/strides. It stores the exporter’s view directly and keeps the exporter alive through view.obj.
Released state is detected by view.obj == NULL, which PyBuffer_Release sets for you.
Its buffer-protocol implementation redirects to the original exporter with the caller’s requested flags.
raw() builds a memoryview and reinterprets it as a contiguous 1-D byte array by pointing shape/strides at the view’s own len/itemsize.
Weakrefs and GC traversal are supported, and deallocation releases the underlying buffer.

CPython's OrderedDict in C: O(1) Insertion Order That Just Works

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

TL;DR: OrderedDict is a C-level dict subclass in Objects/odictobject.c. It bolts a doubly-linked list of keys onto the dict and mirrors that list into an od_fast_nodes array so node lookups stay O(1) — even after the dict resizes underneath it.

Python’s collections.OrderedDict remembers insertion order. You might assume it lives in Lib/collections, but it doesn’t: it’s a built-in type written in C, in Objects/odictobject.c, and it subclasses dict. Let’s trace how it works in CPython 3.15.

Why OrderedDict still exists (dicts are already ordered)

Since 3.7, a plain dict already preserves insertion order. So why bother with OrderedDict?

Because it offers operations a plain dict simply doesn’t have: move_to_end(), FIFO/LIFO popitem(last=...), and order-sensitive equality. It also gives you a stable, explicitly ordered contract that subclasses and tooling can lean on.

Implementing it as a C type that inherits from dict keeps it fast and interoperable with the dict C-API surface wherever possible.

Bottom line: plain dicts handle the common case, but OrderedDict hands you explicit ordering guarantees and order-specific tools a plain dict won’t.

The struct: a dict plus a linked list

PyODictObject embeds a PyDictObject as its first member, so an OrderedDict is a dict at the C level — pass it anywhere that wants a PyDictObject *. On top of that it carries a doubly-linked list of keys plus a “fast node” lookup table:

 c
 
 
// Objects/odictobject.c
struct _odictobject {
 PyDictObject od_dict; /* the underlying dict */
 _ODictNode *od_first; /* first node in the linked list, if any */
 _ODictNode *od_last; /* last node in the linked list, if any */
 _ODictNode **od_fast_nodes; /* hash table that mirrors the dict table */
 Py_ssize_t od_fast_nodes_size;
 void *od_resize_sentinel; /* changes if odict should be resized */
 size_t od_state; /* incremented whenever the LL changes */
 PyObject *od_inst_dict; /* OrderedDict().__dict__ */
 PyObject *od_weakreflist; /* holds weakrefs to the odict */
};

The od_dict member is what makes PyDict_* calls work on an OrderedDict instance. The linked list and fast-node table are the order-preserving layer bolted on top.

Think of it this way: the dict does the heavy lifting — hashing, storage, lookups — and the linked list is only there to remember which key came first.

Nodes store keys, not values

Each node holds a key (owned reference), its hash, and prev/next pointers:

 c
 
 
// Objects/odictobject.c
struct _odictnode {
 PyObject *key;
 Py_hash_t hash;
 _ODictNode *next;
 _ODictNode *prev;
};

#define _odictnode_KEY(node) (node->key)
#define _odictnode_HASH(node) (node->hash)
/* borrowed reference */
#define _odictnode_VALUE(node, od) \
 PyODict_GetItemWithError((PyObject *)od, _odictnode_KEY(node))
#define _odictnode_PREV(node) (node->prev)
#define _odictnode_NEXT(node) (node->next)

#define _odict_FIRST(od) (_PyODictObject_CAST(od)->od_first)
#define _odict_LAST(od) (_PyODictObject_CAST(od)->od_last)
#define _odict_EMPTY(od) (_odict_FIRST(od) == NULL)
#define _odict_FOREACH(od, node) \
 for (node = _odict_FIRST(od); node != NULL; node = _odictnode_NEXT(node))

Notice that a node stores only the key. Values still live in the underlying dict, which keeps the linked list cheap and avoids duplicating value storage. _odictnode_VALUE is just a PyODict_GetItemWithError call away.

O(1) node lookup with od_fast_nodes

A naive linked list on top of dict would make __delitem__ O(n): you have to find the node for a key before you can unlink it. To keep it O(1), CPython mirrors dict’s hash-table layout with an array of node pointers, od_fast_nodes, indexed exactly the way dict’s dk_indices/entries are.

The index for a key comes from running dict’s own lookup and then using pointer arithmetic on the returned entry index:

 c
 
 
// Objects/odictobject.c
static Py_ssize_t
_odict_get_index_raw(PyODictObject *od, PyObject *key, Py_hash_t hash)
{
 PyObject *value = NULL;
 PyDictKeysObject *keys = ((PyDictObject *)od)->ma_keys;
 Py_ssize_t ix;
 ix = _Py_dict_lookup((PyDictObject *)od, key, hash, &value);
 Py_XDECREF(value);
 if (ix == DKIX_EMPTY) {
 return keys->dk_nentries; /* index of new entry */
 }
 if (ix < 0)
 return -1;
 /* We use pointer arithmetic to get the entry's index into the table. */
 return ix;
}

_odict_find_node then just indexes into od_fast_nodes:

 c
 
 
// Objects/odictobject.c
static _ODictNode *
_odict_find_node(PyODictObject *od, PyObject *key)
{
 if (_odict_EMPTY(od)) return NULL;
 Py_hash_t hash = PyObject_Hash(key);
 if (hash == -1) return NULL;
 Py_ssize_t index = _odict_get_index(od, key, hash);
 if (index < 0) return NULL;
 assert(od->od_fast_nodes != NULL);
 return od->od_fast_nodes[index];
}

Here’s the thing: without that mirror array, deleting a key would mean walking the entire list to find its node. With it, you jump straight to the node.

Keeping the mirror in sync when dict resizes

Dict resizes invalidate the correspondence between od_fast_nodes and ma_keys. Instead of updating eagerly, CPython defers: od_resize_sentinel stores the current ma_keys pointer, and _odict_get_index checks whether the sentinel still matches (and whether the table size still matches 1 << dk_log2_size). If not, it rebuilds the whole fast-node table:

 c
 
 
// Objects/odictobject.c
static int
_odict_resize(PyODictObject *od)
{
 Py_ssize_t size, i;
 _ODictNode **fast_nodes, *node;

 size = ONE << (((PyDictObject *)od)->ma_keys->dk_log2_size);
 fast_nodes = PyMem_NEW(_ODictNode *, size);
 if (fast_nodes == NULL) { PyErr_NoMemory(); return -1; }
 for (i = 0; i < size; i++)
 fast_nodes[i] = NULL;

 /* Copy the current nodes into the table. */
 _odict_FOREACH(od, node) {
 i = _odict_get_index_raw(od, _odictnode_KEY(node),
 _odictnode_HASH(node));
 if (i < 0) { PyMem_Free(fast_nodes); return -1; }
 fast_nodes[i] = node;
 }

 PyMem_Free(od->od_fast_nodes);
 od->od_fast_nodes = fast_nodes;
 od->od_fast_nodes_size = size;
 od->od_resize_sentinel = ((PyDictObject *)od)->ma_keys;
 return 0;
}

This lazy rebuild is what lets node lookup stay O(1) amortized without hooking into every dict resize.

Gotcha: the mirror doesn’t update on every dict operation. It’s rebuilt lazily, which means od_fast_nodes can be stale until the next lookup forces a rebuild.

Adding and removing nodes

Insertion appends a node at the tail of the list and records it in the fast table:

 c
 
 
// Objects/odictobject.c
static int
_odict_add_new_node(PyODictObject *od, PyObject *key, Py_hash_t hash)
{
 Py_INCREF(key);
 Py_ssize_t i = _odict_get_index(od, key, hash);
 if (i < 0) { ... Py_DECREF(key); return -1; }
 assert(od->od_fast_nodes != NULL);
 if (od->od_fast_nodes[i] != NULL) {
 /* We already have a node for the key so there's no need to add one. */
 Py_DECREF(key);
 return 0;
 }

 _ODictNode *node = (_ODictNode *)PyMem_Malloc(sizeof(_ODictNode));
 if (node == NULL) { Py_DECREF(key); PyErr_NoMemory(); return -1; }

 _odictnode_KEY(node) = key;
 _odictnode_HASH(node) = hash;
 _odict_add_tail(od, node);
 od->od_fast_nodes[i] = node;
 return 0;
}

Removal is the delicate part. Because of hash collisions, dict’s probe result for a key can change after a different key is deleted — a slot becomes a tombstone and the probe chain shifts. So _odict_clear_node clears the fast-node entry before PyDict_DelItem runs:

 c
 
 
// Objects/odictobject.c
static int
_odict_clear_node(PyODictObject *od, _ODictNode *node, PyObject *key,
 Py_hash_t hash)
{
 Py_ssize_t i = _odict_get_index(od, key, hash);
 if (i < 0) return PyErr_Occurred() ? -1 : 0;

 assert(od->od_fast_nodes != NULL);
 if (node == NULL)
 node = od->od_fast_nodes[i];
 assert(node == od->od_fast_nodes[i]);
 if (node == NULL) return 0;

 // Now clear the node.
 od->od_fast_nodes[i] = NULL;
 _odict_remove_node(od, node);
 _odictnode_DEALLOC(node); /* Py_DECREF(key); PyMem_Free(node) */
 return 0;
}

The public entry points combine this with the underlying dict mutation:

 c
 
 
// Objects/odictobject.c
int
PyODict_DelItem_LockHeld(PyObject *od, PyObject *key)
{
 Py_hash_t hash = PyObject_Hash(key);
 if (hash == -1) return -1;
 int res = _odict_clear_node(_PyODictObject_CAST(od), NULL, key, hash);
 if (res < 0) return -1;
 return _PyDict_DelItem_KnownHash_LockHeld(od, key, hash);
}

static int
_PyODict_SetItem_KnownHash_LockHeld(PyObject *od, PyObject *key,
 PyObject *value, Py_hash_t hash)
{
 int res = _PyDict_SetItem_KnownHash_LockHeld((PyDictObject *)od, key, value, hash);
 if (res == 0) {
 res = _odict_add_new_node(_PyODictObject_CAST(od), key, hash);
 if (res < 0) {
 /* Revert setting the value on the dict */
 PyObject *exc = PyErr_GetRaisedException();
 (void) _PyDict_DelItem_KnownHash(od, key, hash);
 _PyErr_ChainExceptions1(exc);
 }
 }
 return res;
}

Notice the rollback in _PyODict_SetItem_KnownHash_LockHeld: if node creation fails, the dict insertion is undone and the original exception is re-raised.

If you’ve ever been burned by half-applied mutations, this is the pattern to copy: set the dict value first, add the node second, and undo the dict change if the node can’t be created.

od_state: the mutation counter that keeps iterators honest

Every linked-list change bumps od_state. That’s how iterators and internal loops detect concurrent mutation. The iterator snapshots od_state and the size it started with, then compares:

 c
 
 
// Objects/odictobject.c
static PyObject *
odictiter_nextkey_lock_held(odictiterobject *di)
{
 ...
 if (di->di_odict->od_state != di->di_state) {
 PyErr_SetString(PyExc_RuntimeError,
 "OrderedDict mutated during iteration");
 goto done;
 }
 if (di->di_size != PyODict_SIZE(di->di_odict)) {
 PyErr_SetString(PyExc_RuntimeError,
 "OrderedDict changed size during iteration");
 ...
 }
 ...
}

Internal loops such as OrderedDict_copy_impl do the same check after each step, raising RuntimeError("OrderedDict mutated during iteration") if the state changed. __eq__’s _odict_keys_equal likewise snapshots both states and checks them on every key comparison.

The real win: you get the same “dictionary changed size during iteration” safety net plain dicts give you — but for the linked-list order too.

Iterators track keys, not node pointers

A subtlety the source calls out: holding a node pointer across code that might delete it can dangle. So iterators store the current key and re-look-up the node on each step:

 c
 
 
// Objects/odictobject.c
/* In order to protect against modifications during iteration, we track
 * the current key instead of the current node. */
node = _odict_find_node(di->di_odict, di->di_current);
if (node == NULL) { ... /* Must have been deleted. */ ... }
key = di->di_current;
node = reversed ? _odictnode_PREV(node) : _odictnode_NEXT(node);
if (node == NULL) {
 di->di_current = NULL;
} else {
 di->di_current = Py_NewRef(_odictnode_KEY(node));
}

This is the “keep the key in the variable” solution straight from the file’s design notes.

Gotcha: never cache a node pointer across iteration steps. Keep the key, re-look-up the node.

move_to_end: reorder without touching values

move_to_end unlinks the node and re-appends it (at head or tail), but only when it isn’t already there:

 c
 
 
// Objects/odictobject.c
static PyObject *
OrderedDict_move_to_end_impl(PyODictObject *self, PyObject *key, int last)
{
 _ODictNode *node;
 if (_odict_EMPTY(self)) { PyErr_SetObject(PyExc_KeyError, key); return NULL; }
 node = last ? _odict_LAST(self) : _odict_FIRST(self);
 if (key != _odictnode_KEY(node)) {
 node = _odict_find_node(self, key);
 if (node == NULL) { ... return NULL; }
 if (last) {
 if (node != _odict_LAST(self)) {
 _odict_remove_node(self, node);
 _odict_add_tail(self, node);
 }
 } else {
 if (node != _odict_FIRST(self)) {
 _odict_remove_node(self, node);
 _odict_add_head(self, node);
 }
 }
 }
 Py_RETURN_NONE;
}

Because _odict_remove_node/_odict_add_tail only relink pointers and bump od_state — they never touch the dict — the value side is completely unaffected.

Think of it as moving a bookmark: the page (value) stays put, you’re just changing where it sits in the list.

popitem: FIFO/LIFO pops, node first

popitem(last) picks the first or last node, then routes through _odict_popkey_hash, which clears the node first (to avoid a dict resize/reentrancy hazard) and then pops the value from the dict:

 c
 
 
// Objects/odictobject.c
static PyObject *
OrderedDict_popitem_impl(PyODictObject *self, int last)
{
 PyObject *key, *value;
 _ODictNode *node;

 if (_odict_EMPTY(self)) {
 PyErr_SetString(PyExc_KeyError, "dictionary is empty");
 return NULL;
 }

 node = last ? _odict_LAST(self) : _odict_FIRST(self);
 key = Py_NewRef(_odictnode_KEY(node));
 value = _odict_popkey_hash((PyObject *)self, key, NULL,
 _odictnode_HASH(node));
 if (value == NULL) { Py_DECREF(key); return NULL; }
 return _PyTuple_FromPairSteal(key, value);
}

 c
 
 
// Objects/odictobject.c (_odict_popkey_hash)
_ODictNode *node = _odict_find_node_hash(...);
if (node != NULL) {
 int res = _odict_clear_node(..., node, key, hash); /* unlink + free node */
 if (res < 0) goto done;
 if (_PyDict_Pop_KnownHash(...) == 0) {
 value = Py_NewRef(failobj);
 }
}

The ordering matters: clear the node before touching the dict, so a resize can’t invalidate the pointer you’re holding.

Equality: order-sensitive, but cheap to short-circuit

odict_richcompare first delegates to dict’s richcompare — so unequal key sets or values short-circuit — and only if both sides are OrderedDicts and the dict comparison didn’t already decide does it compare key order via _odict_keys_equal:

 c
 
 
// Objects/odictobject.c
static PyObject *
odict_richcompare_lock_held(PyObject *v, PyObject *w, int op)
{
 if (!PyODict_Check(v) || !PyAnyDict_Check(w))
 Py_RETURN_NOTIMPLEMENTED;

 if (op == Py_EQ || op == Py_NE) {
 PyObject *cmp = PyDict_Type.tp_richcompare(v, w, op);
 if (cmp == NULL) return NULL;
 if (!PyODict_Check(w)) return cmp;
 if (op == Py_EQ && cmp == Py_False) return cmp;
 if (op == Py_NE && cmp == Py_True) return cmp;
 Py_DECREF(cmp);

 int eq = _odict_keys_equal(_PyODictObject_CAST(v),
 _PyODictObject_CAST(w));
 if (eq < 0) return NULL;
 PyObject *res = (eq == (op == Py_EQ)) ? Py_True : Py_False;
 return Py_NewRef(res);
 } else {
 Py_RETURN_NOTIMPLEMENTED;
 }
}

Bottom line: two OrderedDicts with the same keys and values are not equal if the insertion order differs. But when the dict-level comparison already fails, you never pay for the order check at all.

Views and reversed()

keys(), values(), and items() return views whose type objects (PyODictKeys_Type and friends) base on the dict view types but override tp_iter to produce an odict_iterator that walks the linked list instead of dk_entries. The iterator supports __reversed__ by starting at _odict_LAST and following prev pointers. The views themselves also expose __reversed__.

Deallocation and GC

odict_dealloc untracks from the GC, clears the instance dict and weakrefs, frees all the nodes, and then chains to dict’s dealloc:

 c
 
 
// Objects/odictobject.c
static void
odict_dealloc(PyObject *op)
{
 PyODictObject *self = _PyODictObject_CAST(op);
 PyObject_GC_UnTrack(self);

 Py_XDECREF(self->od_inst_dict);
 FT_CLEAR_WEAKREFS(op, self->od_weakreflist);

 _odict_clear_nodes(self); /* frees od_fast_nodes + all nodes */
 PyDict_Type.tp_dealloc((PyObject *)self);
}

_odict_clear_nodes frees the fast-node array and walks the list deallocating every node (decrefing its key). odict_traverse visits od_inst_dict and every node’s key so the GC can find cycles.

sizeof

Size accounting adds the fast-node array and the node structs on top of dict’s size:

 c
 
 
// Objects/odictobject.c
static Py_ssize_t
OrderedDict___sizeof___impl(PyODictObject *self)
{
 Py_ssize_t res = _PyDict_SizeOf_LockHeld((PyDictObject *)self);
 res += sizeof(_ODictNode *) * self->od_fast_nodes_size; /* od_fast_nodes */
 if (!_odict_EMPTY(self)) {
 res += sizeof(_ODictNode) * PyODict_SIZE(self); /* linked-list */
 }
 return res;
}

Type registration and the public C API

PyODict_Type has tp_base = &PyDict_Type, tp_name = "collections.OrderedDict", and overrides tp_dealloc, tp_repr, mp_ass_subscript, tp_traverse, tp_clear, tp_richcompare, tp_iter, tp_init, and others. The public C API is small: PyODict_New, PyODict_SetItem/..._LockHeld, PyODict_DelItem/..._LockHeld, plus the PyODict_Check/PyODict_CheckExact macros.

Wrapping up

collections.OrderedDict lives in Objects/odictobject.c as PyODict_Type, a subclass of PyDict_Type. A PyODictObject is a PyDictObject plus a doubly-linked list of _ODictNode (key + hash + prev/next) and an od_fast_nodes array of node pointers mirroring dict’s hash table for O(1) lookup.

The details worth remembering: the mirror is rebuilt lazily when the dict resizes (detected via od_resize_sentinel), keeping node lookup O(1) amortized. Removals clear the fast-node slot before mutating the dict, so collisions can’t corrupt the mapping. od_state plus size snapshots let iterators and internal loops detect mutation and raise RuntimeError. Iterators track the current key rather than a node pointer to avoid dangling pointers. And move_to_end/popitem operate purely on the linked list — values never move.

If you need explicit, order-sensitive semantics beyond what a plain dict offers, OrderedDict delivers them in C, with the same performance discipline as the dict it inherits from.

CPython's object Implementation in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Every Python value is a PyObject *. The type that defines what that pointer points at is object itself, implemented in CPython as PyBaseObject_Type in Objects/typeobject.c. Its instances are the smallest possible Python objects: just a reference count and a type pointer. This article traces that representation and the default behavior that every other type inherits from it.

The PyObject struct

The layout is defined in Include/object.h. In a normal (non-free-threaded) build with 64-bit pointers, it is:

 c
 
 
// Include/object.h
struct _object {
 _Py_ANONYMOUS union {
#if SIZEOF_VOID_P > 4
 int64_t ob_refcnt_full;
 struct {
# if PY_BIG_ENDIAN
 uint16_t ob_flags;
 uint16_t ob_overflow;
 uint32_t ob_refcnt;
# else
 uint32_t ob_refcnt;
 uint16_t ob_overflow;
 uint16_t ob_flags;
# endif
 };
#else
 Py_ssize_t ob_refcnt;
#endif
 _Py_ALIGNED_DEF(_PyObject_MIN_ALIGNMENT, char) _aligner;
 };
 PyTypeObject *ob_type;
};

Two fields matter. ob_refcnt is the reference count; ob_type is the pointer to the object’s type. PyObject_HEAD expands to PyObject ob_base;, so every concrete struct (PyLongObject, PyFloatObject, …) begins with exactly these fields, which is what makes the cast between PyObject * and a more specific pointer valid.

The reference count is manipulated by Py_INCREF/Py_DECREF. Py_DECREF calls the type’s tp_dealloc when the count reaches zero. New objects get their count set to one by _Py_NewReference, which PyObject_Init (and the inline _PyObject_Init) calls after storing the type pointer:

 c
 
 
// Include/internal/pycore_object.h
static inline void
_PyObject_Init(PyObject *op, PyTypeObject *typeobj)
{
 assert(op != NULL);
 Py_SET_TYPE(op, typeobj); // op->ob_type = typeobj
 _Py_INCREF_TYPE(typeobj);
 _Py_NewReference(op); // sets ob_refcnt to 1
}

Immortal objects (such as None, True, False, and static types) use a sentinel refcount (_Py_IMMORTAL_INITIAL_REFCNT) so Py_INCREF/Py_DECREF become no-ops for them.

Identity is just pointer equality:

 c
 
 
// Include/object.h
#define Py_Is(x, y) ((x) == (y))

and id(obj) in Python returns that pointer value (cast to an integer).

ob_size for variable-size objects

Objects whose size is determined at allocation time carry an extra ob_size field. These are PyVarObjects:

 c
 
 
// Include/object.h
struct PyVarObject {
 PyObject ob_base;
 Py_ssize_t ob_size; // Number of items in variable part
};
#define PyObject_VAR_HEAD PyVarObject ob_base;

ob_size is an element count, not a byte count. Types such as list, tuple, bytes and str start with PyObject_VAR_HEAD and allocate tp_basicsize + nitems * tp_itemsize bytes via _PyObject_NewVar:

 c
 
 
// Objects/object.c
PyVarObject *
_PyObject_NewVar(PyTypeObject *tp, Py_ssize_t nitems)
{
 PyVarObject *op;
 const size_t size = _PyObject_VAR_SIZE(tp, nitems);
 op = (PyVarObject *) PyObject_Malloc(size);
 if (op == NULL) {
 return (PyVarObject *) PyErr_NoMemory();
 }
 _PyObject_InitVar(op, tp, nitems); // sets ob_size = nitems
 return op;
}

_PyObject_InitVar stores the count with Py_SET_SIZE, and Py_SIZE(ob) reads it back (with an assertion that the object is not an int/bool, whose ob_size is overloaded for the sign/magnitude representation).

PyBaseObject_Type

The base type table is in Objects/typeobject.c:

 c
 
 
// Objects/typeobject.c
PyTypeObject PyBaseObject_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "object",
 sizeof(PyObject), /* tp_basicsize */
 0, /* tp_itemsize */
 object_dealloc, /* tp_dealloc */
 0, /* tp_vectorcall_offset */
 0, /* tp_getattr */
 0, /* tp_setattr */
 0, /* tp_as_async */
 object_repr, /* tp_repr */
 0, /* tp_as_number */
 0, /* tp_as_sequence */
 0, /* tp_as_mapping */
 PyObject_GenericHash, /* tp_hash */
 0, /* tp_call */
 object_str, /* tp_str */
 PyObject_GenericGetAttr, /* tp_getattro */
 PyObject_GenericSetAttr, /* tp_setattro */
 0, /* tp_as_buffer */
 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags */
 object_doc,
 0, /* tp_traverse */
 0, /* tp_clear */
 object_richcompare, /* tp_richcompare */
 ...
 object_methods, /* tp_methods */
 0, /* tp_members */
 object_getsets, /* tp_getset */
 0, /* tp_base */
 ...
 0, /* tp_dictoffset */
 object_init, /* tp_init */
 PyType_GenericAlloc, /* tp_alloc */
 object_new, /* tp_new */
 PyObject_Free, /* tp_free */
};

Key facts: tp_basicsize is sizeof(PyObject) (16 bytes on 64-bit), tp_itemsize is 0 (fixed size), there is no tp_base (it is the root), and Py_TPFLAGS_BASETYPE allows subclassing. Because tp_dictoffset is zero, a plain object() instance has no instance dictionary and cannot be given attributes.

Default behaviors

Allocation: `object_new`

 c
 
 
// Objects/typeobject.c
static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
 if (excess_args(args, kwds)) {
 if (type->tp_new != object_new) {
 /* subclass overrode __new__: complain here */
 ...
 }
 if (type->tp_init == object_init) {
 PyErr_Format(PyExc_TypeError, "%.200s() takes no arguments",
 type->tp_name);
 return NULL;
 }
 }
 if (type->tp_flags & Py_TPFLAGS_IS_ABSTRACT) {
 /* error: can't instantiate abstract class */
 ...
 }
 PyObject *obj = type->tp_alloc(type, 0); // PyType_GenericAlloc
 if (obj == NULL) {
 return NULL;
 }
 return obj;
}

So object() accepts no arguments; a subclass that overrides neither __new__ nor __init__ gets “takes no arguments”. Abstract classes (those with unimplemented abstract methods) are rejected here.

Initialization: `object_init`

 c
 
 
// Objects/typeobject.c
static int
object_init(PyObject *self, PyObject *args, PyObject *kwds)
{
 PyTypeObject *type = Py_TYPE(self);
 if (excess_args(args, kwds)) {
 if (type->tp_init != object_init) {
 PyErr_SetString(PyExc_TypeError,
 "object.__init__() takes exactly one argument ...");
 return -1;
 }
 if (type->tp_new == object_new) {
 /* __new__ not overridden: complain */
 ...
 }
 }
 return 0;
}

The default __init__ is a no-op; it only enforces the argument rules.

Deallocation: `object_dealloc`

 c
 
 
// Objects/typeobject.c
static void
object_dealloc(PyObject *self)
{
 Py_TYPE(self)->tp_free(self); // PyObject_Free for base object
}

repr: `object_repr`

 c
 
 
// Objects/typeobject.c
static PyObject *
object_repr(PyObject *self)
{
 PyTypeObject *type = Py_TYPE(self);
 PyObject *mod = type_module(type);
 ...
 PyObject *name = type_qualname((PyObject *)type, NULL);
 if (mod != NULL && !_PyUnicode_Equal(mod, &_Py_ID(builtins)))
 rtn = PyUnicode_FromFormat("<%U.%U object at %p>", mod, name, self);
 else
 rtn = PyUnicode_FromFormat("<%s object at %p>", type->tp_name, self);
 ...
 return rtn;
}

A plain instance prints as <object object at 0x...>; types outside builtins are qualified with their module name.

str: `object_str`

 c
 
 
// Objects/typeobject.c
static PyObject *
object_str(PyObject *self)
{
 unaryfunc f = Py_TYPE(self)->tp_repr;
 if (f == NULL)
 f = object_repr;
 return f(self);
}

Unless a subclass overrides __repr__/__str__, str(obj) falls back to repr(obj).

Hash: `PyObject_GenericHash`

 c
 
 
// Python/pyhash.c
Py_hash_t
PyObject_GenericHash(PyObject *obj)
{
 return Py_HashPointer(obj);
}

Py_hash_t
Py_HashPointer(const void *ptr)
{
 Py_hash_t hash = _Py_HashPointerRaw(ptr);
 if (hash == -1) {
 hash = -2;
 }
 return hash;
}

The default hash is derived from the object’s address, so two distinct instances have different hashes and object is hashable. Subclasses that define __eq__ typically set tp_hash = PyObject_HashNotImplemented (or override) to stay consistent.

Rich comparison: `object_richcompare`

 c
 
 
// Objects/typeobject.c
static PyObject *
object_richcompare(PyObject *self, PyObject *other, int op)
{
 switch (op) {
 case Py_EQ:
 res = Py_NewRef((self == other) ? Py_True : Py_NotImplemented);
 break;
 case Py_NE:
 if (Py_TYPE(self)->tp_richcompare == NULL) {
 res = Py_NewRef(Py_NotImplemented);
 break;
 }
 res = (*Py_TYPE(self)->tp_richcompare)(self, other, Py_EQ);
 if (res != NULL && res != Py_NotImplemented) {
 int ok = PyObject_IsTrue(res);
 Py_DECREF(res);
 res = ok ? Py_NewRef(Py_False) : Py_NewRef(Py_True);
 }
 break;
 default:
 res = Py_NewRef(Py_NotImplemented);
 break;
 }
 return res;
}

__eq__ returns True only for the exact same object, otherwise NotImplemented (letting the other operand or the default identity rule decide). __ne__ delegates to __eq__ and inverts. Ordering operators return NotImplemented, so <, >, etc. raise TypeError by default.

Attribute access: `PyObject_GenericGetAttr` / `PyObject_GenericSetAttr`

 c
 
 
// Objects/object.c
PyObject *
PyObject_GenericGetAttr(PyObject *obj, PyObject *name)
{
 return _PyObject_GenericGetAttrWithDict(obj, name, NULL, 0);
}

The generic getter looks up descriptors on the type, then the instance __dict__ (if tp_dictoffset is set). Because object has no tp_dictoffset, instances of object itself have no attributes at all; obj.x = 1 raises AttributeError. Subclasses that need attributes get a dict offset (or managed dicts) from the type machinery.

The one attribute always available is __class__, provided by the object_getsets table:

 c
 
 
// Objects/typeobject.c
static PyGetSetDef object_getsets[] = {
 {"__class__", object_get_class, object_set_class,
 PyDoc_STR("the object's class")},
 {0}
};

`sizeof`

 c
 
 
// Objects/typeobject.c
static PyObject *
object___sizeof___impl(PyObject *self)
{
 Py_ssize_t res, isize;
 res = 0;
 isize = Py_TYPE(self)->tp_itemsize;
 if (isize > 0) {
 res = _PyVarObject_CAST(self)->ob_size * isize;
 }
 res += Py_TYPE(self)->tp_basicsize;
 return PyLong_FromSsize_t(res);
}

So sys.getsizeof(obj) reports tp_basicsize + ob_size * tp_itemsize, i.e. the allocated size of the instance.

Summary

object is the root of the hierarchy: a 16-byte (on 64-bit) struct holding only ob_refcnt and ob_type, with ob_size added for variable-size subtypes. PyBaseObject_Type fills in sensible defaults – identity-based equality and hashing, a pointer-based repr, generic attribute access, no-op init, abstract-class checking in __new__, and PyObject_Free as the deallocator – that every other type either inherits or overrides. All of Python’s object model rests on these two fields and this one type table.

CPython's NotImplemented Implementation in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s NotImplemented is not an exception and not a function result that means “I gave up.” It is a real singleton object of type NotImplementedType, returned from operator implementations to tell the interpreter: “this operand combination is not supported by me – try the other operand, or fall back.” In CPython that is implemented in Objects/object.c as a single static PyObject struct, _Py_NotImplementedStruct, whose type is _PyNotImplemented_Type. This article traces that implementation through the 3.15 source.

The type object

NotImplementedType is a plain, non-subclassable-in-practice type whose slot table is almost entirely zero. The few slots that are set define its identity and its role as a sentinel:

 c
 
 
// Objects/object.c
PyTypeObject _PyNotImplemented_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "NotImplementedType",
 0, /* tp_basicsize: sizeof(PyObject) */
 0, /* tp_itemsize */
 notimplemented_dealloc, /* tp_dealloc */ /*never called*/
 0, /* tp_vectorcall_offset */
 0, /* tp_getattr */
 0, /* tp_setattr */
 0, /* tp_as_async */
 NotImplemented_repr, /* tp_repr */
 &notimplemented_as_number, /* tp_as_number */
 0, /* tp_as_sequence */
 0, /* tp_as_mapping */
 0, /* tp_hash */
 0, /* tp_call */
 0, /* tp_str */
 ...
 Py_TPFLAGS_DEFAULT, /* tp_flags */
 notimplemented_doc, /* tp_doc */
 ...
 0, /* tp_base */ /* filled in by PyType_Ready -> object */
 ...
 notimplemented_methods, /* tp_methods */
 ...
 0, /* tp_init */
 0, /* tp_alloc */
 notimplemented_new, /* tp_new */
};

tp_basicsize is 0, which PyType_Ready turns into sizeof(PyObject): the singleton carries no payload at all, it is just a head. tp_base is 0, so PyType_Ready makes object the base, which is why NotImplemented inherits the default identity equality and identity hash from object.

The type is exported to Python as types.NotImplementedType:

 c
 
 
// Modules/_typesmodule.c
EXPORT_STATIC_TYPE("NotImplementedType", _PyNotImplemented_Type);

The singleton

There is exactly one instance, statically allocated at compile time:

 c
 
 
// Objects/object.c
PyObject _Py_NotImplementedStruct = _PyObject_HEAD_INIT(&_PyNotImplemented_Type);

_PyObject_HEAD_INIT initializes the refcount to an immortal value, sets the static-allocation flag, and points ob_type at _PyNotImplemented_Type. No runtime allocation ever happens.

The public macro resolves to that struct (or, in the limited API 3.13+, to a borrowed reference from the interpreter’s constant table):

 c
 
 
// Include/object.h
PyAPI_DATA(PyObject) _Py_NotImplementedStruct; /* Don't use this directly */

#if defined(Py_LIMITED_API) && Py_LIMITED_API+0 >= 0x030D0000
# define Py_NotImplemented Py_GetConstantBorrowed(Py_CONSTANT_NOT_IMPLEMENTED)
#else
# define Py_NotImplemented (&_Py_NotImplementedStruct)
#endif

Py_RETURN_NOTIMPLEMENTED is the convenience macro for returning it from a C function. In the limited API older than 3.12 it returns a new reference; in 3.12+ it returns the immortal singleton directly:

 c
 
 
// Include/object.h
#if defined(Py_LIMITED_API) && Py_LIMITED_API+0 < 0x030c0000
# define Py_RETURN_NOTIMPLEMENTED return Py_NewRef(Py_NotImplemented)
#else
# define Py_RETURN_NOTIMPLEMENTED return Py_NotImplemented
#endif

Because the object is in the interpreter’s constant table, it is immortal (PEP 683) like None, True, and False:

 c
 
 
// Objects/object.c
static PyObject* constants[] = {
 &_Py_NoneStruct, // Py_CONSTANT_NONE
 (PyObject*)(&_Py_FalseStruct), // Py_CONSTANT_FALSE
 (PyObject*)(&_Py_TrueStruct), // Py_CONSTANT_TRUE
 &_Py_EllipsisObject, // Py_CONSTANT_ELLIPSIS
 &_Py_NotImplementedStruct, // Py_CONSTANT_NOT_IMPLEMENTED
 ...
};

The dealloc guard

The singleton’s refcount should never reach zero, but CPython defends against accidental decref-to-zero by re-immortalizing instead of freeing:

 c
 
 
// Objects/object.c
static void
notimplemented_dealloc(PyObject *notimplemented)
{
 /* This should never get called, but we also don't want to SEGV if
 * we accidentally decref NotImplemented out of existence. Instead,
 * since NotImplemented is an immortal object, re-set the reference count. */
 _Py_SetImmortal(notimplemented);
}

This mirrors bool_dealloc and the None/Ellipsis guards: a statically allocated immortal object has no real deallocator.

Boolean context is an error

NotImplemented is deliberately unusable as a truth value. Its tp_as_number only fills nb_bool, and that slot raises TypeError:

 c
 
 
// Objects/object.c
static int
notimplemented_bool(PyObject *v)
{
 PyErr_SetString(PyExc_TypeError,
 "NotImplemented should not be used in a boolean context");
 return -1;
}

static PyNumberMethods notimplemented_as_number = {
 .nb_bool = notimplemented_bool,
};

So if NotImplemented:, not NotImplemented, and bool(NotImplemented) all raise TypeError: NotImplemented should not be used in a boolean context. This prevents a common bug where a binary operator returns NotImplemented and the caller forgets to check for it, treating it as a truthy result.

repr and pickling

repr(NotImplemented) is the string "NotImplemented" (a fresh str, not a cached one), and __reduce__ returns the same string so unpickling resolves to the singleton by name:

 c
 
 
// Objects/object.c
static PyObject *
NotImplemented_repr(PyObject *op)
{
 return PyUnicode_FromString("NotImplemented");
}

static PyObject *
NotImplemented_reduce(PyObject *op, PyObject *Py_UNUSED(ignored))
{
 return PyUnicode_FromString("NotImplemented");
}

static PyMethodDef notimplemented_methods[] = {
 {"__reduce__", NotImplemented_reduce, METH_NOARGS, NULL},
 {NULL, NULL}
};

Construction: no arguments

NotImplementedType() returns the singleton; passing arguments is an error:

 c
 
 
// Objects/object.c
static PyObject *
notimplemented_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
{
 if (PyTuple_GET_SIZE(args) || (kwargs && PyDict_GET_SIZE(kwargs))) {
 PyErr_SetString(PyExc_TypeError, "NotImplementedType takes no arguments");
 return NULL;
 }
 Py_RETURN_NOTIMPLEMENTED;
}

Note that type is ignored: there is no way to create a second instance, just like True/False for bool.

How binary operators consume it

The whole point of the sentinel is the protocol in Objects/abstract.c. A binary slot (e.g. nb_add) may return Py_NotImplemented to say “I don’t know how to do this.” binary_op1 then tries the reflected/other operand’s slot, and if everything fails, binary_op turns the sentinel into a TypeError:

 c
 
 
// Objects/abstract.c
static PyObject *
binary_op1(PyObject *v, PyObject *w, const int op_slot ...)
{
 binaryfunc slotv = ...; /* v's slot, if any */
 binaryfunc slotw = ...; /* w's slot, if types differ */

 if (slotv) {
 PyObject *x;
 if (slotw && PyType_IsSubtype(Py_TYPE(w), Py_TYPE(v))) {
 x = slotw(v, w); /* try w first when w is a subclass */
 if (x != Py_NotImplemented)
 return x;
 Py_DECREF(x); /* can't do it */
 slotw = NULL;
 }
 x = slotv(v, w);
 if (x != Py_NotImplemented) {
 return x;
 }
 Py_DECREF(x); /* can't do it */
 }
 if (slotw) {
 PyObject *x = slotw(v, w);
 if (x != Py_NotImplemented) {
 return x;
 }
 Py_DECREF(x); /* can't do it */
 }
 Py_RETURN_NOTIMPLEMENTED; /* nobody could do it */
}

static PyObject *
binary_op(PyObject *v, PyObject *w, const int op_slot, const char *op_name)
{
 PyObject *result = BINARY_OP1(v, w, op_slot, op_name);
 if (result == Py_NotImplemented) {
 Py_DECREF(result);
 return binop_type_error(v, w, op_name); /* TypeError */
 }
 return result;
}

The same pattern appears in ternary_op (for pow with a modulus) and in do_richcompare for ordering/comparison operators:

 c
 
 
// Objects/object.c
static PyObject *
do_richcompare(PyThreadState *tstate, PyObject *v, PyObject *w, int op)
{
 ...
 if (!Py_IS_TYPE(v, Py_TYPE(w)) &&
 PyType_IsSubtype(Py_TYPE(w), Py_TYPE(v)) &&
 (f = Py_TYPE(w)->tp_richcompare) != NULL) {
 checked_reverse_op = 1;
 res = (*f)(w, v, _Py_SwappedOp[op]);
 if (res != Py_NotImplemented)
 return res;
 Py_DECREF(res);
 }
 if ((f = Py_TYPE(v)->tp_richcompare) != NULL) {
 res = (*f)(v, w, op);
 if (res != Py_NotImplemented)
 return res;
 Py_DECREF(res);
 }
 if (!checked_reverse_op && (f = Py_TYPE(w)->tp_richcompare) != NULL) {
 res = (*f)(w, v, _Py_SwappedOp[op]);
 if (res != Py_NotImplemented)
 return res;
 Py_DECREF(res);
 }
 /* default: == / != compare by identity; ordering raises TypeError */
 ...
}

The default object_richcompare itself uses NotImplemented as its “I have nothing to say” return value, which is what lets two unrelated objects fall through to identity equality for ==/!= and to a TypeError for ordering:

 c
 
 
// Objects/typeobject.c
static PyObject *
object_richcompare(PyObject *self, PyObject *other, int op)
{
 PyObject *res;
 switch (op) {
 case Py_EQ:
 res = Py_NewRef((self == other) ? Py_True : Py_NotImplemented);
 break;
 case Py_NE:
 ...
 res = Py_NewRef(Py_NotImplemented);
 break;
 default:
 res = Py_NewRef(Py_NotImplemented);
 break;
 }
 return res;
}

The slot-wrapper machinery in typeobject.c (SLOT1BINFULL, etc.) likewise compares returned values against Py_NotImplemented to decide whether to try the reflected dunder (__radd__ and friends) or the other operand.

Memory layout summary

NotImplemented is just a PyObject head: refcount, flags, and type pointer. There is no value, no digit array, no buffer:

text

_Py_NotImplementedStruct
+---------------------------+
| ob_refcnt (immortal) |
+---------------------------+
| ob_flags (static) |
+---------------------------+
| *ob_type (&_PyNotImplemented_Type) |
+---------------------------+

The ABI records its size as 16 bytes on 64-bit builds, matching sizeof(PyObject).

Key takeaways

NotImplemented is a single immortal static PyObject, _Py_NotImplementedStruct, of type _PyNotImplemented_Type (NotImplementedType).
The type is a subclass of object with almost no slots: only tp_repr, tp_as_number (whose nb_bool raises TypeError), tp_methods (__reduce__), and tp_new (which takes no arguments and returns the singleton).
Py_NotImplemented resolves to the struct directly in the full API, or to a borrowed reference from the constant table (Py_CONSTANT_NOT_IMPLEMENTED, index 4) in the limited API 3.13+.
The dealloc guard re-immortalizes instead of freeing.
Binary/ternary operators and rich comparisons treat a returned Py_NotImplemented as “try the other operand”; if nobody can handle it, the sentinel is converted into a TypeError.
bool(NotImplemented) and if NotImplemented: raise TypeError by construction, catching forgotten sentinel checks.

Python on Opensource internals

CPython's Weak Reference Implementation in C (weakref)

The object: PyWeakReference

The per-object weakref list

Construction: get_or_create_weakref

Insertion ordering

Reading a weak reference

Hashing and equality

Clearing: when the target dies

The callback member

Proxy objects

Free-threading

Summary of the data flow

Key takeaways

Further reading

CPython's Union Type Implementation in C (X | Y)

The struct: unionobject

The type: _PyUnion_Type

Deallocation and GC

Equality and hashing

Building a union: unionbuilder

Entry points

X | Y (binary |)

typing.Union[X, Y] and UnionType[X, Y]

Substitution: union.__getitem__

Repr

Attributes and properties

Comparison with typing.Union

Memory layout

Key takeaways

Further reading

CPython's TypeVar, TypeVarTuple, and ParamSpec Implementation in C

Overview

Struct layouts

Lazy evaluation: evaluate_* fields

The NoDefault marker

Construction

Variance and repr

GC and deallocation

ParamSpecArgs / ParamSpecKwargs

| (union) operator

Initialization and exposure

Substitution protocol

TypeAliasType

Key takeaways

Further reading

CPython's types.GenericAlias Implementation in C

The object struct

origin and args

__parameters__ (type variables)

ParamSpec and list-shaped arguments

Substitution: alias[item]

Starred unpacking (*tuple[int])

Attribute access

Hashing and equality

Calling, __mro_entries__, and isinstance

Key takeaways

Further reading

CPython's type Implementation in C (the Metaclass)

The object struct

The real layout: PyHeapTypeObject

tp_dict: the class dictionary

MRO

Type creation

Heap vs static types

Type lookup and the version tag

type.__call__ and instance creation

Slots and inheritance

Subclassing rules

type_new in detail

Managed dicts and __dict__

Garbage collection

Free-threading notes

type itself

Key takeaways

Type watchers and the attribute cache

Further reading

CPython's tuple Implementation in C: Immutable, Inline, and Recycled

The object struct

Memory layout

The object: `PyWeakReference`

Construction: `get_or_create_weakref`

The struct: `unionobject`

The type: `_PyUnion_Type`

Building a union: `unionbuilder`

`X | Y` (binary `|`)

`typing.Union[X, Y]` and `UnionType[X, Y]`

Substitution: `union.getitem`

Comparison with `typing.Union`

Lazy evaluation: `evaluate_*` fields

The `NoDefault` marker

`|` (union) operator

`origin` and `args`

`parameters` (type variables)

Substitution: `alias[item]`

Starred unpacking (`*tuple[int]`)

Calling, `__mro_entries__`, and isinstance

The real layout: `PyHeapTypeObject`

`tp_dict`: the class dictionary

`type.call` and instance creation

`type_new` in detail

Managed dicts and `dict`

`type` itself

The `Interpolation` struct

Constructing an `Interpolation`

The `Template` struct

`Template.new`: normalizing the argument list

`_PyTemplate_Build`: the factory

Iteration: `TemplateIter`

The `superobject` struct

Construction: `super_init_impl`

Zero-argument `super()`

`super` as a descriptor: `super_descr_get`

MRO traversal: `_PySuper_LookupDescr`

Attribute lookup: `super_getattro` and `do_super_lookup`

Field access: attributes via `tp_members`

The type dictionary: sizes and `__match_args__`

Construction from Python: `structseq_new`

repr, reduce, and `replace`

Example: `sys.version_info`

The object: `PySliceObject`

`BUILD_SLICE` in the bytecode

Unpacking a slice: `PySlice_Unpack`

Adjusting indices: `PySlice_AdjustIndices`

`slice.indices(len)`: the Python-facing version

`Ellipsis`