How CPython Implements async generators in C

Jay Hubbert — Tue, 20 Oct 2026 09:00:00 +0000

Python’s async def functions containing yield become async generators: objects that implement __anext__, asend, athrow and aclose. Unlike a regular generator, an async generator is also an awaitable: calling anext(ag) does not run code immediately, it returns an async_generator_asend awaitable that is driven by the event loop. In CPython 3.15 all of this lives in Objects/genobject.c (there is no separate asyncgenobject.c), sharing the same interpreter-frame machinery as generators and coroutines.

The object struct

PyAsyncGenObject is defined through the shared _PyGenObject_HEAD macro, so its layout is identical to a generator’s except for the ag_ prefix:

 c
 
 
// Include/internal/pycore_interpframe_structs.h
struct _PyAsyncGenObject {
 PyObject_HEAD
 PyObject *ag_weakreflist;
 PyObject *ag_name;
 PyObject *ag_qualname;
 _PyErr_StackItem ag_exc_state;
 PyObject *ag_origin_or_finalizer;
 char ag_hooks_inited;
 char ag_closed;
 char ag_running_async;
 int8_t ag_frame_state;
 _PyInterpreterFrame ag_iframe;
};

The two async-specific fields are ag_closed (set once the generator has received GeneratorExit) and ag_origin_or_finalizer, which doubles as storage for the thread’s async-generator finalizer hook. The frame itself is the same _PyInterpreterFrame used by every other callable, so an async generator reuses the normal eval loop and yield/YIELD_VALUE machinery. PyAsyncGen_New allocates through gen_new_with_qualname and then initializes the async fields:

 c
 
 
// Objects/genobject.c
PyObject *PyAsyncGen_New(PyFrameObject *f, PyObject *name, PyObject *qualname) {
 PyAsyncGenObject *ag;
 ag = (PyAsyncGenObject *)gen_new_with_qualname(&PyAsyncGen_Type, f,
 name, qualname);
 ...
 ag->ag_origin_or_finalizer = NULL;
 ag->ag_closed = 0;
 ag->ag_hooks_inited = 0;
 ag->ag_running_async = 0;
 return (PyObject*)ag;
}

`ag_code`

Like gi_code/cr_code, ag_code is a read-only attribute backed by the frame’s executable. ag_getcode just forwards to _gen_getcode, which returns a new reference to _PyGen_GetCode(gen) (i.e. _PyFrame_GetCode(&gen->ag_iframe)):

 c
 
 
// Objects/genobject.c
static PyObject *
ag_getcode(PyObject *gen, void *Py_UNUSED(ignored)) {
 return _gen_getcode((PyGenObject*)gen, "ag_code");
}

static PyObject *
_gen_getcode(PyGenObject *gen, const char *const name) {
 if (PySys_Audit("object.__getattr__", "Os", gen, name) < 0) {
 return NULL;
 }
 return Py_NewRef(_PyGen_GetCode(gen));
}

It is registered in async_gen_getsetlist as {"ag_code", ag_getcode, NULL, NULL}. Because the code object is reachable through the (possibly cleared) frame, ag_code remains available even after the generator is closed.

Async generator hooks

CPython exposes two per-thread hooks, sys.set_asyncgen_firstiter and sys.set_asyncgen_finalizer, stored on the thread state as tstate->async_gen_firstiter and tstate->async_gen_finalizer (see Python/ceval.c). The firstiter hook is invoked the first time an async generator is iterated; the finalizer is stored on the object for later use.

Hook initialization is lazy and guarded by ag_hooks_inited, so it happens on the first call to anext/asend/aclose/athrow:

 c
 
 
// Objects/genobject.c
static int async_gen_init_hooks(PyAsyncGenObject *o) {
 PyThreadState *tstate;
 PyObject *finalizer;
 PyObject *firstiter;

 if (o->ag_hooks_inited) return 0;
 o->ag_hooks_inited = 1;

 tstate = _PyThreadState_GET();

 finalizer = tstate->async_gen_finalizer;
 if (finalizer) {
 o->ag_origin_or_finalizer = Py_NewRef(finalizer);
 }

 firstiter = tstate->async_gen_firstiter;
 if (firstiter) {
 PyObject *res;
 Py_INCREF(firstiter);
 res = PyObject_CallOneArg(firstiter, (PyObject *)o);
 Py_DECREF(firstiter);
 if (res == NULL) return 1; /* propagate exception */
 Py_DECREF(res);
 }
 return 0;
}

Note the ordering: the finalizer is stored (ag_origin_or_finalizer), while the firstiter callback is called immediately and its result discarded. If the firstiter callback raises, the error propagates out of anext() etc.

`anext`/`asend`/`athrow` are awaitables

None of these run the generator directly. async_gen_anext returns an async_generator_asend object (with sendval == NULL), and async_gen_asend wraps the supplied argument. These awaitables (PyAsyncGenASend) carry state (AWAITABLE_STATE_INIT / _ITER / _CLOSED) plus the generator and the value to send. Driving them via .send()/__next__() calls gen_send, then async_gen_unwrap_value converts a wrapped yield value into a StopIteration:

 c
 
 
// Objects/genobject.c
static PyObject *async_gen_unwrap_value(PyAsyncGenObject *gen, PyObject *result) {
 if (result == NULL) {
 if (!PyErr_Occurred()) PyErr_SetNone(PyExc_StopAsyncIteration);
 if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration) ||
 PyErr_ExceptionMatches(PyExc_GeneratorExit)) {
 gen->ag_closed = 1;
 }
 gen->ag_running_async = 0;
 return NULL;
 }
 if (_PyAsyncGenWrappedValue_CheckExact(result)) {
 _PyGen_SetStopIterationValue(((_PyAsyncGenWrappedValue*)result)->agw_val);
 Py_DECREF(result);
 gen->ag_running_async = 0;
 return NULL; /* signals "yielded a value" via StopIteration */
 }
 return result;
}

ag_running_async guards re-entrancy: while a generator is running, a second __anext__() raises “asynchronous generator is already running”.

`aclose`

async_gen_aclose is implemented as athrow(GeneratorExit) in “aclose mode” (agt_typ == NULL). It creates an async_generator_athrow awaitable and, on first send, marks the generator closed and throws GeneratorExit with close_on_genexit = 0 (so a suspended yield from is not auto-closed immediately, allowing the async generator to clean up via awaits):

 c
 
 
// Objects/genobject.c (async_gen_athrow_send, aclose mode)
if (o->agt_typ == NULL) {
 /* aclose() mode */
 o->agt_gen->ag_closed = 1;
 retval = _gen_throw((PyGenObject *)gen, 0,
 PyExc_GeneratorExit, NULL, NULL);
 if (retval && _PyAsyncGenWrappedValue_CheckExact(retval)) {
 Py_DECREF(retval);
 goto yield_close; /* generator yielded during close -> RuntimeError */
 }
}

If the generator yields a value while being closed, CPython sets ASYNC_GEN_IGNORED_EXIT_MSG (“async generator ignored GeneratorExit”) and raises RuntimeError. Otherwise, on completion, StopAsyncIteration/GeneratorExit are swallowed and converted to StopIteration, signalling that the aclose() awaitable is done:

 c
 
 
// Objects/genobject.c
if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration) ||
 PyErr_ExceptionMatches(PyExc_GeneratorExit)) {
 if (o->agt_typ == NULL) {
 PyErr_Clear();
 PyErr_SetNone(PyExc_StopIteration);
 }
}

The ag_closed flag is also consulted by subsequent asend/athrow calls, which then immediately raise StopAsyncIteration without running the body again.

Finalization

gen_finalize (registered as tp_finalize) handles unawaited async generators: if the frame is not finished and ag_closed is false, it invokes the stored finalizer (ag_origin_or_finalizer) so that unclosed async generators are reported (and typically closed) rather than silently leaked.

Summary

PyAsyncGenObject shares _PyGenObject_HEAD(ag) with generators; async-specific state is ag_closed, ag_running_async, ag_hooks_inited and ag_origin_or_finalizer.
ag_code is a read-only getset returning the frame’s code object.
Hooks are initialized lazily on first iteration: the finalizer is stored, firstiter is called immediately.
anext/asend/athrow return awaitables (PyAsyncGenASend / PyAsyncGenAThrow) that drive the generator via gen_send/gen_throw.
aclose is athrow(GeneratorExit) in a special mode that marks the generator closed, forbids yielding, and converts terminal exceptions to StopIteration.

Async on Opensource internals

How CPython Implements async generators in C

The object struct

`ag_code`

Async generator hooks

`anext`/`asend`/`athrow` are awaitables

`aclose`

Finalization

Summary

Further reading

Async on Opensource internals

How CPython Implements async generators in C

The object struct

ag_code

Async generator hooks

anext/asend/athrow are awaitables

aclose

Finalization

Summary

Further reading

`ag_code`

`anext`/`asend`/`athrow` are awaitables

`aclose`