The Interrupt Budget: How V8 Decides When to Optimize a Function
V8 keeps a countdown in the function's feedback cell, spends it on backward jumps, and uses the leftover to decide when a function deserves Maglev or TurboFan.
TL;DR: Every JavaScript function that has a feedback cell also has an
int32interrupt budget (src/objects/feedback-cell.tq). The interpreter subtracts the byte distance of every backward jump from it, and when it goes negative the interpreter calls a runtime function,Runtime_BytecodeBudgetInterrupt_Ignition, which asks theTieringManagerwhether to compile. The budget’s size isinvocation_count × bytecode_length, so a big function gets a bigger budget — but the same number of executions. There is no separate invocation counter driving tiering, and no profiler involved.
“How does V8 know a function is hot?” is one of those questions that gets a vague answer: “it counts invocations”. The real mechanism is more interesting, and it is not a counter of calls at all.
V8 gives every function a budget of bytecode it is allowed to execute. The interpreter spends that budget as it runs. When the budget runs out, V8 asks itself whether the function is worth optimizing. The budget is the hotness detector, the tier-up trigger, and the stability filter all at once.
Where the budget lives (and where it does not)
The first surprise is that the budget is not in the feedback vector. It is in the feedback cell, which is a much smaller and much earlier object:
// src/objects/feedback-cell.tq:5
@cppObjectLayoutDefinition
extern class FeedbackCell extends Struct {
value: Undefined|FeedbackVector|ClosureFeedbackCellArray;
dispatch_handle: int32;
interrupt_budget: int32;
}A FeedbackCell is a Struct — not a heap object with a map, just a
fixed-size blob of memory. Three fields: a pointer to the feedback
vector (or to a closure feedback cell array, or undefined), a dispatch
handle, and the budget.
The distinction matters. The feedback vector is allocated lazily, and only for functions that V8 thinks are worth tracking. The feedback cell is allocated when the function is created. So a function that has never been executed still has a budget, and the budget is the thing that decides whether it gets a feedback vector at all.
There is no interrupt budget in the bytecode array. The bytecode array
lives in src/objects/bytecode-array.tq (not in src/interpreter/)
and has no such field:
// src/objects/bytecode-array.tq:7
@cppObjectLayoutDefinition
extern class BytecodeArray extends ExposedTrustedObject {
const length: Smi;
wrapper: BytecodeWrapper;
source_position_table: ProtectedPointer<TrustedByteArray>;
handler_table: ProtectedPointer<TrustedByteArray>;
constant_pool: ProtectedPointer<TrustedFixedArray>;
frame_size: int32;
parameter_size: uint16;
max_arguments: uint16;
incoming_new_target_or_generator_register: int32;
@if(TAGGED_SIZE_8_BYTES) optional_padding: uint32;
bytes[length]: uint8;
}The budget is in the feedback cell, and the feedback cell is what the
JSFunction points at; the cell is what holds the feedback vector.
How the interpreter spends it
The interpreter decrements the budget on backward jumps. Not on calls, not on returns — on backward jumps. The generated code is straightforward:
// src/interpreter/interpreter-assembler.cc:1179
TNode<Int32T> InterpreterAssembler::UpdateInterruptBudget(
TNode<Int32T> weight) {
TNode<JSFunction> function = LoadFunctionClosure();
TNode<FeedbackCell> feedback_cell = LoadObjectField<FeedbackCell>(
function, offsetof(JSFunction, feedback_cell_));
TNode<Int32T> old_budget = LoadObjectField<Int32T>(
feedback_cell, offsetof(FeedbackCell, interrupt_budget_));
// Update budget by |weight| and check if it reaches zero.
TNode<Int32T> new_budget = Int32Sub(old_budget, weight);
// Update budget.
StoreObjectFieldNoWriteBarrier(
feedback_cell, offsetof(FeedbackCell, interrupt_budget_), new_budget);
return new_budget;
}One load, one subtract, one store, no write barrier (the field is a
plain int32, not a tagged pointer). Then the check:
// src/interpreter/interpreter-assembler.cc:1195
void InterpreterAssembler::DecreaseInterruptBudget(
TNode<Int32T> weight, StackCheckBehavior stack_check_behavior) {
Comment("[ DecreaseInterruptBudget");
Label done(this), interrupt_check(this);
// Assert that the weight is positive.
CSA_DCHECK(this, Int32GreaterThanOrEqual(weight, Int32Constant(0)));
// Make sure we include the current bytecode in the budget calculation.
TNode<Int32T> weight_after_bytecode =
Int32Add(weight, Int32Constant(CurrentBytecodeSize()));
TNode<Int32T> new_budget = UpdateInterruptBudget(weight_after_bytecode);
Branch(Int32GreaterThanOrEqual(new_budget, Int32Constant(0)), &done,
&interrupt_check);
BIND(&interrupt_check);
TNode<JSFunction> function = LoadFunctionClosure();
CallRuntime(stack_check_behavior == kEnableStackCheck
? Runtime::kBytecodeBudgetInterruptWithStackCheck_Ignition
: Runtime::kBytecodeBudgetInterrupt_Ignition,
GetContext(), function);
Goto(&done);
BIND(&done);The weight is the distance of the jump — the number of bytes of bytecode between the jump and its target. A tight loop charges its own body size on every iteration. A forward jump charges nothing.
The backward-jump hook is one line:
// src/interpreter/interpreter-assembler.cc:1252
void InterpreterAssembler::JumpBackward(TNode<IntPtrT> jump_offset) {
DecreaseInterruptBudget(TruncateIntPtrToInt32(jump_offset),
kEnableStackCheck);
JumpToOffset(IntPtrSub(BytecodeOffset(), jump_offset));
}And there is a second charge point, on return, which simulates a backedge to the start of the function so that a function with no loops still ages:
// src/interpreter/interpreter-assembler.cc:1463
void InterpreterAssembler::UpdateInterruptBudgetOnReturn() {
...
// Update profiling count by the number of bytes between the end of the
// current bytecode and the start of the first one, to simulate backedge to
// start of function.
...
TNode<Int32T> profiling_weight =
Int32Sub(TruncateIntPtrToInt32(BytecodeOffset()),
Int32Constant(kFirstBytecodeOffset));
DecreaseInterruptBudget(profiling_weight, kDisableStackCheck);
}So a straight-line function called many times does eventually exhaust its budget — one full pass over its body per call.
The other two tiers have their own versions of the same check. Sparkplug adds a negative weight to the budget and branches:
// src/baseline/x64/baseline-assembler-x64-inl.h:471
void BaselineAssembler::AddToInterruptBudgetAndJumpIfNotExceeded(
int32_t weight, Label* skip_interrupt_label) {
ASM_CODE_COMMENT(masm_);
ScratchRegisterScope scratch_scope(this);
Register feedback_cell = scratch_scope.AcquireScratch();
LoadFeedbackCell(feedback_cell);
__ addl(
FieldOperand(feedback_cell, offsetof(FeedbackCell, interrupt_budget_)),
Immediate(weight));
if (skip_interrupt_label) {
DCHECK_LT(weight, 0);
__ j(greater_equal, skip_interrupt_label);
}
}and Maglev emits explicit ReduceInterruptBudgetForLoop and
ReduceInterruptBudgetForReturn nodes into its graph, scaled by
--osr-from-maglev-interrupt-scale-factor (0.8) so that a Maglev
function reaches its budget slightly sooner than an Ignition one
would.
What the budget is worth
The budget’s size is computed in one place:
// src/execution/tiering-manager.cc:182
// Decrease times of interrupt budget underflow, the reason of not setting
// to INT_MAX is the interrupt budget may overflow when doing add
// operation for forward jump.
constexpr int kMaxInterruptBudget = INT_MAX / 2;
int ScaleInterruptBudget(int64_t invocations, int bytecode_length) {
int64_t budget = invocations * bytecode_length;
return static_cast<int>(std::clamp<int64_t>(budget, 0, kMaxInterruptBudget));
}invocations × bytecode_length. This is the piece people get wrong.
The multiplication by the bytecode length does not mean “big
functions are optimized earlier” — it means the opposite. The budget is
spent at a rate of one bytecode length per pass over the function, so
multiplying by the bytecode length makes the number of executions
required constant, regardless of how big the function is.
A 10-byte function and a 10 000-byte function both need 3000 passes (the TurboFan default) before their budget runs out. The big function simply has a bigger number to spend.
That is the whole design of the heuristic: the unit is executions, the implementation is bytes. V8 could have counted calls; it counts bytes because bytes are what it already has for free (the jump distance is already in a register), and because byte counting naturally handles loops — a function that loops 1000 times per call burns its budget 1000 times faster, which is exactly right.
The budget size depends on which tier the function is heading for:
// src/execution/tiering-manager.cc:258
return ScaleInterruptBudget(int64_t{v8_flags.invocation_count_for_maglev},
bytecode_length);
}
return ScaleInterruptBudget(int64_t{v8_flags.invocation_count_for_turbofan},
bytecode_length);
}and there are special cases. If the function is already being compiled, the budget is set to the maximum so that no further interrupts happen while the compiler is working:
// src/execution/tiering-manager.cc:213
// Avoid interrupts while we're already tiering.
if (tiering_in_progress && !osr_tiering_in_progress) {
return kMaxInterruptBudget;
}And if the function’s bytecode is too big to be worth optimizing, the budget never runs out at all:
// src/execution/tiering-manager.cc:281
DCHECK(function->has_feedback_vector());
if (bytecode_length > v8_flags.max_optimized_bytecode_size) {
return kMaxInterruptBudget;
}--max-optimized-bytecode-size defaults to 60 KB. A function bigger
than that is never optimized, so charging it a budget would be
pointless.
The first budget: getting a feedback vector at all
Before there is a feedback vector, there is a feedback cell with a budget in it, and the first tier-up is Sparkplug (the baseline compiler). Its budget is much smaller:
// src/execution/tiering-manager.cc:275
if (FirstTimeTierUpToSparkplug(isolate, function)) {
return ScaleInterruptBudget(
int64_t{v8_flags.invocation_count_for_feedback_allocation},
bytecode_length);
}--invocation-count-for-feedback-allocation is 8. So after eight
executions, V8 decides the function is worth a feedback vector and a
baseline compilation. This is the “budget for feedback vector
allocation” and it is deliberately tiny: allocating a feedback vector
costs memory, and a function that runs eight times is worth paying for.
The allocation itself happens inside the interrupt handler:
// src/execution/tiering-manager.cc:606
// Ensure that the feedback vector has been allocated.
if (!had_feedback_vector) {
if (compile_sparkplug && function->shared()->cached_tiering_decision() ==
CachedTieringDecision::kPending) {
// Mark the function as compiled with sparkplug before the feedback
// vector is created to initialize the interrupt budget for the next
// tier.
function->shared()->set_cached_tiering_decision(
CachedTieringDecision::kEarlySparkplug);
}
JSFunction::CreateAndAttachFeedbackVector(isolate_, function,
&is_compiled_scope);and the next budget (the one for Maglev or TurboFan) is installed
right after, with kRaise semantics — take the maximum of the old and
the new value, so a budget that has already been partially spent is not
reset to full:
// src/objects/js-function.cc:318
void JSFunction::SetInterruptBudget(
Isolate* isolate, BudgetModification kind,
std::optional<CodeKind> override_active_tier) {
int32_t current = raw_feedback_cell()->interrupt_budget();
int32_t new_budget =
TieringManager::InterruptBudgetFor(isolate, this, override_active_tier);
switch (kind) {
case BudgetModification::kRaise:
new_budget = std::max(current, new_budget);
break;
case BudgetModification::kReduce:
new_budget = std::min(current, new_budget);
break;
case BudgetModification::kReset:
break;
}
raw_feedback_cell()->set_interrupt_budget(new_budget);
}Three modifications: raise (never lower an existing budget), reduce (always lower it), reset (overwrite). The reset is used when the feedback cell is (re)initialized for a closure.
And there is one more wrinkle: the whole point of lazy feedback
allocation is that most functions never get a feedback vector at all.
JSFunction::InitializeFeedbackCell only creates one if it is really
needed:
// src/objects/js-function.cc:730
const bool needs_feedback_vector =
!v8_flags.lazy_feedback_allocation ||
// We also need a feedback vector for certain log events, collecting type
// profile and more precise code coverage.
v8_flags.log_function_events ||
!isolate->is_best_effort_code_coverage() ||
function->shared()->cached_tiering_decision() !=
CachedTieringDecision::kPending;--lazy-feedback-allocation is on by default. A function that is called
once, does nothing interesting, and returns never gets a feedback
vector, never gets a baseline compilation, and never gets optimized. The
budget of 8 is the price of admission.
The tiering state machine
The decision of which tier to go to next is not a simple ladder. It is a six-state machine, and it is written down in the source:
// src/common/globals.h:2957
// State machine:
// S(tate)0: kPending
// S1: kEarlySparkplug
// S2: kDelayMaglev
// S3: kEarlyMaglev
// S4: kEarlyTurbofan
// S5: kNormal
//
// C(ondition)0: sparkplug compile
// C1: maglev compile
// C2: deopt early
// C3: ic was stable early
// C4: turbofan compile
// C5: ic change or deopt
//
// S0 - C0 -> S1 - C1 - C3 -> S3 - C4 -> S4 -|
// | | |
// | |--------------------|
// | |
// C2 C5
// | |
// --> S2 --> S5
enum class CachedTieringDecision : int32_t {
kPending,
kEarlySparkplug,
kDelayMaglev,
kEarlyMaglev,
kEarlyTurbofan,
kNormal,
};It is cached on the SharedFunctionInfo and it is three bits wide:
// src/objects/shared-function-info.tq:58
cached_tiering_decision: CachedTieringDecision: 3 bit;so every function in the program carries its position in the tiering
ladder in three bits of the SFI. The point of the machine is
profile-guided optimization: V8 watches whether a function’s inline
caches keep changing. A function whose ICs are still moving is not worth
optimizing yet, because the optimized code would be based on type
feedback that is about to change. So the machine has a “delay” state
(kDelayMaglev) for functions that deoptimized early, and an “early”
path for functions whose ICs settled early.
Each state has a different budget:
// src/execution/tiering-manager.cc:234
if (v8_flags.profile_guided_optimization) {
switch (cached_tiering_decision) {
case CachedTieringDecision::kDelayMaglev:
return ScaleInterruptBudget(
int64_t{std::max(v8_flags.invocation_count_for_maglev,
v8_flags.minimum_invocations_after_ic_update)} +
v8_flags.invocation_count_for_maglev_with_delay,
bytecode_length);
case CachedTieringDecision::kEarlyMaglev:
case CachedTieringDecision::kEarlyTurbofan:
return ScaleInterruptBudget(
int64_t{v8_flags.invocation_count_for_early_optimization},
bytecode_length);A function that has been delayed gets a longer budget (Maglev’s threshold plus 600 more); a function whose ICs settled early gets a much shorter one (30).
When an inline cache changes
The budget is not the only stability signal. V8 also watches the inline caches. When an IC changes, the function’s type feedback just moved, and any code compiled from the old feedback is suspect. V8 extends the budget to buy time:
// src/execution/tiering-manager.cc:510
OptimizationDecision decision = ShouldOptimize(vector, code_kind);
if (decision.should_optimize()) {
Tagged<SharedFunctionInfo> shared = vector->shared_function_info();
int bytecode_length = shared->GetBytecodeArray(isolate_)->length();
Tagged<FeedbackCell> cell = vector->parent_feedback_cell();
int invocations =
std::max(1, v8_flags.minimum_invocations_after_ic_update.value());
int bytecodes = std::max(
1, std::min(bytecode_length, kMaxInterruptBudget / invocations));
int new_budget = ScaleInterruptBudget(int64_t{invocations}, bytecodes);
int current_budget = cell->interrupt_budget();--minimum-invocations-after-ic-update is 500. After an IC change, the
function must run at least 500 more times before it is eligible for
optimization again. The budget is raised to that, but only if that is
larger than the current budget:
// src/execution/tiering-manager.cc:568
if (new_budget > current_budget) {
if (v8_flags.trace_opt_verbose) {
PrintF("[delaying optimization of %s, IC changed]\n",
shared->DebugNameCStr().get());
}
vector->set_interrupt_budget_reset_by_ic_change(true);
cell->set_interrupt_budget(new_budget);
}and the flag interrupt_budget_reset_by_ic_change records that this
happened, so that the accounting can be corrected later.
The interrupt handler
When the budget goes negative, the interpreter calls a runtime function which does nothing but ask the tiering manager:
// src/runtime/runtime-internal.cc:461
Tagged<Object> BytecodeBudgetInterrupt(Isolate* isolate, RuntimeArguments& args,
CodeKind code_kind) {
HandleScope scope(isolate);
DCHECK(args.length() == 1 ||
(code_kind == CodeKind::MAGLEV && args.length() == 2));
DirectHandle<JSFunction> function = args.at<JSFunction>(0);
function->TraceOptimizationStatus("budget from %s",
CodeKindToString(code_kind));
TRACE_EVENT("v8.execute", "V8.BytecodeBudgetInterrupt");
isolate->tiering_manager()->OnInterruptTick(function, code_kind);
return ReadOnlyRoots(isolate).undefined_value();
}OnInterruptTick is where the actual decision is made. It is a long
function, but the shape is:
// src/execution/tiering-manager.cc:652
// We only tier up beyond sparkplug if we already had a feedback vector.
if (first_time_tiered_up_to_sparkplug) {
// If we didn't have a feedback vector, the interrupt budget has already
// been set by JSFunction::CreateAndAttachFeedbackVector, so no need to
// set it again.
if (had_feedback_vector) {
...
function->SetInterruptBudget(isolate_, BudgetModification::kRaise);
}
return;
}
// Don't tier up if Turbofan is disabled.
// TODO(jgruber): Update this for a multi-tier world.
if (V8_UNLIKELY(!isolate_->use_optimizer())) {
function->SetInterruptBudget(isolate_, BudgetModification::kRaise);
return;
}Note the kRaise in both early exits: the budget is refilled, so the
function will not be interrupted again for another full budget’s worth
of execution. That is the “raise” semantics — the budget is never
lowered by a tick.
Then the decision itself:
// src/execution/tiering-manager.cc:434
OptimizationDecision TieringManager::ShouldOptimize(
Tagged<FeedbackVector> feedback_vector, CodeKind current_code_kind) {
Tagged<SharedFunctionInfo> shared = feedback_vector->shared_function_info();
if (current_code_kind == CodeKind::TURBOFAN_JS) {
return OptimizationDecision::DoNotOptimize();
}
if (TiersUpToMaglev(current_code_kind) &&
shared->PassesFilter(v8_flags.maglev_filter) &&
!shared->maglev_compilation_failed()) {
if (v8_flags.profile_guided_optimization &&
shared->cached_tiering_decision() ==
CachedTieringDecision::kEarlyTurbofan) {
return OptimizationDecision::TurbofanHotAndStable();
}
return OptimizationDecision::Maglev();
}The ladder is: if we are already TurboFan, stop. If Maglev is enabled and the function passes the Maglev filter, go to Maglev (or straight to TurboFan if the PGO state says so). Otherwise, check the TurboFan preconditions and go to TurboFan.
And the last of those preconditions is the size check again:
// src/execution/tiering-manager.cc:468
Tagged<BytecodeArray> bytecode = shared->GetBytecodeArray(isolate_);
if (bytecode->length() > v8_flags.max_optimized_bytecode_size) {
return OptimizationDecision::DoNotOptimize();
}
return OptimizationDecision::TurbofanHotAndStable();The flags you can turn
The whole mechanism is driven by a handful of integers in
src/flags/flag-definitions.h:
// src/flags/flag-definitions.h:1120
// Tiering: Sparkplug / feedback vector allocation.
DEFINE_INT(invocation_count_for_feedback_allocation, 8,
"invocation count required for allocating feedback vectors")
// Tiering: Maglev.
#if defined(ANDROID)
DEFINE_INT(invocation_count_for_maglev, 1000,
"invocation count required for optimizing with Maglev")
#else
DEFINE_INT(invocation_count_for_maglev, 400,
"invocation count required for optimizing with Maglev")
#endif // ANDROID
DEFINE_INT(invocation_count_for_maglev_osr, 100,
"invocation count required for maglev OSR")// src/flags/flag-definitions.h:1145
// Tiering: Turbofan.
DEFINE_INT(invocation_count_for_turbofan, 3000,
"invocation count required for optimizing with TurboFan")
DEFINE_INT(invocation_count_for_osr, 500, "invocation count required for OSR")
DEFINE_FLOAT(invocation_count_for_osr_factor_while_tiering_in_progress, 3.0,
"invocation count factor while waiting for code to compile")
DEFINE_UINT(osr_to_tierup, 1,
"number to decrease the invocation budget by when we follow OSR")
DEFINE_INT(minimum_invocations_after_ic_update, 500,
"How long to minimally wait after IC update before tier up")
DEFINE_INT(minimum_invocations_before_optimization, 2,
"Minimum number of invocations we need before non-OSR optimization")Note that Maglev’s threshold is 1000 on Android and 400 elsewhere - Android devices are slower and more battery-constrained, so V8 makes them wait longer before spending energy on a second compilation.
To pin a tier, you do not set a threshold to infinity; you turn the
tier off. --no-maglev and --no-turbofan are the flags, and the
isolate-level predicate is:
// src/execution/isolate.cc:6910
bool Isolate::use_optimizer() {
// TODO(v8:7700): Update this predicate for a world with multiple tiers.
return (v8_flags.turbofan || v8_flags.maglev) && !serializer_enabled_ &&
CpuFeatures::SupportsOptimizer() && !is_precise_count_code_coverage();
}For debugging there is also the manual override: %OptimizeFunctionOnNextCall
bypasses the budget entirely.
// src/runtime/runtime-test.cc:419
TraceManualRecompile(*function, target_kind, concurrency_mode);
JSFunction::EnsureFeedbackVector(isolate, function, &is_compiled_scope);
if (function->GetActiveTier(isolate) != target_kind) {
function->RequestOptimization(isolate, target_kind, concurrency_mode);
}Note that it forces the feedback vector to exist first — you cannot optimize a function that has no type feedback.
Trade-offs
What the byte-counting budget buys. It measures work done, not calls made. A function called ten million times that does nothing is not hot. A function called three times that each take a second is. Loops are handled naturally, because a loop iteration is a backward jump and therefore costs exactly what it should.
What it costs.
- The budget is a proxy, and proxies are noisy. A function that is called in a burst will exhaust its budget, get queued for optimization, and then sit in the queue while the program moves on. This is why the hysteresis band and the “raise, don’t lower” rule exist.
- The budget must be refilled after every interrupt, which means a function that is almost hot keeps getting interrupted and keeps going through the tiering decision.
- The thresholds are magic numbers. 8, 400, 3000, 500, 30, 600. They were tuned by measurement and they are all overridable, but nobody has a principled reason for any of them.
- Big functions are excluded entirely rather than handled, because optimizing a 60 KB function can blow up the compiler.
What the lazy feedback vector buys. Memory. Most functions in a
program are called once or twice. Giving each of them a feedback vector
would be a large fraction of a modern JS heap for no benefit. The price
is a small amount of complexity in InitializeFeedbackCell and a
second, smaller budget (8 executions) that has to be handled before the
“real” tiering logic even starts.
The bottom line
The mental model is: a function is hot when it has spent its
bytecode budget. V8 gives every function a countdown, spends it as
the interpreter runs, and asks the tiering manager when it runs out.
The countdown’s size is invocation_count × bytecode_length, so the
number of executions needed is the same for every function size — the
bytecode length is there to make the accounting natural, not to favour
big functions.
Above that sit three more layers: the feedback-vector budget (8 executions, then you get a feedback vector and a baseline compilation), the tiering state machine that tracks whether the function’s inline caches have settled, and the IC-change timer that pushes the budget out when the type feedback moves.
The thing people get wrong
“the invocation count” is not the hotness counter. There is an
invocation count, stored in the feedback vector, but it is not what
drives tiering — it is used for OSR decisions and for the
invocation_count_before_stable bookkeeping that feeds the PGO state
machine. The thing that drives tiering is the interrupt budget, and it
is measured in bytecode bytes.
The second misconception is that the budget lives in the feedback vector. It lives in the feedback cell, which is why it exists before there is a feedback vector, and why a function can be tiered up to Sparkplug without ever having had its type feedback collected.
The third is that the budget is per-function. It is per-feedback
cell, and a feedback cell is per (function, native context) pair —
the header of src/objects/feedback-cell.h says it:
This is a special cell used to maintain both the link between a closure and its feedback vector, as well as a way to count the number of closures created for a certain function per native context. There’s at most one FeedbackCell for each function in a native context.
So several closures of the same function, created in the same context, share one budget and are tiered up together. A closure created in a different context gets its own cell, its own budget, and its own trip through the tiering machine.