Opensource internals

Flushing Code: How V8 Gives Memory Back When the Heap is Full

15 min read Jay Hubbert

V8 gives memory back by throwing away the compiled forms of functions that have not run recently. The decision is a wall-clock age counter on the SharedFunctionInfo, and the bytecode is replaced in place by an UncompiledData object.

TL;DR: Every SharedFunctionInfo carries a 16-bit age counter that is bumped by the number of seconds since the last full GC. When the age passes bytecode_old_time (180 seconds by default), the mark-compact collector replaces the function’s BytecodeArray in place with a smaller UncompiledData object, drops the feedback metadata, and resets the closure’s code to CompileLazy. Nothing is recompiled until the function is actually called again.

A JavaScript heap is mostly garbage, and the garbage collector handles that part well. But a long-running server also accumulates a second kind of garbage: code. A service that loads a thousand modules at start-up and then serves requests for a week holds a thousand modules’ worth of bytecode, baseline code, optimized code, feedback vectors, source strings and constant pools - most of which will never be executed again.

V8’s answer is to age code out on a wall clock and throw it away, on the theory that a function that has not run in the last three minutes is not worth the memory it occupies.

Three tiers of code, one function

A single JavaScript function can exist in up to four forms in V8 at the same time:

  1. Bytecode (Ignition). The canonical form. A BytecodeArray on the SharedFunctionInfo.
  2. Baseline code (Sparkplug). A machine-code translation of the bytecode, no feedback, roughly 2-5x faster than the interpreter.
  3. Optimized code (Maglev or TurboFan). Machine code built from the feedback, potentially 100x faster than the interpreter.
  4. The feedback vector, which is the input to tier 3 and is itself allocated per closure.

Flushing has to decide what to throw away, and the answer is a ladder. You can throw away the optimized code cheaply - it is just a Code object, and the GC will collect it if nothing points at it. You can throw away the baseline code. You can throw away the bytecode, but that is the expensive one, because the bytecode is the only thing that can be recompiled without going back to the source text.

The flags are explicit about which rungs are in scope:

cpp 
// src/flags/flag-definitions.h:2843
DEFINE_BOOL(flush_baseline_code, false,
            "flush of baseline code when it has not been executed recently")
DEFINE_BOOL(flush_bytecode, true,
            "flush of bytecode when it has not been executed recently")
DEFINE_INT(bytecode_old_time, 180, "number of seconds before we flush code")
DEFINE_BOOL(stress_flush_code, false, "stress code flushing")
DEFINE_WEAK_IMPLICATION(stress_flush_code, flush_baseline_code)
DEFINE_DEVELOPER_FLAG(trace_flush_code, "trace bytecode flushing")

Bytecode flushing is on by default. Baseline code flushing is off by default - the baseline tier is cheap enough that keeping it is usually worth it.

The age counter is a clock, not a counter

The decision is made per SharedFunctionInfo, and the input is a 16-bit field:

cpp 
// src/objects/shared-function-info.h:384
  static constexpr int kAgeSize = sizeof(uint16_t);
  static constexpr uint16_t kMaxAge = UINT16_MAX;
cpp 
// src/objects/shared-function-info.h:997
  std::atomic<uint16_t> age_ = 0;

The comparison is dead simple:

cpp 
// src/heap/marking-visitor-inl.h:556
template <typename ConcreteVisitor>
bool MarkingVisitorBase<ConcreteVisitor>::IsOld(
    Tagged<SharedFunctionInfo> sfi) const {
  return sfi->age() >= v8_flags.bytecode_old_time;
}

So “old” means “the age counter has reached 180”. But the age counter is not a count of executions or of GCs. It is a wall-clock age in seconds, and the increment comes from the GC tracer:

cpp 
// src/heap/gc-tracer.cc:806
  // Handle code flushing time deltas. Times are incremented conservatively:
  // 1. The first delta is 0s.
  // 2. Any delta is rounded downwards to a full second.
  // 3. 0s-deltas are carried over to the next GC with their precise diff. This
  //    allows for frequent GCs (within a single second) to be attributed
  //    correctly later on.
  // 4. The first non-zero increment after a reset always just increments by 1s.

The increment is the number of seconds between the start of this marking phase and the start of the previous one. The four rules above are all about not lying: a GC that happens 300ms after the last one contributes zero, and the 300ms is carried forward so that the next GC that happens after a real second still counts the right total.

The bump itself is a compare-and-swap loop, because the concurrent marker may be racing with the mutator:

cpp 
// src/heap/marking-visitor-inl.h:562
template <typename ConcreteVisitor>
void MarkingVisitorBase<ConcreteVisitor>::MakeOlder(
    Tagged<SharedFunctionInfo> sfi) const {
  if (code_flushing_increase_ == 0) {
    return;
  }

  uint16_t current_age;
  uint16_t updated_age;
  do {
    current_age = sfi->age();
    // When the age is 0, it was reset by the function prologue in
    // Ignition/Sparkplug. But that might have been some time after the last
    // full GC. So in this case we don't increment the value like we normally
    // would but just set the age to 1. All non-0 values can be incremented as
    // expected (we add the number of seconds since the last GC) as they were
    // definitely last executed before the last full GC.
    updated_age = current_age == 0
                      ? 1
                      : SaturateAdd(current_age, code_flushing_increase_);
  } while (sfi->CompareExchangeAge(current_age, updated_age) != current_age);
}

And there is the crucial detail: running the function resets the age. The reset is a single 16-bit store in the function prologue. The interpreter entry trampoline does it on every call:

cpp 
// src/builtins/x64/builtins-x64.cc:1038
void ResetSharedFunctionInfoAge(MacroAssembler* masm, Register sfi) {
  __ movw(FieldOperand(sfi, offsetof(SharedFunctionInfo, age_)), Immediate(0));
}

void ResetJSFunctionAge(MacroAssembler* masm, Register js_function) {
  const Register shared_function_info(kScratchRegister);
  __ LoadTaggedField(
      shared_function_info,
      FieldOperand(js_function, offsetof(JSFunction, shared_function_info_)));
  ResetSharedFunctionInfoAge(masm, shared_function_info);
}

So the age is a timestamp of last execution, and “old” means “has not been executed for three minutes”. That is a completely different policy from “has been executed few times” - and it is the right one, because the thing you want to reclaim is memory held by code nobody is running.

How the bytecode becomes unreachable

The mechanism is the surprising part, and it is not “the collector deletes the bytecode”. It is “the collector stops looking at the bytecode”.

The marking visitor decides, per SharedFunctionInfo, whether to trace the bytecode array strongly:

cpp 
// src/heap/marking-visitor-inl.h:468
template <typename ConcreteVisitor>
size_t MarkingVisitorBase<ConcreteVisitor>::VisitSharedFunctionInfo(
    Tagged<Map> map, Tagged<SharedFunctionInfo> shared_info,
    MaybeObjectSize maybe_object_size) {
  const bool can_flush_bytecode = HasBytecodeArrayForFlushing(shared_info);

  // We found a BytecodeArray that can be flushed. Increment the age of the SFI.
  if (can_flush_bytecode && !should_keep_ages_unchanged_) {
    MakeOlder(shared_info);
  }

  if (!can_flush_bytecode || !ShouldFlushCode(shared_info)) {
    // If the SharedFunctionInfo doesn't have old bytecode visit the function
    // data strongly.

For an “old” SFI, the branch falls through to the else arm, and the bytecode array is not visited. The SFI still has a pointer to its bytecode, but the GC is no longer told to follow it. The bytecode therefore survives only if something else keeps it alive:

cpp 
// src/heap/marking-visitor-inl.h:506
  } else {
    // In other cases, record as a flushing candidate since we have old
    // bytecode.
    local_weak_objects_->code_flushing_candidates_local.Push(shared_info);
  }
  return Base::VisitSharedFunctionInfo(map, shared_info, maybe_object_size);
}

The most important such “something else” is the stack of a running invocation, which still holds a reference to the BytecodeArray being interpreted. That is why a function that is currently executing is never flushed out from under itself, and why the collector asserts that the bytecode is not marked before it mutates it.

The flush mode

Whether flushing happens at all is decided once per GC cycle, in a function that deliberately does not read flags directly (because it runs on the concurrent marker thread):

cpp 
// src/heap/mark-compact.cc:405
base::EnumSet<CodeFlushMode> GetCodeFlushMode(Isolate* isolate) {
  if (isolate->disable_bytecode_flushing()) {
    return base::EnumSet<CodeFlushMode>();
  }

  base::EnumSet<CodeFlushMode> code_flush_mode;
  if (v8_flags.flush_bytecode) {
    code_flush_mode.Add(CodeFlushMode::kFlushBytecode);
  }

  if (v8_flags.flush_baseline_code) {
    code_flush_mode.Add(CodeFlushMode::kFlushBaselineCode);
  }

  if (v8_flags.stress_flush_code) {
    // This is to check tests accidentally don't miss out on adding either flush
    // bytecode or flush code along with stress flush code. stress_flush_code
    // doesn't do anything if either one of them isn't enabled.
    DCHECK(v8_flags.fuzzing || v8_flags.flush_baseline_code ||
           v8_flags.flush_bytecode);
    code_flush_mode.Add(CodeFlushMode::kForceFlush);
  }

  if (isolate->heap()->IsLastResortGC()) {
    code_flush_mode.Add(CodeFlushMode::kForceFlush);
  }

  return code_flush_mode;
}

Note the last clause. Under memory pressure - V8’s “last resort” GC, the one it runs when it is about to die - the age check is bypassed entirely and everything is flushed. That is the safety valve: the normal policy is a gentle age-based reclamation, and the emergency policy is “throw away all the code and start again”.

Flushing bytecode: in place

The interesting implementation detail is that flushing the bytecode does not allocate a replacement. The BytecodeArray is mutated in place into an UncompiledData object:

cpp 
// src/heap/mark-compact.cc:3615
void MarkCompactCollector::FlushBytecodeFromSFI(
    Tagged<SharedFunctionInfo> shared_info) {
  DCHECK(shared_info->HasBytecodeArray());

  // Retain objects required for uncompiled data.
  Tagged<String> inferred_name = shared_info->inferred_name();
  int start_position = shared_info->StartPosition();
  int end_position = shared_info->EndPosition();

  shared_info->DiscardCompiledMetadata(
      heap_->isolate(),
      [](Tagged<HeapObject> object, ObjectSlot slot,
         Tagged<HeapObject> target) { RecordSlot(object, slot, target); });

The BytecodeArray’s map is swapped for uncompiled_data_without_preparse_data_map, the space past the end of the UncompiledData is turned into a filler object, and the name and source positions are copied into the new object:

cpp 
// src/heap/mark-compact.cc:3682
  // Create a filler object for any left over space in the bytecode array.
  if (!HeapLayout::InAnyLargeSpace(compiled_data)) {
    const int aligned_filler_offset = ALIGN_TO_ALLOCATION_ALIGNMENT(
        sizeof(UncompiledDataWithoutPreparseData));
    heap_->CreateFillerObjectAt(compiled_data.address() + aligned_filler_offset,
                                compiled_data_size - aligned_filler_offset);
  }

The point of doing it in place is that the GC has already decided what to sweep. A BytecodeArray is a large object, possibly in large-object space, and freeing it means the sweeper has to do a compaction dance. Mutating it in place means the memory is simply not reclaimed - the object shrinks in logical size but the pages stay. V8 trades memory for simplicity here: the pages become a filler, and the actual reclamation happens later, when the whole page is swept.

Note which UncompiledData variant this produces: always UncompiledDataWithoutPreparseData. The preparse data from the pre-parser is not preserved by the flush, so a later recompilation has to re-parse from source rather than reuse the pre-parse scope. There is a UncompiledDataWithPreparseData variant too, and it is used by the separate SharedFunctionInfo::DiscardCompiled path, which can keep the pre-parsed scope around.

The feedback goes too

Flushing the bytecode also throws away the FeedbackMetadata, which is the shape of the feedback - the list of which slot is which kind of IC. That metadata lives in a field that is shared with the outer scope info, so discarding it is a slightly delicate operation:

cpp 
// src/objects/shared-function-info.cc:364
void SharedFunctionInfo::DiscardCompiledMetadata(
    Isolate* isolate,
    std::function<void(Tagged<HeapObject> object, ObjectSlot slot,
                       Tagged<HeapObject> target)>
        gc_notify_updated_slot) {
  DisallowGarbageCollection no_gc;
  if (HasFeedbackMetadata()) {
    if (v8_flags.trace_flush_code) {
      CodeTracer::Scope scope(isolate->GetCodeTracer());
      PrintF(scope.file(), "[discarding compiled metadata for ");
      ShortPrint(this, scope.file());
      PrintF(scope.file(), "]\n");
    }

    if (scope_info()->HasOuterScopeInfo()) {
      Tagged<ScopeInfo> outer_scope_info = scope_info()->OuterScopeInfo();

      // Raw setter to avoid validity checks, since we're performing the unusual
      // task of decompiling.
      set_raw_outer_scope_info_or_feedback_metadata(outer_scope_info);

The comment “since we’re performing the unusual task of decompiling” is V8’s own word for it. The SharedFunctionInfo is being taken back to its pre-compilation state.

And the closures themselves are reset. Every JSFunction pointing at the flushed SharedFunctionInfo has to be put back to “needs compiling”:

cpp 
// src/objects/js-function-inl.h:607
void JSFunction::ResetIfCodeFlushed(
    Isolate* isolate,
    std::optional<std::function<void(Tagged<HeapObject> object, ObjectSlot slot,
                             Tagged<HeapObject> target)>>
        gc_notify_updated_slot) {
  const bool kBytecodeCanFlush =
      v8_flags.flush_bytecode || v8_flags.stress_snapshot;
  const bool kBaselineCodeCanFlush =
      v8_flags.flush_baseline_code || v8_flags.stress_snapshot;
  if (!kBytecodeCanFlush && !kBaselineCodeCanFlush) return;

  DCHECK_IMPLIES(NeedsResetDueToFlushedBytecode(isolate), kBytecodeCanFlush);
  if (kBytecodeCanFlush && NeedsResetDueToFlushedBytecode(isolate)) {
    // Bytecode was flushed and function is now uncompiled, reset JSFunction
    // by setting code to CompileLazy and clearing the feedback vector.
    ResetTieringRequests(isolate);
    UpdateCode(isolate, *BUILTIN_CODE(isolate, CompileLazy),
               SKIP_WRITE_BARRIER);
    raw_feedback_cell()->reset_feedback_vector(gc_notify_updated_slot);
    return;
  }

Two things happen here. The closure’s code pointer goes back to CompileLazy - a builtin that, when called, will re-parse and re-compile the function from source. And the feedback vector is dropped, via FeedbackCell::reset_feedback_vector, which puts the cell back to holding the closure feedback cell array.

The baseline-only case is gentler: the code goes to InterpreterEntryTrampoline rather than CompileLazy, because the bytecode is still there.

The compilation cache has to be told

There is one more place that holds on to compiled functions: the compilation cache, which maps source text to SharedFunctionInfo for scripts and evals. A cached SFI whose bytecode has just been flushed is a stale entry, and the cache cleans it up at the start of every mark-compact:

cpp 
// src/codegen/compilation-cache.cc:345
void CompilationCache::MarkCompactPrologue() {
  // Drop SFI entries with flushed bytecode.
  script_.Age();
  eval_global_.Age();
  eval_contextual_.Age();

  // Drop entries in oldest generation.
  reg_exp_.Age();
}

and the script cache’s ageing is literally “clear the entry if the SFI no longer has bytecode”:

cpp 
// src/codegen/compilation-cache.cc:66
void CompilationCacheScript::Age() {
  DisallowGarbageCollection no_gc;
  if (IsUndefined(table_)) return;
  Tagged<CompilationCacheTable> table = Cast<CompilationCacheTable>(table_);

  for (InternalIndex entry : table->IterateEntries()) {
    Tagged<Object> key;
    if (!table->ToKey(entry, &key)) continue;
    DCHECK(IsWeakFixedArray(key));

    Tagged<Object> value = table->PrimaryValueAt(entry);
    if (!IsUndefined(value)) {
      Tagged<SharedFunctionInfo> info = Cast<SharedFunctionInfo>(value);
      // Clear entries after Bytecode was flushed from SFI.
      if (!info->HasBytecodeArray()) {
        table->SetPrimaryValueAt(entry,
                                 ReadOnlyRoots(isolate()).undefined_value(),
                                 SKIP_WRITE_BARRIER);
      }
    }
  }
}

Note that the script cache keeps the entry, with the value blanked, rather than removing it: the key is the source string, and the key is what makes the cache worth having. And the table never shrinks on deletion, because the script cache depends on that:

cpp 
// src/objects/compilation-cache-table.cc:594
void CompilationCacheTable::RemoveEntry(InternalIndex entry) {
  int entry_index = EntryToIndex(entry);
  Tagged<Object> the_hole_value = GetReadOnlyRoots().the_hole_value();
  for (int i = 0; i < kEntrySize; i++) {
    this->set(entry_index + i, the_hole_value, SKIP_WRITE_BARRIER);
  }
  ElementRemoved();

  // This table does not shrink upon deletion. The script cache depends on that
  // fact, because EnsureScriptTableCapacity calls RemoveEntry at a time when
  // shrinking the table would be counterproductive.
}

Before the table is grown, it sweeps out entries whose weak key has been collected - that is the “LRU-ish” part: the cache table’s keys are weak references, so a collected source string takes its entry with it.

Why the interpreter is always the fallback

The reason flushing is safe is the layering. Optimized code is derived from bytecode, and bytecode is derived from source. Flushing optimized code loses nothing but time. Flushing bytecode loses time too - you have to re-parse and re-compile - but it is still correct, because the source is still there and the source is the ground truth.

V8 is careful about one thing though: the baseline code is not derived from bytecode in a way that lets it be regenerated for free, and the optimized code is not derived from bytecode either, it is derived from bytecode plus feedback. So flushing the bytecode also invalidates the feedback, and the next compile starts from nothing.

That is why the policy is gradual:

  • Optimized code is not “flushed” at all in the normal case - it is simply not referenced by anything once the function is reset to CompileLazy, and the GC collects it.
  • Baseline code can be flushed separately, and is off by default.
  • Bytecode is flushed last, and only for functions that have not run in three minutes.

The hot loop that keeps getting flushed

Now the failure mode. Suppose you have a function that runs in a loop but with a long gap between iterations - a polling loop, a cron-ish worker, a timer. Every time it runs, the age is reset. Every time it stops for more than three minutes, the age crosses the threshold and the next GC flushes it.

What happens then? The closure is reset to CompileLazy. The next call re-parses the source, re-compiles the bytecode, re-runs the interpreter until the interrupt budget is exhausted, re-collects the feedback, and re-optimizes. The first call after a flush is a hundred times slower than the steady state.

If the gap is longer than the flush threshold, you pay that cost every time. The function never reaches steady state, because it is flushed before it can stay warm. This is the classic “flushed code” pathology, and it is why bytecode_old_time is a flag you can raise.

There is a second, subtler version of the same problem. Flushing resets the age to 0 when the function next runs, and the age is only bumped by whole seconds since the last GC. If your GC runs more often than once per second, the increment is frequently zero and the age climbs slowly - which is the conservative behaviour the GC tracer’s comment describes.

Trade-offs

What it buys. Memory. A long-lived server that has loaded and evaluated a lot of code can give most of it back, without knowing anything about which code is dead. The --flush-bytecode flag is on by default precisely because a web server’s module graph is mostly cold.

What it costs.

  • A cliff, not a slope. The age check is a comparison against 180. A function is either flushed or it is not.
  • Re-compilation. A flushed function has to be re-parsed, re-compiled and re-optimized. For a big function with a lot of inlining, that is milliseconds.
  • The feedback is lost too. Even if you could keep the bytecode, throwing away the feedback means the next optimized code starts from scratch.
  • The pages may not come back. Because the bytecode is mutated in place, the memory it occupied usually stays allocated as filler until the page is swept for other reasons.

The bottom line

Code flushing in V8 is a wall-clock reclamation policy, not a garbage collection policy. The SharedFunctionInfo carries an age that is incremented by the seconds between GCs and reset to zero every time the function runs. When it passes bytecode_old_time, the bytecode is rewritten in place into an UncompiledData object, the feedback metadata is discarded, and every closure of that function is reset to CompileLazy.

The mental model to keep is that the layers degrade in order of cost. Optimized code is disposable and V8 does not even flush it explicitly - it just stops referencing it. Baseline code is disposable but V8 keeps it by default. Bytecode is the expensive one, because it is the only tier from which the tiers above can be rebuilt without going back to the source, and it is the only tier V8 will throw away on a timer.

The thing people get wrong

“Code flushing is why my code got slower.” Sometimes, but not for the reason people think. Flushing is not triggered by the function being rarely called; it is triggered by the function not having been called at all recently. A function you call in a tight loop is never flushed, no matter how many other functions are.

And: “flushing threw away my optimized code and I have to pay to get it back.” True, but that is not the expensive part. The expensive part is flushing the bytecode, which additionally throws away the feedback and forces a re-parse. If you see a function being re-optimized over and over, look at the gap between its calls, not its call count. The fix is to make the function stay warm - a smaller bytecode_old_time would make it flushed sooner, and a larger one keeps it around longer.

V8InternalsGcMemoryCompiler