Unverified Commit ef247bc8 authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

[turbopack] Remove per-shard modified counts from snapshots (#99955)

## Summary

Removes the per-shard modified counts from Turbopack's task storage.
Each snapshot now scans every shard of the task map instead of skipping
shards that report no modified tasks, and `track_modification` and its
undo no longer update a shared atomic counter.

The counter was there to solve two problems.

- **Skipping unmodified shards.** In a warm `next build` every snapshot
found modified tasks in all 4096 shards, so the counts never skipped
one. In a cold build every task is modified anyway.
- **Making a no-op snapshot cheap.** The persistence loop only snapshots
after enough foreground work has accumulated (1s by default), so a full
scan always follows real work. That makes it unlikely to be a no-op, and
limits how often it can run. Plus in dev `eviction` means unmodified
tasks will be removed from the map, so it will tend to be the case that
resident tasks are dirty

## Verification

Warm `next build` of vercel-site over 21 commits, 2 repeats, GC on. Base
is #99902's head, and the two arms differ only in this change. Both were
built with `build-native --release`, installed through `pack-next`, and
interleaved per commit. Snapshot phases were timed with a bench-only log
that isn't part of this PR.

| warm (40 pairs) | with counts | without | delta [95% CI] |
|---|---|---|---|
| wall | 52.26s | 52.59s | +0.33s [-1.09, +1.75] |
| user CPU | 155.0s | 154.0s | -1.0s [-5.3, +3.4] |
| snapshot capture | 177.9ms | 173.2ms | -4.7ms [-18.9, +9.5] |
| max RSS | 10.34 GB | 10.38 GB | |

Every interval includes zero. Tests that asserted on the counts were
reduced to the behaviour they still describe.

<!-- NEXT_JS_LLM -->
parent 5b370524
......@@ -1253,18 +1253,10 @@ impl TurboTasksBackend {
debug_assert!(self.should_persist());
// Checking after the exclusion begins ensures no concurrent increments can race.
let (snapshot_guard, has_modifications) = self.storage.start_snapshot();
let snapshot_guard = self.storage.start_snapshot();
let snapshot_time = Instant::now();
if !has_modifications && gc_roots_to_persist.is_none() {
// No tasks modified since the last snapshot — drop the guard (which
// calls end_snapshot) and skip the expensive O(N) scan.
drop(snapshot_guard);
return Ok(Some((start, false, gc_outcome)));
}
#[cfg(feature = "print_cache_item_size")]
#[derive(Default)]
struct TaskCacheStats {
......@@ -1474,9 +1466,8 @@ impl TurboTasksBackend {
let task_count = task_snapshots.len();
if task_snapshots.is_empty() && gc_roots_to_persist.is_none() {
// Rare: the modified counts are out of sync with the modified persistable tasks, which
// only happens if every modified task was both new and deleted.
std::hint::cold_path();
// No task was modified since the last snapshot (or every modified task was both new
// and deleted), so there is nothing to write.
return Ok(Some((snapshot_time, false, gc_outcome)));
}
......
......@@ -2550,8 +2550,7 @@ mod cell_data_tracking_tests {
.unwrap()
};
let (snapshot_guard, has_modifications) = storage.start_snapshot();
assert!(has_modifications);
let snapshot_guard = storage.start_snapshot();
let process =
|_: TaskId,
_: &TaskStorage,
......@@ -2620,8 +2619,7 @@ mod cell_data_tracking_tests {
.unwrap()
};
let (snapshot_guard, has_modifications) = storage.start_snapshot();
assert!(has_modifications);
let snapshot_guard = storage.start_snapshot();
// Encodes the live state, like the backend does for tasks that weren't copied.
let process = |task_id: TaskId,
task: &TaskStorage,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment