turbopack: speed up and share reverse postorder ranks (#100452)
Stacked on #100440. The lower PRs of the stack (#100428, #100429, #100430) are merged. ### What? Two changes to the reverse postorder ranks that the RPO stack computes before each fixed-point traversal: 1. **A faster `ModuleGraphSnapshot::reverse_postorder_ranks`, with identical ranks.** - Visited nodes are tracked in one `FixedBitSet` per graph instead of an `FxHashSet<GraphNodeIndex>`. `DiGraph` node indices are dense, and `VisitedModule` aliases are resolved first. - `postorder` is preallocated, and the result map is created with capacity. Collecting a `Result` iterator into a map gives it no capacity hint, so it rehashes as it grows. - One DFS stack is reused across roots. - DFS order and rank numbering are unchanged. - The method now asserts it's called on the whole graph, not on an individual graph layer, where `VisitedModule` nodes can't be resolved. This matches the existing check in `traverse_edges_fixed_point_with_priority`. 2. **One shared rank computation for chunk group info and merged modules.** - The new `ModuleGraphSnapshot::entry_ranks` computes the ranks from the entries of the graph (derived from its chunk groups) on first use and caches them in a `OnceLock` on the snapshot. - `compute_chunk_group_info` and `compute_merged_modules` already computed exactly these ranks (same entries, same order) on the same graph, so they now share one computation. ### Why? The RPO stack adds a rank pass in front of each fixed-point analysis, 5 per build. This removes most of that added cost. The cache lives on the immutable `ModuleGraph` snapshot that the analyses already read through the shared `ReadRef`: - the ranks always belong to exactly the graph being traversed; - there is no extra turbo-tasks function or cell, and no second, separately versioned read inside a task; - there is nothing extra to invalidate, since a changed graph is a new snapshot with an empty cache; - the ranks are computed inside `OnceLock::get_or_init`, so concurrent cold callers wait for one computation instead of racing; - an error is cached too (as a cloneable `SharedError`) and returned to every caller of that snapshot. Not shared: - **`collect_graph`** roots its DFS only at `Entry` chunk groups. Sharing all-entry ranks would change its order on cyclic graphs, so it keeps computing its own ranks (with the faster implementation). - **`compute_binding_usage_info`** runs on different graphs, before unused references are removed, so it keeps computing its own ranks as well. ### Benchmarks > **Historical, not rerun:** measured in the previous review round, before this stack was rebased onto canary `502b6b7d` and before the latest review changes. This round only reran the `turbo-bitset` microbenchmarks in #100566. Real Turbopack `next build --experimental-build-mode compile` with release `next-swc` builds (LTO off, 16 codegen units). #100440 and this PR were each built with the same local timing instrumentation and run 5 times per app, interleaved in randomized order. Times are medians [min–max], summed over all rank computations of a build: | | rank passes per build | #100440 | this PR | |---|---|---|---| | generated app router app, 300 routes | 5 → 4 | 21.1 ms [20.0–23.6] | 14.2 ms [12.7–15.5] | | `bench/nested-deps-app-router-many-pages` | 5 → 4 | 67.9 ms [58.8–70.7] | 25.5 ms [25.1–32.5] | Per caller (many-pages, median): | | binding usage (2×) | chunk group info | collect | merged modules | |---|---|---|---|---| | #100440 | 26.6 ms | 12.4 ms | 11.6 ms | 12.0 ms | | this PR | 12.7 ms | 6.5 ms (shared) | 6.4 ms | cached | - **Visit counts:** identical between both versions, for every fixed-point analysis, in all runs. - **Fixed-point time:** expected to be unchanged. The measured differences are within the run-to-run noise. - **Caveats:** - These are sums of rank-helper timers, not wall-clock savings. - This machine was slower than in the earlier PRs of the stack, so absolute numbers aren't comparable with those PRs. ### How was it verified? - **Exact ranks:** while developing, the previous implementation was kept as a reference and compared against the new one on every call, with `assert_eq!` on the rank maps. The comparison ran across the `turbopack-core` lib tests, the snapshot and execution suites, and real builds of both benchmark apps (4 rank computations per build). There were no mismatches. The reference was removed afterwards. - **New unit tests:** - the shared ranks equal `reverse_postorder_ranks(all_chunk_group_entry_modules())` on a graph with a diamond and a cycle, and a second call returns the same map; - a new snapshot of an edited graph gets fresh ranks, including the new module, while the old snapshot keeps its cached map; - calling `reverse_postorder_ranks` on a graph layer panics. - **Suites and checks:** `cargo test -p turbopack-core --lib`, `cargo test -p turbopack-tests --test snapshot` (no snapshot changes) and `cargo test -p turbopack-tests --test execution` pass. `cargo clippy -p turbopack-core --all-targets -- -D warnings` and `RUSTDOCFLAGS='-D warnings' cargo doc -p turbopack-core --no-deps` are clean. - **Real builds:** the emitted chunks of both benchmark apps are byte-identical to #100440 (sha256 over all JS files in `.next/server/chunks` and `.next/static/chunks`). <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com>
Showing
This diff is collapsed.
Please
register
or
sign in
to comment