turbopack: skip redundant work in merged modules reconciliation (#100238)
### What? Removes redundant work from the reconciliation step of `compute_merged_modules`. This is the third of three stacked PRs; it is based on #100237. 1. intern bitmaps (#100236) 2. parallel merge of the chunk group lists (#100237) 3. reconciliation shortcuts (this PR) ### Why? On the bench app below, only 73 modules actually lead to a split. Each split still repeated work proportional to the common list once per occurrence: ~18.9 M `swap_remove` calls, plus ~19.1 M items inserted into the before/common/after hash sets. All of that ran on one thread for ~1 s. ### How? - **Occurrence updates:** - **Before:** each later module of the common list had its occurrences removed one by one, then one new occurrence was added. - **Now:** they are cleared instead. - **Why it's safe:** every occurrence of such a module lies in the common sequence. Modules in one list share a merge bitmap, so they are in the same chunk groups. - **Check:** `debug_assert!`s check that the set has exactly as many entries as there are occurrences being replaced, and that all of them are present. - **Evidence:** counters across the 54 scope-hoisting fixtures (including the circular ones) and ~120 generated graphs (cycles, CJS, 2–5 entries) never found any other occurrences. - **No release fallback:** this is a deliberate choice. - **Exposed modules:** when a split has no "before" and no "after" list, skip computing the modules to expose. Every reference filter requires membership in one of those lists, so the result was always empty. ### Results Same setup as #100236 and #100237: - **Bench app:** `bench/nested-deps-app-router-many-pages`. - **Builds:** cold `next build` at 32 CPUs, with release binaries built at the base commit with #100236 and #100237, and with this change on top. - **Rounds:** 1 warm-up per arm, then 4 rounds in alternating order. - **Phase times:** from the `overview` light trace. | Metric (s) | #100237 median | this PR median | Median paired change | Faster in | |---|---:|---:|---:|---:| | Wall | 50.50 | 49.60 | −1.5% | 3/4 | | "Compiled in" | 30.00 | 28.80 | −2.0% | 4/4 | | `compute merged modules` | 2.57 | 1.78 | −30.5% | 4/4 | | `map chunk groups` | 0.50 | 0.50 | −1.4% | 2/4 | | `merging chunk group lists` | 0.12 | 0.12 | −0.3% | 2/4 | | `reconciliation` | 1.00 | 0.25 | −75.3% | 4/4 | <details><summary>Per round (#100237 → this PR)</summary> | Round | Order | Wall | "Compiled in" | `compute merged modules` | `map chunk groups` | `merging chunk group lists` | `reconciliation` | |---|---|---:|---:|---:|---:|---:|---:| | 1 | #100237 first | 50.44 → 49.37 | 29.80 → 29.20 | 2.51 → 1.58 | 0.46 → 0.50 | 0.11 → 0.12 | 1.01 → 0.24 | | 2 | this PR first | 52.67 → 49.83 | 30.70 → 28.10 | 2.69 → 1.80 | 0.57 → 0.51 | 0.13 → 0.12 | 0.97 → 0.25 | | 3 | #100237 first | 47.67 → 48.25 | 29.00 → 28.40 | 2.42 → 1.77 | 0.47 → 0.48 | 0.11 → 0.11 | 1.00 → 0.26 | | 4 | this PR first | 50.56 → 50.11 | 30.20 → 30.10 | 2.63 → 1.90 | 0.54 → 0.51 | 0.12 → 0.11 | 1.02 → 0.25 | </details> - **Limits:** one VM, one app, 4 rounds, and no significance test. The wall-time change is within run-to-run noise. - **Whole stack:** across all three PRs, `compute merged modules` goes from ~6.6 s to ~1.8 s on this app. Each PR was measured separately against its parent, so the percentages don't compound exactly. - **Output:** the `.next` output is identical to #100237's. Only the per-build random preview-mode and server-action keys differ. ### Verification - `cargo test -p turbopack-tests --test snapshot --test execution -- scope_hoisting` (2 snapshot and 52 execution fixtures, with unchanged snapshots). These ran as debug builds, so the new assertions were checked. - `cargo test -p turbopack-core` - `cargo clippy -p turbopack-core --all-targets -- -D warnings` - `pnpm build-all` on this branch. <!-- NEXT_JS_LLM --> <!-- fleet d8fee2f9-18db-4936-b59a-f8417b8f4bbc --> Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
Showing
Please
register
or
sign in
to comment