Unverified Commit fa3830c4 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

turbopack: use reverse postorder priority in compute_binding_usage_info (#100440)

Stacked on #100430 (#100428 → #100429 → #100430 → this).

### What?

Two changes in `compute_binding_usage_info`:

1. **Reverse postorder priority.** The fixed-point traversal now uses
`Reverse(rank)` from `ModuleGraphSnapshot::reverse_postorder_ranks`
(added in #100428) as its priority, instead of the constant `0`. Modules
are visited roots first, so in an acyclic graph each module is processed
exactly once, after all of its importers and with its final usage.
Cycles still rely on the fixed point.
2. **No clone of the parent's usage on passthrough edges.** The
`ExportUsage::Passthrough` branch cloned the parent's
`ModuleExportUsageInfo` to work around borrowing two entries of
`used_exports` at once. It now inserts the target's entry, borrows both
entries with `HashMap::get_disjoint_mut`, and merges directly. A
self-edge (`parent == target`) is handled explicitly, because merging a
usage into itself never changes it and `get_disjoint_mut` would panic on
the duplicate key.

### Why?

With the constant priority, the traversal pops the most recently queued
node first, which makes the order DFS-like. That gives no ordering
guarantee: a shared module can be processed before all of its importers,
and is then revisited whenever a later importer adds new export names.

On the apps measured below this happens rarely, so there is no speedup
here. The extra rank traversal makes the analysis **slower** on both
apps. The change is meant as protection against worst cases in apps
where shared modules are imported with different names at many different
depths. In those, the number of revisits is no longer order-dependent on
DAG-shaped graphs.

### Benchmarks

Real Turbopack `next build --experimental-build-mode compile` runs with
release `next-swc` builds (LTO off, 16 codegen units). Builds were
interleaved on the same machine, 3 runs each. Times are the
binding-usage analysis (rank pass + fixed point), summed over both
invocations per build: median [min–max].

| | base (#100430) | + RPO priority | + clone removal (this PR) |
|---|---|---|---|
| generated app router app, 300 routes | 11.2 ms [10.8–11.7] | 14.7 ms
[13.2–16.0] | 14.1 ms [13.4–15.7] |
| `bench/nested-deps-app-router-many-pages` | 36.6 ms [35.5–37.6] | 52.4
ms [51.0–68.8] | 55.7 ms [52.9–57.3] |

| | visits (base → this PR) | rank pass (median) |
|---|---|---|
| generated app | 27,128 → 24,228 | 4.7 ms |
| many-pages | 64,894 → 64,606 | 14.2 ms |

- **Revisits:** RPO removes almost all of them; 25 remain, inside or
right after cycles. Under the constant priority, though, 88% and 99.6%
of modules were already visited only once, so there is little to save.
- **Rank pass:** it is a near-full extra traversal, and it costs more
than the saved revisits on both apps.
- **Clone removal:** no measurable difference within noise on these
apps.

### How was it verified?

- `cargo test -p turbopack-core --lib`, `cargo test -p turbopack-tests
--test snapshot` (no snapshot changes) and `cargo test -p
turbopack-tests --test execution` all pass.
- `cargo clippy -p turbopack-core --all-targets -- -D warnings` and
`RUSTDOCFLAGS='-D warnings' cargo doc -p turbopack-core --no-deps` are
clean.
- Output equivalence on both benchmark apps: I compared a local, not
committed, fingerprint of the analysis result between the base and this
PR, covering per-module used exports, partial-namespace modules and
unused-reference targets. It is identical for both invocations.

<!-- NEXT_JS_LLM -->


<!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 2e133c98
use std::collections::hash_map::Entry; use std::{cmp::Reverse, collections::hash_map::Entry};
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use auto_hash_map::AutoSet; use auto_hash_map::AutoSet;
...@@ -165,10 +165,23 @@ pub async fn compute_binding_usage_info( ...@@ -165,10 +165,23 @@ pub async fn compute_binding_usage_info(
None None
}; };
let entries = graph_ref.all_chunk_group_entry_modules(); // Visit modules in reverse postorder. In an acyclic graph, this processes each module only
// after all modules importing it, i.e. once, with its final usage. Cycles still rely on the
// fixed point and may revisit modules in or after them. This avoids order-dependent
// revisits in DAG-shaped graphs, at the cost of an additional traversal to compute ranks.
let module_rank =
graph_ref.reverse_postorder_ranks(graph_ref.all_chunk_group_entry_modules())?;
let visit_count = graph_ref.traverse_edges_fixed_point_with_priority( let visit_count = graph_ref.traverse_edges_fixed_point_with_priority(
entries.map(|m| (m, 0)), graph_ref
.all_chunk_group_entry_modules()
.map(|m| {
Ok((
m,
Reverse(*module_rank.get(&m).context("Module rank not found")?),
))
})
.collect::<Result<Vec<_>>>()?,
&mut (), &mut (),
|parent, target, _, _| { |parent, target, _, _| {
// Entries are always used // Entries are always used
...@@ -279,14 +292,19 @@ pub async fn compute_binding_usage_info( ...@@ -279,14 +292,19 @@ pub async fn compute_binding_usage_info(
} else { } else {
false false
}; };
let passthrough_usage = used_exports let usage_changed = if parent == target {
.get(&parent) // Merging the usage of a module into itself never changes it (and
.context("parent module must have usage info")? // `get_disjoint_mut` would panic on the duplicate key).
.clone(); false
let usage_changed = used_exports } else {
.entry(target) used_exports.entry(target).or_default();
.or_default() let [Some(parent_usage), Some(target_usage)] =
.add_usage_info(&passthrough_usage); used_exports.get_disjoint_mut([&parent, &target])
else {
bail!("parent module must have usage info");
};
target_usage.add_usage_info(parent_usage)
};
namespace_changed || usage_changed namespace_changed || usage_changed
} }
export_usage => { export_usage => {
...@@ -313,7 +331,13 @@ pub async fn compute_binding_usage_info( ...@@ -313,7 +331,13 @@ pub async fn compute_binding_usage_info(
Ok(GraphTraversalAction::Skip) Ok(GraphTraversalAction::Skip)
} }
}, },
|_, _| Ok(0), |successor, _| {
Ok(Reverse(
*module_rank
.get(&successor)
.context("Module rank not found")?,
))
},
)?; )?;
// Compute cycles and select modules to be 'circuit breakers' // Compute cycles and select modules to be 'circuit breakers'
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment