Unverified Commit a51a2e4c authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

[turbopack] Support traversing the graph in reverse order (#86427)

This makes certain aggregations trivial since we are are guaranteed to only traverse relevant edges.

Use it to fix a bug in the async module identification logic.  Previously we would aggregate async cycles after propagating 'asyncness' through the DFS post order traversal, but this could cause us to fail to propagate async to all reverse dependencies depending on which part of a cycle a node would happen to visit first.

By traversing starting from the async modules we just need to mark everything we find and not worry about cycles at all since the underlying DFS mechanism will ensure we don't loop.  In addition to only requiring a single pass, this is guaranteed to visit fewer nodes in that single pass.

One caveat is that reverse traversals only work within a single `SingleModuleGraph` so a debug assert was added to prevent misuse

Closes #85988
Closes #86391
Closes PACK-5806
parent cf480fc1
......@@ -103,6 +103,19 @@ we attempt to capture traces of the playwright run to make debugging the failure
A test-trace artifact should be uploaded after the workflow completes which can be downloaded, unzipped,
and then inspected with `pnpm playwright show-trace ./path/to/trace`
To attach the chrome debugger to next the easiest approach is to modify the `createNext` call in your test to pass `--inspect` to next.
```js
const next = await createNext({
...
startArgs: =['--inspect'],
})
```
Consider also sett `NEXT_E2E_TEST_TIMEOUT=0`
To debug the test process itself you need to pass the `inspect` flag to the node process running jest. e.g. `IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 NEXT_TEST_MODE=dev node --inspect node_modules/jest/bin/jest.js ...`
### Profiling tests
Add `NEXT_TEST_TRACE=1` to enable test profiling. It's useful for improving our testing infrastructure.
......
use anyhow::Result;
use rustc_hash::FxHashSet;
use turbo_tasks::{ResolvedVc, TryJoinIterExt, Vc};
use turbo_tasks::{ResolvedVc, TryFlatJoinIterExt, Vc};
use crate::{
module::{Module, Modules},
......@@ -57,52 +57,53 @@ async fn compute_async_module_info_single(
let graph = graph.read().await?;
let self_async_modules = graph
.graphs
.first()
.unwrap()
.iter_nodes()
.map(async |node| Ok((node, *node.is_self_async().await?)))
.try_join()
.await?
.into_iter()
.flat_map(|(k, v)| v.then_some(k))
.chain(parent_async_modules.iter().copied())
.collect::<FxHashSet<_>>();
.enumerate_nodes()
.map(async |(_, node)| {
Ok(match node {
super::SingleModuleGraphNode::Module(node) => {
node.is_self_async().await?.then_some(*node)
}
super::SingleModuleGraphNode::VisitedModule { idx: _, module } => {
// If a module is async in the parent then we need to mark reverse dependencies
// async in this graph as well.
parent_async_modules.contains(module).then_some(*module)
}
})
})
.try_flat_join()
.await?;
// To determine which modules are async, we need to propagate the self-async flag to all
// importers, which is done using a postorder traversal of the graph.
//
// This however doesn't cover cycles of async modules, which are handled by determining all
// strongly-connected components, and then marking all the whole SCC as async if one of the
// modules in the SCC is async.
// importers, which is done using a reverse traversal over the graph
// Because we walk edges in the reverse direction we can trivially handle things like cycles
// without actually computing them.
let mut async_modules = FxHashSet::default();
async_modules.extend(self_async_modules.iter());
let mut async_modules = self_async_modules;
graph.traverse_edges_from_entries_dfs(
graph.graphs.first().unwrap().entry_modules(),
graph.traverse_edges_from_entries_dfs_reversed(
self_async_modules,
&mut (),
|_, _, _| Ok(GraphTraversalAction::Continue),
|parent_info, module, _| {
let Some((parent_module, ref_data)) = parent_info else {
// An entry module
return Ok(());
};
if ref_data.chunking_type.is_inherit_async() && async_modules.contains(&module) {
async_modules.insert(parent_module);
}
Ok(())
// child is the previously visited module which must be async
// parent is a new module that depends on it
|child, parent, _state| {
Ok(if let Some((_, edge)) = child {
if edge.chunking_type.is_inherit_async() {
async_modules.insert(parent);
GraphTraversalAction::Continue
} else {
// Wrong edge type to follow
GraphTraversalAction::Exclude
}
} else {
// These are our entry points, just continue
GraphTraversalAction::Continue
})
},
|_, _, _| Ok(()),
)?;
graph.traverse_cycles(
|ref_data| ref_data.chunking_type.is_inherit_async(),
|cycle| {
if cycle.iter().any(|node| async_modules.contains(node)) {
async_modules.extend(cycle.iter().map(|n| **n));
}
Ok(())
},
)?;
// Accumulate the parent modules at the end. Not all parent async modules were in this graph
async_modules.extend(parent_async_modules);
Ok(Vc::cell(async_modules))
}
import { B } from './B'
export function A(n: number) {
if (n > 0) {
B(n - 1)
}
}
import { C } from './C'
import { asyncImportFn } from './asyncImportFn'
export function B(n: number) {
if (n > 0) {
C(n - 1)
asyncImportFn()
}
}
import { A } from './A'
export function C(n: number) {
if (n > 0) {
A(n - 1)
}
}
import { C } from './C'
export function D(n: number) {
if (n > 0) {
C(n - 1)
}
}
import { B } from './B'
export function E(n: number) {
if (n > 0) {
B(n - 1)
}
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms))
}
await sleep(0)
console.log('Imported asyncImportFn')
export function asyncImportFn() {}
import { A } from './A'
import { D } from './D'
/*
* {A,B,C,D}.ts and asyncImportFn.ts have an import graph topology that
* exposes a bug in `compute_async_module_info_single` in turbopack. Requesting
* this page (localhost:3000/api/test) will fail with:
* TypeError: (0 , t.C) is not a function
* This is because C has been marked as an async module, but D hasn't.
*
* route
* | \
* v v
* A<- D
* | \ |
* v \v
* B--->C
* |
* v
* async
*/
it('should handle cycles in async modules', () => {
A(10)
D(10)
expect(true).toBe(true)
})
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment