Unverified Commit bf53f53c authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Turbopack: trace static project files referenced by packages (#98712)

### What?

Allow Turbopack's native asset tracer to follow fully static
application-root file references originating in packages, while
continuing to block dynamic package references from traversing the
application.

Patterns containing alternatives retain only their fully static branches
for project-root tracing. Branches containing any dynamic segment are
discarded. A focused node-file-trace fixture covers a package loading an
application-owned config through `path.resolve()` and `fs.existsSync()`.

### Why?

Turbopack currently redirects every working-directory-relative reference
from `node_modules` to the package directory. This protects builds from
dynamic package expressions accidentally tracing much or all of the
application, but it also prevents safe static configuration lookups such
as `next-i18next.config.js` from reaching standalone output.

The dynamic safeguard should remain, but known static files do not carry
the same risk and should follow runtime working-directory semantics.

Related to #98628.

### How?

Add a pattern-level operation that extracts fully static alternatives
from a normalized reference pattern. The tracing root selection uses
that filtered pattern to choose the configured project working
directory; dynamic-only patterns retain the existing package-directory
fallback.

The shared root-selection path applies this consistently to filesystem,
path, worker, child-process, and specialized traced references. Existing
outside-base and `node_modules` filtering tests continue to cover the
safeguard.

### Verification

- `cargo fmt -- --check`
- `cargo test -p turbopack-core resolve::pattern`
- `cargo test -p turbopack-tracing --test unit`
- `git diff --check`

<!-- NEXT_JS_LLM -->


<!-- fleet fe000a60-fbb8-4835-b6ef-38d5684822a9 -->
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 28f84cea
...@@ -150,6 +150,28 @@ impl Pattern { ...@@ -150,6 +150,28 @@ impl Pattern {
} }
} }
/// Returns the alternatives that contain no dynamic parts.
///
/// The pattern is expected to be normalized so that alternatives are at the top level.
pub fn filter_static(&self) -> Option<Pattern> {
if let Pattern::Alternatives(list) = self {
let mut static_alternatives = list
.iter()
.filter(|alternative| !alternative.has_dynamic_parts())
.cloned()
.collect::<Vec<_>>();
match static_alternatives.len() {
0 => None,
1 => static_alternatives.pop(),
_ => Some(Pattern::Alternatives(static_alternatives)),
}
} else if self.has_dynamic_parts() {
None
} else {
Some(self.clone())
}
}
pub fn constant_prefix(&self) -> &str { pub fn constant_prefix(&self) -> &str {
// The normalized pattern is an Alternative of maximally merged // The normalized pattern is an Alternative of maximally merged
// Concatenations, so extracting the first/only Concatenation child // Concatenations, so extracting the first/only Concatenation child
...@@ -2029,6 +2051,43 @@ mod tests { ...@@ -2029,6 +2051,43 @@ mod tests {
} }
} }
#[test]
fn filter_static() {
let static_a = Pattern::Constant(rcstr!("./next-i18next.config.js"));
let static_b = Pattern::Constant(rcstr!("./i18next.config.js"));
assert_eq!(static_a.filter_static(), Some(static_a.clone()));
assert_eq!(Pattern::Dynamic.filter_static(), None);
assert_eq!(Pattern::DynamicNoSlash.filter_static(), None);
let pattern = Pattern::Alternatives(vec![
static_a.clone(),
static_b.clone(),
Pattern::Dynamic,
Pattern::Concatenation(vec![Pattern::Dynamic, Pattern::Constant(rcstr!("/suffix"))]),
Pattern::Concatenation(vec![
Pattern::Constant(rcstr!("/prefix/")),
Pattern::Dynamic,
]),
]);
assert_eq!(
pattern.filter_static(),
Some(Pattern::Alternatives(vec![static_a, static_b]))
);
assert_eq!(
Pattern::Alternatives(vec![
Pattern::Dynamic,
Pattern::Concatenation(vec![
Pattern::Constant(rcstr!("/prefix/")),
Pattern::Dynamic,
]),
])
.filter_static(),
None
);
}
#[test] #[test]
fn with_normalized_path() { fn with_normalized_path() {
assert!( assert!(
......
...@@ -2032,23 +2032,21 @@ where ...@@ -2032,23 +2032,21 @@ where
ResolveErrorMode::Error ResolveErrorMode::Error
}; };
let get_traced_project_dir = async || -> Result<FileSystemPath> { let get_traced_project_dir = async |pattern: Pattern| -> Result<(FileSystemPath, Pattern)> {
// readFileSync("./foo") should always be relative to the project root, but this is // Relative filesystem references are resolved from the project root at runtime. Inside
// dangerous inside of node_modules as it can cause a lot of false positives in the // node_modules, only trace fully static alternatives from there: a dynamic pattern could
// tracing, if some package does `path.join(dynamic)`, it would include // otherwise include much or all of the project. Dynamic-only patterns continue to fall
// everything from the project root as well. // back to the package directory, preserving package-local tracing without exposing the
// // project root.
// Also, when there's no cwd set (i.e. in a tracing-specific module context, as we if let Some(cwd) = compile_time_info.environment().cwd().owned().await? {
// shouldn't assume a `process.cwd()` for all of node_modules), fallback to if state.allow_project_root_tracing {
// the source file directory. This still allows relative file accesses, just return Ok((cwd, pattern));
// not from the project root. }
if state.allow_project_root_tracing if let Some(static_pattern) = pattern.filter_static() {
&& let Some(cwd) = compile_time_info.environment().cwd().owned().await? return Ok((cwd, static_pattern));
{
Ok(cwd)
} else {
Ok(source.ident().await?.path.parent())
} }
}
Ok((source.ident().await?.path.parent(), pattern))
}; };
let get_issue_source = let get_issue_source =
...@@ -2224,13 +2222,13 @@ where ...@@ -2224,13 +2222,13 @@ where
}; };
// WorkerThreads resolve URLs relative to import.meta.url // WorkerThreads resolve URLs relative to import.meta.url
// and string paths relative to the process root // and string paths relative to the process root
let context_dir = if matches!( let (context_dir, pat) = if matches!(
args.first(), args.first(),
Some(JsValue::Url(_, JsValueUrlKind::Relative)) Some(JsValue::Url(_, JsValueUrlKind::Relative))
) { ) {
origin.into_trait_ref().await?.origin_path().parent() (origin.into_trait_ref().await?.origin_path().parent(), pat)
} else { } else {
get_traced_project_dir().await? get_traced_project_dir(pat).await?
}; };
analysis.add_reference_code_gen( analysis.add_reference_code_gen(
WorkerAssetReference::new_node_worker_thread( WorkerAssetReference::new_node_worker_thread(
...@@ -2563,9 +2561,10 @@ where ...@@ -2563,9 +2561,10 @@ where
return Ok(()); return Ok(());
} }
} }
let (context_dir, pat) = get_traced_project_dir(pat).await?;
analysis.add_reference( analysis.add_reference(
FileSourceReference::new( FileSourceReference::new(
get_traced_project_dir().await?, context_dir,
Pattern::new(pat), Pattern::new(pat),
collect_affecting_sources, collect_affecting_sources,
get_issue_source(), get_issue_source(),
...@@ -2600,9 +2599,10 @@ where ...@@ -2600,9 +2599,10 @@ where
return Ok(()); return Ok(());
} }
} }
let (context_dir, pat) = get_traced_project_dir(pat).await?;
analysis.add_reference( analysis.add_reference(
DirAssetReference::new( DirAssetReference::new(
get_traced_project_dir().await?, context_dir,
Pattern::new(pat), Pattern::new(pat),
get_issue_source(), get_issue_source(),
rcstr!("fs.readdir"), rcstr!("fs.readdir"),
...@@ -2657,9 +2657,10 @@ where ...@@ -2657,9 +2657,10 @@ where
return Ok(()); return Ok(());
} }
} }
let (context_dir, pat) = get_traced_project_dir(pat).await?;
analysis.add_reference( analysis.add_reference(
DirAssetReference::new( DirAssetReference::new(
get_traced_project_dir().await?, context_dir,
Pattern::new(pat), Pattern::new(pat),
get_issue_source(), get_issue_source(),
rcstr!("path.resolve"), rcstr!("path.resolve"),
...@@ -2709,9 +2710,10 @@ where ...@@ -2709,9 +2710,10 @@ where
return Ok(()); return Ok(());
} }
} }
let (context_dir, pat) = get_traced_project_dir(pat).await?;
analysis.add_reference( analysis.add_reference(
DirAssetReference::new( DirAssetReference::new(
get_traced_project_dir().await?, context_dir,
Pattern::new(pat), Pattern::new(pat),
get_issue_source(), get_issue_source(),
rcstr!("path.join"), rcstr!("path.join"),
...@@ -2771,9 +2773,10 @@ where ...@@ -2771,9 +2773,10 @@ where
show_dynamic_warning = true; show_dynamic_warning = true;
} }
if !dynamic || !ignore_dynamic_requests { if !dynamic || !ignore_dynamic_requests {
let (context_dir, pat) = get_traced_project_dir(pat).await?;
analysis.add_reference( analysis.add_reference(
FileSourceReference::new( FileSourceReference::new(
get_traced_project_dir().await?, context_dir,
Pattern::new(pat), Pattern::new(pat),
collect_affecting_sources, collect_affecting_sources,
IssueSource::from_swc_offsets( IssueSource::from_swc_offsets(
...@@ -3013,9 +3016,11 @@ where ...@@ -3013,9 +3016,11 @@ where
.await?; .await?;
js_value_to_pattern(&linked_func_call) js_value_to_pattern(&linked_func_call)
}; };
let (context_dir, abs_pattern) =
get_traced_project_dir(abs_pattern).await?;
analysis.add_reference( analysis.add_reference(
DirAssetReference::new( DirAssetReference::new(
get_traced_project_dir().await?, context_dir,
Pattern::new(abs_pattern), Pattern::new(abs_pattern),
get_issue_source(), get_issue_source(),
rcstr!("express().set"), rcstr!("express().set"),
...@@ -3085,9 +3090,10 @@ where ...@@ -3085,9 +3090,10 @@ where
.await?; .await?;
js_value_to_pattern(&linked_func_call) js_value_to_pattern(&linked_func_call)
}; };
let (context_dir, abs_pattern) = get_traced_project_dir(abs_pattern).await?;
analysis.add_reference( analysis.add_reference(
DirAssetReference::new( DirAssetReference::new(
get_traced_project_dir().await?, context_dir,
Pattern::new(abs_pattern), Pattern::new(abs_pattern),
get_issue_source(), get_issue_source(),
rcstr!("strong-globalize.SetRootDir"), rcstr!("strong-globalize.SetRootDir"),
...@@ -3143,7 +3149,10 @@ where ...@@ -3143,7 +3149,10 @@ where
if args.len() == 2 if args.len() == 2
&& let Some(JsValue::Object { parts, .. }) = args.get(1) && let Some(JsValue::Object { parts, .. }) = args.get(1)
{ {
let context_dir = get_traced_project_dir().await?; // Every `includeDirs` entry below is a constant, so the traced directory is the
// same for all of them and can be computed once from a static pattern.
let (context_dir, _) =
get_traced_project_dir(Pattern::Constant(rcstr!(""))).await?;
let resolved_dirs = parts let resolved_dirs = parts
.iter() .iter()
.filter_map(|object_part| match object_part { .filter_map(|object_part| match object_part {
......
const fs = require('fs')
const path = require('path')
let configPath = './app-config.js'
if (process.env.CONFIG_PATH) {
configPath = process.env.CONFIG_PATH
}
fs.existsSync(path.resolve(configPath))
;[
'package.json',
'test/unit/pkg-project-root-static/app-config.js',
'test/unit/pkg-project-root-static/input.js',
'test/unit/pkg-project-root-static/node_modules/some-pkg/index.js',
'test/unit/pkg-project-root-static/node_modules/some-pkg/package.json',
]
...@@ -173,6 +173,7 @@ static ALLOC: turbo_tasks_malloc::TurboMalloc = turbo_tasks_malloc::TurboMalloc; ...@@ -173,6 +173,7 @@ static ALLOC: turbo_tasks_malloc::TurboMalloc = turbo_tasks_malloc::TurboMalloc;
// #[case::pixelmatch("pixelmatch")] // #[case::pixelmatch("pixelmatch")]
#[case::pkg_dir_outside_base("pkg-dir-outside-base")] #[case::pkg_dir_outside_base("pkg-dir-outside-base")]
#[case::pkg_file_outside_base("pkg-file-outside-base")] #[case::pkg_file_outside_base("pkg-file-outside-base")]
#[case::pkg_project_root_static("pkg-project-root-static")]
// #[case::pkginfo("pkginfo")] // #[case::pkginfo("pkginfo")]
// #[case::pnpm_symlinks("pnpm-symlinks")] // #[case::pnpm_symlinks("pnpm-symlinks")]
// #[case::prisma_photon("prisma-photon")] // #[case::prisma_photon("prisma-photon")]
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment