Unverified Commit f0c1ffc4 authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

Remove ineffective turbo-tasks (#91341)

## Remove ineffective turbo-tasks

Identifies and removes turbo-tasks functions where the task overhead exceeds the value they provide. Each turbo-task carries ~4-6μs execution overhead per miss and ~200-500ns per cache hit, plus allocations and bookkeeping.

### What?

Removes 22 `#[turbo_tasks::function]` implementations across resolve plugins, chunk items, and resolve-result helpers — converting them to plain methods or inlining their work. Changes fall into a few buckets:

- **ResolvePlugin condition handling** (`AfterResolvePluginCondition::matches`, `BeforeResolvePluginCondition::matches`, `after_resolve_condition`, `before_resolve_condition`): conditions now store the resolved `Glob` as a `ReadRef<Glob>` on the plugin struct at construction, so `matches` is a pure sync function and the per-plugin `*_resolve_condition` getters are trivial field reads (no longer turbo-tasks). The `after_resolve` / `before_resolve` hooks themselves stay as `#[turbo_tasks::function]` — they synthesize virtual sources/modules and need memoization on `(self, lookup_path, reference_type, request)` to avoid distinct cells producing duplicate module-graph idents.
    - The basic theory here is that the right level of caching is at `resolve` and at the hook bodies themselves, not the conditions or condition getters.
    - `AfterResolvePluginCondition` and `BeforeResolvePluginCondition` are marked `serialization = "none"` because `ReadRef` cannot be persisted; plugin construction is cheap enough to re-derive on restore.
- **ChunkItem trait methods** (`chunking_context`, `ty`, `content_with_async_module_info`): returned constants or simple field reads, zero cache hits and no `.await` calls (no invalidation value).
- **ResolveResult / ModuleResolveResult helpers** (`primary_modules`, `first_module`, `first_source`, `primary_sources`, `is_unresolvable`, `primary_output_assets`): simple iterators over already-resolved data; converted to plain methods. Added a `Duplicate(usize)` variant to `ModuleResolveResultItem` to handle dedup at construction time instead of in a separate task.
    - The basic idea here is that it is reasonable to consume `ResolveResult/ModuleResolveResult` monolithically, and we get little to no benefit from fine grained access. e.g. `is_unresolved()` in theory that is a valuable turbotask, but since it rarely changes but generally if we change how we resolve an import then we have to regenerate code, so saving a few boolean conditions is unlikely to be very valuable.
- Misc: `EcmascriptModuleAsset::analyze`, `is_types_resolving_enabled`, `next_server::resolve::condition`.

### Impact (vercel-site build, dev first-compile)

| Metric | Before | After | Δ |
|---|---:|---:|---:|
| Total cache hits | 30,885,827 | 29,201,314 | −1,684,513 |
| Total cache misses | 6,473,123 | 5,953,626 | **−519,497** |
| Overall hit rate | 82.67% | 83.06% | +0.39 pp |
| Registered task functions | 1,294 | 1,272 | −22 |

The 22 removed tasks were collectively responsible for ~519K misses per build — each miss previously paying the full execution overhead. Most of the work from `EcmascriptModuleAsset::analyze` naturally migrated into `analyze_ecmascript_module` (the task it was wrapping; +129K hits there).

### On-disk cache size (persistent caching)

Each removed task also stops allocating cache cells on disk. Measured on the same vercel-site build with `.next/cache/turbopack` (persistent cache enabled):

| | Size |
|---|---:|
| canary | 2.56 GiB |
| this branch | 2.46 GiB |
| **saved** | **~100 MiB (−3.81%)** |

### Build-time wall clock and peak memory

Ran `pnpm next build --experimental-build-mode=compile` 5 times on each branch

**Peak RSS — clear reduction:**

| | canary | branch | Δ |
|---|---:|---:|---:|
| min | 19.18 GiB | 18.94 GiB | |
| **median** | **19.22 GiB** | **19.01 GiB** | **−217 MiB (−1.10%)** |
| mean | 19.21 GiB | 19.02 GiB | −199 MiB (−1.01%) |
| max | 19.23 GiB | 19.13 GiB | |

Every branch run has lower RSS than every canary run — the distributions don't overlap. Welch's t = −6.03.

**Wall time — no measurable change:**

| | canary | branch | Δ |
|---|---:|---:|---:|
| min | 62.03s | 60.78s | |
| **median** | **62.61s** | **62.65s** | **+0.04s (+0.06%)** |
| mean | 62.83s | 63.80s | +0.96s (+1.53%) |
| max | 64.25s | 68.23s | |
| stddev | 0.84s | 3.42s | |

Median is flat. The mean difference is within noise (Welch's t = +0.61, n = 5). Branch run-to-run variance is higher — one 68.23s outlier pulls the mean up — so this is neither a regression nor a measurable speedup at this sample size.

<!-- NEXT_JS_LLM_PR -->
parent e83bc939
...@@ -860,7 +860,6 @@ impl AppProject { ...@@ -860,7 +860,6 @@ impl AppProject {
None, None,
ResolveErrorMode::Error, ResolveErrorMode::Error,
) )
.to_resolved()
.await? .await?
.first_module() .first_module()
.await? .await?
......
...@@ -242,10 +242,11 @@ impl ServerNftJsonAsset { ...@@ -242,10 +242,11 @@ impl ServerNftJsonAsset {
None, None,
ResolveErrorMode::Error, ResolveErrorMode::Error,
) )
.await?
.primary_modules() .primary_modules()
.await? .await?
.into_iter() .into_iter()
.map(|m| **m)) .map(|m| *m))
}) })
.try_flat_join() .try_flat_join()
.await?, .await?,
......
...@@ -566,6 +566,7 @@ impl PagesProject { ...@@ -566,6 +566,7 @@ impl PagesProject {
None, None,
) )
.await? .await?
.await?
.first_module() .first_module()
.await? .await?
.context("expected Next.js client runtime to resolve to a module")?; .context("expected Next.js client runtime to resolve to a module")?;
......
...@@ -40,7 +40,6 @@ impl RuntimeEntry { ...@@ -40,7 +40,6 @@ impl RuntimeEntry {
None, None,
ResolveErrorMode::Error, ResolveErrorMode::Error,
) )
.to_resolved()
.await? .await?
.primary_modules() .primary_modules()
.await?; .await?;
......
use std::{io::Write, iter::once}; use std::{io::Write, iter::once};
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use async_trait::async_trait;
use indoc::writedoc; use indoc::writedoc;
use turbo_rcstr::{RcStr, rcstr}; use turbo_rcstr::{RcStr, rcstr};
use turbo_tasks::{ResolvedVc, ValueToString, Vc}; use turbo_tasks::{ResolvedVc, ValueToString, Vc};
...@@ -320,12 +321,10 @@ impl ChunkItem for EcmascriptClientReferenceProxyChunkItem { ...@@ -320,12 +321,10 @@ impl ChunkItem for EcmascriptClientReferenceProxyChunkItem {
self.inner_module.ident() self.inner_module.ident()
} }
#[turbo_tasks::function]
fn chunking_context(&self) -> Vc<Box<dyn ChunkingContext>> { fn chunking_context(&self) -> Vc<Box<dyn ChunkingContext>> {
*self.chunking_context *self.chunking_context
} }
#[turbo_tasks::function]
fn ty(&self) -> Vc<Box<dyn ChunkType>> { fn ty(&self) -> Vc<Box<dyn ChunkType>> {
Vc::upcast(Vc::<EcmascriptChunkType>::default()) Vc::upcast(Vc::<EcmascriptChunkType>::default())
} }
...@@ -336,21 +335,19 @@ impl ChunkItem for EcmascriptClientReferenceProxyChunkItem { ...@@ -336,21 +335,19 @@ impl ChunkItem for EcmascriptClientReferenceProxyChunkItem {
} }
} }
#[async_trait]
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl EcmascriptChunkItem for EcmascriptClientReferenceProxyChunkItem { impl EcmascriptChunkItem for EcmascriptClientReferenceProxyChunkItem {
#[turbo_tasks::function] async fn content_with_async_module_info(
fn content(&self) -> Vc<EcmascriptChunkItemContent> {
self.inner_chunk_item.content()
}
#[turbo_tasks::function]
fn content_with_async_module_info(
&self, &self,
async_module_info: Option<Vc<AsyncModuleInfo>>, async_module_info: Option<Vc<AsyncModuleInfo>>,
estimated: bool, estimated: bool,
) -> Vc<EcmascriptChunkItemContent> { ) -> Result<Vc<EcmascriptChunkItemContent>> {
self.inner_chunk_item self.inner_chunk_item
.into_trait_ref()
.await?
.content_with_async_module_info(async_module_info, estimated) .content_with_async_module_info(async_module_info, estimated)
.await
} }
} }
......
...@@ -52,24 +52,27 @@ struct NextFontLocalFontFileOptions { ...@@ -52,24 +52,27 @@ struct NextFontLocalFontFileOptions {
#[turbo_tasks::value] #[turbo_tasks::value]
pub(crate) struct NextFontLocalResolvePlugin { pub(crate) struct NextFontLocalResolvePlugin {
root: FileSystemPath, root: FileSystemPath,
condition: ResolvedVc<BeforeResolvePluginCondition>,
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl NextFontLocalResolvePlugin { impl NextFontLocalResolvePlugin {
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn new(root: FileSystemPath) -> Vc<Self> { pub async fn new(root: FileSystemPath) -> Result<Vc<Self>> {
NextFontLocalResolvePlugin { root }.cell() let condition = BeforeResolvePluginCondition::from_request_glob(Glob::new(
rcstr!("{next,@vercel/turbopack-next/internal}/font/local/*"),
GlobOptions::default(),
))
.to_resolved()
.await?;
Ok(NextFontLocalResolvePlugin { root, condition }.cell())
} }
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl BeforeResolvePlugin for NextFontLocalResolvePlugin { impl BeforeResolvePlugin for NextFontLocalResolvePlugin {
#[turbo_tasks::function]
fn before_resolve_condition(&self) -> Vc<BeforeResolvePluginCondition> { fn before_resolve_condition(&self) -> Vc<BeforeResolvePluginCondition> {
BeforeResolvePluginCondition::from_request_glob(Glob::new( *self.condition
rcstr!("{next,@vercel/turbopack-next/internal}/font/local/*"),
GlobOptions::default(),
))
} }
#[turbo_tasks::function] #[turbo_tasks::function]
......
...@@ -1269,7 +1269,7 @@ pub async fn try_get_next_package( ...@@ -1269,7 +1269,7 @@ pub async fn try_get_next_package(
Request::parse(Pattern::Constant(rcstr!("next/package.json"))), Request::parse(Pattern::Constant(rcstr!("next/package.json"))),
node_cjs_resolve_options(root.clone()), node_cjs_resolve_options(root.clone()),
); );
if let Some(source) = &*result.first_source().await? { if let Some(source) = result.await?.first_source() {
Ok(Vc::cell(Some(source.ident().await?.path.parent()))) Ok(Vc::cell(Some(source.ident().await?.path.parent())))
} else { } else {
MissingNextFolderIssue { MissingNextFolderIssue {
......
...@@ -42,41 +42,38 @@ pub enum ExternalPredicate { ...@@ -42,41 +42,38 @@ pub enum ExternalPredicate {
/// possible to resolve them at runtime. /// possible to resolve them at runtime.
#[turbo_tasks::value] #[turbo_tasks::value]
pub(crate) struct ExternalCjsModulesResolvePlugin { pub(crate) struct ExternalCjsModulesResolvePlugin {
root: FileSystemPath,
predicate: ResolvedVc<ExternalPredicate>, predicate: ResolvedVc<ExternalPredicate>,
import_externals: bool, import_externals: bool,
condition: ResolvedVc<AfterResolvePluginCondition>,
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl ExternalCjsModulesResolvePlugin { impl ExternalCjsModulesResolvePlugin {
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn new( pub async fn new(
root: FileSystemPath, root: FileSystemPath,
predicate: ResolvedVc<ExternalPredicate>, predicate: ResolvedVc<ExternalPredicate>,
import_externals: bool, import_externals: bool,
) -> Vc<Self> { ) -> Result<Vc<Self>> {
ExternalCjsModulesResolvePlugin { let condition = AfterResolvePluginCondition::new_with_glob(
root, root,
Glob::new(rcstr!("**/node_modules/**"), GlobOptions::default()),
)
.to_resolved()
.await?;
Ok(ExternalCjsModulesResolvePlugin {
predicate, predicate,
import_externals, import_externals,
condition,
} }
.cell() .cell())
} }
} }
#[turbo_tasks::function]
fn condition(root: FileSystemPath) -> Vc<AfterResolvePluginCondition> {
AfterResolvePluginCondition::new_with_glob(
root,
Glob::new(rcstr!("**/node_modules/**"), GlobOptions::default()),
)
}
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin { impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
#[turbo_tasks::function]
fn after_resolve_condition(&self) -> Vc<AfterResolvePluginCondition> { fn after_resolve_condition(&self) -> Vc<AfterResolvePluginCondition> {
condition(self.root.clone()) *self.condition
} }
#[turbo_tasks::function] #[turbo_tasks::function]
...@@ -85,7 +82,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin { ...@@ -85,7 +82,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
fs_path: FileSystemPath, fs_path: FileSystemPath,
lookup_path: FileSystemPath, lookup_path: FileSystemPath,
reference_type: ReferenceType, reference_type: ReferenceType,
request: ResolvedVc<Request>, request: Vc<Request>,
) -> Result<Vc<ResolveResultOption>> { ) -> Result<Vc<ResolveResultOption>> {
let request_value = &*request.await?; let request_value = &*request.await?;
let Request::Module { let Request::Module {
...@@ -112,10 +109,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin { ...@@ -112,10 +109,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
let predicate = self.predicate.await?; let predicate = self.predicate.await?;
let must_be_external = match &*predicate { let must_be_external = match &*predicate {
ExternalPredicate::AllExcept(exceptions) => { ExternalPredicate::AllExcept(exceptions) => {
if *condition(self.root.clone()) if self.condition.await?.matches(&lookup_path) {
.matches(lookup_path.clone())
.await?
{
return Ok(ResolveResultOption::none()); return Ok(ResolveResultOption::none());
} }
...@@ -214,7 +208,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin { ...@@ -214,7 +208,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
Ok(ResolveResultOption::none()) Ok(ResolveResultOption::none())
}; };
let mut request = *request; let mut request = request;
let mut request_str = request_str.to_string(); let mut request_str = request_str.to_string();
let node_resolve_options = if is_esm { let node_resolve_options = if is_esm {
...@@ -230,7 +224,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin { ...@@ -230,7 +224,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
node_resolve_options, node_resolve_options,
); );
let Some(result_from_original_location) = let Some(result_from_original_location) =
*node_resolved_from_original_location.first_source().await? node_resolved_from_original_location.await?.first_source()
else { else {
if is_esm if is_esm
&& !package_subpath.is_empty() && !package_subpath.is_empty()
...@@ -288,7 +282,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin { ...@@ -288,7 +282,7 @@ impl AfterResolvePlugin for ExternalCjsModulesResolvePlugin {
request, request,
node_resolve_options, node_resolve_options,
); );
let resolves_equal = if let Some(result) = *node_resolved.first_source().await? { let resolves_equal = if let Some(result) = node_resolved.await?.first_source() {
let ident = result.ident().await?; let ident = result.ident().await?;
let cjs_path = &ident.path; let cjs_path = &ident.path;
cjs_path == path cjs_path == path
......
...@@ -92,33 +92,35 @@ impl Issue for InvalidImportModuleIssue { ...@@ -92,33 +92,35 @@ impl Issue for InvalidImportModuleIssue {
#[turbo_tasks::value] #[turbo_tasks::value]
pub(crate) struct NextExternalResolvePlugin { pub(crate) struct NextExternalResolvePlugin {
project_path: FileSystemPath, condition: ResolvedVc<AfterResolvePluginCondition>,
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl NextExternalResolvePlugin { impl NextExternalResolvePlugin {
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn new(project_path: FileSystemPath) -> Vc<Self> { pub async fn new(project_path: FileSystemPath) -> Result<Vc<Self>> {
NextExternalResolvePlugin { project_path }.cell() let condition = AfterResolvePluginCondition::new_with_glob(
project_path.root().owned().await?,
Glob::new(
rcstr!("**/next/dist/**/*.{external,runtime.dev,runtime.prod}.js"),
GlobOptions::default(),
),
)
.to_resolved()
.await?;
Ok(NextExternalResolvePlugin { condition }.cell())
} }
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl AfterResolvePlugin for NextExternalResolvePlugin { impl AfterResolvePlugin for NextExternalResolvePlugin {
#[turbo_tasks::function] fn after_resolve_condition(&self) -> Vc<AfterResolvePluginCondition> {
async fn after_resolve_condition(&self) -> Result<Vc<AfterResolvePluginCondition>> { *self.condition
Ok(AfterResolvePluginCondition::new_with_glob(
self.project_path.root().owned().await?,
Glob::new(
rcstr!("**/next/dist/**/*.{external,runtime.dev,runtime.prod}.js"),
GlobOptions::default(),
),
))
} }
#[turbo_tasks::function] #[turbo_tasks::function]
async fn after_resolve( async fn after_resolve(
&self, self: Vc<Self>,
fs_path: FileSystemPath, fs_path: FileSystemPath,
_lookup_path: FileSystemPath, _lookup_path: FileSystemPath,
_reference_type: ReferenceType, _reference_type: ReferenceType,
...@@ -146,33 +148,38 @@ impl AfterResolvePlugin for NextExternalResolvePlugin { ...@@ -146,33 +148,38 @@ impl AfterResolvePlugin for NextExternalResolvePlugin {
#[turbo_tasks::value] #[turbo_tasks::value]
pub(crate) struct NextNodeSharedRuntimeResolvePlugin { pub(crate) struct NextNodeSharedRuntimeResolvePlugin {
root: FileSystemPath,
server_context_type: ServerContextType, server_context_type: ServerContextType,
condition: ResolvedVc<AfterResolvePluginCondition>,
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl NextNodeSharedRuntimeResolvePlugin { impl NextNodeSharedRuntimeResolvePlugin {
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn new(root: FileSystemPath, server_context_type: ServerContextType) -> Vc<Self> { pub async fn new(
NextNodeSharedRuntimeResolvePlugin { root: FileSystemPath,
root, server_context_type: ServerContextType,
) -> Result<Vc<Self>> {
let condition = AfterResolvePluginCondition::new_with_glob(
root.root().owned().await?,
Glob::new(
rcstr!("**/next/dist/**/*.shared-runtime.js"),
GlobOptions::default(),
),
)
.to_resolved()
.await?;
Ok(NextNodeSharedRuntimeResolvePlugin {
server_context_type, server_context_type,
condition,
} }
.cell() .cell())
} }
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl AfterResolvePlugin for NextNodeSharedRuntimeResolvePlugin { impl AfterResolvePlugin for NextNodeSharedRuntimeResolvePlugin {
#[turbo_tasks::function] fn after_resolve_condition(&self) -> Vc<AfterResolvePluginCondition> {
async fn after_resolve_condition(&self) -> Result<Vc<AfterResolvePluginCondition>> { *self.condition
Ok(AfterResolvePluginCondition::new_with_glob(
self.root.root().owned().await?,
Glob::new(
rcstr!("**/next/dist/**/*.shared-runtime.js"),
GlobOptions::default(),
),
))
} }
#[turbo_tasks::function] #[turbo_tasks::function]
...@@ -225,32 +232,34 @@ impl AfterResolvePlugin for NextNodeSharedRuntimeResolvePlugin { ...@@ -225,32 +232,34 @@ impl AfterResolvePlugin for NextNodeSharedRuntimeResolvePlugin {
/// telemetry events if there is a match. /// telemetry events if there is a match.
#[turbo_tasks::value] #[turbo_tasks::value]
pub(crate) struct ModuleFeatureReportResolvePlugin { pub(crate) struct ModuleFeatureReportResolvePlugin {
root: FileSystemPath, condition: ResolvedVc<BeforeResolvePluginCondition>,
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl ModuleFeatureReportResolvePlugin { impl ModuleFeatureReportResolvePlugin {
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn new(root: FileSystemPath) -> Vc<Self> { pub async fn new(_root: FileSystemPath) -> Result<Vc<Self>> {
ModuleFeatureReportResolvePlugin { root }.cell() let condition = BeforeResolvePluginCondition::from_modules(Vc::cell(
FEATURE_MODULES
.keys()
.map(|k| (*k).into())
.collect::<Vec<RcStr>>(),
))
.to_resolved()
.await?;
Ok(ModuleFeatureReportResolvePlugin { condition }.cell())
} }
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl BeforeResolvePlugin for ModuleFeatureReportResolvePlugin { impl BeforeResolvePlugin for ModuleFeatureReportResolvePlugin {
#[turbo_tasks::function]
fn before_resolve_condition(&self) -> Vc<BeforeResolvePluginCondition> { fn before_resolve_condition(&self) -> Vc<BeforeResolvePluginCondition> {
BeforeResolvePluginCondition::from_modules(Vc::cell( *self.condition
FEATURE_MODULES
.keys()
.map(|k| (*k).into())
.collect::<Vec<RcStr>>(),
))
} }
#[turbo_tasks::function] #[turbo_tasks::function]
async fn before_resolve( async fn before_resolve(
&self, self: Vc<Self>,
_lookup_path: FileSystemPath, _lookup_path: FileSystemPath,
_reference_type: ReferenceType, _reference_type: ReferenceType,
request: Vc<Request>, request: Vc<Request>,
...@@ -269,6 +278,7 @@ impl BeforeResolvePlugin for ModuleFeatureReportResolvePlugin { ...@@ -269,6 +278,7 @@ impl BeforeResolvePlugin for ModuleFeatureReportResolvePlugin {
.find(|sub_path| path.is_match(sub_path)); .find(|sub_path| path.is_match(sub_path));
if let Some(sub_path) = sub_path { if let Some(sub_path) = sub_path {
// This is not accurate. we only emit one diagnostic per request not per import
ModuleFeatureTelemetry::new(format!("{module}{sub_path}").into(), 1) ModuleFeatureTelemetry::new(format!("{module}{sub_path}").into(), 1)
.resolved_cell() .resolved_cell()
.emit(); .emit();
...@@ -282,33 +292,35 @@ impl BeforeResolvePlugin for ModuleFeatureReportResolvePlugin { ...@@ -282,33 +292,35 @@ impl BeforeResolvePlugin for ModuleFeatureReportResolvePlugin {
#[turbo_tasks::value] #[turbo_tasks::value]
pub(crate) struct NextSharedRuntimeResolvePlugin { pub(crate) struct NextSharedRuntimeResolvePlugin {
root: FileSystemPath, condition: ResolvedVc<AfterResolvePluginCondition>,
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl NextSharedRuntimeResolvePlugin { impl NextSharedRuntimeResolvePlugin {
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn new(root: FileSystemPath) -> Vc<Self> { pub async fn new(root: FileSystemPath) -> Result<Vc<Self>> {
NextSharedRuntimeResolvePlugin { root }.cell() let condition = AfterResolvePluginCondition::new_with_glob(
root.root().owned().await?,
Glob::new(
rcstr!("**/next/dist/esm/**/*.shared-runtime.js"),
GlobOptions::default(),
),
)
.to_resolved()
.await?;
Ok(NextSharedRuntimeResolvePlugin { condition }.cell())
} }
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl AfterResolvePlugin for NextSharedRuntimeResolvePlugin { impl AfterResolvePlugin for NextSharedRuntimeResolvePlugin {
#[turbo_tasks::function] fn after_resolve_condition(&self) -> Vc<AfterResolvePluginCondition> {
async fn after_resolve_condition(&self) -> Result<Vc<AfterResolvePluginCondition>> { *self.condition
Ok(AfterResolvePluginCondition::new_with_glob(
self.root.root().owned().await?,
Glob::new(
rcstr!("**/next/dist/esm/**/*.shared-runtime.js"),
GlobOptions::default(),
),
))
} }
#[turbo_tasks::function] #[turbo_tasks::function]
async fn after_resolve( async fn after_resolve(
&self, self: Vc<Self>,
fs_path: FileSystemPath, fs_path: FileSystemPath,
_lookup_path: FileSystemPath, _lookup_path: FileSystemPath,
_reference_type: ReferenceType, _reference_type: ReferenceType,
......
...@@ -135,8 +135,10 @@ pub async fn get_swc_ecma_transform_rule_impl( ...@@ -135,8 +135,10 @@ pub async fn get_swc_ecma_transform_rule_impl(
) )
.await?; .await?;
let Some(plugin_module) = let Some(plugin_module) = plugin_wasm_module_resolve_result
&*plugin_wasm_module_resolve_result.first_module().await? .await?
.first_module()
.await?
else { else {
// Ignore unresolvable plugin modules, handle_resolve_error has already emitted // Ignore unresolvable plugin modules, handle_resolve_error has already emitted
// an issue. // an issue.
......
...@@ -260,7 +260,7 @@ async fn detect_react_compiler_target( ...@@ -260,7 +260,7 @@ async fn detect_react_compiler_target(
node_cjs_resolve_options(project_path.root().owned().await?), node_cjs_resolve_options(project_path.root().owned().await?),
); );
let Some(source) = &*react_pkg_result.first_source().await? else { let Some(source) = react_pkg_result.await?.first_source() else {
return Ok(None); return Ok(None);
}; };
...@@ -335,7 +335,7 @@ pub async fn resolve_babel_plugin_react_compiler( ...@@ -335,7 +335,7 @@ pub async fn resolve_babel_plugin_react_compiler(
Request::parse(Pattern::Constant(BABEL_PLUGIN_REACT_COMPILER_PACKAGE_JSON)), Request::parse(Pattern::Constant(BABEL_PLUGIN_REACT_COMPILER_PACKAGE_JSON)),
node_cjs_resolve_options(project_path.root().owned().await?), node_cjs_resolve_options(project_path.root().owned().await?),
); );
let Some(source) = &*babel_plugin_result.first_source().await? else { let Some(source) = babel_plugin_result.await?.first_source() else {
BabelPluginReactCompilerResolutionIssue { BabelPluginReactCompilerResolutionIssue {
failed_resolution: BABEL_PLUGIN_REACT_COMPILER, failed_resolution: BABEL_PLUGIN_REACT_COMPILER,
config_file_path: next_config config_file_path: next_config
......
...@@ -204,7 +204,7 @@ async fn track_glob_internal( ...@@ -204,7 +204,7 @@ async fn track_glob_internal(
types.push(path.get_type()) types.push(path.get_type())
} }
} }
// The most likely case of this is actually a sylink resolution error, it is // The most likely case of this is actually a symlink resolution error, it is
// fine to ignore since the mere act of attempting to resolve it has triggered // fine to ignore since the mere act of attempting to resolve it has triggered
// the ncecessary dependencies. If this file is actually a dependency we should // the ncecessary dependencies. If this file is actually a dependency we should
// get an error in the actual webpack loader when it reads it. // get an error in the actual webpack loader when it reads it.
......
...@@ -63,9 +63,9 @@ pub async fn assert_can_resolve_react_refresh( ...@@ -63,9 +63,9 @@ pub async fn assert_can_resolve_react_refresh(
request, request,
resolve_options, resolve_options,
) )
.first_source(); .await?;
if result.await?.is_some() { if result.first_source().is_some() {
return Ok(ResolveReactRefreshResult::Found(request.to_resolved().await?).cell()); return Ok(ResolveReactRefreshResult::Found(request.to_resolved().await?).cell());
} }
} }
......
...@@ -40,7 +40,6 @@ impl RuntimeEntry { ...@@ -40,7 +40,6 @@ impl RuntimeEntry {
None, None,
ResolveErrorMode::Error, ResolveErrorMode::Error,
) )
.to_resolved()
.await? .await?
.primary_modules() .primary_modules()
.await?; .await?;
......
...@@ -295,6 +295,7 @@ async fn build_internal( ...@@ -295,6 +295,7 @@ async fn build_internal(
origin origin
.resolve_asset(request_vc, origin.resolve_options(), ty) .resolve_asset(request_vc, origin.resolve_options(), ty)
.await? .await?
.await?
.first_module() .first_module()
.await? .await?
.with_context(|| { .with_context(|| {
......
...@@ -138,15 +138,12 @@ pub async fn create_web_entry_source( ...@@ -138,15 +138,12 @@ pub async fn create_web_entry_source(
.into_iter() .into_iter()
.map(|request| async move { .map(|request| async move {
let ty = ReferenceType::Entry(EntryReferenceSubType::Web); let ty = ReferenceType::Entry(EntryReferenceSubType::Web);
Ok(origin origin
.resolve_asset(request, origin.resolve_options(), ty) .resolve_asset(request, origin.resolve_options(), ty)
.await? .await?
.to_resolved()
.await? .await?
.primary_modules() .first_module()
.await? .await
.first()
.copied())
}) })
.try_flat_join() .try_flat_join()
.await?; .await?;
......
...@@ -127,13 +127,26 @@ pub async fn make_chunk_group( ...@@ -127,13 +127,26 @@ pub async fn make_chunk_group(
}) })
.try_join() .try_join()
.await?; .await?;
let async_loader_chunk_items = async_loaders.iter().map(|&chunk_item| { let async_loader_chunk_items = async_loaders
ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(ChunkItemWithAsyncModuleInfo { .iter()
chunk_item, .map(async |&chunk_item| {
module: None, let chunk_type = chunk_item
async_info: None, .into_trait_ref()
.await?
.ty()
.to_resolved()
.await?;
Ok(ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(
ChunkItemWithAsyncModuleInfo {
chunk_item,
chunk_type,
module: None,
async_info: None,
},
))
}) })
}); .try_join()
.await?;
let referenced_output_assets = traced_modules let referenced_output_assets = traced_modules
.into_iter() .into_iter()
......
...@@ -11,7 +11,7 @@ use turbo_tasks::{ ...@@ -11,7 +11,7 @@ use turbo_tasks::{
}; };
use crate::{ use crate::{
chunk::{ChunkItem, ChunkItemWithAsyncModuleInfo, ChunkType, ChunkableModule, ChunkingContext}, chunk::{ChunkItemWithAsyncModuleInfo, ChunkType, ChunkableModule, ChunkingContext},
module_graph::{ module_graph::{
ModuleGraph, ModuleGraph,
async_module_info::AsyncModulesInfo, async_module_info::AsyncModulesInfo,
...@@ -46,8 +46,15 @@ pub async fn attach_async_info_to_chunkable_module( ...@@ -46,8 +46,15 @@ pub async fn attach_async_info_to_chunkable_module(
.as_chunk_item(module_graph, chunking_context) .as_chunk_item(module_graph, chunking_context)
.to_resolved() .to_resolved()
.await?; .await?;
let chunk_type = chunk_item
.into_trait_ref()
.await?
.ty()
.to_resolved()
.await?;
Ok(ChunkItemWithAsyncModuleInfo { Ok(ChunkItemWithAsyncModuleInfo {
chunk_item, chunk_item,
chunk_type,
module: Some(module), module: Some(module),
async_info, async_info,
}) })
...@@ -103,10 +110,9 @@ impl ChunkItemOrBatchWithAsyncModuleInfo { ...@@ -103,10 +110,9 @@ impl ChunkItemOrBatchWithAsyncModuleInfo {
&self, &self,
) -> Result<ChunkItemOrBatchWithAsyncModuleInfoByChunkType> { ) -> Result<ChunkItemOrBatchWithAsyncModuleInfoByChunkType> {
Ok(match self { Ok(match self {
Self::ChunkItem(item) => Either::Left(smallvec![( Self::ChunkItem(item) => {
item.chunk_item.ty().to_resolved().await?, Either::Left(smallvec![(item.chunk_type, Self::ChunkItem(*item))])
Self::ChunkItem(item.clone()) }
)]),
Self::Batch(batch) => Either::Right(batch.split_by_chunk_type().await?), Self::Batch(batch) => Either::Right(batch.split_by_chunk_type().await?),
}) })
} }
...@@ -167,17 +173,15 @@ impl ChunkItemBatchWithAsyncModuleInfo { ...@@ -167,17 +173,15 @@ impl ChunkItemBatchWithAsyncModuleInfo {
let Some((_, first)) = iter.next() else { let Some((_, first)) = iter.next() else {
return Ok(Vc::cell(SmallVec::new())); return Ok(Vc::cell(SmallVec::new()));
}; };
let chunk_type = first.chunk_item.ty().to_resolved().await?; let chunk_type = first.chunk_type;
while let Some((i, item)) = iter.next() { for (i, item) in iter.by_ref() {
let ty = item.chunk_item.ty().to_resolved().await?; let ty = item.chunk_type;
if ty != chunk_type { if ty != chunk_type {
let mut map = FxIndexMap::default(); let mut map = FxIndexMap::default();
map.insert(chunk_type, this.chunk_items[..i].to_vec()); map.insert(chunk_type, this.chunk_items[..i].to_vec());
map.insert(ty, vec![item.clone()]); map.insert(ty, vec![*item]);
for (_, item) in iter { for (_, item) in iter {
map.entry(item.chunk_item.ty().to_resolved().await?) map.entry(item.chunk_type).or_default().push(*item);
.or_default()
.push(item.clone());
} }
return Ok(Vc::cell( return Ok(Vc::cell(
map.into_iter() map.into_iter()
......
...@@ -60,7 +60,7 @@ pub async fn expand_batches( ...@@ -60,7 +60,7 @@ pub async fn expand_batches(
); );
let asset_ident = item.chunk_item.asset_ident().to_string(); let asset_ident = item.chunk_item.asset_ident().to_string();
Ok(ChunkItemOrBatchWithInfo::ChunkItem { Ok(ChunkItemOrBatchWithInfo::ChunkItem {
chunk_item: item.clone(), chunk_item: *item,
size: *size.await?, size: *size.await?,
asset_ident: asset_ident.owned().await?, asset_ident: asset_ident.owned().await?,
}) })
......
...@@ -84,6 +84,7 @@ async fn batch_size( ...@@ -84,6 +84,7 @@ async fn batch_size(
.map( .map(
|&ChunkItemWithAsyncModuleInfo { |&ChunkItemWithAsyncModuleInfo {
chunk_item, chunk_item,
chunk_type: _,
async_info, async_info,
module: _, module: _,
}| { }| {
...@@ -106,18 +107,21 @@ async fn plain_chunk_items_with_info( ...@@ -106,18 +107,21 @@ async fn plain_chunk_items_with_info(
ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(chunk_item_with_info) => { ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(chunk_item_with_info) => {
let ChunkItemWithAsyncModuleInfo { let ChunkItemWithAsyncModuleInfo {
chunk_item, chunk_item,
chunk_type,
async_info, async_info,
module: _, module: _,
} = chunk_item_with_info; } = chunk_item_with_info;
let asset_ident = chunk_item.asset_ident().to_string(); let asset_ident = chunk_item.asset_ident().to_string();
let ty = chunk_item.ty(); let chunk_item_size = chunk_type.chunk_item_size(
let chunk_item_size = chunking_context,
ty.chunk_item_size(chunking_context, *chunk_item, async_info.map(|info| *info)); *chunk_item,
async_info.map(|info| *info),
);
ChunkItemsWithInfo { ChunkItemsWithInfo {
by_type: smallvec![( by_type: smallvec![(
ty.to_resolved().await?, chunk_type,
smallvec![ChunkItemOrBatchWithInfo::ChunkItem { smallvec![ChunkItemOrBatchWithInfo::ChunkItem {
chunk_item: chunk_item_with_info, chunk_item: chunk_item_with_info,
size: *chunk_item_size.await?, size: *chunk_item_size.await?,
...@@ -162,6 +166,7 @@ async fn plain_chunk_items_with_info_with_type( ...@@ -162,6 +166,7 @@ async fn plain_chunk_items_with_info_with_type(
ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(chunk_item_with_info) => { ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(chunk_item_with_info) => {
let &ChunkItemWithAsyncModuleInfo { let &ChunkItemWithAsyncModuleInfo {
chunk_item, chunk_item,
chunk_type: _,
async_info, async_info,
module: _, module: _,
} = chunk_item_with_info; } = chunk_item_with_info;
...@@ -172,7 +177,7 @@ async fn plain_chunk_items_with_info_with_type( ...@@ -172,7 +177,7 @@ async fn plain_chunk_items_with_info_with_type(
Ok(( Ok((
ty, ty,
smallvec![ChunkItemOrBatchWithInfo::ChunkItem { smallvec![ChunkItemOrBatchWithInfo::ChunkItem {
chunk_item: chunk_item_with_info.clone(), chunk_item: *chunk_item_with_info,
size: *chunk_item_size.await?, size: *chunk_item_size.await?,
asset_ident: asset_ident.owned().await?, asset_ident: asset_ident.owned().await?,
}], }],
...@@ -413,7 +418,7 @@ async fn make_chunk( ...@@ -413,7 +418,7 @@ async fn make_chunk(
.into_iter() .into_iter()
.map(|item| match item { .map(|item| match item {
ChunkItemOrBatchWithInfo::ChunkItem { chunk_item, .. } => { ChunkItemOrBatchWithInfo::ChunkItem { chunk_item, .. } => {
ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(chunk_item.clone()) ChunkItemOrBatchWithAsyncModuleInfo::ChunkItem(*chunk_item)
} }
&ChunkItemOrBatchWithInfo::Batch { batch, .. } => { &ChunkItemOrBatchWithInfo::Batch { batch, .. } => {
ChunkItemOrBatchWithAsyncModuleInfo::Batch(batch) ChunkItemOrBatchWithAsyncModuleInfo::Batch(batch)
......
...@@ -48,7 +48,7 @@ pub async fn make_style_production_chunks( ...@@ -48,7 +48,7 @@ pub async fn make_style_production_chunks(
} else { } else {
make_chunk( make_chunk(
vec![&ChunkItemOrBatchWithInfo::ChunkItem { vec![&ChunkItemOrBatchWithInfo::ChunkItem {
chunk_item: chunk_item.clone(), chunk_item: *chunk_item,
size: 0, size: 0,
asset_ident: rcstr!(""), asset_ident: rcstr!(""),
}], }],
......
...@@ -160,13 +160,13 @@ impl ChunkGroupResult { ...@@ -160,13 +160,13 @@ impl ChunkGroupResult {
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl ChunkGroupResult { impl ChunkGroupResult {
#[turbo_tasks::function] #[turbo_tasks::function]
pub async fn output_assets_with_referenced(&self) -> Result<Vc<OutputAssetsWithReferenced>> { pub fn output_assets_with_referenced(&self) -> Vc<OutputAssetsWithReferenced> {
Ok(OutputAssetsWithReferenced { OutputAssetsWithReferenced {
assets: self.assets, assets: self.assets,
referenced_assets: self.referenced_assets, referenced_assets: self.referenced_assets,
references: self.references, references: self.references,
} }
.cell()) .cell()
} }
#[turbo_tasks::function] #[turbo_tasks::function]
......
...@@ -8,7 +8,7 @@ pub(crate) mod chunking_context; ...@@ -8,7 +8,7 @@ pub(crate) mod chunking_context;
pub(crate) mod data; pub(crate) mod data;
pub(crate) mod evaluate; pub(crate) mod evaluate;
use std::fmt::Display; use std::{fmt::Display, hash::Hash};
use anyhow::{Result, bail}; use anyhow::{Result, bail};
use auto_hash_map::AutoSet; use auto_hash_map::AutoSet;
...@@ -453,16 +453,14 @@ pub trait ChunkItem: OutputAssetsReference { ...@@ -453,16 +453,14 @@ pub trait ChunkItem: OutputAssetsReference {
} }
/// The type of chunk this item should be assembled into. /// The type of chunk this item should be assembled into.
#[turbo_tasks::function] fn ty(&self) -> Vc<Box<dyn ChunkType>>;
fn ty(self: Vc<Self>) -> Vc<Box<dyn ChunkType>>;
/// A temporary method to retrieve the module associated with this /// A temporary method to retrieve the module associated with this
/// ChunkItem. TODO: Remove this as part of the chunk refactoring. /// ChunkItem. TODO: Remove this as part of the chunk refactoring.
#[turbo_tasks::function] #[turbo_tasks::function]
fn module(self: Vc<Self>) -> Vc<Box<dyn Module>>; fn module(self: Vc<Self>) -> Vc<Box<dyn Module>>;
#[turbo_tasks::function] fn chunking_context(&self) -> Vc<Box<dyn ChunkingContext>>;
fn chunking_context(self: Vc<Self>) -> Vc<Box<dyn ChunkingContext>>;
} }
#[turbo_tasks::value_trait] #[turbo_tasks::value_trait]
...@@ -514,10 +512,11 @@ impl AsyncModuleInfo { ...@@ -514,10 +512,11 @@ impl AsyncModuleInfo {
} }
#[derive( #[derive(
Debug, Clone, PartialEq, Eq, Hash, TraceRawVcs, TaskInput, NonLocalValue, Encode, Decode, Debug, Clone, Copy, PartialEq, Eq, Hash, TraceRawVcs, TaskInput, NonLocalValue, Encode, Decode,
)] )]
pub struct ChunkItemWithAsyncModuleInfo { pub struct ChunkItemWithAsyncModuleInfo {
pub chunk_item: ResolvedVc<Box<dyn ChunkItem>>, pub chunk_item: ResolvedVc<Box<dyn ChunkItem>>,
pub chunk_type: ResolvedVc<Box<dyn ChunkType>>,
pub module: Option<ResolvedVc<Box<dyn ChunkableModule>>>, pub module: Option<ResolvedVc<Box<dyn ChunkableModule>>>,
pub async_info: Option<ResolvedVc<AsyncModuleInfo>>, pub async_info: Option<ResolvedVc<AsyncModuleInfo>>,
} }
...@@ -535,6 +534,8 @@ where ...@@ -535,6 +534,8 @@ where
async fn id(self: Vc<Self>) -> Result<ModuleId> { async fn id(self: Vc<Self>) -> Result<ModuleId> {
let chunk_item = Vc::upcast_non_strict(self); let chunk_item = Vc::upcast_non_strict(self);
chunk_item chunk_item
.into_trait_ref()
.await?
.chunking_context() .chunking_context()
.chunk_item_id_strategy() .chunk_item_id_strategy()
.await? .await?
......
...@@ -86,7 +86,6 @@ pub async fn children_from_module_references( ...@@ -86,7 +86,6 @@ pub async fn children_from_module_references(
for &module in reference for &module in reference
.resolve_reference() .resolve_reference()
.to_resolved()
.await? .await?
.primary_modules() .primary_modules()
.await? .await?
......
...@@ -12,7 +12,7 @@ use turbo_tasks::{ ...@@ -12,7 +12,7 @@ use turbo_tasks::{
use crate::{ use crate::{
chunk::{ chunk::{
ChunkItem, ChunkItemBatchWithAsyncModuleInfo, ChunkItemWithAsyncModuleInfo, ChunkType, ChunkItemBatchWithAsyncModuleInfo, ChunkItemWithAsyncModuleInfo, ChunkType,
ChunkableModule, ChunkingContext, chunk_item_batch::attach_async_info_to_chunkable_module, ChunkableModule, ChunkingContext, chunk_item_batch::attach_async_info_to_chunkable_module,
}, },
module::{Module, StyleModule, StyleType}, module::{Module, StyleModule, StyleType},
...@@ -232,8 +232,8 @@ pub async fn compute_style_groups( ...@@ -232,8 +232,8 @@ pub async fn compute_style_groups(
chunking_context, chunking_context,
) )
.await?; .await?;
let ty = chunk_item.chunk_item.ty(); let size = *chunk_item
let size = *ty .chunk_type
.chunk_item_size(chunking_context, *chunk_item.chunk_item, None) .chunk_item_size(chunking_context, *chunk_item.chunk_item, None)
.await?; .await?;
Ok((chunk_item, size)) Ok((chunk_item, size))
...@@ -271,7 +271,7 @@ pub async fn compute_style_groups( ...@@ -271,7 +271,7 @@ pub async fn compute_style_groups(
// The list of modules and chunk items that go into the new chunk // The list of modules and chunk items that go into the new chunk
let mut new_chunk_modules = [module].into_iter().collect::<FxHashSet<_>>(); let mut new_chunk_modules = [module].into_iter().collect::<FxHashSet<_>>();
let mut new_chunk_items = vec![info.chunk_item.as_ref().unwrap().clone()]; let mut new_chunk_items = vec![info.chunk_item.unwrap()];
// The current size of the new chunk // The current size of the new chunk
let mut current_size = info.size; let mut current_size = info.size;
...@@ -374,7 +374,7 @@ pub async fn compute_style_groups( ...@@ -374,7 +374,7 @@ pub async fn compute_style_groups(
} }
} }
new_chunk_items.push(info.chunk_item.as_ref().unwrap().clone()); new_chunk_items.push(info.chunk_item.unwrap());
new_chunk_modules.insert(module); new_chunk_modules.insert(module);
*ordered_modules_with_state.get_mut(&module).unwrap() = true; *ordered_modules_with_state.get_mut(&module).unwrap() = true;
continue 'outer; continue 'outer;
......
...@@ -178,15 +178,7 @@ pub async fn primary_referenced_modules(module: Vc<Box<dyn Module>>) -> Result<V ...@@ -178,15 +178,7 @@ pub async fn primary_referenced_modules(module: Vc<Box<dyn Module>>) -> Result<V
.references() .references()
.await? .await?
.iter() .iter()
.map(|reference| async { .map(|reference| async { reference.resolve_reference().await?.primary_modules().await })
reference
.resolve_reference()
.to_resolved()
.await?
.primary_modules()
.owned()
.await
})
.try_join() .try_join()
.await? .await?
.into_iter() .into_iter()
...@@ -231,7 +223,7 @@ pub async fn primary_chunkable_referenced_modules( ...@@ -231,7 +223,7 @@ pub async fn primary_chunkable_referenced_modules(
let resolved = reference let resolved = reference
.resolve_reference() .resolve_reference()
.await? .await?
.primary_modules_ref() .primary_modules()
.await?; .await?;
let binding_usage = if include_binding_usage { let binding_usage = if include_binding_usage {
trait_ref.binding_usage() trait_ref.binding_usage()
......
...@@ -28,7 +28,7 @@ pub async fn handle_resolve_error( ...@@ -28,7 +28,7 @@ pub async fn handle_resolve_error(
) -> Result<Vc<ModuleResolveResult>> { ) -> Result<Vc<ModuleResolveResult>> {
Ok(match result.await { Ok(match result.await {
Ok(result_ref) => { Ok(result_ref) => {
if result_ref.is_unresolvable_ref() { if result_ref.is_unresolvable() {
emit_unresolvable_issue( emit_unresolvable_issue(
error_mode, error_mode,
origin, origin,
...@@ -71,7 +71,7 @@ pub async fn handle_resolve_source_error( ...@@ -71,7 +71,7 @@ pub async fn handle_resolve_source_error(
) -> Result<Vc<ResolveResult>> { ) -> Result<Vc<ResolveResult>> {
Ok(match result.await { Ok(match result.await {
Ok(result_ref) => { Ok(result_ref) => {
if result_ref.is_unresolvable_ref() { if result_ref.is_unresolvable() {
emit_unresolvable_issue( emit_unresolvable_issue(
error_mode, error_mode,
origin, origin,
......
use anyhow::Result; use anyhow::Result;
use rustc_hash::FxHashSet;
use turbo_rcstr::RcStr; use turbo_rcstr::RcStr;
use turbo_tasks::{ResolvedVc, Vc}; use turbo_tasks::{ReadRef, ResolvedVc, Vc};
use turbo_tasks_fs::{FileSystemPath, glob::Glob}; use turbo_tasks_fs::{FileSystemPath, glob::Glob};
use crate::{ use crate::{
...@@ -10,12 +9,17 @@ use crate::{ ...@@ -10,12 +9,17 @@ use crate::{
}; };
/// A condition which determines if the hooks of a resolve plugin gets called. /// A condition which determines if the hooks of a resolve plugin gets called.
#[turbo_tasks::value(shared)] ///
/// The glob is read at construction time and stored as a `ReadRef`, so `matches` is a pure
/// sync function. `serialization = "skip"` because serializing a `ReadRef` is wasteful and
/// recomputing this is very cheap.
#[turbo_tasks::value(serialization = "skip")]
pub enum AfterResolvePluginCondition { pub enum AfterResolvePluginCondition {
Glob { Glob {
root: FileSystemPath, root: FileSystemPath,
glob: ResolvedVc<Glob>, glob: ReadRef<Glob>,
}, },
// these variants are used by utoo
Always, Always,
Never, Never,
} }
...@@ -23,38 +27,31 @@ pub enum AfterResolvePluginCondition { ...@@ -23,38 +27,31 @@ pub enum AfterResolvePluginCondition {
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl AfterResolvePluginCondition { impl AfterResolvePluginCondition {
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn new_with_glob(root: FileSystemPath, glob: ResolvedVc<Glob>) -> Vc<Self> { pub async fn new_with_glob(root: FileSystemPath, glob: ResolvedVc<Glob>) -> Result<Vc<Self>> {
AfterResolvePluginCondition::Glob { root, glob }.cell() let glob = glob.await?;
Ok(AfterResolvePluginCondition::Glob { root, glob }.cell())
} }
} }
#[turbo_tasks::value_impl]
impl AfterResolvePluginCondition { impl AfterResolvePluginCondition {
#[turbo_tasks::function] /// Test whether `fs_path` matches this condition.
pub async fn matches(&self, fs_path: FileSystemPath) -> Result<Vc<bool>> { pub fn matches(&self, fs_path: &FileSystemPath) -> bool {
match self { match self {
AfterResolvePluginCondition::Glob { root, glob } => { AfterResolvePluginCondition::Glob { root, glob } => {
let path = fs_path; root.get_path_to(fs_path).is_some_and(|p| glob.matches(p))
if let Some(path) = root.get_path_to(&path)
&& glob.await?.matches(path)
{
return Ok(Vc::cell(true));
}
Ok(Vc::cell(false))
} }
AfterResolvePluginCondition::Always => Ok(Vc::cell(true)), AfterResolvePluginCondition::Always => true,
AfterResolvePluginCondition::Never => Ok(Vc::cell(false)), AfterResolvePluginCondition::Never => false,
} }
} }
} }
/// A condition which determines if the hooks of a resolve plugin gets called. /// A condition which determines if the hooks of a resolve plugin gets called.
#[turbo_tasks::value(shared)] #[turbo_tasks::value(shared, serialization = "skip")]
pub enum BeforeResolvePluginCondition { pub enum BeforeResolvePluginCondition {
Request(ResolvedVc<Glob>), Request(ReadRef<Glob>),
Modules(FxHashSet<RcStr>), Modules(ReadRef<Vec<RcStr>>),
// These are used by utoo
Always, Always,
Never, Never,
} }
...@@ -63,26 +60,25 @@ pub enum BeforeResolvePluginCondition { ...@@ -63,26 +60,25 @@ pub enum BeforeResolvePluginCondition {
impl BeforeResolvePluginCondition { impl BeforeResolvePluginCondition {
#[turbo_tasks::function] #[turbo_tasks::function]
pub async fn from_modules(modules: ResolvedVc<Vec<RcStr>>) -> Result<Vc<Self>> { pub async fn from_modules(modules: ResolvedVc<Vec<RcStr>>) -> Result<Vc<Self>> {
Ok(BeforeResolvePluginCondition::Modules(modules.await?.iter().cloned().collect()).cell()) Ok(BeforeResolvePluginCondition::Modules(modules.await?).cell())
} }
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn from_request_glob(glob: ResolvedVc<Glob>) -> Vc<Self> { pub async fn from_request_glob(glob: ResolvedVc<Glob>) -> Result<Vc<Self>> {
BeforeResolvePluginCondition::Request(glob).cell() Ok(BeforeResolvePluginCondition::Request(glob.await?).cell())
} }
} }
#[turbo_tasks::value_impl]
impl BeforeResolvePluginCondition { impl BeforeResolvePluginCondition {
#[turbo_tasks::function] /// Test whether `request` matches this condition.
pub async fn matches(&self, request: Vc<Request>) -> Result<Vc<bool>> { pub fn matches(&self, request: &Request) -> bool {
Ok(Vc::cell(match self { match self {
BeforeResolvePluginCondition::Request(glob) => match request.await?.request() { BeforeResolvePluginCondition::Request(glob) => match request.request() {
Some(request) => glob.await?.matches(request.as_str()), Some(request) => glob.matches(request.as_str()),
None => false, None => false,
}, },
BeforeResolvePluginCondition::Modules(modules) => { BeforeResolvePluginCondition::Modules(modules) => {
if let Request::Module { module, .. } = &*request.await? { if let Request::Module { module, .. } = request {
modules.iter().any(|m| module.is_match(m)) modules.iter().any(|m| module.is_match(m))
} else { } else {
false false
...@@ -90,14 +86,13 @@ impl BeforeResolvePluginCondition { ...@@ -90,14 +86,13 @@ impl BeforeResolvePluginCondition {
} }
BeforeResolvePluginCondition::Always => true, BeforeResolvePluginCondition::Always => true,
BeforeResolvePluginCondition::Never => false, BeforeResolvePluginCondition::Never => false,
})) }
} }
} }
#[turbo_tasks::value_trait] #[turbo_tasks::value_trait]
pub trait BeforeResolvePlugin { pub trait BeforeResolvePlugin {
#[turbo_tasks::function] fn before_resolve_condition(&self) -> Vc<BeforeResolvePluginCondition>;
fn before_resolve_condition(self: Vc<Self>) -> Vc<BeforeResolvePluginCondition>;
#[turbo_tasks::function] #[turbo_tasks::function]
fn before_resolve( fn before_resolve(
...@@ -111,8 +106,7 @@ pub trait BeforeResolvePlugin { ...@@ -111,8 +106,7 @@ pub trait BeforeResolvePlugin {
#[turbo_tasks::value_trait] #[turbo_tasks::value_trait]
pub trait AfterResolvePlugin { pub trait AfterResolvePlugin {
/// A condition which determines if the hooks gets called. /// A condition which determines if the hooks gets called.
#[turbo_tasks::function] fn after_resolve_condition(&self) -> Vc<AfterResolvePluginCondition>;
fn after_resolve_condition(self: Vc<Self>) -> Vc<AfterResolvePluginCondition>;
/// This hook gets called when a full filepath has been resolved and the /// This hook gets called when a full filepath has been resolved and the
/// condition matches. If a value is returned it replaces the resolve /// condition matches. If a value is returned it replaces the resolve
......
...@@ -248,12 +248,10 @@ impl ChunkItem for CssModuleChunkItem { ...@@ -248,12 +248,10 @@ impl ChunkItem for CssModuleChunkItem {
self.module.ident() self.module.ident()
} }
#[turbo_tasks::function]
fn chunking_context(&self) -> Vc<Box<dyn ChunkingContext>> { fn chunking_context(&self) -> Vc<Box<dyn ChunkingContext>> {
*self.chunking_context *self.chunking_context
} }
#[turbo_tasks::function]
fn ty(&self) -> Vc<Box<dyn ChunkType>> { fn ty(&self) -> Vc<Box<dyn ChunkType>> {
Vc::upcast(Vc::<CssChunkType>::default()) Vc::upcast(Vc::<CssChunkType>::default())
} }
...@@ -278,7 +276,6 @@ impl CssChunkItem for CssModuleChunkItem { ...@@ -278,7 +276,6 @@ impl CssChunkItem for CssModuleChunkItem {
{ {
for &module in import_ref for &module in import_ref
.resolve_reference() .resolve_reference()
.to_resolved()
.await? .await?
.primary_modules() .primary_modules()
.await? .await?
...@@ -300,7 +297,6 @@ impl CssChunkItem for CssModuleChunkItem { ...@@ -300,7 +297,6 @@ impl CssChunkItem for CssModuleChunkItem {
{ {
for &module in compose_ref for &module in compose_ref
.resolve_reference() .resolve_reference()
.to_resolved()
.await? .await?
.primary_modules() .primary_modules()
.await? .await?
......
...@@ -284,15 +284,16 @@ impl EcmascriptChunkPlaceable for EcmascriptCssModule { ...@@ -284,15 +284,16 @@ impl EcmascriptChunkPlaceable for EcmascriptCssModule {
original: original_name, original: original_name,
from, from,
} => { } => {
let resolved_module = from.resolve_reference().first_module().await?; let resolved_module =
from.resolve_reference().await?.first_module().await?;
let Some(resolved_module) = &*resolved_module else { let Some(resolved_module) = resolved_module else {
// Issue already emitted by CssModuleComposeReference::resolve_reference // Issue already emitted by CssModuleComposeReference::resolve_reference
continue; continue;
}; };
let Some(css_module) = let Some(css_module) =
ResolvedVc::try_downcast_type::<EcmascriptCssModule>(*resolved_module) ResolvedVc::try_downcast_type::<EcmascriptCssModule>(resolved_module)
else { else {
// Issue already emitted by CssModuleComposeReference::resolve_reference // Issue already emitted by CssModuleComposeReference::resolve_reference
continue; continue;
......
...@@ -46,10 +46,10 @@ impl ModuleReference for CssModuleComposeReference { ...@@ -46,10 +46,10 @@ impl ModuleReference for CssModuleComposeReference {
None, None,
); );
let resolved = result.first_module().await?; let resolved = result.await?.first_module().await?;
let file_path = self.origin.origin_path().to_resolved().await?; let file_path = self.origin.origin_path().to_resolved().await?;
if let Some(module) = &*resolved { if let Some(module) = resolved {
if ResolvedVc::try_downcast_type::<EcmascriptCssModule>(*module).is_none() { if ResolvedVc::try_downcast_type::<EcmascriptCssModule>(module).is_none() {
CssModuleComposesIssue { CssModuleComposesIssue {
severity: IssueSeverity::Error, severity: IssueSeverity::Error,
file_path, file_path,
......
...@@ -58,7 +58,7 @@ impl UrlAssetReference { ...@@ -58,7 +58,7 @@ impl UrlAssetReference {
self: Vc<Self>, self: Vc<Self>,
chunking_context: Vc<Box<dyn ChunkingContext>>, chunking_context: Vc<Box<dyn ChunkingContext>>,
) -> Result<Vc<ReferencedAsset>> { ) -> Result<Vc<ReferencedAsset>> {
if let Some(module) = *self.resolve_reference().first_module().await? if let Some(module) = self.resolve_reference().await?.first_module().await?
&& let Some(embeddable) = ResolvedVc::try_downcast::<Box<dyn CssEmbed>>(module) && let Some(embeddable) = ResolvedVc::try_downcast::<Box<dyn CssEmbed>>(module)
{ {
return Ok(ReferencedAsset::Some( return Ok(ReferencedAsset::Some(
......
...@@ -57,12 +57,18 @@ impl ChunkType for EcmascriptChunkType { ...@@ -57,12 +57,18 @@ impl ChunkType for EcmascriptChunkType {
else { else {
bail!("Chunk item is not an ecmascript chunk item but reporting chunk type ecmascript"); bail!("Chunk item is not an ecmascript chunk item but reporting chunk type ecmascript");
}; };
Ok(Vc::cell( let chunk_item = chunk_item.into_trait_ref().await?;
chunk_item let size = match chunk_item
.content_with_async_module_info(async_module_info, true) .content_with_async_module_info(async_module_info, true)
.await .await
.map_or(0, |content| round_chunk_item_size(content.inner_code.len())), {
)) Ok(content) => {
let content = content.await?;
round_chunk_item_size(content.inner_code.len())
}
Err(_) => 0,
};
Ok(Vc::cell(size))
} }
} }
......
use std::io::Write; use std::io::Write;
use anyhow::{Result, bail}; use anyhow::{Result, bail};
use async_trait::async_trait;
use bincode::{Decode, Encode}; use bincode::{Decode, Encode};
use smallvec::SmallVec; use smallvec::SmallVec;
use turbo_rcstr::{RcStr, rcstr}; use turbo_rcstr::{RcStr, rcstr};
...@@ -240,6 +241,7 @@ impl EcmascriptChunkItemWithAsyncInfo { ...@@ -240,6 +241,7 @@ impl EcmascriptChunkItemWithAsyncInfo {
) -> Result<EcmascriptChunkItemWithAsyncInfo> { ) -> Result<EcmascriptChunkItemWithAsyncInfo> {
let ChunkItemWithAsyncModuleInfo { let ChunkItemWithAsyncModuleInfo {
chunk_item, chunk_item,
chunk_type: _,
module: _, module: _,
async_info, async_info,
} = chunk_item; } = chunk_item;
...@@ -255,23 +257,18 @@ impl EcmascriptChunkItemWithAsyncInfo { ...@@ -255,23 +257,18 @@ impl EcmascriptChunkItemWithAsyncInfo {
} }
} }
#[async_trait]
#[turbo_tasks::value_trait] #[turbo_tasks::value_trait]
pub trait EcmascriptChunkItem: ChunkItem + OutputAssetsReference { pub trait EcmascriptChunkItem: ChunkItem + OutputAssetsReference {
#[turbo_tasks::function]
fn content(self: Vc<Self>) -> Vc<EcmascriptChunkItemContent>;
/// Fetches the content of the chunk item with async module info. /// Fetches the content of the chunk item with async module info.
/// When `estimated` is true, it's ok to provide an estimated content, since it's only used for /// When `estimated` is true, it's ok to provide an estimated content, since it's only used for
/// compute the chunking. When `estimated` is true, this function should not invoke other /// compute the chunking. When `estimated` is true, this function should not invoke other
/// chunking operations that would cause cycles. /// chunking operations that would cause cycles.
#[turbo_tasks::function] async fn content_with_async_module_info(
fn content_with_async_module_info( &self,
self: Vc<Self>, async_module_info: Option<Vc<AsyncModuleInfo>>,
_async_module_info: Option<Vc<AsyncModuleInfo>>, estimated: bool,
_estimated: bool, ) -> Result<Vc<EcmascriptChunkItemContent>>;
) -> Vc<EcmascriptChunkItemContent> {
self.content()
}
} }
pub trait EcmascriptChunkItemExt { pub trait EcmascriptChunkItemExt {
...@@ -295,13 +292,18 @@ async fn module_factory_with_code_generation_issue( ...@@ -295,13 +292,18 @@ async fn module_factory_with_code_generation_issue(
chunk_item: Vc<Box<dyn EcmascriptChunkItem>>, chunk_item: Vc<Box<dyn EcmascriptChunkItem>>,
async_module_info: Option<Vc<AsyncModuleInfo>>, async_module_info: Option<Vc<AsyncModuleInfo>>,
) -> Result<Vc<PersistedCode>> { ) -> Result<Vc<PersistedCode>> {
let content = match chunk_item async fn get_content(
.content_with_async_module_info(async_module_info, false) chunk_item: Vc<Box<dyn EcmascriptChunkItem>>,
.await async_module_info: Option<Vc<AsyncModuleInfo>>,
{ ) -> Result<ResolvedVc<PersistedCode>> {
Ok(item) => item.module_factory().await, let chunk_item_ref = chunk_item.into_trait_ref().await?;
Err(err) => Err(err), let content = chunk_item_ref
}; .content_with_async_module_info(async_module_info, false)
.await?
.await?;
content.module_factory().await
}
let content = get_content(chunk_item, async_module_info).await;
Ok(match content { Ok(match content {
Ok(factory) => *factory, Ok(factory) => *factory,
Err(error) => { Err(error) => {
...@@ -372,7 +374,6 @@ impl ChunkItem for EcmascriptModuleChunkItem { ...@@ -372,7 +374,6 @@ impl ChunkItem for EcmascriptModuleChunkItem {
.chunk_item_content_ident(*self.chunking_context, *self.module_graph) .chunk_item_content_ident(*self.chunking_context, *self.module_graph)
} }
#[turbo_tasks::function]
fn ty(&self) -> Vc<Box<dyn ChunkType>> { fn ty(&self) -> Vc<Box<dyn ChunkType>> {
Vc::upcast(Vc::<EcmascriptChunkType>::default()) Vc::upcast(Vc::<EcmascriptChunkType>::default())
} }
...@@ -382,7 +383,6 @@ impl ChunkItem for EcmascriptModuleChunkItem { ...@@ -382,7 +383,6 @@ impl ChunkItem for EcmascriptModuleChunkItem {
Vc::upcast(*self.module) Vc::upcast(*self.module)
} }
#[turbo_tasks::function]
fn chunking_context(&self) -> Vc<Box<dyn ChunkingContext>> { fn chunking_context(&self) -> Vc<Box<dyn ChunkingContext>> {
*self.chunking_context *self.chunking_context
} }
...@@ -397,25 +397,19 @@ impl OutputAssetsReference for EcmascriptModuleChunkItem { ...@@ -397,25 +397,19 @@ impl OutputAssetsReference for EcmascriptModuleChunkItem {
} }
} }
#[async_trait]
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl EcmascriptChunkItem for EcmascriptModuleChunkItem { impl EcmascriptChunkItem for EcmascriptModuleChunkItem {
#[turbo_tasks::function] async fn content_with_async_module_info(
fn content(&self) -> Vc<EcmascriptChunkItemContent> {
self.module
.chunk_item_content(*self.chunking_context, *self.module_graph, None, false)
}
#[turbo_tasks::function]
fn content_with_async_module_info(
&self, &self,
async_module_info: Option<Vc<AsyncModuleInfo>>, async_module_info: Option<Vc<AsyncModuleInfo>>,
estimated: bool, estimated: bool,
) -> Vc<EcmascriptChunkItemContent> { ) -> Result<Vc<EcmascriptChunkItemContent>> {
self.module.chunk_item_content( Ok(self.module.chunk_item_content(
*self.chunking_context, *self.chunking_context,
*self.module_graph, *self.module_graph,
async_module_info, async_module_info,
estimated, estimated,
) ))
} }
} }
...@@ -778,11 +778,6 @@ impl EcmascriptModuleAsset { ...@@ -778,11 +778,6 @@ impl EcmascriptModuleAsset {
*self.source *self.source
} }
#[turbo_tasks::function]
pub fn analyze(self: Vc<Self>) -> Vc<AnalyzeEcmascriptModuleResult> {
analyze_ecmascript_module(self, None)
}
#[turbo_tasks::function] #[turbo_tasks::function]
pub fn options(&self) -> Vc<EcmascriptOptions> { pub fn options(&self) -> Vc<EcmascriptOptions> {
*self.options *self.options
...@@ -790,6 +785,10 @@ impl EcmascriptModuleAsset { ...@@ -790,6 +785,10 @@ impl EcmascriptModuleAsset {
} }
impl EcmascriptModuleAsset { impl EcmascriptModuleAsset {
pub fn analyze(self: Vc<Self>) -> Vc<AnalyzeEcmascriptModuleResult> {
analyze_ecmascript_module(self, None)
}
pub async fn parse(&self) -> Result<Vc<ParseResult>> { pub async fn parse(&self) -> Result<Vc<ParseResult>> {
let options = self.options.await?; let options = self.options.await?;
let node_env = self let node_env = self
......
...@@ -331,7 +331,7 @@ impl ReferencedAsset { ...@@ -331,7 +331,7 @@ impl ReferencedAsset {
pub async fn from_resolve_result(resolve_result: Vc<ModuleResolveResult>) -> Result<Self> { pub async fn from_resolve_result(resolve_result: Vc<ModuleResolveResult>) -> Result<Self> {
// TODO handle multiple keyed results // TODO handle multiple keyed results
let result = resolve_result.await?; let result = resolve_result.await?;
if result.is_unresolvable_ref() { if result.is_unresolvable() {
return Ok(ReferencedAsset::Unresolvable); return Ok(ReferencedAsset::Unresolvable);
} }
for (_, result) in result.primary.iter() { for (_, result) in result.primary.iter() {
...@@ -475,7 +475,7 @@ impl ModuleReference for EsmAssetReference { ...@@ -475,7 +475,7 @@ impl ModuleReference for EsmAssetReference {
loader_request, loader_request,
origin.resolve_options(), origin.resolve_options(),
); );
let loader_fs_path = if let Some(source) = *resolved.first_source().await? { let loader_fs_path = if let Some(source) = resolved.await?.first_source() {
source.ident().await?.path.clone() source.ident().await?.path.clone()
} else { } else {
bail!("Unable to resolve turbopackLoader '{}'", loader.loader); bail!("Unable to resolve turbopackLoader '{}'", loader.loader);
...@@ -544,7 +544,7 @@ impl ModuleReference for EsmAssetReference { ...@@ -544,7 +544,7 @@ impl ModuleReference for EsmAssetReference {
.await?; .await?;
if let Some(ModulePart::Export(export_name)) = &self.export_name { if let Some(ModulePart::Export(export_name)) = &self.export_name {
for &module in result.primary_modules().await? { for &module in result.await?.primary_modules().await?.iter() {
if let Some(module) = ResolvedVc::try_downcast(module) if let Some(module) = ResolvedVc::try_downcast(module)
&& *is_export_missing(*module, export_name.clone()).await? && *is_export_missing(*module, export_name.clone()).await?
{ {
......
...@@ -3622,10 +3622,9 @@ async fn require_resolve_visitor( ...@@ -3622,10 +3622,9 @@ async fn require_resolve_visitor(
.await?; .await?;
let mut values = let mut values =
resolved resolved
.primary_sources()
.await? .await?
.iter() .primary_sources()
.map(|&source| async move { .map(|source| async move {
Ok(require_resolve(source.ident().await?.path.clone()).into()) Ok(require_resolve(source.ident().await?.path.clone()).into())
}) })
.try_join() .try_join()
......
...@@ -85,7 +85,7 @@ pub(crate) enum PatternMapping { ...@@ -85,7 +85,7 @@ pub(crate) enum PatternMapping {
/// ```js /// ```js
/// require(`./images/${name}.png`) /// require(`./images/${name}.png`)
/// ``` /// ```
Map(#[bincode(with = "turbo_bincode::indexmap")] FxIndexMap<String, SinglePatternMapping>), Map(#[bincode(with = "turbo_bincode::indexmap")] FxIndexMap<RcStr, SinglePatternMapping>),
} }
#[derive( #[derive(
...@@ -231,7 +231,7 @@ enum ImportMode { ...@@ -231,7 +231,7 @@ enum ImportMode {
} }
fn create_context_map( fn create_context_map(
map: &FxIndexMap<String, SinglePatternMapping>, map: &FxIndexMap<RcStr, SinglePatternMapping>,
key_expr: &Expr, key_expr: &Expr,
import_mode: ImportMode, import_mode: ImportMode,
) -> Expr { ) -> Expr {
...@@ -304,6 +304,7 @@ async fn to_single_pattern_mapping( ...@@ -304,6 +304,7 @@ async fn to_single_pattern_mapping(
origin: Vc<Box<dyn ResolveOrigin>>, origin: Vc<Box<dyn ResolveOrigin>>,
chunking_context: Vc<Box<dyn ChunkingContext>>, chunking_context: Vc<Box<dyn ChunkingContext>>,
resolve_item: &ModuleResolveResultItem, resolve_item: &ModuleResolveResultItem,
primary: &[(turbopack_core::resolve::RequestKey, ModuleResolveResultItem)],
resolve_type: ResolveType, resolve_type: ResolveType,
) -> Result<SinglePatternMapping> { ) -> Result<SinglePatternMapping> {
let module = match resolve_item { let module = match resolve_item {
...@@ -328,6 +329,16 @@ async fn to_single_pattern_mapping( ...@@ -328,6 +329,16 @@ async fn to_single_pattern_mapping(
.to_unstyled_string(), .to_unstyled_string(),
)); ));
} }
ModuleResolveResultItem::Duplicate(first) => {
return Box::pin(to_single_pattern_mapping(
origin,
chunking_context,
&primary[*first].1,
primary,
resolve_type,
))
.await;
}
ModuleResolveResultItem::Empty | ModuleResolveResultItem::Custom(_) => { ModuleResolveResultItem::Empty | ModuleResolveResultItem::Custom(_) => {
// TODO implement mapping // TODO implement mapping
CodeGenerationIssue { CodeGenerationIssue {
...@@ -405,24 +416,37 @@ impl PatternMapping { ...@@ -405,24 +416,37 @@ impl PatternMapping {
.cell()), .cell()),
1 if !request.request_pattern().await?.has_dynamic_parts() => { 1 if !request.request_pattern().await?.has_dynamic_parts() => {
let resolve_item = &result.primary.first().unwrap().1; let resolve_item = &result.primary.first().unwrap().1;
let single_pattern_mapping = let single_pattern_mapping = to_single_pattern_mapping(
to_single_pattern_mapping(origin, chunking_context, resolve_item, resolve_type) origin,
.await?; chunking_context,
resolve_item,
&result.primary,
resolve_type,
)
.await?;
Ok(PatternMapping::Single(single_pattern_mapping).cell()) Ok(PatternMapping::Single(single_pattern_mapping).cell())
} }
_ => { _ => {
let primary = &result.primary;
let mut set = HashSet::new(); let mut set = HashSet::new();
let map = result let items: Vec<(RcStr, &ModuleResolveResultItem)> = primary
.primary
.iter() .iter()
.filter_map(|(k, v)| { .filter_map(|(k, v)| {
let request = k.request.as_ref()?; let request = k.request.as_ref()?;
set.insert(request).then(|| (request.to_string(), v)) set.insert(request).then(|| (request.clone(), v))
}) })
.collect();
let map = items
.into_iter()
.map(|(k, v)| async move { .map(|(k, v)| async move {
let single_pattern_mapping = let single_pattern_mapping = to_single_pattern_mapping(
to_single_pattern_mapping(origin, chunking_context, v, resolve_type) origin,
.await?; chunking_context,
v,
primary,
resolve_type,
)
.await?;
Ok((k, single_pattern_mapping)) Ok((k, single_pattern_mapping))
}) })
.try_join() .try_join()
......
...@@ -661,7 +661,7 @@ impl EvaluateContext for WebpackLoaderContext { ...@@ -661,7 +661,7 @@ impl EvaluateContext for WebpackLoaderContext {
options, options,
); );
if let Some(source) = *resolved.first_source().await? { if let Some(source) = resolved.await?.first_source() {
if let Some(path) = self.cwd.get_relative_path_to(&source.ident().await?.path) { if let Some(path) = self.cwd.get_relative_path_to(&source.ident().await?.path) {
Ok(ResponseMessage::Resolve { path }) Ok(ResponseMessage::Resolve { path })
} else { } else {
...@@ -702,7 +702,7 @@ impl EvaluateContext for WebpackLoaderContext { ...@@ -702,7 +702,7 @@ impl EvaluateContext for WebpackLoaderContext {
) )
.await?; .await?;
let Some(module) = *resolved.first_module().await? else { let Some(module) = resolved.await?.first_module().await? else {
bail!( bail!(
"importModule: unable to resolve {} in {}", "importModule: unable to resolve {} in {}",
request, request,
......
...@@ -95,10 +95,9 @@ async fn resolve_node_pre_gyp_files( ...@@ -95,10 +95,9 @@ async fn resolve_node_pre_gyp_files(
collect_affecting_sources, collect_affecting_sources,
true, true,
) )
.first_source()
.await?; .await?;
let compile_target = compile_target.await?; let compile_target = compile_target.await?;
if let Some(config_asset) = *config if let Some(config_asset) = config.first_source()
&& let AssetContent::File(file) = &*config_asset.content().await? && let AssetContent::File(file) = &*config_asset.content().await?
&& let FileContent::Content(config_file) = &*file.await? && let FileContent::Content(config_file) = &*file.await?
{ {
...@@ -273,9 +272,11 @@ async fn resolve_node_gyp_build_files( ...@@ -273,9 +272,11 @@ async fn resolve_node_gyp_build_files(
collect_affecting_sources, collect_affecting_sources,
true, true,
); );
if let [binding_gyp] = &gyp_file.primary_sources().await?[..] { let gyp_file = gyp_file.await?;
let mut primary_sources = gyp_file.primary_sources();
if let (Some(binding_gyp), None) = (primary_sources.next(), primary_sources.next()) {
let mut merged_affecting_sources = if collect_affecting_sources { let mut merged_affecting_sources = if collect_affecting_sources {
gyp_file.await?.get_affecting_sources().collect::<Vec<_>>() gyp_file.get_affecting_sources().collect::<Vec<_>>()
} else { } else {
Vec::new() Vec::new()
}; };
...@@ -399,9 +400,8 @@ async fn resolve_node_bindings_files( ...@@ -399,9 +400,8 @@ async fn resolve_node_bindings_files(
collect_affecting_sources, collect_affecting_sources,
true, true,
) )
.first_source()
.await?; .await?;
if let Some(asset) = *resolved if let Some(asset) = resolved.first_source()
&& let AssetContent::File(file) = &*asset.content().await? && let AssetContent::File(file) = &*asset.content().await?
&& let FileContent::Content(_) = &*file.await? && let FileContent::Content(_) = &*file.await?
{ {
......
...@@ -25,7 +25,7 @@ use turbopack_core::{ ...@@ -25,7 +25,7 @@ use turbopack_core::{
pattern::Pattern, pattern::Pattern,
resolve, resolve,
}, },
source::{OptionSource, Source}, source::Source,
}; };
use crate::ecmascript::get_condition_maps; use crate::ecmascript::get_condition_maps;
...@@ -88,7 +88,7 @@ pub async fn read_tsconfigs( ...@@ -88,7 +88,7 @@ pub async fn read_tsconfigs(
configs.push((parsed_data, tsconfig)); configs.push((parsed_data, tsconfig));
if let Some(extends) = json["extends"].as_str() { if let Some(extends) = json["extends"].as_str() {
let resolved = resolve_extends(*tsconfig, extends, resolve_options).await?; let resolved = resolve_extends(*tsconfig, extends, resolve_options).await?;
if let Some(source) = *resolved.await? { if let Some(source) = resolved {
data = source.content().file_content(); data = source.content().file_content();
tsconfig = source; tsconfig = source;
continue; continue;
...@@ -118,7 +118,7 @@ async fn resolve_extends( ...@@ -118,7 +118,7 @@ async fn resolve_extends(
tsconfig: Vc<Box<dyn Source>>, tsconfig: Vc<Box<dyn Source>>,
extends: &str, extends: &str,
resolve_options: Vc<ResolveOptions>, resolve_options: Vc<ResolveOptions>,
) -> Result<Vc<OptionSource>> { ) -> Result<Option<ResolvedVc<Box<dyn Source>>>> {
let parent_dir = tsconfig.ident().await?.path.parent(); let parent_dir = tsconfig.ident().await?.path.parent();
let request = Request::parse_string(extends.into()); let request = Request::parse_string(extends.into());
...@@ -148,18 +148,18 @@ async fn resolve_extends( ...@@ -148,18 +148,18 @@ async fn resolve_extends(
Request::Empty => { Request::Empty => {
let request = Request::parse_string(rcstr!("./tsconfig")); let request = Request::parse_string(rcstr!("./tsconfig"));
Ok(resolve(parent_dir, Ok(resolve(parent_dir,
ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined), request, resolve_options).first_source()) ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined), request, resolve_options).await?.first_source())
} }
// All other types are treated as module imports, and potentially joined with // All other types are treated as module imports, and potentially joined with
// "tsconfig.json". This includes "relative" imports like '.' and '..'. // "tsconfig.json". This includes "relative" imports like '.' and '..'.
_ => { _ => {
let mut result = resolve(parent_dir.clone(), ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined), request, resolve_options).first_source(); let result = resolve(parent_dir.clone(), ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined), request, resolve_options).await?;
if result.await?.is_none() { if let Some(source) = result.first_source() {
let request = Request::parse_string(format!("{extends}/tsconfig").into()); return Ok(Some(source));
result = resolve(parent_dir, ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined), request, resolve_options).first_source();
} }
Ok(result) let request = Request::parse_string(format!("{extends}/tsconfig").into());
Ok(resolve(parent_dir, ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined), request, resolve_options).await?.first_source())
} }
} }
} }
...@@ -169,19 +169,21 @@ async fn resolve_extends_rooted_or_relative( ...@@ -169,19 +169,21 @@ async fn resolve_extends_rooted_or_relative(
request: Vc<Request>, request: Vc<Request>,
resolve_options: Vc<ResolveOptions>, resolve_options: Vc<ResolveOptions>,
path: &str, path: &str,
) -> Result<Vc<OptionSource>> { ) -> Result<Option<ResolvedVc<Box<dyn Source>>>> {
let mut result = resolve( let result = resolve(
lookup_path.clone(), lookup_path.clone(),
ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined), ReferenceType::TypeScript(TypeScriptReferenceSubType::Undefined),
request, request,
resolve_options, resolve_options,
) )
.first_source(); .await?;
let mut result = result.first_source();
// If the file doesn't end with ".json" and we can't find the file, then we have // If the file doesn't end with ".json" and we can't find the file, then we have
// to try again with it. // to try again with it.
// https://github.com/microsoft/TypeScript/blob/611a912d/src/compiler/commandLineParser.ts#L3305 // https://github.com/microsoft/TypeScript/blob/611a912d/src/compiler/commandLineParser.ts#L3305
if !path.ends_with(".json") && result.await?.is_none() { if !path.ends_with(".json") && result.is_none() {
let request = Request::parse_string(format!("{path}.json").into()); let request = Request::parse_string(format!("{path}.json").into());
result = resolve( result = resolve(
lookup_path.clone(), lookup_path.clone(),
...@@ -189,6 +191,7 @@ async fn resolve_extends_rooted_or_relative( ...@@ -189,6 +191,7 @@ async fn resolve_extends_rooted_or_relative(
request, request,
resolve_options, resolve_options,
) )
.await?
.first_source(); .first_source();
} }
Ok(result) Ok(result)
...@@ -424,7 +427,7 @@ pub async fn type_resolve( ...@@ -424,7 +427,7 @@ pub async fn type_resolve(
request, request,
options, options,
); );
if !*result1.is_unresolvable().await? { if !result1.await?.is_unresolvable() {
result1 result1
} else { } else {
resolve( resolve(
......
...@@ -421,20 +421,12 @@ impl ModuleAssetContext { ...@@ -421,20 +421,12 @@ impl ModuleAssetContext {
*self.resolve_options_context *self.resolve_options_context
} }
#[turbo_tasks::function]
pub async fn is_types_resolving_enabled(&self) -> Result<Vc<bool>> {
let resolve_options_context = self.resolve_options_context.await?;
Ok(Vc::cell(
resolve_options_context.enable_types && resolve_options_context.enable_typescript,
))
}
#[turbo_tasks::function] #[turbo_tasks::function]
pub async fn with_types_resolving_enabled(self: Vc<Self>) -> Result<Vc<ModuleAssetContext>> { pub async fn with_types_resolving_enabled(self: Vc<Self>) -> Result<Vc<ModuleAssetContext>> {
if *self.is_types_resolving_enabled().await? { let this = self.await?;
if this.is_types_resolving_enabled().await? {
return Ok(self); return Ok(self);
} }
let this = self.await?;
let resolve_options_context = *this let resolve_options_context = *this
.resolve_options_context .resolve_options_context
.with_types_enabled() .with_types_enabled()
...@@ -452,6 +444,10 @@ impl ModuleAssetContext { ...@@ -452,6 +444,10 @@ impl ModuleAssetContext {
} }
impl ModuleAssetContext { impl ModuleAssetContext {
async fn is_types_resolving_enabled(&self) -> Result<bool> {
let resolve_options_context = self.resolve_options_context.await?;
Ok(resolve_options_context.enable_types && resolve_options_context.enable_typescript)
}
async fn process_with_transition_rules( async fn process_with_transition_rules(
self: Vc<Self>, self: Vc<Self>,
source: ResolvedVc<Box<dyn Source>>, source: ResolvedVc<Box<dyn Source>>,
...@@ -1025,8 +1021,8 @@ impl AssetContext for ModuleAssetContext { ...@@ -1025,8 +1021,8 @@ impl AssetContext for ModuleAssetContext {
); );
let mut result = self.process_resolve_result(*result.to_resolved().await?, reference_type); let mut result = self.process_resolve_result(*result.to_resolved().await?, reference_type);
let this = self.await?;
if *self.is_types_resolving_enabled().await? { if this.is_types_resolving_enabled().await? {
let types_result = type_resolve( let types_result = type_resolve(
Vc::upcast(PlainResolveOrigin::new(Vc::upcast(self), origin_path)), Vc::upcast(PlainResolveOrigin::new(Vc::upcast(self), origin_path)),
request, request,
......
...@@ -12,11 +12,20 @@ Then run this script with the path to the stats.json file to get a report on cac ...@@ -12,11 +12,20 @@ Then run this script with the path to the stats.json file to get a report on cac
The JSON format contains entries like: The JSON format contains entries like:
{ "task_name": { "cache_hit": N, "cache_miss": N } } { "task_name": { "cache_hit": N, "cache_miss": N } }
Usage:
analyze_cache_effectiveness.py <stats.json>
analyze_cache_effectiveness.py --diff <before.json> <after.json> [--top N]
In --diff mode, the script reports the tasks whose hits or misses changed the
most between the two runs — useful for evaluating the impact of removing or
adding `turbo_tasks` caching on specific functions.
""" """
import argparse
import json import json
import sys import sys
from typing import List, Tuple from typing import Dict, List
from dataclasses import dataclass from dataclasses import dataclass
...@@ -54,6 +63,10 @@ def load_task_stats(file_path: str) -> List[TaskStats]: ...@@ -54,6 +63,10 @@ def load_task_stats(file_path: str) -> List[TaskStats]:
return tasks return tasks
def load_task_stats_map(file_path: str) -> Dict[str, TaskStats]:
return {t.name: t for t in load_task_stats(file_path)}
def analyze_tasks(tasks: List[TaskStats]) -> List[TaskStats]: def analyze_tasks(tasks: List[TaskStats]) -> List[TaskStats]:
"""Analyze all tasks and return sorted by wasted cache overhead. """Analyze all tasks and return sorted by wasted cache overhead.
...@@ -114,27 +127,213 @@ def print_analysis(tasks: List[TaskStats]): ...@@ -114,27 +127,213 @@ def print_analysis(tasks: List[TaskStats]):
print(f"Tasks with <50% hit rate: {low_hit_rate_count}") print(f"Tasks with <50% hit rate: {low_hit_rate_count}")
def main(): @dataclass
if len(sys.argv) != 2: class TaskDiff:
print("Usage: python analyze_cache_effectiveness.py <stats.json>") name: str
sys.exit(1) before_hit: int
after_hit: int
before_miss: int
after_miss: int
@property
def delta_hit(self) -> int:
return self.after_hit - self.before_hit
file_path = sys.argv[1] @property
def delta_miss(self) -> int:
return self.after_miss - self.before_miss
try: @staticmethod
tasks = load_task_stats(file_path) def _rate(hit: int, miss: int) -> float:
tasks = analyze_tasks(tasks) total = hit + miss
print_analysis(tasks) return hit / total if total > 0 else 0.0
@property
def before_rate(self) -> float:
return self._rate(self.before_hit, self.before_miss)
@property
def after_rate(self) -> float:
return self._rate(self.after_hit, self.after_miss)
@property
def delta_rate(self) -> float:
return self.after_rate - self.before_rate
def compute_diff(
before: Dict[str, TaskStats], after: Dict[str, TaskStats]
) -> List[TaskDiff]:
names = set(before) | set(after)
diffs = []
zero = TaskStats(name="", cache_hit=0, cache_miss=0)
for name in names:
b = before.get(name, zero)
a = after.get(name, zero)
diffs.append(
TaskDiff(
name=name,
before_hit=b.cache_hit,
after_hit=a.cache_hit,
before_miss=b.cache_miss,
after_miss=a.cache_miss,
)
)
return diffs
def _print_diff_section(
title: str, diffs: List[TaskDiff], key, top: int, reverse: bool
):
print("=" * 100)
print(title)
print("=" * 100)
ordered = sorted(diffs, key=key, reverse=reverse)
header = (
f"{'Δhit':>12} {'Δmiss':>12} {'hit b→a':>22} {'miss b→a':>22} "
f"{'rate b→a':>18} Task"
)
print(header)
print("-" * len(header))
shown = 0
for d in ordered:
# Stop once the signed delta crosses zero in the direction we care about.
v = key(d)
if reverse and v <= 0:
break
if not reverse and v >= 0:
break
hit_str = f"{d.before_hit:,}→{d.after_hit:,}"
miss_str = f"{d.before_miss:,}→{d.after_miss:,}"
rate_str = f"{d.before_rate:.0%}→{d.after_rate:.0%}"
print(
f"{d.delta_hit:>+12,} {d.delta_miss:>+12,} "
f"{hit_str:>22} {miss_str:>22} {rate_str:>18} {d.name}"
)
shown += 1
if shown >= top:
break
if shown == 0:
print("(none)")
print()
def print_diff(
before: Dict[str, TaskStats], after: Dict[str, TaskStats], top: int
):
diffs = compute_diff(before, after)
before_hits = sum(t.cache_hit for t in before.values())
after_hits = sum(t.cache_hit for t in after.values())
before_misses = sum(t.cache_miss for t in before.values())
after_misses = sum(t.cache_miss for t in after.values())
before_total = before_hits + before_misses
after_total = after_hits + after_misses
before_rate = before_hits / before_total if before_total > 0 else 0.0
after_rate = after_hits / after_total if after_total > 0 else 0.0
only_before = set(before) - set(after)
only_after = set(after) - set(before)
except FileNotFoundError: print("Cache statistics diff (before → after)")
print(f"Error: File '{file_path}' not found") print()
print(
f"Hits: {before_hits:>12,} → {after_hits:>12,} "
f"({after_hits - before_hits:+,})"
)
print(
f"Misses: {before_misses:>12,} → {after_misses:>12,} "
f"({after_misses - before_misses:+,})"
)
print(
f"Hit rate: {before_rate:>11.2%} → {after_rate:>11.2%} "
f"({(after_rate - before_rate) * 100:+.2f} pp)"
)
print(
f"Tasks: {len(before):>12,} → {len(after):>12,} "
f"(only in before: {len(only_before)}, only in after: {len(only_after)})"
)
print()
_print_diff_section(
f"Top {top} INCREASES IN HITS (after > before)",
diffs,
key=lambda d: d.delta_hit,
top=top,
reverse=True,
)
_print_diff_section(
f"Top {top} INCREASES IN MISSES (after > before)",
diffs,
key=lambda d: d.delta_miss,
top=top,
reverse=True,
)
_print_diff_section(
f"Top {top} DECREASES IN HITS (after < before)",
diffs,
key=lambda d: d.delta_hit,
top=top,
reverse=False,
)
_print_diff_section(
f"Top {top} DECREASES IN MISSES (after < before)",
diffs,
key=lambda d: d.delta_miss,
top=top,
reverse=False,
)
def main():
parser = argparse.ArgumentParser(
description=(
"Analyze turbo_tasks cache effectiveness, or diff two stats files "
"to see which tasks gained or lost hits/misses."
),
)
parser.add_argument(
"--diff",
action="store_true",
help="Diff mode: compare two stats files and report largest changes.",
)
parser.add_argument(
"--top",
type=int,
default=25,
help="Number of entries to show per section in --diff mode (default: 25).",
)
parser.add_argument(
"files",
nargs="+",
help=(
"Path to stats JSON. In default mode, one file. "
"In --diff mode, two files: <before> <after>."
),
)
args = parser.parse_args()
try:
if args.diff:
if len(args.files) != 2:
parser.error("--diff requires exactly two files: <before> <after>")
before = load_task_stats_map(args.files[0])
after = load_task_stats_map(args.files[1])
print_diff(before, after, args.top)
else:
if len(args.files) != 1:
parser.error("default mode requires exactly one stats file")
tasks = load_task_stats(args.files[0])
tasks = analyze_tasks(tasks)
print_analysis(tasks)
except FileNotFoundError as e:
print(f"Error: File not found: {e.filename}")
sys.exit(1) sys.exit(1)
except json.JSONDecodeError as e: except json.JSONDecodeError as e:
print(f"Error parsing JSON: {e}") print(f"Error parsing JSON: {e}")
sys.exit(1) sys.exit(1)
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
if __name__ == "__main__": if __name__ == "__main__":
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment