Unverified Commit f5649162 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Use graph-backed codegen for client proxies and WASM loaders (#99115)

### What?

Generate **both RSC client-reference proxies and WebAssembly loaders**
under their graph-backed wrapper module identities, using one shared,
higher-level content helper. Remove both generated-path exceptions from
`BindingUsageInfo::used_exports` and the earlier proxy-specific export
filter.

### Why?

The wrapper is the module in Turbopack's graph, but its chunk item
previously delegated code generation to a separately processed virtual
JS module. Export-usage lookup for that proxy/loader missed the graph
and fell back to “all exports used.” In a `sideEffects: false` barrel
this could register a pruned client export and request a missing
`<locals>` module ID. The WebAssembly loader had the same graph-identity
mismatch, hidden by a separate filename-based fallback.

Canary's #99102 now guards against the client-reference missing-ID
symptom by checking pruned references. This PR fixes the upstream
identity mismatch instead so `expand_exports()` sees real usage;
#99102's independent safety checks remain in place.

### How?

`chunk_item_content_with_code_from(graph_module, code_source, ...)`
takes the generated source's analyzed program, references, exports,
source map and async behavior while using the wrapper for the *whole
code-generation module context*. Both client-reference and WASM wrappers
call it. This replaces the public low-level `with_module_context`
setter; it does not change module IDs, WASM output assets, React Flight
registration, or the existing single-chunk WASM rejection. sokra's
`ResolvedVc` suggestions are also applied.

#99090's pruned-reexport-source guard remains necessary: removing it
independently still fails its execution fixture. #99082 proposed
reverting all of #98617 and closed unmerged; it was not a separate fix
to delete.

The neutral production proxy regression and two supplemental
pruned-source execution fixtures remain in the PR.

### Verification

- **WASM can-fail/can-pass:** all 3 existing WASM execution fixtures
pass before removing the fallback; with the fallback removed but old
loader delegation retained, all 3 fail `export usage not found
...wasm_.loader.mjs`; with the shared helper, all 3 pass (instantiation,
imported functions/memory, and `?module` compilation).
- **Proxy can-fail/can-pass:** without the wrapper-context path and
proxy fallback, the production fixture fails `export usage not found
.../__nextjs-internal-proxy.mjs`; with the fix it passes.
- **Rust/build:** 1,282 tests pass across affected crates; clippy (`-D
warnings`), rustfmt and `pnpm build-all` (18/18 tasks) pass.
- **Next.js integration:** Turbopack production 6 suites/12 tests
(proxy, used client references, #99102 regression, edge WASM and WASM
output tracing); development 3 suites/8 tests (proxy, used client
references and edge WASM). Client-reference chunking 1/1. After rebasing
onto current canary, `pnpm build-all` and focused proxy/edge WASM
production tests (3 suites/7 tests) pass again.

<!-- NEXT_JS_LLM -->

<!-- fleet ecdfa248-cd54-41ac-b4a2-c9d49e2a67ee -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: 's avatarLuke Sandberg <210140+lukesandberg@users.noreply.github.com>
parent 0556c918
use std::{io::Write, iter::once}; use std::{io::Write, iter::once};
use anyhow::{Context, Result, bail}; use anyhow::{Result, bail};
use async_trait::async_trait; use async_trait::async_trait;
use indoc::writedoc; use indoc::writedoc;
use turbo_rcstr::{RcStr, rcstr}; use turbo_rcstr::{RcStr, rcstr};
...@@ -29,6 +29,7 @@ use turbopack_ecmascript::{ ...@@ -29,6 +29,7 @@ use turbopack_ecmascript::{
EcmascriptChunkItem, EcmascriptChunkItemContent, EcmascriptChunkPlaceable, EcmascriptChunkItem, EcmascriptChunkItemContent, EcmascriptChunkPlaceable,
EcmascriptChunkType, EcmascriptExports, EcmascriptChunkType, EcmascriptExports,
}, },
chunk_item_content_with_code_from,
runtime_functions::TURBOPACK_EXPORT_NAMESPACE, runtime_functions::TURBOPACK_EXPORT_NAMESPACE,
utils::StringifyJs, utils::StringifyJs,
}; };
...@@ -258,26 +259,19 @@ impl Module for EcmascriptClientReferenceModule { ...@@ -258,26 +259,19 @@ impl Module for EcmascriptClientReferenceModule {
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
impl ChunkableModule for EcmascriptClientReferenceModule { impl ChunkableModule for EcmascriptClientReferenceModule {
#[turbo_tasks::function] #[turbo_tasks::function]
async fn as_chunk_item( fn as_chunk_item(
self: ResolvedVc<Self>, self: ResolvedVc<Self>,
module_graph: Vc<ModuleGraph>, module_graph: ResolvedVc<ModuleGraph>,
chunking_context: ResolvedVc<Box<dyn ChunkingContext>>, chunking_context: ResolvedVc<Box<dyn ChunkingContext>>,
) -> Result<Vc<Box<dyn ChunkItem>>> { ) -> Vc<Box<dyn ChunkItem>> {
let item = self Vc::upcast(
.proxy_module()
.as_chunk_item(module_graph, *chunking_context);
let ecmascript_item =
ResolvedVc::try_downcast::<Box<dyn EcmascriptChunkItem>>(item.to_resolved().await?)
.context("EcmascriptModuleAsset must implement EcmascriptChunkItem")?;
Ok(Vc::upcast(
EcmascriptClientReferenceProxyChunkItem { EcmascriptClientReferenceProxyChunkItem {
inner_module: self, inner_module: self,
inner_chunk_item: ecmascript_item, module_graph,
chunking_context, chunking_context,
} }
.cell(), .cell(),
)) )
} }
} }
...@@ -291,25 +285,28 @@ impl EcmascriptChunkPlaceable for EcmascriptClientReferenceModule { ...@@ -291,25 +285,28 @@ impl EcmascriptChunkPlaceable for EcmascriptClientReferenceModule {
} }
#[turbo_tasks::function] #[turbo_tasks::function]
fn chunk_item_content( async fn chunk_item_content(
self: Vc<Self>, self: ResolvedVc<Self>,
_chunking_context: Vc<Box<dyn ChunkingContext>>, chunking_context: ResolvedVc<Box<dyn ChunkingContext>>,
_module_graph: Vc<ModuleGraph>, _module_graph: Vc<ModuleGraph>,
_async_module_info: Option<Vc<AsyncModuleInfo>>, async_module_info: Option<Vc<AsyncModuleInfo>>,
_estimated: bool, _estimated: bool,
) -> Result<Vc<EcmascriptChunkItemContent>> { ) -> Result<Vc<EcmascriptChunkItemContent>> {
bail!("Attempted to get chunk_item_content for EcmascriptClientReferenceModule") Ok(chunk_item_content_with_code_from(
*ResolvedVc::upcast(self),
self.proxy_module(),
*chunking_context,
async_module_info,
))
} }
} }
/// This wrapper only exists to overwrite the `asset_ident` method of the /// This wrapper uses the client-reference module's graph identity for both its asset identifier
/// wrapped [`Vc<Box<dyn EcmascriptChunkItem>>`]. Otherwise, the asset ident of /// and code generation. The generated proxy supplies the code, but isn't itself in the graph.
/// the chunk item would not be the same as the asset ident of the
/// [`Vc<EcmascriptClientReferenceModule>`].
#[turbo_tasks::value] #[turbo_tasks::value]
struct EcmascriptClientReferenceProxyChunkItem { struct EcmascriptClientReferenceProxyChunkItem {
inner_module: ResolvedVc<EcmascriptClientReferenceModule>, inner_module: ResolvedVc<EcmascriptClientReferenceModule>,
inner_chunk_item: ResolvedVc<Box<dyn EcmascriptChunkItem>>, module_graph: ResolvedVc<ModuleGraph>,
chunking_context: ResolvedVc<Box<dyn ChunkingContext>>, chunking_context: ResolvedVc<Box<dyn ChunkingContext>>,
} }
...@@ -345,11 +342,12 @@ impl EcmascriptChunkItem for EcmascriptClientReferenceProxyChunkItem { ...@@ -345,11 +342,12 @@ impl EcmascriptChunkItem for EcmascriptClientReferenceProxyChunkItem {
async_module_info: Option<Vc<AsyncModuleInfo>>, async_module_info: Option<Vc<AsyncModuleInfo>>,
estimated: bool, estimated: bool,
) -> Result<Vc<EcmascriptChunkItemContent>> { ) -> Result<Vc<EcmascriptChunkItemContent>> {
self.inner_chunk_item Ok(self.inner_module.chunk_item_content(
.into_trait_ref() *self.chunking_context,
.await? *self.module_graph,
.content_with_async_module_info(async_module_info, estimated) async_module_info,
.await estimated,
))
} }
} }
......
import type { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
export default async function NotFound() {
const { bar } = await import('../packages/fixture/src/index.js')
return <main>{bar} not found</main>
}
export default async function Page() {
const { bar } = await import('../packages/fixture/src/index.js')
return <main>{bar}</main>
}
{
"name": "fixture",
"type": "module",
"sideEffects": false
}
import { nextTestSetup } from 'e2e-utils'
describe('pruned client-reference proxy', () => {
const { next } = nextTestSetup({ files: __dirname })
// The barrel also exports a client function. Expanding that export creates an RSC proxy, but
// neither route uses it, so its locals module has no module ID in the chunk graph.
it('builds when a dynamic barrel import only uses a server export', async () => {
const page = await next.render$('/')
expect(page('main').text()).toBe('bar')
const notFound = await next.render$('/missing')
expect(notFound('main').text()).toBe('bar not found')
})
})
...@@ -83,17 +83,10 @@ impl BindingUsageInfo { ...@@ -83,17 +83,10 @@ impl BindingUsageInfo {
) -> Result<Vc<ModuleExportUsage>> { ) -> Result<Vc<ModuleExportUsage>> {
let is_circuit_breaker = self.export_circuit_breakers.contains_key(&module).await?; let is_circuit_breaker = self.export_circuit_breakers.contains_key(&module).await?;
let Some(exports) = self.used_exports.get(&module).await? else { let Some(exports) = self.used_exports.get(&module).await? else {
// There are some module that are codegened, but not referenced in the module graph, bail!(
let ident = module.ident_string().await?; "export usage not found for module: {:?}",
if ident.contains(".wasm_.loader.mjs") || ident.contains("/__nextjs-internal-proxy.") { module.ident_string().await?
// Both the turbopack-wasm `ModuleChunkItem` and `EcmascriptClientReferenceModule` );
// do `self.slightly_different_module().as_chunk_item()`, so the
// module that codegen sees isn't actually in the module graph.
// TODO fix these cases
return Ok(ModuleExportUsage::unknown());
}
bail!("export usage not found for module: {ident:?}");
}; };
let namespace_object_may_escape = let namespace_object_may_escape =
self.partial_namespace_modules.contains_key(&module).await?; self.partial_namespace_modules.contains_key(&module).await?;
......
...@@ -1067,6 +1067,36 @@ pub struct EcmascriptModuleContentOptions { ...@@ -1067,6 +1067,36 @@ pub struct EcmascriptModuleContentOptions {
async_module_info: Option<ResolvedVc<AsyncModuleInfo>>, async_module_info: Option<ResolvedVc<AsyncModuleInfo>>,
} }
/// Generate the code of a processed virtual module under the identity of the module in the
/// graph. The code source supplies its parsed program, references, exports and async behavior;
/// the graph module supplies the identity for export usage and other code generation.
#[turbo_tasks::function]
pub async fn chunk_item_content_with_code_from(
module: ResolvedVc<Box<dyn EcmascriptChunkPlaceable>>,
code_source: ResolvedVc<Box<dyn EcmascriptChunkPlaceable>>,
chunking_context: ResolvedVc<Box<dyn ChunkingContext>>,
async_module_info: Option<Vc<AsyncModuleInfo>>,
) -> Result<Vc<EcmascriptChunkItemContent>> {
let Some(analyzable) = ResolvedVc::try_sidecast::<Box<dyn EcmascriptAnalyzable>>(code_source)
else {
bail!("virtual ECMAScript code source must be analyzable");
};
let source_options = analyzable
.module_content_options(*chunking_context, async_module_info)
.await?;
let mut options = (*source_options).clone();
options.module = module;
let content = EcmascriptModuleContent::new(options.cell());
let async_module_options = code_source
.get_async_module()
.module_options(async_module_info);
Ok(EcmascriptChunkItemContent::new(
content,
*chunking_context,
async_module_options,
))
}
impl EcmascriptModuleContentOptions { impl EcmascriptModuleContentOptions {
async fn merged_code_gens( async fn merged_code_gens(
&self, &self,
......
it('registers the retained source but does not look up the pruned one', async () => {
const { value } = await import('./barrel')
expect(value).toBe('value')
})
it('registers a partially used re-export facade whose other source was pruned', async () => {
const { value } = await import('./barrel')
expect(value).toBe('value')
})
{
"followReexports": true,
"scopeHoisting": true,
"globalModuleIds": true
}
...@@ -21,6 +21,7 @@ use turbopack_ecmascript::{ ...@@ -21,6 +21,7 @@ use turbopack_ecmascript::{
EcmascriptChunkItemContent, EcmascriptChunkPlaceable, EcmascriptExports, EcmascriptChunkItemContent, EcmascriptChunkPlaceable, EcmascriptExports,
ecmascript_chunk_item, ecmascript_chunk_item,
}, },
chunk_item_content_with_code_from,
references::async_module::OptionAsyncModule, references::async_module::OptionAsyncModule,
}; };
...@@ -224,11 +225,11 @@ impl EcmascriptChunkPlaceable for WebAssemblyModuleAsset { ...@@ -224,11 +225,11 @@ impl EcmascriptChunkPlaceable for WebAssemblyModuleAsset {
#[turbo_tasks::function] #[turbo_tasks::function]
async fn chunk_item_content( async fn chunk_item_content(
self: Vc<Self>, self: ResolvedVc<Self>,
chunking_context: Vc<Box<dyn ChunkingContext>>, chunking_context: ResolvedVc<Box<dyn ChunkingContext>>,
module_graph: Vc<ModuleGraph>, _module_graph: Vc<ModuleGraph>,
async_module_info: Option<Vc<AsyncModuleInfo>>, async_module_info: Option<Vc<AsyncModuleInfo>>,
estimated: bool, _estimated: bool,
) -> Result<Vc<EcmascriptChunkItemContent>> { ) -> Result<Vc<EcmascriptChunkItemContent>> {
if matches!( if matches!(
*chunking_context.chunk_loading().await?, *chunking_context.chunk_loading().await?,
...@@ -240,12 +241,13 @@ impl EcmascriptChunkPlaceable for WebAssemblyModuleAsset { ...@@ -240,12 +241,13 @@ impl EcmascriptChunkPlaceable for WebAssemblyModuleAsset {
); );
} }
// Delegate to the loader's chunk item content // Generate the loader's code under this module's graph identity. The loader is a
Ok(self.loader().chunk_item_content( // separately processed virtual module and has no export-usage entry of its own.
chunking_context, Ok(chunk_item_content_with_code_from(
module_graph, *ResolvedVc::upcast(self),
self.loader(),
*chunking_context,
async_module_info, async_module_info,
estimated,
)) ))
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment