Unverified Commit 740d55cd authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Turbopack: Add inline loader configuration via import attributes (#89644)

## What?

Adds support for inline loader configuration in Turbopack via import attributes. Developers can apply webpack-compatible loaders to individual imports using the `with` clause, without needing global `turbopack.rules` configuration.

```tsx
import rawText from '../data.txt' with { turbopackLoader: 'raw-loader', turbopackAs: '*.js' }
import value from '../data.js' with {
  turbopackLoader: 'string-replace-loader',
  turbopackLoaderOptions: '{"search":"PLACEHOLDER","replace":"replaced value"}'
}
```

## Why?

Global `turbopack.rules` applies loaders to all files matching a pattern. This PR provides per-import granularity — useful when only a specific import needs special treatment, avoiding side effects on other imports of the same file type.

## How?

### Import attributes parsing (`turbopack-ecmascript/src/analyzer/imports.rs`)
- Extended `ImportAnnotations` to parse four new attributes: `turbopackLoader`, `turbopackLoaderOptions`, `turbopackAs`, `turbopackModuleType`
- All values are strings per TC39 import attributes spec (`turbopackLoaderOptions` is a JSON-encoded string)
- Added unit tests for the parser

### New structured loader type (`turbopack-core/src/loader.rs`)
- Added `WebpackLoaderItem` struct shared between inline and global loader paths

### Reference type plumbing (`turbopack-core/src/reference_type.rs`, `turbopack-ecmascript/src/references/esm/base.rs`)
- Added `ImportWithTurbopackUse` variant to `EcmaScriptModulesReferenceSubType` carrying the parsed loader, rename_as, and module_type
- ESM reference resolution maps the parsed annotations to this new subtype

### Loader execution (`turbopack/src/lib.rs`)
- In `process_default_internal`, detects `ImportWithTurbopackUse` references and applies the loader as a source transform via the existing webpack loader infrastructure
- Requires `enable_webpack_loaders` to be configured (same as global rules)
- Supports `turbopackAs` (re-processes with new extension) and `turbopackModuleType` (directly sets module type)
- Avoids infinite recursion by switching to plain `Import` reference type on re-processing

### Documentation (`docs/.../turbopack.mdx`)
- Added section documenting the four import attributes with examples
- Added version history entry for v16.2.0

### Testing (`test/development/app-dir/turbopack-import-assertions-use/`)
- E2e test covering raw loader, replace loader with options, `turbopackModuleType: 'ecmascript'`, and `turbopackModuleType: 'json'`
- Turbopack-only (skipped for webpack)
parent 9abda03e
......@@ -151,10 +151,6 @@ pub async fn webpack_loader_options(
.await?,
);
if rules.is_empty() {
return Ok(Vc::cell(None));
}
Ok(Vc::cell(Some(
WebpackLoadersOptions {
rules: ResolvedVc::cell(rules),
......
......@@ -297,6 +297,38 @@ Available module types:
| `raw` | Return raw contents as string |
| `bytes` | Inline contents as bytes |
### Inline loader configuration with import attributes
You can apply a Turbopack loader to an individual import using the `with` clause (import attributes). This is specified per-import rather than globally via `turbopack.rules`.
This is useful when you want to apply a loader to a specific import without affecting all files of that type.
```tsx filename="app/page.tsx"
// Apply a raw loader to import a .txt file as a JavaScript module
import rawText from '../data.txt' with { turbopackLoader: 'raw-loader', turbopackAs: '*.js' }
export default function Page() {
return <p>{rawText}</p>
}
```
The following import attributes are supported:
| Attribute | Description |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
| `turbopackLoader` | The loader to apply (e.g. `'raw-loader'`). |
| `turbopackLoaderOptions` | JSON string of loader options (e.g. `'{"search":"X","replace":"Y"}'`). |
| `turbopackAs` | Rename pattern for the output (same as `turbopack.rules[].as`). For example, `'*.js'` treats the loader output as JavaScript. |
| `turbopackModuleType` | Set the module type for the output (same as `turbopack.rules[].type`). |
Loaders with options pass a JSON-encoded string via `turbopackLoaderOptions`:
```tsx filename="app/page.tsx"
import value from '../data.js' with { turbopackLoader: 'string-replace-loader', turbopackLoaderOptions: '{"search":"PLACEHOLDER","replace":"replaced value"}' }
```
> **Good to know**: Import attributes with `turbopackLoader` are Turbopack-specific and are not supported by webpack. This feature requires the `with` keyword (not `assert`) in your import statements.
### Resolving aliases
Turbopack can be configured to modify module resolution through aliases, similar to webpack's [`resolve.alias`](https://webpack.js.org/configuration/resolve/#resolvealias) configuration.
......@@ -356,6 +388,7 @@ The option automatically adds a polyfill for debug IDs to the JavaScript bundle
| Version | Changes |
| -------- | ---------------------------------------------------- |
| `16.2.0` | `turbopackLoader` import attributes were added. |
| `16.2.0` | `turbopack.rules.*.type` was added. |
| `16.2.0` | `turbopack.rules.*.condition.contentType` was added. |
| `16.2.0` | `turbopack.rules.*.condition.query` was added. |
......
import { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
// @ts-nocheck
// Import a .txt file using a raw loader via turbopackLoader
// turbopackAs tells turbopack to treat the loader output as JavaScript
import rawText from '../data.txt' with { turbopackLoader: 'test-raw-loader', turbopackAs: '*.js' }
// Import a .js file with a replace loader via turbopackLoader with options
import replacedValue from '../data-with-placeholder.js' with { turbopackLoader: 'test-replace-loader', turbopackLoaderOptions: '{"search":"PLACEHOLDER","replace":"turbopackUse works!"}' }
// Import a .txt file using turbopackModuleType to treat loader output as ecmascript
import rawTextViaModuleType from '../data2.txt' with { turbopackLoader: '../node_modules/test-raw-loader/index.js', turbopackModuleType: 'ecmascript' }
// Import a non-.json file and treat it as JSON via turbopackModuleType
import jsonData from '../data.jsonlike' with { turbopackLoader: 'test-identity-loader', turbopackModuleType: 'json' }
export default function Page() {
return (
<div>
<p id="raw">{rawText}</p>
<p id="replaced">{replacedValue}</p>
<p id="module-type">{rawTextViaModuleType}</p>
<p id="json-type">{jsonData.greeting}</p>
</div>
)
}
{"greeting": "Hello from JSON module type"}
\ No newline at end of file
Hello from raw text file
\ No newline at end of file
Hello via module type
\ No newline at end of file
/**
* @type {import('next').NextConfig}
*/
const nextConfig = {}
module.exports = nextConfig
// A loader that passes the source through unchanged
module.exports = function identityLoader(source) {
return source
}
{
"name": "test-identity-loader",
"version": "1.0.0",
"main": "index.js"
}
// A simple loader that exports the raw file content as a string
module.exports = function rawLoader(source) {
return `export default ${JSON.stringify(source)}`
}
{
"name": "test-raw-loader",
"version": "1.0.0",
"main": "index.js"
}
// A loader that replaces a placeholder with a specific value.
// Accepts options: { search: string, replace: string }
module.exports = function replaceLoader(source) {
const options = this.getOptions ? this.getOptions() : {}
const search = options.search || 'PLACEHOLDER'
const replace = options.replace || 'REPLACED'
return source.replaceAll(search, replace)
}
{
"name": "test-replace-loader",
"version": "1.0.0",
"main": "index.js"
}
import { nextTestSetup } from 'e2e-utils'
describe('turbopack-import-assertions-use', () => {
const { next, isTurbopack } = nextTestSetup({
files: __dirname,
// This test is Turbopack-only; turbopackUse is not supported in webpack
skipDeployment: true,
})
if (!isTurbopack) {
it('should skip for webpack', () => {})
return
}
it('should apply raw loader via turbopackUse import assertion', async () => {
const $ = await next.render$('/')
expect($('#raw').text()).toBe('Hello from raw text file')
})
it('should apply replace loader with options via turbopackUse import assertion', async () => {
const $ = await next.render$('/')
expect($('#replaced').text()).toBe('Value is: turbopackUse works!')
})
it('should apply raw loader with turbopackModuleType ecmascript', async () => {
const $ = await next.render$('/')
expect($('#module-type').text()).toBe('Hello via module type')
})
it('should apply identity loader with turbopackModuleType json', async () => {
const $ = await next.render$('/')
expect($('#json-type').text()).toBe('Hello from JSON module type')
})
})
......@@ -23,6 +23,7 @@ pub mod file_source;
pub mod ident;
pub mod introspect;
pub mod issue;
pub mod loader;
pub mod module;
pub mod module_graph;
pub mod node_addon_module;
......
use std::hash::{Hash, Hasher};
use bincode::{Decode, Encode};
use serde::{Deserialize, Serialize};
use turbo_tasks::{NonLocalValue, OperationValue, TaskInput, trace::TraceRawVcs};
use turbo_tasks_fs::FileSystemPath;
#[derive(
Clone,
PartialEq,
Eq,
Debug,
TraceRawVcs,
Serialize,
Deserialize,
NonLocalValue,
OperationValue,
Encode,
Decode,
)]
pub struct WebpackLoaderItem {
pub loader: turbo_rcstr::RcStr,
#[serde(default)]
#[bincode(with = "turbo_bincode::serde_self_describing")]
pub options: serde_json::Map<String, serde_json::Value>,
}
impl Hash for WebpackLoaderItem {
fn hash<H: Hasher>(&self, state: &mut H) {
self.loader.hash(state);
// serde_json::Map doesn't implement Hash, so hash the canonical JSON string.
// serde_json::Map preserves insertion order, and our maps are built
// deterministically from AST traversal, so this is stable.
let options_str = serde_json::to_string(&self.options).unwrap_or_default();
options_str.hash(state);
}
}
impl TaskInput for WebpackLoaderItem {
fn is_transient(&self) -> bool {
false
}
}
/// Like `WebpackLoaderItem`, but with the loader path already resolved to a `FileSystemPath`.
#[derive(Clone, PartialEq, Eq, Debug, TraceRawVcs, NonLocalValue, Encode, Decode)]
pub struct ResolvedWebpackLoaderItem {
pub loader: FileSystemPath,
#[bincode(with = "turbo_bincode::serde_self_describing")]
pub options: serde_json::Map<String, serde_json::Value>,
}
impl Hash for ResolvedWebpackLoaderItem {
fn hash<H: Hasher>(&self, state: &mut H) {
self.loader.hash(state);
let options_str = serde_json::to_string(&self.options).unwrap_or_default();
options_str.hash(state);
}
}
impl TaskInput for ResolvedWebpackLoaderItem {
fn is_transient(&self) -> bool {
false
}
}
#[derive(Debug, Clone)]
#[turbo_tasks::value(shared, transparent)]
pub struct WebpackLoaderItems(pub Vec<WebpackLoaderItem>);
......@@ -5,7 +5,7 @@ use bincode::{Decode, Encode};
use turbo_rcstr::RcStr;
use turbo_tasks::{FxIndexMap, NonLocalValue, ResolvedVc, TaskInput, Vc, trace::TraceRawVcs};
use crate::{module::Module, resolve::ModulePart};
use crate::{loader::ResolvedWebpackLoaderItem, module::Module, resolve::ModulePart};
/// Named references to inner assets. Modules can use them to allow to
/// per-module aliases of some requests to already created module assets.
......@@ -75,6 +75,12 @@ pub enum EcmaScriptModulesReferenceSubType {
ImportPart(ModulePart),
Import,
ImportWithType(RcStr),
/// Import with `turbopackLoader` attribute specifying an inline loader.
ImportWithTurbopackUse {
loader: ResolvedWebpackLoaderItem,
rename_as: Option<RcStr>,
module_type: Option<RcStr>,
},
DynamicImport,
Custom(u8),
#[default]
......@@ -311,6 +317,7 @@ pub enum ReferenceType {
Entry(EntryReferenceSubType),
Runtime,
Internal(ResolvedVc<InnerAssets>),
Loader,
Custom(u8),
#[default]
Undefined,
......@@ -323,6 +330,9 @@ impl Display for ReferenceType {
ReferenceType::CommonJs(_) => "commonjs",
ReferenceType::EcmaScriptModules(sub) => match sub {
EcmaScriptModulesReferenceSubType::ImportPart(_) => "EcmaScript Modules (part)",
EcmaScriptModulesReferenceSubType::ImportWithTurbopackUse { .. } => {
"EcmaScript Modules (turbopackUse)"
}
_ => "EcmaScript Modules",
},
ReferenceType::Css(_) => "css",
......@@ -332,6 +342,7 @@ impl Display for ReferenceType {
ReferenceType::Entry(_) => "entry",
ReferenceType::Runtime => "runtime",
ReferenceType::Internal(_) => "internal",
ReferenceType::Loader => "loader",
ReferenceType::Custom(_) => todo!(),
ReferenceType::Undefined => "undefined",
};
......@@ -379,6 +390,7 @@ impl ReferenceType {
}
ReferenceType::Runtime => matches!(other, ReferenceType::Runtime),
ReferenceType::Internal(_) => matches!(other, ReferenceType::Internal(_)),
ReferenceType::Loader => matches!(other, ReferenceType::Loader),
ReferenceType::Custom(_) => {
todo!()
}
......@@ -390,6 +402,9 @@ impl ReferenceType {
/// combination with [`ModuleRuleCondition::Internal`] to determine if a
/// rule should be applied to an internal asset/reference.
pub fn is_internal(&self) -> bool {
matches!(self, ReferenceType::Internal(_) | ReferenceType::Runtime)
matches!(
self,
ReferenceType::Internal(_) | ReferenceType::Runtime | ReferenceType::Loader
)
}
}
......@@ -14,7 +14,7 @@ use swc_core::{
};
use turbo_rcstr::{RcStr, rcstr};
use turbo_tasks::{FxIndexMap, FxIndexSet, ResolvedVc};
use turbopack_core::{issue::IssueSource, source::Source};
use turbopack_core::{issue::IssueSource, loader::WebpackLoaderItem, source::Source};
use super::{JsValue, ModuleValue, top_level_await::has_top_level_await};
use crate::{
......@@ -31,6 +31,13 @@ pub struct ImportAnnotations {
#[turbo_tasks(trace_ignore)]
#[bincode(with_serde)]
map: BTreeMap<Wtf8Atom, Wtf8Atom>,
/// Parsed turbopack loader configuration from import attributes.
/// e.g. `import "file" with { turbopackLoader: "raw-loader" }`
#[turbo_tasks(trace_ignore)]
#[bincode(with_serde)]
turbopack_loader: Option<WebpackLoaderItem>,
turbopack_rename_as: Option<RcStr>,
turbopack_module_type: Option<RcStr>,
}
/// Enables a specified transition for the annotated import
......@@ -51,30 +58,77 @@ impl ImportAnnotations {
};
let mut map = BTreeMap::new();
// The `with` clause is way more restrictive than `ObjectLit`, it only allows
// string -> value and value can only be a string.
// We just ignore everything else here till the SWC ast is more restrictive.
for (key, value) in with.props.iter().filter_map(|prop| {
let kv = prop.as_prop()?.as_key_value()?;
let Lit::Str(str) = kv.value.as_lit()? else {
return None;
let mut turbopack_loader_name: Option<RcStr> = None;
let mut turbopack_loader_options: serde_json::Map<String, serde_json::Value> =
serde_json::Map::new();
let mut turbopack_rename_as: Option<RcStr> = None;
let mut turbopack_module_type: Option<RcStr> = None;
for prop in &with.props {
let Some(kv) = prop.as_prop().and_then(|p| p.as_key_value()) else {
continue;
};
Some((&kv.key, str))
}) {
let key = match key {
PropName::Ident(ident) => ident.sym.clone().into(),
PropName::Str(str) => str.value.clone(),
// the rest are invalid, ignore for now till SWC ast is correct
let key_str = match &kv.key {
PropName::Ident(ident) => ident.sym.to_string(),
PropName::Str(str) => str.value.to_string_lossy().into_owned(),
_ => continue,
};
map.insert(key, value.value.clone());
// All turbopack* keys are extracted as string values (per TC39 import attributes spec)
match key_str.as_str() {
"turbopackLoader" => {
if let Some(Lit::Str(s)) = kv.value.as_lit() {
turbopack_loader_name =
Some(RcStr::from(s.value.to_string_lossy().into_owned()));
}
}
"turbopackLoaderOptions" => {
if let Some(Lit::Str(s)) = kv.value.as_lit() {
let json_str = s.value.to_string_lossy();
if let Ok(serde_json::Value::Object(map)) = serde_json::from_str(&json_str)
{
turbopack_loader_options = map;
}
}
}
"turbopackAs" => {
if let Some(Lit::Str(s)) = kv.value.as_lit() {
turbopack_rename_as =
Some(RcStr::from(s.value.to_string_lossy().into_owned()));
}
}
"turbopackModuleType" => {
if let Some(Lit::Str(s)) = kv.value.as_lit() {
turbopack_module_type =
Some(RcStr::from(s.value.to_string_lossy().into_owned()));
}
}
_ => {
// For all other keys, only accept string values (per spec)
if let Some(Lit::Str(str)) = kv.value.as_lit() {
let key: Wtf8Atom = match &kv.key {
PropName::Ident(ident) => ident.sym.clone().into(),
PropName::Str(s) => s.value.clone(),
_ => continue,
};
map.insert(key, str.value.clone());
}
}
}
}
ImportAnnotations { map }
let turbopack_loader = turbopack_loader_name.map(|name| WebpackLoaderItem {
loader: name,
options: turbopack_loader_options,
});
ImportAnnotations {
map,
turbopack_loader,
turbopack_rename_as,
turbopack_module_type,
}
}
pub fn parse_dynamic(with: &JsValue) -> Option<ImportAnnotations> {
......@@ -102,7 +156,12 @@ impl ImportAnnotations {
);
}
Some(ImportAnnotations { map })
Some(ImportAnnotations {
map,
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
})
}
/// Returns the content on the transition annotation
......@@ -121,6 +180,26 @@ impl ImportAnnotations {
self.get(&ATTRIBUTE_MODULE_TYPE)
}
/// Returns the turbopackLoader item, if present
pub fn turbopack_loader(&self) -> Option<&WebpackLoaderItem> {
self.turbopack_loader.as_ref()
}
/// Returns the turbopackAs rename configuration, if present
pub fn turbopack_rename_as(&self) -> Option<&RcStr> {
self.turbopack_rename_as.as_ref()
}
/// Returns the turbopackModuleType override, if present
pub fn turbopack_module_type(&self) -> Option<&RcStr> {
self.turbopack_module_type.as_ref()
}
/// Returns true if a turbopack loader is configured
pub fn has_turbopack_loader(&self) -> bool {
self.turbopack_loader.is_some()
}
pub fn get(&self, key: &Wtf8Atom) -> Option<&Wtf8Atom> {
self.map.get(key)
}
......@@ -861,3 +940,90 @@ fn get_import_symbol_from_export(specifier: &ExportSpecifier) -> ImportedSymbol
ExportSpecifier::Namespace(..) => ImportedSymbol::Exports,
}
}
#[cfg(test)]
mod tests {
use swc_core::{atoms::Atom, common::DUMMY_SP, ecma::ast::*};
use super::*;
/// Helper to create a string literal expression
fn str_lit(s: &str) -> Box<Expr> {
Box::new(Expr::Lit(Lit::Str(Str {
span: DUMMY_SP,
value: Atom::from(s).into(),
raw: None,
})))
}
/// Helper to create an ident property name
fn ident_key(s: &str) -> PropName {
PropName::Ident(IdentName {
span: DUMMY_SP,
sym: Atom::from(s),
})
}
/// Helper to create a key-value property
fn kv_prop(key: PropName, value: Box<Expr>) -> PropOrSpread {
PropOrSpread::Prop(Box::new(Prop::KeyValue(KeyValueProp { key, value })))
}
#[test]
fn test_parse_turbopack_loader_annotation() {
// Simulate: with { turbopackLoader: "raw-loader" }
let with = ObjectLit {
span: DUMMY_SP,
props: vec![kv_prop(ident_key("turbopackLoader"), str_lit("raw-loader"))],
};
let annotations = ImportAnnotations::parse(Some(&with));
assert!(annotations.has_turbopack_loader());
let loader = annotations.turbopack_loader().unwrap();
assert_eq!(loader.loader.as_str(), "raw-loader");
assert!(loader.options.is_empty());
}
#[test]
fn test_parse_turbopack_loader_with_options() {
// Simulate: with { turbopackLoader: "my-loader", turbopackLoaderOptions: '{"flag":true}' }
let with = ObjectLit {
span: DUMMY_SP,
props: vec![
kv_prop(ident_key("turbopackLoader"), str_lit("my-loader")),
kv_prop(
ident_key("turbopackLoaderOptions"),
str_lit(r#"{"flag":true}"#),
),
],
};
let annotations = ImportAnnotations::parse(Some(&with));
assert!(annotations.has_turbopack_loader());
let loader = annotations.turbopack_loader().unwrap();
assert_eq!(loader.loader.as_str(), "my-loader");
assert_eq!(loader.options["flag"], serde_json::Value::Bool(true));
}
#[test]
fn test_parse_without_turbopack_loader() {
// Simulate: with { type: "json" }
let with = ObjectLit {
span: DUMMY_SP,
props: vec![kv_prop(ident_key("type"), str_lit("json"))],
};
let annotations = ImportAnnotations::parse(Some(&with));
assert!(!annotations.has_turbopack_loader());
assert!(annotations.module_type().is_some());
}
#[test]
fn test_parse_empty_with() {
let annotations = ImportAnnotations::parse(None);
assert!(!annotations.has_turbopack_loader());
assert!(annotations.module_type().is_none());
}
}
......@@ -18,16 +18,19 @@ use turbopack_core::{
Issue, IssueExt, IssueSeverity, IssueSource, IssueStage, OptionIssueSource,
OptionStyledString, StyledString,
},
loader::ResolvedWebpackLoaderItem,
module::{Module, ModuleSideEffects},
module_graph::binding_usage_info::ModuleExportUsageInfo,
reference::ModuleReference,
reference_type::EcmaScriptModulesReferenceSubType,
reference_type::{EcmaScriptModulesReferenceSubType, ReferenceType},
resolve::{
BindingUsage, ExportUsage, ExternalType, ImportUsage, ModulePart, ModuleResolveResult,
ModuleResolveResultItem, RequestKey, ResolveErrorMode,
origin::{ResolveOrigin, ResolveOriginExt},
parse::Request,
resolve,
},
source::Source,
};
use turbopack_resolve::ecmascript::esm_resolve;
......@@ -408,7 +411,32 @@ impl EsmAssetReference {
impl ModuleReference for EsmAssetReference {
#[turbo_tasks::function]
async fn resolve_reference(&self) -> Result<Vc<ModuleResolveResult>> {
let ty = if let Some(module_type) = self.annotations.module_type() {
let ty = if let Some(loader) = self.annotations.turbopack_loader() {
// Resolve the loader path relative to the importing file
let origin = self.get_origin();
let origin_path = origin.origin_path().await?;
let loader_request = Request::parse(loader.loader.clone().into());
let resolved = resolve(
origin_path.parent(),
ReferenceType::Loader,
loader_request,
origin.resolve_options(),
);
let loader_fs_path = if let Some(source) = *resolved.first_source().await? {
(*source.ident().path().await?).clone()
} else {
bail!("Unable to resolve turbopackLoader '{}'", loader.loader);
};
EcmaScriptModulesReferenceSubType::ImportWithTurbopackUse {
loader: ResolvedWebpackLoaderItem {
loader: loader_fs_path,
options: loader.options.clone(),
},
rename_as: self.annotations.turbopack_rename_as().cloned(),
module_type: self.annotations.turbopack_module_type().cloned(),
}
} else if let Some(module_type) = self.annotations.module_type() {
EcmaScriptModulesReferenceSubType::ImportWithType(RcStr::from(
&*module_type.to_string_lossy(),
))
......
......@@ -123,6 +123,9 @@
module: "./module.js",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -97,6 +97,9 @@
module: "node:module",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -307,6 +310,9 @@
module: "node:module",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -60,6 +60,9 @@
module: "node:module",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -60,6 +60,9 @@
module: "./module.js",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -101,6 +104,9 @@
module: "./module.js",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -8,6 +8,9 @@
module: "x2",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -29,6 +32,9 @@
module: "x1",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -48,6 +54,9 @@
module: "x3",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -73,6 +73,9 @@
module: "react",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -578,6 +581,9 @@
module: "https://esm.sh/preact/jsx-runtime",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -954,6 +960,9 @@
module: "./external",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -1077,6 +1086,9 @@
module: "./external",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -72,6 +72,9 @@
module: "https://esm.sh/preact/jsx-runtime",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -150,6 +153,9 @@
module: "react",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -85,6 +85,9 @@
module: "./assert",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -369,6 +372,9 @@
module: "./assert",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -514,6 +520,9 @@
module: "./assert",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -42,6 +42,9 @@
module: "@upstash/redis",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -1030,6 +1030,9 @@
module: "path",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -83,6 +83,9 @@
module: "path",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -112,6 +115,9 @@
module: "path",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -62,6 +62,9 @@
module: "foo",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -229,6 +232,9 @@
module: "foo",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -361,6 +367,9 @@
module: "foo",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -542,6 +551,9 @@
module: "foo",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -674,6 +686,9 @@
module: "foo",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......@@ -855,6 +870,9 @@
module: "foo",
annotations: ImportAnnotations {
map: {},
turbopack_loader: None,
turbopack_rename_as: None,
turbopack_module_type: None,
},
},
),
......
......@@ -10,8 +10,8 @@ use serde_json::{Map as JsonMap, Value as JsonValue, json};
use serde_with::serde_as;
use turbo_rcstr::{RcStr, rcstr};
use turbo_tasks::{
Completion, NonLocalValue, OperationValue, OperationVc, ResolvedVc, TaskInput, TryJoinIterExt,
ValueToString, Vc, trace::TraceRawVcs,
Completion, OperationVc, ResolvedVc, TaskInput, TryJoinIterExt, ValueToString, Vc,
trace::TraceRawVcs,
};
use turbo_tasks_env::ProcessEnv;
use turbo_tasks_fs::{
......@@ -82,29 +82,7 @@ struct WebpackLoadersProcessingResult {
assets: Option<Vec<EmittedAsset>>,
}
#[derive(
Clone,
PartialEq,
Eq,
Debug,
TraceRawVcs,
Serialize,
Deserialize,
NonLocalValue,
OperationValue,
Encode,
Decode,
)]
pub struct WebpackLoaderItem {
pub loader: RcStr,
#[serde(default)]
#[bincode(with = "turbo_bincode::serde_self_describing")]
pub options: serde_json::Map<String, serde_json::Value>,
}
#[derive(Debug, Clone)]
#[turbo_tasks::value(shared, transparent)]
pub struct WebpackLoaderItems(pub Vec<WebpackLoaderItem>);
pub use turbopack_core::loader::{WebpackLoaderItem, WebpackLoaderItems};
#[turbo_tasks::value]
pub struct WebpackLoaders {
......
......@@ -12,8 +12,10 @@ pub mod global_module_ids;
pub mod module_options;
pub mod transition;
use anyhow::{Result, bail};
use module_options::{ModuleOptions, ModuleOptionsContext, ModuleRuleEffect, ModuleType};
use anyhow::{Context as _, Result, bail};
use module_options::{
ConfiguredModuleType, ModuleOptions, ModuleOptionsContext, ModuleRuleEffect, ModuleType,
};
use tracing::{Instrument, field::Empty};
use turbo_rcstr::{RcStr, rcstr};
use turbo_tasks::{ResolvedVc, TryJoinIterExt, ValueToString, Vc};
......@@ -40,11 +42,12 @@ use turbopack_core::{
parse::Request, resolve,
},
source::Source,
source_transform::SourceTransforms,
};
use turbopack_css::{CssModuleAsset, ModuleCssAsset};
use turbopack_ecmascript::{
AnalyzeMode, EcmascriptInputTransforms, EcmascriptModuleAsset, EcmascriptModuleAssetType,
TreeShakingMode,
EcmascriptOptions, TreeShakingMode,
chunk::EcmascriptChunkPlaceable,
references::{
FollowExportsResult,
......@@ -57,6 +60,7 @@ use turbopack_ecmascript::{
tree_shake::asset::EcmascriptModulePartAsset,
};
use turbopack_json::JsonModuleAsset;
use turbopack_node::transforms::webpack::{WebpackLoaderItem, WebpackLoaderItems, WebpackLoaders};
use turbopack_resolve::{
resolve::resolve_options, resolve_options_context::ResolveOptionsContext,
typescript::type_resolve,
......@@ -65,8 +69,10 @@ use turbopack_static::{css::StaticUrlCssModule, ecma::StaticUrlJsModule};
use turbopack_wasm::{module_asset::WebAssemblyModuleAsset, source::WebAssemblySource};
use crate::{
evaluate_context::node_evaluate_asset_context,
module_options::{
CssOptionsContext, CustomModuleType, EcmascriptOptionsContext, TypescriptTransformOptions,
package_import_map_from_context, package_import_map_from_import_mapping,
},
transition::{Transition, TransitionOptions},
};
......@@ -668,6 +674,150 @@ async fn process_default_internal(
let mut current_source = source;
let mut current_module_type = None;
// Handle turbopackLoader import attributes: apply inline loader as source transform
if let ReferenceType::EcmaScriptModules(
EcmaScriptModulesReferenceSubType::ImportWithTurbopackUse {
ref loader,
ref rename_as,
ref module_type,
},
) = reference_type
{
let module_options_context = module_asset_context.module_options_context().await?;
let webpack_loaders_options = module_options_context
.enable_webpack_loaders
.as_ref()
.context(
"turbopackUse import assertions require webpack loaders to be enabled \
(enable_webpack_loaders)",
)?
.await?;
let execution_context = module_options_context
.execution_context
.context("execution_context is required for turbopackUse import assertions")?;
let execution_context_value = execution_context.await?;
let resolve_options_context = module_asset_context
.resolve_options_context()
.to_resolved()
.await?;
let source_maps = matches!(
module_options_context.ecmascript.source_maps,
SourceMapsType::Full
);
// Determine the import map for loader-runner
let loader_runner_package = webpack_loaders_options.loader_runner_package;
let import_map = if let Some(loader_runner_package) = loader_runner_package {
package_import_map_from_import_mapping(rcstr!("loader-runner"), *loader_runner_package)
} else {
package_import_map_from_context(
rcstr!("loader-runner"),
execution_context_value.project_path.clone(),
)
};
let evaluate_context = node_evaluate_asset_context(
*execution_context,
Some(import_map),
None,
Layer::new(rcstr!("turbopack_use_loaders")),
false,
)
.to_resolved()
.await?;
let loader_relative_path = execution_context_value
.project_path
.get_relative_path_to(&loader.loader)
.context("Loader path must be on project filesystem")?;
let webpack_loader_item = WebpackLoaderItem {
loader: loader_relative_path,
options: loader.options.clone(),
};
let loaders_vc = WebpackLoaderItems(vec![webpack_loader_item]).cell();
let webpack_loaders = WebpackLoaders::new(
*evaluate_context,
*execution_context,
loaders_vc,
rename_as.clone(),
*resolve_options_context,
source_maps,
)
.to_resolved()
.await?;
let transforms = Vc::<SourceTransforms>::cell(vec![ResolvedVc::upcast(webpack_loaders)]);
current_source = transforms.transform(*current_source).to_resolved().await?;
// If turbopackModuleType is specified, skip rule matching and directly
// apply the requested module type with empty transforms (loader output
// is already processed).
if let Some(type_str) = module_type {
let empty_transforms = EcmascriptInputTransforms::empty().to_resolved().await?;
let default_options = EcmascriptOptions::default().resolved_cell();
let effect = ConfiguredModuleType::parse(type_str)?
.into_effect(
empty_transforms,
empty_transforms,
empty_transforms,
default_options,
None,
)
.await?;
match effect {
ModuleRuleEffect::ModuleType(module_type) => {
return apply_module_type(
current_source,
module_asset_context,
module_type.cell(),
part,
inner_assets,
None,
false,
)
.await;
}
ModuleRuleEffect::SourceTransforms(transforms) => {
current_source = transforms.transform(*current_source).to_resolved().await?;
// Fall through to re-process with new ident
}
_ => bail!("Unexpected module rule effect for turbopackModuleType"),
}
}
// If the ident changed (e.g., due to rename_as), re-process from the
// beginning so the new extension is matched by the correct rules.
// Use a plain Import reference type to avoid re-applying turbopackUse
// loaders in the recursive call (which would cause an infinite loop).
if current_source.ident().to_resolved().await? != ident {
let plain_reference_type =
ReferenceType::EcmaScriptModules(EcmaScriptModulesReferenceSubType::Import);
if let Some(transition) = module_asset_context
.await?
.transitions
.await?
.get_by_rules(current_source, &plain_reference_type)
.await?
{
return Ok(transition.process(
*current_source,
module_asset_context,
plain_reference_type,
));
} else {
return Box::pin(process_default(
module_asset_context,
current_source,
plain_reference_type,
processed_rules,
))
.await;
}
}
}
// Collect transforms from ExtendEcmascriptTransforms effects.
// They will be applied when ModuleType is set.
let mut collected_preprocess: Vec<ResolvedVc<EcmascriptInputTransforms>> = Vec::new();
......
......@@ -40,7 +40,7 @@ use turbopack_wasm::source::WebAssemblySourceType;
use crate::evaluate_context::{config_tracing_module_context, node_evaluate_asset_context};
#[turbo_tasks::function]
fn package_import_map_from_import_mapping(
pub(crate) fn package_import_map_from_import_mapping(
package_name: RcStr,
package_mapping: ResolvedVc<ImportMapping>,
) -> Vc<ImportMap> {
......@@ -53,7 +53,7 @@ fn package_import_map_from_import_mapping(
}
#[turbo_tasks::function]
fn package_import_map_from_context(
pub(crate) fn package_import_map_from_context(
package_name: RcStr,
context_path: FileSystemPath,
) -> Vc<ImportMap> {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment