Unverified Commit d46516ce authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

Turbopack: fix error reporting with crashing webpack loaders (#93926)

### What?

When a Turbopack webpack-loader subprocess crashes (e.g. a loader calls
`process.exit()`, a native fatal error, or the IPC socket otherwise
closes mid-message), the error users see today is:

```
- Execution of <WebpackLoadersProcessedAsset as Asset>::content failed
- Execution of WebpackLoadersProcessedAsset::process failed
- Execution of evaluate_webpack_loader failed
- failed to receive message
- reading packet length
- unexpected end of file
```

After this PR, the same crash produces:

```
⨯ ./data/crash.data
Error evaluating Node.js code
Error: Node.js subprocess crashed while evaluating loaders [/path/to/loaders/crash-loader.js]: failed to receive message

Caused by:
- Node.js process exited with exit status: 7
- reading packet length
- unexpected end of file

Debug info:
- failed to receive message
- Node.js process exited with exit status: 7
  Recent process stderr:
  <whatever the loader wrote to stderr before exiting>
- reading packet length
- unexpected end of file
```

### Why?

The original message gave no actionable information: no exit code, no
captured stdout/stderr, no indication of which loader was running. It
also looked like an internal turbopack bug rather than a user-fixable
error, and a transient pool failure could cascade into an unrelated
"issue formatter crashed while reading the source for a code frame"
failure on the way out.

### How?

Four orthogonal fixes, plus a regression test:

1. **Capture stdout/stderr on subprocess crash.** `OutputStreamHandler`
now keeps a bounded ring buffer (last 100 lines per stream) shared with
the owning `NodeJsPoolProcess`. When `NodeJsPoolProcess::recv` fails,
the buffers and the child's exit status are attached to the error via
`anyhow::Error::context`.
2. **Recover from subprocess crash in `pull_operation`.** Instead of
propagating the recv error up through `evaluate_webpack_loader` →
`process()` → `Asset::content` (the cascade above), `pull_operation`
catches it, synthesizes a `StructuredError` via
`evaluate_context.emit_error(...)`, disables process reuse, and returns
`Ok(None)`. This mirrors the existing in-band loader-error path, so the
asset's existing `FileContent::NotFound` degradation kicks in naturally
— `Asset::content` never errors.
3. **Include the loader chain in the error message and issue detail.**
`WebpackLoaderContext` gained a `loader_names: Vec<RcStr>` field. A new
optional `EvaluateContext::crash_context_prefix()` trait method lets
webpack-loader evaluations describe what was being evaluated (\"loaders
[a, b, c]\") in the synthesized crash message. `EvaluationIssue` also
gained an optional `detail` field for the same chain, surfacing it in
`--log-detail` output. PostCSS evaluations are labelled \"postcss\".
4. **Crash-proof the issue formatter.** `PlainSource::from_source` and
`IssueSource::into_plain` previously propagated errors from
`asset.content()` with `?`. They now degrade to `FileContent::NotFound`
(and `range = None`) on read failure, so a future regression in some
other code path can never cause the issue reporter itself to crash on
top of whatever the user was debugging.

### Tests

- Added
`test/e2e/app-dir/webpack-loader-errors/loaders/crash-loader.js`: a
loader that writes a marker to stderr and calls `process.exit(7)`.
- Added an e2e test that fetches `/crash` and asserts the marker, the
absence of the internal cascade, the loader name, and the resource name
are all present in the CLI output.
- All 11 tests in `webpack-loader-errors.test.ts` pass; the 5 Rust
`turbopack-node` pool tests still pass.

Some snapshot/golden tests for error formatting may need updating in CI
since `EvaluationIssue` now emits a non-empty `detail`.

<!-- NEXT_JS_LLM_PR -->
parent 40d7843d
const data = require('../../data/crash.data')
export default function Page() {
return <p>{data.default}</p>
}
module.exports = function crashLoader() {
process.stderr.write('TURBOPACK_CRASH_LOADER_STDERR_MARKER\n')
process.exit(7)
}
......@@ -28,6 +28,10 @@ const nextConfig = {
loaders: [require.resolve('./loaders/fs-error-loader.js')],
as: '*.js',
},
'crash.data': {
loaders: [require.resolve('./loaders/crash-loader.js')],
as: '*.js',
},
},
},
webpack(config) {
......@@ -55,6 +59,10 @@ const nextConfig = {
{
test: /fs-error\.data$/,
use: [require.resolve('./loaders/fs-error-loader.js')],
},
{
test: /crash\.data$/,
use: [require.resolve('./loaders/crash-loader.js')],
}
)
return config
......
......@@ -88,6 +88,31 @@ describe('webpack-loader-errors', () => {
expect(output).toMatch(/\(from .+loaders\/fs-error-loader/)
})
})
// Turbopack-only: webpack runs loaders in-process, so a loader calling
// process.exit() would kill the dev server itself. In Turbopack the
// loader runs in a Node.js subprocess from a worker pool; a hard
// process exit closes the IPC socket mid-message. This used to surface
// as an opaque "failed to receive message / unexpected end of file"
// cascade with no diagnostic context.
it('should surface a useful error when a loader crashes the Node.js subprocess', async () => {
await next.fetch('/crash')
await retry(async () => {
const output = stripAnsi(next.cliOutput)
// The crashing loader wrote to stderr before exiting. With the
// fix, that output is captured and attached to the error.
expect(output).toContain('TURBOPACK_CRASH_LOADER_STDERR_MARKER')
// The crash should not surface as the raw internal cascade.
expect(output).not.toContain(
'<WebpackLoadersProcessedAsset as Asset>::content failed'
)
// The synthesized error should reference both the crashing
// resource and the loader that was running, so the user knows
// exactly which loader to look at.
expect(output).toContain('crash.data')
expect(output).toMatch(/loaders \[[^\]]*loaders\/crash-loader/)
}, 30_000)
})
}
})
......
......@@ -497,7 +497,12 @@ impl IssueSource {
let mut range = match range {
SourceRange::LineColumn(start, end) => Some((start, end)),
SourceRange::ByteOffset(start, end) => {
if let FileLinesContent::Lines(lines) = &*self.source.content().lines().await? {
// Defensively read the content, an error there should not prevent all issue
// formatting. Best practice is for `content` to return `NotFound` instead of
// an error.
if let Ok(content) = self.source.content().lines().await
&& let FileLinesContent::Lines(lines) = &*content
{
let start = find_line_and_column(lines.as_ref(), start);
let end = find_line_and_column(lines.as_ref(), end);
Some((start, end))
......@@ -1034,15 +1039,22 @@ pub struct PlainSource {
impl PlainSource {
#[turbo_tasks::function]
pub async fn from_source(asset: ResolvedVc<Box<dyn Source>>) -> Result<Vc<PlainSource>> {
let asset_content = asset.content().await?;
let content = match *asset_content {
AssetContent::File(file_content) => file_content.await?,
AssetContent::Redirect { .. } => ReadRef::new_owned(FileContent::NotFound),
// Defensively read the content, an error there should not prevent all issue
// formatting. Best practice is for `content` to return `NotFound` instead of
// an error.
let content = if let Ok(asset_content) = asset.content().await
&& let AssetContent::File(file_content) = &*asset_content
&& let Ok(file_content) = file_content.await
{
file_content
} else {
ReadRef::new_owned(FileContent::NotFound)
};
let ident = asset.ident();
Ok(PlainSource {
ident: asset.ident().to_string().owned().await?,
file_path: asset.ident().await?.path.to_string_ref().await?,
ident: ident.to_string().owned().await?,
file_path: ident.await?.path.to_string_ref().await?,
content,
}
.cell())
......
......@@ -352,6 +352,15 @@ pub trait EvaluateContext {
state: Self::State,
pool: &EvaluatePool,
) -> impl Future<Output = Result<()>> + Send;
/// Optional human-readable prefix describing *what was being evaluated*,
/// included verbatim in the message of the synthetic [`StructuredError`]
/// emitted when the Node.js subprocess crashes mid-evaluation. For
/// webpack-loader evaluations this is the loader chain ("loaders
/// [foo, bar]"). The default returns `None`.
fn crash_context_prefix(&self) -> Option<RcStr> {
None
}
}
pub async fn custom_evaluate(evaluate_context: impl EvaluateContext) -> Result<Vc<Option<RcStr>>> {
......@@ -546,7 +555,34 @@ async fn pull_operation<T: EvaluateContext>(
let _guard = duration_span!("Node.js evaluation");
loop {
let message = serde_json::from_slice(&operation.recv().await?)?;
let recv_result = operation.recv().await;
let bytes = match recv_result {
Ok(bytes) => bytes,
Err(err) => {
// The Node.js subprocess crashed (or some other IPC failure
// closed the connection) before sending a response. Convert
// this into a synthesized issue with whatever diagnostic
// context the pool managed to capture, so the user sees a
// real error message instead of an internal turbo-tasks
// execution-failed cascade.
let message = match evaluate_context.crash_context_prefix() {
Some(prefix) => format!(
"Node.js subprocess crashed while evaluating {}: {}",
prefix,
PrettyPrintError(&err)
),
None => format!(
"Node.js subprocess crashed while evaluating: {}",
PrettyPrintError(&err)
),
};
let synthetic = StructuredError::from_message("Error".to_string(), message);
evaluate_context.emit_error(synthetic, pool).await?;
operation.disallow_reuse();
return Ok(None);
}
};
let message = serde_json::from_slice(&bytes)?;
match message {
EvalJavaScriptIncomingMessage::Error(error) => {
......@@ -647,6 +683,7 @@ impl EvaluateContext for BasicEvaluateContext {
assets_for_source_mapping: pool.assets_for_source_mapping,
assets_root: pool.assets_root.clone(),
root_path: self.chunking_context.root_path().owned().await?,
detail: None,
}
.resolved_cell()
.emit();
......@@ -684,6 +721,10 @@ pub struct EvaluationIssue {
pub assets_for_source_mapping: ResolvedVc<AssetsForSourceMapping>,
pub assets_root: FileSystemPath,
pub root_path: FileSystemPath,
/// Optional extra context shown only when log details are enabled — e.g.
/// the loader chain that was running when a webpack-loader subprocess
/// errored.
pub detail: Option<RcStr>,
}
#[async_trait]
......@@ -715,6 +756,10 @@ impl Issue for EvaluationIssue {
)))
}
async fn detail(&self) -> Result<Option<StyledString>> {
Ok(self.detail.clone().map(StyledString::Text))
}
fn source(&self) -> Option<IssueSource> {
Some(self.source)
}
......
use std::{
collections::VecDeque,
future::Future,
mem::take,
path::{Path, PathBuf},
......@@ -46,6 +47,11 @@ struct NodeJsPoolProcess {
connection: TcpStream,
stdout_handler: OutputStreamHandler<ChildStdout, Stdout>,
stderr_handler: OutputStreamHandler<ChildStderr, Stderr>,
/// Shared ring buffer of recent stdout lines; lives independently of
/// `stdout_handler` so the post-mortem on a recv failure can still read
/// it after the handler future has returned an error.
recent_stdout: RecentLines,
recent_stderr: RecentLines,
debug: bool,
cpu_time_invested: Duration,
}
......@@ -91,6 +97,17 @@ static GLOBAL_OUTPUT_LOCK: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_ne
static MARKER: &[u8] = b"TURBOPACK_OUTPUT_";
static MARKER_STR: &str = "TURBOPACK_OUTPUT_";
/// Maximum number of recent output lines retained per stream for diagnostic
/// purposes — included in the error message when the subprocess crashes
/// before completing a recv.
const RECENT_OUTPUT_LINES: usize = 100;
/// A bounded ring buffer of recent lines from a child stream. Shared with the
/// owning [`NodeJsPoolProcess`] so that the post-mortem code on a recv failure
/// can read what the child wrote even if the handler future has already
/// returned an error (and dropped its local state).
type RecentLines = Arc<Mutex<VecDeque<Vec<u8>>>>;
struct OutputStreamHandler<R: AsyncRead + Unpin, W: AsyncWrite + Unpin> {
stream: BufReader<R>,
shared: SharedOutputSet,
......@@ -98,6 +115,7 @@ struct OutputStreamHandler<R: AsyncRead + Unpin, W: AsyncWrite + Unpin> {
root: FileSystemPath,
project_dir: FileSystemPath,
final_stream: W,
recent_lines: RecentLines,
}
impl<R: AsyncRead + Unpin, W: AsyncWrite + Unpin> OutputStreamHandler<R, W> {
......@@ -113,6 +131,7 @@ impl<R: AsyncRead + Unpin, W: AsyncWrite + Unpin> OutputStreamHandler<R, W> {
root,
project_dir,
final_stream,
recent_lines,
} = self;
async fn write_final<W: AsyncWrite + Unpin>(
......@@ -183,9 +202,17 @@ impl<R: AsyncRead + Unpin, W: AsyncWrite + Unpin> OutputStreamHandler<R, W> {
{
bail!("stream closed unexpectedly")
}
if buffer.len() - start == MARKER.len() + 2
&& &buffer[start..buffer.len() - 2] == MARKER
{
// Mirror the just-read line into the recent-lines ring buffer for
// diagnostic capture on subprocess crash. Skip protocol markers.
let line_is_marker = buffer.len() - start == MARKER.len() + 2
&& &buffer[start..buffer.len() - 2] == MARKER;
if !line_is_marker {
let mut lines = recent_lines.lock();
if lines.len() >= RECENT_OUTPUT_LINES {
lines.pop_front();
}
lines.push_back(buffer[start..].to_vec());
} else {
// This is new line
buffer.pop();
// This is the type
......@@ -373,6 +400,11 @@ impl NodeJsPoolProcess {
let child_stdout = BufReader::new(child.stdout.take().unwrap());
let child_stderr = BufReader::new(child.stderr.take().unwrap());
let recent_stdout: RecentLines =
Arc::new(Mutex::new(VecDeque::with_capacity(RECENT_OUTPUT_LINES)));
let recent_stderr: RecentLines =
Arc::new(Mutex::new(VecDeque::with_capacity(RECENT_OUTPUT_LINES)));
let stdout_handler = OutputStreamHandler {
stream: child_stdout,
shared: shared_stdout,
......@@ -380,6 +412,7 @@ impl NodeJsPoolProcess {
root: assets_root.clone(),
project_dir: project_dir.clone(),
final_stream: stdout(),
recent_lines: recent_stdout.clone(),
};
let stderr_handler = OutputStreamHandler {
stream: child_stderr,
......@@ -388,6 +421,7 @@ impl NodeJsPoolProcess {
root: assets_root.clone(),
project_dir: project_dir.clone(),
final_stream: stderr(),
recent_lines: recent_stderr.clone(),
};
let mut process = Self {
......@@ -395,6 +429,8 @@ impl NodeJsPoolProcess {
connection,
stdout_handler,
stderr_handler,
recent_stdout,
recent_stderr,
debug,
cpu_time_invested: Duration::ZERO,
};
......@@ -455,11 +491,56 @@ impl NodeJsPoolProcess {
self.stdout_handler.handle_operation(),
self.stderr_handler.handle_operation(),
);
let result: Bytes = result?;
let result = match result {
Err(err) => {
// The IPC read failed — the most common cause is the child
// process crashing or exiting before sending a response.
// Attach whatever output the child wrote (recent lines from
// both stream handlers + the child's exit status, if
// available) so the user sees something diagnostic instead
// of an opaque "unexpected end of file" cascade.
return Err(self.diagnostic_recv_error(err).await);
}
Ok(result) => result,
};
stdout.context("unable to handle stdout from the Node.js process in a structured way")?;
stderr.context("unable to handle stderr from the Node.js process in a structured way")?;
Ok(result)
}
/// Build a contextualized error for a failed [`Self::recv`]. Includes
/// the captured stdout/stderr ring buffers and, if the child has
/// terminated, its exit status.
async fn diagnostic_recv_error(&mut self, err: anyhow::Error) -> anyhow::Error {
let exit_status = match self.child.as_mut() {
Some(child) => match timeout(Duration::from_secs(2), child.wait()).await {
Ok(Ok(status)) => Some(status),
_ => None,
},
None => None,
};
let mut output = String::new();
let stdout_lines: Vec<Vec<u8>> = self.recent_stdout.lock().iter().cloned().collect();
let stderr_lines: Vec<Vec<u8>> = self.recent_stderr.lock().iter().cloned().collect();
if !stdout_lines.is_empty() {
output.push_str("\nRecent process stdout:\n");
for line in stdout_lines {
output.push_str(&String::from_utf8_lossy(&line));
}
}
if !stderr_lines.is_empty() {
output.push_str("\nRecent process stderr:\n");
for line in stderr_lines {
output.push_str(&String::from_utf8_lossy(&line));
}
}
let exit_note = match exit_status {
Some(status) => format!("Node.js process exited with {status}"),
None => "Node.js process is still running or its exit status is unavailable".into(),
};
// ast-grep-ignore: no-context-format
err.context(format!("{exit_note}{output}"))
}
async fn send(&mut self, packet_data: Bytes) -> Result<()> {
self.connection
.write_u32(
......
......@@ -275,6 +275,19 @@ pub struct StructuredError {
}
impl StructuredError {
/// Construct a [`StructuredError`] from a free-form message with no stack
/// frames or cause. Used when synthesizing an error from contexts that do
/// not have a real JavaScript stack trace, such as a Node.js subprocess
/// crash before any response was received.
pub fn from_message(name: String, message: String) -> Self {
Self {
name,
message,
stack: Vec::new(),
cause: None,
}
}
pub async fn print(
&self,
assets_for_source_mapping: Vc<AssetsForSourceMapping>,
......
......@@ -84,40 +84,35 @@ pub struct PostCssTransformOptions {
#[turbo_tasks::function]
fn postcss_configs() -> Vc<Vec<RcStr>> {
Vc::cell(
[
".postcssrc",
".postcssrc.json",
".postcssrc.yaml",
".postcssrc.yml",
".postcssrc.js",
".postcssrc.mjs",
".postcssrc.cjs",
".postcssrc.ts",
".postcssrc.mts",
".postcssrc.cts",
".config/postcssrc",
".config/postcssrc.json",
".config/postcssrc.yaml",
".config/postcssrc.yml",
".config/postcssrc.js",
".config/postcssrc.mjs",
".config/postcssrc.cjs",
".config/postcssrc.ts",
".config/postcssrc.mts",
".config/postcssrc.cts",
"postcss.config.js",
"postcss.config.mjs",
"postcss.config.cjs",
"postcss.config.ts",
"postcss.config.mts",
"postcss.config.cts",
"postcss.config.json",
]
.into_iter()
.map(RcStr::from)
.collect(),
)
Vc::cell(vec![
rcstr!(".postcssrc"),
rcstr!(".postcssrc.json"),
rcstr!(".postcssrc.yaml"),
rcstr!(".postcssrc.yml"),
rcstr!(".postcssrc.js"),
rcstr!(".postcssrc.mjs"),
rcstr!(".postcssrc.cjs"),
rcstr!(".postcssrc.ts"),
rcstr!(".postcssrc.mts"),
rcstr!(".postcssrc.cts"),
rcstr!(".config/postcssrc"),
rcstr!(".config/postcssrc.json"),
rcstr!(".config/postcssrc.yaml"),
rcstr!(".config/postcssrc.yml"),
rcstr!(".config/postcssrc.js"),
rcstr!(".config/postcssrc.mjs"),
rcstr!(".config/postcssrc.cjs"),
rcstr!(".config/postcssrc.ts"),
rcstr!(".config/postcssrc.mts"),
rcstr!(".config/postcssrc.cts"),
rcstr!("postcss.config.js"),
rcstr!("postcss.config.mjs"),
rcstr!("postcss.config.cjs"),
rcstr!("postcss.config.ts"),
rcstr!("postcss.config.mts"),
rcstr!("postcss.config.cts"),
rcstr!("postcss.config.json"),
])
}
#[turbo_tasks::value]
......@@ -588,6 +583,7 @@ impl PostCssTransformedAsset {
ResolvedVc::cell(source_map.into()),
],
additional_invalidation: config_changed,
loader_names: vec![turbo_rcstr::rcstr!("postcss")],
})
.await?;
......
......@@ -300,6 +300,7 @@ impl WebpackLoadersProcessedAsset {
project_path
);
};
let loader_names: Vec<RcStr> = loaders.iter().map(|l| l.loader.clone()).collect();
let config_value = evaluate_webpack_loader(WebpackLoaderContext {
entries,
cwd: project_path.clone(),
......@@ -320,6 +321,7 @@ impl WebpackLoadersProcessedAsset {
ResolvedVc::cell(transform.source_maps.into()),
],
additional_invalidation: Completion::immutable().to_resolved().await?,
loader_names,
})
.await?;
......@@ -507,6 +509,34 @@ pub struct WebpackLoaderContext {
pub asset_context: ResolvedVc<Box<dyn AssetContext>>,
pub args: Vec<ResolvedVc<JsonValue>>,
pub additional_invalidation: ResolvedVc<Completion>,
/// Names of the loaders being applied to the source, in pipeline order.
/// Used to enrich error messages and issue details so users know which
/// loader chain was running when an error occurred.
pub loader_names: Vec<RcStr>,
}
impl WebpackLoaderContext {
/// Format the loader chain as "loaders [a, b, c]" for inclusion in
/// error messages and issue detail text. Returns `None` if there are
/// no loaders, which should not normally happen but keeps the helper
/// well-defined.
fn loader_chain_description(&self) -> Option<RcStr> {
if self.loader_names.is_empty() {
None
} else {
Some(
format!(
"loaders [{}]",
self.loader_names
.iter()
.map(|n| n.as_str())
.collect::<Vec<_>>()
.join(", ")
)
.into(),
)
}
}
}
impl EvaluateContext for WebpackLoaderContext {
......@@ -544,6 +574,10 @@ impl EvaluateContext for WebpackLoaderContext {
true
}
fn crash_context_prefix(&self) -> Option<RcStr> {
self.loader_chain_description()
}
async fn emit_error(&self, error: StructuredError, pool: &EvaluatePool) -> Result<()> {
EvaluationIssue {
error,
......@@ -551,6 +585,7 @@ impl EvaluateContext for WebpackLoaderContext {
assets_for_source_mapping: pool.assets_for_source_mapping,
assets_root: pool.assets_root.clone(),
root_path: self.chunking_context.root_path().owned().await?,
detail: self.loader_chain_description(),
}
.resolved_cell()
.emit();
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment