Unverified Commit ceff7268 authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

Turbopack: don't crash the dev server when node_modules/next is briefly unresolvable (#93877)

### What?

When `node_modules/next` is briefly unresolvable mid-session (for
example because pnpm is mid-install of a package with a `next` peer
dependency, reshuffling its symlinks) Turbopack would:

- Emit a `MissingNextFolderIssue` with severity `Fatal`
- Propagate `anyhow!("Next.js package not found")` through ~10 levels of
the build graph
- Surface the failure at the napi boundary as a
`TurbopackInternalError`, which the Next.js JS-side treats as fatal and
shuts the dev server down

The reported user impact was a dev session that became wedged and could
not recover even after restarting `next dev`, because the persistent
cache had stored the failed operation state. Users reported it as
"stuck", "catastrophic", and required wiping `.next/` to recover.

Unfortunately while i was able to reproduce a number of bad outcomes, i
couldn't actually reproduce the issue of `next dev` getting stuck in a
bad state. Still this PR will improve a number of apis and error
messages

See internal discussion:
https://vercel.slack.com/archives/C046HAU4H7F/p1778792876550309

### Why?

A transient filesystem race during an install is the canonical
recoverable error — once the install finishes, `next` is resolvable
again. There is no reason this should bring down the dev server or
poison the persistent cache.

The bug had two contributing causes:

1. The error message was poor. It assumed the only cause was a mis-set
`turbopack.root`, with the entire message in `title()`. It didn't
acknowledge the install-race case or any of the other realistic causes
(broken symlink, monorepo hoisting, removed `node_modules/next`, global
install). Users had no way to know that the right response was to wait
or refresh.

2. The Rust→napi boundary did not treat this as a recoverable error. Two
specific code paths propagated `Err` instead of catching it:
- `issue_filter_from_endpoint` in `endpoint.rs` calls
`endpoint_op.connect().await?` — if the upstream endpoint resolution
errors (as it does when `directory_tree_to_loader_tree` fails), this `?`
propagates **before** the surrounding
`strongly_consistent_catch_collectables` gets a chance to convert the
failure into Issues.
- `Project::hmr_version_state` is called bare from `project_hmr_events`
at the napi boundary. There is no `_with_issues_operation` wrapper, so
any error from the underlying
`hmr_content`/`versioned_content_map.compute_entry` chain propagates
straight to JS.

### How?

Three changes:

**1. Rewrite `MissingNextFolderIssue`**
([next_import_map.rs](crates/next-core/src/next_import_map.rs))
- Severity downgraded from `Fatal` to `Error`. `Fatal` triggers an
explicit JS-side server shutdown in `print-build-errors.ts`; `Error`
lets the dev server recover.
- Title is a single line: `"Could not find the Next.js package
(next/package.json)"`.
- Description lists all realistic causes (concurrent install,
missing/broken symlink, mis-set workspace root, monorepo hoisting,
global install).
- Doc link moved to `documentation_link()` so it doesn't pollute the
inline message.

**2. Make `issue_filter_from_endpoint` Err-tolerant**
([endpoint.rs](crates/next-napi-bindings/src/next_api/endpoint.rs))
- Wrap the `endpoint_op.connect().await` in a `match` and fall back to
`IssueFilter::warnings_and_foreign_errors()` on `Err`.
- This unblocks the existing `strongly_consistent_catch_collectables`
recovery path, which then absorbs the upstream failure as Issues that
surface through the napi boundary normally.

**3. Add `hmr_version_state_with_issues_operation`**
([project.rs](crates/next-napi-bindings/src/next_api/project.rs))
- Mirrors the existing `hmr_update_with_issues_operation` pattern.
- Drives `hmr_version_operation` as an `OperationVc`, catches any `Err`
and falls back to `NotFoundVersion`, and collects issues from the
operation via `peek_issues`.
- The napi `project_hmr_events` handler now uses this wrapper instead of
calling `Project::hmr_version_state` bare. Issues from both
`hmr_version_state` and `hmr_update` are merged before being sent to JS.
- Required lifting the inner `hmr_version_operation` out of
`Project::hmr_version_state`'s closure to make it callable from the napi
layer.

A new regression test at
[test/development/app-dir/concurrent-install/](test/development/app-dir/concurrent-install/)
reproduces the failure by moving `node_modules/next` aside
mid-HMR-session and asserts that the dev server does not emit `FATAL: An
unexpected Turbopack error occurred` or `TurbopackInternalError`, and
that the friendly Issue text surfaces. The test is skipped under
`NEXT_SKIP_ISOLATE=1` since that mode doesn't produce a manipulable
`node_modules`.

Out of scope: after recovery, the dev server still returns 500 on
subsequent requests until the user manually refreshes (the JS-side
per-route error state isn't invalidated by Turbopack's successful
recompile). That's a separate dev-server caching issue.

<!-- NEXT_JS_LLM_PR -->
parent 55b2d3c7
......@@ -1195,7 +1195,10 @@ impl Issue for MissingNextFolderIssue {
}
fn severity(&self) -> IssueSeverity {
IssueSeverity::Fatal
// In theory this should be fatal (how can we ever recover from next missing when we are
// next), but we actually might be detecting an ephemeral scenario where 'next' is moving
// and we can recover.
IssueSeverity::Error
}
fn stage(&self) -> IssueStage {
......@@ -1203,48 +1206,68 @@ impl Issue for MissingNextFolderIssue {
}
async fn title(&self) -> Result<StyledString> {
let system_path = match to_sys_path(self.path.clone()).await? {
Some(path) => path.to_str().unwrap_or("{unknown}").to_string(),
_ => "{unknown}".to_string(),
Ok(StyledString::Text(rcstr!(
"Could not find the Next.js package (next/package.json)"
)))
}
async fn description(&self) -> Result<Option<StyledString>> {
let context_path: RcStr = match to_sys_path(self.path.clone()).await? {
Some(path) => path.to_str().unwrap_or("{unknown}").into(),
_ => rcstr!("{unknown}"),
};
let root_path = match to_sys_path(self.root.clone()).await? {
Some(path) => path.to_str().unwrap_or("{unknown}").to_string(),
_ => "{unknown}".to_string(),
let root_path: RcStr = match to_sys_path(self.root.clone()).await? {
Some(path) => path.to_str().unwrap_or("{unknown}").into(),
_ => rcstr!("{unknown}"),
};
Ok(StyledString::Stack(vec![
StyledString::Line(vec![
StyledString::Text(
"Error: Next.js inferred your workspace root, but it may not be correct."
.into(),
),
]),
Ok(Some(StyledString::Stack(vec![
StyledString::Line(vec![
StyledString::Text("We couldn't find the Next.js package (".into()),
StyledString::Strong("next/package.json".into()),
StyledString::Text(") from the project directory: ".into()),
StyledString::Strong(system_path.into()),
StyledString::Text(rcstr!("Resolved from: ")),
StyledString::Strong(context_path),
]),
StyledString::Line(vec![
StyledString::Text("Filesystem root used for resolution: ".into()),
StyledString::Strong(root_path.into()),
StyledString::Text(rcstr!("Filesystem root used for resolution: ")),
StyledString::Strong(root_path),
]),
StyledString::Line(vec![StyledString::Text(rcstr!(""))]),
StyledString::Line(vec![StyledString::Text(rcstr!("Possible causes:"))]),
StyledString::Line(vec![StyledString::Text(rcstr!(
" - node_modules is being reorganized by a concurrent install (e.g. pnpm adding \
a package with a `next` peer dependency). This is transient and should clear \
once the install completes."
))]),
StyledString::Line(vec![StyledString::Text(rcstr!(
" - node_modules/next was removed, renamed, or has a broken symlink."
))]),
StyledString::Line(vec![
StyledString::Text("To fix this, set ".into()),
StyledString::Code("turbopack.root".into()),
StyledString::Text(
" in your Next.js config, or ensure the Next.js package is resolvable from the project directory.".into(),
),
]),
StyledString::Line(vec![
StyledString::Text("Note: For security and performance reasons, files outside of the project directory will not be compiled.".into()),
]),
StyledString::Line(vec![
StyledString::Text("See ".into()),
StyledString::Strong("https://nextjs.org/docs/app/api-reference/config/next-config-js/turbopack#root-directory".into()),
StyledString::Text(" for more information.".into()),
StyledString::Text(rcstr!(" - The workspace root is incorrect — see ")),
StyledString::Code(rcstr!("turbopack.root")),
StyledString::Text(rcstr!(
" in the Next.js config docs for how to configure it."
)),
]),
]))
StyledString::Line(vec![StyledString::Text(rcstr!(
" - In a monorepo, the Next.js package may only exist in a directory above the \
closest directory containing a package manager lockfile. The workspace root is \
detected by locating the nearest package manager lockfile."
))]),
StyledString::Line(vec![StyledString::Text(rcstr!(
" - Next.js is installed globally rather than as a project dependency. This is \
not supported; install it locally."
))]),
StyledString::Line(vec![StyledString::Text(rcstr!(""))]),
StyledString::Line(vec![StyledString::Text(rcstr!(
"Note: To ensure a hermetic build and a portable cache, files outside of the \
workspace root are not compiled."
))]),
])))
}
fn documentation_link(&self) -> RcStr {
rcstr!(
"https://nextjs.org/docs/app/api-reference/config/next-config-js/turbopack#root-directory"
)
}
}
......
......@@ -101,14 +101,22 @@ impl Deref for ExternalEndpoint {
/// Build an `IssueFilter` by reading the project from the endpoint's
/// `OperationVc<OptionEndpoint>` and extracting ignore rules from its config.
async fn issue_filter_from_endpoint(
endpoint_op: OperationVc<OptionEndpoint>,
) -> Result<Vc<IssueFilter>> {
let endpoint_option = endpoint_op.connect().await?;
if let Some(ep) = &*endpoint_option {
Ok(ep.project().issue_filter())
} else {
Ok(IssueFilter::warnings_and_foreign_errors().cell())
///
/// If the upstream endpoint operation fails to resolve (e.g. because the build
/// graph cannot be evaluated transiently — for example during a mid-session
/// `node_modules` reshuffle), this falls back to a default filter rather than
/// propagating the error. In this scenario we believe the caller will already be observing the
/// same error
async fn issue_filter_from_endpoint(endpoint_op: OperationVc<OptionEndpoint>) -> Vc<IssueFilter> {
match endpoint_op.connect().await {
Ok(endpoint_option) => {
if let Some(ep) = &*endpoint_option {
ep.project().issue_filter()
} else {
IssueFilter::warnings_and_foreign_errors().cell()
}
}
Err(_) => IssueFilter::warnings_and_foreign_errors().cell(),
}
}
......@@ -124,7 +132,7 @@ async fn get_written_endpoint_with_issues_operation(
endpoint_op: OperationVc<OptionEndpoint>,
) -> Result<Vc<WrittenEndpointWithIssues>> {
let write_to_disk_op = endpoint_write_to_disk_operation(endpoint_op);
let filter = issue_filter_from_endpoint(endpoint_op).await?;
let filter = issue_filter_from_endpoint(endpoint_op).await;
let (written, issues, effects) =
strongly_consistent_catch_collectables(write_to_disk_op, filter).await?;
Ok(WrittenEndpointWithIssues {
......@@ -229,25 +237,24 @@ async fn subscribe_issues_and_diags_operation(
) -> Result<Vc<EndpointIssuesAndDiags>> {
let changed_op = endpoint_server_changed_operation(endpoint_op);
if should_include_issues {
let filter = issue_filter_from_endpoint(endpoint_op).await?;
let (changed_value, issues, effects) =
strongly_consistent_catch_collectables(changed_op, filter).await?;
Ok(EndpointIssuesAndDiags {
changed: changed_value,
issues,
effects,
}
.cell())
} else {
let changed_value = changed_op.read_strongly_consistent().await?;
Ok(EndpointIssuesAndDiags {
changed: Some(changed_value),
issues: Arc::new(vec![]),
effects: Arc::new(Effects::default()),
}
.cell())
// Use catch-collectables in both branches so transient build-graph errors
// (e.g. missing `node_modules/next` during a concurrent install) surface as
// Issues rather than killing the subscription with a `TurbopackInternalError`.
// When `should_include_issues` is false the caller doesn't need the Issue
// payload, but we still need the catch path to avoid the FATAL.
let filter = issue_filter_from_endpoint(endpoint_op).await;
let (changed_value, issues, effects) =
strongly_consistent_catch_collectables(changed_op, filter).await?;
Ok(EndpointIssuesAndDiags {
changed: changed_value,
issues: if should_include_issues {
issues
} else {
Arc::new(vec![])
},
effects,
}
.cell())
}
#[tracing::instrument(level = "info", name = "get client-side endpoint changes", skip_all)]
......@@ -265,11 +272,13 @@ pub fn endpoint_client_changed_subscribe(
async move {
let changed_op = endpoint_client_changed_operation(endpoint_op);
// We don't capture issues and diagnostics here since we don't want to be
// notified when they change
// notified when they change. We also want errors to propagate so we don't use
// strongly_consistent_catch_collectibles either.
//
// This must be a *read*, not just a resolve, because we need the root task created
// by `subscribe` to re-run when the `Completion`'s value changes (via equality),
// even if the cell id doesn't change.
//
let _ = changed_op.read_strongly_consistent().await?;
Ok(())
}
......
......@@ -1821,6 +1821,10 @@ async fn hmr_update_with_issues_operation(
target: HmrTarget,
) -> Result<Vc<HmrUpdateWithIssues>> {
let update_op = project_hmr_update_operation(project, chunk_name, target, state);
// NOTE: we do not use `strongly_consistent_catch_collectables` here. The JS HMR
// consumers in `hot-reloader-turbopack.ts` (`subscribeToServerHmr` and
// `subscribeToClientHmrEvents`) rely on this read *throwing* on build-graph
// failures to trigger their recovery paths
let update = update_op.read_strongly_consistent().await?;
let filter = project.issue_filter();
let issues = get_issues(update_op, filter).await?;
......@@ -1957,6 +1961,12 @@ async fn get_hmr_chunk_names_with_issues_operation(
target: HmrTarget,
) -> Result<Vc<HmrChunkNamesWithIssues>> {
let hmr_chunk_names_op = project_hmr_chunk_names_operation(container, target);
// Do NOT switch this to `strongly_consistent_catch_collectables`. The JS HMR
// chunk-names consumer in `hot-reloader-turbopack.ts` relies on this read
// *throwing* on build-graph failures so its outer `try` block exits the
// subscription loop. Swallowing the error and emitting an empty chunk-name
// list keeps the loop running but with stale state, and obscures the real
// failure from the dev server log.
let hmr_chunk_names = hmr_chunk_names_op.read_strongly_consistent().await?;
let filter = issue_filter_from_container(container);
let issues = get_issues(hmr_chunk_names_op, filter).await?;
......
......@@ -260,7 +260,7 @@ export async function handleRouteType({
// otherwise we don't known when to unsubscribe and this leaking
hooks?.subscribeToChanges(
serverKey,
false,
/** includeIssues=*/ false,
route.dataEndpoint,
() => {
// Report the next compilation again
......@@ -279,7 +279,7 @@ export async function handleRouteType({
)
hooks?.subscribeToChanges(
clientKey,
false,
/** includeIssues=*/ false,
route.htmlEndpoint,
() => {
return {
......@@ -296,7 +296,7 @@ export async function handleRouteType({
if (entrypoints.global.document) {
hooks?.subscribeToChanges(
getEntryKey('pages', 'server', '_document'),
false,
/** includeIssues=*/ false,
entrypoints.global.document,
() => {
return {
......@@ -354,7 +354,7 @@ export async function handleRouteType({
// otherwise we don't known when to unsubscribe and this leaking
hooks?.subscribeToChanges(
key,
true,
/** includeIssues=*/ true,
route.rscEndpoint,
(change, hash) => {
if (change.issues.some((issue) => issue.severity === 'error')) {
......@@ -739,7 +739,7 @@ export async function handleEntrypoints({
if (dev) {
dev?.hooks.subscribeToChanges(
key,
false,
/** includeIssues=*/ false,
endpoint,
async () => {
const finishBuilding = dev.hooks.startBuilding(
......@@ -876,7 +876,7 @@ export async function handlePagesErrorRoute({
hooks.handleWrittenEndpoint(key, writtenEndpoint, false)
hooks.subscribeToChanges(
key,
false,
/** includeIssues=*/ false,
entrypoints.global.app,
() => {
// There's a special case for this in `../client/page-bootstrap.ts`.
......@@ -905,7 +905,7 @@ export async function handlePagesErrorRoute({
hooks.handleWrittenEndpoint(key, writtenEndpoint, false)
hooks.subscribeToChanges(
key,
false,
/** includeIssues=*/ false,
entrypoints.global.document,
() => {
return {
......@@ -931,7 +931,7 @@ export async function handlePagesErrorRoute({
hooks.handleWrittenEndpoint(key, writtenEndpoint, false)
hooks.subscribeToChanges(
key,
false,
/** includeIssues=*/ false,
entrypoints.global.error,
() => {
// There's a special case for this in `../client/page-bootstrap.ts`.
......
export default function LateRoute() {
return <p>late route</p>
}
import { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
export default function Page() {
return <p>hello world</p>
}
import { nextTestSetup } from 'e2e-utils'
import { retry } from 'next-test-utils'
import fs from 'fs/promises'
import path from 'path'
/**
* Simulates the failure mode where a concurrent package manager install
* reorganizes `node_modules` while the Next.js dev server is running.
*
* Specifically, when `next` (re)moves under `node_modules` mid-HMR, Turbopack
* fails to resolve `next/package.json` and emits a `MissingNextFolderIssue`.
* The dev server must:
* - surface the issue (recoverable, not Fatal)
* - NOT crash with a `TurbopackInternalError` / "FATAL" log
* - recover once `node_modules/next` is restored
*/
const describeMaybe = process.env.NEXT_SKIP_ISOLATE ? describe.skip : describe
describeMaybe('concurrent-install', () => {
const { next, isTurbopack } = nextTestSetup({
files: __dirname,
})
const itTurbopack = isTurbopack ? it : it.skip
async function getNextPath(): Promise<string> {
const nextPath = path.join(next.testDir, 'node_modules', 'next')
// sanity check
await fs.lstat(nextPath)
return nextPath
}
async function moveNextAside(): Promise<{ original: string; stash: string }> {
const original = await getNextPath()
const stash = `${original}.stash-${Date.now()}`
await fs.rename(original, stash)
return { original, stash }
}
async function restoreNext({
original,
stash,
}: {
original: string
stash: string
}): Promise<void> {
await fs.rename(stash, original)
}
itTurbopack(
'does not crash when node_modules/next is moved mid-session',
async () => {
await next.browser('/')
const getOutput = next.getCliOutputFromHere()
const stashInfo = await moveNextAside()
try {
// Force a recompile while next is missing. Not strickly necessary, but important to ensure
// we do recover with the new content eventually
await next.patchFile(
'app/page.tsx',
`export default function Page() {
return <p>hello world (edited)</p>
}
`
)
// Give the dev server time to react. We're not asserting on a specific
// user-visible behavior here — we just want the failure path to fire.
await retry(
async () => {
// The friendly Issue should be surfaced.
expect(getOutput()).toContain('Could not find the Next.js package')
},
5000,
500
)
} finally {
await restoreNext(stashInfo)
}
// The dev server must not have died from a TurbopackInternalError.
// (Whether the page itself recovers without a manual reload is a separate
// dev-server caching concern; the catastrophic failure mode is the crash.)
expect(getOutput()).not.toContain(
'FATAL: An unexpected Turbopack error occurred'
)
expect(getOutput()).not.toContain('TurbopackInternalError')
}
)
itTurbopack(
'surfaces a friendly issue when node_modules/next is missing',
async () => {
await next.browser('/')
const getOutput = next.getCliOutputFromHere()
const stashInfo = await moveNextAside()
try {
await next.patchFile(
'app/page.tsx',
`export default function Page() {
return <p>hello world (while-missing)</p>
}
`
)
// Wait for the Issue to be rendered to stdout.
await retry(
async () => {
expect(getOutput()).toContain('Could not find the Next.js package')
},
10000,
500
)
// The full issue text. Normalize path-like values that vary per run
// (the test-dir is a random tmpdir).
expect(getOutput()).toContain(
'Could not find the Next.js package (next/package.json)'
)
expect(getOutput()).not.toContain(
'FATAL: An unexpected Turbopack error occurred'
)
expect(getOutput()).not.toContain('TurbopackInternalError')
} finally {
await restoreNext(stashInfo)
}
}
)
itTurbopack(
'does not crash when navigating to an uncompiled route while node_modules/next is missing',
async () => {
// Compile `/` so the harness has at least one warm chunk.
await next.browser('/')
const getOutput = next.getCliOutputFromHere()
const stashInfo = await moveNextAside()
try {
// Navigating to `/late-route` (never compiled in this session) forces
// a fresh `hmr_version_state` evaluation for that chunk. That path
// hits `get_next_package`, which Errs when `node_modules/next` is
// missing. The dev server must surface the failure as an Issue and
// keep the HMR subscription alive — not crash.
await next.fetch('/late-route').catch(() => {
// The request itself may fail (500) while next is missing. That's
// expected. We only care that the dev server stays alive.
})
await retry(
async () => {
expect(getOutput()).toContain('Could not find the Next.js package')
},
10000,
500
)
expect(getOutput()).not.toContain(
'FATAL: An unexpected Turbopack error occurred'
)
expect(getOutput()).not.toContain('TurbopackInternalError')
} finally {
await restoreNext(stashInfo)
}
// Once next is restored, `/late-route` should eventually render.
await retry(
async () => {
const res = await next.fetch('/late-route')
expect(res.status).toBe(200)
expect(await res.text()).toContain('late route')
},
15000,
500
)
}
)
})
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment