- 28 Jul, 2026 4 commits
-
-
next-js-bot[bot] authored
-
Andrew Clark authored
Removes the "allow-runtime" prefetch config, and turns its behavior on implicitly wherever Partial Prefetching is enabled. The original motivation for "allow-runtime" was to give apps more control over server costs triggered by prefetches. Until a route explicitly opts in, prefetches would only be served from the CDN, not from the server. The problem, though, was it was very confusing to know when to add or remove this configuration. The incentive for many apps was to add it everywhere, with no clear signal for when to remove it. Our updated thinking is that Partial Prefetching itself already provides sufficient protection against runaway prefetching costs: per-link prefetches only happen on Link components that explicitly opt in with the prefetch prop. The optimizations landed earlier in this stack also make allow-runtime less necessary: on pages where all the content is statically renderable, prefetches are served from the static cache and no runtime request is ever issued; only a page that accesses non-static data is prefetched at runtime. The upshot of this decision is that runtime versus static becomes an internal optimization; the same content gets prefetched regardless of whether or how Next.js is able to optimize it.
-
Andrew Clark authored
If we're reasonably confident that a segment can be prefetched statically without omitting data that would have been included during a runtime prefetch (e.g. cookies), the client should attempt prefetch the segment statically instead of going straight to a runtime request. The decision for whether to do this is based on the ShouldAttemptStaticPrefetch added in previous steps. If it turns out the static response is not sufficient, then it will fall back to a runtime request. This makes prefetching cheaper for pages that are fully statically renderable. We can make the optimization better in the future with more reliable per-segment computation of the ShouldAttemptStaticPrefetch, but the current approach should at least work for fully static pages, which is what's most important. This optimization applies during both the Shell phase and the Speculative phase of the prefetching algorithm.
-
next-js-bot[bot] authored
-
- 27 Jul, 2026 23 commits
-
-
next-js-bot[bot] authored
-
Andrew Clark authored
This is a refactor of the prefetching algorithm to block the prefetch task from completing until the response is received. Currently, a prefetch task is fire-and-forget: once a prefetch's requests are spawned, the scheduler marks the originating task as complete regardless of what the server ends up returning. However, certain planned optimizations require us to verify the response from the server to determine whether additional work is necessary. For example, we may attempt to do a static prefetch instead of a runtime prefetch if we optimistically assume that the static prefetch does not vary on runtime data. We need to verify that assumption is correct before marking the task as complete. Now, whenever a pass over the task spawns a segment request, or encounters an entry whose response hasn't arrived yet, the task registers itself on the entry and exits as Blocked instead of Done. When the entry resolves, the blocked task is pinged and the pass re-runs against the received data. Only a pass that observes every response it cares about advances the task to the next phase or completes it. Rejected entries are handled per segment: a rejection counts as an observed response, so the pass skips that segment and keeps prefetching the rest. The task never registers on a rejected entry, since nothing ever pings one. Retries are governed by the existing per-entry logic. Also fixes a cache bug this exposed: when a revalidation response is keyed at a more generic vary path than the stale partial entry that prompted it (because the segment turned out not to vary on some param), the stale entry shadowed the result on every lookup — the upgraded data was unreachable and the revalidation was wasted. With blocking, that dead end became an infinite loop: each fulfillment re-pinged the blocked task, which re-read the same stale entry and spawned another revalidation. Upserting (or re-keying) a segment now deletes settled entries at more specific vary paths that the incoming entry supersedes. The router-act test helper's "at least one request initiated" watchdog is adjusted to account for the new blocking behavior: countable requests can now be legitimately gated on an in-flight App Shell response (which the watchdog intentionally doesn't count). On expiry it now proceeds to response processing if the act scope intercepted a shell request, and keeps waiting while shell requests are in flight elsewhere on the page, instead of timing out.
-
David Alexandru Ilie authored
Draft replacement for #96267 after restructuring the Request Insights stack. ## Summary - Request Insights bookkeeping used `Date.now()`, which Cache Components patches to detect unstable application wall-clock reads. - Framework bookkeeping could therefore be incorrectly attributed to user code. - Implicit timestamps now use `performance.timeOrigin + performance.now()`. - Explicit OpenTelemetry timestamp inputs remain unchanged. - Tests distinguish framework bookkeeping from a genuine application `Date.now()` call. ## Verification ```bash pnpm build-all pnpm --filter=next types pnpm --filter=next build pnpm test-unit --runTestsByPath packages/next/src/server/lib/trace/local-span-recorder.test.ts packages/next/src/server/lib/trace/span-store.test.ts packages/next/src/server/lib/trace/request-insights.test.ts pnpm test-dev-turbo test/development/app-dir/request-insights/request-insights.test.ts pnpm test-dev-webpack test/development/app-dir/request-insights/request-insights.test.ts ``` <!-- NEXT_JS_LLM -->
-
dan authored
Applies the key accounting from #96229 to the other in-memory LRUs with byte budgets: - the incremental memory cache - the default `use cache` handler - the dev `staticPathsCache` and `serverComponentsHmrCache` - the dev source-map URL cache. Keys were free against each budget in the same way; unlike the filesystem route cache there are no production leak reports for these, so this is preventative and kept separate. Count-based caches and the disk-byte LRU are unchanged. ## Test Plan Each size callback change adds `cacheKey.length` to whatever it returned before. The memory cache callback is also restructured from early returns into if/else chains; each `CachedRouteKind` branch returns the same value as before, plus the key length. To see the change do something, run this against the built memory cache: ```js const { getMemoryCache } = require('next/dist/server/lib/incremental-cache/memory-cache.external') const cache = getMemoryCache(1024 * 1024) // 1 MB budget for (let i = 0; i < 20000; i++) { cache.set('/api/data?' + 'p'.repeat(1000) + i, { value: { kind: 'FETCH', data: { body: 'ok' } }, lastModified: 1, }) } console.log(cache.size, cache.currentSize) ``` Before this change, all 20,000 entries are admitted: the cache counts 260 KB against its 1 MB budget while the keys alone hold ~20 MB, and it keeps growing as long as new keys arrive. After, it holds 1,020 entries and the counted size stops at the budget. For regressions: `test/production/app-dir/revalidate` and `test/e2e/app-dir/use-cache-default-handler-expire-zero` pass in prod mode (these exercise the fetch cache and the default `use cache` handler), plus the `packages/next/src/server/lib` jest suites and `pnpm --filter=next types`. Co-authored-by:Marcos Hernanz <96699542+marcoshernanz@users.noreply.github.com>
-
dan authored
`getItem` stores `null` for paths that don't resolve, but checks the cached value with `if (lruResult)`, so a cached `null` is never served: every request for an unknown path — including every dynamic-route request — re-runs the lookup. It's been this way since the cache was introduced in #52492. Return the cached entry whenever the key is present. This can't go stale: in production the item sets are only populated during `setupFsCheck`, and the LRU is disabled in dev. Cached misses are stored as a dedicated sentinel instead of `null`. Since serving a cached miss has no observable effect, no test can catch this regressing — that is also why it went unnoticed. With the sentinel it's a type error instead: reintroducing the truthy check fails `pnpm types` (verified). Same pattern as `invalidSourceMap` in `source-maps.ts`. Stacked on #96229, which bounds what negative entries can retain. ## Test Plan No direct test: the change is unobservable by design, and the existing e2e suites assert that it stays that way. The regression guard is the type system, as above. `test/e2e/app-dir/not-found/basic` passes in prod mode (14/14), exercising the negative-cache path.
-
dan authored
Fixes #94890 Adopted from #96187 and #94699, with both authors credited on the commit. The router server keeps an in-memory cache that maps request URL paths to what they resolve to: a page, a static file, or nothing. It's bounded by a size budget, but the size of each entry was computed from the cached value only. The URL string used as the key was free, and "resolves to nothing" entries counted as size 1. So a long-lived server receiving many unique URLs — bot crawls, long slugs, `/order/:id`-style traffic — could accumulate up to ~1M URL strings while the cache believed it was within budget. That looks like a slow memory leak that tracks how many distinct URLs you've served rather than how much traffic you get. #94890 has several production reports, including two deployments that patched key accounting in and saw memory go flat. The idea of the fix is to make the cache's bookkeeping match what an entry actually keeps alive, so the budget becomes a real memory bound. That takes three things: 1. Count the key string, not just the value. 2. Count the fixed cost an entry has regardless of string lengths — the per-entry bookkeeping objects, measured at ~120 bytes. Without this, a flood of *short* unique URLs stays "cheap" on paper while the bookkeeping itself grows unbounded. 3. Copy strings before caching them. When V8 takes a substring of a long string (which is how the pathname is extracted from the request URL), it doesn't copy the characters — it creates a small object that points into the original string, and the original can't be garbage-collected while the substring is alive. So caching a 30-char pathname could actually hold the entire URL in memory, kilobyte query string and all, and the bookkeeping only sees the 30 chars. Copying the string before caching makes it hold only its own characters. With the bookkeeping honest, the budget is set to 8 MiB, roughly the real memory the cache could already use for legitimate entries before this change, so capacity isn't reduced — the previous attempt at this fix (#94942) stalled on that concern. ## Test Plan A/B on a production `next start` serving 60k requests with unique long URLs (the reported traffic shape): canary grows +79 MB and keeps growing; this branch grows +9 MB and plateaus — the budget working as intended. The per-entry costs above were measured directly on the cache with forced GC (they fit `118 + key length` bytes within a few percent, which is also what predicts the A/B numbers). `lru-cache` unit tests (26/26) and `pnpm --filter=next types` pass. --------- Co-authored-by:
Marcos Hernanz <96699542+marcoshernanz@users.noreply.github.com> Co-authored-by:
dmchoi77 <dm.choi.dev@gmail.com>
-
Benjamin Woodruff authored
If we get metadata events, or if we (on Linux with inotify) get file access events, these aren't useful, and we filter them, but we were unintentionally extending the deadline for the batch because of them.
-
Benjamin Woodruff authored
Previously we we had 5 total `FxHashSet`s in `BatchedInvalidations`. Now we've got one `FxHashMap` and one `FxHashSet`. There was potential overlap in paths represented in all of these sets. The old implementation was suboptimal for two reasons: - We ended up cloning and storing `PathBuf` more times. (Note: there's a good future opportunity for an arena here too) - If a path was in both `path` and `path_and_children`, we'd later try to invalidate paths associated with that twice. This will be a no-op as we have a lock on the invalidator map and the duplicated path has already removed from the invalidator map, but it makes us do extra map lookups that aren't needed. This still keeps around a separate set for `new_paths` so that we can more efficiently iterate over them. It's possible it could benefit from being a flag in the same set, but it's less obvious.
-
Joseph authored
Closes: https://github.com/vercel/next.js/issues/96204 The fix to manage rewrites introduced in https://github.com/vercel/next.js/pull/82236 PR made it so that, push/replace without explicit pathname broke the `as` pattern, for example in `with-vercel-blob`. Also documents the resolution rule in the `useRouter` API reference.
-
Jimmy Miller authored
I get this is a bit of a silly change in some ways. It isn't going to have some massive impact. But using dhat I got these stats on a smallish sample nextjs project | metric | A (parse.rs change) | B (baseline) | delta | | ------------------------ | -------------------- | ------------ | ------------------- | | total bytes allocated | 438.4 MiB | 442.6 MiB | -4.3 MiB (-0.94%) | | total allocation count | 1,672,688 | 1,872,347 | -199,659 (-10.66%) Eliminating ~200k allocations from this small change just felt worth it.
-
Vercel Release Bot authored
Co-authored-by:next-js-bot[bot] <279046576+next-js-bot[bot]@users.noreply.github.com>
-
dan authored
- `detectContentType` test was too aggressive about asserting completion in 1ms, let's give it several tries. - We weren't correctly setting up pnpm overrides in tests, which breaks things whenever canary comes out.
-
dan authored
This test suite used to be disabled in CC runs, which feels like a bit of a hazard for developing CC. Let's enable it.
-
dan authored
Reverts the revert of #95878. This failed on `main` due to React bugs (https://github.com/react/react/pull/37061?) which are fixed now.
-
dan authored
Test-only. The test assertion was racy: it asserted no with-query-param RSC request during hash-only navigations, but a late prefetch of the `#link-to-query-param` link makes exactly that request (failed 3/3 on a prod CI shard). Assert the actual invariants instead: - hash-only navigations issue no navigation RSC requests at all (prefetches, identified by their headers, are ignored) - a query change *does* issue one — with `prefetch={false}` on the link so it can't be served from the prefetch cache. -
Aurora Scharff authored
### What? Restructure the AI agents guide into a step-by-step progression and close the coverage gaps against the 16.2 and 16.3 AI blog posts. ### Why? The guide read as a feature list and was missing agent-facing features the blog posts announced: the actionable error menus with the **Copy prompt** button and per-error `/docs/messages` pages, the `.md` / `llms.txt` docs access path, and the `.next/dev/lock` behavior that keeps agents from stacking dev servers. ### How? Four steps now sequence the setup (bundled docs, runtime visibility, errors, skills), with the new content folded in where it belongs. The skills intro states the knowledge-vs-workflow split (bundled docs are the source of truth, skills are for workflows). Cross-links the instant navigation AI workflow and the building guide's `--debug-prerender` section, and the terminal example uses the current message wording rather than the blog's pre-stable snapshot. The `#existing-projects` anchor and per-skill anchors are unchanged for published inbound links; instant-navigation's link to runtime visibility is updated to the new heading.
-
Sebastian "Sebbie" Silbermann authored
[`httpxy`](https://github.com/unjs/httpxy) is a faster and actively maintained variant of `http-proxy`. It also doesn't need any manual patches to avoid security vulnerabilities in Next.js. `httpxy` is used by Nuxt and maintained by the Nitro team.
-
Sebastian "Sebbie" Silbermann authored
The previous `@mswjs/interceptors` has a bug where the response it forwarded never set the [`complete` field](https://nodejs.org/api/http.html#messagecomplete). This wasn't an issue so far (or not reported) but is an issue for `httpxy` which relies on an accurate `complete` field. Bumping `@mswjs/interceptor` now so that we can switch from `http-proxy` to `httpxy`.
-
Andrew Clark authored
Adds a server-computed signal to static per-segment prefetch responses that tells the client whether a runtime prefetch request would return more content than the static response already contains. The signal is computed by tracking whether the prerender accessed any data source that hangs during a static prerender but would resolve during a runtime prerender. For example: cookies, headers, fallback params, and search params. The information is encoded two places: - As a prefetch hint called ShouldAttemptStaticPrefetch. This tells the client that it's worth attempting to do a static prefetch instead of a runtime one. It's only an optimization, though: if the static response ends up being insufficient, the client will follow it up with a runtime request. It's semantically OK if there are false positives. - Embedded in the static segment response. This tells the client whether the static response is missing data that would have been included in a runtime response. Unlike the prefetch hint, this value must never falsely claim that no runtime request is needed. A follow-up will update the client to skip the runtime request using this information. This commit only encodes the signal into the response.
-
Tim Neutkens authored
## Summary - record deferred Instant Insights work as a separate typed Request Insights item - keep validation spans, fetches, timing, and live updates separate from the foreground request - label Instant Insights consistently in DevTools and CLI output - stack this observability change on #95939, which moves validation off the navigation response path ## Verification - `pnpm jest --runInBand packages/next/src/server/lib/trace/request-insights.test.ts packages/next/src/server/lib/trace/local-span-recorder.test.ts packages/next/src/next-devtools/dev-overlay/components/request-insights/trace-viewer.test.ts packages/next/src/next-devtools/dev-overlay/shared.test.ts` - `NEXT_SKIP_ISOLATE=1 NEXT_TEST_PREFER_OFFLINE=1 pnpm test-dev-turbo test/development/app-dir/request-insights/request-insights.test.ts` - `NEXT_SKIP_ISOLATE=1 NEXT_TEST_PREFER_OFFLINE=1 pnpm test-dev-webpack test/development/app-dir/request-insights/request-insights.test.ts` - `pnpm --filter=next types` <!-- NEXT_JS_LLM --> --------- Co-authored-by:David Ilie <david@davidilie.com>
-
Sam Poder authored
This PR builds on https://github.com/vercel/next.js/pull/95716 by adding support tree-shaking exports exported using the `module.exports = {...}` syntax
-
Sam Poder authored
Used Claude + this tool (https://github.com/vercel/next.js/pull/96158) to look at Turbopack's runtime. With a large fixture with a lot of imports, this part was bailing out: ``` { "kind": "deopt", "bailoutType": "deopt-eager", "reason": "wrong map", "functionName": "getChunkRelativeUrl", "original": { "source": "turbopack:///[turbopack]/browser/runtime/base/runtime-base.ts", "line": 486, "column": 11 }, "severity": "high" } ``` After the fix, this went away. --------- Co-authored-by:
Luke Sandberg <lukesandberg@users.noreply.github.com>
-
Andrew Clark authored
Updates the per-segment, static prefetch response type to include a bytes offset for the App Shell. This will allow the client to extract a shell response from a more complete response, like we already do for page-level prerender responses. All of the significant changes are confined to the server, during the step when a full page prerender is used to create per-segment responses (collect-segment-data). The overall strategy is the same one used at the page level. The input data is unblocked in distinct stages, separated by macrotasks. At each stage, we record the number of bytes that have been emitted to the output stream. Then, before closing the final stream, we embed the byte offsets into the response. During this phase, no actual user code runs: all we're doing is decoding and reencoding an existing Flight response. So the timing logic is more straightforward. For example, we don't need to worry about dynamic data leaking into the result. This commit is purely an in-place refactor. Subsequent steps will update the client to take advantage of the new offsets.
-
- 26 Jul, 2026 1 commit
-
-
Tim Neutkens authored
## Summary Add Vite-compatible `caseSensitive` support to `import.meta.glob()`. Matching remains case-sensitive by default, while `caseSensitive: false` applies case-insensitive matching to directory traversal, filenames, positive patterns, and negative patterns. The option is included in matcher equality and non-default virtual module identity, and is exposed through diagnostics, TypeScript declarations, and the Turbopack options reference. ## Verification - `cargo test -p turbo-tasks-fs glob` - `cargo nextest run -p turbopack-tests -E "test(import_meta_glob) and not test(import_meta_glob_errors)"` - `cargo fmt --all -- --check` - `pnpm build-all` and `pnpm --filter=next types` are blocked by current-canary `sharp.default` type errors in `packages/next/src/server/image-optimizer.ts` <!-- NEXT_JS_LLM -->
-
- 25 Jul, 2026 12 commits
-
-
next-js-bot[bot] authored
-
Pete Hunt authored
### What? Fire-and-forget `setTimeout` calls in the Edge runtime sandbox (e.g. from middleware) are tracked forever by Next.js's sandbox `TimeoutsManager`, even after the timeout has run. In a long-lived standalone/self-hosted server this makes tracked timeout resources grow monotonically, producing step-like heap growth and eventual OOM. ### Why? `TimeoutsManager.add()` pushes each timeout id into a `resources` array. When a one-shot timeout fires naturally, `webSetTimeoutPolyfill`'s `finally` clears the underlying Node timer (the workaround for nodejs/node#53335) but never removes the id from the manager. The id is dropped only if user code calls `clearTimeout(id)` or the whole module context is torn down (`clearModuleContext`, introduced in #57235). The natural-completion path — module context stays alive, timeout finishes on its own — was missing, so ids accumulate for the lifetime of the process. ### How? - `TimeoutsManager.create()` now wraps the callback so the timeout untracks its own id once it has run. The Node-timer workaround and the `this`/args binding in `webSetTimeoutPolyfill` are preserved. - Refactored the base `ResourceManager.remove()` into `untrack()` (stop tracking) + `destroy()`, so natural completion can release tracking without a redundant `clearTimeout`. - Intervals are intentionally unchanged: they fire repeatedly and must remain tracked until `clearInterval`/`removeAll`. - Added a unit test covering natural release, no monotonic growth across repeated rounds, the `clearTimeout`/`remove` path, callback `this`/args binding, and that intervals stay tracked. Verified: `pnpm --filter=next types` clean, prettier/eslint clean, and the new unit test fails on the current behavior (reverting only the release logic) and passes with the fix. Fixes #95094 --- **Update:** this branch now lands @berry95's commit from #94794 first (`Set`-based resource tracking, which also makes `untrack`/`remove` O(1)), with the fix rebased on top. --------- Co-authored-by:
Berry Kim <wlsdud40@gmail.com> Co-authored-by:
Dan Abramov <dan.abramov@gmail.com>
-
dan authored
In a project whose absolute path contains characters that percent-encode in URLs — e.g. a space — the dev overlay showed no code frame and raw `file://` URLs instead of source locations for server frames. Two encoding mismatches caused this: - Turbopack's `trace_source` percent-decoded the source map's original file URL before comparing it against the still-encoded project root URI, so the containment check failed ("Original file ... outside project") for any project path with such characters. The comparisons now stay in the encoded domain and only the outputs are decoded back into paths. - React synthesizes stack frame `file:` URLs by prepending the scheme to a filesystem path, so the overlay receives URLs with e.g. raw spaces. These aren't well-formed, and they don't match the `url.pathToFileURL` form Node.js keys its source map cache by. The overlay middleware now re-encodes them through WHATWG URL parsing, which tolerates such input. ## Test Plan New tests. Known cases that remain broken are commented out with a TODO. These could be resolved if https://github.com/react/react/pull/37105 or equivalent lands. ## Before <img width="825" height="732" alt="Screenshot 2026-07-25 at 14 39 18" src="https://github.com/user-attachments/assets/0f119e52-ec86-495a-bf5c-5339faa1d606" /> ## After <img width="1126" height="733" alt="Screenshot 2026-07-25 at 14 58 14" src="https://github.com/user-attachments/assets/a263ee8a-4cfe-4358-9a8e-2e58b4b16aa5" /> Co-authored-by:
Claude Fable 5 <noreply@anthropic.com>
-
dan authored
Vibed. This should help with running React PRs on sandboxes to measure impact on `bench/`. ## Usage ``` use bench skill to to measure impact of https://github.com/react/react/pull/37066 ``` - Should be able to point at React or Next PRs - Compares against mergebase - Grabs React builds from CI when possible - Before benching, asserts both React builds are green, or that tests pass - Runs on many sandboxes with interleaved runs - Caches reusable stuff (builds) between sandboxes - Uses `bench/` examples with realistic payloads ## Test Plan Running a bench: <img width="1445" height="584" alt="Screenshot 2026-07-20 at 03 08 02" src="https://github.com/user-attachments/assets/ac8b6aac-1e62-4695-9a59-fd0577a8a441" /> Bench that shows good results: <img width="1078" height="725" alt="Screenshot 2026-07-20 at 03 46 20" src="https://github.com/user-attachments/assets/ce49a806-f1b1-497c-a3dd-e5f8d08ea25b" /> Neutral bench: <img width="1032" height="455" alt="Screenshot 2026-07-20 at 03 47 27" src="https://github.com/user-attachments/assets/de3b700c-4875-4c70-b1c1-55cdddd73a1a" /> --------- Co-authored-by:
Claude Fable 5 <noreply@anthropic.com>
-
Hendrik Liebau authored
Node.js caches source maps per isolate, so the dev validation worker only has maps for the chunk files loaded on its own thread. For chunks it never loads, such as a module behind a dynamic `import()`, it reads the `.map` that Turbopack writes next to the chunk. Webpack keeps its dev source maps in the compiler instead, so the worker has nothing to read and reports those frames with the position they have in the compiled output rather than in the source. This stops using the worker when the bundler is Webpack, so validation runs in process again, where those frames resolve, and `experimental.devValidationWorker` now has no effect there. Dev performance under Webpack goes back to what it was before validation moved to a worker, which is the better trade: the worker exists to keep the event loop responsive during rapid navigation, and that is not worth losing the source location on validation errors.
-
Hendrik Liebau authored
Node.js caches source maps per isolate, so the dev validation worker only has maps for the chunk files loaded on its own thread. It never renders server components, and the built entry reaches segment modules through lazy getters, so it loads whatever `loadComponents` pulls in and nothing else. A module the bundler splits into a chunk of its own, as Turbopack does for a dynamic `import()`, is therefore missing from that cache entirely, and the frames pointing into it are logged without a source location. In-process validation resolves them, so this is a regression from moving validation onto a worker. On the main thread the same miss is covered by asking Turbopack for the map through the `Project` handle, which cannot cross a thread boundary. The worker instead reads the `.map` that Turbopack already wrote next to the chunk, which needs no project handle and, unlike Node's cache, does not depend on the chunk having been evaluated on this thread. Lookups are restricted to `distDir` so a stack frame cannot point the reader at an arbitrary file, and both hits and misses are memoised. Installing that lookup also required moving `bundlerFindSourceMapPayload` onto a `globalThis` symbol. `patch-error-inspect` is bundled into several runtimes that each get their own copy, and the copy that registers the implementation, here the worker bundle, is not the copy that symbolicates the frame, which is the app-page bundle's. The code frame renderer in the same file is already shared this way for the same reason. Webpack keeps its dev source maps in the compiler rather than writing them next to the chunks, so there is nothing for the worker to read and its frames are unchanged. A follow-up turns the worker off for Webpack so that validation runs in process, where those frames still resolve.
-
Hendrik Liebau authored
Running `test/e2e/app-dir/instant-validation/suspense-boundaries.test.ts` locally on Node.js 24 currently fails in the two "missing suspense around search params" cases: stacks that should read `app/…/page.tsx:40:18` are logged as raw `about://React/Prefetch/file:///…/%5Broot-of-the-server%5D…` URLs, so the validation errors printed to the terminal no longer point at any source. CI is green on the same commit, because it runs Node.js 20. The dev validation worker (#96153) and the switch to `file:` source maps for Turbopack (#95946) landed hours apart, and the worker's branch predated the source map change, so the two first met on canary, where the older Node.js hid the result. Symbolicating a frame that React spliced in from another environment means decoding the chunk path out of its `about://React/…` URL and asking Node.js for that chunk's source map. Decoding returns the path as written, while Node.js keys its cache by the `pathToFileURL` encoding of the same path, and from Node.js 22 on that encoding escapes the brackets in Turbopack's `[root-of-the-server]` chunk names, so the lookup misses. Node.js 20 leaves brackets alone, which is why the same code resolves there. PR #95946 anticipated the mismatch and left it as a follow-up, because on the main thread a miss is only wasteful: Turbopack hands back the chunk's map instead, which means parsing one the process had already parsed and cached. The validation worker runs on its own thread with no access to the Turbopack project, so there the miss is final. Node.js also accepts the plain path as a cache key, unambiguously for both a CommonJS absolute path and an ESM `file:` URL, so we retry with it before giving up. Making React's fake frame URLs reversible, as proposed in https://github.com/react/react/pull/37105, would let the first lookup succeed on its own and retire the retry. The test added here pins down what the worker can symbolicate at all. It caches maps per chunk file in its own isolate and never renders server components, so a statically imported module sits in a chunk it has already loaded and resolves, while a dynamically imported one gets a chunk of its own that is never loaded and does not. The snapshots for that second route record broken output on purpose: in-process validation resolves the frame, so losing it is a regression from moving validation onto a worker, and it affects Turbopack (no location) and Webpack (compiled positions) alike. Follow-ups address each. The snapshots are recorded with the retry applied rather than before it. Without it Turbopack's output differs between Node.js 20 and 22, and the broken URLs embed the test's temporary install directory, which changes on every run, so no stable recording exists. One consequence is that CI cannot fail on a regression of the retry itself; that was checked by running the new suite under Node.js 24.
-
Hendrik Liebau authored
The dev validation snapshot flattened both fallback param sets into arrays of entries before handing them to the worker, and the worker rebuilt a `Map` from each one on the other side. Neither step is needed, because the structured clone that carries the snapshot across the `worker_threads` boundary supports `Map` natively, so the maps can travel as they are. Dropping the conversion on both ends also lets the two snapshot fields be typed as `OpaqueFallbackRouteParams` directly, and it makes them consistent with `rootParams` and `implicitTags`, which the snapshot already passes through untouched. The neighbouring `headers` spread stays as it is, because `Headers` is not a structured-cloneable type and those entries genuinely do have to be materialized before the snapshot is sent.
-
dan authored
Follow-up to https://github.com/vercel/next.js/pull/95946. Every fake stack frame is eval'd as its own script, so error formatting resolved a source map once per frame. This sucks for two reasons: `SourceMap#payload` returns a deep clone of the payload *on every access*, and constructing a `SyncSourceMapConsumer` indexes the whole payload. So an error with 50 fake frames pointing into one chunk cloned that chunk's map 50 times and indexed it 50 times. This change caches both steps per map instead of per script. `findSourceMapPayload` wraps `module.findSourceMap` and reads payload only once per `SourceMap` instance. This works because Node.js keeps one instance per script, so the instance is a reliable identity for "same map". (This also helps `filterStackFrameDEV`, which cloned the payload for every frame it filtered.) Consumers are cached in a WeakMap with (payload, base URL) pairs as a key since that's what they vary by. (Note that consumer sharing only works because the payload memo makes references stable, that's why they're in one commit.) This lets us share computation for identical sourcemaps despite them being used by different virtual frames. I previously tried to do this by keying a map from a physical filename to the sourcemap, but as noted in https://github.com/vercel/next.js/pull/95946#discussion_r3613450061, each fake frame could technically receive a different map, even if we don't do that. Caching based on returned value's identity lets us preserve the ability to be more granular later without worrying that the cache will start behaving incorrectly. ## Test Plan Microbench (20 errors × 20 fake frames, 5.6MB map, native path): canary 688.2 ms/error → this commit 0.2 ms/error. On a real devserver route with an artificially deep error (50-deep stacks in a chunk with a 19MB map), the route serve time goes down from ~400ms to ~70ms. Co-authored-by:
Claude Fable 5 <noreply@anthropic.com>
-
Pete Hunt authored
## Summary Fixes #94919. When a client disconnects mid-response, the compression middleware's zlib stream is never released, and every aborted compressed response permanently retains its deflate state (~256 KiB). On a server taking traffic that aborts mid-stream (bots, CDNs, users navigating away) RSS climbs until the process is OOM-killed. `router-server.ts` applies the vendored `compression` middleware, which ends its zlib stream *only* from inside its own `res.end()` wrapper. On a client disconnect Next destroys the response instead, so that wrapper never runs and the stream stays open. An open zlib stream is pinned by its native handle, so it is not merely uncollected-for-now — it survives GC entirely. This is why the reported symptom looks unlike a normal JS leak: the JS heap stays nearly flat while RSS grows without bound, because the retained deflate state is native memory. It affects `next start`, `next dev`, and custom servers — anything going through the router server with the default `compress: true`. Deployments that let a proxy handle compression never see it. Note that ending the stream is not sufficient to recover it: its `data` handler writes to the already-destroyed response, `res.write()` returns `false`, the middleware pauses the stream, and it never reaches `end`. It has to be destroyed. `releaseCompressionStream` carries a comment explaining this so the cleanup isn't "simplified" back into a no-op later. The stream is reached through the one handle the middleware exposes: it forwards `res.on('drain', …)` to the zlib stream once that stream exists, and `EventEmitter#on` returns the emitter it was called on. Upstream `compression@1.8.1` still has no premature-close handling, so this can't be fixed by bumping the vendored copy. The added test asserts that assumption and will fail if a future vendored version handles this itself, signalling that the workaround can be dropped. ### Before / after Measured with a minimal custom Node server (no Express/axios) rendering a 1.25 MB dynamic App Router page, 8 rounds × 300 requests cancelled after the first chunk, `heapUsed`/`rss` sampled **after forced GC** each round. Same build, cleanup toggled behind a temporary env var: | | without cleanup | with cleanup | | --- | --- | --- | | zlib contexts never released | 2400 of 2400 (100%) | **0** of 2600 | | RSS | 98.2 → 1069.6 MiB, linear | 100 → 401.5 MiB, plateaus | | `external` | 3.6 → 76.5 MiB | 3.6 → 4.5 MiB | | `heapUsed` | 21.2 → 79.1 MiB | 21.2 → 27.4 MiB | Without the cleanup, RSS grows by a steady ~93 MiB per 300 aborted requests across rounds 2–8 (≈310 KiB per request, consistent with zlib's default deflate state plus per-stream buffers), and every unreleased stream is *still reachable after repeated forced GC* — permanent retention rather than GC lag. With the cleanup, all 2600 contexts are closed and RSS plateaus, with the residual difference from baseline being allocator retention rather than per-request growth. As a control, 200 fully-read responses issued into the same un-fixed process all closed normally, isolating mid-stream disconnects as the trigger. The issue also attributes the leak to a retained RSC element tree held via the `WeakMap` in `use-flight-response.tsx`. That part did not reproduce on canary: across 8000 aborted requests spanning dynamic renders, `redirect()`, `notFound()` and render throws, and 3000 full reads of the same 1.25 MB payload, post-GC heap stayed flat at ~29 MiB. The native `zlib_memory` line in the reporter's heap-snapshot diff was the actual cause. ## Verification - `pnpm test-start-turbo test/e2e/custom-server/custom-server.test.ts` and `pnpm test-dev-turbo test/e2e/custom-server/custom-server.test.ts` — covers `Content-Encoding: gzip` through a custom server, i.e. the code path being changed (46 and 45 passing) - `npx jest packages/next/src/server/lib` — 188 passing, including the 4 new cases - Manual: verified a gzipped response still decompresses byte-identical to the same response with `accept-encoding: identity` (1,250,721 bytes), with no errors or `MaxListenersExceededWarning` under sustained load - Manual: the before/after table above, re-measured against this branch's HEAD <!-- NEXT_JS_LLM --> Co-authored-by:Pete Hunt <239742+petehunt@users.noreply.github.com>
-
Hendrik Liebau authored
This moves the dev-mode Cache Components validation renders off the dev server's main thread onto a worker thread, so rapid navigation no longer starves the event loop. The worker crosses into the app-page bundle exactly once, calling a new `ComponentMod.routeModule.runValidationInDev` entry that rebuilds the render context, work store, and request store from a serializable snapshot and runs the whole validation there, so the client prerender and the user's client components resolve the single app-page React instance rather than a second copy. A thin worker shell (`dev-validation-worker.ts`) plus a single-worker pool (`dev-validation-worker-pool.ts`) load the user bundle via `loadComponents`, install code-frame support for CLI output, and forward the validation errors back as Flight bytes; the main thread only delivers them to the overlay through `sendErrorsToBrowser`. The snapshot, the globalThis-symbol handoff, and the shared error-delivery helpers live in their own modules. The dev server installs the worker when `experimental.devValidationWorker` is not `false`, and `runDevValidationInBackground` uses it when present, falling back to the in-process path otherwise. A one-slot `SharedArrayBuffer` propagates a supersede abort into the worker so a newer navigation cancels an in-flight validation. The runtime bundle gains an `app-worker` entry for the worker with the `build/swc` boundary externalized for the code-frame native binding, and `patch-error-inspect.ts` now backs its code-frame renderer with a globalThis symbol so all copies of the module share it across the thread. The synthetic `bench/dev-validation` benchmark navigates back-to-back, so every click lands inside the validation window — the worst case for main-thread contention. Browser-observed navigation TTFB in that window, worker vs in-process: | Route | Worker (p50/p95/max) | In-process (p50/p95/max) | | ------ | -------------------- | ------------------------ | | client | 19 / 24 / 27 ms | 40 / 66 / 7762 ms | | server | 42 / 45 / 46 ms | 122 / 158 / 252 ms | | sprite | 109 / 117 / 169 ms | 196 / 208 / 299 ms | The steady-state difference is only tens of milliseconds; the effect that matters is the tail. In-process, a navigation that collides with an in-flight validation render can stall for seconds (this run peaked at ~7.8s on the client route, and the peak varies run to run) because a staged render does not yield until it finishes; off-thread the main thread stays free and that stall disappears. Read these as an upper bound, not a speedup that generalizes. The work moved off-thread is the validation render's CPU — bounded, route-dependent, and free of IO — so it does not grow with the main render's cost. In an app whose main render is dominated by IO the same absolute saving is a small fraction of the request, and it only appears when a navigation lands in the brief validation window; at ordinary click speed it is largely invisible. This is a dev-only responsiveness improvement whose benefit varies widely with the app and the navigation pattern. As a follow-up, the validation work that still runs on the main thread could move to the worker as well. When the main render can't be reused for validation (for example after a cache miss), the validation re-renders on the main thread to produce its inputs, resuming from the Resume Data Cache (RDC) the main render already filled, and only then hands the resulting Flight chunks to the worker. A later iteration could run those renders inside the worker too, transporting the (serializable) RDC so the worker can resume from the filled caches rather than reading them back on the main thread. closes NAR-895
-
Hendrik Liebau authored
This adds `bench/dev-validation/`, wired as `pnpm bench:dev-validation`, which measures how much dev-mode Cache Components validation contends for the dev server's event loop during rapid navigation, and how much running it on a worker thread relieves that. It toggles `experimental.devValidationWorker` (added in the previous commit) to A/B the two configurations on the same build. Until the worker implementation lands the flag is inert and the A/B shows no delta. The fixture generates one route per family (`client`, `server`, `sprite`), each nested several layout segments deep under a `(routes)` route group. Validation renders a combined payload at every URL depth, so a deeper route means more validation work per navigation, which mirrors a realistically deep app rather than a single flat segment. The runner clicks a family's `<Link>` repeatedly, since navigating to the current route re-renders and re-validates it on every click. The routes carry no `instant` config because dev validation applies to page segments by default at the warning level. The three families isolate the client prerender, the Flight re-encode plus owner-stack work, and the Flight payload size, respectively. The signal is browser-observed TTFB taken from Playwright's own network timing, because it includes the time a request waits for the event loop while validation monopolizes it. We deliberately do not use the CLI's logged request durations: the dev server starts that clock inside the request handler, after the loop has already yielded to the request, so the queue wait is invisible to it. The runner prints each configuration's absolute TTFB (p50/p95/max) side by side rather than a ratio. The time the worker frees is the validation render's CPU, which is bounded, route-dependent, and does no IO, so a ratio would overstate a win that does not scale with total request time. Because the clicks are back-to-back the numbers are a worst case — navigations that land inside the validation window — and the `max` tail is the honest headline: it is the main-thread stall the worker removes.
-