- 27 Oct, 2026 31 commits
-
-
next-js-bot[bot] authored
-
Tobias Koppers authored
### What? `turbo_bitset::Bitset` can now use different storage for its blocks. A new public `BlockStorage` trait defines what that storage must support. - `Bitset` is now `Bitset<S: BlockStorage = Vec<usize>>`, so every existing user keeps `Vec<usize>` storage. - The trait is implemented for `Vec<usize>`, `SmallVec<[usize; N]>` and `ArrayVec<usize, N>`. `turbo-bitset` now depends on `smallvec` and `arrayvec` unconditionally. - All `Bitset` methods and operators, plus `Encode`, `Decode` and `ShrinkToFit`, work with any storage. Set operations require both sides to use the same storage. The `turbo-tasks` integrations remain on the default `Bitset`. - The encoding doesn't depend on the storage, so bytes written with one storage decode into any other. - With `ArrayVec`, operations that would need more than its capacity of `N` blocks panic. Decoding a set that doesn't fit returns a `DecodeError`. - `shrink_to_fit` only touches the storage when that frees capacity. An `ArrayVec` or an inline `SmallVec` is left unchanged. ### Why? This lets small sets use inline storage where that helps, without making every `Bitset` user change its storage type. ### How? - **Bounded storage:** when an operation would exceed an `ArrayVec`'s capacity, empty blocks at either end of the storage are dropped first. This fixes a case where an insert at the back panicked although the bits fit, because earlier growth at the front had filled the capacity with zero blocks. - **`intersection`:** builds its result with `FromIterator`, so a `Vec` keeps its exact-size allocation. - **Inlining:** only the small methods of the non-generic `Vec` storage are marked `#[inline]`. The generic `SmallVec` and `ArrayVec` implementations and `Iter::next` have no annotation; removing those had no effect on the benchmark binary (see below). - **Tests:** the shared semantic tests run for all three storages. New tests cover `ArrayVec` overflow and decode errors, `SmallVec` inline/spill behaviour, `shrink_to_fit`, zero-capacity storage, and encoding equality across storages. The benchmarks gain cases for `SmallVec` and `ArrayVec`. ### Verification - `cargo fmt --check -p turbo-bitset -p turbo-tasks -p turbopack-core` - `cargo clippy -q -p turbo-bitset -p turbo-tasks -p turbopack-core -p next-api --all-targets -- -D warnings` - `RUSTDOCFLAGS="-D warnings" cargo doc -q --no-deps -p turbo-bitset -p turbopack-core` - `cargo check -q -p next-api` - `cargo test -q -p turbo-bitset` - `cargo test -q -p turbo-tasks --lib` - `cargo test -q -p turbopack-core --lib` - `cargo test -q -p turbopack-tests --test snapshot` (no snapshot changes) - `cargo bench -q -p turbo-bitset --bench mod -- --test 'bitset/storage_'` (216 cases) - `cargo run -q --release -p turbo-bitset --example sizes` gives output identical to before this change. - Not run: a 32-bit cross-check, because the `i686-unknown-linux-gnu` standard library is not installed locally. The test and benchmark changes avoid assuming a 64-bit `usize`. ### Performance caveats I compared release benchmark binaries for the previous `Vec`-only implementation and this change. Each side was built in its own target directory, I checked that the binary hashes differ, and the runs alternated old/new/new/old. - Before the final storage changes, the full comparison (336 cases) averaged **0.992×** of the old timings. - Those final changes switched `intersection` to `collect` and added `#[inline]`. After them, a focused rerun of 84 previously slower cases averaged **1.040×**: | Group | New / old (geomean) | | --- | ---: | | `intersection` | 0.907 | | `union` | 0.974 | | `contains` | 1.049 | | `insert` | 1.057 | | `iter` | 1.112 | | `from_iter` | 1.165 | - **`iter` and `contains`:** some inputs remain slower even though their normalized hot-loop instructions are the same in both binaries. The cause is unresolved. - **Empty `from_iter`:** about 3 ns becomes about 5 ns because of an extra out-of-line call. ### `#[inline]` annotations At `17be1d5e`, which annotated `Iter::next` and every `BlockStorage` method, I compared builds with and without all 43 of those annotations: full `mod` bench, 768 cases, runs alternating with/without/without/with, separate target dirs, release profile (thin LTO, one codegen unit). - Without any of them, the timings were **1.091×** overall (1.086× and 1.095× in the two pairs). The same binary run twice drifted 1.024× and 1.033×, but per group the same-binary drift reached about 30%, so this is a shared-VM microbenchmark trend, not a proven whole-build effect. - Removing only the annotation on `Iter::next`, or only the 28 on the generic `SmallVec`/`ArrayVec` implementations, gives a bench binary **byte-identical** to the annotated one. - Removing the 14 on the `Vec` implementation reproduces the whole difference. Without them, `Vec`'s `with_capacity` and `resize` wrappers remain separate functions in the binary, even with thin LTO. - So only the `Vec` annotations are kept. The bench binary of this PR's tip is byte-identical to the one built at `17be1d5e`. <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
-
Tobias Koppers authored
### What? When the dev server answers the Chrome DevTools workspace request (`/.well-known/appspecific/com.chrome.devtools.json`) and there's no persisted UUID yet, it now creates `<distDir>/cache` before writing `chrome-devtools-workspace-uuid`. ### Why? The UUID was written with a bare `writeFile`. If the cache directory didn't exist at the time of the first DevTools request (e.g. it was removed while `next dev` was running), the write failed with `ENOENT`. That logged the "needs to be reconnected after the next page reload" warning and kept the UUID in memory only, so the next server process generated a new UUID and the DevTools workspace had to be reconnected. On a normal startup the directory usually already exists because other startup writers create it, which is why this rarely shows up. ### How? - The directory is created recursively inside the existing `try/catch`. If creating the directory or writing the file fails (e.g. an unwritable path), the endpoint still responds with an in-memory UUID and logs the existing warning. **Test:** a dev-only e2e test (`// @force-gate dev`) in `test/e2e/chrome-devtools-workspace/`, using a dedicated fixture: - It waits for the asynchronous startup writes into the cache directory (`.rscinfo`, plus `next-devtools-config.json` with webpack), then removes the directory. - It then checks the endpoint returns a UUID, that the same UUID is on disk, and that no persistence warning is logged. After restarting the server, the same UUID must be returned. - The fixture disables the Turbopack filesystem cache so it can't recreate the directory mid-test. ### Verification - The new test fails without the fix and passes with it under `test-dev-turbo` and `test-dev-webpack`. - The existing `test/e2e/chrome-devtools-workspace/` suites still pass in dev (Turbopack and webpack) and in `test-start-turbo`. - Manual check: when the cache path is a regular file, so the directory can't be created, the endpoint still returns a UUID and logs the warning instead of crashing. <!-- NEXT_JS_LLM --> <!-- fleet f94f85f4-f8af-450a-a1ca-28243552f5f0 --> Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
Chad Hietala authored
### What? This PR adds a new example, `examples/eve`. The example runs an [eve](https://eve.dev) agent and a Next.js App Router app as one project. It uses the `eve/next` integration. <img width="1300" height="747" alt="image" src="https://github.com/user-attachments/assets/18c99c82-53cc-42d6-ad44-9ef153516f56" /> ### Why? Next.js does not have an example that shows how to add an eve agent to an app. This example gives a minimal starting point. You can create the example with `create-next-app --example eve`. You can deploy it with the Deploy button. The agent and the app use the same origin. They deploy as one Vercel project. ### How? **`next.config.ts`** - The `withEve()` function wraps the Next.js config. - `withEve()` puts the agent from `agent/` at `/eve/v1/*`. This path is on the same origin as the app. - When you run `next dev`, the eve dev server starts with Next.js. Next.js sends the eve routes to the eve dev server. **`agent/`** - `agent.ts` sets the model. The agent connects to the model through Vercel AI Gateway. - `instructions.md` contains the system prompt. The agent is a weather assistant. - `tools/get_weather.ts` is a typed tool. It uses a `zod` schema for the input. It gives fake weather data. - `channels/eve.ts` sets the auth policy for browser requests. The policy uses `vercelOidc()`, `localDev()`, and `none()`. **`app/chat.tsx`** This file is a Client Component. It uses the `useEveAgent` hook from `eve/react`. The component does these tasks: - It shows the messages as they stream in. - It shows the tool calls. - It sends new messages into a turn that is still running ("steer"). - It stops a turn that is still running. - It starts a new session. - It prewarms the agent when the user types. **`README.md`** The README tells you how to: - Set the model key (`AI_GATEWAY_API_KEY` or Vercel OIDC). - Run the app in development. - Build the app for production. You must run `eve build` before `next build`. - Deploy the app with the Deploy button. You must use Node.js 24 or later. ### Security The channel uses `none()`. Because of this, a deployment from the Deploy button accepts anonymous requests. Any person who has the URL can use the agent. These requests use the AI Gateway credits of the person who deployed the app. The README gives this warning. It also gives a link to the eve auth guide. --------- Co-authored-by:
Joseph <joseph.chamochumbi@vercel.com>
-
Luke Sandberg authored
Varint encoding works well for things like lengths/indicies which are naturally 'zero biased' but many of our 'id' types are not like that Switch them to use a fixed encoding This includes: * `TaskId` mostly 5 bytes before, now 4 bytes always * `CellId` always 5 bytes before, now 4 bytes always * `RawVc` always 9 bytes before, 8 bytes now * `FunctionId` mostly 3 bytes before, now 2 bytes * `ValueTypeId` mostly 3 bytes before now 2 bytes * `TraitTypeId` transformed to a `NonZeroU8` so it always takes the fixed int encode path naturally In principle it is possible that we might still benefit from varint encoding if most of the uses of these ids were biased towards smaller values, but at least in practice this isn't the case. ### Results Measured on a large app comparing to canary | | canary | this PR | change | per-round range | |---|---|---|---|---| | **Cache size after cold build** | 2.646 GB | 2.546 GB | **−3.8%** | −3.8 to −3.7% | | **Cache size after warm build** (compacted) | 2.628 GB | 2.527 GB | **−3.8%** | −3.9 to −3.7% | | **Warm build user CPU** (mostly decoding) | 43.8s | 42.2s | **−3.7%** | −5.1 to −2.3% | | Cold build user CPU | 482.6s | 483.0s | +0.1% | −1.4 to +1.0% | | Wall time, cold / warm | 75.3s / 31.2s | 76.1s / 31.4s | within noise | | | Peak RSS, cold / warm | | | −0.2% / −0.8% | |
-
next-js-bot[bot] authored
-
Jan Kassens authored
## Summary Fix compressed responses received by Node HTTP clients when `experimental.testProxy` is enabled. Requests outside a test context and proxy `continue` responses now use native HTTP passthrough, preserving the original compressed bytes and headers. Normalize stale `Content-Encoding` and `Content-Length` headers when serializing fetched responses with supported compression encodings, including cloned fetch responses. Preserve explicitly constructed mock responses as supplied, including deliberately compressed bytes. Handlers that construct a response with a modified body must update its headers themselves. Preserve bodyless responses such as HEAD so their representation metadata remains available. Add regression coverage for chunked responses and compressed bodies whose decoded length is shorter or longer than the upstream content length, across native passthrough, proxy continue, fetched responses, and modified responses. Also cover explicitly compressed mocks, fetched-response clones, unsupported encodings, and HEAD metadata. ## Verification - All 36 testmode integration tests passed in development mode with webpack, using the published native binding because the local native build has an incompatible lockfile API. - Confirmed the standalone gzip reproduction fails against the unchanged build and passes with native passthrough. <!-- NEXT_JS_LLM -->
-
Steven authored
- Release notes: https://github.com/lovell/sharp/releases/tag/v0.35.5
-
Jan Kassens authored
## Summary Proposes repository-level worktree setup for Codex and Claude Code, replacing the worktree instructions in AGENTS.md. A shared script installs dependencies and builds Next.js with its workspace dependencies. Codex runs it through a local environment; Claude creates a detached HEAD worktree and runs the same setup. Claude cleanup preserves uncommitted changes and commits that are not reachable from a branch or tag. Codex's environment must be selected when starting a worktree chat. ## Verification - Checked creation, isolated setup, detached HEAD behavior, cleanup protections, and setup failure handling in a temporary Git repository with dependency installation and builds stubbed. - Formatting, ESLint, shell syntax, and configuration checks passed. - Full dependency installation and build were not run. <!-- NEXT_JS_LLM -->
-
Aurora Scharff authored
The October pass@1 study left seven tasks in `audit` because their graders, framework contracts, or failures were not trustworthy enough to classify. This resolves every task: three repaired tasks move to regression, two valid competency tasks remain discriminatory, and two invalid public measurements are unpublished. ## Classification | Task | Before | Change and evidence | Final | | --- | --- | --- | --- | | `026-no-serial-await` | Generic JavaScript task with solution leakage and source-shape grading | Current score cannot support a Next.js competency claim | Unpublished; removed from runnable suites | | `029-use-cache-directive` | Missing Cache Components setup and incomplete flow verification | Requires Cache Components and verifies the cached read, form-bound action, shared tag, and two-argument revalidation as one flow. Gemini failed both scored treatments; confirmations were mixed | Competency | | `030-app-router-migration-hard` | Broad migration with incomplete coverage and unstable judges | Current score cannot distinguish migration ability from grader gaps | Unpublished; held out until split or substantially repaired | | `035-connection-dynamic` | Accepted `io()` for a real-request requirement and used three variable judge checks | Requires `connection()`, preserves the stable frame, and consolidates the request-specific behavior check. All 8 valid treatments passed | Regression | | `038-refresh-settings` | Grader did not connect the mutation, UI, and `refresh()` | Ties the complete Server Action flow together and rejects redirects. All 9 valid calibration results passed | Regression | | `041-optimize-ppr-shell` | Vague prompt and one implementation-oriented judge | Names the visible shell and independently revealing sections, keeps Cache Components enabled, and grades the rendered component tree. All 8 valid treatments passed | Regression | | `051-prefetch-until-navigation` | Broad source checks around navigation-stage prefetching | Scopes the title prefetch and deferred details to the destination. Frontier passed; Luna Base failed its scored attempt and both confirmations, while Luna AGENTS.md passed | Competency | The final suite contains 22 regression tasks and 7 competency tasks. No task remains in `audit`. `044-uses-nextjs` remains outside these suites because it measures framework preference for a0. ## Repaired-task results Pass@1 against the exact `16.5.0-canary.27` package: | Model | Treatment | `029` | `035` | `041` | | --- | --- | ---: | ---: | ---: | | Claude Opus 5.5 | Base | 1/1 | 1/1 | 1/1 | | Claude Opus 5.5 | AGENTS.md | 1/1 | 1/1 | 1/1 | | GPT-6 Sol | Base | 1/1 | 1/1 | 1/1 | | GPT-6 Sol | AGENTS.md | 1/1 | 1/1 | 1/1 | | Gemini 4 Argon | Base | 0/1 | 1/1 | 1/1 | | Gemini 4 Argon | AGENTS.md | 0/1 | 1/1 | 1/1 | | GPT-6 Luna | Base | 1/1 | 1/1 | 1/1 | | GPT-6 Luna | AGENTS.md | discarded | 1/1 | 1/1 | The discarded Luna `029` attempt timed out without a transcript and is not a model result. Diagnostic reruns do not change pass@1: Gemini Base passed one of two confirmations and repeated the same substantive failure once; Gemini AGENTS.md passed both confirmations. The two scored Gemini cells remain failed and should be reported as unstable. ## Validation - `pnpm prettier ... --check` on all changed files - `npx eslint --config eslint.config.mjs` on all changed graders - `pnpm eval --suite competency --dry` - `pnpm eval --suite regression --dry` - `git diff --check` Stacked on #100742, which adds suite selection and the classification protocol. <!-- NEXT_JS_LLM -->
-
Tobias Koppers authored
### What? `ExecuteContext::child_context()` in `turbo-tasks-backend` now reads the current task priority on the calling thread. The resulting `ChildExecuteContext` stores a priority that is already known, instead of an optional one. ### Why? Both users of `child_context()` hand the child context to `scope_bounded` helper threads: - the parallel path of `connect_children` (≥10k new children) - the parallel invalidation of output dependents Before this change the child context copied an override that is usually unset. On a helper thread, `get_current_task_priority()` then fell back to the task-local task state. That state only exists on the thread running the task, so helper threads got `TaskPriority::initial()`. Tasks scheduled or made dirty from those chunks (e.g. via `make_task_dirty_internal`) therefore got a priority that depended on which thread ran the chunk. ### How? The priority is resolved in `child_context()`, on the thread that creates the context. An explicit override still wins; otherwise the task-local priority is read before the work moves to another thread. Making the field non-optional means a child context can no longer carry an unresolved priority. Putting the fix in `child_context()` covers both parallel users, and any future one. Neither user relied on the late lookup, because the child context always runs on another thread. Paths that don't go parallel are unchanged. Verification: - `cargo check -p turbo-tasks-backend --all-targets` - `cargo clippy -p turbo-tasks-backend --all-targets` - `cargo nextest run -p turbo-tasks-backend` (329 passed) This PR does not add a regression test for priority on the parallel paths. Such a test could build on the existing `dirty_parent_priority_for_testing` hook used in `tests/invalidate_all.rs`. <!-- NEXT_JS_LLM --> <!-- fleet 458ef681-a3c9-47b5-a9fe-b400713e9995 --> Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
Will Binns-Smith authored
## Summary The bundle optimizer ranks client code by size, so it misses small modules that shouldn't be in the browser at all: a database schema, a token-signing helper, an agent's system prompt. This adds a size-independent leak pass to step 4 that walks every client-reachable module in the analyzer export and checks it against content heuristics. Matches are reported in a separate "Possible server-only leaks" section with route, shortest import chain, matched heuristics and compressed bytes. They stay out of the size ranking and deltas, and the agent leaves flagged modules and their import chains untouched in both modes, since moving them is the owner's call and a size metric can't verify it. The scan lives inline in SKILL.md rather than in a reference file. It runs on every invocation, and in eval runs the agent skipped a reference pointer in fix mode and missed a finding. Also rewords one existing sentence in step 4's Done line. Evals: - `agent-059-bundle-audit-server-leaks`: audit mode. Two leaks (prompt-like schema descriptions under `lib/server`, agent instructions reached through a shared util) sit well below a lodash import in size. Negative controls cover UI copy under `description`/`instructions`/`prompt` keys, a `NEXT_PUBLIC_` read and a `server-only` module used only by a Server Component. - `agent-060-bundle-fix-server-leaks`: fix mode on the same app. Checks the lodash fix still happens, the leaks are reported as unfixed, and none of the flagged modules or their importers are edited. ## Verification - `pnpm eval agent-059-bundle-audit-server-leaks`: `skills` passes, `baseline` and `agents-md` fail - `pnpm eval agent-060-bundle-fix-server-leaks`: `skills` passes, `baseline` and `agents-md` fail (both edit the flagged modules) - Not rerun: both evals after dropping the env-read case and the `/pricing` route from the fixtures <!-- NEXT_JS_LLM --> --------- Co-authored-by:vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>
-
Benjamin Woodruff authored
The webpack runner needs to compute the absolute path to the loader implementation, and it cannot compute it relative to the cwd if we're using additional roots. Adds this struct: ``` /// An argument passed to the evaluated JavaScript. #[turbo_tasks::task_input] #[derive(Clone, Hash, Debug, PartialEq, Eq, Encode, Decode)] pub enum EvaluateArg { Vc(ResolvedVc<JsonValue>), /// Passed as a string: A path relative to [`EvaluateContext::cwd`] when both are on the same /// filesystem, otherwise an absolute system path (e.g. for an additional root). /// /// This is converted when the evaluation runs, so that machine-specific system paths aren't /// stored in task inputs or cells. Path(FileSystemPath), } ``` The `Path` variant allows us to defer conversion to an absolute system path until after we've crossed the turbo-tasks boundary, right before we call the webpack loader. -
Benjamin Woodruff authored
This implements auto-configuration (behind a flag for now) for pnpm 11+. pnpm 10 supports GVS, but `node_modules/.modules.yaml` is actually YAML on 10. 11 changes it to a JSON file, which lets us avoid pulling in a YAML parser. IMO, this is fine as GVS on pnpm 10 is very experimental (it seems like it doesn't play nicely with `packageExtensions`). TODO in future PRs: - bun, nub, aube support - figure out how to support `NODE_PATH` so that hoisting works for broken packages with phantom dependencies (users can work around this with `packageExtensions`, pnpm documents this problem in the GVS docs) - Switch on GVS for (most of) the Next.js e2e tests so that we have some good coverage of this feature. https://github.com/vercel/next.js/pull/100337
-
Tobias Koppers authored
### What? Next.js now persists its server-side storage cache in Docker environments, the same as other environments. This covers the RSC / Server Actions encryption key, preview-mode keys, and the development Chrome DevTools workspace identifier. ### Why? The shared storage-directory helper previously disabled `.next/cache` persistence whenever Docker was detected. As a result, Docker environments could not reuse cached RSC encryption secrets across separate key-generation processes. ### How? The shared helper now always resolves the cache directory beneath the configured dist directory. Because storage is always available, each consumer drops the now-unreachable fallback path and uses the existing cache read/write flow directly. Existing key validation, expiry-based rotation during builds, and `NEXT_SERVER_ACTIONS_ENCRYPTION_KEY` precedence are unchanged. Telemetry is intentionally unchanged: it has its own storage-location logic for CI and Docker, which already stores telemetry configuration in `.next/cache` there. ### Verification - Ran a focused probe against the updated source with Docker detection mocked. It confirmed RSC encryption-key cache write and reuse, expired-key handling for runtime and builds, explicit encryption-key precedence, and preview-key cache write and reuse. - `pnpm --filter=next types` - Not run: full Next.js integration tests. <!-- NEXT_JS_LLM --> <!-- fleet 3a3e1bf2-4e32-499c-8f71-ae9dd2d64f16 --> Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
Tobias Koppers authored
-
Tobias Koppers authored
### What? Run new and changed deploy tests in PR CI for both deploy builders, and fold the separate cache-components job into the same matrix: - `builder`: `adapter` (`NEXT_ENABLE_ADAPTER=1`, current behavior) and `non-adapter` (`NEXT_ENABLE_ADAPTER=0`) - `cache-components`: `false` / `true` (replaces the separate `test-new-tests-deploy-cache-components` job) `test-new-tests-deploy` now runs group (5) × builder (2) × cache components (2) = 20 entries. Per entry, the env and command are the same as before: - cache components off sets `__NEXT_EXPERIMENTAL_STRICT_ROUTE_TYPES=true` - cache components on sets `__NEXT_CACHE_COMPONENTS`, `__NEXT_EXPERIMENTAL_CACHED_NAVIGATIONS` and the combined `NEXT_EXTERNAL_TESTS_FILTERS` Adapter entries keep their existing `stepName`s, so their test-report and Datadog history continues. Non-adapter entries get a `-non-adapter` suffix. Job names include the variant, builder and group. ### Why? The release deploy tests run both builders, but PR CI only checked new and changed deploy tests against the adapter. Changes that behave differently with the non-adapter builder were only caught after release. A recent example is a `resume-data-cache.partial-prefetching` failure that only showed up in the non-adapter Turbopack release shards. ### How? - Parsed the workflow and expanded the matrix to 20 entries. Each rendered `afterBuild` script was run in an empty environment with `node` stubbed out. Every entry exports the same env and runs the same command, `stepName` and display name as the matching entry from the previous two jobs. The rest of the workflow is unchanged. - `prettier --check .github/workflows/build_and_test.yml` <!-- NEXT_JS_LLM --> <!-- fleet 115592cd-38e9-4ec3-904b-f1c778e1e1d9 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
nt authored
Problem: Complete PPR pages were saved without the .rsc file that disk-cache reads require. After memory eviction or restart, those reads missed and pages rendered again. Change: Save the complete RSC payload for non-postponed PPR pages. Continue excluding fallbacks and postponed pages. Regression evidence: Without the fix, the complete-PPR unit test fails; the other 21 pass. With the fix, all 22 pass. HTTP validation: Disk-only repeat requests and cache persistence across restart pass under both Turbopack and Webpack. Fixes #100670 Fixes #98306 Closes #98307 Co-authored-by:Shtokarev Yuriy <shtokarev@gmail.com>
-
Tobias Koppers authored
## Summary When Turbopack builds an app's client reference manifests, it builds one per app entry (page). For each client reference a page uses, `build_manifest` recomputed: - the reference's server path; - its client, SSR and RSC module ids (each read goes through the chunking context's id strategy, the module's ident and a keyed module-id map read); - for every JS chunk, its component chunks (the merged chunk's runtime info plus each component chunk's path and file size). None of these depend on the page, and pages usually share most of their client references. So the number of reads grew as pages × references, and with `experimental.turbopackChunking.generateComponentChunks` as pages × references × chunks × component chunks. Every read was also another dependency edge on each page's `build_manifest` task. This PR moves that work into two turbo-tasks functions that are keyed only on page-independent inputs: - `client_reference_ids`: keyed on the client reference module, the client chunking context and the optional SSR chunking context. It returns the server path and the client, SSR and RSC module ids. - `client_chunk_components`: keyed on the chunk and the client-relative path. It returns the chunk's component chunk paths and sizes. Each page's manifest now does one cached read per reference and one per chunk. Values and insertion order are unchanged, so the generated manifests are the same. After an edit, only the tasks whose inputs changed rerun. ## Verification - Compared every `*_client-reference-manifest.js` from full `next build` runs with and without this change. All were byte-identical: - a synthetic 300-page app with 40 client references per page, default config: 302 files; - a 1000-page app with 80 references per page and `turbopackChunking: { generateComponentChunks: true, minComponentChunkSize: 0 }`: 1002 files, including merged-chunk entries. - `next dev`: after editing a client component, the edit shows up in the SSR output and the page's client reference manifest still lists the component. - `cargo clippy -p next-core --all-targets -- -D warnings`, `cargo fmt --check` - Trace counts on the 1000-page app confirm the tasks are shared: 1002 manifests ran `client_reference_ids` 408 times and `client_chunk_components` 402 times. - Rough local timings (one 8-core Linux machine, non-LTO release bindings, `next build --experimental-build-mode=compile`; exploratory, not a validated benchmark): - **Default config:** no measurable end-to-end difference; traced `build_manifest` time drops about 15%. - **Component chunks forced on (`minComponentChunkSize: 0`):** traced `build_manifest` time drops from about 63 s to about 6–8 s. Build wall time drops from about 53.8 s to about 48.5 s (median of 8 interleaved runs). - Not run locally: Edge runtime, persistent-cache rebuilds and the e2e suites (left to CI). <!-- NEXT_JS_LLM --> <!-- fleet 27c4bd4e-c7b6-4b23-85a1-32cf20f0a948 --> --------- Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
ejc3 <ejc3@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Fleet Dev Agent <fleet-dev-agent@users.noreply.github.com>
-
Batrić Perović authored
### What? `FileSystemCache` dates tag revalidations and cache entries with the wall clock, but since #94457 `areTagsExpired` compares them against the monotonic clock (`performance.timeOrigin + performance.now()`). When the wall clock runs ahead, a revalidation is dated in the future, so it's ignored and the old page keeps being served. ### Why? I ran into this with `next start`, when our E2E tests kept getting stale pages after `revalidateTag`. In a fresh 16.4.2 app with the wall clock 5ms ahead, 30 out of 30 reads right after a revalidation returned the old page. With this change, it's 0 of 30. The two clocks drift apart in ordinary setups, for example when NTP, a VM or WSL2 corrects the wall clock while the server runs. ### How? Use the monotonic clock everywhere that `FileSystemCache` dates something, as `incremental-cache/index.ts` already does: - `revalidateTag` and `set()` stamp with `performance.timeOrigin + performance.now()` instead of `Date.now()`. - `set()` also stores that timestamp on disk, as `routeCacheLastModified` in the `.meta` file and as `lastModified` in the fetch cache entry. Reads used to take the file's mtime(wall-clock time). Files without a stored time, from `next build` or an older Next.js version, still fall back to the mtime. They were written before the server started, so their mtime is earlier than any revalidation it records. This matters when entries are read from disk: with `cacheMaxMemorySize: 0`, or after an eviction from the memory cache. With only the in-memory stamps fixed, a page written just before a revalidation comes back with an mtime that's ahead, so it isn't even marked stale. Running `next start` with the memory cache off and the wall clock stepped 1s ahead, 20 of 40 revalidations left the old page in place that way. With both changes, it's 0 of 40. Notes: - `routeCacheLastModified` was only written when a build seed was promoted. Runtime writes set it now, so the "preserves seed age" test checks for it instead of its absence. - Fetch cache files are kept across deploys, and they gain a `lastModified` field. - `test/production/app-dir/file-system-cache-clock` offsets `performance.timeOrigin` at startup and checks the first request after `revalidateTag` for an ISR page, an ISR route handler and a dynamic page, with the default memory cache and `cacheMaxMemorySize: 0`. It also restarts on a fetch cache file without `lastModified`. It fails on canary, and with only the in-memory fix the `cacheMaxMemorySize: 0` cases fail. - Ran the new test with `test-start-webpack` and `test-start-turbo`, and four existing revalidation and ISR suites with `test-start-webpack`. Not run in dev.
-
Tobias Koppers authored
### What? Removes the `FixedBitSet` and `RoaringBitmap` comparisons from the `turbo-bitset` benchmarks and the `sizes` example. - Every benchmark group now measures `Bitset` only. The persistence-path `serialize` / `deserialize` and `hash_encode` benchmarks are unchanged. - `sizes` reports `Bitset`'s in-memory and serialized sizes only. - The `fixedbitset` and `roaring` dev-dependencies are removed from `turbo-bitset`. `roaring` has no other user, so it is also removed from the workspace and `Cargo.lock`. `fixedbitset` stays in the workspace for its other users. - The benchmark validation now checks `Bitset` against `BTreeSet` reference sets, outside the timed code. ### Why? The comparisons were for choosing the data structure. The design-choice data stays in #100566. Keeping them in the regular benchmark suite triples its runtime and keeps two dev-only dependencies around. ### How? `Bitset`'s implementation is unchanged, so there are no new benchmark numbers. The benchmarks still use the same inputs and persistence writer and reader. Stacked on #100630. <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
-
Tobias Koppers authored
Stacked on #100568, the top of the `Bitset` stack: #100566 (crate) → #100643 → #100567 → #100568 → this. ## Summary The PRs below move the module-graph traversals onto `Bitset`, but they still convert to `RoaringBitmapWrapper` at the boundary, and persisted values and production chunking stay Roaring. This PR finishes that, using the bincode encoding and set operations that #100566 adds: - **turbo-tasks** depends on `turbo-bitset` and implements `TaskInput`, `NonLocalValue` and `ValueDebugFormat` for `Bitset`, next to its other foreign-type impls. - **turbopack-core** drops `RoaringBitmapWrapper` and the `roaring` dependency: - `ModuleToChunkGroups`, `ChunkingHeuristicsInfo::priority_routes`, the module and chunk-item batch types, and production chunking use `Bitset` directly. - So do the local sets: `inherits_from`, cluster propagation, `cluster_groups` and `pairs_with`. - `Bitset::to_roaring()` (and its tests and benchmark) is removed, and `roaring` becomes a dev-dependency of `turbo-bitset` (for the benchmarks and the `sizes` example). - `next-api` drops its unused `roaring` dependency, which the workspace's unused-dependency lint reports once `turbopack-core` no longer uses it. **Cache compatibility:** the persisted encoding and value hashes of these types change (task inputs are hashed with fixed-width integers, for which the `Bitset` encoding always uses the bitmap form). The persistent cache is versioned by `git describe` (`turbo-tasks-backend/src/database/db_versioning.rs`), so normal builds won't read old entries. With `TURBO_ENGINE_VERSION`, `TURBO_ENGINE_DISABLE_VERSIONING` or `TURBO_ENGINE_IGNORE_DIRTY` in a dirty checkout, clear the cache when switching between binaries with and without this change. ## Results > **Historical, not rerun:** measured in the previous review round, before this stack was rebased onto canary `502b6b7d` and before the latest review changes on `b149cf8a` (#100566), `c7ea9063` (#100643), `e322a1c1` (#100567), `6536eeea` (#100568) and `417a41fd` (#100630). This round only reran the `turbo-bitset` microbenchmarks in #100566. Base is #100568's head, so the encoding of #100566 is the same on both sides; only what this PR persists as `Bitset` instead of Roaring differs. ### Full builds Full `next build` (including prerendering) with uninstrumented `next-swc` built with the repository's `release` profile (thin LTO), on a shared VM with 8 vCPUs and 16 GB. 5 randomized, interleaved runs per variant and app, median with min–max. Each run: 1. **cold**: removes `.next` and builds; 2. **cache**: measures `.next/cache` (the Turbopack build cache) with `du -sb`; 3. **warm**: removes everything in `.next` except that variant's own cache and builds again. Caches are never shared between variants. Both binaries were built from a checkout with uncommitted changes, so every build ran with `TURBO_ENGINE_IGNORE_DIRTY=1` to keep the filesystem cache enabled (each build logs that it is). | full `next build` | base (#100568) | this PR | |---|---|---| | `bench/nested-deps-app-router-many-pages` (3,000 routes): cold build | 53.1 s (51.7–54.1) | 52.2 s (51–56.5) | | `bench/nested-deps-app-router-many-pages` (3,000 routes): `.next/cache` size | 762.95 MB (762.85–763.19) | 761.49 MB (761.24–761.84) | | `bench/nested-deps-app-router-many-pages` (3,000 routes): warm build (own cache) | 35.8 s (34.8–36.7) | 35.5 s (34.8–36.6) | | generated 300-route app: cold build | 11.3 s (10.8–11.7) | 11.1 s (10.9–11.7) | | generated 300-route app: `.next/cache` size | 213.84 MB (213.45–214.45) | 213.85 MB (213.76–213.94) | | generated 300-route app: warm build (own cache) | 5.42 s (5.14–5.5) | 5.46 s (5.18–5.57) | All 40 builds succeeded with the same routes per app. The cache is reused: warm builds take 48–68% of the cold time for both variants and rewrite part of the cache. On `many-pages` the cache is 1.46 MB (0.19%) smaller with non-overlapping ranges; on the generated app the difference is within the run-to-run variation (±0.5 MB). Cold and warm times overlap, so this makes no end-to-end speed claim. <details> <summary>Charts: full builds (linear)</summary> Median wall time in seconds. Ranges overlap; exact values are in the table above. ```mermaid xychart-beta title "bench/nested-deps-app-router-many-pages: full build, median s" x-axis ["cold", "warm"] y-axis "s" bar "base (#100568)" [53.1, 35.8] bar "this PR" [52.2, 35.5] ``` ```mermaid xychart-beta title "generated 300-route app: full build, median s" x-axis ["cold", "warm"] y-axis "s" bar "base (#100568)" [11.3, 5.42] bar "this PR" [11.1, 5.46] ``` </details> ### Encoded sizes Logical set payloads from local-only size probes (bytes of the encoded values, not the cache database, so keys, metadata and compression aren't included), same compile-mode builds as below. They are identical in every run: | logical payload | base (#100568) | this PR | |---|---|---| | `bench/nested-deps-app-router-many-pages` (3,000 routes): encoded `ModuleToChunkGroups` values (33,779 sets) | 40,212,209 B | **9,711,974 B** (-76%) | | `bench/nested-deps-app-router-many-pages` (3,000 routes): encoded `ModuleBatch` chunk groups (4,132 sets) | 721,522 B | **97,208 B** (-87%) | | `bench/nested-deps-app-router-many-pages` (3,000 routes): encoded `ModuleBatchGroup` chunk groups (3,021 sets) | 138,211 B | **27,960 B** (-80%) | | `bench/nested-deps-app-router-many-pages` (3,000 routes): per-module set p50 / p99 / max | 19 / 6,023 / 8,211 B | **5 / 1,006 / 2,501 B** | | generated 300-route app: encoded `ModuleToChunkGroups` values (12,240 sets) | 1,971,644 B | **434,660 B** (-78%) | | generated 300-route app: encoded `ModuleBatch` chunk groups (2,101 sets) | 220,751 B | **42,221 B** (-81%) | | generated 300-route app: encoded `ModuleBatchGroup` chunk groups (629 sets) | 30,165 B | **5,578 B** (-82%) | | generated 300-route app: per-module set p50 / p99 / max | 19 / 1,241 / 3,687 B | **3 / 235 / 339 B** | ### Stage times `next build --experimental-build-mode compile` with release `next-swc` (LTO off, 16 codegen units) built from each PR's head plus local-only stage timers. Median of 5 randomized, interleaved runs per variant, min–max in parentheses, in ms. All five layers of the stack were measured in the same interleaved set. | stage (ms) | previous PR | this PR | |---|---|---| | `bench/nested-deps-app-router-many-pages` (3,000 routes): heuristics + building the result | 37.7 (37.1–39.4) | **5.17 (4.66–5.39)** | | `bench/nested-deps-app-router-many-pages` (3,000 routes): production chunk merging (summed) | 317 (313–326) | **249 (240–253)** | | generated 300-route app: heuristics + building the result | 3.71 (3.57–3.78) | **0.62 (0.574–0.657)** | | generated 300-route app: production chunk merging (summed) | 19.3 (18.7–20.5) | **14.2 (13.6–14.8)** | <details> <summary>Charts: stage times (linear)</summary> Median ms per stage; the first series is the previous PR. Exact values and ranges are in the table above. ```mermaid xychart-beta title "bench/nested-deps-app-router-many-pages, median ms" x-axis ["heuristics + building the result", "production chunk merging (summed)"] y-axis "ms" bar "previous PR" [37.7, 317] bar "this PR" [5.17, 249] ``` ```mermaid xychart-beta title "generated 300-route app, median ms" x-axis ["heuristics + building the result", "production chunk merging (summed)"] y-axis "ms" bar "previous PR" [3.71, 19.3] bar "this PR" [0.62, 14.2] ``` </details> Sum of the measured module-graph stages per run (chunk group info and building its result, collect, merged modules and interning), whole stack: | sum of module-graph stages (ms) | #100566 (before the usage PRs) | top of stack (#100630) | |---|---|---| | `bench/nested-deps-app-router-many-pages` (3,000 routes) | 750 (746–767) | **102 (99.7–113)** | | generated 300-route app | 36.3 (35.1–38) | **17.8 (17.5–18.1)** | <details> <summary>Charts: stage sum per layer (linear)</summary> Median ms of the summed module-graph stages, one bar per layer of the stack. Exact values for the ends are in the table above. ```mermaid xychart-beta title "bench/nested-deps-app-router-many-pages: stage sum per layer, median ms" x-axis ["#100566", "#100643", "#100567", "#100568", "#100630"] y-axis "ms" bar "stage sum" [750, 455, 288, 128, 102] ``` ```mermaid xychart-beta title "generated 300-route app: stage sum per layer, median ms" x-axis ["#100566", "#100643", "#100567", "#100568", "#100630"] y-axis "ms" bar "stage sum" [36.3, 28.8, 27.1, 21, 17.8] ``` </details> Visit counts and the emitted output are identical across all layers on both apps (14,028 and 2,142 files, same digests). ## Verification - `cargo test -p turbo-bitset`, `-p turbo-tasks --lib`, `-p turbopack-core --lib`, the turbopack snapshot tests (no snapshot changes) and execution tests. - `cargo fmt --check`, `cargo clippy` (touched crates plus `next-api`, `next-core`, all targets) and `cargo doc` with `-D warnings` (no unused-dependency warning). - `cargo tree -p turbopack-core -e normal` no longer contains `roaring`. - The builds above, with timers and probes kept local only. <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
Tobias Koppers authored
Stacked on #100567 (`collect_graph`). Stack: #100566 → #100643 → #100567 → this → #100630. ## Summary `compute_merged_modules` now uses `Bitset` for its per-module merge group sets during the fixed-point traversal, the same change as in the PRs below. Like #100643, it drops the first-visit `is_zero()` check plus `clone()`, as `union_with` copies into a set that stores no blocks. The bitmap interning step that follows (which gives each distinct set a dense id) also works on `Bitset`. Hashing a `Bitset` hashes its words directly; hashing a `RoaringBitmapWrapper` serializes the bitmap. The merged module result types are unchanged. ## Benchmarks > **Historical, not rerun:** measured in the previous review round, before this stack was rebased onto canary `502b6b7d` and before the latest review changes on `b149cf8a` (#100566), `c7ea9063` (#100643), `e322a1c1` (#100567), `6536eeea` (#100568) and `417a41fd` (#100630). This round only reran the `turbo-bitset` microbenchmarks in #100566. `next build --experimental-build-mode compile` with release `next-swc` (LTO off, 16 codegen units) built from each PR's head plus local-only stage timers, on a shared VM with 8 vCPUs and 16 GB. Each number is the median of 5 randomized, interleaved runs per variant, with the min–max range in parentheses, in ms. All five layers of the stack were measured in the same interleaved set, so adjacent PRs compare directly. | stage (ms) | previous PR | this PR | |---|---|---| | `bench/nested-deps-app-router-many-pages` (3,000 routes): merged-modules fixed point | 109 (108–110) | **22.9 (22.4–24.5)** | | `bench/nested-deps-app-router-many-pages` (3,000 routes): bitmap interning | 75.9 (73.1–81.5) | **6.72 (6.16–6.87)** | | generated 300-route app: merged-modules fixed point | 9.14 (8.63–13.7) | **5.87 (5.79–6.04)** | | generated 300-route app: bitmap interning | 3.59 (3.49–6.33) | **0.672 (0.631–0.69)** | Visit counts are identical, and the emitted JS chunks are byte-identical across all layers of the stack on both apps. Whole-build time is too noisy on these apps to show a reliable end-to-end difference. The sum over the whole stack is in #100630. <details> <summary>Charts: stage times (linear)</summary> Median ms per stage; the first series is the previous PR. Exact values and ranges are in the table above. ```mermaid xychart-beta title "bench/nested-deps-app-router-many-pages, median ms" x-axis ["merged-modules fixed point", "bitmap interning"] y-axis "ms" bar "previous PR" [109, 75.9] bar "this PR" [22.9, 6.72] ``` ```mermaid xychart-beta title "generated 300-route app, median ms" x-axis ["merged-modules fixed point", "bitmap interning"] y-axis "ms" bar "previous PR" [9.14, 3.59] bar "this PR" [5.87, 0.672] ``` </details> ## Verification - `cargo test -p turbopack-core --lib` (including the interning tests), all turbopack snapshot tests and all turbopack execution tests pass. - The real builds above. <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
Tobias Koppers authored
Stacked on #100643 (`compute_chunk_group_info`), on top of #100566, which adds `turbo-bitset`. Stack: #100566 → #100643 → this → #100568 → #100630. ## Summary `collect_graph` now uses `Bitset` for its per-module entry membership sets during the fixed-point traversal, the same change as in `compute_chunk_group_info`. It never stores these sets, so nothing needs converting back to `RoaringBitmapWrapper`. Like #100643, it drops the first-visit `is_zero()` check plus `clone()`, as `union_with` copies into a set that stores no blocks. When matching collected references to entries, the old code built a Roaring intersection of the collecting module's and the emitted module's memberships. It now walks the collecting module's entries and keeps those that the emitted module's membership `contains`. That allocates no intermediate set, and the order and the skip-when-empty behavior are unchanged. ## Benchmarks > **Historical, not rerun:** measured in the previous review round, before this stack was rebased onto canary `502b6b7d` and before the latest review changes on `b149cf8a` (#100566), `c7ea9063` (#100643), `e322a1c1` (#100567), `6536eeea` (#100568) and `417a41fd` (#100630). This round only reran the `turbo-bitset` microbenchmarks in #100566. `next build --experimental-build-mode compile` with release `next-swc` (LTO off, 16 codegen units) built from each PR's head plus local-only stage timers, on a shared VM with 8 vCPUs and 16 GB. Each number is the median of 5 randomized, interleaved runs per variant, with the min–max range in parentheses, in ms. All five layers of the stack were measured in the same interleaved set, so adjacent PRs compare directly. | stage (ms) | previous PR | this PR | |---|---|---| | `bench/nested-deps-app-router-many-pages` (3,000 routes): `collect_graph` fixed point | 189 (186–195) | **16 (15–17.1)** | | generated 300-route app: `collect_graph` fixed point | 6.38 (6.32–6.68) | **3.37 (3.33–4.02)** | Visit counts are identical, and the emitted JS chunks are byte-identical across all layers of the stack on both apps. Whole-build time is too noisy on these apps to show a reliable end-to-end difference. <details> <summary>Charts: stage times (linear)</summary> Median ms per stage; the first series is the previous PR. Exact values and ranges are in the table above. ```mermaid xychart-beta title "bench/nested-deps-app-router-many-pages, median ms" x-axis ["`collect_graph` fixed point"] y-axis "ms" bar "previous PR" [189] bar "this PR" [16] ``` ```mermaid xychart-beta title "generated 300-route app, median ms" x-axis ["`collect_graph` fixed point"] y-axis "ms" bar "previous PR" [6.38] bar "this PR" [3.37] ``` </details> ## Verification - `cargo test -p turbopack-core --lib`, the turbopack snapshot tests and the collect execution tests pass. All execution tests pass on the top of the stack. - The real builds above. <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
Tobias Koppers authored
Stacked on #100566, which adds `turbo-bitset`. Stack: #100566 → this → #100567 (`collect_graph`) → #100568 (`compute_merged_modules`) → #100630 (persisted chunk group sets). ## Summary `compute_chunk_group_info` tracked each module's chunk groups with `RoaringBitmapWrapper` during its fixed-point traversal. The traversal checks "would `current |= parent` change `current`?" on every edge and then does the union; on Roaring's array containers both cost time proportional to the number of bits, which made this the slowest of the module graph passes on `bench/nested-deps-app-router-many-pages`. It now uses `Bitset` for the working per-module sets, and converts them with `to_roaring()` when building the result. `ModuleToChunkGroups`, the batch types and production chunking still use `RoaringBitmapWrapper`, so persisted values are unchanged (#100630 switches those). The old code called `RoaringBitmapWrapper::is_proper_superset` (which actually computed `!parent.is_subset(current)`) and then `|=`. Now `current.union_with(parent)` does both in one pass and returns whether it added a bit. The separate first-visit `is_zero()` check plus `clone()` is gone too: `union_with` copies into a set that stores no blocks. This doesn't change the traversal, as a parent's set is never empty when its children are visited. ## Benchmarks > **Historical, not rerun:** measured in the previous review round, before this stack was rebased onto canary `502b6b7d` and before the latest review changes on `b149cf8a` (#100566), `c7ea9063` (#100643), `e322a1c1` (#100567), `6536eeea` (#100568) and `417a41fd` (#100630). This round only reran the `turbo-bitset` microbenchmarks in #100566. `next build --experimental-build-mode compile` with release `next-swc` (LTO off, 16 codegen units) built from each PR's head plus local-only stage timers, on a shared VM with 8 vCPUs and 16 GB. Each number is the median of 5 randomized, interleaved runs per variant, with the min–max range in parentheses, in ms. All five layers of the stack were measured in the same interleaved set, so adjacent PRs compare directly. | stage (ms) | previous PR | this PR | |---|---|---| | `bench/nested-deps-app-router-many-pages` (3,000 routes): chunk-group fixed point | 379 (375–384) | **49.9 (45.8–54.2)** | | `bench/nested-deps-app-router-many-pages` (3,000 routes): building `ChunkGroupInfo` | 3.73 (3.16–4.08) | **37.4 (35.7–42.2)** | | generated 300-route app: chunk-group fixed point | 17.3 (16.4–18.1) | **7.09 (6.93–7.33)** | | generated 300-route app: building `ChunkGroupInfo` | 0.444 (0.417–0.471) | **3.57 (3.35–3.62)** | Building `ChunkGroupInfo` now includes the `to_roaring()` conversion, which #100630 removes. Visit counts are identical, and the emitted JS chunks are byte-identical across all layers of the stack on both apps (14,028 and 2,142 files, same digests). Whole-build time is too noisy on these apps to show a reliable end-to-end difference. The microbenchmarks are in #100566. <details> <summary>Charts: stage times (linear)</summary> Median ms per stage; the first series is the previous PR. Exact values and ranges are in the table above. ```mermaid xychart-beta title "bench/nested-deps-app-router-many-pages, median ms" x-axis ["chunk-group fixed point", "building `ChunkGroupInfo`"] y-axis "ms" bar "previous PR" [379, 3.73] bar "this PR" [49.9, 37.4] ``` ```mermaid xychart-beta title "generated 300-route app, median ms" x-axis ["chunk-group fixed point", "building `ChunkGroupInfo`"] y-axis "ms" bar "previous PR" [17.3, 0.444] bar "this PR" [7.09, 3.57] ``` </details> ## Verification - `cargo test -p turbopack-core --lib` and the turbopack snapshot tests pass without snapshot changes; the execution tests pass on the top of the stack. - The real builds above. <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-
Tobias Koppers authored
Stacked on #100452. This PR only adds the crate; the PRs above use it: #100643 (`compute_chunk_group_info`) → #100567 (`collect_graph`) → #100568 (`compute_merged_modules`) → #100630 (persisted chunk group sets) → #100724 (removes the FixedBitSet/Roaring comparisons from the benchmarks). ## Summary The module graph fixed-point traversals track set membership with `RoaringBitmap`, which is built for compact storage, not speed. They check "would `current |= parent` change `current`?" on every edge and then do the union; on array containers both cost time proportional to the number of bits. This PR adds a new `turbo-bitset` crate with `Bitset`: a `Vec<usize>` of blocks plus an offset counted in whole blocks. - Bit `i` is stored at `i - offset * usize::BITS`, so a set only stores the word range between its lowest and highest bit. Sets whose bits are clustered at high indices stay small. - Unions, intersections, differences, subset checks, `is_disjoint` and `intersection_len` are word loops over the overlapping range, so their cost is proportional to the range, not the number of bits. The subset check reduces chunks of 4 words before branching, so it vectorizes. - `union_with` returns whether it added a bit. It ORs the words and collects the added bits in the same loop, so a traversal checks for a change and applies the union in one pass instead of a subset check plus `|=`. If the set stores no blocks (a module's first visit), it copies `other` without its zero edge blocks, reusing a reservation or allocating exactly the copied blocks, so the traversals need no separate `is_zero()` check and `clone()`. - `contains`, `insert` and `remove` check the stored range with a single comparison. Growth is amortized at both ends: growing at the back uses the `Vec`'s growth, and growing at the front moves the blocks once to the end of the (reused or doubled) capacity, which leaves headroom for further front growth. So inserting in any order, including descending, moves each block O(log n) times. `extend` inserts the items directly. - `Bitset::with_len(bits: u32)` stores exactly the zeroed blocks for the bits `0..bits` at offset 0, in an allocation without spare capacity, so inserting any of them doesn't grow the storage. `Bitset::with_capacity(bits: u32)` reserves capacity for that many bits without storing any block; the first insert still picks the offset. - Equality and hashing depend only on the bits, not on the stored range. `count_ones()` counts over the stored range (the name says it isn't free), `is_zero()` checks for no bits. `Bitset` implements `ShrinkToFit`. - The storage is a plain `Vec`. A `SmallVec<[usize; INLINE]>` variant was tried and reverted: it adds an inline-or-heap branch to every block access, and no user needs inline blocks. Against it, `Vec` measured 0.967× overall in the microbenchmarks; unions right after a clone are slower, because `Vec::clone` allocates exactly the stored blocks (details in the review thread). Clones keep the default exact capacity, to avoid extra memory per module set in the traversals. - `to_roaring()` converts back container by container (2^16 bits at a time) straight from the words, with the same container types (and serialization) as inserting the bits one by one. The PRs above use it at the boundary until #100630 removes it. - No `#[inline]` hints: the crate relies on thin LTO. Removing the 8 hints it had (one went away with the hand-written varint writer) changed 41 of 480 benchmark cases by ≥ 5% consistently (24 faster, 17 slower; geometric mean 0.99×), and restoring the decoder's most likely candidate made 3 cases faster and 3 slower, so none is kept. ### Bincode encoding `Encode`/`Decode` (in `src/encode.rs`) start with a signed length header: - `0`: the empty set. - `len > 0`, the **bitmap form**: a `u32` byte offset, then `len` raw bytes, with leading and trailing zero bytes trimmed. - `len = -n < 0`, the **list form**: the payload length `len` the bitmap form would have, then the first bit and `n - 1` differences (each `≥ 1`), all `u32` (varints under the standard config). The bits must cover exactly that payload range. Which form is written: - With varint integers (how Turbopack persists), the list form is used iff `5 · n < 4 · len`, for `n` bits and a bitmap payload of `len` bytes. That only needs the bit count (counting stops once it reaches the threshold), not the sizes of the list items. - If the bitmap is chosen, its payload is at most 1.25× the list's items, as every item takes at least one byte. - If the list is chosen, the bits are more than 10 apart on average, and only differences above 250 take more than one byte; their sum is below `8 · len`, so the differences take fewer bytes than the bitmap payload. - The rule ignores the headers and the first item, so for sets with very few bits the chosen form can be a few bytes (at most 13) larger than the other. - For a set spanning its whole range, the switch is at 10% density. The 10%-of-16,384-bits benchmark input has exactly 1,638 bits and a 2,047-byte payload, just at the threshold (5 · 1,638 = 8,190 ≥ 4 · 2,047 = 8,188), so it is written as a bitmap. - With fixed-width integers (how task inputs are hashed), always the bitmap form. Integers are encoded with bincode's own `Encode` impls. The list items are encoded with the outer encoder's config into a 1 KB stack buffer (bincode's `SliceWriter`), which is written in chunks, a block at a time; encoding doesn't allocate. The bitmap bytes are little-endian on every platform: on little-endian targets they are written and read with a single copy of the storage viewed as bytes (documented `unsafe` slice casts), big-endian targets convert each block. Decoding allocates the storage once (from the bitmap length in both forms), claiming it from the decoder's byte limit first. List items are set into a register-held block, and runs of single-byte varints are handled in batches, still claiming 4 bytes per item like `u32::decode`. Malformed input is rejected with a `DecodeError`: an out-of-range header or payload, a bitmap or list past the `u32` range, a zero or overflowing difference, a list item outside its payload range or a list that doesn't end in its last byte, and truncated input. ## Benchmarks All numbers in this section are criterion point estimates from `cargo bench -p turbo-bitset` (workspace release settings, thin LTO, `RUSTFLAGS="-C llvm-args=-x86-branches-within-32B-boundaries"`, rustc 1.101.0-nightly 2026-10-05; bincode 2.0.1, fixedbitset 0.5.7, roaring 0.11.5, codspeed-criterion-compat 4.3.0), on this PR's `Vec`-backed code before the sizing constructors and `union_with -> bool` were added (tree of `d7d69b85`), on a shared VM with 8 vCPUs and 16 GB. ### Microbenchmarks Each operation compares `Bitset` with a plain `FixedBitSet` (indexed from bit 0) and `RoaringBitmap` 0.11.5. Inputs are `empty`, `1`, `4` and `16` random bits, plus densities that set at least 32 bits (10%, 50%, 100% of 1,024 bits; 1%, 10%, 50%, 100% of 16,384 bits; 0.1%, 1%, 10%, 50%, 100% of 65,536 bits). Every input has exactly its count, or exactly ⌊universe · density⌋ distinct bits, picked by selection sampling from a seeded `SmallRng`, starting at bit 0. The `sizes` example uses the same inputs (`benches/common/inputs.rs`). Results are checked for agreement outside the measurement. Cells are Bitset / FixedBitSet / RoaringBitmap in ns, 16,384-bit universe: | operation | 1 bit | 16 bits | 1% | 10% | 100% | |---|---|---|---|---|---| | union (`a \|= b`) | 52 / 9.8 / 24 | 178 / 81 / 92 | 75 / 79 / 600 | 78 / 78 / 5,633 | 101 / 84 / 834 | | superset, holds | 2.3 / 3.1 / 3.8 | 58 / 90 / 32 | 63 / 89 / 263 | 62 / 89 / 2,728 | 72 / 89 / 384 | | intersection | 2.9 / 19 / 23 | 102 / 106 / 52 | 90 / 102 / 350 | 88 / 101 / 3,383 | 86 / 102 / 918 | | contains (1,024 lookups) | 774 / 593 / 3,234 | 731 / 768 / 4,309 | 765 / 749 / 9,437 | 750 / 748 / 12,535 | 766 / 747 / 2,474 | | insert (random order) | 19 / 24 / 35 | 249 / 119 / 255 | 387 / 261 / 3,709 | 2,225 / 1,695 / 69,182 | 19,087 / 14,452 / 328,090 | | iter | 2.3 / 57 / 8.0 | 120 / 157 / 52 | 282 / 256 / 389 | 1,337 / 1,098 / 3,832 | 15,188 / 13,820 / 44,408 | <details> <summary>Charts: microbenchmarks (log scale)</summary> log10 of the time in ns (1 = 10 ns, 3 = 1 µs), 16,384-bit universe. Exact values are in the table above. ```mermaid xychart-beta title "union (a |= b)" x-axis ["1 bit", "16 bits", "1%", "10%", "100%"] y-axis "log10(ns)" line "Bitset" [1.72, 2.25, 1.88, 1.89, 2.0] line "FixedBitSet" [0.99, 1.91, 1.9, 1.89, 1.92] line "RoaringBitmap" [1.38, 1.97, 2.78, 3.75, 2.92] ``` ```mermaid xychart-beta title "superset, holds" x-axis ["1 bit", "16 bits", "1%", "10%", "100%"] y-axis "log10(ns)" line "Bitset" [0.37, 1.77, 1.8, 1.79, 1.86] line "FixedBitSet" [0.49, 1.95, 1.95, 1.95, 1.95] line "RoaringBitmap" [0.58, 1.51, 2.42, 3.44, 2.58] ``` ```mermaid xychart-beta title "intersection" x-axis ["1 bit", "16 bits", "1%", "10%", "100%"] y-axis "log10(ns)" line "Bitset" [0.46, 2.01, 1.96, 1.94, 1.93] line "FixedBitSet" [1.27, 2.03, 2.01, 2.0, 2.01] line "RoaringBitmap" [1.36, 1.72, 2.54, 3.53, 2.96] ``` ```mermaid xychart-beta title "contains (1,024 lookups)" x-axis ["1 bit", "16 bits", "1%", "10%", "100%"] y-axis "log10(ns)" line "Bitset" [2.89, 2.86, 2.88, 2.88, 2.88] line "FixedBitSet" [2.77, 2.89, 2.87, 2.87, 2.87] line "RoaringBitmap" [3.51, 3.63, 3.97, 4.1, 3.39] ``` ```mermaid xychart-beta title "insert (random order)" x-axis ["1 bit", "16 bits", "1%", "10%", "100%"] y-axis "log10(ns)" line "Bitset" [1.29, 2.4, 2.59, 3.35, 4.28] line "FixedBitSet" [1.39, 2.08, 2.42, 3.23, 4.16] line "RoaringBitmap" [1.54, 2.41, 3.57, 4.84, 5.52] ``` ```mermaid xychart-beta title "iter" x-axis ["1 bit", "16 bits", "1%", "10%", "100%"] y-axis "log10(ns)" line "Bitset" [0.36, 2.08, 2.45, 3.13, 4.18] line "FixedBitSet" [1.76, 2.2, 2.41, 3.04, 4.14] line "RoaringBitmap" [0.91, 1.72, 2.59, 3.58, 4.65] ``` </details> ### Ordered inserts Building a set by inserting its bits one by one in a fixed random order, ascending (the set only grows at the back) and descending (it only grows at the front). FixedBitSet uses `grow_and_insert`. Bitset / FixedBitSet / RoaringBitmap in ns, 16,384-bit universe: | order | 1 bit | 16 bits | 1% | 10% | 100% | |---|---|---|---|---|---| | random order | 19 / 24 / 35 | 249 / 119 / 255 | 387 / 261 / 3,709 | 2,225 / 1,695 / 69,182 | 19,087 / 14,452 / 328,090 | | ascending | 20 / 25 / 35 | 337 / 362 / 241 | 971 / 1,038 / 2,879 | 3,385 / 7,760 / 29,593 | 31,185 / 71,557 / 160,400 | | descending | 19 / 25 / 36 | 209 / 74 / 286 | 417 / 201 / 4,570 | 2,431 / 1,666 / 82,091 | 29,920 / 27,721 / 393,320 | Before/after A/B of this round's growth change (front growth with headroom instead of an exact shift, and `extend` inserting items directly instead of buffering them). Both sides use the new inputs and benchmarks; "before" is the previous `ensure_blocks`/`Extend` on the rebased code. Two interleaved rounds per side (before, after, after, before), mean in ns: | input | `insert_desc` before → after | `from_iter` before → after | |---|---|---| | `low-1024-16bits` | 167 → 116 (0.69×) | 70 → 146 (2.08×) | | `low-1024-10%` | 377 → 209 (0.55×) | 277 → 292 (1.05×) | | `low-16384-16bits` | 406 → 227 (0.56×) | 108 → 296 (2.76×) | | `low-16384-1%` | 2,497 → 413 (0.17×) | 424 → 1,046 (2.47×) | | `low-16384-10%` | 7,285 → 2,378 (0.33×) | 3,889 → 3,392 (0.87×) | | `low-16384-100%` | 35,372 → 30,340 (0.86×) | 47,816 → 29,290 (0.61×) | | `low-65536-0.1%` | 3,036 → 774 (0.25×) | 250 → 861 (3.45×) | | `low-65536-1%` | 23,598 → 1,235 (0.05×) | 1,597 → 3,178 (1.99×) | | `low-65536-10%` | 57,686 → 11,312 (0.20×) | 15,433 → 12,488 (0.81×) | | `low-65536-100%` | 171,340 → 132,520 (0.77×) | 191,165 → 115,290 (0.60×) | Descending inserts used to move every stored block on each new lowest block, which is quadratic; they are now up to 19× faster. `from_iter` is 13–40% faster for 10% to 100% of 16,384 and 65,536 bits, but **1.8–3.5× slower for the sparse inputs** (4 and 16 bits, 0.1% and 1%; e.g. 0.1% of 65,536 bits: 250 → 861 ns). The removed buffered `extend` computed the whole range first and allocated once; inserting directly grows the storage several times. <details> <summary>Chart: descending inserts before vs after (log scale)</summary> log10 of the time in ns, mean of the two rounds per side. Exact values are in the table above. ```mermaid xychart-beta title "insert_desc" x-axis ["1024-16bits", "1024-10%", "16384-16bits", "16384-1%", "16384-10%", "16384-100%", "65536-0.1%", "65536-1%", "65536-10%", "65536-100%"] y-axis "log10(ns)" bar "before" [2.22, 2.58, 2.61, 3.4, 3.86, 4.55, 3.48, 4.37, 4.76, 5.23] bar "after" [2.06, 2.32, 2.36, 2.62, 3.38, 4.48, 2.89, 3.09, 4.05, 5.12] ``` </details> ### Serialization With the writer and reader Turbopack persists with (`turbo_bincode`: `TurboBincodeWriter` into a new `TurboBincodeBuffer`, `TurboBincodeReader`, standard config). Roaring goes through serde, as `RoaringBitmapWrapper` persists it. Bitset / RoaringBitmap in ns: | universe | 1 bit | 16 bits | 0.1% | 1% | 10% | 50% | 100% | |---|---|---|---|---|---|---|---| | 1,024: serialize | 33 / 102 | 83 / 184 | - | - | 62 / 347 | 62 / 904 | 63 / 1,467 | | 16,384: serialize | 41 / 102 | 394 / 177 | - | 704 / 404 | 286 / 2,020 | 143 / 1,501 | 147 / 1,529 | | 65,536: serialize | 42 / 102 | 1,316 / 164 | 1,514 / 255 | 2,726 / 1,074 | 889 / 1,435 | 278 / 2,136 | 207 / 1,445 | | 1,024: deserialize | 57 / 137 | 67 / 153 | - | - | 53 / 208 | 54 / 627 | 53 / 1,060 | | 16,384: deserialize | 47 / 132 | 143 / 132 | - | 332 / 267 | 91 / 1,579 | 107 / 1,148 | 89 / 1,145 | | 65,536: deserialize | 48 / 174 | 193 / 172 | 443 / 187 | 1,078 / 833 | 150 / 1,172 | 149 / 1,141 | 153 / 1,140 | For the bitmap-form inputs, `Bitset` is 1.6–23× faster than Roaring to serialize and 2.4–20× faster to deserialize. The list-form inputs (4 and 16 bits, 0.1% and 1%) depend on the width of the stored range: on 1,024 bits `Bitset` is 1.7–2.3× faster both ways, but on 16,384 and 65,536 bits it is up to 11× slower to serialize (4 bits in 65,536) and up to 2.4× slower to deserialize (0.1% of 65,536) than Roaring's sorted arrays. Both directions walk the stored word range, not just the set bits. <details> <summary>Charts: serialization (log scale)</summary> log10 of the time in ns, `TurboBincodeWriter`/`TurboBincodeReader`. Points that set fewer than 32 bits are left out. Exact values are in the table above. ```mermaid xychart-beta title "1,024: serialize" x-axis ["1 bit", "16 bits", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset" [1.52, 1.92, 1.79, 1.79, 1.8] line "RoaringBitmap" [2.01, 2.26, 2.54, 2.96, 3.17] ``` ```mermaid xychart-beta title "16,384: serialize" x-axis ["1 bit", "16 bits", "1%", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset" [1.62, 2.6, 2.85, 2.46, 2.16, 2.17] line "RoaringBitmap" [2.01, 2.25, 2.61, 3.31, 3.18, 3.18] ``` ```mermaid xychart-beta title "65,536: serialize" x-axis ["1 bit", "16 bits", "0.1%", "1%", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset" [1.62, 3.12, 3.18, 3.44, 2.95, 2.44, 2.32] line "RoaringBitmap" [2.01, 2.21, 2.41, 3.03, 3.16, 3.33, 3.16] ``` ```mermaid xychart-beta title "1,024: deserialize" x-axis ["1 bit", "16 bits", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset" [1.75, 1.83, 1.73, 1.73, 1.72] line "RoaringBitmap" [2.14, 2.18, 2.32, 2.8, 3.03] ``` ```mermaid xychart-beta title "16,384: deserialize" x-axis ["1 bit", "16 bits", "1%", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset" [1.67, 2.16, 2.52, 1.96, 2.03, 1.95] line "RoaringBitmap" [2.12, 2.12, 2.43, 3.2, 3.06, 3.06] ``` ```mermaid xychart-beta title "65,536: deserialize" x-axis ["1 bit", "16 bits", "0.1%", "1%", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset" [1.68, 2.29, 2.65, 3.03, 2.18, 2.17, 2.19] line "RoaringBitmap" [2.24, 2.24, 2.27, 2.92, 3.07, 3.06, 3.06] ``` </details> Before/after A/B of removing the decoder's batch copy (the bytes are now read straight from `peek_read`). Both builds differ only in `decode_list`; same scheme as above: | input (form) | `deserialize` before → after | |---|---| | `low-1024-16bits` (list) | 68 → 60 (0.88×) | | `low-16384-4bits` (list) | 95 → 86 (0.90×) | | `low-16384-16bits` (list) | 206 → 174 (0.84×) | | `low-16384-1%` (list) | 358 → 344 (0.96×) | | `low-16384-10%` (bitmap) | 91 → 112 (1.24×) | | `low-65536-16bits` (list) | 219 → 187 (0.85×) | | `low-65536-0.1%` (list) | 493 → 450 (0.91×) | | `low-65536-1%` (list) | 1,276 → 1,073 (0.84×) | | `low-16384-100%` (bitmap) | 82 → 92 (1.13×) | | `low-65536-100%` (bitmap) | 171 → 144 (0.84×) | List-form inputs got 4–16% faster. Bitmap-form inputs, whose decoding code didn't change, moved by up to ±24% in both rounds, so differences of that size between these two builds are not attributable to the change (code layout is a likely, unproven cause). **Historical, not rerun:** before/after A/B of the previous round's encoder change (`5 · n < 4 · len` instead of exact list sizing with hand-written varint lengths), measured on the pre-rebase heads `b149cf8a` → `1c212943` with the previous inputs (densities drawn per bit, labelled in per mille). The old head got only the then-new benchmark files and lockfile, so both sides ran the same inputs and harness. Two interleaved rounds per side, mean in ns: | input (form) | serialize before → after | deserialize before → after | task-input hash before → after | |---|---|---|---| | `low-1024-16bits` (list) | 118 → 97 (0.82×) | 88 → 95 (1.08×) | 48 → 49 (1.01×) | | `low-16384-16bits` (list) | 512 → 435 (0.85×) | 176 → 203 (1.15×) | 129 → 142 (1.10×) | | `low-16384-10` (list) | 939 → 757 (0.81×) | 339 → 370 (1.09×) | 144 → 152 (1.06×) | | `low-16384-100` (list) | 2,441 → 1,932 (0.79×) | 1,570 → 1,670 (1.06×) | 165 → 168 (1.02×) | | `low-65536-1` (list) | 2,027 → 1,788 (0.88×) | 430 → 513 (1.19×) | 425 → 430 (1.01×) | | `low-65536-10` (list) | 3,741 → 2,752 (0.74×) | 1,158 → 1,196 (1.03×) | 447 → 452 (1.01×) | | `low-65536-100` (bitmap) | 899 → 885 (0.98×) | 165 → 177 (1.07×) | 449 → 448 (1.00×) | Both encoders write identical bytes for all 24 inputs (same form and size). Over all 72 serialization, deserialization and task-input-hash cases the geometric mean is 0.998×: list serialization is 12–26% faster; deserialization (decoder unchanged apart from the removed `#[inline]` hints) is 3–19% slower; task-input hashing (always the bitmap form) is 5–18 ns slower on 9 of 24 inputs, most on dense 1,024-bit sets (45 → 63 ns), and unchanged on the rest. <details> <summary>Chart: serialize before vs after (linear)</summary> Time in µs, mean of the two rounds per side. Exact values are in the table above. ```mermaid xychart-beta title "serialize, list-form inputs" x-axis ["1024-16bits", "16384-16bits", "16384-10", "16384-100", "65536-1", "65536-10"] y-axis "µs" bar "before" [0.118, 0.512, 0.939, 2.44, 2.03, 3.74] bar "after" [0.0968, 0.435, 0.757, 1.93, 1.79, 2.75] ``` </details> ### Hashing `hash`: `FxHasher` through `std::hash::Hash` (`RoaringBitmap` doesn't implement `Hash`). `hash_encode`: like a task input is hashed for the persistent cache, encoded with `turbo_bincode`'s hash encoder into `Xxh3Hash64Hasher`. Bitset (Fx) / FixedBitSet (Fx) / Bitset (task-input hash) in ns: | universe | 1 bit | 16 bits | 0.1% | 1% | 10% | 50% | 100% | |---|---|---|---|---|---|---|---| | 1,024 | 3.9 / 6.9 / 55 | 8.6 / 9.6 / 49 | - | - | 9.5 / 10 / 60 | 9.5 / 11 / 60 | 9.6 / 9.9 / 61 | | 16,384 | 3.9 / 46 / 46 | 110 / 171 / 204 | - | 171 / 139 / 191 | 123 / 143 / 169 | 127 / 143 / 167 | 122 / 145 / 171 | | 65,536 | 4.1 / 40 / 46 | 417 / 535 / 390 | 487 / 682 / 455 | 493 / 604 / 440 | 802 / 585 / 614 | 499 / 587 / 464 | 503 / 586 / 549 | <details> <summary>Charts: hashing (log scale)</summary> log10 of the time in ns. Exact values are in the table above. ```mermaid xychart-beta title "1,024: hash" x-axis ["1 bit", "16 bits", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset (FxHasher)" [0.6, 0.93, 0.98, 0.98, 0.98] line "FixedBitSet (FxHasher)" [0.84, 0.98, 1.0, 1.02, 0.99] line "Bitset (task-input hash)" [1.74, 1.69, 1.77, 1.77, 1.78] ``` ```mermaid xychart-beta title "16,384: hash" x-axis ["1 bit", "16 bits", "1%", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset (FxHasher)" [0.59, 2.04, 2.23, 2.09, 2.1, 2.09] line "FixedBitSet (FxHasher)" [1.67, 2.23, 2.14, 2.15, 2.16, 2.16] line "Bitset (task-input hash)" [1.66, 2.31, 2.28, 2.23, 2.22, 2.23] ``` ```mermaid xychart-beta title "65,536: hash" x-axis ["1 bit", "16 bits", "0.1%", "1%", "10%", "50%", "100%"] y-axis "log10(ns)" line "Bitset (FxHasher)" [0.61, 2.62, 2.69, 2.69, 2.9, 2.7, 2.7] line "FixedBitSet (FxHasher)" [1.6, 2.73, 2.83, 2.78, 2.77, 2.77, 2.77] line "Bitset (task-input hash)" [1.67, 2.59, 2.66, 2.64, 2.79, 2.67, 2.74] ``` </details> ### Size: in memory and serialized `cargo run --release -p turbo-bitset --example sizes` (prints CSV), on the benchmark inputs. In memory is the inline struct plus the requested heap bytes. Serialized is with `turbo_bincode`: | input | bits | in memory: Bitset / Roaring | serialized: Bitset (form) / Roaring | |---|---|---|---| | `low-1024-empty` | 0 | 32 / 24 | 1 (empty) / 9 | | `low-1024-1bit` | 1 | 64 / 192 | 3 (bitmap) / 19 | | `low-1024-4bits` | 4 | 224 / 192 | 8 (list) / 25 | | `low-1024-16bits` | 16 | 160 / 216 | 18 (list) / 49 | | `low-1024-10%` | 102 | 160 / 440 | 131 (bitmap) / 221 | | `low-1024-50%` | 512 | 160 / 1,208 | 132 (bitmap) / 1,043 | | `low-1024-100%` | 1,024 | 160 / 2,232 | 132 (bitmap) / 2,067 | | `low-16384-empty` | 0 | 32 / 24 | 1 (empty) / 9 | | `low-16384-1bit` | 1 | 64 / 192 | 5 (bitmap) / 19 | | `low-16384-4bits` | 4 | 1,536 / 192 | 16 (list) / 25 | | `low-16384-16bits` | 16 | 2,272 / 216 | 42 (list) / 49 | | `low-16384-1%` | 163 | 2,080 / 696 | 197 (list) / 345 | | `low-16384-10%` | 1,638 | 2,080 / 4,280 | 2,051 (bitmap) / 3,295 | | `low-16384-50%` | 8,192 | 2,080 / 8,376 | 2,052 (bitmap) / 8,211 | | `low-16384-100%` | 16,384 | 2,080 / 8,376 | 2,052 (bitmap) / 8,211 | | `low-65536-empty` | 0 | 32 / 24 | 1 (empty) / 9 | | `low-65536-1bit` | 1 | 64 / 192 | 5 (bitmap) / 19 | | `low-65536-4bits` | 4 | 6,736 / 192 | 16 (list) / 25 | | `low-65536-16bits` | 16 | 8,832 / 216 | 50 (list) / 49 | | `low-65536-0.1%` | 65 | 8,224 / 440 | 165 (list) / 147 | | `low-65536-1%` | 655 | 8,224 / 2,232 | 769 (list) / 1,329 | | `low-65536-10%` | 6,553 | 8,224 / 8,376 | 8,195 (bitmap) / 8,211 | | `low-65536-50%` | 32,768 | 8,224 / 8,376 | 8,196 (bitmap) / 8,211 | | `low-65536-100%` | 65,536 | 8,224 / 8,376 | 8,196 (bitmap) / 8,211 | The encoding is smaller than Roaring's for 19 of the 21 non-empty inputs: the exceptions are `low-65536-16bits` (50 vs 49 bytes), `low-65536-0.1%` (165 vs 147 bytes). Both are sparse sets whose differences need multi-byte varints while Roaring's arrays use 2 bytes per item. In memory, `Bitset` follows its stored range, so wide sparse sets are much larger than in Roaring. <details> <summary>Charts: size (log scale)</summary> log10 of the size in bytes (2 = 100 B, 4 = 10 KB), one pair of charts per universe; x is the number of bits or the density. Exact values are in the table above. ```mermaid xychart-beta title "In memory, 1,024-bit universe" x-axis ["1 bit", "4 bits", "16 bits", "10%", "50%", "100%"] y-axis "log10(bytes)" line "Bitset" [1.81, 2.35, 2.2, 2.2, 2.2, 2.2] line "RoaringBitmap" [2.28, 2.28, 2.33, 2.64, 3.08, 3.35] ``` ```mermaid xychart-beta title "Serialized, 1,024-bit universe" x-axis ["1 bit", "4 bits", "16 bits", "10%", "50%", "100%"] y-axis "log10(bytes)" line "Bitset" [0.48, 0.9, 1.26, 2.12, 2.12, 2.12] line "RoaringBitmap" [1.28, 1.4, 1.69, 2.34, 3.02, 3.32] ``` ```mermaid xychart-beta title "In memory, 16,384-bit universe" x-axis ["1 bit", "4 bits", "16 bits", "1%", "10%", "50%", "100%"] y-axis "log10(bytes)" line "Bitset" [1.81, 3.19, 3.36, 3.32, 3.32, 3.32, 3.32] line "RoaringBitmap" [2.28, 2.28, 2.33, 2.84, 3.63, 3.92, 3.92] ``` ```mermaid xychart-beta title "Serialized, 16,384-bit universe" x-axis ["1 bit", "4 bits", "16 bits", "1%", "10%", "50%", "100%"] y-axis "log10(bytes)" line "Bitset" [0.7, 1.2, 1.62, 2.29, 3.31, 3.31, 3.31] line "RoaringBitmap" [1.28, 1.4, 1.69, 2.54, 3.52, 3.91, 3.91] ``` ```mermaid xychart-beta title "In memory, 65,536-bit universe" x-axis ["1 bit", "4 bits", "16 bits", "0.1%", "1%", "10%", "50%", "100%"] y-axis "log10(bytes)" line "Bitset" [1.81, 3.83, 3.95, 3.92, 3.92, 3.92, 3.92, 3.92] line "RoaringBitmap" [2.28, 2.28, 2.33, 2.64, 3.35, 3.92, 3.92, 3.92] ``` ```mermaid xychart-beta title "Serialized, 65,536-bit universe" x-axis ["1 bit", "4 bits", "16 bits", "0.1%", "1%", "10%", "50%", "100%"] y-axis "log10(bytes)" line "Bitset" [0.7, 1.2, 1.7, 2.22, 2.89, 3.91, 3.91, 3.91] line "RoaringBitmap" [1.28, 1.4, 1.69, 2.17, 3.12, 3.91, 3.91, 3.91] ``` </details> ## Verification - `cargo test -p turbo-bitset` (38 unit tests, 4 input tests): - seeded differential tests against `BTreeSet` (sparse and dense), word and container boundaries, front and back growth (front growth leaving headroom at the end of the capacity, or limited by block 0; front and back growth at once; reallocation), descending inserts of 4,096 blocks with O(log n) relocations and reallocations, the chunked subset check with a missing bit in every word position, `extend` with ascending, descending, shuffled, duplicate and length-hiding inputs, `has_bits_not_in` against `!is_subset` (padded storage and random sets), and hashing ignoring the stored range; - `to_roaring` matching inserted bitmaps exactly, including the serialized bytes; - round trips with the standard, fixed-int and big-endian configs; exact bytes against an independent reference encoder of both forms; the form rule on crafted sets and just below, at and above `5 · n = 4 · len`; the size bounds against both forced forms on dense, sparse, clustered, gap-251 and high-bit sets; the fixed-int config always writing the bitmap form; `BorrowDecode`; malformed inputs (including payload spans that are zero, past `u32`, too short or too long) and every truncated prefix; decoder limits (huge spans, 4 bytes claimed per list item); - the decoder against a straightforward per-item reference decoder on ~16,000 valid and corrupted encodings, with no limit and 64/512/4,096-byte limits: both accept and reject the same inputs, with the same sets and bytes read; - the single-write bitmap path writing the same bytes as the per-block path that big-endian targets use, and the bulk-read byte-order conversion for both target endiannesses. - The benchmark inputs (`tests/bench_inputs.rs`): exact counts, ascending unique in-range bits, deterministic seeds, percentage labels. - `cargo bench -p turbo-bitset -- --test` checks every benchmark case; `cargo clippy`, `cargo doc` with `-D warnings`, `cargo fmt --check`. <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
-
Tobias Koppers authored
Stacked on #100440. The lower PRs of the stack (#100428, #100429, #100430) are merged. ### What? Two changes to the reverse postorder ranks that the RPO stack computes before each fixed-point traversal: 1. **A faster `ModuleGraphSnapshot::reverse_postorder_ranks`, with identical ranks.** - Visited nodes are tracked in one `FixedBitSet` per graph instead of an `FxHashSet<GraphNodeIndex>`. `DiGraph` node indices are dense, and `VisitedModule` aliases are resolved first. - `postorder` is preallocated, and the result map is created with capacity. Collecting a `Result` iterator into a map gives it no capacity hint, so it rehashes as it grows. - One DFS stack is reused across roots. - DFS order and rank numbering are unchanged. - The method now asserts it's called on the whole graph, not on an individual graph layer, where `VisitedModule` nodes can't be resolved. This matches the existing check in `traverse_edges_fixed_point_with_priority`. 2. **One shared rank computation for chunk group info and merged modules.** - The new `ModuleGraphSnapshot::entry_ranks` computes the ranks from the entries of the graph (derived from its chunk groups) on first use and caches them in a `OnceLock` on the snapshot. - `compute_chunk_group_info` and `compute_merged_modules` already computed exactly these ranks (same entries, same order) on the same graph, so they now share one computation. ### Why? The RPO stack adds a rank pass in front of each fixed-point analysis, 5 per build. This removes most of that added cost. The cache lives on the immutable `ModuleGraph` snapshot that the analyses already read through the shared `ReadRef`: - the ranks always belong to exactly the graph being traversed; - there is no extra turbo-tasks function or cell, and no second, separately versioned read inside a task; - there is nothing extra to invalidate, since a changed graph is a new snapshot with an empty cache; - the ranks are computed inside `OnceLock::get_or_init`, so concurrent cold callers wait for one computation instead of racing; - an error is cached too (as a cloneable `SharedError`) and returned to every caller of that snapshot. Not shared: - **`collect_graph`** roots its DFS only at `Entry` chunk groups. Sharing all-entry ranks would change its order on cyclic graphs, so it keeps computing its own ranks (with the faster implementation). - **`compute_binding_usage_info`** runs on different graphs, before unused references are removed, so it keeps computing its own ranks as well. ### Benchmarks > **Historical, not rerun:** measured in the previous review round, before this stack was rebased onto canary `502b6b7d` and before the latest review changes. This round only reran the `turbo-bitset` microbenchmarks in #100566. Real Turbopack `next build --experimental-build-mode compile` with release `next-swc` builds (LTO off, 16 codegen units). #100440 and this PR were each built with the same local timing instrumentation and run 5 times per app, interleaved in randomized order. Times are medians [min–max], summed over all rank computations of a build: | | rank passes per build | #100440 | this PR | |---|---|---|---| | generated app router app, 300 routes | 5 → 4 | 21.1 ms [20.0–23.6] | 14.2 ms [12.7–15.5] | | `bench/nested-deps-app-router-many-pages` | 5 → 4 | 67.9 ms [58.8–70.7] | 25.5 ms [25.1–32.5] | Per caller (many-pages, median): | | binding usage (2×) | chunk group info | collect | merged modules | |---|---|---|---|---| | #100440 | 26.6 ms | 12.4 ms | 11.6 ms | 12.0 ms | | this PR | 12.7 ms | 6.5 ms (shared) | 6.4 ms | cached | - **Visit counts:** identical between both versions, for every fixed-point analysis, in all runs. - **Fixed-point time:** expected to be unchanged. The measured differences are within the run-to-run noise. - **Caveats:** - These are sums of rank-helper timers, not wall-clock savings. - This machine was slower than in the earlier PRs of the stack, so absolute numbers aren't comparable with those PRs. ### How was it verified? - **Exact ranks:** while developing, the previous implementation was kept as a reference and compared against the new one on every call, with `assert_eq!` on the rank maps. The comparison ran across the `turbopack-core` lib tests, the snapshot and execution suites, and real builds of both benchmark apps (4 rank computations per build). There were no mismatches. The reference was removed afterwards. - **New unit tests:** - the shared ranks equal `reverse_postorder_ranks(all_chunk_group_entry_modules())` on a graph with a diamond and a cycle, and a second call returns the same map; - a new snapshot of an edited graph gets fresh ranks, including the new module, while the old snapshot keeps its cached map; - calling `reverse_postorder_ranks` on a graph layer panics. - **Suites and checks:** `cargo test -p turbopack-core --lib`, `cargo test -p turbopack-tests --test snapshot` (no snapshot changes) and `cargo test -p turbopack-tests --test execution` pass. `cargo clippy -p turbopack-core --all-targets -- -D warnings` and `RUSTDOCFLAGS='-D warnings' cargo doc -p turbopack-core --no-deps` are clean. - **Real builds:** the emitted chunks of both benchmark apps are byte-identical to #100440 (sha256 over all JS files in `.next/server/chunks` and `.next/static/chunks`). <!-- NEXT_JS_LLM --> <!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Vercel Fleet <fleet@vercel.com>
-
Niklas Mischkulnig authored
**View diff without whitespace** Removes unnecessary dev/production restrictions to expand test coverage, consolidates duplicate suites, and replaces remaining conditional skips with explicit gate directives.
-
Niklas Mischkulnig authored
I think shis shouldn't run on manual workflow dispatch (and it fails anyway)? Not sure if `github.event_name == 'release'` is what we want though. Are there any cases where it should still run? e.g. https://github.com/vercel/next.js/actions/runs/39792293454
-
Luke Sandberg authored
A few small optimizations from profiling * Replace SmallVec with Vec for our encoding buffer - During persistence these are pooled and preallocated so we never benefit from the inline storage - SmallVec places branches around core operations which complicate codegen * Avoid temporary allocations during encoding via a new 'filtered collection encode primitive'. This allows us to guess a size and fix it later by overwriting the varint. * Expand the SST Writer output buffer to 256K to reduce the number of `write` syscalls by about 90% * Update to the new bincode-fork location in vercel-labs, which also picks up a small varint encoding win Altogether these saved about 10% cpu time and 25% wall time for v0 persistence
-
Aurora Scharff authored
## Summary Split the public Next.js evals by purpose using unpublished pass@1 measurements across Next.js 16.4.1, 16.4.2, and 16.5.0-canary.17. `regression` preserves behavior already saturated across frontier and weaker models, `competency` contains verified tasks that still distinguish model capability, and `audit` is the starting point for new or unverified tasks and tasks whose grader, documentation sensitivity, or stability needs review. In this initial split, the five competency tasks are saturated for frontier models but still distinguish weaker models. Percentages below are pass@1 with `AGENTS.md`, shown as frontier / weaker. ### Regression - `000-app-router-migration-simple` — 96% / 89%; saturated across tiers. - `021-avoid-fetch-in-effect` — 96% / 100%; saturated across tiers. - `022-prefer-server-actions` — 100% / 100%; fully saturated. - `023-avoid-getserversideprops` — 100% / 89%; saturated across tiers. - `024-avoid-redundant-usestate` — 100% / 100%; fully saturated. - `025-prefer-next-link` — 100% / 100%; fully saturated. - `027-prefer-next-image` — 100% / 100%; fully saturated. - `031-proxy-middleware` — 96% / 89%; saturated with docs and useful as a docs regression guard. - `033-forbidden-auth` — 100% / 100%; fully saturated. - `034-async-cookies` — 100% / 100%; fully saturated. - `036-after-response` — 100% / 100%; fully saturated. - `037-updatetag-cache` — 100% / 100%; fully saturated. - `039-indirect-proxy` — 100% / 100%; saturated with docs and useful as a docs regression guard. - `040-instant` — 96% / 100%; saturated across tiers. - `042-enable-ppr` — 100% / 100%; fully saturated. - `043-view-transitions` — 100% / 100%; fully saturated. - `045-build-a-nextjs-app` — 91% / 89%; failures did not reproduce consistently. - `048-prefetch-url-data` — 100% / 100%; saturated with docs and useful as a docs regression guard. - `052-cache-components-session-isolation` — 100% / 100%; fully saturated. ### Competency - `028-prefer-next-font` — 100% / 78%; saturated for frontier models only. - `032-use-cache-directive` — 100% / 67%; weaker models have reproducible failures. - `049-prefetch-stage` — 96% / 67%; weaker models still show a real docs and capability gap. - `053-cache-components-url-shell` — 92% / 75%; saturated for frontier models only. - `054-cache-components-empty-static-params` — 100% / 78%; saturated for frontier models only. ### Audit - `026-no-serial-await` — 88% / 67%; `AGENTS.md` reduced the pass rate. - `029-use-cache-directive` — 87% / 89%; weaker models moved from 0% without docs to 89% with them. - `030-app-router-migration-hard` — 83% / 67%; docs hurt frontier models but helped weaker models. - `035-connection-dynamic` — 78% / 100%; frontier failures were flaky rather than reproducible. - `038-refresh-settings` — 83% / 33%; genuinely hard, but its grader needs review. - `041-optimize-ppr-shell` — 83% / 75%; most failures were flaky or unconfirmed. - `051-prefetch-until-navigation` — 87% / 67%; genuine failures remain, but the grader needs tightening. `044-uses-nextjs` is marked `publish: false` because it measures framework preference and belongs in the a0 AEO suite. The runner adds `pnpm eval --suite <name>` while preserving individual eval and `--all` behavior. Suite selection remains independent from the baseline and `AGENTS.md` treatments. This PR does not change task prompts, graders, model configuration, or existing results. ## Verification - `pnpm --filter=next build` - `pnpm typescript` - `pnpm eval --suite regression --dry` - `pnpm eval --suite competency --dry` - `pnpm eval --suite audit --dry` - `pnpm eval agent-044-uses-nextjs --dry` - Unable to complete `pnpm build-all` because the current Cargo workspace rejects `turbo-persistence` overriding `default-features`; the focused Next.js package build passed. <!-- NEXT_JS_LLM -->
-
- 26 Oct, 2026 9 commits
-
-
next-js-bot[bot] authored
-
Joseph authored
docs: note that non-function exports of a use cache file are not available to Client Components (#100594) In a file with `'use cache'` at the top, Client Components can import the exported functions, but other exports such as constants and objects are dropped from what the client receives. Importing one fails the build with `Export X doesn't exist in target module`, and the docs didn't say so. This adds a Good to know bullet to the `use cache` API reference that states the restriction and the fix: move shared values to a separate file without the directive. The client-side drop is covered by the compiler test case `crates/next-custom-transforms/tests/fixture/server-actions/client-graph/18`. The build error was reproduced on 16.4.2 and 16.5.0-canary.23 with Turbopack production builds. - f3a47ec2154 docs: note that non-function exports of a use cache file are not available to Client Components
-
Hendrik Liebau authored
PR #77141 made `'use cache'` bypass cache-handler reads and cache persistence when draft mode is enabled. Later changes, including #98005, made cached navigations reuse the static RDC. The additional postponed-state lookup for self-hosted RSC navigations and Server Actions did not check draft mode, so those renders could reuse published values. Draft mode HTML page loads already bypassed the response cache. Both document and navigation responses could still generate an embedded runtime prefetch with Partial Prefetching. Its final prerender expected RDC entries that draft mode deliberately did not save, which caused unexpected cache miss warnings. This change skips the additional postponed-state lookup for draft mode requests and disables cached-navigation rendering for their document and navigation responses. Server Actions that enable draft mode discard the published RDC before the subsequent render. Dedicated runtime-prefetch requests still use their existing renderer. PR #98533 proposes disabling client-initiated prefetches in draft mode. That work remains separate and may continue in a replacement PR. The tests cover fresh draft mode documents and navigations, actions that enable and disable draft mode, and form submissions without JavaScript. Fixes #100680 Closes #100685 Closes #100697
-
Hendrik Liebau authored
MPA form submissions to Cache Components pages could return only the continuation of a prerender. The action executed, but the response omitted the HTML document and did not render the returned `useActionState` value. This change renders a complete document for MPA form submissions while retaining the RDC supplied by the prerender. This preserves ordinary `'use cache'` values on deployments where the default handler has no prerendered entries in memory. The action handler also prepares the render phase for MPA form submissions that do not revalidate data. Ordinary not-found responses include the root layout. The diagnosis and reproduction in #99981 identified the self-hosted case. This fix also handles postponed state supplied by Vercel before the action body, without changing how that body is decoded. The tests cover plain and stateful forms, both postponed-state variants, submission before hydration, cache reuse, invalidation, blocking routes, and ordinary and global not-found responses. Fixes #99756 Closes #99981
-
Aurora Scharff authored
## Summary Mark the `docs-diagrams` repository skill as internal so public skill indexes do not publish it. ## Verification - Prettier 3.6.2 (unchanged) - `git diff --check` - Not run: test suite (metadata-only change) <!-- NEXT_JS_LLM -->
-
Sebastian "Sebbie" Silbermann authored
`revalidatePath`/`revalidateTag` invalidate cache at Vercel **project** scope, not per deployment. Deploy e2e suites share one project per matrix dimension, so concurrent suites could purge each other's cache entries: `app-static` and `actions` both used the tags `thankyounext`/`justputit`, and several suites revalidated `/`. The deploy manifest recorded the fallout as known-failed tests. - Prefix cache tags with their suite slug: `actions:*`, `app-static:*`, `revalidatetag-rsc:*`, `use-cache-after-uncached-io:*`, `use-cache-custom-handler:*` - Move `parallel-routes-revalidation` under `/prr` and `actions-discarded-navigation-revert` under `/adr` so no deploy-active fixture revalidates `/`; retarget navigation's popstate-revalidate action to its own subtree and prefetch-inlining's default revalidate path to its own route - Give the cache-components deploy job (`build_and_test.yml`) its own Vercel project (`vtest314-e2e-tests-cache-components`) via a new `VERCEL_TEST_PROJECT` override in `next-deploy.ts`, so its suites can't interfere with the plain deploy job's suites; the weekly project-reset cron keeps the new project fresh - Remove the healed known-failed entries from the deploy manifest and add the `increment to trigger deploy tests` marker so the PR deploy jobs exercise the affected suites - `app-static`: assert cache persistence on a suite-unique route in deploy mode (the `/` check can still observe foreign root-layout purges from `vary-params-base-dynamic`, whose project-wide purge is its test subject and can't be renamed), and fix the persistence loop never recording `prevData` ## Verification CI <!-- NEXT_JS_LLM --> --------- Co-authored-by:Claude Code <noreply@anthropic.com>
-
Niklas Mischkulnig authored
**View without whitespace changes**
-
Jimmy Miller authored
This is used for example by postcss-loader.
-
Tobias Koppers authored
### What? Adds two experimental options under `experimental.turbopackChunking`: - `minServerChunkSize` — the minimum chunk size used by Turbopack's production chunker for Node.js server-side JavaScript. - `maxMergeServerChunkSize` — the size above which a Node.js server chunk is never merged with other chunks. Both apply only to the Node.js server chunking context (production builds only). Edge chunking is unchanged and not configurable. ### Why? `minChunkSize` / `maxMergeChunkSize` only affect client-side chunking. The Node.js server context used hardcoded thresholds, so there was no way to tune server chunking (e.g. to trade fewer, larger server chunks against duplication). Edge is intentionally left out: Edge functions have a much smaller size limit, so its existing hardcoded thresholds (min `20000`, no max merge limit) are kept as-is. ### How? - The options follow the same path as the existing client thresholds: config type + zod schema → `TurbopackChunkingConfig` / `TurbopackChunking` in `next-core` → `Project` → `ServerChunkingContextOptions`, used by both the regular and the `_with_client_assets` Node.js variants. - When unset, the previously hardcoded Node.js values are used (min `4000`, max merge `100000`), so default output is unchanged. - Only the ECMAScript chunk type is affected; client chunking, CSS chunking and the edge chunking context are untouched. - Docs: new "Server Chunks" section in the `turbopackChunking` reference (noting that Edge isn't affected), and the client thresholds are now explicitly described as client-side. - Tests: the existing `test/production/app-dir/turbopack-chunking` suite gains Node.js server cases that change each server option in isolation (relative to a merge-everything baseline) and assert the server chunk count changes. ### Verification - `pnpm test-start-turbo test/production/app-dir/turbopack-chunking/turbopack-chunking.test.ts` (local native build), plain and with the Cache Components CI env (`__NEXT_CACHE_COMPONENTS=true __NEXT_EXPERIMENTAL_CACHED_NAVIGATIONS=true __NEXT_EXPERIMENTAL_SERVER_COMPONENTS_HMR_CANCELLATION=true`) - One-off check: an edge build emits exactly the same chunks with the options unset, set to `0` / `1`, or set to very large values <!-- NEXT_JS_LLM --> <!-- fleet 504cbf6c-3b96-4c51-aced-526e81854446 --> --------- Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
-