Write webpack dev manifests atomically (#100112)
### What? In webpack dev, manifest files (`*manifest.json` / `*manifest.js`, e.g. `next-font-manifest.json` and `page_client-reference-manifest.js`) are now written atomically: the content goes to a temp file next to the manifest, which is then renamed over it. Other emitted assets are written as before. ### Why? This fixes a canary hard flake in `test/e2e/app-dir/not-found/basic` › "should render the 404 page when the file is removed, and restore the page when re-added" ([example](https://github.com/vercel/next.js/actions/runs/38003705301/job/114068060547)). It is a real dev-server bug: - Renaming `app/page.js` away makes `/` render `/_not-found`. That entry is already built, so the render doesn't wait for the in-flight rebuild. - Webpack writes emitted assets in place (it truncates first). `RouteModule.loadManifests` can therefore read a 0-byte manifest. Instrumented local runs caught `.next/dev/server/app/_not-found/page_client-reference-manifest.js` (`Manifest file is empty`) and `.next/dev/server/next-font-manifest.json` (`Unexpected end of JSON input`). CI shows the same errors on every retry. - The request returns a 500, the browser lands on the pages error page, which never recovers through app-dir HMR, and the test times out after 120 s. Turbopack already writes its manifests with `writeFileAtomic` and isn't affected. ### How? - `createAtomicManifestOutputFileSystem(fs)` returns an output FS that inherits from webpack's shared `graceful-fs` object without mutating it. Manifest writes go through the existing sync `writeFileAtomic` helper (`lib/fs/write-atomic.ts`, now accepting `Buffer`). That helper is the same one Turbopack's manifests use, and it already handles temp naming, cleanup and the Windows-tolerant `renameSync`. All other writes and methods go to the original FS. - One wrapped FS is created next to the shared `inputFileSystem` and assigned to every compiler in the multi-compiler. The fallback compiler is wrapped too. - This assumes the disk-backed dev output that Next.js uses. The dev server reads these manifests from disk with Node `fs` anyway. ### Verification Full test file, `pnpm test-dev-webpack`, CI env (`NEXT_TEST_REACT_VERSION=18.3.1`, `__NEXT_EXPERIMENTAL_STRICT_ROUTE_TYPES=true`): | | Runs | Manifest read errors | Failures | |---|---|---|---| | canary | 8 | 5 | 5 | | this PR | 10 | 0 | 2* | \* Both remaining failures happened at the *restore* step: an HMR refresh arrived before the app router initialized. That's a separate root cause, fixed independently in #100113, and with both PRs applied the file passed 10/10. - `pnpm test-dev-turbo`: 3/3 passed. `pnpm test-start-webpack`: passed. - After the switch to `writeFileAtomic`, with #100113's fix applied: 5/5 full-file `pnpm test-dev-webpack` runs passed in the same CI env, with no manifest read errors. - Unit tests run against a real temp dir. They cover Buffer bytes being preserved and the target keeping its complete old content at the rename boundary (then the complete new one). They also cover errors reaching the callback exactly once, no leftover temp files, delegation of non-manifest writes, and the original FS not being mutated. <!-- NEXT_JS_LLM --> <!-- fleet e9d6c475-aeff-42d9-a058-210b6631f951 --> --------- Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Showing
Please
register
or
sign in
to comment