1. 12 Aug, 2025 17 commits
  2. 11 Aug, 2025 22 commits
    • Jude Gao's avatar
      Fix: Disambiguate Mediapartners-Google user agent (#82536) · 4dc34109
      Jude Gao authored
      `Mediapartners-Google` used to match both `HEADLESS_BROWSER_BOT_UA_RE`
      and `HTML_LIMITED_BOT_UA_RE`, causing inconsistent state values for
      `getBotType`, `isDomBotUA`, and `isHtmlLimitedBotUA`.
      4dc34109
    • Benjamin Woodruff's avatar
      Turbopack: Rework watcher data structures, store watched list and watcher behind… · 0ad13118
      Benjamin Woodruff authored
      Turbopack: Rework watcher data structures, store watched list and watcher behind the same RwLock (#82258)
      
      Reworks the data structures used for the watcher to achieve a few goals:
      
      - I want to avoid multiple locks, as I don't think we actually need two different locks, and multiple locks leaves more room for bugs (e.g. lock inversion).
      - The information about if we're in non-recursive mode or recursive mode needs to be available outside of the lock for performance reasons.
      - The `watching`/`watched` set needs to be ordered. We don't have a convenient concurrent BTreeSet or Trie/Radix Tree pulled in, and I'd like to avoid adding more dependencies. We end up locking `notify_watcher` anywhere we'd need a write lock anyways, so it might block a few reads, but otherwise it's not that bad.
      
      Fixes in this PR:
      
      - Fixes an issue where multiple concurrent calls to `ensure_watching` for the same directory could cause one of those calls to exit early. This may cause a file in that directory to be read before the filesystem watcher is set up, potentially missing events.
      - Improves the time complexity of `restore_if_watching` using an ordered set to avoid iterating over all of `watched` to find child directories. See `OrderedPathSetExt`. This time complexity problem is introduced in the previous PR, #82130.
      
      Other changes:
      
      - It turns out that we don't need to explicitly set up any watchers during `start_watching` except for the `root_dir`. If paths have already been read, they'll set up watchers before re-reading the file when we invalidate it! This simplifies things, and means there's less stuff we need to track if you never start the watcher (i.e. `next build`).
      - Added a `--start-watching-late` option to the fuzzer to test calling `start_watching` after all the initial reads happen, to help validate the safety of the claim in the previous bullet point.
      
      Minor (aesthetic) changes:
      
      - Use past tense: `watching` is now `watched` and `ensure_watching` is now `ensure_watched`, as the `watched` list includes both currently active paths *and* previously watched but deleted paths.
      - Shorten the names of non-public types, because we have a lot more of them, and the names would get long otherwise
      
      ## Benchmarking
      
      This replaces a `DashSet<_>` with (essentially) a `RwLock<BTreeSet<_>>`. There's some risk of increased contention, particularly in the read-only fast-path for `ensure_watched`/`ensure_watching`.
      
      Under the assumption that the most heavily lock-contended use-case for this code is after restoring from persistent cache on a large repository:
      
      1. Check out `vercel-site` in `front`.
      2. Hack the new tracing span onto the canary version.
      3. Build a release version with `pnpm pack-next --project ~/front/apps/vercel-site --no-js-build -- --release`
      4. `rm -rf .next`
      5. `NEXT_TURBOPACK_TRACING='turbo_tasks_fs' TURBO_ENGINE_IGNORE_DIRTY=1 pnpm next dev --turbopack`
      6. Load `http://localhost:3000/vercel`. Wait for it to fully finish loading. Stop the server with `ctrl+c`.
      7. Restart the server, wait for it to finish initializing, and then exit.
      8. Start the trace viewer: `pnpm next internal trace .next/trace-turbopack`, and look at the aggregated times of `ensure_watched`/`ensure_watching`.
      
      Results:
      
      before (canary): 313ms ensure_watching (2235x spans)
      after (this pr): 279ms ensure_watched (2235x spans)
      
      So no significant difference.
      0ad13118
    • nextjs-bot's avatar
      v15.4.2-canary.38 · 78c544b1
      nextjs-bot authored
      78c544b1
    • Joseph's avatar
      docs: experimental config browserDebugInfoInTerminal (#82532) · d75d90f3
      Joseph authored
      Adding docs for `browserDebugInfoInTerminal`
      d75d90f3
    • Niklas Mischkulnig's avatar
      Test that there are no server-side CSS chunks (#82471) · 161a8ba0
      Niklas Mischkulnig authored
      A few months ago, this was fixed  for Turbopack. Adding a regression test now
      161a8ba0
    • nextjs-bot's avatar
      v15.4.2-canary.37 · d2512647
      nextjs-bot authored
      d2512647
    • Vercel Release Bot's avatar
      Update Rspack production test manifest (#82525) · c40ce7e0
      Vercel Release Bot authored
      This auto-generated PR updates the production integration test manifest used when testing Rspack.
      c40ce7e0
    • Vercel Release Bot's avatar
      Update Rspack development test manifest (#82526) · c3c0b78f
      Vercel Release Bot authored
      This auto-generated PR updates the development integration test manifest used when testing Rspack.
      c3c0b78f
    • Steven's avatar
      fix: add `?dpl` to fonts in `/_next/static/media` part 2 (#82488) · e822c809
      Steven authored
      This is a follow up to PR https://github.com/vercel/next.js/pull/82384
      which fixed it for `<link>` but not for `next/font` that embeds local
      fonts in css.
      
      
      https://nextjs.org/docs/14/app/building-your-application/deploying#version-skew
      e822c809
    • Steven's avatar
      fix(next/image): use `image-size` pkg as additional format detector (#82538) · ee8286f1
      Steven authored
      In a previous PR https://github.com/vercel/next.js/pull/82118 we added a
      fallback to `sharp().metadata()` when the magic number didn't detect the
      image format in order to improve detection.
      
      But `sharp` can be slow to read metadata, so we should avoid it if we
      can.
      
      So this PR uses the existing `image-size` package to detect the content
      type in JS and only then does it fallback to the native `sharp` code if
      necessary.
      
      No tests were added since this doesn't change the behavior, only
      performance.
      ee8286f1
    • Benjamin Woodruff's avatar
      chore: Ignore `compiled` folder in ripgrep/ast-grep/ag/etc (#82482) · 2c1a3d15
      Benjamin Woodruff authored
      These files need to be committed (why they're not in `.gitignore`), but I'm tired of `ripgrep` capturing these generated files.
      2c1a3d15
    • nextjs-bot's avatar
      v15.4.2-canary.36 · b1c93c72
      nextjs-bot authored
      b1c93c72
    • nextjs-bot's avatar
    • Niklas Mischkulnig's avatar
      Turbopack: respect `productionBrowserSourceMaps` (#78014) · f23f9354
      Niklas Mischkulnig authored
      Let's try this again
      
      ~~Blocked by https://github.com/vercel/next.js/pull/78058~~
      Blocked by https://github.com/vercel/next.js/pull/82448
      
      - [x] Add regression test
      - [x] Update failing assertions
      
      Closes PACK-4304
      f23f9354
    • Niklas Mischkulnig's avatar
      Turbopack: fix CSS module references, take 2 (#82448) · 0ec5511d
      Niklas Mischkulnig authored
      Replaces and closes https://github.com/vercel/next.js/pull/78058
      Closes PACK-4324
      
      Before, you got duplicate modules because
      `transition_to_client(postcss(webpack(source in rsc)) != postcss(webpack(transition_to_client(source in rsc))`
      
      Now, we have:
      ```
      rsc layer
      --ESM import->
      CSSModuleAsset(in rsc, contains raw SCSS source)
      --CssReferenceSubType::Inner->
      CssAsset(first in client layer, then processed with Webpack loader)
      ```
      
      This aligns it with when you import CSS from a client component, in both cases the CSS module is first transitioned into the client layer, and then processed with Webpack
      0ec5511d
    • Benjamin Woodruff's avatar
      Turbopack: Restore watchers for children when using non-recursive FS watcher (#82130) · 13899e5f
      Benjamin Woodruff authored
      **This should be merged alongside #82258, as this introduces a potentially high time complexity operation that #82258 solves.**
      
      With this PR and the previous one, my fuzzer shows that we capture every (currently tested) file change correctly:
      
      ```
      rm -rf /tmp/fuzz && cargo run --release -p turbo-tasks-fuzz -- fs-watcher --fs-root /tmp/fuzz
      ```
      
      However, this does add an `O(n)` iteration of the `watching` map that I'm working on cleaning up (why this is still a draft).
      
      I'll do something similar to https://github.com/vercel/next.js/pull/82133, but it's a bit more complicated to switch to a `BTreeSet` because it's currently using a `DashSet`, and I don't want to introduce more locking than I have to.
      13899e5f
    • nextjs-bot's avatar
    • Luke Sandberg's avatar
      [turbopack] Fix the build under `--features=tokio_tracing` (#81979) · 5f6cb1c6
      Luke Sandberg authored
      ## Fix the build under the `tokio_tracing` feature
      
      https://github.com/vercel/next.js/pull/80738 changed the definition of LocalTaskSpec and moved the Display trait impl down to LocalTaskType but updating this use was missed since it is guarded by a feature.
      5f6cb1c6
    • Luke Sandberg's avatar
      [turbopack] Promote `exports` and module to factory parameters for cjs (#82285) · e9a5067e
      Luke Sandberg authored
      ## Optimze cjs codegen by promoting `module` and `exports` to factory parameters
      
      *tl;dr; destructing is hard, lets go shopping!*
      
      Currently we prefix factories for cjs modules with
      `var {m:module, e:exports} = __turbopack_context__;`
      
      This is problematic since swc is unable to eliminate these bindings when they are dead (too afraid of es5 getter properties and side effects 😭).  Furthermore we are playing a game of telephone with ourselves since our caller basically is doing
      
      `factory({m:module, e: module.exports})` when they call us (this is skimming over a bunch of details, ofc).
      
      Instead just pass them directly as function parameters.  One downside of this is that we still pass them to `esm` factories even though they are ignored (and simply not declared by the factories).  This is not too costly since the caller has trivial access already and browsers are pretty good at handling/ignoring extra parameters (see https://v8.dev/blog/adaptor-frame for the clever way v8 optimizes this).
      
      ### Alternatives
      
      The main alternative considered was compile time replacing references to `module` and `exports` with `__turbopack_context__.(m|e)`. Unfortunately this breaks subtle edge cases of the cjs spec since it is allowed for users to rewrite `module` and `exports` with assignments  (e.g. `exports = 'bad idea'`).  This is generally a linter error but not unheard of and we cannot break it.
      
      The other alternative considered was enhancing `swc` to inline and dead code eliminate destructuring patterns, but from playing with the playgound it looks like swc is pretty conservative about moving/inlining variables across side effectful statements, so we would still be paying the cost of the declaration in many cases.
      
      ## Turbopack Performance Comparison: Head vs Change
      
      As expected this is a substantial improvement to commonjs evaluation. The ESM results are kind of confusing.  I generally expected 'neutral' since the cost of passing a dead parameters is so small in modern vms.
      
      | Test Case | Metric | Head (ms) | Change (ms) | Speedup | Improvement |
      |-----------|--------|-----------|-------------|---------|-------------|
      | **Client CommonJS** | Load - Average | 31.61 | 24.38 | **1.30x** | **+29.6%** |
      | | Load - Median | 32.05 | 24.30 | **1.32x** | **+31.9%** |
      | | Load - P75 | 33.10 | 24.90 | **1.33x** | **+32.9%** |
      | | Execute - Average | 25.96 | 19.80 | **1.31x** | **+31.1%** |
      | | Execute - Median | 25.90 | 19.50 | **1.33x** | **+32.8%** |
      | | Execute - P75 | 26.50 | 20.20 | **1.31x** | **+31.2%** |
      | **Client ESM** | Load - Average | 22.28 | 21.94 | **1.02x** | **+1.6%** |
      | | Load - Median | 21.80 | 21.60 | **1.01x** | **+0.9%** |
      | | Load - P75 | 23.50 | 22.10 | **1.06x** | **+6.3%** |
      | | Execute - Average | 50.77 | 48.12 | **1.06x** | **+5.5%** |
      | | Execute - Median | 50.85 | 47.55 | **1.07x** | **+6.9%** |
      | | Execute - P75 | 51.60 | 48.30 | **1.07x** | **+6.8%** |
      | **Pages API CommonJS** | Load - Average | 24.18 | 23.33 | **1.04x** | **+3.7%** |
      | | Load - Median | 23.27 | 23.57 | 0.99x | -1.3% |
      | | Load - P75 | 25.29 | 23.96 | **1.06x** | **+5.5%** |
      | | Execute - Average | 32.23 | 24.26 | **1.33x** | **+32.8%** |
      | | Execute - Median | 31.16 | 24.34 | **1.28x** | **+28.0%** |
      | | Execute - P75 | 33.42 | 24.52 | **1.36x** | **+36.3%** |
      | **Pages API ESM** | Load - Average | 23.48 | 28.91 | 0.81x | -18.8% |
      | | Load - Median | 23.13 | 28.91 | 0.80x | -20.0% |
      | | Load - P75 | 25.55 | 29.57 | 0.86x | -13.6% |
      | | Execute - Average | 63.01 | 55.94 | **1.13x** | **+12.6%** |
      | | Execute - Median | 61.56 | 55.22 | **1.11x** | **+11.5%** |
      | | Execute - P75 | 69.56 | 56.40 | **1.23x** | **+23.3%** |
      
      **Legend:**
      - **Bold values** indicate performance improvements (speedups)
      - Regular values indicate performance regressions (slowdowns)
      - Speedup = Head Time ÷ Change Time (>1.0 = faster, <1.0 = slower)
      - P75 = 75th percentile
      e9a5067e
    • Luke Sandberg's avatar
      [turbopack] Fix a small issue in the analyzer where we wouldn't skip assignments… · 74a0aa2f
      Luke Sandberg authored
      [turbopack] Fix a small issue in the analyzer where we wouldn't skip assignments to free vars that were just identifiers (#82392)
      
      Correctly skip assignments when replacing free variables
      
      Drop an `Option<..>` for a parameter that is always supplied.
      
      These are fixes i tripped over when attempting to compile time replace `module` and `exports`. That approach failed (see #82285 for what happened instead), but we can keep these cleanups.
      74a0aa2f
    • Tobias Koppers's avatar
      Turbopack: increase time before persisting (#82403) · 2c3e9d83
      Tobias Koppers authored
      ### What?
      
      Decrease the frequency of persisting since this is very expensive.
      2c3e9d83
    • Andrew Clark's avatar
      Delete abandoned segment parsing code (#82458) · 1af1a28a
      Andrew Clark authored
      Originally, as part of the clientParamParsing feature, I expected that I
      would omit param values from the FlightRouterState when sending data
      from the server, and add it on the client.
      
      However, I've since realized this is unnecessary because
      FlightRouterState is only used during runtime server responses. Not in
      static responses, like per-segment prefetches.
      
      Since the goal of clientParamParsing is to omit param values from static
      responses, we don't need to change anything about FlightRouterState.
      
      We may still want to update how params are processed on the server to
      more closely match the implementation on the client, but that still
      doesn't require any changes to FlightRouterState.
      
      So, this deletes all the code related to adding client-parsed params to
      FlightRouterState on the client. None of this was being used anywhere;
      it was added by previous PRs in anticipation that it would eventually be
      used. But now we don't need it.
      1af1a28a
  3. 09 Aug, 2025 1 commit