Unverified Commit eab3ab87 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Add `next internal static-routes-info` CLI command (#93399)

### What?

Adds a new internal CLI subcommand:

```
next internal static-routes-info [directory] [options]
```

It runs against an already-built Next.js app (`.next/` from `next
build`), reads the manifests under `distDir`, and reports per-route
bundle sizes split into six file categories. Supports markdown (default)
or JSON output, sorting, limiting, and per-category file listings.

### Why?

We want a **static, build-output-based** way to compare different
chunking strategies for JS and CSS without running the user's app.
Existing tooling either requires running the app
(`@next/bundle-analyzer` is a webpack plugin), is bundler-specific, or
aggregates per-bundle/per-asset rather than per-route. This tool answers
the concrete question "how much JS / CSS does each route ship today?"
purely by reading static manifests, so it can be diffed across builds,
branches, or bundlers (Turbopack vs webpack) and used to evaluate
chunking changes.

It is namespaced under `next internal` because the output format and
category boundaries are tied to internal manifest shapes; we don't
intend to make it a stable public surface yet.

### How?

#### Command surface

```
Usage: next internal static-routes-info [directory] [options]

Options:
  --json        Output as JSON instead of markdown.
  --limit <n>   Only show the first N routes after sorting (totals always reflect all routes).
  --sort <key>  Sort routes by: name (default, ascending), or one of
                client, client-js, client-css, client-map,
                server, server-bundled-js, server-unbundled, server-map,
                total (descending).
  --files       Include the list of files (relative to the output directory)
                per category in the JSON output. Requires --json.
  -h, --help    Displays this message.
```

`--limit` and `--sort` always consider every route for the totals; only
the displayed table is trimmed/reordered. `--files` is only meaningful
in JSON output and errors otherwise. Invalid `--sort` keys error with
the valid set listed.

#### Six categories per route

Each file the tool sees is placed into **exactly one** of these six
buckets, so totals are not double-counted:

| Category | Description |
| ------------------ |
--------------------------------------------------------------------------------------------------
|
| Client JS | `.js` chunks loaded by the browser |
| Client CSS | `.css` files loaded by the browser |
| Client Source Maps | `.map` files for client JS / CSS |
| Server Bundled JS | `.js` chunks executed on the server (App Router,
Pages SSR, route handlers, middleware) |
| Server Unbundled | Files traced via `*.nft.json` outside `distDir`
(typically `node_modules` deps for `serverExternalPackages` / Pages SSR)
|
| Server Source Maps | `.map` files for server JS, including `.map`s
referenced from nft.json |

Source maps are discovered three ways: `.map` extension matches, `//#
sourceMappingURL=...` trailers in JS, and `/*# sourceMappingURL=...*/`
trailers in CSS. Maps always go into the `Maps` category even when the
manifest puts them next to their bundle, so they never inflate Bundled
or CSS counts. `sourceMappingURL` reads are memoized per chunk so a
chunk shared by N routes is opened once.

#### Two-step measurement

1. **Capture per-route file sets** by reading manifests:
- `pages-manifest.json` and `app-path-routes-manifest.json` for the
route list.
- `<entry>.nft.json` for server-bundled chunks plus traced node_modules
deps.
- `<entry>_client-reference-manifest.js` for App Router client JS/CSS
(both `entryJSFiles` / `entryCSSFiles` on Turbopack and
`clientModules.chunks` on webpack — the parser handles both layouts).
- `build-manifest.json` for shared App Router root chunks
(`rootMainFiles`) and Pages Router client chunks.
   - `middleware-manifest.json` for middleware and edge route handlers.
2. **Deduplicate inside each per-route category, then measure.** A
global `lstat` cache stats every unique path once across the whole run;
per-category sets dedupe via string equality. Files are routed by
extension (`.map` → Maps, `.css` → CSS, `.js` → bundled / unbundled
depending on whether the path stays inside `distDir`) at the point they
enter a set, so e.g. an `.nft.json` referencing both bundle and `.map`
paths places each in the right bucket.

#### Shared metric

For each route, every category also carries a `sharedAvg`: the average
size of the *intersection* between this route and each peer route of the
same type. Computed as

```
sharedAvg = (Σ over peers p: |files(this) ∩ files(p)|) / number_of_peers
```

with both file count and bytes reported. The metric is also expressed as
a percentage of the route's own count and bytes (`percentCount`,
`percentBytes`) to make sharing easy to interpret at a glance — e.g.
`5.3 files (88%) / 424.12 KB (100%)` means "88% of this route's files,
and effectively all of its bytes, are also shipped by an average peer".
Routes with no peers (only one route of their type) get `null`. Note
that percentages are NOT commutative across peers (they're divided by
each route's own count/bytes) while raw intersection numbers are.

#### Output

Markdown (default), with three sections — `## Routes`, `## Shared (avg
per other route of same type)`, `## Totals` — each rendered as a
fixed-width aligned table. Empty cells render as `-` (and routes with no
peers in the Shared section as `n/a`) so meaningful values stand out:

```
## Routes

| Route        | Type       | Client JS           | Client CSS      | Client Source Maps | Server Bundled JS    | Server Unbundled    | Server Source Maps  |
| ------------ | ---------- | ------------------- | --------------- | ------------------ | -------------------- | ------------------- | ------------------- |
| /            | app-page   | 6 files / 424.40 KB | 2 files / 153 B | -                  | 16 files / 384.47 KB | 140 files / 1.37 MB | 16 files / 2.31 MB  |
| /api/edge    | app-route  | -                   | -               | -                  | 9 files / 296.82 KB  | -                   | 4 files / 1.53 MB   |
…

## Shared (avg per other route of same type)

| Route        | Type     | Client JS                          | Client CSS                   | Client Source Maps | Server Bundled JS                | Server Unbundled                  | Server Source Maps             |
| ------------ | -------- | ---------------------------------- | ---------------------------- | ------------------ | -------------------------------- | --------------------------------- | ------------------------------ |
| /            | app-page | 5.3 files (88%) / 424.12 KB (100%) | 1.3 files (63%) / 52 B (34%) | -                  | 11 files (69%) / 357.52 KB (93%) | 140 files (100%) / 1.37 MB (100%) | 11 files (69%) / 2.19 MB (95%) |
…
```

JSON has the same per-category structure (count + bytes + sharedAvg +
optional files list when `--files` is used) with identical category
ordering: `clientJs`, `clientCss`, `clientMaps`, `serverBundled`,
`serverUnbundled`, `serverMaps`. JSON values are exact (e.g. `0/0` is
preserved as `{count:0, bytes:0}` rather than `-`) so machine consumers
aren't affected by the markdown placeholder. Totals also expose dedup'd
`files` arrays under `--files`.

#### Route types

Reported types: `app-page`, `app-route`, `pages`, `pages-static`,
`pages-api`, `middleware`. App Router route handlers with `runtime:
'edge'` report as `app-route` (not a separate `edge-function`) so
they're directly comparable with their Node-runtime peers. Middleware is
a first-class type rather than being lumped under edge-function.

#### Robust manifest parsing

`_client-reference-manifest.js` is a JS module, not JSON. Both bundlers
emit it but with different layouts:

- Turbopack: multi-line, with a `for (const key in
MANIFEST[entry].clientModules) MANIFEST[entry].clientModules[k] = val`
suffix when a deployment ID is set.
- Webpack: single-line, no whitespace around `=`.

We extract the JSON body without evaluating the file. The implementation
locates the `globalThis.__RSC_MANIFEST[` anchor, walks the JS string
literal that holds the entry name (honoring `\\` escapes), then
balance-walks the `{...}` body. This handles entry names that contain
`]` characters, e.g.

```js
globalThis.__RSC_MANIFEST["/(dashboard)/[teamSlug]/(team)/~/stores/(store-details)/blob/[storeId]/page"] = {...}
```

Any structural surprise (anchor missing, unterminated string/object,
JSON parse failure) throws with the file path and offset — we never
silently undercount client JS/CSS for a route. Only file-not-found stays
as a `null` return — that's a normal case for server entries with no
client-reference manifest (middleware, route handlers, etc.).

### Tests

`test/production/static-routes-info/` is a real fixture covering every
route type:

- App Router: `/`, `/about`, `/no-client`, `/items/[itemId]` (a dynamic
segment inside a `(group)` route group, which forces `]` to appear
unescaped in the manifest entry name and exercises the parser), plus the
auto-generated `/_not-found`.
- App Router route handlers: `/api/node` (default Node runtime),
`/api/edge` (`runtime: 'edge'`).
- Pages Router: `/pages-ssr`, `/pages-ssr-2` (siblings sharing chunks),
`/pages-static`, `/api/hello`.
- Middleware: `middleware.ts`.
- A shared lib (`lib/shared.ts`) imported by both pages-router siblings
and `/`-`/about` to give the shared-avg metric something non-trivial to
measure.
- A `'use client'` `Counter` component imported by `/` and `/about` (but
not `/no-client`), which itself imports `counter.module.css`. Routes
that import Counter must ship strictly more client JS (and on Turbopack,
more client CSS) than `/no-client` — this is asserted, and it's the
cross-bundler regression check for the App Router client-JS collection
on webpack via `clientModules.chunks` (without it, every webpack
app-page reports `clientJs.count = 0`).

The test file (`static-routes-info.test.ts`) covers all the above plus
output formats, sort options, limit semantics, file-list integrity,
totals dedup, shared-avg correctness against a hand-computed reference,
markdown/JSON consistency, and the empty-cell `-` placeholder. The
shared-avg metric is verified three independent ways: against a
from-scratch reimplementation that walks the `--files` lists and
recomputes every (route, category) cell; against a "sharedAvg.count ==
own.count IFF every peer is a strict superset" invariant that makes 100%
values load-bearing; and against a hand-known case where one route ships
a chunk no peer does, forcing strictly-below-100% sharing. 31 tests,
passing on both Turbopack and webpack.

The tool was also exercised against `bench/basic-app`,
`bench/heavy-npm-deps`, `bench/nested-deps`, `bench/app-router-server`,
and `bench/nested-deps-app-router` while developing.

### Notes for reviewers

- New error codes added to `errors.json` for the manifest-parser throws
and other invariant violations.
- The command is registered under `next internal`; not advertised in
user-facing docs by design.
- Webpack quirk documented in the test: `flight-manifest-plugin.ts`'s
`mergeManifest` merges every app-page's `entryCSSFiles` into every other
route's CRM, so per-route CSS attribution on webpack is inherently fuzzy
— the test asserts CSS attribution on Turbopack only, and the comment
explains why.

<!-- NEXT_JS_LLM_PR -->

---------
Co-authored-by: 's avatarv-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com>
Co-authored-by: 's avatarClaude <noreply@anthropic.com>
Co-authored-by: 's avatarTobias Koppers <sokra@users.noreply.github.com>
parent 5d0dab5f
......@@ -1230,6 +1230,14 @@
"1229": "Route \"%s\": Next.js encountered uncached or runtime data in \\`generateMetadata()\\`.\\n\\nThis route's metadata is blocked, but the rest of its content can be prerendered.\\n\\nWays to fix this:\\n - Use a static metadata export instead of \\`generateMetadata()\\`\\n - Cache the metadata with \\`\"use cache\"\\` in \\`generateMetadata()\\`\\n - Add a dynamic data access (e.g. \\`await connection()\\`) to the page to render it at request time\\n\\nLearn more: https://nextjs.org/docs/messages/next-prerender-dynamic-metadata",
"1230": "Route \"%s\": Next.js encountered runtime data in \\`generateMetadata()\\`.\\n\\nThis route's metadata is blocked, but the rest of its content can be prerendered. \\`cookies()\\`, \\`headers()\\`, \\`params\\`, or \\`searchParams\\` accessed in \\`generateMetadata()\\` cause it to run dynamically.\\n\\nWays to fix this:\\n - Use a static metadata export instead of \\`generateMetadata()\\`\\n - Add a dynamic data access (e.g. \\`await connection()\\`) to the page to render it at request time\\n\\nLearn more: https://nextjs.org/docs/messages/next-prerender-dynamic-metadata",
"1231": "Route \"%s\": Next.js encountered uncached data in \\`generateMetadata()\\`.\\n\\nThis route's metadata is blocked, but the rest of its content can be prerendered. \\`fetch(...)\\` or \\`connection()\\` accessed in \\`generateMetadata()\\` cause it to run dynamically.\\n\\nWays to fix this:\\n - Cache the metadata with \\`\"use cache\"\\` in \\`generateMetadata()\\`\\n - Add a dynamic data access (e.g. \\`await connection()\\`) to the page to render it at request time\\n\\nLearn more: https://nextjs.org/docs/messages/next-prerender-dynamic-metadata",
"1235": "unreachable sort key: %s",
"1236": "Expected '%s' after entry name in %s at offset %s, got %s.",
"1237": "Could not find 'globalThis.__RSC_MANIFEST[' in %s; client reference manifest format may have changed.",
"1238": "Unterminated entry-name string literal in %s.",
"1239": "Failed to parse JSON body of %s: %s",
"1240": "Expected '{' after 'globalThis.__RSC_MANIFEST[...] =' in %s at offset %s, got %s.",
"1241": "Expected string literal as entry name in %s at offset %s, got %s.",
"1242": "Unterminated JSON object in %s.",
"1244": "Route \"%s\": Next.js encountered %s during the initial render.\\n\\nThis value must either be prerendered or computed per request.\\n\\nWays to fix this:\\n - Render at request time by adding a dynamic data access (e.g. \\`await connection()\\`) before this call\\n - Prerender and cache the value with \\`\"use cache\"\\`\\n - Render the value on the client with \\`\"use client\"\\`\\n%s\\nLearn more: %s",
"1245": "Route \"%s\": Next.js encountered %s in a Client Component.\\n\\nThis value would be evaluated during the prerender and fixed at build time, instead of recomputed on each visit.\\n\\nWays to fix this:\\n - Wrap the Client Component in \\`<Suspense fallback={...}>\\`\\n - Move the read into a \\`useEffect\\` or event handler\\n\\nLearn more: %s",
"1247": "Route \"%s\": Next.js encountered %s without an explicit rendering intent.\\n\\nThis value can change between renders, so it must be either prerendered or computed later.\\n\\nWays to fix this:\\n - Render at request time by adding a dynamic data access (e.g. \\`await connection()\\`) before this call\\n - Prerender and cache the value with \\`\"use cache\"\\`\\n - Render the value on the client with \\`\"use client\"\\`\\n%s\\nLearn more: %s"
......
......@@ -700,4 +700,46 @@ internal
})
.usage('[directory] [options]')
internal
.command('static-routes-info')
.description(
'Analyze a built Next.js app and report per-route bundle sizes across server bundled JS, server source maps, server unbundled, client JS, client source maps, and client CSS categories.'
)
.argument(
'[directory]',
`A directory containing the built Next.js application. ${italic(
'If no directory is provided, the current directory will be used.'
)}`
)
.option('--json', 'Output as JSON instead of markdown.')
.option(
'--limit <n>',
'Only show the first N routes after sorting (totals always reflect all routes).',
parseInt
)
.option(
'--sort <key>',
'Sort routes by: name (default, ascending), or one of client, client-js, client-css, client-map, server, server-bundled-js, server-unbundled, server-map, total (descending).'
)
.option(
'--files',
'Include the list of files (relative to the output directory) per category in the JSON output. Requires --json.'
)
.action(
(
directory: string,
options: {
json?: boolean
limit?: number
sort?: string
files?: boolean
}
) => {
return import('../cli/internal/static-routes-info.js').then((mod) =>
mod.staticRoutesInfoCli(options, directory)
)
}
)
.usage('[directory] [options]')
program.parse(process.argv)
This diff is collapsed.
// Route name: `/items/[itemId]` (the `(group)` segment is silent).
// The internal entry name in the client-reference manifest is
// `app/(group)/items/[itemId]/page` — note the `]` characters that
// appear unescaped inside the entry-key string. This fixture exists
// to exercise `parseClientReferenceManifest`'s string-literal walking
// for `]`-containing entry names; a naïve `[^\]]*` regex breaks here.
export default async function ItemPage({
params,
}: {
params: Promise<{ itemId: string }>
}) {
const { itemId } = await params
return <p>item {itemId}</p>
}
// Second app-page so we can test the sharedAvg metric (which requires
// at least 2 routes of the same type). Imports the same shared module
// and the same client component as `app/page.tsx` so the tool's per-route
// file sets actually overlap beyond just framework/layout chunks.
import { sharedHelper } from '../../lib/shared'
import Counter from '../../components/Counter'
export default function About() {
return (
<p data-len={sharedHelper()}>
about
<Counter />
</p>
)
}
export const runtime = 'edge'
export async function GET() {
return Response.json({ ok: true })
}
export const dynamic = 'force-dynamic'
export async function GET() {
return Response.json({ ok: true })
}
import { ReactNode } from 'react'
// Layout imports the global stylesheet — this is the typical Next.js
// pattern where `globals.css` is imported by the root layout and so
// applies to every app-page route (not just one). The tool must pick up
// CSS from the layout segment via `entryCSSFiles[<layout>]` for all
// app-pages, not just the one that directly imports `globals.css`.
import './globals.css'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
// Third app-page that deliberately does NOT import the `Counter` client
// component. Used as the negative reference in the "client components
// contribute per-route client JS" assertion: routes that import Counter
// must ship strictly more client JS bytes than this route.
export default function NoClient() {
return <p>no-client</p>
}
import { sharedHelper } from '../lib/shared'
import Counter from '../components/Counter'
export default function Page() {
return (
<p className="hello" data-len={sharedHelper()}>
app-page
<Counter />
</p>
)
}
'use client'
// Client component imported by App Router pages. Imports a CSS module
// (`counter.module.css`) so that `clientModules` in the per-route
// `_client-reference-manifest.js` references a CSS module entry — and the
// tool must surface its extracted .css file via `entryCSSFiles` for routes
// that transitively import this component.
import { useState } from 'react'
import styles from './counter.module.css'
export default function Counter() {
const [count, setCount] = useState(0)
return (
<button className={styles.btn} onClick={() => setCount(count + 1)}>
count is {count}
</button>
)
}
.btn {
color: white;
background: black;
font-weight: bold;
padding: 8px 16px;
border-radius: 4px;
border: none;
cursor: pointer;
}
// Shared module imported by multiple routes so the static-routes-info tool
// can observe deliberate chunk sharing in its `sharedAvg` metric. The string
// is large enough that, however the bundler decides to chunk it (a separate
// shared chunk, or inlined into each importer), the resulting bundle output
// includes it and any shared chunk picked by the bundler is detectable.
export const SHARED_PAYLOAD = 'shared-route-info-payload-' + 'x'.repeat(4096)
export function sharedHelper(): number {
return SHARED_PAYLOAD.length
}
import { NextResponse } from 'next/server'
// `middleware.ts` is reported as the `middleware` route type. We only need it
// to exist for the test — its behavior doesn't matter.
export function middleware() {
return NextResponse.next()
}
export const config = {
matcher: '/about',
}
/**
* @type {import('next').NextConfig}
*/
const nextConfig = {}
module.exports = nextConfig
// Pages Router API route — exercises the `pages-api` route type.
import type { NextApiRequest, NextApiResponse } from 'next'
export default function handler(_req: NextApiRequest, res: NextApiResponse) {
res.status(200).json({ ok: true })
}
// Second SSR page so the `pages` route type has a peer for the sharedAvg
// metric. Imports the same shared module as `pages-ssr.tsx` so we get
// observable chunk overlap beyond the standard `_app` / framework chunks.
import { sharedHelper } from '../lib/shared'
export default function ServerPage2({ len }: { len: number }) {
return <p data-len={len}>pages-ssr-2</p>
}
export async function getServerSideProps() {
return { props: { len: sharedHelper() } }
}
// Server-rendered Pages Router page — `getServerSideProps` forces a `.js`
// server entry, exercises the `pages` route type.
import { sharedHelper } from '../lib/shared'
export default function ServerPage({ len }: { len: number }) {
return <p data-len={len}>pages-ssr</p>
}
export async function getServerSideProps() {
return { props: { len: sharedHelper() } }
}
// Statically pre-rendered Pages Router page (no data-fetching) — produces a
// `.html` server entry, exercises the `pages-static` route type.
export default function StaticPage() {
return <p>pages-static</p>
}
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment