Unverified Commit 9ce8d136 authored by Andrew Clark's avatar Andrew Clark Committed by GitHub

experimental.prefetchInlining: bundle segment prefetches into a single response (#90555)

## Background

Next.js 16 introduced per-segment prefetching through the Client Segment
Cache. Rather than fetching all data for a route in a single request,
the client issues individual requests for each segment in the route
tree. This design improves cache efficiency: shared layouts between
sibling routes (e.g., /dashboard/settings and /dashboard/profile sharing
a /dashboard layout) are fetched once and reused from the client cache,
avoiding redundant data transfer.

The trade-off is request volume. A route with N segments now produces N
prefetch requests instead of one. Users upgrading from older versions
notice significantly more network activity in their devtools, even
though the total bytes transferred may be similar or lower due to
deduplication.

Per-segment fetching is still a reasonable default for many sites. These
prefetch requests are served from cache, they're fast, and they run in
parallel. The main scenario where the trade-off breaks down is
deployment environments that charge per-request. But even setting aside
cost, there is a theoretical performance threshold where very small
segments are better off inlined — the per-request overhead (connection
setup, headers, scheduling) exceeds the cost of transferring duplicate
bytes. This is analogous to JS bundlers, which inline small modules
rather than creating separate chunks, because the overhead of an
additional script tag or dynamic import outweighs the bytes saved.

## What this change does

This adds `experimental.prefetchInlining`, a boolean option in
next.config.js. When enabled, the server bundles all segment data for a
route into a single `/_inlined` response rather than serving each
segment individually. The tree prefetch (`/_tree`), which provides route
structure metadata, remains a separate request — but optimistic routing
(#88965) eliminates that request entirely by predicting the route
structure client-side. With both features enabled, prefetching is
effectively one request per link.

The fundamental trade-off is straightforward: inlining reduces request
count at the cost of deduplication. Each inlined response includes its
own copy of any shared layout data, so two sibling routes will each
transfer the shared layout rather than sharing a single cached copy.
This is the same trade-off that compilers and bundlers face when
deciding whether to inline a function: inlining eliminates the overhead
of indirection (here, extra HTTP requests) but increases total size when
the same data appears in multiple call sites.

## Future direction

The boolean flag is a stepping stone. The observation that there is a
natural size threshold below which inlining is strictly better — where
per-request overhead dominates the cost of any duplicate bytes — points
toward a size-based heuristic, analogous to how compilers choose an
inlining threshold. Small segments would be inlined automatically;
segments exceeding a byte threshold would be "outlined" into separate
requests where deduplication can take effect. For most applications,
this would require no configuration. For applications with specific
latency or bandwidth constraints, an option to adjust the threshold
would let developers tune their position on the requests-vs-bytes curve.
Adaptive heuristics based on network conditions are also possible,
though further out.
parent 4d2ac245
......@@ -238,6 +238,9 @@ export function getDefineEnv({
'process.env.__NEXT_DYNAMIC_ON_HOVER': Boolean(
config.experimental.dynamicOnHover
),
'process.env.__NEXT_PREFETCH_INLINING': Boolean(
config.experimental.prefetchInlining
),
'process.env.__NEXT_OPTIMISTIC_CLIENT_CACHE':
config.experimental.optimisticClientCache ?? true,
'process.env.__NEXT_MIDDLEWARE_PREFETCH':
......
......@@ -726,6 +726,7 @@ export async function handler(
nextConfig.experimental.optimisticRouting
),
inlineCss: Boolean(nextConfig.experimental.inlineCss),
prefetchInlining: Boolean(nextConfig.experimental.prefetchInlining),
authInterrupts: Boolean(nextConfig.experimental.authInterrupts),
clientTraceMetadata:
nextConfig.experimental.clientTraceMetadata || ([] as any),
......
......@@ -163,6 +163,7 @@ async function requestHandler(
dynamicOnHover: Boolean(nextConfig.experimental.dynamicOnHover),
optimisticRouting: Boolean(nextConfig.experimental.optimisticRouting),
inlineCss: Boolean(nextConfig.experimental.inlineCss),
prefetchInlining: Boolean(nextConfig.experimental.prefetchInlining),
authInterrupts: Boolean(nextConfig.experimental.authInterrupts),
clientTraceMetadata:
nextConfig.experimental.clientTraceMetadata || ([] as any),
......
......@@ -2,6 +2,8 @@ import type {
TreePrefetch,
RootTreePrefetch,
SegmentPrefetch,
InlinedPrefetchResponse,
InlinedSegmentPrefetch,
} from '../../../server/app-render/collect-segment-data'
import type {
CacheNodeSeedData,
......@@ -1984,6 +1986,176 @@ export async function fetchSegmentOnCacheMiss(
}
}
// TODO: The inlined prefetch flow below is temporary. Eventually, inlining
// will be the default behavior controlled by a size heuristic rather than a
// boolean flag. At that point, the per-segment and inlined fetch paths will
// merge, and these separate functions will be removed.
//
// The call site in the scheduler is guarded by
// process.env.__NEXT_PREFETCH_INLINING, so these functions are
// dead-code-eliminated from the client bundle when the feature is disabled.
export async function fetchInlinedSegmentsOnCacheMiss(
route: FulfilledRouteCacheEntry,
routeKey: RouteCacheKey,
tree: RouteTree,
spawnedEntries: Map<SegmentRequestKey, PendingSegmentCacheEntry>
): Promise<PrefetchSubtaskResult<null> | null> {
// When prefetch inlining is enabled, all segment data for a route is bundled
// into a single /_inlined response instead of individual per-segment
// requests. This function fetches that response and walks the tree to fill
// all segment cache entries at once.
const url = new URL(route.canonicalUrl, location.origin)
const nextUrl = routeKey.nextUrl
const headers: RequestHeaders = {
[RSC_HEADER]: '1',
[NEXT_ROUTER_PREFETCH_HEADER]: '1',
[NEXT_ROUTER_SEGMENT_PREFETCH_HEADER]: '/_inlined',
}
if (nextUrl !== null) {
headers[NEXT_URL] = nextUrl
}
addInstantPrefetchHeaderIfLocked(headers)
try {
const response = await fetchPrefetchResponse(url, headers)
if (
!response ||
!response.ok ||
response.status === 204 ||
(response.headers.get(NEXT_DID_POSTPONE_HEADER) !== '2' &&
!isOutputExportMode) ||
!response.body
) {
rejectSegmentEntriesIfStillPending(spawnedEntries, Date.now() + 10 * 1000)
return null
}
const closed = createPromiseWithResolvers<void>()
const prefetchStream = createPrefetchResponseStream(
response.body,
closed.resolve,
function onResponseSizeUpdate() {
// For inlined responses, size tracking per segment is approximate.
// We don't track individual sizes since they're all in one response.
}
)
const serverData =
await createFromNextReadableStream<InlinedPrefetchResponse>(
prefetchStream,
headers,
{ allowPartialStream: true }
)
if (
(response.headers.get(NEXT_NAV_DEPLOYMENT_ID_HEADER) ??
serverData.tree.segment.buildId) !== getNavigationBuildId()
) {
rejectSegmentEntriesIfStillPending(spawnedEntries, Date.now() + 10 * 1000)
return null
}
const now = Date.now()
// Walk the inlined tree in parallel with the RouteTree and fill
// segment cache entries.
fillInlinedSegmentEntries(now, route, tree, serverData.tree, spawnedEntries)
// Fill the head entry.
const headStaleAt = now + getStaleTimeMs(serverData.head.staleTime)
const headKey = route.metadata.requestKey
const ownedHeadEntry = spawnedEntries.get(headKey)
if (ownedHeadEntry !== undefined) {
fulfillSegmentCacheEntry(
ownedHeadEntry,
serverData.head.rsc,
headStaleAt,
serverData.head.isPartial
)
} else {
// The head was already cached. Try to upsert if the entry is empty.
const existingEntry = readOrCreateSegmentCacheEntry(
now,
FetchStrategy.PPR,
route.metadata
)
if (existingEntry.status === EntryStatus.Empty) {
fulfillSegmentCacheEntry(
upgradeToPendingSegment(existingEntry, FetchStrategy.PPR),
serverData.head.rsc,
headStaleAt,
serverData.head.isPartial
)
}
}
// Reject any remaining entries that were not fulfilled by the response.
rejectSegmentEntriesIfStillPending(spawnedEntries, Date.now() + 10 * 1000)
return { value: null, closed: closed.promise }
} catch (error) {
rejectSegmentEntriesIfStillPending(spawnedEntries, Date.now() + 10 * 1000)
return null
}
}
function fillInlinedSegmentEntries(
now: number,
route: FulfilledRouteCacheEntry,
tree: RouteTree,
inlinedNode: InlinedSegmentPrefetch,
spawnedEntries: Map<SegmentRequestKey, PendingSegmentCacheEntry>
): void {
// Check if the spawned entries map has an entry for this segment's key.
const segment = inlinedNode.segment
const staleAt = now + getStaleTimeMs(segment.staleTime)
const ownedEntry = spawnedEntries.get(tree.requestKey)
if (ownedEntry !== undefined) {
// We own this entry. Fulfill it directly.
fulfillSegmentCacheEntry(
ownedEntry,
segment.rsc,
staleAt,
segment.isPartial
)
} else {
// Not owned by us — this is extra data from the inlined response for a
// segment that was already cached. Try to upsert if the entry is empty.
const existingEntry = readOrCreateSegmentCacheEntry(
now,
FetchStrategy.PPR,
tree
)
if (existingEntry.status === EntryStatus.Empty) {
fulfillSegmentCacheEntry(
upgradeToPendingSegment(existingEntry, FetchStrategy.PPR),
segment.rsc,
staleAt,
segment.isPartial
)
}
}
// Recurse into children.
if (tree.slots !== null && inlinedNode.slots !== null) {
for (const parallelRouteKey in tree.slots) {
const childTree = tree.slots[parallelRouteKey]
const childInlinedNode = inlinedNode.slots[parallelRouteKey]
if (childInlinedNode !== undefined) {
fillInlinedSegmentEntries(
now,
route,
childTree,
childInlinedNode,
spawnedEntries
)
}
}
}
}
export async function fetchSegmentPrefetchesUsingDynamicRequest(
task: PrefetchTask,
route: FulfilledRouteCacheEntry,
......
......@@ -10,6 +10,7 @@ import {
readOrCreateSegmentCacheEntry,
fetchRouteOnCacheMiss,
fetchSegmentOnCacheMiss,
fetchInlinedSegmentsOnCacheMiss,
EntryStatus,
type FulfilledRouteCacheEntry,
type RouteCacheEntry,
......@@ -793,6 +794,7 @@ function pingRootRouteTree(
)
}
}
return PrefetchTaskExitStatus.Done
}
case FetchStrategy.Full:
......@@ -1482,6 +1484,25 @@ function pingStaticSegmentData(
): void {
switch (segment.status) {
case EntryStatus.Empty:
if (process.env.__NEXT_PREFETCH_INLINING) {
// All segment data for this route is bundled into a single
// /_inlined response. Walk the full route tree to collect all
// Empty segments, upgrade them to Pending, and spawn one
// bundled request. Subsequent calls for other segments in the
// same tree will see them as Pending and skip.
const inlinedEntries = collectInlinedEntries(now, route)
if (inlinedEntries.size > 0) {
spawnPrefetchSubtask(
fetchInlinedSegmentsOnCacheMiss(
route,
routeKey,
route.tree,
inlinedEntries
)
)
}
break
}
// Upgrade to Pending so we know there's already a request in progress
spawnPrefetchSubtask(
fetchSegmentOnCacheMiss(
......@@ -1557,6 +1578,51 @@ function pingStaticSegmentData(
// entry, which is handled by `fetchSegmentOnCacheMiss`).
}
/**
* Walks the RouteTree (including the head metadata) and collects any segments
* that are still Empty into a Map, upgrading them to Pending. These entries
* will be fulfilled by the inlined prefetch response.
*/
function collectInlinedEntries(
now: number,
route: FulfilledRouteCacheEntry
): Map<SegmentRequestKey, PendingSegmentCacheEntry> {
const entries = new Map<SegmentRequestKey, PendingSegmentCacheEntry>()
collectInlinedEntriesImpl(now, route.tree, entries)
// Also collect the head/metadata entry.
const headEntry = readOrCreateSegmentCacheEntry(
now,
FetchStrategy.PPR,
route.metadata
)
if (headEntry.status === EntryStatus.Empty) {
entries.set(
route.metadata.requestKey,
upgradeToPendingSegment(headEntry, FetchStrategy.PPR)
)
}
return entries
}
function collectInlinedEntriesImpl(
now: number,
tree: RouteTree,
entries: Map<SegmentRequestKey, PendingSegmentCacheEntry>
): void {
const entry = readOrCreateSegmentCacheEntry(now, FetchStrategy.PPR, tree)
if (entry.status === EntryStatus.Empty) {
entries.set(
tree.requestKey,
upgradeToPendingSegment(entry, FetchStrategy.PPR)
)
}
if (tree.slots !== null) {
for (const parallelRouteKey in tree.slots) {
collectInlinedEntriesImpl(now, tree.slots[parallelRouteKey], entries)
}
}
}
function pingPPRSegmentRevalidation(
now: number,
route: FulfilledRouteCacheEntry,
......
......@@ -509,6 +509,7 @@ async function exportAppImpl(
dynamicOnHover: nextConfig.experimental.dynamicOnHover ?? false,
optimisticRouting: nextConfig.experimental.optimisticRouting ?? false,
inlineCss: nextConfig.experimental.inlineCss ?? false,
prefetchInlining: nextConfig.experimental.prefetchInlining ?? false,
authInterrupts: !!nextConfig.experimental.authInterrupts,
maxPostponedStateSizeBytes: parseMaxPostponedStateSize(
nextConfig.experimental.maxPostponedStateSize
......
......@@ -6148,7 +6148,8 @@ async function collectSegmentData(
fullPageDataBuffer,
staleTime,
clientModules,
serverConsumerManifest
serverConsumerManifest,
renderOpts.experimental.prefetchInlining
)
}
......
......@@ -160,6 +160,7 @@ export interface RenderOptsPartial {
dynamicOnHover: boolean
optimisticRouting: boolean
inlineCss: boolean
prefetchInlining: boolean
authInterrupts: boolean
/**
......
......@@ -573,6 +573,8 @@ export default abstract class Server<
optimisticRouting:
this.nextConfig.experimental.optimisticRouting ?? false,
inlineCss: this.nextConfig.experimental.inlineCss ?? false,
prefetchInlining:
this.nextConfig.experimental.prefetchInlining ?? false,
authInterrupts: !!this.nextConfig.experimental.authInterrupts,
maxPostponedStateSizeBytes: parseMaxPostponedStateSize(
this.nextConfig.experimental.maxPostponedStateSize
......
......@@ -221,6 +221,7 @@ export const experimentalSchema = {
dynamicOnHover: z.boolean().optional(),
optimisticRouting: z.boolean().optional(),
varyParams: z.boolean().optional(),
prefetchInlining: z.boolean().optional(),
disableOptimizedLoading: z.boolean().optional(),
disablePostcssPresetEnv: z.boolean().optional(),
cacheComponents: z.boolean().optional(),
......
......@@ -355,6 +355,7 @@ export interface ExperimentalConfig {
dynamicOnHover?: boolean
optimisticRouting?: boolean
varyParams?: boolean
prefetchInlining?: boolean
preloadEntriesOnStart?: boolean
clientRouterFilter?: boolean
clientRouterFilterRedirects?: boolean
......@@ -1647,6 +1648,7 @@ export const defaultConfig = Object.freeze({
clientParamParsingOrigins: undefined,
dynamicOnHover: false,
varyParams: false,
prefetchInlining: false,
preloadEntriesOnStart: true,
clientRouterFilter: true,
clientRouterFilterRedirects: false,
......@@ -1792,6 +1794,7 @@ export interface NextConfigRuntime {
| 'dynamicOnHover'
| 'optimisticRouting'
| 'inlineCss'
| 'prefetchInlining'
| 'authInterrupts'
| 'clientTraceMetadata'
| 'clientParamParsingOrigins'
......@@ -1856,6 +1859,7 @@ export function getNextConfigRuntime(
dynamicOnHover: ex.dynamicOnHover,
optimisticRouting: ex.optimisticRouting,
inlineCss: ex.inlineCss,
prefetchInlining: ex.prefetchInlining,
authInterrupts: ex.authInterrupts,
clientTraceMetadata: ex.clientTraceMetadata,
clientParamParsingOrigins: ex.clientParamParsingOrigins,
......
import { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
import { LinkAccordion } from '../components/link-accordion'
export default function Home() {
return (
<div>
<h1>Home</h1>
<ul>
<li>
<LinkAccordion href="/shared/a/b/c">Route A</LinkAccordion>
</li>
<li>
<LinkAccordion href="/shared/a/d/e">Route B</LinkAccordion>
</li>
<li>
<LinkAccordion href="/shared/a/b/c" id="duplicate-a">
Route A (duplicate)
</LinkAccordion>
</li>
</ul>
</div>
)
}
export default function PageC() {
return <p id="page-c">Page C</p>
}
import { ReactNode } from 'react'
export default function BLayout({ children }: { children: ReactNode }) {
return <div id="b-layout">{children}</div>
}
export default function PageE() {
return <p id="page-e">Page E</p>
}
import { ReactNode } from 'react'
export default function DLayout({ children }: { children: ReactNode }) {
return <div id="d-layout">{children}</div>
}
import { ReactNode } from 'react'
export default function ALayout({ children }: { children: ReactNode }) {
return <div id="a-layout">{children}</div>
}
import { ReactNode } from 'react'
export default function SharedLayout({ children }: { children: ReactNode }) {
return <div id="shared-layout">{children}</div>
}
'use client'
import Link, { type LinkProps } from 'next/link'
import { useState } from 'react'
export function LinkAccordion({
href,
children,
id,
}: {
href: LinkProps['href']
children: React.ReactNode
id?: string
}) {
const [isVisible, setIsVisible] = useState(false)
return (
<>
<input
type="checkbox"
checked={isVisible}
onChange={() => setIsVisible(!isVisible)}
data-link-accordion={id ?? href}
/>
{isVisible ? (
<Link href={href}>{children}</Link>
) : (
`${children} (link is hidden)`
)}
</>
)
}
/**
* @type {import('next').NextConfig}
*/
const nextConfig = {
cacheComponents: true,
experimental: {
prefetchInlining: true,
},
}
module.exports = nextConfig
import { nextTestSetup } from 'e2e-utils'
import type * as Playwright from 'playwright'
import { createRouterAct } from 'router-act'
describe('prefetch inlining', () => {
const { next, isNextDev } = nextTestSetup({
files: __dirname,
})
if (isNextDev) {
it('disabled in development', () => {})
return
}
it('bundles all segment data into a single request per route', async () => {
// The test app has two routes that are 5 segments deep:
// /shared/a/b/c and /shared/a/d/e
// Without inlining, prefetching each route would issue one request per
// segment plus one for the head (6+ requests). With inlining enabled,
// all segment data is bundled into a single /_inlined response, so
// revealing a link should produce at most 2 prefetch requests per route:
// one for /_tree and one for /_inlined.
let rscRequestCount = 0
let page: Playwright.Page
const browser = await next.browser('/', {
beforePageLoad(p: Playwright.Page) {
page = p
p.on('request', (request: Playwright.Request) => {
if (request.headers()['rsc']) {
rscRequestCount++
}
})
},
})
const act = createRouterAct(page!)
// Reveal the first link to trigger a prefetch of /shared/a/b/c
await act(
async () => {
await browser
.elementByCss('input[data-link-accordion="/shared/a/b/c"]')
.click()
},
{
includes: 'Page C',
}
)
// Snapshot the request count before revealing the second link
const countBeforeSecondPrefetch = rscRequestCount
// Reveal the second link to trigger a prefetch of /shared/a/d/e
await act(
async () => {
await browser
.elementByCss('input[data-link-accordion="/shared/a/d/e"]')
.click()
},
{
includes: 'Page E',
}
)
// The delta should be at most 2 requests (/_tree + /_inlined).
// Without inlining, there would be 6+ individual segment requests.
const delta = rscRequestCount - countBeforeSecondPrefetch
expect(delta).toBeLessThanOrEqual(2)
// Navigate to the second route. Because the data was fully prefetched,
// there should be no additional requests.
await act(async () => {
await browser.elementByCss('a[href="/shared/a/d/e"]').click()
}, 'no-requests')
// Verify the page rendered correctly
const text = await browser.elementByCss('#page-e').text()
expect(text).toBe('Page E')
})
it('deduplicates inlined prefetch requests for the same route', async () => {
// Two links point to the same route (/shared/a/b/c). When the first
// link is revealed, its prefetch spawns an inlined request. While
// that request is still pending, revealing the second link should
// not spawn any additional requests because the segments are already
// Pending from the first task.
let page: Playwright.Page
const browser = await next.browser('/', {
beforePageLoad(p: Playwright.Page) {
page = p
},
})
const act = createRouterAct(page!)
await act(
async () => {
// Reveal the first link, blocking its response so the prefetch
// stays in-flight.
await act(async () => {
await browser
.elementByCss('input[data-link-accordion="/shared/a/b/c"]')
.click()
}, 'block')
// While the first prefetch is still pending, reveal a second
// link to the same route. This should not spawn any new
// requests because the segments are already Pending.
await act(async () => {
await browser
.elementByCss('input[data-link-accordion="duplicate-a"]')
.click()
}, 'no-requests')
},
{
includes: 'Page C',
}
)
// Navigate to the route. Because the data was fully prefetched,
// no additional requests are needed.
await act(async () => {
await browser.elementByCss('a[href="/shared/a/b/c"]').click()
}, 'no-requests')
const text = await browser.elementByCss('#page-c').text()
expect(text).toBe('Page C')
})
})
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment