Unverified Commit 16eab3ac authored by Andrew Clark's avatar Andrew Clark Committed by GitHub

[backport] Fix: Optimistic routing bugs leading to repeated prefetch loops (#97325)

Backports #97128 ("Fix: Optimistic routing bugs leading to repeated
prefetch loops") to `next-16-3`.

Fixes an infinite prefetch loop in optimistic routing. A proxy that
rewrites every URL to inject a leading path segment — i18n with the
default locale hidden from the URL, so `/one/two` is served as
`/en/one/two` — combined with a fully dynamic target route like
`/[locale]/[...pages]`, produces a prefetch that never settles. When a
prefetch response came back, the client never checked whether the
rendered tree matched the one it had requested. When the params baked
into the request were wrong because of the rewrite, the task kept
re-attempting to fulfill the missing data and looped forever.

The prefetch path now records a dynamic-rewrite marker on the local
route definition when the response's tree diverges from the requested
one, which disables further optimistic resolution of that route and
invalidates the entries derived from it. This is the same strategy
normal navigation responses already used, now applied to the prefetch
path.

A second, related bug is also fixed: parallel routes with conflicting
dynamic params at the same level (`@modal/[...catchAll]` next to
`[username]`) can't be distinguished by the traversal algorithm, which
assumes each segment resolves independently without inspecting sibling
branches. When that scenario is detected, optimistic routing is now
disabled for the conflicting route rather than binding one branch's URL
parts to another branch's params.

### Verification on this branch

Cherry-picked from `5942b37a`. Three source files conflicted, all
because `next-16-3` predates the post-16.3.0 refactor that split the
client's response decoder into its own module:

- `segment-cache/decode-server-response.ts` — this file does not exist
on this branch; it was extracted out of `navigation.ts` after 16.3.0.
Its change (tracking whether the decoded tree's segment identities
diverge from the base request tree) was ported into `navigation.ts`,
where the equivalent logic still lives. Canary decodes a compact
transport tree and compares segment identities inline during the walk;
this branch instead merges the response patch onto the base
`FlightRouterState` before converting it to a `RouteTree`. The check is
therefore re-expressed as a read-only walk in
`convertServerPatchToFullTree` that compares each server patch against
the base tree it is applied to, following canary's descent
(`convertServerPatchToFullTreeImpl`) exactly and using the same
comparison rules: segment identity via `matchSegment`, page segments
matched modulo embedded search params, server-filled `__DEFAULT__`
segments ignored, and inactive parallel-route branches (those carrying a
refresh state in the base) skipped. The result is threaded out through
`NavigationSeed.treeDivergedFromBase`.
- `segment-cache/cache.ts` — the consuming block in
`fetchSegmentPrefetchesUsingDynamicRequest` (mark-as-dynamic-rewrite,
invalidate derived entries, reject with immediate expiry) is
byte-identical to canary. The surrounding conflicts were only
incidental: the `RouteTreeAccumulator` type gaining
`treeDivergedFromBase`, and canary's transport-decoder call vs. this
branch's `convertRootFlightRouterStateToRouteTree`. Resolved by keeping
this branch's calls and adding the new field to the accumulators.
Canary's inline `NavigationSeed` construction in
`writeDynamicTreeResponseIntoCache` does not apply here because that
function delegates to `convertServerPatchToFullTree`, which now carries
the field.
- `router-reducer/create-initial-router-state.ts` — a one-line
accumulator initializer; kept this branch's
`convertRootFlightRouterStateToRouteTree` call and added
`treeDivergedFromBase: false`.

The other touched source files applied cleanly and are byte-identical to
canary: `segment-cache/optimistic-routes.ts` (the parallel-route
conflict detection) and `route-params.ts` (exporting
`canonicalizeURLPart`), along with `ppr-navigations.ts`, the small
cleanly-applied hunk in `navigation.ts`, and every test fixture.

`tsc` passes on the `next` package for all files touched here.
Behavioral verification is left to CI on this branch; the change carries
the `proxy-prefix-rewrite-prefetch-loop` and
`modal-catchall-sibling-dynamic-prefetch-loop` end-to-end suites plus
additions to the `optimistic-routing` suite.

Opened as a draft because the `decode-server-response.ts` →
`navigation.ts` port is a manual re-implementation across the refactor
boundary rather than a mechanical cherry-pick. The divergence walk in
`convertServerPatchToFullTree` is the part most worth a close review.

<!-- NEXT_JS_LLM -->
parent 75548c9a
...@@ -83,7 +83,7 @@ export function createInitialRouterState({ ...@@ -83,7 +83,7 @@ export function createInitialRouterState({
// these trees with InliningHintsStale, which causes the route cache entry // these trees with InliningHintsStale, which causes the route cache entry
// to be immediately expired. The next prefetch will re-fetch the tree with // to be immediately expired. The next prefetch will re-fetch the tree with
// correct hints from the /_tree response. // correct hints from the /_tree response.
const acc = { metadataVaryPath: null } const acc = { metadataVaryPath: null, treeDivergedFromBase: false }
const initialRouteTree = convertRootFlightRouterStateToRouteTree( const initialRouteTree = convertRootFlightRouterStateToRouteTree(
initialTree, initialTree,
initialRenderedSearch as NormalizedSearch, initialRenderedSearch as NormalizedSearch,
......
...@@ -932,7 +932,7 @@ function reuseActiveSegmentInDefaultSlot( ...@@ -932,7 +932,7 @@ function reuseActiveSegmentInDefaultSlot(
reusedRenderedSearch = oldRootRefreshState.renderedSearch reusedRenderedSearch = oldRootRefreshState.renderedSearch
} }
const acc = { metadataVaryPath: null } const acc = { metadataVaryPath: null, treeDivergedFromBase: false }
const reusedRouteTree = convertReusedFlightRouterStateToRouteTree( const reusedRouteTree = convertReusedFlightRouterStateToRouteTree(
parentRouteTree, parentRouteTree,
parallelRouteKey, parallelRouteKey,
......
...@@ -1534,6 +1534,9 @@ function rejectSegmentCacheEntry( ...@@ -1534,6 +1534,9 @@ function rejectSegmentCacheEntry(
type RouteTreeAccumulator = { type RouteTreeAccumulator = {
metadataVaryPath: PageVaryPath | null metadataVaryPath: PageVaryPath | null
// Whether the decoded tree's segment identities diverged from the base
// tree it was overlaid onto. See NavigationSeed.treeDivergedFromBase.
treeDivergedFromBase: boolean
} }
function convertRootTreePrefetchToRouteTree( function convertRootTreePrefetchToRouteTree(
...@@ -2097,7 +2100,10 @@ export async function fetchRouteOnCacheMiss( ...@@ -2097,7 +2100,10 @@ export async function fetchRouteOnCacheMiss(
// //
// During this traversal, we accumulate additional data into this // During this traversal, we accumulate additional data into this
// "accumulator" object. // "accumulator" object.
const acc: RouteTreeAccumulator = { metadataVaryPath: null } const acc: RouteTreeAccumulator = {
metadataVaryPath: null,
treeDivergedFromBase: false,
}
const routeTree = convertRootTreePrefetchToRouteTree( const routeTree = convertRootTreePrefetchToRouteTree(
serverData, serverData,
renderedPathname, renderedPathname,
...@@ -3305,6 +3311,39 @@ export async function fetchSegmentPrefetchesUsingDynamicRequest( ...@@ -3305,6 +3311,39 @@ export async function fetchSegmentPrefetchesUsingDynamicRequest(
// Not needed for prefetch responses; pass unknown to use the default. // Not needed for prefetch responses; pass unknown to use the default.
UnknownDynamicStaleTime UnknownDynamicStaleTime
) )
if (
navigationSeed.treeDivergedFromBase &&
// A head-only request uses the MetadataOnlyRequestTree stub rather than
// a tree derived from the route entry, so divergence from it carries
// no signal.
// TODO: This special case goes away once convertServerPatchToFullTree
// diffs against the base RouteTree (route.tree) instead of the
// request tree.
dynamicRequestTree !== MetadataOnlyRequestTree
) {
// The server rendered a different route tree than the one we requested:
// the URL has a rewrite that behaves dynamically, so the params baked
// into the request are wrong and the server can never fulfill it. Mark
// the route entry — which doubles as the stored prediction pattern, so
// this also disables a bad prediction (see matchKnownRoute) that would
// otherwise be re-derived on every retry — and invalidate entries that
// were derived from it. This mirrors dispatchRetryDueToTreeMismatch on
// the navigation path. It can't loop: the refetched route entry is
// built from the server's response, so it only mismatches again if the
// rewrite's behavior changes again.
markRouteEntryAsDynamicRewrite(route)
invalidateRouteCacheEntries(key.nextUrl, task.treeAtTimeOfPrefetch)
// Reject with an immediate expiration instead of the usual backoff: the
// invalidation above triggers a re-prefetch, which per the above does
// not loop.
// TODO: Consider also bounding retries with a counter on the task
// object, so a prefetch that repeatedly fails to settle backs off
// regardless of the reason.
rejectSegmentEntriesIfStillPending(spawnedEntries, -1)
return null
}
// Aside from writing the data into the cache, this function also returns // Aside from writing the data into the cache, this function also returns
// the entries that were fulfilled, so we can streamingly update their sizes // the entries that were fulfilled, so we can streamingly update their sizes
// in the LRU as more data comes in. // in the LRU as more data comes in.
...@@ -3407,7 +3446,10 @@ function writeDynamicTreeResponseIntoCache( ...@@ -3407,7 +3446,10 @@ function writeDynamicTreeResponseIntoCache(
// //
// During this traversal, we accumulate additional data into this // During this traversal, we accumulate additional data into this
// "accumulator" object. // "accumulator" object.
const acc: RouteTreeAccumulator = { metadataVaryPath: null } const acc: RouteTreeAccumulator = {
metadataVaryPath: null,
treeDivergedFromBase: false,
}
const routeTree = convertRootFlightRouterStateToRouteTree( const routeTree = convertRootFlightRouterStateToRouteTree(
flightRouterState, flightRouterState,
renderedSearch, renderedSearch,
......
...@@ -3,6 +3,7 @@ import type { ...@@ -3,6 +3,7 @@ import type {
FlightRouterState, FlightRouterState,
FlightSegmentPath, FlightSegmentPath,
ScrollRef, ScrollRef,
Segment,
} from '../../../shared/lib/app-router-types' } from '../../../shared/lib/app-router-types'
import type { CacheNode } from '../../../shared/lib/app-router-types' import type { CacheNode } from '../../../shared/lib/app-router-types'
import type { HeadData } from '../../../shared/lib/app-router-types' import type { HeadData } from '../../../shared/lib/app-router-types'
...@@ -50,6 +51,11 @@ import { computeChangedPath } from '../router-reducer/compute-changed-path' ...@@ -50,6 +51,11 @@ import { computeChangedPath } from '../router-reducer/compute-changed-path'
import { isJavaScriptURLString } from '../../lib/javascript-url' import { isJavaScriptURLString } from '../../lib/javascript-url'
import { UnknownDynamicStaleTime, computeDynamicStaleAt } from './bfcache' import { UnknownDynamicStaleTime, computeDynamicStaleAt } from './bfcache'
import { createLinkPrefetchPartialError } from '../../../shared/lib/instant-messages' import { createLinkPrefetchPartialError } from '../../../shared/lib/instant-messages'
import { matchSegment } from '../match-segments'
import {
DEFAULT_SEGMENT_KEY,
PAGE_SEGMENT_KEY,
} from '../../../shared/lib/segment'
/** /**
* Navigate to a new URL, using the Segment Cache to construct a response. * Navigate to a new URL, using the Segment Cache to construct a response.
...@@ -421,6 +427,8 @@ function navigateUsingPrefetchedRouteTree( ...@@ -421,6 +427,8 @@ function navigateUsingPrefetchedRouteTree(
data: null, data: null,
head: null, head: null,
dynamicStaleAt: computeDynamicStaleAt(now, UnknownDynamicStaleTime), dynamicStaleAt: computeDynamicStaleAt(now, UnknownDynamicStaleTime),
// Not derived from a server response; no base to diverge from.
treeDivergedFromBase: false,
} }
return navigateToKnownRoute( return navigateToKnownRoute(
now, now,
...@@ -884,6 +892,16 @@ export type NavigationSeed = { ...@@ -884,6 +892,16 @@ export type NavigationSeed = {
data: CacheNodeSeedData | null data: CacheNodeSeedData | null
head: HeadData | null head: HeadData | null
dynamicStaleAt: number dynamicStaleAt: number
// Whether the response rendered a segment whose identity differs from the
// base tree's at the same position (inactive parallel route branches are
// expected to differ and don't count). Only meaningful when the base is a
// request tree derived from a cached route entry, as during a prefetch:
// divergence then means the entry doesn't describe what the server renders
// — the URL has a rewrite that behaves dynamically (see
// fetchSegmentPrefetchesUsingDynamicRequest). During a navigation the base
// is the current page's tree, so divergence carries no signal. False when
// there was no base to compare against.
treeDivergedFromBase: boolean
} }
export function convertServerPatchToFullTree( export function convertServerPatchToFullTree(
...@@ -912,6 +930,11 @@ export function convertServerPatchToFullTree( ...@@ -912,6 +930,11 @@ export function convertServerPatchToFullTree(
let baseTree: FlightRouterState = currentTree let baseTree: FlightRouterState = currentTree
let baseData: CacheNodeSeedData | null = null let baseData: CacheNodeSeedData | null = null
let head: HeadData | null = null let head: HeadData | null = null
// Whether any patch rendered a segment whose identity differs from the base
// (request) tree at the same position. See NavigationSeed.treeDivergedFromBase.
// Compared against the original `currentTree`, not the progressively-merged
// `baseTree`, so each patch is checked against what was actually requested.
let treeDivergedFromBase = false
if (flightData !== null) { if (flightData !== null) {
for (const { for (const {
segmentPath, segmentPath,
...@@ -919,6 +942,13 @@ export function convertServerPatchToFullTree( ...@@ -919,6 +942,13 @@ export function convertServerPatchToFullTree(
seedData: dataPatch, seedData: dataPatch,
head: headPatch, head: headPatch,
} of flightData) { } of flightData) {
if (!treeDivergedFromBase) {
treeDivergedFromBase = didServerPatchDivergeFromBase(
currentTree,
segmentPath,
treePatch
)
}
const result = convertServerPatchToFullTreeImpl( const result = convertServerPatchToFullTreeImpl(
baseTree, baseTree,
baseData, baseData,
...@@ -943,7 +973,7 @@ export function convertServerPatchToFullTree( ...@@ -943,7 +973,7 @@ export function convertServerPatchToFullTree(
// TODO: Eventually, FlightRouterState will evolve to being a transport format // TODO: Eventually, FlightRouterState will evolve to being a transport format
// only. The RouteTree type will become the main type used for dealing with // only. The RouteTree type will become the main type used for dealing with
// routes on the client, and we'll store it in the state directly. // routes on the client, and we'll store it in the state directly.
const acc = { metadataVaryPath: null } const acc = { metadataVaryPath: null, treeDivergedFromBase: false }
const routeTree = convertRootFlightRouterStateToRouteTree( const routeTree = convertRootFlightRouterStateToRouteTree(
finalFlightRouterState, finalFlightRouterState,
renderedSearch as NormalizedSearch, renderedSearch as NormalizedSearch,
...@@ -957,7 +987,97 @@ export function convertServerPatchToFullTree( ...@@ -957,7 +987,97 @@ export function convertServerPatchToFullTree(
renderedSearch, renderedSearch,
head, head,
dynamicStaleAt: computeDynamicStaleAt(now, dynamicStaleTimeSeconds), dynamicStaleAt: computeDynamicStaleAt(now, dynamicStaleTimeSeconds),
treeDivergedFromBase,
}
}
// Whether a server patch's rendered tree diverges in segment identity from the
// base (request) tree it was applied to. Mirrors the comparison performed while
// decoding a server response in newer versions of the client (see
// NavigationSeed.treeDivergedFromBase). Divergence means the server rendered a
// different route than the one we requested — e.g. a URL rewrite that behaves
// dynamically — so a prefetch built from the base tree can never be fulfilled.
//
// The base-tree descent mirrors convertServerPatchToFullTreeImpl: segmentPath
// is a repeating [parallelRouteKey, segment, ...] pattern; segmentPath[i] keys
// into the children, segmentPath[i + 1] is the server's segment at that
// position.
function didServerPatchDivergeFromBase(
currentTree: FlightRouterState,
segmentPath: FlightSegmentPath,
treePatch: FlightRouterState
): boolean {
let baseNode: FlightRouterState = currentTree
for (let i = 0; i + 1 < segmentPath.length; i += 2) {
const parallelRouteKey: string = segmentPath[i]
const serverSegment: Segment = segmentPath[i + 1]
const childBase: FlightRouterState | undefined =
baseNode[1][parallelRouteKey]
if (childBase === undefined) {
// The base tree doesn't have this branch. Unless the server merely
// filled it with a default, the trees have different structures.
return serverSegment !== DEFAULT_SEGMENT_KEY
}
if (segmentIdentityDivergesFromBase(serverSegment, childBase[0])) {
return true
}
baseNode = childBase
}
return detectTreeDivergenceFromBase(treePatch, baseNode)
}
// Recursively compares a server-rendered subtree against the base subtree at
// the same position. Inactive parallel route branches — which carry a refresh
// state in the base — are expected to differ and are skipped.
function detectTreeDivergenceFromBase(
serverNode: FlightRouterState,
baseNode: FlightRouterState
): boolean {
if (segmentIdentityDivergesFromBase(serverNode[0], baseNode[0])) {
return true
}
const serverChildren = serverNode[1]
const baseChildren = baseNode[1]
for (const parallelRouteKey in serverChildren) {
const childServer = serverChildren[parallelRouteKey]
const childBase = baseChildren[parallelRouteKey]
if (childBase === undefined) {
// A slot the base tree doesn't have. Unless the server merely filled it
// with a default, the trees have different structures.
if (childServer[0] !== DEFAULT_SEGMENT_KEY) {
return true
}
} else if ((childBase[2] ?? null) !== null) {
// The base branch carries a refresh state: an inactive parallel route
// reused from a different route (e.g. a "default" slot). The server's
// answer is expected to differ, so skip the branch.
} else if (detectTreeDivergenceFromBase(childServer, childBase)) {
return true
}
}
return false
}
// Whether two segments at the same position claim different identities. Page
// segments match modulo embedded search params (validated separately, see
// getRenderedSearch), and a default filled in by the server is not a claim
// about the position's identity.
function segmentIdentityDivergesFromBase(
serverSegment: Segment,
baseSegment: Segment
): boolean {
if (
typeof serverSegment === 'string' &&
typeof baseSegment === 'string' &&
serverSegment.startsWith(PAGE_SEGMENT_KEY) &&
baseSegment.startsWith(PAGE_SEGMENT_KEY)
) {
return false
}
if (serverSegment === DEFAULT_SEGMENT_KEY) {
return false
} }
return !matchSegment(baseSegment, serverSegment)
} }
function convertServerPatchToFullTreeImpl( function convertServerPatchToFullTreeImpl(
......
...@@ -55,7 +55,10 @@ import { ...@@ -55,7 +55,10 @@ import {
createMetadataRouteTree, createMetadataRouteTree,
} from './cache' } from './cache'
import { isValueExpired } from './cache-map' import { isValueExpired } from './cache-map'
import { doesStaticSegmentAppearInURL } from '../../route-params' import {
canonicalizeURLPart,
doesStaticSegmentAppearInURL,
} from '../../route-params'
import type { NormalizedPathname, NormalizedSearch } from './cache-key' import type { NormalizedPathname, NormalizedSearch } from './cache-key'
import { splitPathnameIntoParts } from './cache-key' import { splitPathnameIntoParts } from './cache-key'
import { import {
...@@ -112,8 +115,21 @@ type KnownRoutePartBase = { ...@@ -112,8 +115,21 @@ type KnownRoutePartBase = {
// learned its structure yet. // learned its structure yet.
pattern: FulfilledRouteCacheEntry | null pattern: FulfilledRouteCacheEntry | null
// True when parallel route branches disagree about the dynamic segment at
// this level — different param name or type, e.g. an @modal/[...catchAll]
// slot alongside [username]. The trie can only model one dynamic child per
// level, so prediction below this level would bind one branch's URL parts
// to another branch's params. Once set, discovery stops storing patterns
// beneath this level and matching bails out to server resolution.
//
// TODO: Consider including conflicting sibling dynamic params in the route
// tree, like we do for static siblings, and attempting to match both.
hasConflictingDynamicChildren: boolean
// TODO: For prefix rewrite support. When true, this part may not appear in // TODO: For prefix rewrite support. When true, this part may not appear in
// the candidate URL because it was injected by a rewrite. // the candidate URL because it was injected by a rewrite. Today, discovery
// refuses to store a pattern for such routes (see the cache key comparison
// in discoverKnownRoutePart); this field would let them be predicted.
// mayBeSkippedInURL: boolean // mayBeSkippedInURL: boolean
} }
...@@ -176,6 +192,7 @@ function createEmptyPart(): KnownRoutePart { ...@@ -176,6 +192,7 @@ function createEmptyPart(): KnownRoutePart {
dynamicChildParamName: null, dynamicChildParamName: null,
dynamicChildParamType: null, dynamicChildParamType: null,
pattern: null, pattern: null,
hasConflictingDynamicChildren: false,
} }
} }
...@@ -310,9 +327,11 @@ function handleMismatchDueToRewrite( ...@@ -310,9 +327,11 @@ function handleMismatchDueToRewrite(
} }
/** /**
* Gets or creates the dynamic child node for a KnownRoutePart. * Gets or creates the dynamic child node for a KnownRoutePart. A node can
* A node can have at most one dynamic child (you can't have both [slug] and * have at most one dynamic child. Sibling filesystem routes can't declare two
* [id] at the same route level), so we either return existing or create new. * different params at the same level, but parallel route branches can (e.g.
* @modal/[...catchAll] alongside [username]) — the caller detects that case
* and marks the level as conflicted instead of calling this.
*/ */
function discoverDynamicChild( function discoverDynamicChild(
part: KnownRoutePart, part: KnownRoutePart,
...@@ -410,6 +429,7 @@ function discoverKnownRoutePart( ...@@ -410,6 +429,7 @@ function discoverKnownRoutePart(
} else { } else {
// Dynamic segment tuple: [paramName, paramCacheKey, paramType, staticSiblings] // Dynamic segment tuple: [paramName, paramCacheKey, paramType, staticSiblings]
const paramName: string = segment[0] const paramName: string = segment[0]
const paramCacheKey: string = segment[1]
const paramType: DynamicParamTypesShort = segment[2] const paramType: DynamicParamTypesShort = segment[2]
const staticSiblings: readonly string[] | null = segment[3] const staticSiblings: readonly string[] | null = segment[3]
...@@ -453,6 +473,102 @@ function discoverKnownRoutePart( ...@@ -453,6 +473,102 @@ function discoverKnownRoutePart(
) )
} }
// The param's cache key holds the value parsed from the *rendered*
// pathname. If the URL part(s) this segment would consume don't equal
// that value, the response was rewrite-affected in a way that shifts
// which URL part maps to which segment (e.g. a proxy injected a leading
// locale segment). A static segment catches this above by failing to
// match its URL part; a dynamic segment consumes whatever part is in
// front of it, so compare against the rendered value instead. Bail out.
switch (paramType) {
case 'd': {
// Canonicalize the URL part to the same encoded form the server used
// for the cache key.
if (
urlPart !== null &&
canonicalizeURLPart(urlPart) !== paramCacheKey
) {
return handleMismatchDueToRewrite(
existingEntry,
now,
pathname,
search,
nextUrl,
fullTree,
metadataVaryPath,
couldBeIntercepted,
canonicalUrl,
supportsPerSegmentPrefetching
)
}
break
}
case 'c':
case 'oc': {
// Catch-alls consume every remaining URL part; their cache keys are
// the rendered parts joined with '/' (empty string for an empty
// optional catch-all). Comparing the joined remainder also catches a
// rewrite that appended segments the URL doesn't have.
const joinedRemainingParts = pathnameParts
.slice(partIndex)
.map(canonicalizeURLPart)
.join('/')
if (joinedRemainingParts !== paramCacheKey) {
return handleMismatchDueToRewrite(
existingEntry,
now,
pathname,
search,
nextUrl,
fullTree,
metadataVaryPath,
couldBeIntercepted,
canonicalUrl,
supportsPerSegmentPrefetching
)
}
break
}
case 'ci(..)(..)':
case 'ci(.)':
case 'ci(..)':
case 'ci(...)':
case 'di(..)(..)':
case 'di(.)':
case 'di(..)':
case 'di(...)':
// Interception params embed relative markers in their values, and
// patterns containing them are never used for prediction anyway (see
// matchKnownRoutePart), so skip the comparison.
break
default:
paramType satisfies never
}
if (
parentKnownRoutePart.hasConflictingDynamicChildren ||
(parentKnownRoutePart.dynamicChild !== null &&
(parentKnownRoutePart.dynamicChildParamName !== paramName ||
parentKnownRoutePart.dynamicChildParamType !== paramType))
) {
// A different parallel route branch already claimed the dynamic child
// at this level with a different param. Mark the level as conflicted
// so matching bails out, and don't store a pattern via this branch.
parentKnownRoutePart.hasConflictingDynamicChildren = true
return handleMismatchDueToRewrite(
existingEntry,
now,
pathname,
search,
nextUrl,
fullTree,
metadataVaryPath,
couldBeIntercepted,
canonicalUrl,
supportsPerSegmentPrefetching
)
}
// URL matches route structure. Build the known route tree. // URL matches route structure. Build the known route tree.
knownRoutePart = discoverDynamicChild( knownRoutePart = discoverDynamicChild(
parentKnownRoutePart, parentKnownRoutePart,
...@@ -779,8 +895,10 @@ function matchKnownRoutePart( ...@@ -779,8 +895,10 @@ function matchKnownRoutePart(
} }
} }
// Try dynamic child // Try dynamic child. Skip it entirely if parallel route branches disagree
if (part.dynamicChild !== null) { // about the dynamic segment at this level — any pattern stored beneath it
// was learned under a conflicting model.
if (part.dynamicChild !== null && !part.hasConflictingDynamicChildren) {
const dynamicPart = part.dynamicChild const dynamicPart = part.dynamicChild
const paramName = part.dynamicChildParamName const paramName = part.dynamicChildParamName
const paramType = part.dynamicChildParamType const paramType = part.dynamicChildParamType
......
...@@ -64,7 +64,7 @@ export function getRenderedPathname( ...@@ -64,7 +64,7 @@ export function getRenderedPathname(
// `encodeURIComponent` percent-encodes them. To produce the same canonical // `encodeURIComponent` percent-encodes them. To produce the same canonical
// form on the client (and avoid double-encoding `%xx` sequences such as // form on the client (and avoid double-encoding `%xx` sequences such as
// `%2F` → `%252F`), we decode the URL part first and re-encode it. // `%2F` → `%252F`), we decode the URL part first and re-encode it.
function canonicalizeURLPart(part: string): string { export function canonicalizeURLPart(part: string): string {
try { try {
return encodeURIComponent(decodeURIComponent(part)) return encodeURIComponent(decodeURIComponent(part))
} catch { } catch {
......
// The documented "close modal on navigation" pattern from the parallel
// routes docs: a catch-all in the root @modal slot that renders nothing.
export default function ModalCatchAll() {
return null
}
import { Suspense } from 'react'
import { LinkAccordion } from '../../../components/link-accordion'
async function Followers({
params,
}: {
params: Promise<{ username: string }>
}) {
const { username } = await params
const users = Array.from({ length: 10 }, (_, i) => `user${i}`)
return (
<section>
<h2 id="followers-heading">Followers of {username}</h2>
<ul>
{users.map((u) => (
<li key={u}>
<LinkAccordion href={`/${u}`}>{u}</LinkAccordion>
</li>
))}
</ul>
</section>
)
}
export default function FollowersPage({
params,
}: {
params: Promise<{ username: string }>
}) {
return (
<Suspense fallback={null}>
<Followers params={params} />
</Suspense>
)
}
import { Suspense } from 'react'
import { LinkAccordion } from '../../components/link-accordion'
// Params are read inside Suspense throughout this fixture so the routes can
// be prerendered when Cache Components is enabled (CI runs every suite with
// __NEXT_CACHE_COMPONENTS=true).
async function UserHeading({
params,
}: {
params: Promise<{ username: string }>
}) {
const { username } = await params
return <h1 id="user-layout-heading">{username}</h1>
}
export default function UserLayout({
children,
params,
}: {
children: React.ReactNode
params: Promise<{ username: string }>
}) {
return (
<div>
<header>
<Suspense fallback={null}>
<UserHeading params={params} />
</Suspense>
<LinkAccordion href="/viewer">my profile</LinkAccordion>
</header>
{children}
</div>
)
}
import { Suspense } from 'react'
async function Profile({ params }: { params: Promise<{ username: string }> }) {
const { username } = await params
return (
<section>
<h2 id="profile-heading">Profile of {username}</h2>
</section>
)
}
export default function ProfilePage({
params,
}: {
params: Promise<{ username: string }>
}) {
return (
<Suspense fallback={null}>
<Profile params={params} />
</Suspense>
)
}
export default function RootLayout({
children,
modal,
}: {
children: React.ReactNode
modal: React.ReactNode
}) {
return (
<html lang="en">
<body>
{children}
{modal}
</body>
</html>
)
}
import { LinkAccordion } from '../components/link-accordion'
export default function Home() {
return (
<main>
<h1>Home</h1>
<LinkAccordion href="/alice/followers">alice's followers</LinkAccordion>
</main>
)
}
'use client'
import Link, { type LinkProps } from 'next/link'
import { useState } from 'react'
export function LinkAccordion({
href,
children,
prefetch,
}: {
href: LinkProps['href']
children: React.ReactNode
prefetch?: LinkProps['prefetch']
}) {
// This component controls when prefetching happens by hiding the Link until
// the checkbox is clicked. Prefetch triggers when the Link enters the viewport.
const [isVisible, setIsVisible] = useState(false)
return (
<>
<input
type="checkbox"
checked={isVisible}
onChange={() => setIsVisible(!isVisible)}
data-link-accordion={href}
/>
{isVisible ? (
<Link href={href} prefetch={prefetch}>
{children}
</Link>
) : (
`${children} (link is hidden)`
)}
</>
)
}
import { nextTestSetup } from 'e2e-utils'
import { createRouterAct } from 'router-act'
import type { Page, Request } from 'playwright'
// Regression fixture for https://github.com/vercel/next.js/issues/97135
//
// Route shape (no rewrites, no special config):
// app/@modal/[...catchAll]/page <- catch-all in a root parallel slot
// app/[username]/followers/page <- current page nested under a sibling
// dynamic segment
//
// On an initial load of /alice/followers, Known Routes discovery
// (segment-cache/optimistic-routes.ts) walks BOTH parallel branches against
// the same URL parts. The @modal branch's [...catchAll] segment reuses the
// dynamic trie node that the children branch just created for [username]
// (discoverDynamicChild does not compare param name/type), absorbs all
// remaining URL parts, and stores the full /alice/followers route entry as
// the [username] node's direct pattern. Any subsequent 1-part URL
// (/user3, /viewer, ...) then matches that poisoned pattern and is predicted
// as the *followers* page, which the server can never satisfy -> the
// prefetch task livelocks, re-issuing the same request hundreds of times
// per second.
describe('modal-catchall-sibling-dynamic-prefetch-loop', () => {
const { next, isNextDev } = nextTestSetup({
files: __dirname,
})
if (isNextDev) {
// Only reproduces in a production build; dev does not predict routes
// from a learned pattern (it resolves routes on demand).
test('skipped in dev mode', () => {})
return
}
// On a buggy build the loop re-requests the same URL indefinitely, so
// `act` never settles. Instead of waiting for the jest timeout, fail fast
// once any single pathname has been requested this many times.
const MAX_RSC_REQUESTS_PER_PATH = 25
it('does not loop prefetches when a root slot catch-all shares a trie node with a sibling dynamic segment', async () => {
let act: ReturnType<typeof createRouterAct>
const rscRequestCounts = new Map<string, number>()
let excessRequests: string | null = null
let rejectOnExcess!: (error: Error) => void
const excessDetected = new Promise<never>((_, reject) => {
rejectOnExcess = reject
})
// Mark as handled in case it fires while no race is pending; the final
// assertion on `excessRequests` still catches it.
excessDetected.catch(() => {})
async function actExpectingToSettle<T>(
scope: () => Promise<T> | T,
config?: Parameters<typeof act>[1]
): Promise<unknown> {
const actPromise = act(scope, config)
// If the race rejects first, `act` keeps running until the page is
// torn down; swallow its eventual rejection so it doesn't surface as
// an unhandled promise rejection.
actPromise.catch(() => {})
return Promise.race([actPromise, excessDetected])
}
const browser = await next.browser('/alice/followers', {
beforePageLoad(page: Page) {
act = createRouterAct(page)
page.on('request', (request: Request) => {
const headers = request.headers()
if (headers['rsc'] === undefined) {
return
}
const pathname = new URL(request.url()).pathname
const count = (rscRequestCounts.get(pathname) ?? 0) + 1
rscRequestCounts.set(pathname, count)
if (count > MAX_RSC_REQUESTS_PER_PATH && excessRequests === null) {
excessRequests =
`Observed more than ${MAX_RSC_REQUESTS_PER_PATH} RSC requests ` +
`for ${pathname} (state tree of last request: ` +
`${headers['next-router-state-tree'] ?? '<none>'})`
rejectOnExcess(
new Error(
'Prefetch livelock detected: the client keeps re-requesting ' +
'the same URL. ' +
excessRequests
)
)
}
})
},
})
// The initial page load is when the client learns the (poisoned) route
// pattern for /alice/followers.
expect(await browser.elementById('followers-heading').text()).toBe(
'Followers of alice'
)
// Reveal a single link whose URL matches the poisoned pattern. Pre-fix,
// this immediately enters the livelock (~hundreds of requests/sec).
await actExpectingToSettle(async () => {
const toggle = await browser.elementByCss(
'input[data-link-accordion="/user3"]'
)
await toggle.click()
})
expect(excessRequests).toBeNull()
// Reveal several more links in one act scope to recreate the issue's
// many-visible-links scenario (concurrent prefetch tasks all matching
// the same poisoned pattern).
await actExpectingToSettle(async () => {
for (const href of ['/user0', '/user1', '/user5', '/viewer']) {
const toggle = await browser.elementByCss(
`input[data-link-accordion="${href}"]`
)
await toggle.click()
}
})
expect(excessRequests).toBeNull()
// Navigate to one of the prefetched links and confirm it renders the
// real profile page (not a mispredicted followers page).
const link = await browser.elementByCss('a[href="/user3"]')
await link.click()
expect(await browser.elementById('profile-heading').text()).toBe(
'Profile of user3'
)
expect(excessRequests).toBeNull()
})
})
/** @type {import('next').NextConfig} */
// Intentionally default config: the reproduction in
// https://github.com/vercel/next.js/issues/97135 requires no rewrites, no
// cacheComponents, and no special flags. Optimistic routing (Known Routes
// discovery) is enabled by default.
module.exports = {}
...@@ -131,6 +131,22 @@ export default function Home() { ...@@ -131,6 +131,22 @@ export default function Home() {
</li> </li>
</ul> </ul>
<h2>Rewrite Detection (Prefetch Misprediction)</h2>
<p>
/products/promo/[id] is rewritten to /products/sale/[id]. The rewrite
preserves the /products/[category]/[id] shape, so a pattern learned from
a non-rewritten product URL will match and mispredict category as
&quot;promo&quot;. The prefetch response must detect the mismatch and
mark the pattern instead of rendering the wrong params.
</p>
<ul>
<li>
<LinkAccordion href="/products/promo/gadget" prefetch={true}>
Promo Gadget (rewritten to sale/gadget)
</LinkAccordion>
</li>
</ul>
<h2>Rewrite Detection (Search Params)</h2> <h2>Rewrite Detection (Search Params)</h2>
<p> <p>
These URLs are rewritten based on search params. /search-rewrite?v=X These URLs are rewritten based on search params. /search-rewrite?v=X
......
...@@ -392,6 +392,66 @@ describe('optimistic-routing', () => { ...@@ -392,6 +392,66 @@ describe('optimistic-routing', () => {
]) ])
}) })
it('rewrite detection (prefetch): detects mispredicted prefetch when a shape-preserving rewrite changes the params', async () => {
let act: ReturnType<typeof createRouterAct>
const browser = await next.browser('/', {
beforePageLoad(page) {
act = createRouterAct(page)
},
})
// Step 1: Prefetch /products/electronics/phone-1 to learn the
// /products/[category]/[id] pattern. This URL is not rewritten, so the
// pattern can be used for prediction.
const revealProduct1 = await browser.elementByCss(
'input[data-link-accordion="/products/electronics/phone-1"]'
)
await act(
async () => {
await revealProduct1.click()
},
{
includes: 'Loading',
}
)
// Step 2: Reveal /products/promo/gadget (prefetch={true}). The URL
// matches the learned pattern, so the client predicts the route with
// category="promo". But the proxy rewrites this URL to
// /products/sale/gadget — shape-preserving, so undetectable at learn
// time. The prefetch response must reveal the mismatch; the client marks
// the pattern and re-prefetches using server resolution.
await act(async () => {
const revealPromo = await browser.elementByCss(
'input[data-link-accordion="/products/promo/gadget"]'
)
await revealPromo.click()
})
// Step 3: Navigate. The page must render the server's params
// (category="sale") with no intermediate render of the mispredicted
// params (category="promo").
const linkPromo = await browser.elementByCss(
'a[href="/products/promo/gadget"]'
)
await act(async () => {
await linkPromo.click()
})
const productTitle = await browser.elementById('product-title')
expect(await productTitle.text()).toBe('Product: sale/gadget')
// If the router had rendered the mispredicted route tree, we'd see an
// entry with category "promo" before the corrected one.
expect(await getRenderedRouteHistory(browser)).toEqual([
{ url: '/', params: {} },
{
url: '/products/promo/gadget',
params: { category: 'sale', id: 'gadget' },
},
])
})
it('rewrite detection (search params): does not use cached pattern when search params cause different rewrite', async () => { it('rewrite detection (search params): does not use cached pattern when search params cause different rewrite', async () => {
let act: ReturnType<typeof createRouterAct> let act: ReturnType<typeof createRouterAct>
const browser = await next.browser('/', { const browser = await next.browser('/', {
......
...@@ -14,6 +14,16 @@ export function proxy(request: NextRequest) { ...@@ -14,6 +14,16 @@ export function proxy(request: NextRequest) {
return NextResponse.rewrite(url) return NextResponse.rewrite(url)
} }
// Rewrite /products/promo/[id] to /products/sale/[id]
// Unlike /rewritten/* above, this preserves the URL's shape: both sides
// match /products/[category]/[id], so the rewrite is only discoverable from
// the response. Exercises the prefetch-side mispredict detection.
if (pathname.startsWith('/products/promo/')) {
const url = request.nextUrl.clone()
url.pathname = pathname.replace('/products/promo/', '/products/sale/')
return NextResponse.rewrite(url)
}
// Rewrite /search-rewrite?v=alpha to /rewrite-alpha // Rewrite /search-rewrite?v=alpha to /rewrite-alpha
// Rewrite /search-rewrite?v=beta to /rewrite-beta // Rewrite /search-rewrite?v=beta to /rewrite-beta
// This tests that search param rewrites are correctly detected. // This tests that search param rewrites are correctly detected.
...@@ -31,5 +41,5 @@ export function proxy(request: NextRequest) { ...@@ -31,5 +41,5 @@ export function proxy(request: NextRequest) {
} }
export const config = { export const config = {
matcher: ['/rewritten/:path*', '/search-rewrite'], matcher: ['/rewritten/:path*', '/search-rewrite', '/products/promo/:path*'],
} }
import { Suspense } from 'react'
import { LinkAccordion } from '../../../components/link-accordion'
// One accordion-hidden link per ancestor of the current catch-all path.
// `/one/two` is the interesting one: it has enough URL parts to fully match
// the learned `/[locale]/[...pages]` pattern, so a buggy client predicts the
// route (binding `locale` to "one") instead of asking the server. The links
// are hidden behind LinkAccordion checkboxes so the test controls exactly
// when each prefetch happens (inside an `act` scope).
async function Content({
params,
}: Pick<PageProps<'/[locale]/[...pages]'>, 'params'>) {
const { locale, pages } = await params
return (
<>
{/* A param-derived string with a terminator, unique per path, so the
test can assert that a prefetch response was rendered with the
server-resolved params (locale "en"), not a mispredicted binding. */}
<p id="params">{`params:${locale}:${pages.join('/')}:end`}</p>
<nav>
{pages.slice(0, -1).map((_, index) => {
const href = `/${pages.slice(0, index + 1).join('/')}`
return (
<div key={href}>
<LinkAccordion href={href}>{href}</LinkAccordion>
</div>
)
})}
</nav>
</>
)
}
export default function Page(props: PageProps<'/[locale]/[...pages]'>) {
return (
<main>
<Suspense fallback={<p>loading content</p>}>
<Content params={props.params} />
</Suspense>
<h1 id="page-title">Page content</h1>
</main>
)
}
import { locale } from 'next/root-params'
export function generateStaticParams() {
return [{ locale: 'en' }]
}
export default async function RootLayout({
children,
}: LayoutProps<'/[locale]'>) {
// An i18n library reads the locale root param to pick up the active locale
// (e.g. next-intl's getRequestConfig). Without this read the param is
// unused and the bug does not reproduce.
const activeLocale = await locale()
return (
<html lang={activeLocale}>
<body>{children}</body>
</html>
)
}
'use client'
import Link, { type LinkProps } from 'next/link'
import { useState } from 'react'
export function LinkAccordion({
href,
children,
prefetch,
}: {
href: LinkProps['href']
children: React.ReactNode
prefetch?: LinkProps['prefetch']
}) {
// This component controls when prefetching happens by hiding the Link until
// the checkbox is clicked. Prefetch triggers when the Link enters the viewport.
const [isVisible, setIsVisible] = useState(false)
return (
<>
<input
type="checkbox"
checked={isVisible}
onChange={() => setIsVisible(!isVisible)}
data-link-accordion={href}
/>
{isVisible ? (
<Link href={href} prefetch={prefetch}>
{children}
</Link>
) : (
`${children} (link is hidden)`
)}
</>
)
}
/** @type {import('next').NextConfig} */
module.exports = {
cacheComponents: true,
partialPrefetching: true,
}
import { nextTestSetup } from 'e2e-utils'
import { retry } from 'next-test-utils'
import { createRouterAct } from 'router-act'
import type { Page, Request } from 'playwright'
const NEXT_ROUTER_STATE_TREE_HEADER = 'next-router-state-tree'
// The proxy rewrites every request to `/en/...`, so `locale` is always "en".
// A request whose state tree binds `locale` to anything else matched the
// `/[locale]/[...pages]` pattern without accounting for the injected segment.
const MISPREDICTED_TREE_PATTERN = /"locale","(?!en")/
function decodeStateTree(value: string | undefined): string | null {
if (value === undefined) {
return null
}
// The header is sent percent-encoded.
try {
return decodeURIComponent(value)
} catch {
return value
}
}
describe('proxy-prefix-rewrite-prefetch-loop', () => {
const { next, isNextDev } = nextTestSetup({
files: __dirname,
})
if (isNextDev) {
// Only reproduces in a production build; dev does not predict routes from
// a learned pattern (it resolves routes on demand).
test('skipped in dev mode', () => {})
return
}
it('does not prefetch an ancestor link in a loop when the proxy injects a path segment', async () => {
let act: ReturnType<typeof createRouterAct>
// Every router request must carry a state tree with the server-resolved
// locale ("en"). Pre-fix, revealing a link that fully matched the learned
// pattern issued a prefetch with a mispredicted tree (locale bound to
// "one") that the server could never satisfy, so the prefetch task
// retried it in a tight serial loop. Under that loop, `act`'s flush keeps
// observing new requests and never settles — so instead of waiting for a
// jest timeout, we race each `act` against a promise that rejects as soon
// as the first mispredicted request is observed. This fails fast with a
// legible error on a buggy build and is a no-op on a correct one.
const mispredictedTrees: string[] = []
let rejectOnMisprediction!: (error: Error) => void
const mispredictionDetected = new Promise<never>((_, reject) => {
rejectOnMisprediction = reject
})
// Mark the rejection as handled in case it fires while no race is
// pending; the final assertion on `mispredictedTrees` still catches it.
mispredictionDetected.catch(() => {})
async function actExpectingNoMisprediction<T>(
scope: () => Promise<T> | T,
config?: Parameters<typeof act>[1]
): Promise<unknown> {
const actPromise = act(scope, config)
// If the race rejects first, `act` keeps running until the page is torn
// down; swallow its eventual rejection so it doesn't surface as an
// unhandled promise rejection.
actPromise.catch(() => {})
return Promise.race([actPromise, mispredictionDetected])
}
const browser = await next.browser('/one/two/three/four', {
beforePageLoad(page: Page) {
act = createRouterAct(page)
page.on('request', (request: Request) => {
const headers = request.headers()
if (headers['rsc'] === undefined) {
return
}
const stateTree = decodeStateTree(
headers[NEXT_ROUTER_STATE_TREE_HEADER]
)
if (stateTree !== null && MISPREDICTED_TREE_PATTERN.test(stateTree)) {
mispredictedTrees.push(stateTree)
rejectOnMisprediction(
new Error(
'Observed a router request whose next-router-state-tree ' +
'binds `locale` to a value other than "en". The proxy ' +
'rewrites every path to /en/..., so the server can never ' +
'satisfy this tree; a client that issues it predicted the ' +
'route without accounting for the injected segment.\n\n' +
`State tree: ${stateTree}`
)
)
}
})
},
})
// The initial page load renders the proxied route (`/en/one/two/three/
// four`) and is when the client learns (or, with the fix, declines to
// learn) the `/[locale]/[...pages]` pattern for `/one/two/three/four`.
expect(await browser.elementById('params').text()).toBe(
'params:en:one/two/three/four:end'
)
// Reveal the `/one/two` link. It is the first ancestor with enough URL
// parts to fully match the learned pattern (2 parts = [locale] + at least
// one for [...pages]). The client must not predict the route from the
// pattern; it must ask the server instead, via a route tree (`/_tree`)
// prefetch — asserted by matching the route trie shape in a static
// prefetch response. Pre-fix, no `/_tree` request happens at all: the
// pattern fully matches, so the client predicts the route and issues the
// mispredicted runtime prefetch that loops forever; the race above turns
// that into a fast, descriptive failure. `act` returning at all also
// proves the prefetch queue settled instead of looping.
await actExpectingNoMisprediction(
async () => {
const toggle = await browser.elementByCss(
'input[data-link-accordion="/one/two"]'
)
await toggle.click()
},
{ includes: '"name":"pages"', kind: 'static' }
)
// Navigate to the prefetched ancestor. The navigation response must
// contain content rendered with the server-resolved params (locale "en",
// pages ["one", "two"]) — proving the request tree wasn't shaped by a
// mispredicted binding of locale to "one".
await actExpectingNoMisprediction(
async () => {
const link = await browser.elementByCss('a[href="/one/two"]')
await link.click()
},
{ includes: 'params:en:one/two:end' }
)
// Both the old and new page render `#params`, so wait for the soft
// navigation to commit the new content.
await retry(async () => {
expect(await browser.elementById('params').text()).toBe(
'params:en:one/two:end'
)
})
// The sharpest signal of the bug: no request during the entire test ever
// carried a mispredicted state tree.
expect(mispredictedTrees).toEqual([])
})
})
import { NextResponse, type NextRequest } from 'next/server'
// Injects a locale as a leading path segment, so the rendered pathname always
// has one more segment than the requested one. This is what i18n libraries do
// when the default locale is hidden from the URL (e.g. next-intl's
// `localePrefix: 'as-needed'`).
export default function proxy(request: NextRequest) {
const { pathname } = request.nextUrl
if (pathname.startsWith('/en')) {
return NextResponse.next()
}
const url = request.nextUrl.clone()
url.pathname = `/en${pathname}`
return NextResponse.rewrite(url)
}
export const config = {
matcher: ['/((?!_next|_vercel|.*\\..*).*)'],
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment