Unverified Commit 12b802b2 authored by Andrew Clark's avatar Andrew Clark Committed by GitHub

Convert per-segment prefetches to NavigationFlightResponse format (#96877)

This is the last of the changes to convert all RSC requests to a single
unified response type, NavigationFlightResponse.

The old format was a positional array whose ordering had to be kept in
sync by hand between the server and the client. The new format is the
same root-anchored tree used by every other response, so the dedicated
client write path for per-segment responses is deleted and these
responses go through the shared one.

A few functions are renamed as their roles change:
convertServerPatchToFullTree becomes createNavigationSeed (it now also
accepts trees with no base to overlay), fetchSegmentsOnCacheMiss becomes
fetchSegmentPrefetchesUsingStaticRequest (the counterpart of
fetchSegmentPrefetchesUsingRuntimeRequest), and writeSeedDataIntoCache
becomes writeTreeDataIntoCache (seed data no longer exists as a
concept).

During the unification, a few inconsistencies and oversights were caught
and fixed: response data with no matching pending entry is written as a
detached entry instead of dropped, and a response's byte size is spread
across the entries it actually fulfilled rather than the old chain-based
count. The fallback retry loop also marks itself pending before the
first cache write instead of after, closing a window where the scheduler
could spawn a duplicate revalidation.

The second commit unifies cache keying for prefetched segments. There
were two implementations of the logic that decides which cache key a
prefetched segment is written to, one for per-segment responses and one
for responses produced by a live render. Now there's one: if the server
reported which params the segment depends on, key the entry using those;
otherwise fall back to the keying implied by how the payload was
fetched. A segment is only treated as param-independent when the server
says so. Two precedence fixes were needed to merge them, and a few other
inconsistencies found during the unification are fixed along the way;
see the commit message for details.
parent 0c5881a3
...@@ -1469,5 +1469,6 @@ ...@@ -1469,5 +1469,6 @@
"1468": "Missing workUnitStore in createComponentTree", "1468": "Missing workUnitStore in createComponentTree",
"1469": "The payload's transport tree is missing a slot that exists in the loader tree: %s", "1469": "The payload's transport tree is missing a slot that exists in the loader tree: %s",
"1470": "`deflateResumeDataCache` should not be called in edge runtime.", "1470": "`deflateResumeDataCache` should not be called in edge runtime.",
"1471": "Cannot resolve a dynamic segment that has no param value: the response provides no rendered pathname to parse it from." "1471": "Cannot resolve a dynamic segment that has no param value: the response provides no rendered pathname to parse it from.",
"1472": "Cannot convert a server response with no transport data and no base tree."
} }
...@@ -86,6 +86,15 @@ export function createInitialRouterState({ ...@@ -86,6 +86,15 @@ export function createInitialRouterState({
// There's no base tree to overlay onto; the initial payload is a full // There's no base tree to overlay onto; the initial payload is a full
// render from the root. // render from the root.
null, null,
// The initial payload may still be streaming in while we hydrate, so its
// vary params can't be drained here — and nothing consumes them from
// this tree. The segment-cache write below re-decodes the transport data
// with the payload's root params once the stale time has resolved.
null,
// Same for partiality: only segment-cache writes consume it, and the
// write below re-decodes with the payload's actual response-level value.
// Pass the conservative value here.
true,
// The initial payload always includes the param values in the tree // The initial payload always includes the param values in the tree
// (fallback shells are patched with the parsed values before this runs — // (fallback shells are patched with the parsed values before this runs —
// see createInitialRSCPayloadFromFallbackPrerender), so there's no // see createInitialRSCPayloadFromFallbackPrerender), so there's no
...@@ -127,8 +136,9 @@ export function createInitialRouterState({ ...@@ -127,8 +136,9 @@ export function createInitialRouterState({
// Intentionally holding off on doing this until we decide how the Cached // Intentionally holding off on doing this until we decide how the Cached
// Navigations behavior should work in combination with App Shells. // Navigations behavior should work in combination with App Shells.
// Write the initial seed data into the segment cache so subsequent // Write the initial payload's segment data into the segment cache so
// navigations to the initial page can serve cached segments instantly. // subsequent navigations to the initial page can serve cached
// segments instantly.
if (initialStaleTime !== undefined) { if (initialStaleTime !== undefined) {
if ( if (
initialStaticStageByteLength !== undefined && initialStaticStageByteLength !== undefined &&
...@@ -161,17 +171,19 @@ export function createInitialRouterState({ ...@@ -161,17 +171,19 @@ export function createInitialRouterState({
// rendered normally, we just won't write into the cache. // rendered normally, we just won't write into the cache.
}) })
} else { } else {
// Fully static page — cache the entire decoded seed data as-is. We're // Fully static page — cache the initial payload's segment data as-is.
// not using the initial response here (which would allow us to combine // We're not using the initial response here (which would allow us to
// the two branches) to avoid unnecessary decoding of the Flight data, // combine the two branches) to avoid unnecessary decoding of the
// since we can just take the seed data that we already decoded during // Flight data, since we can just take the segment data that we
// hydration and write it into the cache directly. // already decoded during hydration and write it into the
// cache directly.
spawnStaticStageCacheWrite( spawnStaticStageCacheWrite(
Date.now(), Date.now(),
// The transport subset of the initial payload, already decoded // The transport subset of the initial payload, already decoded
// during hydration. Synthesized so the helper can resolve the // during hydration. `u` (the runtime-data verdict) is deliberately
// stale time from the response's own `s` field, the same as the // omitted from this synthesized subset — its writes record their
// truncated branch above. // strategy unrefined — while the truncated branch above forwards
// the decoded payload's own `u`.
{ {
t: initialTransportData, t: initialTransportData,
r: initialRootVaryParams, r: initialRootVaryParams,
......
...@@ -21,7 +21,7 @@ import { ...@@ -21,7 +21,7 @@ import {
import { isNavigatingToNewRootLayout } from './is-navigating-to-new-root-layout' import { isNavigatingToNewRootLayout } from './is-navigating-to-new-root-layout'
import { getLastCommittedTree } from './reducers/committed-state' import { getLastCommittedTree } from './reducers/committed-state'
import { import {
convertServerPatchToFullTree, createNavigationSeed,
type NavigationSeed, type NavigationSeed,
} from '../segment-cache/decode-server-response' } from '../segment-cache/decode-server-response'
import { import {
...@@ -1818,10 +1818,16 @@ async function fetchMissingDynamicData( ...@@ -1818,10 +1818,16 @@ async function fetchMissingDynamicData(
} }
const now = Date.now() const now = Date.now()
const seed = convertServerPatchToFullTree( const seed = createNavigationSeed(
now, now,
task.route, task.route,
result.transportData, result.transportData,
// Navigation responses stream in incrementally, so their vary params
// can't be drained here — and nothing consumes them from a navigation
// seed (only segment-cache writes read vary params, and those decode
// their own, buffered, payloads).
null,
result.isResponsePartial,
// Navigation responses always include the param values in the tree, so // Navigation responses always include the param values in the tree, so
// there's no pathname to parse them from (nor a need to). // there's no pathname to parse them from (nor a need to).
null, null,
...@@ -1946,7 +1952,7 @@ function writeDynamicDataIntoNavigationTask( ...@@ -1946,7 +1952,7 @@ function writeDynamicDataIntoNavigationTask(
): boolean { ): boolean {
// A non-null data object means the response accounted for this segment, // A non-null data object means the response accounted for this segment,
// even if it didn't render it (data.rsc may still be null, e.g. for the // even if it didn't render it (data.rsc may still be null, e.g. for the
// intermediate segments on the path to a patched subtree). // intermediate segments on the path to a rendered subtree).
const dynamicData = serverRouteTree.data const dynamicData = serverRouteTree.data
if (task.status === NavigationTaskStatus.Pending && dynamicData !== null) { if (task.status === NavigationTaskStatus.Pending && dynamicData !== null) {
task.status = NavigationTaskStatus.Fulfilled task.status = NavigationTaskStatus.Fulfilled
......
...@@ -5,7 +5,7 @@ import type { ...@@ -5,7 +5,7 @@ import type {
} from '../router-reducer-types' } from '../router-reducer-types'
import { ScrollBehavior } from '../router-reducer-types' import { ScrollBehavior } from '../router-reducer-types'
import { navigateToKnownRoute } from '../../segment-cache/navigation' import { navigateToKnownRoute } from '../../segment-cache/navigation'
import { convertServerPatchToFullTree } from '../../segment-cache/decode-server-response' import { createNavigationSeed } from '../../segment-cache/decode-server-response'
import { import {
invalidateSegmentCacheEntries, invalidateSegmentCacheEntries,
segmentCacheMap, segmentCacheMap,
...@@ -73,13 +73,15 @@ export function refreshDynamicData( ...@@ -73,13 +73,15 @@ export function refreshDynamicData(
const now = Date.now() const now = Date.now()
// TODO: Store the dynamic stale time on the top-level state so it's known // TODO: Store the dynamic stale time on the top-level state so it's known
// during restores and refreshes. // during restores and refreshes.
const refreshSeed = convertServerPatchToFullTree( const refreshSeed = createNavigationSeed(
now, now,
currentFlightRouterState, currentFlightRouterState,
// No transport data (and so no pathname to parse params from) — this // No transport data (and so no vary params, no partiality, and no
// converts the base tree alone. // pathname to parse params from) — this converts the base tree alone.
null, null,
null, null,
true,
null,
currentRenderedSearch, currentRenderedSearch,
UnknownDynamicStaleTime UnknownDynamicStaleTime
) )
......
...@@ -16,7 +16,7 @@ import { ...@@ -16,7 +16,7 @@ import {
completeHardNavigation, completeHardNavigation,
completeTraverseNavigation, completeTraverseNavigation,
} from '../../segment-cache/navigation' } from '../../segment-cache/navigation'
import { convertServerPatchToFullTree } from '../../segment-cache/decode-server-response' import { createNavigationSeed } from '../../segment-cache/decode-server-response'
import { segmentCacheMap } from '../../segment-cache/cache' import { segmentCacheMap } from '../../segment-cache/cache'
import { UnknownDynamicStaleTime } from '../../segment-cache/bfcache' import { UnknownDynamicStaleTime } from '../../segment-cache/bfcache'
...@@ -53,13 +53,15 @@ export function restoreReducer( ...@@ -53,13 +53,15 @@ export function restoreReducer(
separateRefreshUrls: null, separateRefreshUrls: null,
scrollRef: null, scrollRef: null,
} }
const restoreSeed = convertServerPatchToFullTree( const restoreSeed = createNavigationSeed(
now, now,
treeToRestore, treeToRestore,
// No transport data (and so no pathname to parse params from) — this // No transport data (and so no vary params, no partiality, and no
// converts the base tree alone. // pathname to parse params from) — this converts the base tree alone.
null, null,
null, null,
true,
null,
renderedSearch, renderedSearch,
UnknownDynamicStaleTime UnknownDynamicStaleTime
) )
......
...@@ -57,7 +57,7 @@ import { ...@@ -57,7 +57,7 @@ import {
navigateToKnownRoute, navigateToKnownRoute,
navigate, navigate,
} from '../../segment-cache/navigation' } from '../../segment-cache/navigation'
import { convertServerPatchToFullTree } from '../../segment-cache/decode-server-response' import { createNavigationSeed } from '../../segment-cache/decode-server-response'
import { discoverKnownRoute } from '../../segment-cache/optimistic-routes' import { discoverKnownRoute } from '../../segment-cache/optimistic-routes'
import type { NormalizedSearch } from '../../segment-cache/cache-key' import type { NormalizedSearch } from '../../segment-cache/cache-key'
import { import {
...@@ -477,10 +477,19 @@ export function serverActionReducer( ...@@ -477,10 +477,19 @@ export function serverActionReducer(
const now = Date.now() const now = Date.now()
// TODO: Store the dynamic stale time on the top-level state so it's // TODO: Store the dynamic stale time on the top-level state so it's
// known during restores and refreshes. // known during restores and refreshes.
const redirectSeed = convertServerPatchToFullTree( const redirectSeed = createNavigationSeed(
now, now,
currentFlightRouterState, currentFlightRouterState,
flightData, flightData,
// Action responses stream in incrementally, so their vary params
// can't be drained here — and nothing consumes them from a
// navigation seed (only segment-cache writes read vary params, and
// those decode their own, buffered, payloads).
null,
// Same for partiality: only segment-cache writes consume it, and
// action responses are never written to the segment cache. Pass
// the conservative value.
true,
// Navigation responses always include the param values in the // Navigation responses always include the param values in the
// tree, so there's no pathname to parse them from (nor a need to). // tree, so there's no pathname to parse them from (nor a need to).
null, null,
......
...@@ -37,7 +37,7 @@ import { isJavaScriptURLString } from '../../lib/javascript-url' ...@@ -37,7 +37,7 @@ import { isJavaScriptURLString } from '../../lib/javascript-url'
import { UnknownDynamicStaleTime, computeDynamicStaleAt } from './bfcache' import { UnknownDynamicStaleTime, computeDynamicStaleAt } from './bfcache'
import { createLinkPrefetchPartialError } from '../../../shared/lib/instant-messages' import { createLinkPrefetchPartialError } from '../../../shared/lib/instant-messages'
import { import {
convertServerPatchToFullTree, createNavigationSeed,
type NavigationSeed, type NavigationSeed,
} from './decode-server-response' } from './decode-server-response'
...@@ -410,6 +410,7 @@ function navigateUsingPrefetchedRouteTree( ...@@ -410,6 +410,7 @@ function navigateUsingPrefetchedRouteTree(
head: null, head: null,
isHeadPartial: true, isHeadPartial: true,
headVaryParams: null, headVaryParams: null,
headStaleTimeSeconds: null,
dynamicStaleAt: computeDynamicStaleAt(now, UnknownDynamicStaleTime), dynamicStaleAt: computeDynamicStaleAt(now, UnknownDynamicStaleTime),
// Not derived from a server response; no base to diverge from. // Not derived from a server response; no base to diverge from.
treeDivergedFromBase: false, treeDivergedFromBase: false,
...@@ -522,10 +523,16 @@ async function navigateToUnknownRoute( ...@@ -522,10 +523,16 @@ async function navigateToUnknownRoute(
// Since the response format of dynamic requests and prefetches is slightly // Since the response format of dynamic requests and prefetches is slightly
// different, we'll need to massage the data a bit. Create FlightRouterState // different, we'll need to massage the data a bit. Create FlightRouterState
// tree that simulates what we'd receive as the result of a prefetch. // tree that simulates what we'd receive as the result of a prefetch.
const navigationSeed = convertServerPatchToFullTree( const navigationSeed = createNavigationSeed(
now, now,
currentFlightRouterState, currentFlightRouterState,
transportData, transportData,
// Navigation responses stream in incrementally, so their vary params
// can't be drained here — and nothing consumes them from a navigation
// seed (only segment-cache writes read vary params, and those decode
// their own, buffered, payloads).
null,
isResponsePartial,
// Navigation responses always include the param values in the tree, so // Navigation responses always include the param values in the tree, so
// there's no pathname to parse them from (nor a need to). // there's no pathname to parse them from (nor a need to).
null, null,
......
...@@ -56,6 +56,19 @@ export function getRenderedPathname( ...@@ -56,6 +56,19 @@ export function getRenderedPathname(
) as NormalizedPathname ) as NormalizedPathname
} }
/**
* Like getRenderedPathname, but derived from the request URL rather than the
* response. Used in output: "export" mode, where the response URL has the
* segment filename appended to the pathname — and where rewrites don't
* exist, so the request pathname is always the rendered pathname.
*/
export function getPathnameFromRequestURL(url: URL): NormalizedPathname {
const pathname = url.pathname
return (
hasBasePath(pathname) ? removeBasePath(pathname) : pathname
) as NormalizedPathname
}
// Pathname parts come from `URL.pathname.split('/')`, so they are already // Pathname parts come from `URL.pathname.split('/')`, so they are already
// in the encoded form the URL parser produces. The server-side equivalent // in the encoded form the URL parser produces. The server-side equivalent
// (`get-dynamic-param.ts`) starts from a decoded param value and applies // (`get-dynamic-param.ts`) starts from a decoded param value and applies
......
...@@ -839,7 +839,7 @@ async function generateDynamicRSCPayload( ...@@ -839,7 +839,7 @@ async function generateDynamicRSCPayload(
baseResponse.a = options.shellByteLengthPromise baseResponse.a = options.shellByteLengthPromise
} }
if (options?.shellUsedSessionDataPromise !== undefined) { if (options?.shellUsedSessionDataPromise !== undefined) {
baseResponse.u = options.shellUsedSessionDataPromise baseResponse.w = options.shellUsedSessionDataPromise
} }
if (options?.runtimePrefetchStream !== undefined) { if (options?.runtimePrefetchStream !== undefined) {
...@@ -10449,6 +10449,7 @@ async function collectSegmentData( ...@@ -10449,6 +10449,7 @@ async function collectSegmentData(
// measure, but the static-prefetch hint still rides the manifest — // measure, but the static-prefetch hint still rides the manifest —
// collectPrefetchHints then only builds the tree shape carrying it. // collectPrefetchHints then only builds the tree shape carrying it.
hints = await ComponentMod.collectPrefetchHints( hints = await ComponentMod.collectPrefetchHints(
renderOpts.cacheComponents,
fullPageDataBuffer, fullPageDataBuffer,
staleTime, staleTime,
clientModules, clientModules,
......
...@@ -730,7 +730,10 @@ type SegmentData = { ...@@ -730,7 +730,10 @@ type SegmentData = {
function createSegmentData(data: TransportSegmentData): SegmentData { function createSegmentData(data: TransportSegmentData): SegmentData {
return { return {
node: data.r, node: data.r,
isPartial: data.p, // Page payloads always carry the boolean form of `p`; the promise
// encoding only appears in per-segment prefetch responses. Treat it
// conservatively as partial if it ever shows up here.
isPartial: typeof data.p === 'boolean' ? data.p : true,
varyParams: data.v, varyParams: data.v,
} }
} }
......
...@@ -218,7 +218,7 @@ export const enum PrefetchHint { ...@@ -218,7 +218,7 @@ export const enum PrefetchHint {
// it separately. Set at build time by the segment size measurement pass. // it separately. Set at build time by the segment size measurement pass.
InlinedIntoChild = 0b1000000, InlinedIntoChild = 0b1000000,
// On a __PAGE__: this page's response includes the head (metadata/viewport) // On a __PAGE__: this page's response includes the head (metadata/viewport)
// at the end of its SegmentPrefetch[] array. // in its transport data (PartialTransportData's `h`).
HeadInlinedIntoSelf = 0b10000000, HeadInlinedIntoSelf = 0b10000000,
// On the root hint node: the head was NOT inlined into any page — fetch // On the root hint node: the head was NOT inlined into any page — fetch
// it separately. Absence of this bit means the head is bundled into a page. // it separately. Absence of this bit means the head is bundled into a page.
...@@ -248,9 +248,9 @@ export const enum PrefetchHint { ...@@ -248,9 +248,9 @@ export const enum PrefetchHint {
// This segment or one of its descendants prefetches "eagerly" — i.e. its // This segment or one of its descendants prefetches "eagerly" — i.e. its
// effective prefetch strategy is anything other than 'partial'. Used by // effective prefetch strategy is anything other than 'partial'. Used by
// App Shells: a non-eager subtree relies on the shared app shell and skips // the scheduler's phasing: a non-eager subtree relies on the shell the
// its Speculative prefetch. Propagates upward so the root reflects the // Shell phase prefetches and skips its Speculative prefetch. Propagates
// entire subtree. // upward so the root reflects the entire subtree.
SubtreeHasEagerPrefetch = 0b1000000000000, SubtreeHasEagerPrefetch = 0b1000000000000,
// This segment or one of its descendants exports `instant = false`, // This segment or one of its descendants exports `instant = false`,
// explicitly opting out of Partial Prefetching. Propagates upward so the root // explicitly opting out of Partial Prefetching. Propagates upward so the root
...@@ -284,15 +284,15 @@ export const enum PrefetchHint { ...@@ -284,15 +284,15 @@ export const enum PrefetchHint {
/** /**
* Bitmask for checking whether a segment's static prefetch is skipped — i.e. * Bitmask for checking whether a segment's static prefetch is skipped — i.e.
* the server emits no static data for it (its slot in a segment bundle is * the server emits no static data for it (its node in a segment response
* null, and it participates in the bundle chain only as a pass-through) and * carries identity only, and it participates in a bundle only as a
* the client never issues a static request for it. * pass-through) and the client never issues a static request for it.
* *
* Static prefetching is disabled ONLY by `prefetch: 'force-disabled'` * Static prefetching is disabled ONLY by `prefetch: 'force-disabled'`
* (PrefetchDisabled). Notably, Partial Prefetching segments DO have static * (PrefetchDisabled). Notably, Partial Prefetching segments DO have static
* data even though they require runtime completeness: the server emits it * data even though they require runtime completeness: the server emits it
* unconditionally — it can't be gated on the ShouldAttemptStaticPrefetch * unconditionally — it can't be gated on the ShouldAttemptStaticPrefetch
* hint, because null-slot positions in segment bundles must be deterministic * hint, because which segments carry static data must be deterministic
* from build-time config. A runtime request may still be needed for the * from build-time config. A runtime request may still be needed for the
* segment, but the scheduler may attempt a static prefetch first (per the * segment, but the scheduler may attempt a static prefetch first (per the
* ShouldAttemptStaticPrefetch hint) and skip the runtime request if the * ShouldAttemptStaticPrefetch hint) and skip the runtime request if the
...@@ -322,8 +322,9 @@ export const SubtreePrefetchHints = ...@@ -322,8 +322,9 @@ export const SubtreePrefetchHints =
* *
* Used wherever a route tree is assembled bottom-up: on the server when building * Used wherever a route tree is assembled bottom-up: on the server when building
* a transport tree (createTransportTreeFromLoaderTree) and on the client when * a transport tree (createTransportTreeFromLoaderTree) and on the client when
* merging a navigation patch into the existing tree (convertServerPatchToFullTree). * overlaying a navigation response onto the existing tree
* Keep these in sync by routing both through this helper. * (createNavigationSeed). Keep these in sync by routing both through
* this helper.
*/ */
export function propagateSubtreeBits( export function propagateSubtreeBits(
parentHints: number, parentHints: number,
...@@ -429,7 +430,7 @@ export type InitialRSCPayload = { ...@@ -429,7 +430,7 @@ export type InitialRSCPayload = {
* happened during a stage it includes, pending (⇒ no access) otherwise. * happened during a stage it includes, pending (⇒ no access) otherwise.
* Unlike an async iterable, a pending promise costs Flight no abort * Unlike an async iterable, a pending promise costs Flight no abort
* listener on the render. Used when generating per-segment prefetch * listener on the render. Used when generating per-segment prefetch
* responses (forwarded as the response-level `needsRuntimeRequest`). * responses (forwarded as each response's own `u`).
* The build-constant `PrefetchHint.ShouldAttemptStaticPrefetch` is * The build-constant `PrefetchHint.ShouldAttemptStaticPrefetch` is
* tracked directly on the prerender store instead (its * tracked directly on the prerender store instead (its
* `shouldAttemptStaticPrefetch` cell) — it needs neither stream * `shouldAttemptStaticPrefetch` cell) — it needs neither stream
...@@ -481,6 +482,19 @@ type NavigationFlightResponseBase = { ...@@ -481,6 +482,19 @@ type NavigationFlightResponseBase = {
/** /**
* transport data — present iff the response carries a SPA payload. * transport data — present iff the response carries a SPA payload.
* Absent when the response renders nothing (see `n` for MPA navigations). * Absent when the response renders nothing (see `n` for MPA navigations).
*
* In a per-segment prefetch response the tree is root-anchored with `d`
* present on exactly the segments whose data is bundled into the response
* (the requested segment and any ancestors inlined into it, per the
* build-time prefetch hints); the remaining nodes — the spine above the
* bundle, and segments with static prefetching disabled — carry identity
* but no data. Every node carries the same hints the `/_tree` response
* emits for it, so the tree decodes like any other transport tree. Data
* nodes use the staged encodings: `p` is a promise and `s` is present
* (see TransportSegmentData). The head (`h`) is present iff it's bundled
* into the response: the standalone head response (whose tree is a bare
* root identity), or a page response whose bundle accepted the head
* (PrefetchHint.HeadInlinedIntoSelf).
*/ */
t?: PartialTransportData t?: PartialTransportData
/** /**
...@@ -490,22 +504,77 @@ type NavigationFlightResponseBase = { ...@@ -490,22 +504,77 @@ type NavigationFlightResponseBase = {
* or a build-id mismatch), but the client honors it. * or a build-id mismatch), but the client honors it.
*/ */
n?: string n?: string
/** staleTime - Only present in dynamic runtime prefetch responses. */ /** staleTime - Only present in live-render responses.
* Per-segment prefetch responses carry staleTime per node instead (see
* TransportSegmentData). */
s?: AsyncIterable<number> s?: AsyncIterable<number>
/** staticStageByteLength - Resolves when the static stage ends. */ /** staticStageByteLength - Resolves when the static stage ends. */
l?: Promise<number> l?: Promise<number>
/** /**
* shellByteLength - Resolves when the shell stage ends. * shellByteLength - Resolves when the shell stage ends.
* If it resolves to null, then the shell is the same as the main response. * If it resolves to null, then the shell is the same as the main response.
* */ *
* In a per-segment prefetch response this is the shell byte boundary the
* client uses for the shell double-decode: re-decoding the buffered
* response truncated at the offset yields the shell variant of every
* segment (param-dependent content reduced to still-pending references).
* `0` means no shell exists (the page wasn't produced by staged
* rendering) — never a valid offset, so it doubles as the "none"
* sentinel. The resolution row flushes past the boundary, so a truncated
* shell decode reads it as pending, which is harmless. The client's
* buffered read leans on the same fact from the other side: 0 read from
* an unfulfilled `a` implies a bug — see the shell-extraction comment in
* fetchAndWritePerSegmentPrefetchResponse (segment-cache/cache.ts).
*/
a?: Promise<number | null> a?: Promise<number | null>
/** /**
* shellUsedSessionData - true if resolving session data * needsRuntimeRequest — presence means the response carries a stage-scoped
* unblocked new content in the shell. * runtime-data verdict. Per-segment prefetch responses always emit one,
* NOTE: only use this in runtime/session prefetch requests * and so does any prerendered page payload, which embeds the prerender's
* where we have a proper session shell. * runtime-data probe — that covers navigation responses served from
* */ * prerender output and the truncated initial payload alike. Live renders
* emit none. Only per-segment responses additionally promise-encode the
* head's partiality (see TransportSegmentData). The value is the page's
* runtime-data-access flag (the page payload's own `u`; per-segment
* responses forward it from the staged decode of the page data).
* Fulfilled `true` means a runtime prefetch would return more
* than this static response; pending or fulfilled `false` means it
* wouldn't. Promise-encoded so the answer rewinds with a truncated
* decode: the fulfillment row lands on the same side of the shell byte
* boundary (`a`) as the access it records. Tracking is page-global, so it
* lives on the response, not on each segment; per-segment granularity
* comes from combining it with each segment's partiality:
* needsRuntimeRequest(segment) = (settled true) && (isPartial pending).
* The derived value must never falsely claim that no runtime request is
* needed, so every fallback is conservative: pages that carry no `u`
* (legacy render paths) forward an already-resolved `true`. Unlike the
* build-constant prefetch hints, this is computed per render and may
* change between responses for the same build — it reflects THIS
* response.
*/
u?: Promise<boolean> u?: Promise<boolean>
/**
* shellUsedSessionData — present only in runtime-shell prefetch
* responses: true if resolving session data unblocked new content in the
* shell. Only meaningful where there's a proper session shell.
* Promise-encoded so the answer rewinds with a truncated decode.
* Currently has no client consumer; emitted for forward compatibility.
*/
w?: Promise<boolean>
/**
* isUpgradeableISRFallback — present (true) iff this is a per-segment
* prefetch response generated from a fallback shell render (i.e. the page
* had not yet been prerendered with concrete params, so it was rendered
* with `fallbackRouteParams`). The client uses this to schedule a retry,
* since a more complete version may become available once the server's
* background regeneration finishes.
*
* Note: this is distinct from per-segment partiality. A fully-prerendered
* PPR page can have partial segments (dynamic holes filled by runtime
* requests); those should not be retried. This flag specifically means a
* more complete *static* version may become available.
*/
f?: boolean
/** /**
* rootVaryParams - the root params accessed anywhere in the response, emitted * rootVaryParams - the root params accessed anywhere in the response, emitted
* once. The client unions these into the head and every segment's vary * once. The client unions these into the head and every segment's vary
......
...@@ -63,8 +63,28 @@ export type TransportDynamicSegment = { ...@@ -63,8 +63,28 @@ export type TransportDynamicSegment = {
export type TransportSegmentData = { export type TransportSegmentData = {
/** rsc — the React node for this segment; null = skipped (not rendered) */ /** rsc — the React node for this segment; null = skipped (not rendered) */
r: React.ReactNode r: React.ReactNode
/** isPartial — contains unresolved dynamic holes (static prerender) */ /**
p: boolean * isPartial — whether anything in the segment's output is not fully
* resolved: dynamic holes, runtime holes, anything suspended.
*
* Navigation responses carry a plain boolean, known at render time. It is
* a render-wide constant (isPossiblyPartialResponse in
* create-component-tree.tsx), identical on every node and only
* conservatively accurate, so the client resolves a boolean-form node's
* actual partiality from the response-level signal instead (see
* decodeTransportNode).
*
* Per-segment prefetch responses carry the rewindable promise encoding
* instead: fulfilled = complete, pending forever = partial. Partiality is
* only discovered there by probing the buffered prerender (a partial
* segment's probe never resolves, the same way Flight encodes the holes
* themselves), and the encoding must survive the client's shell
* double-decode — the response buffer is decoded a second time truncated
* at the shell byte boundary (`a`), and a fulfillment row that landed past
* the boundary must read as "partial" in that decode. The promise form is
* only ever read off a fully-buffered decode's thenable status.
*/
p: boolean | Promise<void>
/** /**
* varyParams — an iterable of the route params this segment's output * varyParams — an iterable of the route params this segment's output
* depends on (one name per yield, deduped). Used by the client router to * depends on (one name per yield, deduped). Used by the client router to
...@@ -83,6 +103,18 @@ export type TransportSegmentData = { ...@@ -83,6 +103,18 @@ export type TransportSegmentData = {
* reused when those specific params match. * reused when those specific params match.
*/ */
v: VaryParamsIterable | null v: VaryParamsIterable | null
/**
* staleTime in seconds — present only in per-segment prefetch responses,
* where staleness is tracked per node. Navigation responses carry
* staleness at the response level instead (the wrapper's `s` iterable or
* the Next-Router-Stale-Time header).
*
* An async iterable rather than a plain number because the final value is
* only known late in the stream, and the iterable form survives a
* truncated/rewound shell decode (read via thenable status from the
* buffered response). The client takes the last yielded value.
*/
s?: AsyncIterable<number>
} }
/** /**
...@@ -136,10 +168,10 @@ export type FullTransportNode = TransportNodeShape & { ...@@ -136,10 +168,10 @@ export type FullTransportNode = TransportNodeShape & {
* - `h` omitted: on a skipped node, the client keeps its existing hints for * - `h` omitted: on a skipped node, the client keeps its existing hints for
* the segment; on any other node it means the hints are zero. * the segment; on any other node it means the hints are zero.
* *
* TODO: A node with `d` omitted currently ends the client's cache write for * The client's cache write descends through nodes with `d` omitted (see
* its subtree (see writeSeedDataIntoCache), so "skip a middle segment but * writeTreeDataIntoCache), so data may appear at any depth below a no-claim
* render below it" is not expressible yet. Extend the decode/write semantics * node — the identity spine of a per-segment prefetch response is the
* when a producer needs that. * canonical producer of that shape.
*/ */
export type PartialTransportNode = TransportNodeShape & { export type PartialTransportNode = TransportNodeShape & {
/** data — omitted when this segment was not rendered in this response */ /** data — omitted when this segment was not rendered in this response */
...@@ -184,6 +216,47 @@ export function createSkippedSegmentData(): TransportSegmentData { ...@@ -184,6 +216,47 @@ export function createSkippedSegmentData(): TransportSegmentData {
return { r: null, p: true, v: null } return { r: null, p: true, v: null }
} }
/**
* Reads a late-resolving value from a fully-buffered Flight decode via the
* thenable's status. Flight sets `status`/`value` on a row's promise once
* its bytes are processed, and a buffered decode processes every byte
* synchronously, so any row that made it into the payload is readable
* without awaiting. Returns `unresolvedValue` for a row that is pending or
* absent in this decode — e.g. one whose fulfillment landed past a
* truncated shell decode's boundary; that's what scopes a response's
* late-resolving signals to the payload being decoded. Returns
* `rejectedValue` (defaults to `unresolvedValue`) for a rejected row — an
* aborted render errors rows that were still pending when it happened.
*
* Shared by client and server: the client reads buffered prefetch
* responses; the server (collect-segment-data) reads the buffered page
* payload it re-serializes into per-segment responses.
*/
export function readFulfilledValue<T, TFallback>(
valueFromServer: PromiseLike<T>,
unresolvedValue: TFallback,
rejectedValue: T | TFallback = unresolvedValue
): T | TFallback {
const thenable = valueFromServer as PromiseLike<T> & {
status?: string
value?: T
}
// Force Flight to unwrap a received-but-not-yet-settled row.
thenable.then(noop, noop)
switch (thenable.status) {
case 'fulfilled':
return thenable.value as T
case 'rejected':
return rejectedValue
// No status yet: the row is still pending, or absent from this decode.
case undefined:
default:
return unresolvedValue
}
}
const noop = () => {}
/** /**
* Converts a segment's client/server-internal representation to its wire * Converts a segment's client/server-internal representation to its wire
* representation. * representation.
...@@ -212,10 +285,12 @@ export function transportSegmentToSegment( ...@@ -212,10 +285,12 @@ export function transportSegmentToSegment(
} }
return [ return [
transportSegment.n, transportSegment.n,
// TODO: `k` may be null when the client is expected to parse the param // `k` may be null when the client is expected to parse the param value
// value from the URL. Navigation responses always include the key today; // from the URL (per-segment prefetch responses). Callers that need the
// this becomes relevant when per-segment prefetch responses converge on // real value resolve it from the rendered pathname instead of using this
// this format. // function (see resolveTransportSegment in decode-server-response); the
// remaining callers are value-insensitive (segment request keys, which
// never include param values) or only see concrete keys.
transportSegment.k ?? '', transportSegment.k ?? '',
transportSegment.t, transportSegment.t,
transportSegment.s, transportSegment.s,
......
...@@ -4,6 +4,8 @@ ...@@ -4,6 +4,8 @@
* This module is shared between server and client. * This module is shared between server and client.
*/ */
import { readFulfilledValue } from '../rsc-transport'
export type VaryParams = Set<string> export type VaryParams = Set<string>
/** /**
...@@ -23,26 +25,15 @@ export type VaryParams = Set<string> ...@@ -23,26 +25,15 @@ export type VaryParams = Set<string>
*/ */
export type VaryParamsIterable = AsyncIterable<string> export type VaryParamsIterable = AsyncIterable<string>
/**
* A Flight "chunk": when an `AsyncIterable` arrives over a fully-buffered Flight
* stream, each `iterator.next()` returns one of these instead of a native
* Promise. It follows the React thenable protocol — once `status` is
* `'fulfilled'`, `value` (the `IteratorResult`) can be read synchronously.
*/
type IteratorResultChunk = PromiseLike<IteratorResult<string>> & {
status?: 'pending' | 'resolved_model' | 'fulfilled' | 'rejected' | string
value?: IteratorResult<string>
}
/** /**
* Synchronously drains a vary params `AsyncIterable`, adding each yielded name * Synchronously drains a vary params `AsyncIterable`, adding each yielded name
* to `target`. * to `target`.
* *
* By the time this runs (on the client, or in collectSegmentData), the Flight * By the time this runs (on the client, or in collectSegmentData), the Flight
* stream has been fully buffered, so every yielded value is already * stream has been fully buffered, so every yielded value is already
* materialized and can be read without awaiting. We force each iterator result * materialized and can be read without awaiting: each iterator result is a
* to resolve synchronously using the same `.then(noop)` trick React uses * Flight chunk whose settled state `readFulfilledValue` reads off the
* internally, then read its `status`/`value` directly. * thenable's status.
* *
* We add "every param yielded up to the point the stream suspends": a * We add "every param yielded up to the point the stream suspends": a
* normally-closed iterable drains fully, while one left hanging (a sync-I/O * normally-closed iterable drains fully, while one left hanging (a sync-I/O
...@@ -58,21 +49,10 @@ function drainVaryParams( ...@@ -58,21 +49,10 @@ function drainVaryParams(
): void { ): void {
const iterator = iterable[Symbol.asyncIterator]() const iterator = iterable[Symbol.asyncIterator]()
while (true) { while (true) {
const chunk = iterator.next() as IteratorResultChunk const step = readFulfilledValue(iterator.next(), undefined)
// Attach a no-op listener to force Flight to synchronously resolve the if (step === undefined || step.done) {
// chunk. A freshly-arrived result may be in an intermediate // Either the stream suspended here — everything yielded before this
// 'resolved_model' state (data received but not unwrapped); calling // point has already been added — or the iterable finished cleanly.
// .then() transitions it to 'fulfilled', making the value available
// synchronously. (A native Promise has no `status` and simply reads as
// not-fulfilled below, so this can never hang.)
chunk.then(noop, noop)
if (chunk.status !== 'fulfilled' || chunk.value === undefined) {
// The stream suspended here. Everything yielded before this point has
// already been added.
return
}
const step = chunk.value
if (step.done) {
return return
} }
target.add(step.value) target.add(step.value)
...@@ -117,5 +97,3 @@ export function readVaryParams( ...@@ -117,5 +97,3 @@ export function readVaryParams(
drainVaryParams(rootIterable, varyParams) drainVaryParams(rootIterable, varyParams)
return varyParams return varyParams
} }
const noop = () => {}
...@@ -23,7 +23,7 @@ const APP_SHELL_PREFETCH_VALUE = '3' ...@@ -23,7 +23,7 @@ const APP_SHELL_PREFETCH_VALUE = '3'
// - 'static': per-segment static prefetches. These carry the // - 'static': per-segment static prefetches. These carry the
// `next-router-segment-prefetch` header (NEXT_ROUTER_SEGMENT_PREFETCH_HEADER // `next-router-segment-prefetch` header (NEXT_ROUTER_SEGMENT_PREFETCH_HEADER
// in the client), which is sent both by the per-segment data fetch // in the client), which is sent both by the per-segment data fetch
// (`fetchSegmentsOnCacheMissImpl`) and the route tree fetch // (`fetchSegmentPrefetchesUsingStaticRequest`) and the route tree fetch
// (`fetchRouteOnCacheMiss`). // (`fetchRouteOnCacheMiss`).
// - 'runtime': dynamic prefetch requests, issued by // - 'runtime': dynamic prefetch requests, issued by
// `fetchSegmentPrefetchesUsingRuntimeRequest` in the client. These carry a // `fetchSegmentPrefetchesUsingRuntimeRequest` in the client. These carry a
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment