Unverified Commit f4449e8f authored by Janka Uryga's avatar Janka Uryga Committed by GitHub

Instant validation for caches excluded from prerenders (#98342)

Follow-up to #98339. Turns out we were not testing how caches excluded
from static/runtime prerenders and app shells behave in Instant
Validation, and it was somewhat broken. This PR adds test coverage and
fixes some bugs I found along the way.

### Fixes

- `use-cache-wrapper` was incorrectly gating cache delays in "request"
stores on `NODE_ENV === "development"`, which does not include
build-time instant validation. The correct check is now implemented in
`isValidationRender`
- `use-cache-wrapper` has divergent behavior for caches with `stale <
MIN_SHELL_STALE` across app shells and PPR/static shells, which needs to
be tracked so that we know that the same render can't be used for both
Instant and Static Shell validation. When we see a cache entry like
that, we now call `trackIncompatibleShellContent()`
- A render that had a cache miss could still report that it's compatible
with both SSV and IV, because we only do the above for a cache hit. As a
result so we'd incorrectly reuse `LAZY_FULL_RENDER` for both. Cache
misses now result in a `trackIncompatibleShellContent()` call to avoid
this

### Tests

We now have tests for:
- `stale < MIN_SHELL_STALE` - excluded from app shells, but included in
static and runtime prefetches
- `stale < MIN_PREFETCHABLE_STALE` - excluded from all prerenders
- `expire < MIN_PRERENDERABLE_EXPIRE` - excluded from static prerenders,
but allowed in runtime prerenders

Due to bugs mentioned above, some of the added tests were failing before
the fixes (mostly the ones that expect an error -- passing a "no
validation errors" test is easy, just don't create any dynamic holes)

In build, these tests incorrectly reported no errors when they should've
failed IV:

- `invalid - unguarded non-prefetchable cache (with short stale)` (both
PPF and non-PPF)
- `non-app shell validation > invalid - unguarded non-prerenderable
cache with short expire` in build:
- `app shell validation > invalid - unguarded cache with a
shorter-than-shell staleTime` in build:

This is because we were missing cache delays in build-time instant
validation (now fixed with `isValidationRender`), so the caches weren't
dynamic holes at all.

The `stale < MIN_SHELL_STALE` tests (`app shell validation > invalid -
unguarded cache with a shorter-than-shell staleTime`) were also failing
in dev:
- **{initial load, client navigation} with cold caches**: Should be an
IV error, but is an SSV error. The initial render had cache misses, so
we did a warm-cache full rerender with runtime shells, which resolved
`await nonShellCache()` in the Runtime stage. But the initial render
**did not track incompatible data**, so we incorrectly re-used it for
SSV and IV. The cache was resolved in `Runtime` so SSV saw a runtime
hole and `await nonShellCache()` errored in SSV with `Next.js
encountered runtime data during prerendering.`
- **client navigation with warm caches**: same as above, except there
weren't cache misses, so incorrectly reused the *original* runtime-shell
render for SSV and IV
- **initial load with warm caches**: No redbox when IV should've
errored. The main render was an initial load and did not use runtime
shells, so it resolved `await nonShellCache()` in `PrefetchStatic`. We
had no cache misses and **did not track incompatible data**, so we
incorrectly re-used the main non-runtime-shell render for SSV and IV.
The cache resolved in `PrefetchStatic` and it wasn't a hole in
`ShellRuntime` so no error was reported.
parent e974969d
...@@ -335,7 +335,10 @@ import type { ...@@ -335,7 +335,10 @@ import type {
import { ResponseCookies } from '../web/spec-extension/cookies' import { ResponseCookies } from '../web/spec-extension/cookies'
import { isInstantValidationError } from './instant-validation/instant-validation-error' import { isInstantValidationError } from './instant-validation/instant-validation-error'
import { createPromiseWithResolvers } from '../../shared/lib/promise-with-resolvers' import { createPromiseWithResolvers } from '../../shared/lib/promise-with-resolvers'
import { RENDER_STAGES_BY_DATA_KIND } from '../dynamic-rendering-utils' import {
RENDER_STAGES_BY_DATA_KIND,
trackIncompatibleShellContent,
} from '../dynamic-rendering-utils'
import type { import type {
PrefetchedSegmentStage, PrefetchedSegmentStage,
SegmentStage, SegmentStage,
...@@ -5106,11 +5109,20 @@ async function prepareValidationInputsInPartialPrefetching( ...@@ -5106,11 +5109,20 @@ async function prepareValidationInputsInPartialPrefetching(
const needsInstantValidation = const needsInstantValidation =
await anySegmentNeedsInstantValidationInDev(loaderTree) await anySegmentNeedsInstantValidationInDev(loaderTree)
// Certain APIs (static `params`, `unstable_navigation()`, `unstable_prefetch()`) resolve // Certain APIs resolve in either static or runtime stages depending on the context.
// in either static or runtime stages depending on the context (see `needsAppShell`). // (see `needsAppShell` and callsites of `trackIncompatibleShellContent`)
// This includes:
// - static `params`
// - `unstable_navigation()` and `unstable_prefetch()`
// - caches with `stale < MIN_SHELL_STALE` are excluded from app shells
// but are included in static prerenders
// If one of these APIs is used, the render can't be used for both Instant Validation and // If one of these APIs is used, the render can't be used for both Instant Validation and
// Static Shell Validation and we'll need to perform a secondary render. // Static Shell Validation and we'll need to perform a secondary render.
// All relevant uses are tracked on the request store. //
// Note that a cache miss *also* sets this, because otherwise we might not
// have accurate tracking, especially for short-stale caches.
// This prevents us from using the same rerender for both validations
// if there's a chance that the cache miss might be hiding incompatible data.
const areStagesCompatible = !requestStore.hasIncompatibleShellContent const areStagesCompatible = !requestStore.hasIncompatibleShellContent
const LAZY_FULL_RENDER = createLazyDevValidationInputs(async () => { const LAZY_FULL_RENDER = createLazyDevValidationInputs(async () => {
...@@ -5463,6 +5475,7 @@ interface StreamStagedRenderInDevOptions extends StagedDevRenderOptions { ...@@ -5463,6 +5475,7 @@ interface StreamStagedRenderInDevOptions extends StagedDevRenderOptions {
* the render turns out to be prod-representative. * the render turns out to be prod-representative.
*/ */
async function streamStagedRenderInDev({ async function streamStagedRenderInDev({
prefetchMode,
ctx, ctx,
requestStore, requestStore,
rscPayload, rscPayload,
...@@ -5537,6 +5550,8 @@ async function streamStagedRenderInDev({ ...@@ -5537,6 +5550,8 @@ async function streamStagedRenderInDev({
// shell stage is flushing (see `checkForCacheMiss`). // shell stage is flushing (see `checkForCacheMiss`).
let reportedColdCache = false let reportedColdCache = false
let trackedIncompatibleShellContent = false
// Runs at each stage boundary. Latches the running cache-miss verdict and // Runs at each stage boundary. Latches the running cache-miss verdict and
// returns it, so a boundary can reveal the shell as soon as a miss is seen // returns it, so a boundary can reveal the shell as soon as a miss is seen
// (and so dev validation can later tell whether the streamed render is // (and so dev validation can later tell whether the streamed render is
...@@ -5546,6 +5561,36 @@ async function streamStagedRenderInDev({ ...@@ -5546,6 +5561,36 @@ async function streamStagedRenderInDev({
if (cacheSignal.hasPendingReads()) { if (cacheSignal.hasPendingReads()) {
hadCacheMiss = true hadCacheMiss = true
if (prefetchMode === PrefetchingMode.Partial) {
// If we had a cache miss (and we'll need an app shell for validation),
// we *may* have content that is incompatible between an app shell and a static shell:
//
// - A cache miss might defer content to a stage where it is no longer
// considered incompatible and would not be tracked as such
// - Caches with `stale < MIN_SHELL_STALE` should be excluded from app shells,
// but included in static prerenders (which we can only do when the cache
// is a hit and its cache life is known)
//
// We need to signal this to `prepareValidationInputsInPartialPrefetching`.
// It would not re-use a render with a cache miss anyway, but it tries to
// optimize the follow-up rerenders into a single one based on whether
// stages are compatible, and we can't do that if they *might* use incompatible
// data due to a cache miss.
//
// TODO(app-shells): optimize this (and other `trackIncompatibleShellContent`
// callsites) to only consider stages that are relevant for validation.
if (trackedIncompatibleShellContent) {
// It's enough to report the first miss.
} else {
trackedIncompatibleShellContent = true
trackIncompatibleShellContent(
requestStore,
'cache miss during main render'
)
}
}
// The cold-cache indicator reflects the shell only. A cache read still // The cold-cache indicator reflects the shell only. A cache read still
// pending while a shell stage flushes (`currentStage <= // pending while a shell stage flushes (`currentStage <=
// revealAfterStage`, using the ordered `RenderStage` values) is part of // revealAfterStage`, using the ordered `RenderStage` values) is part of
......
...@@ -82,6 +82,11 @@ export interface RequestStore extends CommonWorkUnitStore { ...@@ -82,6 +82,11 @@ export interface RequestStore extends CommonWorkUnitStore {
* Certain APIs have different behavior in static and runtime prerenders. * Certain APIs have different behavior in static and runtime prerenders.
* - if `false`, they will follow static semantics * - if `false`, they will follow static semantics
* - if `true`, they will follow runtime semantics * - if `true`, they will follow runtime semantics
*
* NOTE: Whenever the stage of a promise varies on `needsAppShell`,
* we should also call `trackIncompatibleShellContent` to signal this.
* Otherwise, instant validation or static shell validation might incorrectly
* use a render that resolves it at an inappropriate time.
* */ * */
needsAppShell?: boolean // DEV-only needsAppShell?: boolean // DEV-only
/** /**
......
...@@ -305,7 +305,39 @@ function trackRuntimeDataAccessedImpl( ...@@ -305,7 +305,39 @@ function trackRuntimeDataAccessedImpl(
} }
} }
export function trackIncompatibleShellContent(workUnitStore: RequestStore) { /**
* Signals that we cannot recover both a runtime shell and a static (PPR) shell
* from the same render. Use this whenever the stage of a promise varies on
* `RequestStore.needsAppShell`.
* */
export function trackIncompatibleShellContent(
workUnitStore: RequestStore,
reason: string
) {
const { stagedRendering } = workUnitStore
if (!stagedRendering) {
return
}
// TODO(app-shells): optimize this to only consider stages that are relevant for validation.
// We should only track incompatible content when it can affect them.
// For now, we simply exclude everything that happens in the dynamic stage.
// (Note that we also need to account for cache misses that move things to a
// different stage -- those should also preemptively set `hasIncompatibleShellContent`
// because there's a chance that a render with warm caches would set it)
const { currentStage } = stagedRendering
if (
currentStage === RenderStage.Dynamic ||
currentStage === RenderStage.Abandoned
) {
return
}
if (process.env.NEXT_PRIVATE_DEBUG_VALIDATION) {
const workStore = workAsyncStorage.getStore()!
console.log(
`Route ${workStore.route}: Incompatible shell content: ${reason}`
)
}
workUnitStore.hasIncompatibleShellContent = true workUnitStore.hasIncompatibleShellContent = true
} }
......
...@@ -98,7 +98,7 @@ export function unstable_prefetch(): Promise<void> { ...@@ -98,7 +98,7 @@ export function unstable_prefetch(): Promise<void> {
const { stagedRendering } = workUnitStore const { stagedRendering } = workUnitStore
if (stagedRendering) { if (stagedRendering) {
// We can either recover a static shell or a runtime shell, but not both. // We can either recover a static shell or a runtime shell, but not both.
trackIncompatibleShellContent(workUnitStore) trackIncompatibleShellContent(workUnitStore, '`unstable_prefetch()`')
const stage = workUnitStore.needsAppShell const stage = workUnitStore.needsAppShell
? RENDER_STAGES_BY_DATA_KIND.runtimeLinkData // Match the timing of 'prerender-runtime'. ? RENDER_STAGES_BY_DATA_KIND.runtimeLinkData // Match the timing of 'prerender-runtime'.
: RENDER_STAGES_BY_DATA_KIND.staticLinkData // Match the timing of 'prerender'. : RENDER_STAGES_BY_DATA_KIND.staticLinkData // Match the timing of 'prerender'.
...@@ -251,7 +251,7 @@ export function unstable_navigation(): Promise<void> { ...@@ -251,7 +251,7 @@ export function unstable_navigation(): Promise<void> {
const { stagedRendering } = workUnitStore const { stagedRendering } = workUnitStore
if (stagedRendering) { if (stagedRendering) {
// We can either recover a static shell or a runtime shell, but not both. // We can either recover a static shell or a runtime shell, but not both.
trackIncompatibleShellContent(workUnitStore) trackIncompatibleShellContent(workUnitStore, '`unstable_navigation()`')
const stage = workUnitStore.needsAppShell const stage = workUnitStore.needsAppShell
? RenderStage.NavigationRuntime // Match the timing of 'prerender-runtime'. ? RenderStage.NavigationRuntime // Match the timing of 'prerender-runtime'.
: RenderStage.NavigationStatic // Match the timing of 'prerender'. : RenderStage.NavigationStatic // Match the timing of 'prerender'.
......
...@@ -586,7 +586,11 @@ function createStagedRenderParamsImpl( ...@@ -586,7 +586,11 @@ function createStagedRenderParamsImpl(
// If static params are accessed, we can recover a static shell or a session shell, but not both. // If static params are accessed, we can recover a static shell or a session shell, but not both.
return trackPromiseUsed( return trackPromiseUsed(
promise, promise,
trackIncompatibleShellContent.bind(null, workUnitStore) trackIncompatibleShellContent.bind(
null,
workUnitStore,
'static `params`'
)
) )
} else { } else {
return promise return promise
......
...@@ -45,6 +45,7 @@ import { ...@@ -45,6 +45,7 @@ import {
makePrefetchHangingPromise, makePrefetchHangingPromise,
makeUntrackedHangingPromise, makeUntrackedHangingPromise,
RENDER_STAGES_BY_DATA_KIND, RENDER_STAGES_BY_DATA_KIND,
trackIncompatibleShellContent,
} from '../dynamic-rendering-utils' } from '../dynamic-rendering-utils'
import type { ClientReferenceManifest } from '../../build/webpack/plugins/flight-manifest-plugin' import type { ClientReferenceManifest } from '../../build/webpack/plugins/flight-manifest-plugin'
...@@ -1058,7 +1059,7 @@ function maybePropagateCacheEntryMetadata( ...@@ -1058,7 +1059,7 @@ function maybePropagateCacheEntryMetadata(
} }
case 'request': { case 'request': {
if ( if (
process.env.NODE_ENV === 'development' && isValidationRender(outerWorkUnitStore) &&
outerWorkUnitStore.cacheSignal outerWorkUnitStore.cacheSignal
) { ) {
// If we're filling caches for a dev request, apply the same logic as // If we're filling caches for a dev request, apply the same logic as
...@@ -1986,9 +1987,9 @@ export async function cache( ...@@ -1986,9 +1987,9 @@ export async function cache(
break break
} }
case 'request': { case 'request': {
if (process.env.NODE_ENV === 'development') { if (isValidationRender(outerWorkUnitStore)) {
// Similar to runtime prerenders, private caches should not resolve in the static stage // Similar to runtime prerenders, private caches should not resolve in the static stage
// of a dev request, so we delay them. // of a validation request, so we delay them.
await makeDevtoolsIOAwarePromise( await makeDevtoolsIOAwarePromise(
undefined, undefined,
outerWorkUnitStore, outerWorkUnitStore,
...@@ -2453,7 +2454,7 @@ export async function cache( ...@@ -2453,7 +2454,7 @@ export async function cache(
break break
} }
case 'request': { case 'request': {
if (process.env.NODE_ENV === 'development') { if (isValidationRender(workUnitStore)) {
// These throws force an explicit cache life decision on an // These throws force an explicit cache life decision on an
// outer cache that a nested cache would otherwise silently // outer cache that a nested cache would otherwise silently
// shorten (see `shouldReportNestedCacheError` above). Otherwise // shorten (see `shouldReportNestedCacheError` above). Otherwise
...@@ -2604,11 +2605,7 @@ export async function cache( ...@@ -2604,11 +2605,7 @@ export async function cache(
break break
} }
case 'request': { case 'request': {
// A request store in `next start` never delays caches — shells if (isValidationRender(workUnitStore)) {
// are produced by separate (runtime) prerenders, which apply
// the exclusions above. In dev, the request render is also used
// to recover shells, so we delay the entry here to match.
if (process.env.NODE_ENV === 'development') {
// End the cache signal read (once, in case an earlier block // End the cache signal read (once, in case an earlier block
// already did) so the delayed value isn't counted as a pending // already did) so the delayed value isn't counted as a pending
// read at a staged rendering boundary. // read at a staged rendering boundary.
...@@ -2616,21 +2613,32 @@ export async function cache( ...@@ -2616,21 +2613,32 @@ export async function cache(
cacheSignal.endRead() cacheSignal.endRead()
cacheSignalReadEnded = true cacheSignalReadEnded = true
} }
// An unprefetchable entry is excluded from prerenders, so it
// resolves in the dynamic stage. Otherwise, a dynamic request
// generally recovers a static shell, so the entry can resolve
// in the static link data stage. If we need to recover a
// session shell instead, as indicated by `needsAppShell`,
// the entry must resolve after the session data stage that
// the shell includes.
let stage: AdvanceableRenderStage let stage: AdvanceableRenderStage
if (!isPrefetchable) { if (!isPrefetchable) {
// An unprefetchable entry is excluded from prerenders, so it
// resolves in the dynamic stage.
stage = RenderStage.Dynamic stage = RenderStage.Dynamic
} else if (workUnitStore.needsAppShell) {
stage = RENDER_STAGES_BY_DATA_KIND.runtimeLinkData
} else { } else {
stage = RENDER_STAGES_BY_DATA_KIND.staticLinkData // If the entry would be be excluded from the shell, treat it as
// if it were link data.
// (Note that this is still correct without PPF or in static shell validation,
// where we don't use runtime shells and include static link data)
trackIncompatibleShellContent(
workUnitStore,
'"use cache" excluded from app shells due to a short staletime'
)
stage = workUnitStore.needsAppShell
? RENDER_STAGES_BY_DATA_KIND.runtimeLinkData
: RENDER_STAGES_BY_DATA_KIND.staticLinkData
} }
debug?.(
logPrefix,
'delaying entry',
serializedCacheKey,
`until after the ${RenderStage[stage]} stage due to short stale value:`,
rdcResult.entry.stale
)
await makeDevtoolsIOAwarePromise( await makeDevtoolsIOAwarePromise(
undefined, undefined,
workUnitStore, workUnitStore,
...@@ -3172,7 +3180,7 @@ export async function cache( ...@@ -3172,7 +3180,7 @@ export async function cache(
}) })
return hangingPromise return hangingPromise
case 'request': { case 'request': {
if (process.env.NODE_ENV === 'development') { if (isValidationRender(workUnitStore)) {
// A short-lived entry is a dynamic hole, excluded from the // A short-lived entry is a dynamic hole, excluded from the
// static shell, so we end the cache signal read here (the // static shell, so we end the cache signal read here (the
// prerender case does the same) to avoid this cache hit being // prerender case does the same) to avoid this cache hit being
...@@ -3183,7 +3191,14 @@ export async function cache( ...@@ -3183,7 +3191,14 @@ export async function cache(
cacheSignal.endRead() cacheSignal.endRead()
cacheSignalReadEnded = true cacheSignalReadEnded = true
} }
const stage = RENDER_STAGES_BY_DATA_KIND.sessionData
debug?.(
logPrefix,
'delaying entry',
serializedCacheKey,
`until after the ${RenderStage[stage]} stage due to short expire value:`,
entry.expire
)
await makeDevtoolsIOAwarePromise( await makeDevtoolsIOAwarePromise(
undefined, undefined,
workUnitStore, workUnitStore,
...@@ -3205,16 +3220,14 @@ export async function cache( ...@@ -3205,16 +3220,14 @@ export async function cache(
} }
if (entry !== undefined && entry.stale < MIN_SHELL_STALE) { if (entry !== undefined && entry.stale < MIN_SHELL_STALE) {
const isPrefetchable = entry.stale >= MIN_PREFETCHABLE_STALE
switch (workUnitStore.type) { switch (workUnitStore.type) {
case 'request': { case 'request': {
// Same as the resume data cache read path: the entry's stale // Same as the resume data cache read path: the entry's stale
// time is short enough that it's excluded from shells, or, if // time is short enough that it's excluded from shells, or, if
// it's below `MIN_PREFETCHABLE_STALE`, from prerenders // it's below `MIN_PREFETCHABLE_STALE`, from prerenders
// entirely. A request store in `next start` never delays // entirely.
// caches — shells are produced by separate (runtime) if (isValidationRender(workUnitStore)) {
// prerenders. In dev, the request render is also used to
// recover shells, so we delay the entry here to match.
if (process.env.NODE_ENV === 'development') {
// End the cache signal read (once, in case the expire block // End the cache signal read (once, in case the expire block
// above already did) so the delayed value isn't counted as a // above already did) so the delayed value isn't counted as a
// pending read at a staged rendering boundary. // pending read at a staged rendering boundary.
...@@ -3222,21 +3235,31 @@ export async function cache( ...@@ -3222,21 +3235,31 @@ export async function cache(
cacheSignal.endRead() cacheSignal.endRead()
cacheSignalReadEnded = true cacheSignalReadEnded = true
} }
// An unprefetchable entry is excluded from prerenders, so it
// resolves in the dynamic stage. Otherwise, a dynamic request
// generally recovers a static shell, so the entry can resolve
// in the static link data stage. If we need to recover a
// session shell instead, as indicated by `needsAppShell`,
// the entry must resolve after the session data stage that
// the shell includes.
let stage: AdvanceableRenderStage let stage: AdvanceableRenderStage
if (entry.stale < MIN_PREFETCHABLE_STALE) { if (!isPrefetchable) {
// An unprefetchable entry is excluded from prerenders, so it
// resolves in the dynamic stage.
stage = RenderStage.Dynamic stage = RenderStage.Dynamic
} else if (workUnitStore.needsAppShell) {
stage = RENDER_STAGES_BY_DATA_KIND.runtimeLinkData
} else { } else {
stage = RENDER_STAGES_BY_DATA_KIND.staticLinkData // If the entry would be be excluded from the shell, treat it as
// if it were link data.
// (Note that this is still correct without PPF or in static shell validation,
// where we don't use runtime shells and include static link data)
trackIncompatibleShellContent(
workUnitStore,
'"use cache" excluded from app shells due to a short staletime'
)
stage = workUnitStore.needsAppShell
? RENDER_STAGES_BY_DATA_KIND.runtimeLinkData
: RENDER_STAGES_BY_DATA_KIND.staticLinkData
} }
debug?.(
logPrefix,
'delaying entry',
serializedCacheKey,
`until after the ${RenderStage[stage]} stage due to short stale value:`,
entry.stale
)
await makeDevtoolsIOAwarePromise( await makeDevtoolsIOAwarePromise(
undefined, undefined,
workUnitStore, workUnitStore,
...@@ -3666,6 +3689,21 @@ async function computeCacheKeyImplementationPart( ...@@ -3666,6 +3689,21 @@ async function computeCacheKeyImplementationPart(
} }
} }
/**
* A request store in `next start` never delays caches.
* We only apply delays when the render will be used for
* validation purposes.
*/
function isValidationRender(requestStore: RequestStore): boolean {
return !!(
process.env.__NEXT_CACHE_COMPONENTS &&
// In dev we're always running validation.
(process.env.__NEXT_DEV_SERVER ||
// Build-time instant validation also uses a 'request' store.
requestStore.validationSamples)
)
}
/** /**
* Returns `true` if the `'use cache'` function is the page component itself, * Returns `true` if the `'use cache'` function is the page component itself,
* or `generateMetadata`/`generateViewport` in a page file. * or `generateMetadata`/`generateViewport` in a page file.
......
import { Instant } from 'next'
import { cacheLife } from 'next/cache'
export const instant: Instant = {
level: 'experimental-error',
}
export const prefetch = 'partial'
export default function Page() {
return (
<main>
<p>
This page uses uses a cache that is excluded from the shell due to a
short staletime, so we can't render a shell.
</p>
<PrefetchContent />
</main>
)
}
async function PrefetchContent() {
await nonShellCache()
return <div>{`Prefetch content`}</div>
}
async function nonShellCache() {
'use cache'
cacheLife({ stale: 300 - 1 }) // smaller than MIN_SHELL_STALE
await new Promise((resolve) => setTimeout(resolve))
return Date.now()
}
...@@ -12,7 +12,7 @@ export default async function Page() { ...@@ -12,7 +12,7 @@ export default async function Page() {
cacheLife('minutes') cacheLife('minutes')
return ( return (
<main> <main>
<h2>App Shells</h2> <h1>App Shells</h1>
<ul> <ul>
<li> <li>
<DebugLinks href="/shells/valid-session-only" /> <DebugLinks href="/shells/valid-session-only" />
...@@ -51,6 +51,19 @@ export default async function Page() { ...@@ -51,6 +51,19 @@ export default async function Page() {
<DebugLinks href="/shells/valid-prefetch-with-suspense" /> <DebugLinks href="/shells/valid-prefetch-with-suspense" />
</li> </li>
</ul> </ul>
<h2>Excluded caches</h2>
<ul>
<li>
<DebugLinks href="/shells/invalid-non-shell-cache" />
</li>
<li>
<DebugLinks href="/shells/valid-non-shell-cache" />
</li>
<li>
<DebugLinks href="/shells/valid-non-prerenderable-cache" />
</li>
</ul>
</main> </main>
) )
} }
export const instant = false
export default function IgnoreStaticShellValidationLayout({ children }) {
return children
}
import { Instant } from 'next'
import { cacheLife } from 'next/cache'
export const instant: Instant = {
level: 'experimental-error',
}
export const prefetch = 'partial'
export default function Page() {
return (
<main>
<p>
This page uses uses a cache that is excluded from static prerenders due
to a short expire. In Partial Prefetching, we can use a runtime
prerender instead, so this is allowed.
</p>
<SessionContent />
</main>
)
}
async function SessionContent() {
await nonPrerenderableCache()
return <div>{`Session content`}</div>
}
async function nonPrerenderableCache() {
'use cache'
cacheLife({ expire: 300 - 1 }) // smaller than MIN_PRERENDERABLE_EXPIRE
await new Promise((resolve) => setTimeout(resolve))
return Date.now()
}
import { Suspense } from 'react'
import { cacheLife } from 'next/cache'
import { Instant } from 'next'
export const instant: Instant = {
level: 'experimental-error',
}
export const prefetch = 'partial'
export default function Page() {
return (
<main>
<p>
This page uses uses a cache that is excluded from the shell due to a
short staletime, but it guards it with Suspense, so we can render a
shell.
</p>
<Suspense fallback={<div>Loading prefetch content...</div>}>
<PrefetchContent />
</Suspense>
</main>
)
}
async function PrefetchContent() {
await nonShellCache()
return <div id="prefetch-content">{`Prefetch content`}</div>
}
async function nonShellCache() {
'use cache'
cacheLife({ stale: 300 - 1 }) // smaller than MIN_SHELL_STALE
await new Promise((resolve) => setTimeout(resolve))
return Date.now()
}
import { Instant } from 'next'
import { cacheLife } from 'next/cache'
export const instant: Instant = {
level: 'experimental-error',
}
export default function Page() {
return (
<main>
<p>
This page uses uses a cache that is excluded from prerenders due to a
short staletime, so we can't prefetch it in any way.
</p>
<DynamicContent />
</main>
)
}
async function DynamicContent() {
await nonPrefetchableCache()
return <div>{`Dynamic content`}</div>
}
async function nonPrefetchableCache() {
'use cache'
cacheLife({ stale: 30 - 1 }) // smaller than MIN_PREFETCHABLE_STALE
await new Promise((resolve) => setTimeout(resolve))
return Date.now()
}
import { Instant } from 'next'
import { cacheLife } from 'next/cache'
export const instant: Instant = {
level: 'experimental-error',
}
export default function Page() {
return (
<main>
<p>
This page uses uses a cache that is excluded from static prerenders due
to a short expire. Outside of Partial Prefetching we only use static
prerenders, so this is not allowed.
</p>
<SessionContent />
</main>
)
}
async function SessionContent() {
await nonPrerenderableCache()
return <div>{`Session content`}</div>
}
async function nonPrerenderableCache() {
'use cache'
cacheLife({ expire: 300 - 1 }) // smaller than MIN_PRERENDERABLE_EXPIRE
await new Promise((resolve) => setTimeout(resolve))
return Date.now()
}
import { Instant } from 'next'
import { cacheLife } from 'next/cache'
export const instant: Instant = {
level: 'experimental-error',
}
export default function Page() {
return (
<main>
<p>
This page uses uses a cache that is excluded from the shell due to a
short staletime. The cache is still included in static prerenders, so
it's allowed outside of Partial Prefetching.
</p>
<PrefetchContent />
</main>
)
}
async function PrefetchContent() {
await nonShellCache()
return <div>{`Prefetch content`}</div>
}
async function nonShellCache() {
'use cache'
cacheLife({ stale: 300 - 1 }) // smaller than MIN_SHELL_STALE
await new Promise((resolve) => setTimeout(resolve))
return Date.now()
}
...@@ -300,6 +300,13 @@ export default async function Page() { ...@@ -300,6 +300,13 @@ export default async function Page() {
</li> </li>
</ul> </ul>
<h2>Excluded caches</h2>
<ul>
<li>
<DebugLinks href="/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache" />
</li>
</ul>
<h2>Without partialPrefetching</h2> <h2>Without partialPrefetching</h2>
<ul> <ul>
<li> <li>
...@@ -311,6 +318,12 @@ export default async function Page() { ...@@ -311,6 +318,12 @@ export default async function Page() {
<li> <li>
<DebugLinks href="/suspense-in-root/non-app-shell/valid-unguarded-prefetch" /> <DebugLinks href="/suspense-in-root/non-app-shell/valid-unguarded-prefetch" />
</li> </li>
<li>
<DebugLinks href="/suspense-in-root/non-app-shell/valid-unguarded-non-shell-cache" />
</li>
<li>
<DebugLinks href="/suspense-in-root/non-app-shell/invalid-unguarded-non-prerenderable-cache" />
</li>
</ul> </ul>
<h2>Blocking await attribution</h2> <h2>Blocking await attribution</h2>
......
...@@ -13,11 +13,14 @@ export interface InstantValidationCaseContext { ...@@ -13,11 +13,14 @@ export interface InstantValidationCaseContext {
* Navigate to a page either via initial load or soft navigation. * Navigate to a page either via initial load or soft navigation.
* For soft nav, navigates to the index page first, then clicks the link. * For soft nav, navigates to the index page first, then clicks the link.
*/ */
navigateTo: (href: string) => Promise<Playwright> navigateTo: (href: string, browser?: Playwright) => Promise<Playwright>
warmCachesAndNavigateTo: (href: string) => Promise<Playwright>
restartDevServerToEnsureColdCaches(): Promise<void>
expectNoDevValidationErrors: ( expectNoDevValidationErrors: (
browser: Playwright, browser: Playwright,
url: string url: string
) => Promise<void> ) => Promise<void>
getCliOutputSinceMark: () => string getCliOutputSinceMark: () => string
/** Prerender a single page with `--experimental-build-mode generate` */ /** Prerender a single page with `--experimental-build-mode generate` */
prerender: (pathname: string) => ReturnType<NextInstance['build']> prerender: (pathname: string) => ReturnType<NextInstance['build']>
...@@ -44,7 +47,7 @@ export function runInstantValidationTests( ...@@ -44,7 +47,7 @@ export function runInstantValidationTests(
const { next, skipped, isNextDev, isNextStart, isTurbopack } = const { next, skipped, isNextDev, isNextStart, isTurbopack } =
nextTestSetup({ nextTestSetup({
files: __dirname, files: __dirname,
skipStart: true, skipStart: true, // for `prerender`
skipDeployment: true, skipDeployment: true,
env: { env: {
NEXT_TEST_LOG_VALIDATION: '1', NEXT_TEST_LOG_VALIDATION: '1',
...@@ -58,19 +61,6 @@ export function runInstantValidationTests( ...@@ -58,19 +61,6 @@ export function runInstantValidationTests(
return return
} }
if (isNextStart) {
beforeAll(async () => {
await next.build({ args: ['--experimental-build-mode', 'compile'] })
})
afterEach(async () => {
await next.stop()
})
} else {
beforeAll(async () => {
await next.start()
})
}
let currentCliOutputIndex = 0 let currentCliOutputIndex = 0
beforeEach(() => { beforeEach(() => {
currentCliOutputIndex = next.cliOutput.length currentCliOutputIndex = next.cliOutput.length
...@@ -84,7 +74,35 @@ export function runInstantValidationTests( ...@@ -84,7 +74,35 @@ export function runInstantValidationTests(
return next.cliOutput.slice(currentCliOutputIndex) return next.cliOutput.slice(currentCliOutputIndex)
} }
async function restartDevServerToEnsureColdCaches() {
if (isNextDev) {
// Ensure caches are cold.
await next.stop()
await next.start()
// Restarting the server resets `next.cliOutput`.
currentCliOutputIndex = 0
}
}
if (isNextStart) {
// Compile the app first so that `prerender` can run individual prerenders.
beforeAll(async () => {
await next.build({ args: ['--experimental-build-mode', 'compile'] })
})
afterEach(async () => {
await next.stop()
})
} else {
// We set `skipStart` for prerender, so in dev we have to start the server manually.
beforeAll(async () => {
await next.start()
})
}
const prerender = async (pathname: string) => { const prerender = async (pathname: string) => {
if (!isNextStart) {
throw new Error('prerender() can only be used in `next start`')
}
const args = [ const args = [
'--experimental-build-mode', '--experimental-build-mode',
'generate', 'generate',
...@@ -113,14 +131,22 @@ export function runInstantValidationTests( ...@@ -113,14 +131,22 @@ export function runInstantValidationTests(
: [{ isClientNav: false, description: 'build' }] : [{ isClientNav: false, description: 'build' }]
describe.each(cases)('$description', ({ isClientNav }) => { describe.each(cases)('$description', ({ isClientNav }) => {
const resolveRelativeHref = (href: string) => new URL(href, next.url).href
/** /**
* Navigate to a page either via initial load or soft navigation. * Navigate to a page either via initial load or soft navigation.
* For soft nav, navigates to the index page first, then clicks the link. * For soft nav, navigates to the index page first, then clicks the link.
*/ */
async function navigateTo(href: string) { async function navigateTo(href: string, existingBrowser?: Playwright) {
if (!isClientNav) { if (!isClientNav) {
// Initial load - navigate directly // Initial load - navigate directly
const browser = await next.browser(href) let browser: Playwright
if (existingBrowser) {
await existingBrowser.get(resolveRelativeHref(href))
browser = existingBrowser
} else {
browser = await next.browser(href)
}
await browser.elementByCss('main') await browser.elementByCss('main')
return browser return browser
} }
...@@ -152,7 +178,13 @@ export function runInstantValidationTests( ...@@ -152,7 +178,13 @@ export function runInstantValidationTests(
throw new Error(`Could not find index page for ${href}`) throw new Error(`Could not find index page for ${href}`)
})() })()
const browser = await next.browser(indexPage) let browser: Playwright
if (existingBrowser) {
await existingBrowser.get(resolveRelativeHref(indexPage))
browser = existingBrowser
} else {
browser = await next.browser(indexPage)
}
const initialRootLayoutTimestamp = await browser const initialRootLayoutTimestamp = await browser
.elementById('root-layout-timestamp') .elementById('root-layout-timestamp')
...@@ -183,6 +215,27 @@ export function runInstantValidationTests( ...@@ -183,6 +215,27 @@ export function runInstantValidationTests(
return browser return browser
} }
async function warmCachesAndNavigateTo(
href: string
): Promise<Playwright> {
// First, visit the page once to warm any caches it contains.
// We add a dummy query param to differentiate validation logs
// from the two visits.
const warmingUrl = new URL(href, 'http://__n')
warmingUrl.searchParams.append('__warming_caches__', Date.now() + '')
const warmingHref = warmingUrl.pathname + warmingUrl.search
const browser = await next.browser(warmingHref)
// We wait for the validation to finish. This is longer than
// strictly necessary, we only need to wait for the initial render to finish,
// but this makes the output predictable.
await waitForValidation(warmingHref, getCliOutputSinceMark)
// Then, test either hard or client nav as usual.
await navigateTo(href, browser)
return browser
}
registerTests({ registerTests({
next, next,
isNextDev, isNextDev,
...@@ -190,6 +243,8 @@ export function runInstantValidationTests( ...@@ -190,6 +243,8 @@ export function runInstantValidationTests(
isTurbopack, isTurbopack,
isClientNav, isClientNav,
navigateTo, navigateTo,
warmCachesAndNavigateTo,
restartDevServerToEnsureColdCaches,
expectNoDevValidationErrors, expectNoDevValidationErrors,
getCliOutputSinceMark, getCliOutputSinceMark,
prerender, prerender,
......
...@@ -14,6 +14,8 @@ export function registerSuspenseBoundariesTests( ...@@ -14,6 +14,8 @@ export function registerSuspenseBoundariesTests(
isNextDev, isNextDev,
isClientNav, isClientNav,
navigateTo, navigateTo,
warmCachesAndNavigateTo,
restartDevServerToEnsureColdCaches,
expectNoDevValidationErrors, expectNoDevValidationErrors,
getCliOutputSinceMark, getCliOutputSinceMark,
prerender, prerender,
...@@ -931,6 +933,114 @@ export function registerSuspenseBoundariesTests( ...@@ -931,6 +933,114 @@ export function registerSuspenseBoundariesTests(
} }
}) })
describe('excluded caches', () => {
// Non-prefetchable caches (`stale < MIN_PREFETCHABLE_STALE`) are not allowed
// without suspense regardless of Partial Prefetching.
// TODO(app-shells): We currently report these caches as uncached data.
// The suggested fixes might be confusing, because they don't mention
// that increasing `stale` would help.
// Also, a cache being uncached sounds like a contradiction.
describe('invalid - unguarded non-prefetchable cache (with short stale)', () => {
beforeEach(async () => {
await restartDevServerToEnsureColdCaches()
})
const route =
'/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache'
it('with cold caches', async () => {
if (isNextDev) {
const browser = await navigateTo(route)
await expect(browser).toDisplayCollapsedRedbox(`
{
"cause": [
{
"label": "Caused by: Instant Validation",
"source": "app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (4:33) @ instant
> 4 | export const instant: Instant = {
| ^",
"stack": [
"instant app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (4:33)",
"Set.forEach <anonymous>",
],
},
],
"description": "Next.js encountered uncached data during a navigation.",
"environmentLabel": "Server",
"label": "Instant",
"source": "app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (21:9) @ DynamicContent
> 21 | await nonPrefetchableCache()
| ^",
"stack": [
"DynamicContent app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (21:9)",
"Page app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (15:7)",
],
}
`)
} else {
const result = await prerender(route)
expect(extractBuildValidationError(result.cliOutput))
.toMatchInlineSnapshot(`
"Error: Route "/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache": Next.js encountered uncached data during prerendering or a navigation.
\`fetch(...)\` or \`connection()\` accessed outside of \`<Suspense>\` prevents the route from being prerendered or the navigation from being instant, leading to a slower user experience.
Ways to fix this:
- [stream] Provide a placeholder with \`<Suspense fallback={...}>\` around the data access
- [cache] Cache the data access with \`"use cache"\` (does not apply to \`connection()\`)
- [block] Set \`export const instant = false\` to allow a blocking route
Learn more: https://nextjs.org/docs/messages/blocking-prerender-dynamic
at main (<anonymous>)
at body (<anonymous>)
at html (<anonymous>)
at a (<anonymous>)
Build-time instant validation failed for route "/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache".
To get a more detailed stack trace and pinpoint the issue, try one of the following:
- Start the app in development mode by running \`next dev\`, then open "/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache" in your browser to investigate the error.
- Rerun the production build with \`next build --debug-prerender\` to generate better stack traces.
Stopping prerender due to instant validation errors."
`)
expect(result.exitCode).toBe(1)
}
})
if (isNextDev) {
it('with warm caches', async () => {
const browser = await warmCachesAndNavigateTo(route)
await expect(browser).toDisplayCollapsedRedbox(`
{
"cause": [
{
"label": "Caused by: Instant Validation",
"source": "app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (4:33) @ instant
> 4 | export const instant: Instant = {
| ^",
"stack": [
"instant app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (4:33)",
"Set.forEach <anonymous>",
],
},
],
"description": "Next.js encountered uncached data during a navigation.",
"environmentLabel": "Server",
"label": "Instant",
"source": "app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (21:9) @ DynamicContent
> 21 | await nonPrefetchableCache()
| ^",
"stack": [
"DynamicContent app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (21:9)",
"Page app/suspense-in-root/excluded-caches/invalid-non-prefetchable-cache/page.tsx (15:7)",
],
}
`)
})
}
})
})
it('valid - no suspense needed around dynamic in page if loading.js is present', async () => { it('valid - no suspense needed around dynamic in page if loading.js is present', async () => {
if (isNextDev) { if (isNextDev) {
const browser = await navigateTo( const browser = await navigateTo(
......
...@@ -109,7 +109,7 @@ export function extractValidationOutput( ...@@ -109,7 +109,7 @@ export function extractValidationOutput(
export function normalizeValidationUrl(url: string): string { export function normalizeValidationUrl(url: string): string {
// RSC requests include ?_rsc=... in the URL. Strip it so the event URL // RSC requests include ?_rsc=... in the URL. Strip it so the event URL
// matches what browser.url() returns (which has no _rsc param). // matches what browser.url() returns (which has no _rsc param).
const parsed = new URL(url, 'http://n') const parsed = new URL(url, 'http://__n')
parsed.searchParams.delete('_rsc') parsed.searchParams.delete('_rsc')
return parsed.pathname + parsed.search + parsed.hash return parsed.pathname + parsed.search + parsed.hash
} }
...@@ -118,7 +118,7 @@ export async function waitForValidationStart( ...@@ -118,7 +118,7 @@ export async function waitForValidationStart(
targetUrl: string, targetUrl: string,
getOutput: () => string getOutput: () => string
): Promise<ValidationStartEvent> { ): Promise<ValidationStartEvent> {
const parsedTargetUrl = new URL(targetUrl) const parsedTargetUrl = new URL(targetUrl, 'http://__n')
const relativeTargetUrl = const relativeTargetUrl =
parsedTargetUrl.pathname + parsedTargetUrl.search + parsedTargetUrl.hash parsedTargetUrl.pathname + parsedTargetUrl.search + parsedTargetUrl.hash
......
...@@ -62,6 +62,7 @@ async function getCodeHashes( ...@@ -62,6 +62,7 @@ async function getCodeHashes(
"NEXT_PRIVATE_DEBUG_CACHE", "NEXT_PRIVATE_DEBUG_CACHE",
"__NEXT_DEV_SERVER", "__NEXT_DEV_SERVER",
"NEXT_PRIVATE_DEBUG_RUNTIME_DATA", "NEXT_PRIVATE_DEBUG_RUNTIME_DATA",
"NEXT_PRIVATE_DEBUG_VALIDATION",
"NEXT_OTEL_VERBOSE", "NEXT_OTEL_VERBOSE",
"NEXT_OTEL_PERFORMANCE_PREFIX", "NEXT_OTEL_PERFORMANCE_PREFIX",
"NEXT_SERVER_ACTIONS_ENCRYPTION_KEY", "NEXT_SERVER_ACTIONS_ENCRYPTION_KEY",
...@@ -76,6 +77,7 @@ async function getCodeHashes( ...@@ -76,6 +77,7 @@ async function getCodeHashes(
"NEXT_PRIVATE_DEBUG_CACHE", "NEXT_PRIVATE_DEBUG_CACHE",
"__NEXT_DEV_SERVER", "__NEXT_DEV_SERVER",
"NEXT_PRIVATE_DEBUG_RUNTIME_DATA", "NEXT_PRIVATE_DEBUG_RUNTIME_DATA",
"NEXT_PRIVATE_DEBUG_VALIDATION",
"NEXT_OTEL_VERBOSE", "NEXT_OTEL_VERBOSE",
"NEXT_OTEL_PERFORMANCE_PREFIX", "NEXT_OTEL_PERFORMANCE_PREFIX",
"NEXT_SERVER_ACTIONS_ENCRYPTION_KEY", "NEXT_SERVER_ACTIONS_ENCRYPTION_KEY",
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment