Unverified Commit 658c6005 authored by Jimmy Lai's avatar Jimmy Lai Committed by GitHub

app-router: new client-side cache semantics (#48383)

This PR implements new cache semantics for the app router on the client.

## Context

Currently, on the App Router, every Link navigation is prefetched and
kept forever in the cache. This means that once you visit it, you will
always see the same version of the page for the duration of your
navigation.

## This PR

This PR introduces new semantics for how the App Router will cache
during navigations. Here's a TL;DR of the changes:
- all navigations (prefetched/unprefetched) are cached for a maximum of
30s from the time it was last accessed or created (in this order).
- in addition to this, the App Router will cache differently depending
on the `prefetch` prop passed to a `<Link>` component:
  - `prefetch={undefined}`/default behaviour:
- the router will prefetch the full page for static pages/partially for
dynamic pages
    - if accessed within 30s, it will use the cache
- after that, if accessed within 5 mins, it will re-fetch and suspend
below the nearest loading.js
- after those 5 mins, it will re-fetch the full content (with a new
loading.js boundary)
  - `prefetch={false}`:
    - the router will not prefetch anything
    - if accessed within 30s again, it will re-use the page
    - after that, it will re-fetch fully
  - `prefetch={true}`
- this will prefetch the full content of your page, dynamic or static
    - if accessed within 5 mins, it will re-use the page

## Follow ups

- we may add another API to control the cache TTL at the page level
- a way to opt-in for prefetch on hover even with prefetch={false}


<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:

## For Contributors

### Improving Documentation or adding/fixing Examples

- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md

### Fixing a bug

- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md

### Adding a feature

- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md



## For Maintainers

- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change

### What?

### Why?

### How?

Closes NEXT-
Fixes #

-->

link NEXT-1011
parent db008670
...@@ -25,6 +25,7 @@ import { ...@@ -25,6 +25,7 @@ import {
ACTION_REFRESH, ACTION_REFRESH,
ACTION_RESTORE, ACTION_RESTORE,
ACTION_SERVER_PATCH, ACTION_SERVER_PATCH,
PrefetchKind,
} from './router-reducer/router-reducer-types' } from './router-reducer/router-reducer-types'
import { createHrefFromUrl } from './router-reducer/create-href-from-url' import { createHrefFromUrl } from './router-reducer/create-href-from-url'
import { import {
...@@ -234,7 +235,7 @@ function Router({ ...@@ -234,7 +235,7 @@ function Router({
const routerInstance: AppRouterInstance = { const routerInstance: AppRouterInstance = {
back: () => window.history.back(), back: () => window.history.back(),
forward: () => window.history.forward(), forward: () => window.history.forward(),
prefetch: async (href) => { prefetch: async (href, options) => {
// If prefetch has already been triggered, don't trigger it again. // If prefetch has already been triggered, don't trigger it again.
if (isBot(window.navigator.userAgent)) { if (isBot(window.navigator.userAgent)) {
return return
...@@ -244,12 +245,12 @@ function Router({ ...@@ -244,12 +245,12 @@ function Router({
if (isExternalURL(url)) { if (isExternalURL(url)) {
return return
} }
// @ts-ignore startTransition exists // @ts-ignore startTransition exists
React.startTransition(() => { React.startTransition(() => {
dispatch({ dispatch({
type: ACTION_PREFETCH, type: ACTION_PREFETCH,
url, url,
kind: options?.kind ?? PrefetchKind.FULL,
}) })
}) })
}, },
......
...@@ -277,16 +277,7 @@ function InnerLayoutRouter({ ...@@ -277,16 +277,7 @@ function InnerLayoutRouter({
// TODO-APP: verify if this can be null based on user code // TODO-APP: verify if this can be null based on user code
childProp.current !== null childProp.current !== null
) { ) {
if (childNode) { if (!childNode) {
if (childNode.status === CacheStates.LAZY_INITIALIZED) {
// @ts-expect-error we're changing it's type!
childNode.status = CacheStates.READY
// @ts-expect-error
childNode.subTreeData = childProp.current
// Mutates the prop in order to clean up the memory associated with the subTreeData as it is now part of the cache.
childProp.current = null
}
} else {
// Add the segment's subTreeData to the cache. // Add the segment's subTreeData to the cache.
// This writes to the cache when there is no item in the cache yet. It never *overwrites* existing cache items which is why it's safe in concurrent mode. // This writes to the cache when there is no item in the cache yet. It never *overwrites* existing cache items which is why it's safe in concurrent mode.
childNodes.set(cacheKey, { childNodes.set(cacheKey, {
...@@ -295,10 +286,15 @@ function InnerLayoutRouter({ ...@@ -295,10 +286,15 @@ function InnerLayoutRouter({
subTreeData: childProp.current, subTreeData: childProp.current,
parallelRoutes: new Map(), parallelRoutes: new Map(),
}) })
// Mutates the prop in order to clean up the memory associated with the subTreeData as it is now part of the cache.
childProp.current = null
// In the above case childNode was set on childNodes, so we have to get it from the cacheNodes again. // In the above case childNode was set on childNodes, so we have to get it from the cacheNodes again.
childNode = childNodes.get(cacheKey) childNode = childNodes.get(cacheKey)
} else {
if (childNode.status === CacheStates.LAZY_INITIALIZED) {
// @ts-expect-error we're changing it's type!
childNode.status = CacheStates.READY
// @ts-expect-error
childNode.subTreeData = childProp.current
}
} }
} }
......
...@@ -7,7 +7,7 @@ export function applyFlightData( ...@@ -7,7 +7,7 @@ export function applyFlightData(
existingCache: CacheNode, existingCache: CacheNode,
cache: CacheNode, cache: CacheNode,
flightDataPath: FlightDataPath, flightDataPath: FlightDataPath,
wasPrefetched?: boolean wasPrefetched: boolean = false
): boolean { ): boolean {
// The one before last item is the router state tree patch // The one before last item is the router state tree patch
const [treePatch, subTreeData, head] = flightDataPath.slice(-3) const [treePatch, subTreeData, head] = flightDataPath.slice(-3)
...@@ -33,7 +33,12 @@ export function applyFlightData( ...@@ -33,7 +33,12 @@ export function applyFlightData(
cache.subTreeData = existingCache.subTreeData cache.subTreeData = existingCache.subTreeData
cache.parallelRoutes = new Map(existingCache.parallelRoutes) cache.parallelRoutes = new Map(existingCache.parallelRoutes)
// Create a copy of the existing cache with the subTreeData applied. // Create a copy of the existing cache with the subTreeData applied.
fillCacheWithNewSubTreeData(cache, existingCache, flightDataPath) fillCacheWithNewSubTreeData(
cache,
existingCache,
flightDataPath,
wasPrefetched
)
} }
return true return true
......
...@@ -14,6 +14,7 @@ import { ...@@ -14,6 +14,7 @@ import {
} from '../app-router-headers' } from '../app-router-headers'
import { urlToUrlWithoutFlightMarker } from '../app-router' import { urlToUrlWithoutFlightMarker } from '../app-router'
import { callServer } from '../../app-call-server' import { callServer } from '../../app-call-server'
import { PrefetchKind } from './router-reducer-types'
/** /**
* Fetch the flight data for the provided url. Takes in the current router state to decide what to render server-side. * Fetch the flight data for the provided url. Takes in the current router state to decide what to render server-side.
...@@ -23,7 +24,7 @@ export async function fetchServerResponse( ...@@ -23,7 +24,7 @@ export async function fetchServerResponse(
url: URL, url: URL,
flightRouterState: FlightRouterState, flightRouterState: FlightRouterState,
nextUrl: string | null, nextUrl: string | null,
prefetch?: true prefetchKind?: PrefetchKind
): Promise<[FlightData: FlightData, canonicalUrlOverride: URL | undefined]> { ): Promise<[FlightData: FlightData, canonicalUrlOverride: URL | undefined]> {
const headers: { const headers: {
[RSC]: '1' [RSC]: '1'
...@@ -36,8 +37,14 @@ export async function fetchServerResponse( ...@@ -36,8 +37,14 @@ export async function fetchServerResponse(
// Provide the current router state // Provide the current router state
[NEXT_ROUTER_STATE_TREE]: JSON.stringify(flightRouterState), [NEXT_ROUTER_STATE_TREE]: JSON.stringify(flightRouterState),
} }
if (prefetch) {
// Enable prefetch response /**
* Three cases:
* - `prefetchKind` is `undefined`, it means it's a normal navigation, so we want to prefetch the page data fully
* - `prefetchKind` is `full` - we want to prefetch the whole page so same as above
* - `prefetchKind` is `auto` - if the page is dynamic, prefetch the page data partially, if static prefetch the page data fully
*/
if (prefetchKind === PrefetchKind.AUTO) {
headers[NEXT_ROUTER_PREFETCH] = '1' headers[NEXT_ROUTER_PREFETCH] = '1'
} }
......
import { FlightSegmentPath } from '../../../server/app-render/types'
import { CacheNode, CacheStates } from '../../../shared/lib/app-router-context' import { CacheNode, CacheStates } from '../../../shared/lib/app-router-context'
import { createRouterCacheKey } from './create-router-cache-key'
import { fetchServerResponse } from './fetch-server-response' import { fetchServerResponse } from './fetch-server-response'
/** /**
...@@ -7,19 +9,24 @@ import { fetchServerResponse } from './fetch-server-response' ...@@ -7,19 +9,24 @@ import { fetchServerResponse } from './fetch-server-response'
export function fillCacheWithDataProperty( export function fillCacheWithDataProperty(
newCache: CacheNode, newCache: CacheNode,
existingCache: CacheNode, existingCache: CacheNode,
segments: string[], flightSegmentPath: FlightSegmentPath,
fetchResponse: () => ReturnType<typeof fetchServerResponse> fetchResponse: () => ReturnType<typeof fetchServerResponse>,
bailOnParallelRoutes: boolean = false
): { bailOptimistic: boolean } | undefined { ): { bailOptimistic: boolean } | undefined {
const isLastEntry = segments.length === 1 const isLastEntry = flightSegmentPath.length <= 2
const parallelRouteKey = 'children' const [parallelRouteKey, segment] = flightSegmentPath
const [segment] = segments const cacheKey = createRouterCacheKey(segment)
const existingChildSegmentMap = const existingChildSegmentMap =
existingCache.parallelRoutes.get(parallelRouteKey) existingCache.parallelRoutes.get(parallelRouteKey)
if (!existingChildSegmentMap) { if (
!existingChildSegmentMap ||
(bailOnParallelRoutes && existingCache.parallelRoutes.size > 1)
) {
// Bailout because the existing cache does not have the path to the leaf node // Bailout because the existing cache does not have the path to the leaf node
// or the existing cache has multiple parallel routes
// Will trigger lazy fetch in layout-router because of missing segment // Will trigger lazy fetch in layout-router because of missing segment
return { bailOptimistic: true } return { bailOptimistic: true }
} }
...@@ -31,8 +38,8 @@ export function fillCacheWithDataProperty( ...@@ -31,8 +38,8 @@ export function fillCacheWithDataProperty(
newCache.parallelRoutes.set(parallelRouteKey, childSegmentMap) newCache.parallelRoutes.set(parallelRouteKey, childSegmentMap)
} }
const existingChildCacheNode = existingChildSegmentMap.get(segment) const existingChildCacheNode = existingChildSegmentMap.get(cacheKey)
let childCacheNode = childSegmentMap.get(segment) let childCacheNode = childSegmentMap.get(cacheKey)
// In case of last segment start off the fetch at this level and don't copy further down. // In case of last segment start off the fetch at this level and don't copy further down.
if (isLastEntry) { if (isLastEntry) {
...@@ -41,7 +48,7 @@ export function fillCacheWithDataProperty( ...@@ -41,7 +48,7 @@ export function fillCacheWithDataProperty(
!childCacheNode.data || !childCacheNode.data ||
childCacheNode === existingChildCacheNode childCacheNode === existingChildCacheNode
) { ) {
childSegmentMap.set(segment, { childSegmentMap.set(cacheKey, {
status: CacheStates.DATA_FETCH, status: CacheStates.DATA_FETCH,
data: fetchResponse(), data: fetchResponse(),
subTreeData: null, subTreeData: null,
...@@ -54,7 +61,7 @@ export function fillCacheWithDataProperty( ...@@ -54,7 +61,7 @@ export function fillCacheWithDataProperty(
if (!childCacheNode || !existingChildCacheNode) { if (!childCacheNode || !existingChildCacheNode) {
// Start fetch in the place where the existing cache doesn't have the data yet. // Start fetch in the place where the existing cache doesn't have the data yet.
if (!childCacheNode) { if (!childCacheNode) {
childSegmentMap.set(segment, { childSegmentMap.set(cacheKey, {
status: CacheStates.DATA_FETCH, status: CacheStates.DATA_FETCH,
data: fetchResponse(), data: fetchResponse(),
subTreeData: null, subTreeData: null,
...@@ -71,13 +78,13 @@ export function fillCacheWithDataProperty( ...@@ -71,13 +78,13 @@ export function fillCacheWithDataProperty(
subTreeData: childCacheNode.subTreeData, subTreeData: childCacheNode.subTreeData,
parallelRoutes: new Map(childCacheNode.parallelRoutes), parallelRoutes: new Map(childCacheNode.parallelRoutes),
} as CacheNode } as CacheNode
childSegmentMap.set(segment, childCacheNode) childSegmentMap.set(cacheKey, childCacheNode)
} }
return fillCacheWithDataProperty( return fillCacheWithDataProperty(
childCacheNode, childCacheNode,
existingChildCacheNode, existingChildCacheNode,
segments.slice(1), flightSegmentPath.slice(2),
fetchResponse fetchResponse
) )
} }
...@@ -78,7 +78,7 @@ describe('fillCacheWithNewSubtreeData', () => { ...@@ -78,7 +78,7 @@ describe('fillCacheWithNewSubtreeData', () => {
// Mirrors the way router-reducer values are passed in. // Mirrors the way router-reducer values are passed in.
const flightDataPath = flightData[0] const flightDataPath = flightData[0]
fillCacheWithNewSubTreeData(cache, existingCache, flightDataPath) fillCacheWithNewSubTreeData(cache, existingCache, flightDataPath, false)
const expectedCache: CacheNode = { const expectedCache: CacheNode = {
data: null, data: null,
......
...@@ -10,7 +10,8 @@ import { createRouterCacheKey } from './create-router-cache-key' ...@@ -10,7 +10,8 @@ import { createRouterCacheKey } from './create-router-cache-key'
export function fillCacheWithNewSubTreeData( export function fillCacheWithNewSubTreeData(
newCache: CacheNode, newCache: CacheNode,
existingCache: CacheNode, existingCache: CacheNode,
flightDataPath: FlightDataPath flightDataPath: FlightDataPath,
wasPrefetched?: boolean
): void { ): void {
const isLastEntry = flightDataPath.length <= 5 const isLastEntry = flightDataPath.length <= 5
const [parallelRouteKey, segment] = flightDataPath const [parallelRouteKey, segment] = flightDataPath
...@@ -63,7 +64,8 @@ export function fillCacheWithNewSubTreeData( ...@@ -63,7 +64,8 @@ export function fillCacheWithNewSubTreeData(
childCacheNode, childCacheNode,
existingChildCacheNode, existingChildCacheNode,
flightDataPath[2], flightDataPath[2],
flightDataPath[4] flightDataPath[4],
wasPrefetched
) )
childSegmentMap.set(cacheKey, childCacheNode) childSegmentMap.set(cacheKey, childCacheNode)
...@@ -90,6 +92,7 @@ export function fillCacheWithNewSubTreeData( ...@@ -90,6 +92,7 @@ export function fillCacheWithNewSubTreeData(
fillCacheWithNewSubTreeData( fillCacheWithNewSubTreeData(
childCacheNode, childCacheNode,
existingChildCacheNode, existingChildCacheNode,
flightDataPath.slice(2) flightDataPath.slice(2),
wasPrefetched
) )
} }
import { PrefetchCacheEntry } from './router-reducer-types'
const FIVE_MINUTES = 5 * 60 * 1000
const THIRTY_SECONDS = 30 * 1000
export enum PrefetchCacheEntryStatus {
fresh = 'fresh',
reusable = 'reusable',
expired = 'expired',
stale = 'stale',
}
export function getPrefetchEntryCacheStatus({
kind,
prefetchTime,
lastUsedTime,
}: PrefetchCacheEntry): PrefetchCacheEntryStatus {
// if the cache entry was prefetched or read less than 30s ago, then we want to re-use it
if (Date.now() < (lastUsedTime ?? prefetchTime) + THIRTY_SECONDS) {
return lastUsedTime
? PrefetchCacheEntryStatus.reusable
: PrefetchCacheEntryStatus.fresh
}
// if the cache entry was prefetched less than 5 mins ago, then we want to re-use only the loading state
if (kind === 'auto') {
if (Date.now() < prefetchTime + FIVE_MINUTES) {
return PrefetchCacheEntryStatus.stale
}
}
// if the cache entry was prefetched less than 5 mins ago and was a "full" prefetch, then we want to re-use it "full
if (kind === 'full') {
if (Date.now() < prefetchTime + FIVE_MINUTES) {
return PrefetchCacheEntryStatus.reusable
}
}
return PrefetchCacheEntryStatus.expired
}
...@@ -86,7 +86,7 @@ describe('invalidateCacheBelowFlightSegmentPath', () => { ...@@ -86,7 +86,7 @@ describe('invalidateCacheBelowFlightSegmentPath', () => {
// @ts-expect-error TODO-APP: investigate why this is not a TS error in router-reducer. // @ts-expect-error TODO-APP: investigate why this is not a TS error in router-reducer.
cache.subTreeData = existingCache.subTreeData cache.subTreeData = existingCache.subTreeData
// Create a copy of the existing cache with the subTreeData applied. // Create a copy of the existing cache with the subTreeData applied.
fillCacheWithNewSubTreeData(cache, existingCache, flightDataPath) fillCacheWithNewSubTreeData(cache, existingCache, flightDataPath, false)
// Invalidate the cache below the flight segment path. This should remove the 'about' node. // Invalidate the cache below the flight segment path. This should remove the 'about' node.
invalidateCacheBelowFlightSegmentPath( invalidateCacheBelowFlightSegmentPath(
......
...@@ -80,6 +80,7 @@ import { ...@@ -80,6 +80,7 @@ import {
ACTION_NAVIGATE, ACTION_NAVIGATE,
ACTION_PREFETCH, ACTION_PREFETCH,
PrefetchAction, PrefetchAction,
PrefetchKind,
} from '../router-reducer-types' } from '../router-reducer-types'
import { navigateReducer } from './navigate-reducer' import { navigateReducer } from './navigate-reducer'
import { prefetchReducer } from './prefetch-reducer' import { prefetchReducer } from './prefetch-reducer'
...@@ -1005,6 +1006,7 @@ describe('navigateReducer', () => { ...@@ -1005,6 +1006,7 @@ describe('navigateReducer', () => {
const prefetchAction: PrefetchAction = { const prefetchAction: PrefetchAction = {
type: ACTION_PREFETCH, type: ACTION_PREFETCH,
url, url,
kind: PrefetchKind.AUTO,
} }
const state = createInitialRouterState({ const state = createInitialRouterState({
...@@ -1086,6 +1088,9 @@ describe('navigateReducer', () => { ...@@ -1086,6 +1088,9 @@ describe('navigateReducer', () => {
'/linking/about', '/linking/about',
{ {
data: record, data: record,
kind: PrefetchKind.AUTO,
lastUsedTime: null,
prefetchTime: expect.any(Number),
treeAtTimeOfPrefetch: [ treeAtTimeOfPrefetch: [
'', '',
{ {
......
...@@ -38,7 +38,11 @@ import { ...@@ -38,7 +38,11 @@ import {
CacheStates, CacheStates,
} from '../../../../shared/lib/app-router-context' } from '../../../../shared/lib/app-router-context'
import { createInitialRouterState } from '../create-initial-router-state' import { createInitialRouterState } from '../create-initial-router-state'
import { PrefetchAction, ACTION_PREFETCH } from '../router-reducer-types' import {
PrefetchAction,
ACTION_PREFETCH,
PrefetchKind,
} from '../router-reducer-types'
import { prefetchReducer } from './prefetch-reducer' import { prefetchReducer } from './prefetch-reducer'
import { fetchServerResponse } from '../fetch-server-response' import { fetchServerResponse } from '../fetch-server-response'
import { createRecordFromThenable } from '../create-record-from-thenable' import { createRecordFromThenable } from '../create-record-from-thenable'
...@@ -128,11 +132,12 @@ describe('prefetchReducer', () => { ...@@ -128,11 +132,12 @@ describe('prefetchReducer', () => {
url, url,
initialTree, initialTree,
null, null,
true PrefetchKind.AUTO
) )
const action: PrefetchAction = { const action: PrefetchAction = {
type: ACTION_PREFETCH, type: ACTION_PREFETCH,
url, url,
kind: PrefetchKind.AUTO,
} }
const newState = await runPromiseThrowChain(() => const newState = await runPromiseThrowChain(() =>
...@@ -149,6 +154,9 @@ describe('prefetchReducer', () => { ...@@ -149,6 +154,9 @@ describe('prefetchReducer', () => {
'/linking/about', '/linking/about',
{ {
data: record, data: record,
kind: PrefetchKind.AUTO,
lastUsedTime: null,
prefetchTime: expect.any(Number),
treeAtTimeOfPrefetch: [ treeAtTimeOfPrefetch: [
'', '',
{ {
...@@ -273,11 +281,12 @@ describe('prefetchReducer', () => { ...@@ -273,11 +281,12 @@ describe('prefetchReducer', () => {
url, url,
initialTree, initialTree,
null, null,
true PrefetchKind.AUTO
) )
const action: PrefetchAction = { const action: PrefetchAction = {
type: ACTION_PREFETCH, type: ACTION_PREFETCH,
url, url,
kind: PrefetchKind.AUTO,
} }
await runPromiseThrowChain(() => prefetchReducer(state, action)) await runPromiseThrowChain(() => prefetchReducer(state, action))
...@@ -296,6 +305,9 @@ describe('prefetchReducer', () => { ...@@ -296,6 +305,9 @@ describe('prefetchReducer', () => {
'/linking/about', '/linking/about',
{ {
data: record, data: record,
prefetchTime: expect.any(Number),
kind: PrefetchKind.AUTO,
lastUsedTime: null,
treeAtTimeOfPrefetch: [ treeAtTimeOfPrefetch: [
'', '',
{ {
......
...@@ -4,13 +4,18 @@ import { ...@@ -4,13 +4,18 @@ import {
PrefetchAction, PrefetchAction,
ReducerState, ReducerState,
ReadonlyReducerState, ReadonlyReducerState,
PrefetchKind,
} from '../router-reducer-types' } from '../router-reducer-types'
import { createRecordFromThenable } from '../create-record-from-thenable' import { createRecordFromThenable } from '../create-record-from-thenable'
import { prunePrefetchCache } from './prune-prefetch-cache'
export function prefetchReducer( export function prefetchReducer(
state: ReadonlyReducerState, state: ReadonlyReducerState,
action: PrefetchAction action: PrefetchAction
): ReducerState { ): ReducerState {
// let's prune the prefetch cache before we do anything else
prunePrefetchCache(state.prefetchCache)
const { url } = action const { url } = action
const href = createHrefFromUrl( const href = createHrefFromUrl(
url, url,
...@@ -18,9 +23,32 @@ export function prefetchReducer( ...@@ -18,9 +23,32 @@ export function prefetchReducer(
false false
) )
// If the href was already prefetched it is not necessary to prefetch it again const cacheEntry = state.prefetchCache.get(href)
if (state.prefetchCache.has(href)) { if (cacheEntry) {
return state /**
* If the cache entry present was marked as temporary, it means that we prefetched it from the navigate reducer,
* where we didn't have the prefetch intent. We want to update it to the new, more accurate, kind here.
*/
if (cacheEntry.kind === PrefetchKind.TEMPORARY) {
console.log(href, action.kind, cacheEntry)
state.prefetchCache.set(href, {
...cacheEntry,
kind: action.kind,
})
}
/**
* if the prefetch action was a full prefetch and that the current cache entry wasn't one, we want to re-prefetch,
* otherwise we can re-use the current cache entry
**/
if (
!(
cacheEntry.kind === PrefetchKind.AUTO &&
action.kind === PrefetchKind.FULL
)
) {
return state
}
} }
// fetchServerResponse is intentionally not awaited so that it can be unwrapped in the navigate-reducer // fetchServerResponse is intentionally not awaited so that it can be unwrapped in the navigate-reducer
...@@ -30,7 +58,7 @@ export function prefetchReducer( ...@@ -30,7 +58,7 @@ export function prefetchReducer(
// initialTree is used when history.state.tree is missing because the history state is set in `useEffect` below, it being missing means this is the hydration case. // initialTree is used when history.state.tree is missing because the history state is set in `useEffect` below, it being missing means this is the hydration case.
state.tree, state.tree,
state.nextUrl, state.nextUrl,
true action.kind
) )
) )
...@@ -39,6 +67,9 @@ export function prefetchReducer( ...@@ -39,6 +67,9 @@ export function prefetchReducer(
// Create new tree based on the flightSegmentPath and router state patch // Create new tree based on the flightSegmentPath and router state patch
treeAtTimeOfPrefetch: state.tree, treeAtTimeOfPrefetch: state.tree,
data: serverResponse, data: serverResponse,
kind: action.kind,
prefetchTime: Date.now(),
lastUsedTime: null,
}) })
return state return state
......
import type { ReducerState } from '../router-reducer-types'
import {
PrefetchCacheEntryStatus,
getPrefetchEntryCacheStatus,
} from '../get-prefetch-cache-entry-status'
export function prunePrefetchCache(
prefetchCache: ReducerState['prefetchCache']
) {
for (const [href, prefetchCacheEntry] of prefetchCache) {
if (
getPrefetchEntryCacheStatus(prefetchCacheEntry) ===
PrefetchCacheEntryStatus.expired
) {
prefetchCache.delete(href)
}
}
}
...@@ -117,6 +117,19 @@ export interface ServerPatchAction { ...@@ -117,6 +117,19 @@ export interface ServerPatchAction {
mutable: Mutable mutable: Mutable
} }
/**
* PrefetchKind defines the type of prefetching that should be done.
* - `auto` - if the page is dynamic, prefetch the page data partially, if static prefetch the page data fully.
* - `full` - prefetch the page data fully.
* - `temporary` - a temporary prefetch entry is added to the cache, this is used when prefetch={false} is used in next/link or when you push a route programmatically.
*/
export enum PrefetchKind {
AUTO = 'auto',
FULL = 'full',
TEMPORARY = 'temporary',
}
/** /**
* Prefetch adds the provided FlightData to the prefetch cache * Prefetch adds the provided FlightData to the prefetch cache
* - Creates the router state tree based on the patch in FlightData * - Creates the router state tree based on the patch in FlightData
...@@ -126,6 +139,7 @@ export interface ServerPatchAction { ...@@ -126,6 +139,7 @@ export interface ServerPatchAction {
export interface PrefetchAction { export interface PrefetchAction {
type: typeof ACTION_PREFETCH type: typeof ACTION_PREFETCH
url: URL url: URL
kind: PrefetchKind
} }
interface PushRef { interface PushRef {
...@@ -154,6 +168,14 @@ export type FocusAndScrollRef = { ...@@ -154,6 +168,14 @@ export type FocusAndScrollRef = {
segmentPaths: FlightSegmentPath[] segmentPaths: FlightSegmentPath[]
} }
export type PrefetchCacheEntry = {
treeAtTimeOfPrefetch: FlightRouterState
data: ReturnType<typeof fetchServerResponse> | null
kind: PrefetchKind
prefetchTime: number
lastUsedTime: number | null
}
/** /**
* Handles keeping the state of app-router. * Handles keeping the state of app-router.
*/ */
...@@ -173,13 +195,7 @@ export type AppRouterState = { ...@@ -173,13 +195,7 @@ export type AppRouterState = {
/** /**
* Cache that holds prefetched Flight responses keyed by url. * Cache that holds prefetched Flight responses keyed by url.
*/ */
prefetchCache: Map< prefetchCache: Map<string, PrefetchCacheEntry>
string,
{
treeAtTimeOfPrefetch: FlightRouterState
data: ReturnType<typeof fetchServerResponse> | null
}
>
/** /**
* Decides if the update should create a new history entry and if the navigation has to trigger a browser navigation. * Decides if the update should create a new history entry and if the navigation has to trigger a browser navigation.
*/ */
......
...@@ -16,10 +16,12 @@ import { RouterContext } from '../shared/lib/router-context' ...@@ -16,10 +16,12 @@ import { RouterContext } from '../shared/lib/router-context'
import { import {
AppRouterContext, AppRouterContext,
AppRouterInstance, AppRouterInstance,
PrefetchOptions as AppRouterPrefetchOptions,
} from '../shared/lib/app-router-context' } from '../shared/lib/app-router-context'
import { useIntersection } from './use-intersection' import { useIntersection } from './use-intersection'
import { getDomainLocale } from './get-domain-locale' import { getDomainLocale } from './get-domain-locale'
import { addBasePath } from './add-base-path' import { addBasePath } from './add-base-path'
import { PrefetchKind } from './components/router-reducer/router-reducer-types'
type Url = string | UrlObject type Url = string | UrlObject
type RequiredKeys<T> = { type RequiredKeys<T> = {
...@@ -120,6 +122,7 @@ function prefetch( ...@@ -120,6 +122,7 @@ function prefetch(
href: string, href: string,
as: string, as: string,
options: PrefetchOptions, options: PrefetchOptions,
appOptions: AppRouterPrefetchOptions,
isAppRouter: boolean isAppRouter: boolean
): void { ): void {
if (typeof window === 'undefined') { if (typeof window === 'undefined') {
...@@ -154,11 +157,15 @@ function prefetch( ...@@ -154,11 +157,15 @@ function prefetch(
prefetched.add(prefetchedKey) prefetched.add(prefetchedKey)
} }
const prefetchPromise = isAppRouter
? (router as AppRouterInstance).prefetch(href, appOptions)
: (router as NextRouter).prefetch(href, as, options)
// Prefetch the JSON page if asked (only in the client) // Prefetch the JSON page if asked (only in the client)
// We need to handle a prefetch error here since we may be // We need to handle a prefetch error here since we may be
// loading with priority which can reject but we don't // loading with priority which can reject but we don't
// want to force navigation since this is only a prefetch // want to force navigation since this is only a prefetch
Promise.resolve(router.prefetch(href, as, options)).catch((err) => { Promise.resolve(prefetchPromise).catch((err) => {
if (process.env.NODE_ENV !== 'production') { if (process.env.NODE_ENV !== 'production') {
// rethrow to show invalid URL errors // rethrow to show invalid URL errors
throw err throw err
...@@ -248,6 +255,51 @@ function formatStringOrUrl(urlObjOrString: UrlObject | string): string { ...@@ -248,6 +255,51 @@ function formatStringOrUrl(urlObjOrString: UrlObject | string): string {
*/ */
const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>( const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>(
function LinkComponent(props, forwardedRef) { function LinkComponent(props, forwardedRef) {
let children: React.ReactNode
const {
href: hrefProp,
as: asProp,
children: childrenProp,
prefetch: prefetchProp = null,
passHref,
replace,
shallow,
scroll,
locale,
onClick,
onMouseEnter: onMouseEnterProp,
onTouchStart: onTouchStartProp,
// @ts-expect-error this is inlined as a literal boolean not a string
legacyBehavior = process.env.__NEXT_NEW_LINK_BEHAVIOR === false,
...restProps
} = props
children = childrenProp
if (
legacyBehavior &&
(typeof children === 'string' || typeof children === 'number')
) {
children = <a>{children}</a>
}
const prefetchEnabled = prefetchProp !== false
/**
* The possible states for prefetch are:
* - null: this is the default "auto" mode, where we will prefetch partially if the link is in the viewport
* - true: we will prefetch if the link is visible and prefetch the full page, not just partially
* - false: we will not prefetch if in the viewport at all
*/
const appPrefetchKind =
prefetchProp === null ? PrefetchKind.AUTO : PrefetchKind.FULL
const pagesRouter = React.useContext(RouterContext)
const appRouter = React.useContext(AppRouterContext)
const router = pagesRouter ?? appRouter
// We're in the app directory if there is no pages router.
const isAppRouter = !pagesRouter
if (process.env.NODE_ENV !== 'production') { if (process.env.NODE_ENV !== 'production') {
function createPropError(args: { function createPropError(args: {
key: string key: string
...@@ -361,7 +413,7 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>( ...@@ -361,7 +413,7 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>(
// This hook is in a conditional but that is ok because `process.env.NODE_ENV` never changes // This hook is in a conditional but that is ok because `process.env.NODE_ENV` never changes
// eslint-disable-next-line react-hooks/rules-of-hooks // eslint-disable-next-line react-hooks/rules-of-hooks
const hasWarned = React.useRef(false) const hasWarned = React.useRef(false)
if (props.prefetch && !hasWarned.current) { if (props.prefetch && !hasWarned.current && !isAppRouter) {
hasWarned.current = true hasWarned.current = true
console.warn( console.warn(
'Next.js auto-prefetches automatically based on viewport. The prefetch attribute is no longer needed. More: https://nextjs.org/docs/messages/prefetch-true-deprecated' 'Next.js auto-prefetches automatically based on viewport. The prefetch attribute is no longer needed. More: https://nextjs.org/docs/messages/prefetch-true-deprecated'
...@@ -369,44 +421,6 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>( ...@@ -369,44 +421,6 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>(
} }
} }
let children: React.ReactNode
const {
href: hrefProp,
as: asProp,
children: childrenProp,
prefetch: prefetchProp,
passHref,
replace,
shallow,
scroll,
locale,
onClick,
onMouseEnter: onMouseEnterProp,
onTouchStart: onTouchStartProp,
// @ts-expect-error this is inlined as a literal boolean not a string
legacyBehavior = process.env.__NEXT_NEW_LINK_BEHAVIOR === false,
...restProps
} = props
children = childrenProp
if (
legacyBehavior &&
(typeof children === 'string' || typeof children === 'number')
) {
children = <a>{children}</a>
}
const prefetchEnabled = prefetchProp !== false
const pagesRouter = React.useContext(RouterContext)
const appRouter = React.useContext(AppRouterContext)
const router = pagesRouter ?? appRouter
// We're in the app directory if there is no pages router.
const isAppRouter = !pagesRouter
if (process.env.NODE_ENV !== 'production') { if (process.env.NODE_ENV !== 'production') {
if (isAppRouter && !asProp) { if (isAppRouter && !asProp) {
let href: string | undefined let href: string | undefined
...@@ -546,7 +560,16 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>( ...@@ -546,7 +560,16 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>(
} }
// Prefetch the URL. // Prefetch the URL.
prefetch(router, href, as, { locale }, isAppRouter) prefetch(
router,
href,
as,
{ locale },
{
kind: appPrefetchKind,
},
isAppRouter
)
}, [ }, [
as, as,
href, href,
...@@ -556,6 +579,7 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>( ...@@ -556,6 +579,7 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>(
pagesRouter?.locale, pagesRouter?.locale,
router, router,
isAppRouter, isAppRouter,
appPrefetchKind,
]) ])
const childProps: { const childProps: {
...@@ -639,6 +663,9 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>( ...@@ -639,6 +663,9 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>(
// @see {https://github.com/vercel/next.js/discussions/40268?sort=top#discussioncomment-3572642} // @see {https://github.com/vercel/next.js/discussions/40268?sort=top#discussioncomment-3572642}
bypassPrefetchedCheck: true, bypassPrefetchedCheck: true,
}, },
{
kind: appPrefetchKind,
},
isAppRouter isAppRouter
) )
}, },
...@@ -673,6 +700,9 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>( ...@@ -673,6 +700,9 @@ const Link = React.forwardRef<HTMLAnchorElement, LinkPropsReal>(
// @see {https://github.com/vercel/next.js/discussions/40268?sort=top#discussioncomment-3572642} // @see {https://github.com/vercel/next.js/discussions/40268?sort=top#discussioncomment-3572642}
bypassPrefetchedCheck: true, bypassPrefetchedCheck: true,
}, },
{
kind: appPrefetchKind,
},
isAppRouter isAppRouter
) )
}, },
......
'use client' 'use client'
import { FocusAndScrollRef } from '../../client/components/router-reducer/router-reducer-types' import {
FocusAndScrollRef,
PrefetchKind,
} from '../../client/components/router-reducer/router-reducer-types'
import type { fetchServerResponse } from '../../client/components/router-reducer/fetch-server-response' import type { fetchServerResponse } from '../../client/components/router-reducer/fetch-server-response'
import type { import type {
FlightRouterState, FlightRouterState,
...@@ -68,6 +71,10 @@ export interface NavigateOptions { ...@@ -68,6 +71,10 @@ export interface NavigateOptions {
forceOptimisticNavigation?: boolean forceOptimisticNavigation?: boolean
} }
export interface PrefetchOptions {
kind: PrefetchKind
}
export interface AppRouterInstance { export interface AppRouterInstance {
/** /**
* Navigate to the previous history entry. * Navigate to the previous history entry.
...@@ -94,7 +101,7 @@ export interface AppRouterInstance { ...@@ -94,7 +101,7 @@ export interface AppRouterInstance {
/** /**
* Prefetch the provided href. * Prefetch the provided href.
*/ */
prefetch(href: string): void prefetch(href: string, options?: PrefetchOptions): void
} }
export const AppRouterContext = React.createContext<AppRouterInstance | null>( export const AppRouterContext = React.createContext<AppRouterInstance | null>(
......
export default async function Page() {
const randomNumber = Math.random()
return (
<div>
<div>LOADING</div>
<div id="loading">{randomNumber}</div>
</div>
)
}
import Link from 'next/link'
export default async function Page({ searchParams: { timeout } }) {
const randomNumber = await new Promise((resolve) => {
setTimeout(
() => {
resolve(Math.random())
},
timeout !== undefined ? Number.parseInt(timeout, 10) : 0
)
})
return (
<>
<div>
<Link href="/"> Back to Home </Link>
</div>
<div id="random-number">{randomNumber}</div>
</>
)
}
export default function Root({ children }) {
return (
<html>
<head></head>
<body>{children}</body>
</html>
)
}
import Link from 'next/link'
export default function HomePage() {
return (
<>
<div>
<Link href="/0?timeout=0" prefetch={true}>
To Random Number - prefetch: true
</Link>
</div>
<div>
<Link href="/1">To Random Number - prefetch: auto</Link>
</div>
<div>
<Link href="/2" prefetch={false}>
To Random Number 2 - prefetch: false
</Link>
</div>
<div>
<Link href="/1?timeout=1000">
To Random Number - prefetch: auto, slow
</Link>
</div>
</>
)
}
This diff is collapsed.
module.exports = {
experimental: {
appDir: true,
},
}
import { createNextDescribe } from 'e2e-utils' import { createNextDescribe } from 'e2e-utils'
import { check, waitFor } from 'next-test-utils' import { check, waitFor } from 'next-test-utils'
const browserConfigWithFixedTime = {
beforePageLoad: (page) => {
page.addInitScript(() => {
const startTime = new Date()
const fixedTime = new Date('2023-04-17T00:00:00Z')
// Override the Date constructor
// @ts-ignore
// eslint-disable-next-line no-native-reassign
Date = class extends Date {
constructor() {
super()
// @ts-ignore
return new startTime.constructor(fixedTime)
}
static now() {
return fixedTime.getTime()
}
}
})
},
}
createNextDescribe( createNextDescribe(
'app dir prefetching', 'app dir prefetching',
{ {
...@@ -15,7 +39,7 @@ createNextDescribe( ...@@ -15,7 +39,7 @@ createNextDescribe(
} }
it('should show layout eagerly when prefetched with loading one level down', async () => { it('should show layout eagerly when prefetched with loading one level down', async () => {
const browser = await next.browser('/') const browser = await next.browser('/', browserConfigWithFixedTime)
// Ensure the page is prefetched // Ensure the page is prefetched
await waitFor(1000) await waitFor(1000)
...@@ -50,7 +74,7 @@ createNextDescribe( ...@@ -50,7 +74,7 @@ createNextDescribe(
}) })
it('should not fetch again when a static page was prefetched', async () => { it('should not fetch again when a static page was prefetched', async () => {
const browser = await next.browser('/404') const browser = await next.browser('/404', browserConfigWithFixedTime)
let requests: string[] = [] let requests: string[] = []
browser.on('request', (req) => { browser.on('request', (req) => {
...@@ -76,7 +100,7 @@ createNextDescribe( ...@@ -76,7 +100,7 @@ createNextDescribe(
}) })
it('should not fetch again when a static page was prefetched when navigating to it twice', async () => { it('should not fetch again when a static page was prefetched when navigating to it twice', async () => {
const browser = await next.browser('/404') const browser = await next.browser('/404', browserConfigWithFixedTime)
let requests: string[] = [] let requests: string[] = []
browser.on('request', (req) => { browser.on('request', (req) => {
......
...@@ -522,17 +522,10 @@ createNextDescribe( ...@@ -522,17 +522,10 @@ createNextDescribe(
await browser.waitForElementByCss('#render-id-456') await browser.waitForElementByCss('#render-id-456')
expect(await browser.eval('window.history.length')).toBe(3) expect(await browser.eval('window.history.length')).toBe(3)
// Get the id on the rendered page.
const firstID = await browser.elementById('render-id-456').text()
// Go back, and redo the navigation by clicking the link. // Go back, and redo the navigation by clicking the link.
await browser.back() await browser.back()
await browser.elementById('link').click() await browser.elementById('link').click()
await browser.waitForElementByCss('#render-id-456') await browser.waitForElementByCss('#render-id-456')
// Get the id again, and compare, they should not be the same.
const secondID = await browser.elementById('render-id-456').text()
expect(secondID).not.toBe(firstID)
} finally { } finally {
await browser.close() await browser.close()
} }
...@@ -549,9 +542,6 @@ createNextDescribe( ...@@ -549,9 +542,6 @@ createNextDescribe(
await browser.waitForElementByCss('#render-id-456') await browser.waitForElementByCss('#render-id-456')
expect(await browser.eval('window.history.length')).toBe(2) expect(await browser.eval('window.history.length')).toBe(2)
// Get the date again, and compare, they should not be the same.
const firstId = await browser.elementById('render-id-456').text()
// Navigate to the subpage, verify that the history entry was NOT added. // Navigate to the subpage, verify that the history entry was NOT added.
await browser.elementById('link').click() await browser.elementById('link').click()
await browser.waitForElementByCss('#render-id-123') await browser.waitForElementByCss('#render-id-123')
...@@ -561,10 +551,6 @@ createNextDescribe( ...@@ -561,10 +551,6 @@ createNextDescribe(
await browser.elementById('link').click() await browser.elementById('link').click()
await browser.waitForElementByCss('#render-id-456') await browser.waitForElementByCss('#render-id-456')
expect(await browser.eval('window.history.length')).toBe(2) expect(await browser.eval('window.history.length')).toBe(2)
// Get the date again, and compare, they should not be the same.
const secondId = await browser.elementById('render-id-456').text()
expect(firstId).not.toBe(secondId)
} finally { } finally {
await browser.close() await browser.close()
} }
......
...@@ -122,10 +122,13 @@ export class BrowserInterface implements PromiseLike<any> { ...@@ -122,10 +122,13 @@ export class BrowserInterface implements PromiseLike<any> {
async getAttribute<T = any>(name: string): Promise<T> { async getAttribute<T = any>(name: string): Promise<T> {
return return
} }
async eval<T = any>(snippet: string | Function): Promise<T> { async eval<T = any>(snippet: string | Function, ...args: any[]): Promise<T> {
return return
} }
async evalAsync<T = any>(snippet: string | Function): Promise<T> { async evalAsync<T = any>(
snippet: string | Function,
...args: any[]
): Promise<T> {
return return
} }
async text(): Promise<string> { async text(): Promise<string> {
...@@ -148,4 +151,6 @@ export class BrowserInterface implements PromiseLike<any> { ...@@ -148,4 +151,6 @@ export class BrowserInterface implements PromiseLike<any> {
async url(): Promise<string> { async url(): Promise<string> {
return '' return ''
} }
async waitForIdleNetwork(): Promise<void> {}
} }
...@@ -350,10 +350,10 @@ export class Playwright extends BrowserInterface { ...@@ -350,10 +350,10 @@ export class Playwright extends BrowserInterface {
}) })
} }
eval<T = any>(snippet): Promise<T> { eval<T = any>(fn: any, ...args: any[]): Promise<T> {
return this.chainWithReturnValue(() => return this.chainWithReturnValue(() =>
page page
.evaluate(snippet) .evaluate(fn, ...args)
.catch((err) => { .catch((err) => {
console.error('eval error:', err) console.error('eval error:', err)
return null return null
...@@ -365,15 +365,15 @@ export class Playwright extends BrowserInterface { ...@@ -365,15 +365,15 @@ export class Playwright extends BrowserInterface {
) )
} }
async evalAsync<T = any>(snippet) { async evalAsync<T = any>(fn: any, ...args: any[]) {
if (typeof snippet === 'function') { if (typeof fn === 'function') {
snippet = snippet.toString() fn = fn.toString()
} }
if (snippet.includes(`var callback = arguments[arguments.length - 1]`)) { if (fn.includes(`var callback = arguments[arguments.length - 1]`)) {
snippet = `(function() { fn = `(function() {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
const origFunc = ${snippet} const origFunc = ${fn}
try { try {
origFunc(resolve) origFunc(resolve)
} catch (err) { } catch (err) {
...@@ -383,7 +383,7 @@ export class Playwright extends BrowserInterface { ...@@ -383,7 +383,7 @@ export class Playwright extends BrowserInterface {
})()` })()`
} }
return page.evaluate<T>(snippet).catch(() => null) return page.evaluate<T>(fn).catch(() => null)
} }
async log() { async log() {
...@@ -397,4 +397,10 @@ export class Playwright extends BrowserInterface { ...@@ -397,4 +397,10 @@ export class Playwright extends BrowserInterface {
async url() { async url() {
return this.chain(() => page.evaluate('window.location.href')) as any return this.chain(() => page.evaluate('window.location.href')) as any
} }
async waitForIdleNetwork(): Promise<void> {
return this.chain(() => {
return page.waitForLoadState('networkidle')
})
}
} }
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment