Unverified Commit 323a17c1 authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

[backport] Discard only cache entries that predate a tag revalidation, and reuse…

[backport] Discard only cache entries that predate a tag revalidation, and reuse completed entries (#97314)

Backports #96726 ("Discard only cache entries that predate a tag
revalidation") and #96727 ("Reuse completed cache entries for the rest
of a request") to `next-16-3`.

Calling `updateTag()` in a server action made every later read of a
cache carrying that tag regenerate for the remainder of the request,
including reads of an entry that had just been generated after the
invalidation and therefore already reflected it. Two sequential reads of
the same cache function during the re-render produced two different
values within a single render, and each one repeated the work.
`isRecentlyRevalidatedTag` only asked whether a tag appeared in
`pendingRevalidatedTags`, and that array lives for the whole
`WorkStore`, which spans a server action and the render that follows it.
Each pending revalidated tag now records a `revalidatedAt` timestamp
taken from the same clock as `CacheEntry.timestamp`, and the renamed
`isRevalidatedAfter` reports an entry as stale only when the
revalidation is newer than the entry.

The second change fixes a `'use cache: private'` function executing its
body twice when it is called twice in one request, which defeats the
preload-then-read shape that motivates a preload in the first place. The
intra-request dedupe map dropped an entry as soon as its fill completed,
so it only ever covered concurrent invocations, and private caches have
no cache handler in production to fall back to. Completed invocations
now move into `completedCacheInvocations` on the work store instead of
being dropped.

The two are backported together because #96727 builds on the discard
checks that #96726 restructures. They were stacked in this order on
canary, and #96727 conflicts in
`test/e2e/app-dir/use-cache/use-cache.test.ts` when applied alone.

### Verification on this branch

Cherry-picked from `5da1c1ae` and `1ff6a803` in that order, with no
conflicts. All source files are byte-identical to canary. Behavioral
verification is left to CI on this branch; the changes carry the
`action-dedupe`, `dedup-sequential` and `private-dedup-sequential`
fixtures in the `use-cache` suite, and a handler-call count assertion in
`use-cache-custom-handler`.

<!-- NEXT_JS_LLM -->
parent 6d67f46a
...@@ -6339,6 +6339,7 @@ function buildDevValidationWorkStore( ...@@ -6339,6 +6339,7 @@ function buildDevValidationWorkStore(
cacheLifeProfiles: message.nextConfigSerializable.cacheLifeProfiles, cacheLifeProfiles: message.nextConfigSerializable.cacheLifeProfiles,
buildId: message.buildId, buildId: message.buildId,
deploymentId: message.deploymentId, deploymentId: message.deploymentId,
requestStartTime: message.request.requestStartTime,
previouslyRevalidatedTags: [], previouslyRevalidatedTags: [],
refreshTagsByCacheKind: new Map(), refreshTagsByCacheKind: new Map(),
runInCleanSnapshot: createSnapshot(), runInCleanSnapshot: createSnapshot(),
...@@ -7914,6 +7915,7 @@ async function validateInstantConfigInBuildWithSample( ...@@ -7914,6 +7915,7 @@ async function validateInstantConfigInBuildWithSample(
isStaticGeneration: false, isStaticGeneration: false,
page: outerWorkStore.page, page: outerWorkStore.page,
route: outerWorkStore.route, route: outerWorkStore.route,
requestStartTime: outerWorkStore.requestStartTime,
incrementalCache: outerWorkStore.incrementalCache, incrementalCache: outerWorkStore.incrementalCache,
cacheLifeProfiles: outerWorkStore.cacheLifeProfiles, cacheLifeProfiles: outerWorkStore.cacheLifeProfiles,
useCacheTimeout: outerWorkStore.useCacheTimeout, useCacheTimeout: outerWorkStore.useCacheTimeout,
......
...@@ -55,6 +55,7 @@ export interface DevValidationRequestSnapshot { ...@@ -55,6 +55,7 @@ export interface DevValidationRequestSnapshot {
isDraftMode: boolean isDraftMode: boolean
isHmrRefresh: boolean isHmrRefresh: boolean
hmrRefreshHash: string | undefined hmrRefreshHash: string | undefined
requestStartTime: number
} }
/** /**
......
...@@ -75,6 +75,7 @@ export async function buildDevValidationSnapshot( ...@@ -75,6 +75,7 @@ export async function buildDevValidationSnapshot(
isDraftMode: requestStore.draftMode.isEnabled, isDraftMode: requestStore.draftMode.isEnabled,
isHmrRefresh: requestStore.isHmrRefresh ?? false, isHmrRefresh: requestStore.isHmrRefresh ?? false,
hmrRefreshHash: requestStore.hmrRefreshHash, hmrRefreshHash: requestStore.hmrRefreshHash,
requestStartTime: ctx.workStore.requestStartTime,
}, },
interpolatedParams: ctx.interpolatedParams, interpolatedParams: ctx.interpolatedParams,
requestFallbackRouteParams: ctx.fallbackRouteParams, requestFallbackRouteParams: ctx.fallbackRouteParams,
......
...@@ -67,6 +67,14 @@ export interface WorkStore { ...@@ -67,6 +67,14 @@ export interface WorkStore {
pendingRevalidatedTags?: Array<{ pendingRevalidatedTags?: Array<{
tag: string tag: string
profile?: string | { stale?: number; revalidate?: number; expire?: number } profile?: string | { stale?: number; revalidate?: number; expire?: number }
/**
* When the tag was revalidated, on the same clock as `CacheEntry.timestamp`
* (`performance.timeOrigin + performance.now()`). A cache entry created
* before this is stale; one created after it already reflects the
* revalidation and can still be served. Re-revalidating a tag moves this
* forward.
*/
revalidatedAt: number
}> }>
/** /**
...@@ -76,6 +84,15 @@ export interface WorkStore { ...@@ -76,6 +84,15 @@ export interface WorkStore {
*/ */
readonly previouslyRevalidatedTags: readonly string[] readonly previouslyRevalidatedTags: readonly string[]
/**
* When this request started, on the same clock as `CacheEntry.timestamp`.
* `previouslyRevalidatedTags` carry no timestamp of their own, having been
* revalidated by an earlier request, so they are treated as revalidated at
* this instant: entries predating the request are discarded, while entries
* generated during it are not.
*/
readonly requestStartTime: number
/** /**
* This map contains lazy results so that we can evaluate them when the first * This map contains lazy results so that we can evaluate them when the first
* cache entry is read. It allows us to skip refreshing tags if no caches are * cache entry is read. It allows us to skip refreshing tags if no caches are
...@@ -96,6 +113,21 @@ export interface WorkStore { ...@@ -96,6 +113,21 @@ export interface WorkStore {
*/ */
pendingCacheInvocations?: Map<string, Promise<SharedCacheResult>> pendingCacheInvocations?: Map<string, Promise<SharedCacheResult>>
/**
* Invocations from this request that have already completed, keyed the same
* way as `pendingCacheInvocations`. Entries move here when their fill
* finishes rather than being dropped, so a later invocation of the same cache
* function reuses the entry instead of repeating the cache handler lookup
* and, on a miss, the work.
*
* Only populated for kinds where that saves something real: private caches,
* which have no cache handler in production, and kinds whose handler was
* supplied by the platform or by `cacheHandlers` config, whose reads may be
* remote. A built-in handler read is a map lookup, so retaining its entries
* would cost memory for nothing.
*/
completedCacheInvocations?: Map<string, Promise<SharedCacheResult>>
/** /**
* Set by the dev-server's hang-detection probe worker (see * Set by the dev-server's hang-detection probe worker (see
* `use-cache-probe-worker.ts`) to switch `cache()` into a one-shot fill * `use-cache-probe-worker.ts`) to switch `cache()` into a one-shot fill
......
...@@ -157,6 +157,7 @@ export function createWorkStore({ ...@@ -157,6 +157,7 @@ export function createWorkStore({
cacheComponentsEnabled: renderOpts.cacheComponents, cacheComponentsEnabled: renderOpts.cacheComponents,
validationLevel: renderOpts.validationLevel, validationLevel: renderOpts.validationLevel,
previouslyRevalidatedTags, previouslyRevalidatedTags,
requestStartTime: performance.timeOrigin + performance.now(),
refreshTagsByCacheKind: createRefreshTagsByCacheKind(), refreshTagsByCacheKind: createRefreshTagsByCacheKind(),
runInCleanSnapshot: createSnapshot(), runInCleanSnapshot: createSnapshot(),
shouldTrackFetchMetrics, shouldTrackFetchMetrics,
......
...@@ -208,6 +208,7 @@ function buildProbeWorkStore(msg: ProbeMessage): WorkStore { ...@@ -208,6 +208,7 @@ function buildProbeWorkStore(msg: ProbeMessage): WorkStore {
cacheLifeProfiles: msg.nextConfigSerializable.cacheLifeProfiles, cacheLifeProfiles: msg.nextConfigSerializable.cacheLifeProfiles,
buildId: msg.buildId, buildId: msg.buildId,
deploymentId: msg.deploymentId, deploymentId: msg.deploymentId,
requestStartTime: msg.request.requestStartTime,
// Empty values for cache-handler / RDC bookkeeping. The `useCacheProbeMode` // Empty values for cache-handler / RDC bookkeeping. The `useCacheProbeMode`
// branch in `cache()` returns before any code that reads or writes these // branch in `cache()` returns before any code that reads or writes these
// fields, so the values can never be observed. // fields, so the values can never be observed.
......
...@@ -15,6 +15,7 @@ const handlersSymbol = Symbol.for('@next/cache-handlers') ...@@ -15,6 +15,7 @@ const handlersSymbol = Symbol.for('@next/cache-handlers')
const handlersMapSymbol = Symbol.for('@next/cache-handlers-map') const handlersMapSymbol = Symbol.for('@next/cache-handlers-map')
const handlersSetSymbol = Symbol.for('@next/cache-handlers-set') const handlersSetSymbol = Symbol.for('@next/cache-handlers-set')
const privateHandlerSymbol = Symbol.for('@next/cache-handlers-private') const privateHandlerSymbol = Symbol.for('@next/cache-handlers-private')
const builtInHandlersSymbol = Symbol.for('@next/cache-handlers-built-in')
const devFrontHandlersSymbol = Symbol.for('@next/cache-handlers-dev-fronts') const devFrontHandlersSymbol = Symbol.for('@next/cache-handlers-dev-fronts')
const devTieredHandlersSymbol = Symbol.for('@next/cache-handlers-dev-tiered') const devTieredHandlersSymbol = Symbol.for('@next/cache-handlers-dev-tiered')
const memoryCacheDisabledSymbol = Symbol.for( const memoryCacheDisabledSymbol = Symbol.for(
...@@ -41,6 +42,7 @@ const reference: typeof globalThis & { ...@@ -41,6 +42,7 @@ const reference: typeof globalThis & {
} }
[handlersMapSymbol]?: Map<string, CacheHandler> [handlersMapSymbol]?: Map<string, CacheHandler>
[handlersSetSymbol]?: Set<CacheHandler> [handlersSetSymbol]?: Set<CacheHandler>
[builtInHandlersSymbol]?: Set<CacheHandler>
// DEV-only // DEV-only
[privateHandlerSymbol]?: CacheHandler [privateHandlerSymbol]?: CacheHandler
[devFrontHandlersSymbol]?: Map<string, CacheHandler> [devFrontHandlersSymbol]?: Map<string, CacheHandler>
...@@ -64,6 +66,8 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean { ...@@ -64,6 +66,8 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean {
debug?.('initializing cache handlers') debug?.('initializing cache handlers')
const handlersMap = new Map<string, CacheHandler>() const handlersMap = new Map<string, CacheHandler>()
reference[handlersMapSymbol] = handlersMap reference[handlersMapSymbol] = handlersMap
const builtInHandlers = new Set<CacheHandler>()
reference[builtInHandlersSymbol] = builtInHandlers
// In development, `cacheMaxMemorySize: 0` would make the built-in default // In development, `cacheMaxMemorySize: 0` would make the built-in default
// handler a no-op, so every reload would miss. Use a real in-memory size // handler a no-op, so every reload would miss. Use a real in-memory size
...@@ -84,6 +88,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean { ...@@ -84,6 +88,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean {
} else { } else {
debug?.('setting "default" cache handler from default') debug?.('setting "default" cache handler from default')
fallback = createDefaultCacheHandler(builtInSize) fallback = createDefaultCacheHandler(builtInSize)
builtInHandlers.add(fallback)
} }
handlersMap.set('default', fallback) handlersMap.set('default', fallback)
...@@ -97,6 +102,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean { ...@@ -97,6 +102,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean {
} }
} else { } else {
const handler = createDefaultCacheHandler(builtInSize) const handler = createDefaultCacheHandler(builtInSize)
builtInHandlers.add(handler)
debug?.('setting "default" cache handler from default') debug?.('setting "default" cache handler from default')
handlersMap.set('default', handler) handlersMap.set('default', handler)
...@@ -189,6 +195,27 @@ export function isCustomCacheHandler(kind: string): boolean { ...@@ -189,6 +195,27 @@ export function isCustomCacheHandler(kind: string): boolean {
return reference[devFrontHandlersSymbol]?.has(kind) ?? false return reference[devFrontHandlersSymbol]?.has(kind) ?? false
} }
/**
* Whether `kind` resolves to one of the in-memory handlers that
* `initializeCacheHandlers` constructs itself, rather than one supplied by the
* platform through the `@next/cache-handlers` global or by `cacheHandlers`
* config. Reads from a built-in handler are map lookups, so repeating one
* within a request costs nothing; reads from a supplied handler may be a
* network round trip. Unlike `isCustomCacheHandler`, this holds in production.
*
* Tracking the instances rather than the kinds is what keeps aliasing correct:
* self-hosted, `remote` resolves to the same handler as `default` and so
* reports as built-in, while a platform-supplied `remote` does not.
*/
export function isBuiltInCacheHandler(kind: string): boolean {
const handler = reference[handlersMapSymbol]?.get(kind)
return (
handler !== undefined &&
Boolean(reference[builtInHandlersSymbol]?.has(handler))
)
}
/** /**
* Get the dev-only tiered cache handler for a custom `kind`: a fast built-in * Get the dev-only tiered cache handler for a custom `kind`: a fast built-in
* in-memory front handler in front of the user-configured backing handler, so * in-memory front handler in front of the user-configured backing handler, so
......
...@@ -25,6 +25,7 @@ export type UseCacheProbeRequestSnapshot = { ...@@ -25,6 +25,7 @@ export type UseCacheProbeRequestSnapshot = {
isDraftMode: boolean isDraftMode: boolean
isHmrRefresh: boolean isHmrRefresh: boolean
hmrRefreshHash: string | undefined hmrRefreshHash: string | undefined
requestStartTime: number
} }
/** /**
......
...@@ -116,6 +116,7 @@ export function setupProbeScheduler( ...@@ -116,6 +116,7 @@ export function setupProbeScheduler(
isDraftMode: workStore.isDraftMode ?? false, isDraftMode: workStore.isDraftMode ?? false,
isHmrRefresh: outerRequestStore.isHmrRefresh ?? false, isHmrRefresh: outerRequestStore.isHmrRefresh ?? false,
hmrRefreshHash: outerRequestStore.hmrRefreshHash, hmrRefreshHash: outerRequestStore.hmrRefreshHash,
requestStartTime: workStore.requestStartTime,
}, },
timeoutMs: probeInternalTimeoutMs, timeoutMs: probeInternalTimeoutMs,
}).then( }).then(
......
...@@ -211,6 +211,8 @@ function revalidate( ...@@ -211,6 +211,8 @@ function revalidate(
store.pendingRevalidatedTags = [] store.pendingRevalidatedTags = []
} }
const revalidatedAt = performance.timeOrigin + performance.now()
for (const tag of tags) { for (const tag of tags) {
const existingIndex = store.pendingRevalidatedTags.findIndex((item) => { const existingIndex = store.pendingRevalidatedTags.findIndex((item) => {
if (item.tag !== tag) return false if (item.tag !== tag) return false
...@@ -227,7 +229,13 @@ function revalidate( ...@@ -227,7 +229,13 @@ function revalidate(
store.pendingRevalidatedTags.push({ store.pendingRevalidatedTags.push({
tag, tag,
profile, profile,
revalidatedAt,
}) })
} else {
// Revalidating a tag again invalidates everything produced up to now, so
// the latest revalidation is the one that decides which entries are
// stale.
store.pendingRevalidatedTags[existingIndex].revalidatedAt = revalidatedAt
} }
} }
......
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
async function getData(params: { id: number }) {
'use cache'
return `${params.id}:${new Date().toISOString()}`
}
export default async function Page() {
await connection()
// A fresh object literal per call, so the React.cache memo misses on
// reference equality, and a delay between them so the second call is outside
// the in-flight dedupe window. Reuse therefore has to come from a stored
// entry.
const first = await getData({ id: 1 })
await setTimeout(100)
const second = await getData({ id: 1 })
return (
<div>
<p id="first">{first}</p>
<p id="second">{second}</p>
</div>
)
}
...@@ -156,4 +156,27 @@ describe('use-cache-custom-handler', () => { ...@@ -156,4 +156,27 @@ describe('use-cache-custom-handler', () => {
) )
}) })
}) })
it('should reach the cache handler only once for sequential calls in a request', async () => {
const $ = await next.render$('/sequential-dedupe')
expect($('#first').text()).toBe($('#second').text())
await retry(async () => {
const cliOutput = next.cliOutput.slice(outputIndex)
// The first call misses and fills. The second is served from the entry
// retained for this request, so it never reaches the handler, which for a
// registered handler can be a remote round trip.
expect(cliOutput).toIncludeRepeated(`ModernCustomCacheHandler::get`, 1)
// Matched up to the escaped opening bracket of the logged cache key,
// since `::set-resolved-entry` would otherwise count as another `::set`.
// The matcher compiles its argument as a regular expression.
expect(cliOutput).toIncludeRepeated(
`ModernCustomCacheHandler::set \\[`,
1
)
})
})
}) })
import { cacheTag, updateTag } from 'next/cache'
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
let invocations = 0
async function getData(params: { id: number }) {
'use cache'
cacheTag('action-dedupe')
return `${params.id}:${++invocations}`
}
// Each call passes a fresh object literal so the React.cache memo misses on
// reference equality, and the delay puts the second call outside the in-flight
// dedupe window. Both calls therefore depend on a stored entry being reused.
async function readTwice() {
const first = await getData({ id: 1 })
await setTimeout(100)
const second = await getData({ id: 1 })
return { first, second }
}
export default async function Page() {
await connection()
const { first, second } = await readTwice()
return (
<div>
<p className="first">{first}</p>
<p className="second">{second}</p>
<form>
<button
id="revalidate"
formAction={async () => {
'use server'
const { first: a1, second: a2 } = await readTwice()
console.log(`action-dedupe: revalidate action read ${a1} ${a2}`)
updateTag('action-dedupe')
}}
>
Read twice and revalidate
</button>{' '}
<button
id="no-revalidate"
formAction={async () => {
'use server'
const { first: a1, second: a2 } = await readTwice()
console.log(`action-dedupe: plain action read ${a1} ${a2}`)
}}
>
Read twice only
</button>
</form>
</div>
)
}
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
let invocations = 0
async function getData(params: { id: number }) {
'use cache'
return `${params.id}:${++invocations}`
}
export default async function Page() {
await connection()
// The public counterpart of `/private-dedup-sequential`, with the same call
// shape: a fresh object literal per call so the React.cache memo misses, and
// a delay that puts the second call outside the in-flight dedupe window. Here
// the cache handler serves the second call.
const first = await getData({ id: 1 })
await setTimeout(100)
const second = await getData({ id: 1 })
return (
<div>
<p className="first">{first}</p>
<p className="second">{second}</p>
</div>
)
}
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
let invocations = 0
async function getData(params: { id: number }) {
'use cache: private'
return `${params.id}:${++invocations}`
}
export default async function Page() {
await connection()
// Each call passes a fresh object literal, so the React.cache memo that wraps
// every cache function misses on reference equality and the lookup falls
// through to the serialized cache key.
const first = await getData({ id: 1 })
// The intra-request dedupe map drops an entry once its fill completes, so
// this delay puts the second call outside the in-flight window. Reuse then
// depends on the cache store rather than on joining a pending fill.
await setTimeout(100)
const second = await getData({ id: 1 })
return (
<div>
<p className="first">{first}</p>
<p className="second">{second}</p>
</div>
)
}
...@@ -1647,13 +1647,59 @@ describe('use-cache', () => { ...@@ -1647,13 +1647,59 @@ describe('use-cache', () => {
expect(randA).toBe(randB) expect(randA).toBe(randB)
}) })
it('should dedupe private caches within a single request', async () => { it('should dedupe private caches for concurrent calls within a single request', async () => {
const browser = await next.browser('/private-dedup') const browser = await next.browser('/private-dedup')
const first = await browser.elementByCss('.rand:nth-of-type(1)').text() const first = await browser.elementByCss('.rand:nth-of-type(1)').text()
const second = await browser.elementByCss('.rand:nth-of-type(2)').text() const second = await browser.elementByCss('.rand:nth-of-type(2)').text()
expect(first).toBe(second) expect(first).toBe(second)
}) })
it('should dedupe private caches for sequential calls within a single request', async () => {
const $ = await next.render$('/private-dedup-sequential')
expect($('.first').text()).toBe($('.second').text())
})
it('should dedupe caches for sequential calls within a single request', async () => {
const $ = await next.render$('/dedup-sequential')
expect($('.first').text()).toBe($('.second').text())
})
it('should dedupe sequential calls in a server action and its subsequent render', async () => {
const browser = await next.browser('/action-dedupe')
const initial = await browser.elementByCss('.first').text()
expect(await browser.elementByCss('.second').text()).toBe(initial)
// With no revalidation, the action's two reads and the re-render's two
// reads all serve the entry that the initial render produced.
await browser.elementById('no-revalidate').click()
await retry(async () => {
expect(next.cliOutput).toContain(
`action-dedupe: plain action read ${initial} ${initial}`
)
})
expect(await browser.elementByCss('.first').text()).toBe(initial)
expect(await browser.elementByCss('.second').text()).toBe(initial)
// The action reads the still-valid entry twice, then revalidates the tag.
// The re-render must not reuse that entry, but its own two reads must share
// the entry the first of them regenerated.
await browser.elementById('revalidate').click()
await retry(async () => {
expect(await browser.elementByCss('.first').text()).not.toBe(initial)
})
const revalidated = await browser.elementByCss('.first').text()
expect(await browser.elementByCss('.second').text()).toBe(revalidated)
expect(next.cliOutput).toContain(
`action-dedupe: revalidate action read ${initial} ${initial}`
)
})
it('dedupes private caches across concurrent requests in dev but not in production', async () => { it('dedupes private caches across concurrent requests in dev but not in production', async () => {
const [first$, second$] = await Promise.all([ const [first$, second$] = await Promise.all([
next.render$('/private-dedup-cross-request'), next.render$('/private-dedup-cross-request'),
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment