Unverified Commit 1ff6a803 authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

Reuse completed cache entries for the rest of a request (#96727)

Calling the same `'use cache: private'` function twice in one request
executed its body twice in production. Preloading at the top of a
segment and reading the same function again lower down for
composability, which is the shape that motivates a preload in the first
place, therefore did the work twice instead of once.

The intra-request dedupe map dropped an entry as soon as its fill
completed, so it only ever covered concurrent invocations. A later
invocation fell through to the cache handler, and the `React.cache` memo
wrapping every cache function missed whenever the arguments were not
reference-equal. Public caches got a handler hit out of that, but
private caches have no handler in production and their entries are
excluded from the immutable Resume Data Cache of a dynamic request, so
nothing had stored the entry.

Completed invocations now move into `completedCacheInvocations` on the
work store instead of being dropped, and a later invocation joins that
entry. The pending map keeps its previous semantics untouched, because a
concurrent joiner shares a fill that is genuinely in flight and must not
re-run the discard checks against it, whereas a completed entry is a
stored value and is only reused when the caller has not asked to bypass
caches and nothing has invalidated it since. Private caches still get no
cache handler, so the map is what backs them, and it lives on the work
store and so cannot carry request-derived data beyond the request that
produced it.

Retention is limited to the kinds where it saves real work: private
caches, and kinds whose handler came from the platform or from
`cacheHandlers` config, where a read can be a network round trip.
`isBuiltInCacheHandler` answers that by tracking the handler instances
`initializeCacheHandlers` constructs itself. Recording instances rather
than kinds is what keeps aliasing right, since a self-hosted `remote`
resolves to the very same in-memory handler as `default` and retaining
its entries would duplicate that cache for nothing, while a
platform-supplied `remote` does not.

The `use-cache-custom-handler` suite covers the mechanism rather than
the observable value, counting `::get` calls to assert that two
sequential reads reach the handler once. Value equality alone cannot
distinguish a retained entry from a handler hit, which is also why the
new `dedup-sequential` fixture passes without this change and serves
only as a regression pin next to `private-dedup-sequential`.
parent 5da1c1ae
...@@ -111,6 +111,21 @@ export interface WorkStore { ...@@ -111,6 +111,21 @@ export interface WorkStore {
*/ */
pendingCacheInvocations?: Map<string, Promise<SharedCacheResult>> pendingCacheInvocations?: Map<string, Promise<SharedCacheResult>>
/**
* Invocations from this request that have already completed, keyed the same
* way as `pendingCacheInvocations`. Entries move here when their fill
* finishes rather than being dropped, so a later invocation of the same cache
* function reuses the entry instead of repeating the cache handler lookup
* and, on a miss, the work.
*
* Only populated for kinds where that saves something real: private caches,
* which have no cache handler in production, and kinds whose handler was
* supplied by the platform or by `cacheHandlers` config, whose reads may be
* remote. A built-in handler read is a map lookup, so retaining its entries
* would cost memory for nothing.
*/
completedCacheInvocations?: Map<string, Promise<SharedCacheResult>>
/** /**
* Set by the dev-server's hang-detection probe worker (see * Set by the dev-server's hang-detection probe worker (see
* `use-cache-probe-worker.ts`) to switch `cache()` into a one-shot fill * `use-cache-probe-worker.ts`) to switch `cache()` into a one-shot fill
......
...@@ -15,6 +15,7 @@ const handlersSymbol = Symbol.for('@next/cache-handlers') ...@@ -15,6 +15,7 @@ const handlersSymbol = Symbol.for('@next/cache-handlers')
const handlersMapSymbol = Symbol.for('@next/cache-handlers-map') const handlersMapSymbol = Symbol.for('@next/cache-handlers-map')
const handlersSetSymbol = Symbol.for('@next/cache-handlers-set') const handlersSetSymbol = Symbol.for('@next/cache-handlers-set')
const privateHandlerSymbol = Symbol.for('@next/cache-handlers-private') const privateHandlerSymbol = Symbol.for('@next/cache-handlers-private')
const builtInHandlersSymbol = Symbol.for('@next/cache-handlers-built-in')
const devFrontHandlersSymbol = Symbol.for('@next/cache-handlers-dev-fronts') const devFrontHandlersSymbol = Symbol.for('@next/cache-handlers-dev-fronts')
const devTieredHandlersSymbol = Symbol.for('@next/cache-handlers-dev-tiered') const devTieredHandlersSymbol = Symbol.for('@next/cache-handlers-dev-tiered')
const memoryCacheDisabledSymbol = Symbol.for( const memoryCacheDisabledSymbol = Symbol.for(
...@@ -41,6 +42,7 @@ const reference: typeof globalThis & { ...@@ -41,6 +42,7 @@ const reference: typeof globalThis & {
} }
[handlersMapSymbol]?: Map<string, CacheHandler> [handlersMapSymbol]?: Map<string, CacheHandler>
[handlersSetSymbol]?: Set<CacheHandler> [handlersSetSymbol]?: Set<CacheHandler>
[builtInHandlersSymbol]?: Set<CacheHandler>
// DEV-only // DEV-only
[privateHandlerSymbol]?: CacheHandler [privateHandlerSymbol]?: CacheHandler
[devFrontHandlersSymbol]?: Map<string, CacheHandler> [devFrontHandlersSymbol]?: Map<string, CacheHandler>
...@@ -64,6 +66,8 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean { ...@@ -64,6 +66,8 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean {
debug?.('initializing cache handlers') debug?.('initializing cache handlers')
const handlersMap = new Map<string, CacheHandler>() const handlersMap = new Map<string, CacheHandler>()
reference[handlersMapSymbol] = handlersMap reference[handlersMapSymbol] = handlersMap
const builtInHandlers = new Set<CacheHandler>()
reference[builtInHandlersSymbol] = builtInHandlers
// In development, `cacheMaxMemorySize: 0` would make the built-in default // In development, `cacheMaxMemorySize: 0` would make the built-in default
// handler a no-op, so every reload would miss. Use a real in-memory size // handler a no-op, so every reload would miss. Use a real in-memory size
...@@ -84,6 +88,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean { ...@@ -84,6 +88,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean {
} else { } else {
debug?.('setting "default" cache handler from default') debug?.('setting "default" cache handler from default')
fallback = createDefaultCacheHandler(builtInSize) fallback = createDefaultCacheHandler(builtInSize)
builtInHandlers.add(fallback)
} }
handlersMap.set('default', fallback) handlersMap.set('default', fallback)
...@@ -97,6 +102,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean { ...@@ -97,6 +102,7 @@ export function initializeCacheHandlers(cacheMaxMemorySize: number): boolean {
} }
} else { } else {
const handler = createDefaultCacheHandler(builtInSize) const handler = createDefaultCacheHandler(builtInSize)
builtInHandlers.add(handler)
debug?.('setting "default" cache handler from default') debug?.('setting "default" cache handler from default')
handlersMap.set('default', handler) handlersMap.set('default', handler)
...@@ -189,6 +195,27 @@ export function isCustomCacheHandler(kind: string): boolean { ...@@ -189,6 +195,27 @@ export function isCustomCacheHandler(kind: string): boolean {
return reference[devFrontHandlersSymbol]?.has(kind) ?? false return reference[devFrontHandlersSymbol]?.has(kind) ?? false
} }
/**
* Whether `kind` resolves to one of the in-memory handlers that
* `initializeCacheHandlers` constructs itself, rather than one supplied by the
* platform through the `@next/cache-handlers` global or by `cacheHandlers`
* config. Reads from a built-in handler are map lookups, so repeating one
* within a request costs nothing; reads from a supplied handler may be a
* network round trip. Unlike `isCustomCacheHandler`, this holds in production.
*
* Tracking the instances rather than the kinds is what keeps aliasing correct:
* self-hosted, `remote` resolves to the same handler as `default` and so
* reports as built-in, while a platform-supplied `remote` does not.
*/
export function isBuiltInCacheHandler(kind: string): boolean {
const handler = reference[handlersMapSymbol]?.get(kind)
return (
handler !== undefined &&
Boolean(reference[builtInHandlersSymbol]?.has(handler))
)
}
/** /**
* Get the dev-only tiered cache handler for a custom `kind`: a fast built-in * Get the dev-only tiered cache handler for a custom `kind`: a fast built-in
* in-memory front handler in front of the user-configured backing handler, so * in-memory front handler in front of the user-configured backing handler, so
......
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
async function getData(params: { id: number }) {
'use cache'
return `${params.id}:${new Date().toISOString()}`
}
export default async function Page() {
await connection()
// A fresh object literal per call, so the React.cache memo misses on
// reference equality, and a delay between them so the second call is outside
// the in-flight dedupe window. Reuse therefore has to come from a stored
// entry.
const first = await getData({ id: 1 })
await setTimeout(100)
const second = await getData({ id: 1 })
return (
<div>
<p id="first">{first}</p>
<p id="second">{second}</p>
</div>
)
}
...@@ -156,4 +156,27 @@ describe('use-cache-custom-handler', () => { ...@@ -156,4 +156,27 @@ describe('use-cache-custom-handler', () => {
) )
}) })
}) })
it('should reach the cache handler only once for sequential calls in a request', async () => {
const $ = await next.render$('/sequential-dedupe')
expect($('#first').text()).toBe($('#second').text())
await retry(async () => {
const cliOutput = next.cliOutput.slice(outputIndex)
// The first call misses and fills. The second is served from the entry
// retained for this request, so it never reaches the handler, which for a
// registered handler can be a remote round trip.
expect(cliOutput).toIncludeRepeated(`ModernCustomCacheHandler::get`, 1)
// Matched up to the escaped opening bracket of the logged cache key,
// since `::set-resolved-entry` would otherwise count as another `::set`.
// The matcher compiles its argument as a regular expression.
expect(cliOutput).toIncludeRepeated(
`ModernCustomCacheHandler::set \\[`,
1
)
})
})
}) })
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
let invocations = 0
async function getData(params: { id: number }) {
'use cache'
return `${params.id}:${++invocations}`
}
export default async function Page() {
await connection()
// The public counterpart of `/private-dedup-sequential`, with the same call
// shape: a fresh object literal per call so the React.cache memo misses, and
// a delay that puts the second call outside the in-flight dedupe window. Here
// the cache handler serves the second call.
const first = await getData({ id: 1 })
await setTimeout(100)
const second = await getData({ id: 1 })
return (
<div>
<p className="first">{first}</p>
<p className="second">{second}</p>
</div>
)
}
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
let invocations = 0
async function getData(params: { id: number }) {
'use cache: private'
return `${params.id}:${++invocations}`
}
export default async function Page() {
await connection()
// Each call passes a fresh object literal, so the React.cache memo that wraps
// every cache function misses on reference equality and the lookup falls
// through to the serialized cache key.
const first = await getData({ id: 1 })
// The intra-request dedupe map drops an entry once its fill completes, so
// this delay puts the second call outside the in-flight window. Reuse then
// depends on the cache store rather than on joining a pending fill.
await setTimeout(100)
const second = await getData({ id: 1 })
return (
<div>
<p className="first">{first}</p>
<p className="second">{second}</p>
</div>
)
}
...@@ -1668,13 +1668,23 @@ describe('use-cache', () => { ...@@ -1668,13 +1668,23 @@ describe('use-cache', () => {
expect(randA).toBe(randB) expect(randA).toBe(randB)
}) })
it('should dedupe private caches within a single request', async () => { it('should dedupe private caches for concurrent calls within a single request', async () => {
const browser = await next.browser('/private-dedup') const browser = await next.browser('/private-dedup')
const first = await browser.elementByCss('.rand:nth-of-type(1)').text() const first = await browser.elementByCss('.rand:nth-of-type(1)').text()
const second = await browser.elementByCss('.rand:nth-of-type(2)').text() const second = await browser.elementByCss('.rand:nth-of-type(2)').text()
expect(first).toBe(second) expect(first).toBe(second)
}) })
it('should dedupe private caches for sequential calls within a single request', async () => {
const $ = await next.render$('/private-dedup-sequential')
expect($('.first').text()).toBe($('.second').text())
})
it('should dedupe caches for sequential calls within a single request', async () => {
const $ = await next.render$('/dedup-sequential')
expect($('.first').text()).toBe($('.second').text())
})
it('should dedupe sequential calls in a server action and its subsequent render', async () => { it('should dedupe sequential calls in a server action and its subsequent render', async () => {
const browser = await next.browser('/action-dedupe') const browser = await next.browser('/action-dedupe')
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment