Unverified Commit 1066fbf7 authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

Improve deduping of concurrent `'use cache'` invocations (#91830)

With #71286, we implemented deduping of cache entries under certain
circumstances. For example, the following constructed example was fixed
with the PR:

```js
async function getCachedRandom() {
  'use cache'
  return Math.random()
}

const rand1 = await getCachedRandom()
const rand2 = await getCachedRandom()

assert(rand1 === rand2)
```

However, this implementation relied on awaiting the two calls
sequentially. When rendering components, this can usually not be
guaranteed. E.g. the following example was not properly deduped:

```jsx
async function Cached() {
  'use cache'
  return <p>{Math.random()}</p>
}

export default function Page() {
  return (
    <>
      <Cached />
      <Cached />
    </>
  )
}
```

This did render the same value, but only because we triggered two render
passes, and the last cached value was used for both elements in the
final render pass. But during the first render pass, the `Cached`
function was called twice, and the cache entry was also set twice.

With #75786, we also fixed the render scenario by wrapping the cached
function in `React.cache`. This however did not work for route handlers.
E.g. the first example rewritten as follows, and used in a route
handler, still wouldn't be deduped:

```js
const [rand1, rand2] = await Promise.all([
  getCachedRandom(),
  getCachedRandom(),
])
```

Furthermore, with this solution, nested cached functions could not be
deduped across different outer cache scopes. This is because each cache
scope creates its own `React.cache` scope. Example:

```jsx
async function Inner() {
  'use cache'
  return <p>{Math.random()}</p>
}

async function Outer1() {
  'use cache'
  return <Inner />
}

async function Outer2() {
  'use cache'
  return <Inner />
}

export default function Page() {
  return (
    <>
      <Outer1 />
      <Outer2 />
    </>
  )
}
```

This PR introduces two-layer invocation tracking that deduplicates these
cases without changing how cache handlers are implemented. Only the
first invocation (the "leader") performs the cache handler lookup and
generation. Subsequent invocations ("joiners") tee the leader's result
stream instead.

The deduplication scope starts after the Resume Data Cache (RDC) lookup
and before the cache handler `get`. The RDC phase is excluded because it
throws synchronous errors (dynamic usage errors) that need individual
stack traces per call site, and RDC lookups are local with no network
savings from deduplication.

Intra-request deduplication is stored on the `WorkStore` and keyed by
`serializedCacheKey` (the coarse key, which is safe because root params
are identical within a single request). Cross-request deduplication is
stored in a module-scope map keyed by `cacheHandlerKey`, which may
include root params on the warm path. Cross-request joiners must await
metadata for root param verification before forking the stream. If the
key mismatches (different root params), the joiner retries with a
recomputed key. Because metadata is checked before `fork()` is called, a
mismatched joiner never consumes a stream from the wrong entry.

Stream tee-ing is lazy via the `SharedCacheEntry` class: the leader
calls `fork()` to get its copy, and each joiner calls `fork()` on
demand. If no joiners exist, only one tee occurs. The
`SharedCacheResult` discriminated union wraps either a
`SharedCacheEntry` (for the cached case) or a hanging promise (for
`prerender-dynamic`).

Each invocation still decodes the stream independently via
`createFromReadableStream` with its own `temporaryReferences`, because
sharing the decoded result would cause cache poisoning when components
receive different non-serializable props (e.g. `children`).

Admittedly, this adds significant complexity to the `cache()` function.
Two classes help keep it manageable:
- `SharedCacheEntry` encapsulates stream ownership and lazy tee-ing so
that call sites don't need to reason about which streams have been
consumed or need cloning.
- `ResolvableSharedCacheResult` manages the deferred promise, map
registration, and lazy cleanup. Entries stay in the dedup maps until
collection completes (so late-arriving invocations can join while the
leader streams), then clean up automatically on resolve or reject.

A follow-up refactoring to extract the cache handler lookup and
generation into a separate function would help break up the function's
size.

closes #78703
parent a7d15733
...@@ -5,6 +5,7 @@ import type { DeepReadonly } from '../../shared/lib/deep-readonly' ...@@ -5,6 +5,7 @@ import type { DeepReadonly } from '../../shared/lib/deep-readonly'
import type { AppSegmentConfig } from '../../build/segment-config/app/app-segment-config' import type { AppSegmentConfig } from '../../build/segment-config/app/app-segment-config'
import type { AfterContext } from '../after/after-context' import type { AfterContext } from '../after/after-context'
import type { CacheLife } from '../use-cache/cache-life' import type { CacheLife } from '../use-cache/cache-life'
import type { SharedCacheResult } from '../use-cache/use-cache-wrapper'
// Share the instance module in the next-shared layer // Share the instance module in the next-shared layer
import { workAsyncStorageInstance } from './work-async-storage-instance' with { 'turbopack-transition': 'next-shared' } import { workAsyncStorageInstance } from './work-async-storage-instance' with { 'turbopack-transition': 'next-shared' }
...@@ -82,6 +83,16 @@ export interface WorkStore { ...@@ -82,6 +83,16 @@ export interface WorkStore {
fetchMetrics?: FetchMetrics fetchMetrics?: FetchMetrics
shouldTrackFetchMetrics: boolean shouldTrackFetchMetrics: boolean
/**
* Tracks pending `"use cache"` invocations within the current request scope,
* keyed by the serialized cache key (coarse key). Used for intra-request
* deduplication: when multiple components call the same cache function within
* a single request, only the first one runs (cache handler lookup +
* generation), and joiners tee its result stream.
* Root params are identical within a request, so the coarse key is sufficient.
*/
pendingCacheInvocations?: Map<string, Promise<SharedCacheResult>>
isDraftMode?: boolean isDraftMode?: boolean
isUnstableNoStore?: boolean isUnstableNoStore?: boolean
isPrefetchRequest?: boolean isPrefetchRequest?: boolean
......
...@@ -11,7 +11,7 @@ import type { CollectedCacheResult } from '../use-cache/use-cache-wrapper' ...@@ -11,7 +11,7 @@ import type { CollectedCacheResult } from '../use-cache/use-cache-wrapper'
*/ */
type CacheStore<T> = Pick< type CacheStore<T> = Pick<
Map<string, T>, Map<string, T>,
'entries' | 'keys' | 'size' | 'get' | 'set' | typeof Symbol.iterator 'entries' | 'keys' | 'size' | 'get' | 'set' | 'has' | typeof Symbol.iterator
> >
/** /**
......
import { ReactNode, Suspense } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>
<Suspense>{children}</Suspense>
</body>
</html>
)
}
export function generateStaticParams() {
return [
{ countryCode: 'ca', lang: 'en' },
{ countryCode: 'ca', lang: 'fr' },
]
}
import { countryCode, lang } from 'next/root-params'
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
async function getCachedData() {
'use cache: remote'
// Simulate I/O latency so concurrent requests overlap.
await setTimeout(1000)
return {
countryCode: await countryCode(),
lang: await lang(),
random: Math.random(),
}
}
export default async function Page() {
await connection()
const result = await getCachedData()
return (
<p>
<span id="country">{result.countryCode}</span>{' '}
<span id="lang">{result.lang}</span>{' '}
<span id="random">{result.random}</span>
</p>
)
}
import type { NextConfig } from 'next'
const nextConfig: NextConfig = {
experimental: {
useCache: true,
rootParams: true,
},
}
export default nextConfig
...@@ -285,3 +285,35 @@ describe('app-root-param-getters - cache - at build', () => { ...@@ -285,3 +285,35 @@ describe('app-root-param-getters - cache - at build', () => {
}) })
} }
}) })
describe('app-root-param-getters - cache dedup with root params', () => {
const { next, skipped } = nextTestSetup({
files: join(__dirname, 'fixtures', 'use-cache-dedup'),
// In deploy mode, concurrent requests could hit different lambdas.
skipDeployment: true,
})
if (skipped) return
it('should dedupe same root params and isolate different root params', async () => {
// Three concurrent requests: ca/en, ca/fr, ca/fr.
const [$en, $fr1, $fr2] = await Promise.all([
next.render$('/ca/en'),
next.render$('/ca/fr'),
next.render$('/ca/fr'),
])
const randomEn = $en('#random').text()
const randomFr1 = $fr1('#random').text()
const randomFr2 = $fr2('#random').text()
expect(randomEn).toBeTruthy()
expect(randomFr1).toBeTruthy()
// ca/en and ca/fr should have different results (isolation).
expect(randomEn).not.toBe(randomFr1)
// Both ca/fr requests should have the same result (deduped).
expect(randomFr1).toBe(randomFr2)
})
})
import { cacheLife, cacheTag } from 'next/cache'
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
async function Inner() {
'use cache'
await setTimeout(1000)
// This should lower the default cache lives of the outer cache scopes.
cacheLife({ revalidate: 180, expire: 300 })
cacheTag('inner')
return <p id="inner">{Math.random()}</p>
}
async function Outer1() {
'use cache'
cacheTag('outer1')
return <Inner />
}
async function Outer2() {
'use cache'
cacheTag('outer2')
return <Inner />
}
export default async function Page() {
await connection()
return (
<>
<Outer1 />
<Outer2 />
</>
)
}
...@@ -2,6 +2,8 @@ ...@@ -2,6 +2,8 @@
const defaultCacheHandler = const defaultCacheHandler =
require('next/dist/server/lib/cache-handlers/default.external').default require('next/dist/server/lib/cache-handlers/default.external').default
const { AsyncLocalStorage } = require('node:async_hooks')
const snapshot = AsyncLocalStorage.snapshot()
/** /**
* @type {import('next/dist/server/lib/cache-handlers/types').CacheHandler} * @type {import('next/dist/server/lib/cache-handlers/types').CacheHandler}
...@@ -14,6 +16,15 @@ const cacheHandler = { ...@@ -14,6 +16,15 @@ const cacheHandler = {
async set(cacheKey, pendingEntry) { async set(cacheKey, pendingEntry) {
console.log('ModernCustomCacheHandler::set', cacheKey) console.log('ModernCustomCacheHandler::set', cacheKey)
pendingEntry.then(({ revalidate, expire, tags }) => {
snapshot(() => {
console.log(
`ModernCustomCacheHandler::set-resolved-entry revalidate: ${revalidate}, expire: ${expire}, tags: ${tags}\n ${cacheKey}`
)
})
})
return defaultCacheHandler.set(cacheKey, pendingEntry) return defaultCacheHandler.set(cacheKey, pendingEntry)
}, },
......
...@@ -130,4 +130,27 @@ describe('use-cache-custom-handler', () => { ...@@ -130,4 +130,27 @@ describe('use-cache-custom-handler', () => {
) )
}) })
} }
it('should dedupe nested caches across different outer cache scopes, and still propagate cache life/tags correctly', async () => {
await next.fetch('/nested')
await retry(async () => {
const cliOutput = next.cliOutput.slice(outputIndex)
expect(cliOutput).toIncludeRepeated(
`ModernCustomCacheHandler::set-resolved-entry revalidate: 180, expire: 300, tags: inner`,
1
)
expect(cliOutput).toIncludeRepeated(
`ModernCustomCacheHandler::set-resolved-entry revalidate: 180, expire: 300, tags: outer1,inner`,
1
)
expect(cliOutput).toIncludeRepeated(
`ModernCustomCacheHandler::set-resolved-entry revalidate: 180, expire: 300, tags: outer2,inner`,
1
)
})
})
}) })
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
async function getCachedRandom() {
'use cache'
// Simulate I/O latency so concurrent requests overlap.
await setTimeout(1000)
return Math.random()
}
export async function GET() {
await connection()
const rand = await getCachedRandom()
return Response.json({ rand })
}
...@@ -15,6 +15,24 @@ describe('use-cache-route-handler-only', () => { ...@@ -15,6 +15,24 @@ describe('use-cache-route-handler-only', () => {
expect(date1).toBe(date2) expect(date1).toBe(date2)
}) })
if (!isNextDeploy) {
// In deploy mode, concurrent requests could hit different lambdas.
it('should dedupe concurrent cache invocations in route handlers', async () => {
// Restart to ensure a cold server. Without this, prior test activity
// can affect request timing enough for the concurrent requests to no
// longer overlap reliably.
await next.stop()
await next.start()
const [{ rand: first }, { rand: second }] = await Promise.all([
next.fetch('/node-dynamic').then((response) => response.json()),
next.fetch('/node-dynamic').then((response) => response.json()),
])
expect(first).toBe(second)
})
}
it('should be able to revalidate prerendered route handlers', async () => { it('should be able to revalidate prerendered route handlers', async () => {
const response1 = await next.fetch('/node') const response1 = await next.fetch('/node')
const { date1: date1a } = await response1.json() const { date1: date1a } = await response1.json()
......
import { cacheLife } from 'next/cache' import { cacheLife } from 'next/cache'
import { connection } from 'next/server' import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
async function getInnerData(id: string) { async function getInnerData(id: string) {
'use cache' 'use cache'
await setTimeout(1000)
return new Date().toISOString() return new Date().toISOString()
} }
...@@ -12,6 +15,8 @@ async function getOuterData(id: string) { ...@@ -12,6 +15,8 @@ async function getOuterData(id: string) {
cacheLife({ revalidate: 5 }) cacheLife({ revalidate: 5 })
console.log('use-cache-swr: generating outer data')
const innerData = await getInnerData('inner') const innerData = await getInnerData('inner')
return { outer: new Date().toISOString(), inner: innerData } return { outer: new Date().toISOString(), inner: innerData }
......
// @ts-check // @ts-check
const { setTimeout } = require('timers/promises')
/** /**
* A persistent cache handler that does NOT drop entries at revalidate time. * A persistent cache handler that does NOT drop entries at revalidate time.
* This allows the framework's SWR code path to trigger, unlike the default * This allows the framework's SWR code path to trigger, unlike the default
...@@ -22,6 +24,8 @@ const cacheHandler = { ...@@ -22,6 +24,8 @@ const cacheHandler = {
await pendingPromise await pendingPromise
} }
// Simulate some latency when fetching from a remote cache.
await setTimeout(200)
const entry = store.get(cacheKey) const entry = store.get(cacheKey)
if (!entry) { if (!entry) {
console.log('PersistentCacheHandler::get', cacheKey, softTags, '-> miss') console.log('PersistentCacheHandler::get', cacheKey, softTags, '-> miss')
......
...@@ -78,4 +78,37 @@ describe('use-cache-swr', () => { ...@@ -78,4 +78,37 @@ describe('use-cache-swr', () => {
// cache by its "inner" sentinel argument in the key. // cache by its "inner" sentinel argument in the key.
expect(cliOutput).toMatch(/PersistentCacheHandler::get.*"inner".*_N_T_\//) expect(cliOutput).toMatch(/PersistentCacheHandler::get.*"inner".*_N_T_\//)
}) })
it('should dedupe SWR regens across concurrent requests', async () => {
const browser = await next.browser('/')
await browser.elementById('outer-data').text()
// Wait for the outer cache to go stale (revalidate: 5).
await new Promise((resolve) => setTimeout(resolve, 6000))
// Reset output index to capture only the SWR-related logs.
outputIndex = next.cliOutput.length
// Fire multiple concurrent requests that all find the stale entry.
// Only one of them should trigger a background regen.
await Promise.all([next.fetch('/'), next.fetch('/'), next.fetch('/')])
// Wait for the background regen to complete.
await retry(() => {
const regenOutput = next.cliOutput.slice(outputIndex)
expect(regenOutput).toInclude('use-cache-swr: generating outer data')
})
const cliOutput = next.cliOutput.slice(outputIndex)
// The cache function should have been executed only once across all
// concurrent requests, not once per request.
const generationCalls = cliOutput.split('\n').filter(
(line) =>
line.includes('use-cache-swr: generating outer data') &&
// Ignore replayed logs that have a Cache badge.
!line.includes(' Cache ')
)
expect(generationCalls).toHaveLength(1)
})
}) })
import { connection } from 'next/server'
async function CachedWrapper({ children }: { children: React.ReactNode }) {
'use cache: remote'
return (
<div className="wrapper">
<p className="rand">{Math.random()}</p>
<p className="children">{children}</p>
</div>
)
}
export default async function Page() {
await connection()
return (
<div>
<CachedWrapper>
<span>Child A</span>
</CachedWrapper>
<CachedWrapper>
<span>Child B</span>
</CachedWrapper>
</div>
)
}
import { connection } from 'next/server'
import { Suspense } from 'react'
import { setTimeout } from 'timers/promises'
async function Inner() {
await setTimeout(1000)
return <p className="inner">{Math.random()}</p>
}
// We're only using the id to force using a different cache key for different
// test scenarios.
async function InnerCached({ id }: { id: string; name: string }) {
'use cache: remote'
return (
<Suspense fallback={<p className="loading">Loading...</p>}>
<Inner />
</Suspense>
)
}
async function OuterCached1({ id }: { id: string; name: string }) {
'use cache: remote'
return <InnerCached id={id} name="inner" />
}
async function OuterCached2({ id }: { id: string; name: string }) {
'use cache: remote'
return <InnerCached id={id} name="inner" />
}
export default async function Page({
params,
}: {
params: Promise<{ id: string }>
}) {
await connection()
const { id } = await params
return (
<>
<OuterCached1 id={id} name="outer1" />
<OuterCached2 id={id} name="outer2" />
</>
)
}
import { connection } from 'next/server'
import { setTimeout } from 'timers/promises'
async function PrivateCached() {
'use cache: private'
await setTimeout(1000)
return <p className="rand">{Math.random()}</p>
}
export default async function Page() {
await connection()
return (
<div>
<PrivateCached />
<PrivateCached />
</div>
)
}
import { connection } from 'next/server'
import { Suspense } from 'react'
import { setTimeout } from 'timers/promises'
async function DelayedContent() {
await setTimeout(1000)
return <p className="content">{Math.random()}</p>
}
async function Cached() {
// We use a custom no-store cache handler here to ensure we're actually
// testing the deduping behavior in the cache wrapper, and not a pending set
// locking mechanism that the cache handler might implement (as the default
// handler does).
'use cache: no-store'
return (
<Suspense fallback={<p className="loading">Loading...</p>}>
<DelayedContent />
</Suspense>
)
}
export default async function Page() {
await connection()
return <Cached />
}
...@@ -6,6 +6,7 @@ const nextConfig = { ...@@ -6,6 +6,7 @@ const nextConfig = {
custom: require.resolve( custom: require.resolve(
'next/dist/server/lib/cache-handlers/default.external' 'next/dist/server/lib/cache-handlers/default.external'
), ),
'no-store': require.resolve('./no-store-handler.js'),
}, },
experimental: { experimental: {
useCache: true, useCache: true,
......
// @ts-check
/**
* @type {import('next/dist/server/lib/cache-handlers/types').CacheHandler}
*/
const noStoreCacheHandler = {
async get(_cacheKey, _softTags) {
return undefined
},
async set(_cacheKey, _pendingEntry) {},
async refreshTags() {},
async getExpiration(_tags) {
return Infinity
},
async updateTags(_tags) {},
}
module.exports = noStoreCacheHandler
...@@ -30,6 +30,19 @@ describe('use-cache', () => { ...@@ -30,6 +30,19 @@ describe('use-cache', () => {
return return
} }
let cliOutputLength: number
beforeEach(() => {
cliOutputLength = next.cliOutput.length
})
afterEach(async () => {
// eslint-disable-next-line jest/no-standalone-expect
expect(next.cliOutput.slice(cliOutputLength)).not.toContain(
'unhandledRejection'
)
})
it('should cache results', async () => { it('should cache results', async () => {
const browser = await next.browser(`/?n=1`) const browser = await next.browser(`/?n=1`)
expect(await browser.waitForElementByCss('#x').text()).toBe('1') expect(await browser.waitForElementByCss('#x').text()).toBe('1')
...@@ -1489,6 +1502,103 @@ describe('use-cache', () => { ...@@ -1489,6 +1502,103 @@ describe('use-cache', () => {
await assertNoConsoleErrors(browser) await assertNoConsoleErrors(browser)
}) })
it('should dedupe shared inner caches across different outer caches', async () => {
const browser = await next.browser('/nested/1')
const first = await browser.elementByCss('.inner:nth-of-type(1)').text()
const second = await browser.elementByCss('.inner:nth-of-type(2)').text()
expect(first).toBe(second)
})
if (!isNextDeploy) {
// In deploy mode, concurrent requests could hit different instances.
it('should dedupe a streaming cache across concurrent requests', async () => {
const [first, second] = await Promise.all([
next.render('/streaming'),
// Delay the second request to ensure deduping also works when the
// first request has already started streaming.
new Promise<string>((resolve) =>
setTimeout(() => resolve(next.render('/streaming')), 500)
),
])
// Both requests should contain the cached content.
expect(first).toContain('<p class="content">')
expect(second).toContain('<p class="content">')
// Both requests should get the same cached value.
const getContent = (html: string) =>
html.match(/<p class="content">([^<]+)<\/p>/)?.[1]
expect(getContent(first)).toBe(getContent(second))
// The leader streams with a loading boundary visible in the initial HTML,
// while the cross-request joiner resolves from the fully collected result
// with no loading boundary. We don't know which request is the leader,
// but exactly one should have it.
expect([first, second]).toSatisfy(function onlyOneRequestStreams([
a,
b,
]: string[]) {
return (
(a.includes('<p class="loading">') &&
!b.includes('<p class="loading">')) ||
(!a.includes('<p class="loading">') &&
b.includes('<p class="loading">'))
)
})
})
}
it('should resolve different children correctly when deduping', async () => {
const browser = await next.browser('/cached-with-children')
const childA = await browser
.elementByCss('.wrapper:first-child .children')
.text()
const childB = await browser
.elementByCss('.wrapper:last-child .children')
.text()
expect(childA).toBe('Child A')
expect(childB).toBe('Child B')
// The random value from the cache function should be the same for both
// wrappers, confirming the invocation was actually deduped.
const randA = await browser
.elementByCss('.wrapper:first-child .rand')
.text()
const randB = await browser.elementByCss('.wrapper:last-child .rand').text()
expect(randA).toBe(randB)
})
it('should dedupe private caches within a single request', async () => {
const browser = await next.browser('/private-dedup')
const first = await browser.elementByCss('.rand:nth-of-type(1)').text()
const second = await browser.elementByCss('.rand:nth-of-type(2)').text()
expect(first).toBe(second)
})
it('should not dedupe private caches across concurrent requests', async () => {
const [first$, second$] = await Promise.all([
next.render$('/private-dedup'),
next.render$('/private-dedup'),
])
const firstValue = first$('.rand').first().text()
const secondValue = second$('.rand').first().text()
// Across requests, private caches must NOT be deduped.
expect(firstValue).not.toBe(secondValue)
})
it('should stream the result of a deduped invocation', async () => {
const html = await next
.fetch('/nested/2')
.then((response) => response.text())
// The loading boundaries of both inner cache functions are expected to be
// shown while the page is loading.
expect(html).toIncludeRepeated('<p class="loading">Loading...</p>', 2)
})
}) })
async function getSanitizedLogs(browser: Playwright): Promise<string[]> { async function getSanitizedLogs(browser: Playwright): Promise<string[]> {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment