Unverified Commit 00b2275c authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

docs: Fix the documented contract for custom cache handlers (#98039)

The documented in-memory handler stored the resolved entry and returned
it from every `get`. A stored stream keeps the request that produced the
entry reachable. The second cache hit also fails, because it reads a
stream that another reader already consumed. The render then reports
"Invalid state: ReadableStream is locked".

The "Handling Streams" section described a second, milder mistake. It
told a handler to tee the stored stream in `get`. One branch goes back
into the store, and the other serves the current read. That pattern
serves every hit correctly. The built-in handler used it until #97941,
so the page described the framework's own implementation. The stored
branch retains the same request, and it also holds the source stream and
its buffered chunks. The size a store accounts for covers none of that.
The page now states that cost, because a reader who thinks of teeing
would otherwise read the rule as inapplicable.

`get` told a handler to drop an entry once it is past `revalidate`.
Next.js compares `timestamp` against `expire` on every read, and it
treats a too-old entry as a miss. It applies the same check to
`revalidate` when the result goes into another server cache. A handler
therefore needs no age check at all. The built-in handler still drops at
`revalidate`, as a deliberate policy for an in-memory cache, and the
page now presents that as a policy instead of a rule.

Something still has to reclaim entries, so the page states that eviction
belongs to the handler. Next.js never deletes from it. An age check at
serve time is no substitute, because it only reaches the keys that a
reader still asks for.

The page now follows an order in which no rule appears after the code
that depends on it. The stream rules sit in one section ahead of the API
reference. `CacheEntry Type` moves above the methods that reference it.
`set` precedes `get`, because a handler writes an entry before it reads
one. The tag and error sections move above the examples.

`types.ts` described `refreshTags` as periodic, and the page described
it as a call before each request. Neither is accurate. It runs once per
request, before the first cache read for its kind. A request that reads
nothing from a handler never calls it.

Neither `types.ts` nor the page documented the `revalidate: -1` signal,
which the built-in handler relies on. A negative value always lies in
the past, so it serves an entry with a stale tag one more time while
Next.js generates a fresh one.

`types.ts` now states these rules as well, so a handler author no longer
has to read the built-in implementation to find them. It also covers the
`pendingEntry` promise, which a handler must not retain either.

`streamFromBuffer` now records why it has to stay a default stream. A
byte stream transfers the buffer of every chunk it receives. That would
detach the buffer an in-memory handler serves many reads from. The
comment in `tiered-cache-handler.ts` about cancelling a teed stream now
says that the backing handler is user-configured, because the built-in
one no longer returns such a branch.

The handler test from #97941 now records a limit of its own. It only
fails on Node 20 and 22, whose `AsyncLocalStorage` attaches the active
store to every promise. CI runs Node 20.9, so a regression fails there.
A local run on a newer Node passes.

Nothing in this change affects runtime behavior.
parent d4c838a7
...@@ -12,6 +12,20 @@ setFlagsFromString('--expose-gc') ...@@ -12,6 +12,20 @@ setFlagsFromString('--expose-gc')
const forceGarbageCollection = runInNewContext('gc') as () => void const forceGarbageCollection = runInNewContext('gc') as () => void
describe('default use cache handler', () => { describe('default use cache handler', () => {
/**
* These tests guard the handler against retaining the request that filled or
* read an entry. Whether a regression here fails depends on the
* `AsyncLocalStorage` implementation of the environment:
*
* - Node 20 and 22 attach the active store to every promise, so a retained
* stream keeps the store reachable, and these tests fail on a regression.
* - Node 24 and later use `AsyncContextFrame`, and these tests pass with or
* without the retention.
*
* CI runs Node 20.9, so the guard holds there. A regression is invisible to a
* developer who runs the suite on Node 24 or later.
*/
it('does not retain the async context that populated an entry', async () => { it('does not retain the async context that populated an entry', async () => {
const handler = createDefaultCacheHandler(1024 * 1024) const handler = createDefaultCacheHandler(1024 * 1024)
const requestStoreRef = await runInRequestContext(() => const requestStoreRef = await runInRequestContext(() =>
......
...@@ -5,9 +5,24 @@ export type Timestamp = number ...@@ -5,9 +5,24 @@ export type Timestamp = number
export interface CacheEntry { export interface CacheEntry {
/** /**
* The ReadableStream can error and only have partial data so any cache * The serialized value of the entry. A cache handler consumes this stream in
* handlers need to handle this case and decide to keep the partial cache * `set` and persists what it delivers. A handler can buffer the stream, or it
* around or not. * can pipe it to its storage as chunks arrive. Two rules apply either way:
*
* - A handler must return a new stream from every `get`.
* - A handler must not retain the stream after `set` resolves.
*
* A stream can be read one time only. A second reader of the same stream gets
* no data, or an error while the first reader holds the lock. A retained
* stream also keeps a reference to the async context of the request that
* created it, which keeps the state of that request reachable for as long as
* the entry lives.
*
* The same applies to the `pendingEntry` promise that `set` receives. A
* handler must not put that promise in a map that outlives the request.
*
* The stream can error and deliver partial data. Each handler decides whether
* it keeps the partial entry or discards it.
*/ */
value: ReadableStream<Uint8Array> value: ReadableStream<Uint8Array>
...@@ -30,11 +45,22 @@ export interface CacheEntry { ...@@ -30,11 +45,22 @@ export interface CacheEntry {
/** /**
* How long the entry is allowed to be used (should be longer than revalidate) * How long the entry is allowed to be used (should be longer than revalidate)
* [duration in seconds] * [duration in seconds]
*
* This is the hard limit. Next.js compares it against `timestamp` on every
* read and treats a too-old entry as a miss, so a handler does not need to
* check the age of an entry before it returns one. The dev server raises the
* limit to five minutes when `expire` is shorter, to keep reloads fast.
*/ */
expire: number expire: number
/** /**
* How long until the entry should be revalidated [duration in seconds] * How long until the entry should be revalidated [duration in seconds]
*
* An entry that is past `revalidate` but within `expire` is still served, and
* Next.js generates a fresh one in the background. A negative value always
* lies in the past, so it forces that background refresh on the next read.
* The built-in handler returns `-1` for an entry whose tag is stale, which
* serves the entry one more time and replaces it.
*/ */
revalidate: number revalidate: number
} }
...@@ -42,7 +68,10 @@ export interface CacheEntry { ...@@ -42,7 +68,10 @@ export interface CacheEntry {
export interface CacheHandler { export interface CacheHandler {
/** /**
* Retrieve a cache entry for the given cache key, if available. Will return * Retrieve a cache entry for the given cache key, if available. Will return
* undefined if there's no valid entry, or if the given soft tags are stale. * undefined if there's nothing stored, or if the given soft tags are stale.
*
* Each call returns a new `value` stream over the stored bytes. See
* `CacheEntry.value`.
*/ */
get(cacheKey: string, softTags: string[]): Promise<undefined | CacheEntry> get(cacheKey: string, softTags: string[]): Promise<undefined | CacheEntry>
...@@ -53,12 +82,20 @@ export interface CacheHandler { ...@@ -53,12 +82,20 @@ export interface CacheHandler {
* before the pending entry is complete, the cache handler must wait for the * before the pending entry is complete, the cache handler must wait for the
* `set` operation to finish, before returning the entry, instead of returning * `set` operation to finish, before returning the entry, instead of returning
* undefined. * undefined.
*
* The handler takes ownership of the entry's `value` stream, and consumes it
* exactly once. See `CacheEntry.value`.
*
* The handler also owns eviction. Next.js never removes an entry from the
* store, so the store needs a mechanism of its own, such as a time to live
* from `expire`, or a size-bounded LRU.
*/ */
set(cacheKey: string, pendingEntry: Promise<CacheEntry>): Promise<void> set(cacheKey: string, pendingEntry: Promise<CacheEntry>): Promise<void>
/** /**
* This function may be called periodically, but always before starting a new * This function is called once per request, before the first cache read for
* request. If applicable, it should communicate with the tags service to * this handler's kind. A request that reads nothing from this handler does
* not call it. If applicable, it should communicate with the tags service to
* refresh the local tags manifest accordingly. * refresh the local tags manifest accordingly.
*/ */
refreshTags(): Promise<void> refreshTags(): Promise<void>
......
...@@ -87,6 +87,17 @@ export function streamFromString(str: string): ReadableStream<Uint8Array> { ...@@ -87,6 +87,17 @@ export function streamFromString(str: string): ReadableStream<Uint8Array> {
}) })
} }
/**
* Creates a stream that delivers `chunk` and then closes.
*
* The stream is a default stream, and it has to stay one. A byte stream (`type:
* 'bytes'`) transfers the buffer of each chunk that it receives, which detaches
* `chunk`. A caller that keeps `chunk` to serve more than one read, such as an
* in-memory cache handler, loses the data on the first read.
*
* Every reader receives the same `chunk` instance. A reader that modifies it
* changes what later readers see.
*/
export function streamFromBuffer(chunk: Buffer): ReadableStream<Uint8Array> { export function streamFromBuffer(chunk: Buffer): ReadableStream<Uint8Array> {
return new ReadableStream({ return new ReadableStream({
start(controller) { start(controller) {
......
...@@ -158,10 +158,12 @@ async function reconcileFrontFromBacking( ...@@ -158,10 +158,12 @@ async function reconcileFrontFromBacking(
if (backingEntry.timestamp > frontEntry.timestamp) { if (backingEntry.timestamp > frontEntry.timestamp) {
await front.set(cacheKey, Promise.resolve(backingEntry)) await front.set(cacheKey, Promise.resolve(backingEntry))
} else { } else {
// The front is already up to date, so the backing entry goes unused. // The front is already up to date, so the backing entry goes unused. This
// Release its stream without awaiting: a teed stream's `cancel()` only // code releases its stream and does not await the result. The backing
// settles once the sibling branch (retained by the backing handler) is // handler is user-configured, and it can return one branch of a teed
// also cancelled, so awaiting it here would hang the reconcile. // stream. `cancel()` on such a branch settles only after the sibling
// branch is cancelled too, and the handler retains that sibling, so an
// await here would hang the reconcile.
void backingEntry.value.cancel() void backingEntry.value.cancel()
} }
} catch { } catch {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment