Unverified Commit 4286a42e authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

Encode the cache item name built by `unstable_cache` (#96937)

A cache implementation may serialize cache metadata into HTTP request
headers, whose values are limited to Latin-1. `unstable_cache` assembles
a cache item name from the request URL and the name of the cached
callback, and neither part was encoded. When that name holds a character
above U+00FF the conversion throws before the request is dispatched, so
the read never reaches the cache and the write that follows it fails the
same way. Nothing is stored, nothing is found, and the entry falls back
to the origin on every render.

The name is built by `getFetchUrlPrefix`, which reads the pathname and
the search parameters out of the request URL. The pathname stays
percent-encoded, but `URLSearchParams` returns decoded keys and values,
so a non-ASCII query parameter is the reachable case: it applies to any
dynamic route that calls `unstable_cache`, whether or not the route
reads `searchParams`, and therefore also to a parameter a caller
appends. A callback whose name holds such a character is affected too,
though a production build usually renames the binding.

This change encodes the assembled name with `encodeHeaderSafe`. That
helper only replaces characters outside the class Node accepts in a
header value, so the separating spaces and the URL punctuation are
preserved and the name keeps its documented shape. Every name that is
representable today is returned unchanged, so this is inert for existing
entries. The item name is a label: it is not the cache key, which is
derived separately from the callback's key parts and arguments, and the
Suspense Cache API neither parses nor matches on it.

The test covers the two parts of the name on separate routes so a
failure names the part it comes from, and asserts the constraint rather
than either input, since which inputs are live depends on the bundler
and on minification. A deployment checks it through the real cache
handler implementation, where the failure shows up as an entry that is
recomputed on every request.

fixes #76286
parent 1f0cd93b
......@@ -2,6 +2,7 @@ import type { IncrementalCache } from '../../lib/incremental-cache'
import { CACHE_ONE_YEAR_SECONDS } from '../../../lib/constants'
import { validateRevalidate, validateTags } from '../../lib/patch-fetch'
import { encodeHeaderSafe } from '../../lib/encode-header-safe'
import {
workAsyncStorage,
type WorkStore,
......@@ -138,7 +139,20 @@ export function unstable_cache<T extends Callback>(
const cacheKey =
await incrementalCache.generateSimpleCacheKey(invocationKey)
// $urlWithPath,$sortedQueryStringKeys,$hashOfEveryThingElse
const fetchUrl = `unstable_cache ${fetchUrlPrefix} ${cb.name ? ` ${cb.name}` : cacheKey}`
//
// A cache implementation may serialize this name into an HTTP request
// header, so it is encoded here. Both parts can carry a character above
// U+00FF: the search parameters are decoded, and a JavaScript identifier
// may hold one. The character class leaves the separating spaces and the
// URL punctuation untouched, so the shape above is preserved.
//
// `toWellFormed` replaces lone surrogates, which `cb.name` can hold and
// which `encodeURIComponent` rejects. The name identifies the call for
// debug metrics, so a replacement character is an acceptable trade for
// not failing the render.
const fetchUrl = encodeHeaderSafe(
`unstable_cache ${fetchUrlPrefix} ${cb.name ? ` ${cb.name}` : cacheKey}`.toWellFormed()
)
const fetchIdx =
(workStore ? workStore.nextFetchId : noStoreFetchIdx) ?? 1
......
import { unstable_cache } from 'next/cache'
import { connection } from 'next/server'
// Returns a different value on every call, so a cache that is not working is
// visible in the rendered output.
const RANDOM_ENDPOINT = 'https://next-data-api-endpoint.vercel.app/api/random'
// The callback is anonymous, so the item name is the request URL plus a hash.
// This route covers the part of the name that comes from the URL.
const getConfiguration = unstable_cache(
async () => {
const res = await fetch(`${RANDOM_ENDPOINT}?unstable-cache`)
return (await res.text()).trim()
},
['configuration'],
{ tags: ['ascii-tag'], revalidate: 3600 }
)
// The fetch cache reaches the same cache layer, but derives its item name from
// the request URL, which is already percent-encoded. It is a control: it has to
// keep caching for the very request that breaks the entry above.
async function getFetchedValue() {
const res = await fetch(`${RANDOM_ENDPOINT}?fetch-cache`, {
next: { revalidate: 3600, tags: ['ascii-fetch-tag'] },
})
return (await res.text()).trim()
}
export default async function Page() {
// The item name only carries the request URL for a dynamic render. A
// prerender uses the route pattern instead, which is always ASCII.
//
// The page never reads `searchParams`. The query string reaches the item name
// through the request URL either way, so a dynamic route is affected by a
// query it does not look at, and by one a caller appends to it.
await connection()
return (
<>
<p id="cached">{await getConfiguration()}</p>
<p id="fetched">{await getFetchedValue()}</p>
</>
)
}
import { ReactNode, Suspense } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>
<Suspense fallback={<p>Loading...</p>}>{children}</Suspense>
</body>
</html>
)
}
import { unstable_cache } from 'next/cache'
import { connection } from 'next/server'
const RANDOM_ENDPOINT = 'https://next-data-api-endpoint.vercel.app/api/random'
// `encodeURIComponent` rejects a lone surrogate outright, so an item name
// holding one fails the render rather than the cache read. That makes this a
// stricter case than a name that is merely unrepresentable in a header.
//
// A lone surrogate cannot be written as an identifier, so the name is attached
// through a computed key. The key is a string literal, which a bundler cannot
// rename, so this route carries the constraint in every mode.
const LONE_SURROGATE = '\uD800'
const getConfiguration = unstable_cache(
{
[LONE_SURROGATE]: async () => {
const res = await fetch(`${RANDOM_ENDPOINT}?lone-surrogate`)
return (await res.text()).trim()
},
}[LONE_SURROGATE],
['lone-surrogate'],
{ revalidate: 3600 }
)
export default async function Page() {
await connection()
return <p id="cached">{await getConfiguration()}</p>
}
import { unstable_cache } from 'next/cache'
import { connection } from 'next/server'
const RANDOM_ENDPOINT = 'https://next-data-api-endpoint.vercel.app/api/random'
// A JavaScript identifier may hold characters above U+00FF, and a named
// callback puts its name into the cache item name. This route is requested
// under a pure ASCII URL, so the name is the only part of the item name that
// can be unrepresentable, and a failure here points at nothing else.
//
// A production build renames the binding, so this route only carries the
// constraint in development.
const pobierzKonfigurację = async () => {
const res = await fetch(`${RANDOM_ENDPOINT}?named-callback`)
return (await res.text()).trim()
}
const getConfiguration = unstable_cache(
pobierzKonfigurację,
['named-configuration'],
{ revalidate: 3600 }
)
export default async function Page() {
await connection()
return <p id="cached">{await getConfiguration()}</p>
}
// Reports any value that Next.js hands to the cache layer and that cannot be
// carried in an HTTP header.
//
// A cache implementation may serialize the tags, the soft tags, and the cache
// item name into request headers. Header values are limited to Latin-1, so a
// character above U+00FF cannot be represented and the conversion throws. A
// value that fails it never reaches the cache, and the read is
// indistinguishable from a miss.
//
// Only the value is constrained, not the header it ends up in, so the probe
// uses an arbitrary header name.
const CHECKED_FIELDS = ['tags', 'softTags', 'fetchUrl']
const PROBE_HEADER = 'x-cache-probe'
function reportUnrepresentableValues(operation, ctx) {
for (const field of CHECKED_FIELDS) {
const raw = ctx[field]
const value = Array.isArray(raw) ? raw.join(',') : raw
if (!value) {
continue
}
try {
new Headers({ [PROBE_HEADER]: value })
} catch {
console.error(
`CACHE_PROBE unsafe ${operation} ${field} ${JSON.stringify(value)}`
)
}
}
}
module.exports = class CacheHandler {
async get(key, ctx = {}) {
// Logged on every read so a test can tell "nothing unrepresentable" apart
// from "the handler was never consulted".
console.error(`CACHE_PROBE get`)
reportUnrepresentableValues('get', ctx)
return null
}
async set(key, data, ctx = {}) {
reportUnrepresentableValues('set', ctx)
}
async revalidateTag() {}
resetRequestCache() {}
}
/**
* @type {import('next').NextConfig}
*/
// A deployment checks the constraint through the real cache implementation,
// which serializes cache metadata into request headers itself. Everywhere else
// the fixture supplies a handler that applies the same conversion.
const nextConfig = process.env.VERCEL
? {}
: {
cacheHandler: require.resolve('./cache-handler.js'),
// The in-memory cache would answer repeat lookups without consulting the
// handler, which is where the local assertions read from.
cacheMaxMemorySize: 0,
}
module.exports = nextConfig
import { nextTestSetup } from 'e2e-utils'
import { retry } from 'next-test-utils'
// Cache metadata reaches a cache implementation through HTTP request headers,
// whose values are limited to Latin-1. Every tag, soft tag, and cache item name
// therefore has to be representable there. A value that is not gets lost before
// it reaches the cache, so the entry is never found and never stored, and the
// route calls the origin on every render without reporting an error.
//
// `unstable_cache` assembles its item name from the request URL and the name of
// the cached callback, so the fixture covers those two parts on separate
// routes. `/[slug]` is requested with a non-ASCII segment and a non-ASCII query
// parameter and holds an anonymous callback. `/named-callback` is requested
// under a pure ASCII URL and holds a callback whose name is non-ASCII. A
// failure therefore names the part of the item name it comes from.
describe('non-ASCII cache item name', () => {
const { next, isNextDeploy, isNextDev } = nextTestSetup({
files: __dirname,
})
const urlPath = `/${encodeURIComponent('odzież')}?q=${encodeURIComponent('模型')}`
const namedCallbackPath = '/named-callback'
const loneSurrogatePath = '/lone-surrogate'
async function readValues() {
const $ = await next.render$(urlPath)
const values = {
cached: $('#cached').text(),
fetched: $('#fetched').text(),
}
// Both entries hold a random number. Asserting the shape keeps a render
// that produced neither from reading as two equal results.
for (const value of Object.values(values)) {
expect(value).toMatch(/^0\.\d+$/)
}
return values
}
async function expectEveryValueRepresentable(requestPath: string) {
const outputIndex = next.cliOutput.length
const res = await next.fetch(requestPath)
expect(res.status).toBe(200)
await retry(async () => {
// Without this the assertion would also hold when the handler is never
// consulted at all.
expect(next.cliOutput.slice(outputIndex)).toContain('CACHE_PROBE get')
})
expect(next.cliOutput.slice(outputIndex)).not.toContain(
'CACHE_PROBE unsafe'
)
}
if (isNextDeploy) {
it('keeps caching a request with a non-ASCII query parameter', async () => {
// The real cache implementation performs the conversion, so an item name
// it cannot carry surfaces as a cache that never returns anything: the
// entry is recomputed and the rendered value changes on every request.
// The fetched value has to stay put throughout, which separates a broken
// item name from a cache that is down.
//
// Only the URL route is covered here. A production build renames the
// callback on the other one, which leaves nothing unrepresentable in its
// item name.
await readValues()
await retry(async () => {
const first = await readValues()
const second = await readValues()
expect(second).toEqual(first)
})
})
} else {
it('encodes a non-ASCII query parameter in the item name', async () => {
await expectEveryValueRepresentable(urlPath)
})
it('encodes a lone surrogate in the callback name', async () => {
// `encodeURIComponent` rejects a lone surrogate, which fails the render
// instead of the cache read. The response is still a 200 that carries an
// error, so the assertion has to be on the rendered output.
const $ = await next.render$(loneSurrogatePath)
expect($('#cached').text()).toMatch(/^0\.\d+$/)
})
// A production build renames the callback, which leaves nothing
// unrepresentable in its item name, so the constraint is only observable in
// development.
if (isNextDev) {
it('encodes a non-ASCII callback name in the item name', async () => {
await expectEveryValueRepresentable(namedCallbackPath)
})
}
}
})
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment