Unverified Commit 3a4477cb authored by Batrić Perović's avatar Batrić Perović Committed by GitHub

Stamp FileSystemCache revalidations and entries with the monotonic clock (#100357)

### What?
`FileSystemCache` dates tag revalidations and cache entries with the
wall clock, but since #94457 `areTagsExpired` compares them against the
monotonic clock (`performance.timeOrigin + performance.now()`).
When the wall clock runs ahead, a revalidation is dated in the future,
so it's ignored and the old page keeps being served.

### Why?
I ran into this with `next start`, when our E2E tests kept getting stale
pages after `revalidateTag`. In a fresh 16.4.2 app with the wall clock
5ms ahead, 30 out of 30 reads right after a revalidation returned the
old page. With this change, it's 0 of 30.

The two clocks drift apart in ordinary setups, for example when NTP, a
VM or WSL2 corrects the wall clock while the server runs.

### How?
Use the monotonic clock everywhere that `FileSystemCache` dates
something, as `incremental-cache/index.ts` already does:

- `revalidateTag` and `set()` stamp with `performance.timeOrigin +
performance.now()` instead of `Date.now()`.
- `set()` also stores that timestamp on disk, as
`routeCacheLastModified` in the `.meta` file and as `lastModified` in
the fetch cache entry. Reads used to take the file's mtime(wall-clock
time).
Files without a stored time, from `next build` or an older Next.js
version, still fall back to the mtime. They were written before the
server started, so their mtime is earlier than any revalidation it
records.
This matters when entries are read from disk: with `cacheMaxMemorySize:
0`, or after an eviction from the memory cache. With only the in-memory
stamps fixed, a page written just before a revalidation comes back with
an mtime that's ahead, so it isn't even marked stale. Running `next
start` with the memory cache off and the wall clock stepped 1s ahead, 20
of 40 revalidations left the old page in place that way. With both
changes, it's 0 of 40.

Notes:
- `routeCacheLastModified` was only written when a build seed was
promoted. Runtime writes set it now, so the "preserves seed age" test
checks for it instead of its absence.
- Fetch cache files are kept across deploys, and they gain a
`lastModified` field.
- `test/production/app-dir/file-system-cache-clock` offsets
`performance.timeOrigin` at startup and checks the first request after
`revalidateTag` for an ISR page, an ISR route handler and a dynamic
page, with the default memory cache and `cacheMaxMemorySize: 0`. It also
restarts on a fetch cache file without `lastModified`. It fails on
canary, and with only the in-memory fix the `cacheMaxMemorySize: 0`
cases fail.
- Ran the new test with `test-start-webpack` and `test-start-turbo`, and
four existing revalidation and ISR suites with `test-start-webpack`. Not
run in dev.
parent 94e2c32d
...@@ -22,6 +22,10 @@ export type RouteMetadata = { ...@@ -22,6 +22,10 @@ export type RouteMetadata = {
cacheControl?: CacheControl cacheControl?: CacheControl
/** Identifies the response-cache entry represented by this build artifact. */ /** Identifies the response-cache entry represented by this build artifact. */
routeCache?: RouteCacheMetadata routeCache?: RouteCacheMetadata
/** Original payload mtime retained when a build seed is promoted. */ /**
* When the incremental cache wrote the entry, on the clock that tag
* revalidations are stamped with. A promoted build seed keeps the mtime of
* its payload instead.
*/
routeCacheLastModified?: number routeCacheLastModified?: number
} }
...@@ -27,6 +27,12 @@ import { areTagsExpired, tagsManifest } from './tags-manifest.external' ...@@ -27,6 +27,12 @@ import { areTagsExpired, tagsManifest } from './tags-manifest.external'
import { MultiFileWriter } from '../../../lib/multi-file-writer' import { MultiFileWriter } from '../../../lib/multi-file-writer'
import { getMemoryCache } from './memory-cache.external' import { getMemoryCache } from './memory-cache.external'
/**
* A fetch cache entry as written to disk. `lastModified` is absent from entries
* written before it was stored, which fall back to the file's mtime.
*/
type StoredFetchValue = CachedFetchValue & { lastModified?: number }
type FileSystemCacheContext = Omit< type FileSystemCacheContext = Omit<
CacheHandlerContext, CacheHandlerContext,
'fs' | 'serverDistDir' 'fs' | 'serverDistDir'
...@@ -109,7 +115,7 @@ export default class FileSystemCache implements CacheHandler { ...@@ -109,7 +115,7 @@ export default class FileSystemCache implements CacheHandler {
return return
} }
const now = Date.now() const now = Math.floor(performance.timeOrigin + performance.now())
for (const tag of tags) { for (const tag of tags) {
const existingEntry = tagsManifest.get(tag) || {} const existingEntry = tagsManifest.get(tag) || {}
...@@ -248,10 +254,10 @@ export default class FileSystemCache implements CacheHandler { ...@@ -248,10 +254,10 @@ export default class FileSystemCache implements CacheHandler {
if (!this.flushToDisk) return null if (!this.flushToDisk) return null
const lastModified = mtime.getTime() const { lastModified, ...parsedData }: StoredFetchValue =
const parsedData: CachedFetchValue = JSON.parse(fileData) JSON.parse(fileData)
data = { data = {
lastModified, lastModified: lastModified ?? mtime.getTime(),
value: parsedData, value: parsedData,
} }
...@@ -618,9 +624,14 @@ export default class FileSystemCache implements CacheHandler { ...@@ -618,9 +624,14 @@ export default class FileSystemCache implements CacheHandler {
const seedState = this.getSeedReadState(key) const seedState = this.getSeedReadState(key)
if (seedState) seedState.version++ if (seedState) seedState.version++
// areTagsExpired compares with the monotonic clock, so the entry is stamped
// with it in memory and on disk. Without a stored stamp, a read from disk
// would fall back to the file's mtime, which is wall-clock time.
const lastModified = Math.floor(performance.timeOrigin + performance.now())
FileSystemCache.memoryCache?.set(key, { FileSystemCache.memoryCache?.set(key, {
value: data, value: data,
lastModified: Date.now(), lastModified,
cacheControl, cacheControl,
}) })
...@@ -651,6 +662,7 @@ export default class FileSystemCache implements CacheHandler { ...@@ -651,6 +662,7 @@ export default class FileSystemCache implements CacheHandler {
segmentPaths: undefined, segmentPaths: undefined,
prefetchHints: undefined, prefetchHints: undefined,
cacheControl, cacheControl,
routeCacheLastModified: lastModified,
} }
writer.append( writer.append(
...@@ -706,6 +718,7 @@ export default class FileSystemCache implements CacheHandler { ...@@ -706,6 +718,7 @@ export default class FileSystemCache implements CacheHandler {
segmentPaths, segmentPaths,
prefetchHints: undefined, prefetchHints: undefined,
cacheControl, cacheControl,
routeCacheLastModified: lastModified,
} }
writer.append( writer.append(
...@@ -720,6 +733,7 @@ export default class FileSystemCache implements CacheHandler { ...@@ -720,6 +733,7 @@ export default class FileSystemCache implements CacheHandler {
segmentPaths: undefined, segmentPaths: undefined,
prefetchHints: undefined, prefetchHints: undefined,
cacheControl, cacheControl,
routeCacheLastModified: lastModified,
} }
writer.append( writer.append(
htmlPath.replace(/\.html$/, NEXT_META_SUFFIX), htmlPath.replace(/\.html$/, NEXT_META_SUFFIX),
...@@ -733,7 +747,8 @@ export default class FileSystemCache implements CacheHandler { ...@@ -733,7 +747,8 @@ export default class FileSystemCache implements CacheHandler {
JSON.stringify({ JSON.stringify({
...data, ...data,
tags: ctx.fetchCache ? ctx.tags : [], tags: ctx.fetchCache ? ctx.tags : [],
}) lastModified,
} satisfies StoredFetchValue)
) )
} }
......
import { Suspense } from 'react'
import { connection } from 'next/server'
import { getData } from '../../lib/data'
async function Data() {
// Rendered per request, so every request reads the fetch cache.
await connection()
return <p id="data">{await getData('dynamic')}</p>
}
export default function Page() {
return (
<Suspense>
<Data />
</Suspense>
)
}
import type { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
import { revalidateTag } from 'next/cache'
export async function POST() {
revalidateTag('data', { expire: 0 })
return Response.json({ revalidated: true })
}
import { getData } from '../../../lib/data'
// Only this slug is prerendered. The tests request a different one, so its
// entries are written at runtime.
export function generateStaticParams() {
return [{ slug: 'build' }]
}
export async function GET(
_request: Request,
{ params }: { params: Promise<{ slug: string }> }
) {
const { slug } = await params
return new Response(await getData(`route/${slug}`))
}
import { getData } from '../../../lib/data'
// Only this slug is prerendered. The tests request a different one, so its
// entries are written at runtime.
export function generateStaticParams() {
return [{ slug: 'build' }]
}
export default async function Page({
params,
}: {
params: Promise<{ slug: string }>
}) {
// Awaited outside any Suspense boundary, so each slug is rendered on demand
// into its own response cache entry.
const { slug } = await params
return <p id="data">{await getData(`static/${slug}`)}</p>
}
import { nextTestSetup } from 'e2e-utils'
import { gate } from 'next-test-utils'
import { load } from 'cheerio'
import { createServer } from 'node:http'
import type { AddressInfo } from 'node:net'
import { promises as fs } from 'node:fs'
import { join } from 'node:path'
import { getRouteCacheKey } from 'next/dist/server/lib/route-cache-key'
import { RouteKind } from 'next/dist/server/route-kind'
// Tag revalidations and cache entries must be dated with the same clock as
// the expiry check, including entries read back from disk. Instrumentation
// puts the monotonic clock behind the wall clock, so an entry dated with
// Date.now() or a file mtime would look newer than a later revalidation.
describe('file-system-cache-clock', () => {
// Returns a new value on every request, so a cached value is recognizable.
let version = 0
const service = createServer((req, res) => {
res.end(`${req.url} ${++version}`)
})
beforeAll(async () => {
await new Promise<void>((resolve) =>
service.listen(0, '127.0.0.1', resolve)
)
})
afterAll(async () => {
await new Promise<void>((resolve, reject) =>
service.close((err) => (err ? reject(err) : resolve()))
)
})
describe.each([
['the default memory cache', undefined],
['cacheMaxMemorySize: 0', '0'],
])('with %s', (_name, cacheMaxMemorySize) => {
const { next } = nextTestSetup({
files: __dirname,
skipStart: true,
env: {
TIME_ORIGIN_OFFSET_MS: '60000',
...(cacheMaxMemorySize !== undefined && {
CACHE_MAX_MEMORY_SIZE: cacheMaxMemorySize,
}),
},
})
beforeAll(async () => {
const { port } = service.address() as AddressInfo
next.env.DATA_SERVICE_URL = `http://127.0.0.1:${port}`
await next.start()
})
async function request(pathname: string) {
const res = await next.fetch(pathname)
expect(res.status).toBe(200)
const body = await res.text()
return {
cache: res.headers.get('x-nextjs-cache'),
data: pathname.startsWith('/route/')
? body
: load(body)('#data').text(),
}
}
async function getData(pathname: string) {
return (await request(pathname)).data
}
// The response cache entry of the ISR page, as written by the default
// file-system cache handler.
async function readPageEntry(pathname: string) {
const key = getRouteCacheKey(pathname, {
kind: RouteKind.APP_PAGE,
sourceRoute: '/static/[slug]/page',
})
return fs.readFile(
join(next.testDir, '.next/server', `${key}.html`),
'utf8'
)
}
async function revalidate() {
const res = await next.fetch('/revalidate', { method: 'POST' })
expect(res.status).toBe(200)
await res.text()
}
// Only the "build" slug is prerendered, so this entry is written at
// runtime.
it('serves fresh data for an ISR page right after revalidateTag', async () => {
const pathname = '/static/runtime'
const rendered = await request(pathname)
expect(rendered.data).toStartWith(`${pathname} `)
// The page has its own response cache entry, so the assertions below do
// not pass through fetch caching alone.
expect(await readPageEntry(pathname)).toContain(rendered.data)
const cached = await request(pathname)
expect(cached.data).toBe(rendered.data)
// The file-system cache doesn't write the RSC payload of a PPR page, so
// it can't read that entry back. With the memory cache off, the page is
// rendered again from the cached fetch.
const isPPR = await gate((c) => c.cacheComponents)
expect(cached.cache).toBe(
isPPR && cacheMaxMemorySize === '0' ? 'MISS' : 'HIT'
)
await revalidate()
const fresh = await request(pathname)
expect(fresh.cache).toBe('MISS')
expect(fresh.data).toStartWith(`${pathname} `)
expect(fresh.data).not.toBe(rendered.data)
expect(await readPageEntry(pathname)).toContain(fresh.data)
})
it('serves fresh data for an ISR route handler right after revalidateTag', async () => {
const pathname = '/route/runtime'
const rendered = await request(pathname)
expect(rendered.data).toStartWith(`${pathname} `)
expect(await request(pathname)).toEqual({
cache: 'HIT',
data: rendered.data,
})
await revalidate()
const fresh = await request(pathname)
expect(fresh.cache).toBe('MISS')
expect(fresh.data).toStartWith(`${pathname} `)
expect(fresh.data).not.toBe(rendered.data)
})
it('serves fresh data for a dynamic page right after revalidateTag', async () => {
const cached = await getData('/dynamic')
expect(cached).toStartWith('/dynamic ')
expect(await getData('/dynamic')).toBe(cached)
await revalidate()
const fresh = await getData('/dynamic')
expect(fresh).toStartWith('/dynamic ')
expect(fresh).not.toBe(cached)
})
it('reads a fetch cache entry written without a stored timestamp', async () => {
const cached = await getData('/dynamic')
expect(cached).toStartWith('/dynamic ')
await next.stop()
// Find the entry by its cached value, then rewrite it in the format
// written before timestamps were stored, with an earlier mtime.
const fetchCacheDir = join(next.testDir, '.next/cache/fetch-cache')
const entries = await Promise.all(
(await fs.readdir(fetchCacheDir)).map(async (file) => {
const filePath = join(fetchCacheDir, file)
return { filePath, content: await fs.readFile(filePath, 'utf8') }
})
)
const matches = entries.filter(({ content }) => {
const data = JSON.parse(content).data
return data && Buffer.from(data.body, 'base64').toString() === cached
})
expect(matches).toHaveLength(1)
const [{ filePath, content }] = matches
const { lastModified, ...entry } = JSON.parse(content)
expect(typeof lastModified).toBe('number')
await fs.writeFile(filePath, JSON.stringify(entry))
const modified = new Date(Date.now() - 60_000)
await fs.utimes(filePath, modified, modified)
await next.start({ skipBuild: true })
expect(await getData('/dynamic')).toBe(cached)
await revalidate()
expect(await getData('/dynamic')).not.toBe(cached)
// The fresh value was written in the new format.
expect(
JSON.parse(await fs.readFile(filePath, 'utf8')).lastModified
).toEqual(expect.any(Number))
})
})
})
export function register() {
// Puts the monotonic clock behind the wall clock, as after the wall clock is
// stepped forward while the server runs. Date.now() and file mtimes keep the
// real time.
const offset = Number(process.env.TIME_ORIGIN_OFFSET_MS)
if (offset) {
Object.defineProperty(performance, 'timeOrigin', {
value: performance.timeOrigin - offset,
})
}
}
// The path is part of the fetch URL, so every route and slug has its own
// fetch cache entry. The revalidate interval also sets the cache lifetime of
// the ISR routes that call this.
export async function getData(path: string) {
const res = await fetch(`${process.env.DATA_SERVICE_URL}/${path}`, {
next: { tags: ['data'], revalidate: 3600 },
})
return res.text()
}
/** @type {import('next').NextConfig} */
module.exports = {
cacheMaxMemorySize: process.env.CACHE_MAX_MEMORY_SIZE
? Number(process.env.CACHE_MAX_MEMORY_SIZE)
: undefined,
}
...@@ -240,6 +240,9 @@ describe('FileSystemCache route-scoped build seeds', () => { ...@@ -240,6 +240,9 @@ describe('FileSystemCache route-scoped build seeds', () => {
expect(restarted?.value).toMatchObject({ html: 'build html' }) expect(restarted?.value).toMatchObject({ html: 'build html' })
expect(restarted?.lastModified).toBe(modified.getTime()) expect(restarted?.lastModified).toBe(modified.getTime())
const beforeRuntimeWrite = Math.floor(
performance.timeOrigin + performance.now()
)
await createCache().set( await createCache().set(
key, key,
{ {
...@@ -253,13 +256,14 @@ describe('FileSystemCache route-scoped build seeds', () => { ...@@ -253,13 +256,14 @@ describe('FileSystemCache route-scoped build seeds', () => {
) )
const afterRuntimeWrite = await createCache().get(key, getContext) const afterRuntimeWrite = await createCache().get(key, getContext)
expect(afterRuntimeWrite?.value).toMatchObject({ html: 'runtime html' }) expect(afterRuntimeWrite?.value).toMatchObject({ html: 'runtime html' })
expect(afterRuntimeWrite?.lastModified).toBeGreaterThan(modified.getTime()) // The runtime write replaces the seed's age with its own, on the clock
expect(afterRuntimeWrite?.lastModified).toBe( // that tag revalidations are stamped with.
(await fs.stat(`${scopedBase}.html`)).mtime.getTime() expect(afterRuntimeWrite?.lastModified).toBeGreaterThanOrEqual(
beforeRuntimeWrite
) )
expect( expect(
JSON.parse(await fs.readFile(`${scopedBase}.meta`, 'utf8')) JSON.parse(await fs.readFile(`${scopedBase}.meta`, 'utf8'))
).not.toHaveProperty('routeCacheLastModified') ).toHaveProperty('routeCacheLastModified', afterRuntimeWrite?.lastModified)
}) })
it.each([ it.each([
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment