Unverified Commit 579ff601 authored by Eddy (Frontend Engineer)'s avatar Eddy (Frontend Engineer) Committed by GitHub

example(cache-handler-redis): fix connection leak, hangs on a down Redis, and…

example(cache-handler-redis): fix connection leak, hangs on a down Redis, and cross-page tag invalidation (#98716)

Three runtime problems in `examples/cache-handler-redis`.

### 1. One new Redis connection per request, never closed

Next.js constructs the singular `cacheHandler` class once per request
(`new CurCacheHandler(...)` in `IncrementalCache`, which
`route-module.ts` creates per request). The example's constructor calls
`createClient()` + `connect()`, so every request opens a connection that
is never closed.

```
$ redis-cli client list | wc -l      # before
102
$ for i in $(seq 1 10); do curl -s -o /dev/null localhost:3000/cet; done
$ redis-cli client list | wc -l
132                                  # +3 per request
```

With Redis' default `maxclients 10000`, a single instance runs out after
a few thousand requests.

### 2. Requests hang while Redis is unavailable

The README says the handlers "degrade gracefully when Redis is
unavailable, so the app still builds and runs, just without a shared
cache". At runtime they don't: with Redis stopped, every request on
every instance blocks until Redis comes back.

```
$ docker stop cache-handler-redis
$ curl -s -o /dev/null -m 60 -w "%{http_code} %{time_total}s\n" localhost:3000/cet
000 60.007305s                       # (uncapped: 188s, returned the moment Redis was started again)
```

Same when the app is started while Redis is down. Cause: node-redis
keeps retrying in the background and `client.connect()` does not settle
until a connection succeeds. Isolated:

```js
const c = createClient({ url: "redis://localhost:6379" }); c.on("error", () => {});
await Promise.race([c.connect(), new Promise(r => setTimeout(() => r("pending"), 15000))]);
// -> "pending" after 15s, isOpen=true isReady=false   (redis@6.2.1)
```

Because each request built a new handler, each request awaited a fresh,
never-settling `connect()` in `getClient()`. `remote-cache-handler.js`
has one module-level client, but its `getClient()` awaits the same
promise, so it hangs the same way if the app starts while Redis is down.

### 3. `updateTag` never reaches other pages' remote entries

The remote handler's `get` never checks the entry's own tags. Next.js
only passes soft tags to `getExpiration`, and the `"use cache"` wrapper
only knows about tags revalidated in the current request, so
`updateTag("time-data")` from `/cet` left the `/gmt` entry stale on
every instance, including the one that ran it, until it expired
(`cacheLife` `expire: 3600`). The [`cacheHandlers`
docs](https://nextjs.org/docs/app/api-reference/config/next-config-js/cacheHandlers#get)
say `get` should report an entry whose tag was invalidated as missing or
stale.

### Fix

- Hoist the client in `cache-handler.js` to module scope (the pattern
`remote-cache-handler.js` already uses).
- `getClient()` awaits the connect promise raced against a 1s
`unref()`'d timer (suggested in review), then returns the client when
`isReady` and `null` otherwise. Requests during startup still wait for
the connection, and a down Redis costs one bounded wait instead of
blocking every request. The client keeps retrying and `isReady` flips
back on its own.
- `disableOfflineQueue: true`, so a command issued while the connection
is down rejects immediately (`ClientOfflineError`) instead of being
queued for the 5s command timeout.
- Entry `get` / `set` catch the Redis call only and degrade to a miss.
- Remote `get` compares the entry's tags against their revalidation
timestamps (one `MGET`) and misses when any is newer, the same
comparison as the built-in handler. `getExpiration` returns `Date.now()`
when Redis can't answer, so the entry is discarded rather than served.
- `revalidateTag` / `updateTags` throw when Redis isn't ready, so an
invalidation that never reached Redis surfaces as an error instead of a
success whose entries come back once Redis does.

### After

Two `next start` instances on one Redis, same script on `next@16.3.5`
and `16.4.0-canary.35` (identical results), canary's handlers vs this
PR:

| scenario | before | after |
| --- | --- | --- |
| `updateTag` from `/cet`, read `/gmt` on both instances | stale |
refreshed |
| `revalidateTag("time-data", "max")` | neither page refreshed | both
refreshed |
| tag lookup fails | 500 | 200, regenerated |
| request while Redis is stopped | no response | 200, uncached |
| `updateTag` while Redis is stopped | no response | 500, error logged |
| app started without Redis | no response | first ISR read waits ≤1.1s
once |
| Redis connections, 200 requests | 173 → 773 | 7 → 7 |
| `/cet` under load (autocannon, 10 connections × 10s, `16.3.5`) | 332
req/s, hits Redis `maxclients` | 1,106 req/s median, p99 20ms, 3
connections |
| 10 readers during 20 `updateTag`s: reads served a value from before a
completed invalidation | 3,089 of 3,089 | 0 of 33,899 |
| Redis down ~4s under load | timeouts, caching doesn't come back | 0
errors, caching resumes |

The tag check costs one Redis round trip per remote hit (−9 to −13% on a
route that only reads one remote entry). Handler-level cases, full
tables and the benchmark breakdown are in [this
comment](https://github.com/vercel/next.js/pull/98716#issuecomment-5724918096).
parent 9014f046
......@@ -46,9 +46,9 @@ The `/[timezone]` page renders a mostly static shell and, inside it, a `'use cac
- **ISR cache (`cache-handler.js`):** stores the prerendered page entries as JSON under a `nextjs:cache:` prefix, and tracks which keys belong to each tag in a Redis set (`nextjs:tag:<tag>`). `revalidateTag` deletes every key associated with a tag.
- **Remote cache (`remote-cache-handler.js`):** stores each `'use cache: remote'` entry under a `nextjs:use-cache:` prefix (the streamed value is base64-encoded). Tag revalidation is timestamp-based: `updateTags` records `nextjs:use-cache-tag:<tag>` = now, and `getExpiration` reports the latest time so Next treats older entries as stale. Clicking **Revalidate** calls [`updateTag('time-data')`](https://nextjs.org/docs/app/api-reference/functions/updateTag), which regenerates the remote entry.
- **Remote cache (`remote-cache-handler.js`):** stores each `'use cache: remote'` entry under a `nextjs:use-cache:` prefix (the streamed value is base64-encoded). Tag revalidation is timestamp-based: `updateTags` records `nextjs:use-cache-tag:<tag>` = now. On a hit, `get` compares the entry's own tags (from `cacheTag`) against those timestamps and reports a miss if any is newer, and `getExpiration` reports the latest time for the route's soft tags so Next discards older entries. Clicking **Revalidate** calls [`updateTag('time-data')`](https://nextjs.org/docs/app/api-reference/functions/updateTag), which regenerates the remote entry.
- **Building without Redis:** both handlers skip connecting during `next build` (they check `NEXT_PHASE`) and degrade gracefully when Redis is unavailable, so the app still builds and runs, just without a shared cache.
- **Building without Redis:** both handlers skip connecting during `next build` (they check `NEXT_PHASE`) and degrade gracefully when Redis is unavailable, so the app still builds and runs, just without a shared cache. Revalidation is the exception: `revalidateTag` and `updateTags` throw while Redis is unavailable, because an invalidation that never reached Redis would be lost, and the old entries would be served again once Redis is back.
- **Redis server setup:** ensure your Redis server is running before starting the app. Configure the connection with `REDIS_URL` (defaults to `redis://localhost:6379`).
......
......@@ -43,49 +43,77 @@ function deserialize(text) {
});
}
// Next.js constructs the `cacheHandler` class once per request, so the Redis
// client must live at module scope: creating it in the constructor would open
// a new connection on every request and never close it.
const client = createClient({
url: process.env.REDIS_URL ?? "redis://localhost:6379",
// Fail commands immediately while the connection is down instead of queueing
// them until Redis is back.
disableOfflineQueue: true,
});
// Redis won't work without error handling. Do not throw here, otherwise the
// client won't reconnect after a connection drop.
client.on("error", (error) => {
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis client error:", error);
}
});
// Connecting to Redis during `next build` can cause issues, so we only connect
// at runtime.
const connection =
process.env.NEXT_PHASE === PHASE_PRODUCTION_BUILD
? Promise.resolve()
: client.connect().catch((error) => {
console.warn("Failed to connect to Redis:", error);
});
// `connect()` stays pending for as long as Redis is unreachable, so cap the
// wait: requests arriving during startup wait for the connection at most
// once, and while Redis is down every request is served uncached instead of
// blocking. The client keeps retrying in the background, so `isReady` flips
// back on its own once Redis is reachable again.
const CONNECT_TIMEOUT_MS = 1000;
const ready = Promise.race([
connection,
// `unref()` so this timer never keeps the process alive.
new Promise((resolve) => setTimeout(resolve, CONNECT_TIMEOUT_MS).unref()),
]);
// Resolve a connected client, or `null` when Redis is unavailable so the app
// keeps working (without a shared cache) instead of hanging or crashing.
async function getClient() {
await ready;
return client.isReady ? client : null;
}
module.exports = class CacheHandler {
constructor(options) {
this.options = options;
this.client = createClient({
url: process.env.REDIS_URL ?? "redis://localhost:6379",
});
// Redis won't work without error handling. Do not throw here, otherwise
// the client won't reconnect after a connection drop.
this.client.on("error", (error) => {
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis client error:", error);
}
});
// Connecting to Redis during `next build` can cause issues, so we only
// connect at runtime. `this.connection` resolves once the client is ready.
this.connection =
process.env.NEXT_PHASE === PHASE_PRODUCTION_BUILD
? Promise.resolve()
: this.client.connect().catch((error) => {
console.warn("Failed to connect to Redis:", error);
});
}
// Resolve a connected client, or `null` when Redis is unavailable so the
// app keeps working (without a shared cache) instead of crashing.
async getClient() {
await this.connection;
return this.client.isReady ? this.client : null;
}
async get(key) {
const client = await this.getClient();
const client = await getClient();
if (!client) return null;
const entry = await client.get(CACHE_PREFIX + key);
let entry;
try {
entry = await client.get(CACHE_PREFIX + key);
} catch (error) {
// A connection dropping mid-request degrades to a cache miss.
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis get failed:", error);
}
return null;
}
return entry ? deserialize(entry) : null;
}
async set(key, data, ctx) {
const client = await this.getClient();
const client = await getClient();
if (!client || !data) return;
// Collect tags from both sources: `ctx.tags` (fetch entries) and the
......@@ -105,19 +133,29 @@ module.exports = class CacheHandler {
? { expiration: { type: "EX", value: Math.max(1, Math.ceil(expire)) } }
: {};
await client.set(
CACHE_PREFIX + key,
serialize({ value: data, lastModified: Date.now(), tags }),
options,
);
const value = serialize({ value: data, lastModified: Date.now(), tags });
try {
await client.set(CACHE_PREFIX + key, value, options);
// Index this key under each of its tags so `revalidateTag` can find it.
await Promise.all(tags.map((tag) => client.sAdd(TAG_PREFIX + tag, key)));
// Index this key under each of its tags so `revalidateTag` can find it.
await Promise.all(tags.map((tag) => client.sAdd(TAG_PREFIX + tag, key)));
} catch (error) {
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis set failed:", error);
}
}
}
async revalidateTag(tags) {
const client = await this.getClient();
if (!client) return;
const client = await getClient();
// Don't report success for a revalidation that never reached Redis: once
// Redis is back, every instance would serve the old entries again.
if (!client) {
throw new Error(
"Redis is unavailable, so the tag revalidation was not recorded",
);
}
// `tags` is either a single tag or an array of tags.
for (const tag of [tags].flat()) {
......
......@@ -13,6 +13,9 @@ const TAG_PREFIX = "nextjs:use-cache-tag:";
const client = createClient({
url: process.env.REDIS_URL ?? "redis://localhost:6379",
// Fail commands immediately while the connection is down instead of queueing
// them until Redis is back.
disableOfflineQueue: true,
});
client.on("error", (error) => {
......@@ -28,8 +31,20 @@ const connection =
console.warn("Failed to connect to Redis (remote cache):", error);
});
// `connect()` stays pending for as long as Redis is unreachable, so cap the
// wait: requests arriving during startup wait for the connection at most
// once, and while Redis is down every request is served uncached instead of
// blocking. The client keeps retrying in the background, so `isReady` flips
// back on its own once Redis is reachable again.
const CONNECT_TIMEOUT_MS = 1000;
const ready = Promise.race([
connection,
// `unref()` so this timer never keeps the process alive.
new Promise((resolve) => setTimeout(resolve, CONNECT_TIMEOUT_MS).unref()),
]);
async function getClient() {
await connection;
await ready;
return client.isReady ? client : null;
}
......@@ -38,7 +53,16 @@ module.exports = {
const redis = await getClient();
if (!redis) return undefined;
const stored = await redis.get(ENTRY_PREFIX + cacheKey);
let stored;
try {
stored = await redis.get(ENTRY_PREFIX + cacheKey);
} catch (error) {
// A connection dropping mid-request degrades to a cache miss.
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis get failed (remote cache):", error);
}
return undefined;
}
if (!stored) return undefined;
const data = JSON.parse(stored);
......@@ -50,6 +74,33 @@ module.exports = {
return undefined;
}
// Next.js only asks `getExpiration` about the route's soft tags, so the
// entry's own tags (from `cacheTag`) are checked here. If any of them was
// revalidated after this entry was created, on this instance or another,
// the entry is out of date: report a miss so Next.js regenerates it.
if (data.tags.length) {
let revalidatedAt;
try {
revalidatedAt = await redis.mGet(
data.tags.map((tag) => TAG_PREFIX + tag),
);
} catch (error) {
// Without the tag timestamps we can't tell, so don't serve it.
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis tag lookup failed (remote cache):", error);
}
return undefined;
}
if (
revalidatedAt.some(
(time) => time !== null && Number(time) > data.timestamp,
)
) {
return undefined;
}
}
return {
// `value` must be a stream; rebuild it from the stored bytes.
value: new ReadableStream({
......@@ -99,38 +150,60 @@ module.exports = {
}
: {};
await redis.set(
ENTRY_PREFIX + cacheKey,
JSON.stringify({
value: bytes.toString("base64"),
tags: entry.tags,
stale: entry.stale,
timestamp: entry.timestamp,
expire: entry.expire,
revalidate: entry.revalidate,
}),
options,
);
const value = JSON.stringify({
value: bytes.toString("base64"),
tags: entry.tags,
stale: entry.stale,
timestamp: entry.timestamp,
expire: entry.expire,
revalidate: entry.revalidate,
});
try {
await redis.set(ENTRY_PREFIX + cacheKey, value, options);
} catch (error) {
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis set failed (remote cache):", error);
}
}
},
// Redis is the single source of truth and every read hits it, so there's no
// local tag state to sync between requests.
async refreshTags() {},
// Return the most recent revalidation time across `tags`. Next treats an
// entry as stale when this is newer than the entry's `timestamp`.
// Return the most recent revalidation time across `tags`. Next.js calls
// this after a hit with the route's soft tags (the implicit `_N_T_` tags
// that `revalidatePath` uses) and discards the entry when the result is at
// or after the entry's `timestamp`.
async getExpiration(tags) {
if (!tags.length) return 0;
// If Redis can't answer, report the tags as revalidated just now: Next.js
// then discards the entry and regenerates it, the same miss `get` falls
// back to. Returning `0` would instead serve an entry that a
// `revalidatePath` may already have invalidated.
const redis = await getClient();
if (!redis || !tags.length) return 0;
if (!redis) return Date.now();
let values;
try {
values = await redis.mGet(tags.map((tag) => TAG_PREFIX + tag));
} catch (error) {
if (process.env.NEXT_PRIVATE_DEBUG_CACHE) {
console.warn("Redis tag lookup failed (remote cache):", error);
}
return Date.now();
}
const values = await redis.mGet(tags.map((tag) => TAG_PREFIX + tag));
const timestamps = values.filter(Boolean).map(Number);
return timestamps.length ? Math.max(...timestamps) : 0;
},
// Record when each tag was last revalidated so `getExpiration` can report
// it. There's one key per distinct tag, overwritten in place, so a small
// fixed tag set (like this example's single `time-data` tag) never grows.
// Record when each tag was last revalidated, for `get` (the entry's own
// tags) and `getExpiration` (soft tags) to compare against. There's one key
// per distinct tag, overwritten in place, so a small fixed tag set (like
// this example's single `time-data` tag) never grows.
//
// An app that mints many distinct, short-lived tags (e.g. `user-<id>`) would
// instead keep a key per tag forever. To bound that, give each key a TTL
......@@ -140,7 +213,13 @@ module.exports = {
// still cached and serve it as fresh when it should be stale.
async updateTags(tags) {
const redis = await getClient();
if (!redis) return;
// Don't report success for a revalidation Redis never recorded: once Redis
// is back, every instance would serve the old entries again.
if (!redis) {
throw new Error(
"Redis is unavailable, so the tag revalidation was not recorded",
);
}
const now = String(Date.now());
await Promise.all(tags.map((tag) => redis.set(TAG_PREFIX + tag, now)));
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment