Count URL key length in the filesystem route cache LRU (#96229)
Fixes #94890 Adopted from #96187 and #94699, with both authors credited on the commit. The router server keeps an in-memory cache that maps request URL paths to what they resolve to: a page, a static file, or nothing. It's bounded by a size budget, but the size of each entry was computed from the cached value only. The URL string used as the key was free, and "resolves to nothing" entries counted as size 1. So a long-lived server receiving many unique URLs — bot crawls, long slugs, `/order/:id`-style traffic — could accumulate up to ~1M URL strings while the cache believed it was within budget. That looks like a slow memory leak that tracks how many distinct URLs you've served rather than how much traffic you get. #94890 has several production reports, including two deployments that patched key accounting in and saw memory go flat. The idea of the fix is to make the cache's bookkeeping match what an entry actually keeps alive, so the budget becomes a real memory bound. That takes three things: 1. Count the key string, not just the value. 2. Count the fixed cost an entry has regardless of string lengths — the per-entry bookkeeping objects, measured at ~120 bytes. Without this, a flood of *short* unique URLs stays "cheap" on paper while the bookkeeping itself grows unbounded. 3. Copy strings before caching them. When V8 takes a substring of a long string (which is how the pathname is extracted from the request URL), it doesn't copy the characters — it creates a small object that points into the original string, and the original can't be garbage-collected while the substring is alive. So caching a 30-char pathname could actually hold the entire URL in memory, kilobyte query string and all, and the bookkeeping only sees the 30 chars. Copying the string before caching makes it hold only its own characters. With the bookkeeping honest, the budget is set to 8 MiB, roughly the real memory the cache could already use for legitimate entries before this change, so capacity isn't reduced — the previous attempt at this fix (#94942) stalled on that concern. ## Test Plan A/B on a production `next start` serving 60k requests with unique long URLs (the reported traffic shape): canary grows +79 MB and keeps growing; this branch grows +9 MB and plateaus — the budget working as intended. The per-entry costs above were measured directly on the cache with forced GC (they fit `118 + key length` bytes within a few percent, which is also what predicts the A/B numbers). `lru-cache` unit tests (26/26) and `pnpm --filter=next types` pass. --------- Co-authored-by:Marcos Hernanz <96699542+marcoshernanz@users.noreply.github.com> Co-authored-by:
dmchoi77 <dm.choi.dev@gmail.com>
Showing
Please
register
or
sign in
to comment