Unverified Commit 269020a0 authored by Alexander Savelyev's avatar Alexander Savelyev Committed by GitHub

Disable 2mb limit for custom incrementalCacheHandler (#59976)

### Fixing a bug

### What?

Disable 2MB limit for custom incrementalCacheHandler

### Why?

The limit is necessary because `FetchCache` has a 2MB limit, but it
seems there was a miscommunication regarding the key coincidence, where
`fetchCache` is a flag indicating that the method is called from fetch,
rather than indicating that the `FetchCache` Provider is currently being
used.

We do not use Vercel, and as I understand it, we do not have the
opportunity to use this functionality.

In any case, it is more important for us to increase the limits, and in
some cases, using a file storage is even preferable.

### How?

I have created a flag that determines whether the use of `FetchCache` is
possible at least in theory - if no custom provider is passed, and
additionally configured it so that it is not an implementation of
`FetchCache` as a protection against special individuals (*like me :)*).

If everything is fine, I will write proper tests.

Also, I would like to recommend making `FileSystemCache` public (_i.e.
support it as public functionality_) so that it can be imported and
extended or simply used to fix only it.

Fixes #48324 (partially)

---------
Co-authored-by: 's avatarJJ Kasper <jj@jjsweb.site>
parent fbd5e918
...@@ -65,6 +65,7 @@ export class CacheHandler { ...@@ -65,6 +65,7 @@ export class CacheHandler {
export class IncrementalCache implements IncrementalCacheType { export class IncrementalCache implements IncrementalCacheType {
dev?: boolean dev?: boolean
cacheHandler?: CacheHandler cacheHandler?: CacheHandler
hasCustomCacheHandler: boolean
prerenderManifest: PrerenderManifest prerenderManifest: PrerenderManifest
requestHeaders: Record<string, undefined | string | string[]> requestHeaders: Record<string, undefined | string | string[]>
requestProtocol?: 'http' | 'https' requestProtocol?: 'http' | 'https'
...@@ -112,6 +113,7 @@ export class IncrementalCache implements IncrementalCacheType { ...@@ -112,6 +113,7 @@ export class IncrementalCache implements IncrementalCacheType {
experimental: { ppr: boolean } experimental: { ppr: boolean }
}) { }) {
const debug = !!process.env.NEXT_PRIVATE_DEBUG_CACHE const debug = !!process.env.NEXT_PRIVATE_DEBUG_CACHE
this.hasCustomCacheHandler = Boolean(CurCacheHandler)
if (!CurCacheHandler) { if (!CurCacheHandler) {
if (fs && serverDistDir) { if (fs && serverDistDir) {
if (debug) { if (debug) {
...@@ -555,8 +557,14 @@ export class IncrementalCache implements IncrementalCacheType { ...@@ -555,8 +557,14 @@ export class IncrementalCache implements IncrementalCacheType {
} }
if (this.dev && !ctx.fetchCache) return if (this.dev && !ctx.fetchCache) return
// fetchCache has upper limit of 2MB per-entry currently // FetchCache has upper limit of 2MB per-entry currently
if (ctx.fetchCache && JSON.stringify(data).length > 2 * 1024 * 1024) { if (
ctx.fetchCache &&
// we don't show this error/warning when a custom cache handler is being used
// as it might not have this limit
this.hasCustomCacheHandler &&
JSON.stringify(data).length > 2 * 1024 * 1024
) {
if (this.dev) { if (this.dev) {
throw new Error(`fetch for over 2MB of data can not be cached`) throw new Error(`fetch for over 2MB of data can not be cached`)
} }
......
...@@ -516,6 +516,8 @@ createNextDescribe( ...@@ -516,6 +516,8 @@ createNextDescribe(
'ssr-forced/page.js', 'ssr-forced/page.js',
'articles/works.rsc', 'articles/works.rsc',
'force-cache/page.js', 'force-cache/page.js',
'force-cache/large-data/page.js',
'force-cache/large-data/page_client-reference-manifest.js',
'ssg-draft-mode.html', 'ssg-draft-mode.html',
'articles/works.html', 'articles/works.html',
'no-store/static.rsc', 'no-store/static.rsc',
...@@ -533,6 +535,7 @@ createNextDescribe( ...@@ -533,6 +535,7 @@ createNextDescribe(
'force-static-fetch-no-store/page_client-reference-manifest.js', 'force-static-fetch-no-store/page_client-reference-manifest.js',
'force-static/first.rsc', 'force-static/first.rsc',
'api/draft-mode/route.js', 'api/draft-mode/route.js',
'api/large-data/route.js',
'blog/tim/first-post.rsc', 'blog/tim/first-post.rsc',
'force-static/first.html', 'force-static/first.html',
'force-static/second.rsc', 'force-static/second.rsc',
...@@ -770,6 +773,26 @@ createNextDescribe( ...@@ -770,6 +773,26 @@ createNextDescribe(
"initialRevalidateSeconds": false, "initialRevalidateSeconds": false,
"srcRoute": "/", "srcRoute": "/",
}, },
"/api/large-data": {
"dataRoute": null,
"experimentalBypassFor": [
{
"key": "Next-Action",
"type": "header",
},
{
"key": "content-type",
"type": "header",
"value": "multipart/form-data",
},
],
"initialHeaders": {
"content-type": "application/json",
"x-next-cache-tags": "_N_T_/layout,_N_T_/api/layout,_N_T_/api/large-data/layout,_N_T_/api/large-data/route,_N_T_/api/large-data",
},
"initialRevalidateSeconds": false,
"srcRoute": "/api/large-data",
},
"/articles/works": { "/articles/works": {
"dataRoute": "/articles/works.rsc", "dataRoute": "/articles/works.rsc",
"experimentalBypassFor": [ "experimentalBypassFor": [
...@@ -3053,6 +3076,12 @@ createNextDescribe( ...@@ -3053,6 +3076,12 @@ createNextDescribe(
expect(next.cliOutput).toContain('cache-handler get') expect(next.cliOutput).toContain('cache-handler get')
expect(next.cliOutput).toContain('cache-handler set') expect(next.cliOutput).toContain('cache-handler set')
}) })
it('should load large data only once when using custom cache handler and force-cache mode', async () => {
await next.fetch('/force-cache/large-data')
await next.fetch('/force-cache/large-data')
expect(next.cliOutput.match('Load data').length).toBe(1)
})
} }
} }
) )
import { NextResponse } from 'next/server'
export async function GET() {
console.log('Load data')
return NextResponse.json({
content: Array.from(new Array(3 * 1024 * 1024))
.map(() => 1)
.join(''),
})
}
export default async function Page() {
const resp = await fetch(
`http://localhost:${process.env.PORT}/api/large-data`,
{
cache: 'force-cache',
}
)
const data = await resp.json()
return <p id="content">{data.content}</p>
}
export const dynamic = 'force-dynamic'
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment