Unverified Commit 707b1e3b authored by Shu Ding's avatar Shu Ding Committed by GitHub

Limit sharp's concurrency (#53385)

Sharp by default spawns #cpu_cores threads to process images which could lead to large memory consumption. This PR caps the `concurrency` value especially on dev.

Locally I see a small memory improvement (10~20 MB) from this, but it will mostly affect long-running servers.

Related: https://github.com/lovell/sharp/issues/2607Co-authored-by: 's avatarSteven <229881+styfle@users.noreply.github.com>
parent 10cb7406
import { createHash } from 'crypto' import { createHash } from 'crypto'
import { promises } from 'fs' import { promises } from 'fs'
import { cpus } from 'os'
import type { IncomingMessage, ServerResponse } from 'http' import type { IncomingMessage, ServerResponse } from 'http'
import { mediaType } from 'next/dist/compiled/@hapi/accept' import { mediaType } from 'next/dist/compiled/@hapi/accept'
import chalk from 'next/dist/compiled/chalk' import chalk from 'next/dist/compiled/chalk'
...@@ -43,15 +44,17 @@ const VECTOR_TYPES = [SVG] ...@@ -43,15 +44,17 @@ const VECTOR_TYPES = [SVG]
const BLUR_IMG_SIZE = 8 // should match `next-image-loader` const BLUR_IMG_SIZE = 8 // should match `next-image-loader`
const BLUR_QUALITY = 70 // should match `next-image-loader` const BLUR_QUALITY = 70 // should match `next-image-loader`
let sharp: let sharp: typeof import('sharp') | undefined
| ((
input?: string | Buffer,
options?: import('sharp').SharpOptions
) => import('sharp').Sharp)
| undefined
try { try {
sharp = require(process.env.NEXT_SHARP_PATH || 'sharp') sharp = require(process.env.NEXT_SHARP_PATH || 'sharp')
if (sharp && sharp.concurrency() > 1) {
// Reducing concurrency should reduce the memory usage too.
// We more aggressively reduce in dev but also reduce in prod.
// https://sharp.pixelplumbing.com/api-utility#concurrency
const divisor = process.env.NODE_ENV === 'development' ? 4 : 2
sharp.concurrency(Math.floor(Math.max(cpus().length / divisor, 1)))
}
} catch (e) { } catch (e) {
// Sharp not present on the server, Squoosh fallback will be used // Sharp not present on the server, Squoosh fallback will be used
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment