Unverified Commit d54d43d4 authored by Shu Ding's avatar Shu Ding Committed by GitHub

Use only 1 worker for type checking and linting (#45639)

During build, there's no need to start N workers for type checking and
linting, because there's no concurrency at all for these tasks (both
TypeScript and ESLint are checking all files globally).

During my test of `next build` for a small size project of 5 pages, this
change reduces the peak memory usage by 545MB (as each worker holds a
huge dependency chain). Before (there are too many worker threads
initiated):

<img width="1278" alt="before"
src="https://user-images.githubusercontent.com/3676859/217116925-5594a45c-f1bb-4d11-8699-f6fe12ce7ee5.png">

After:

<img width="1189" alt="CleanShot-2023-02-07-WfQTnB1D@2x"
src="https://user-images.githubusercontent.com/3676859/217116940-a08a256e-b80d-4836-8b97-2837c845435e.png">

It improves the CPU usage as well.

In the future we can spin up multiple workers and assign sub-task for
each, but that only makes sense when the number of pages is large.

NEXT-470
parent 1596e12b
...@@ -160,14 +160,13 @@ function verifyTypeScriptSetup( ...@@ -160,14 +160,13 @@ function verifyTypeScriptSetup(
tsconfigPath: string, tsconfigPath: string,
disableStaticImages: boolean, disableStaticImages: boolean,
cacheDir: string | undefined, cacheDir: string | undefined,
numWorkers: number | undefined,
enableWorkerThreads: boolean | undefined, enableWorkerThreads: boolean | undefined,
isAppDirEnabled: boolean isAppDirEnabled: boolean
) { ) {
const typeCheckWorker = new JestWorker( const typeCheckWorker = new JestWorker(
require.resolve('../lib/verifyTypeScriptSetup'), require.resolve('../lib/verifyTypeScriptSetup'),
{ {
numWorkers, numWorkers: 1,
enableWorkerThreads, enableWorkerThreads,
maxRetries: 0, maxRetries: 0,
} }
...@@ -409,7 +408,6 @@ export default async function build( ...@@ -409,7 +408,6 @@ export default async function build(
config.typescript.tsconfigPath, config.typescript.tsconfigPath,
config.images.disableStaticImages, config.images.disableStaticImages,
cacheDir, cacheDir,
config.experimental.cpus,
config.experimental.workerThreads, config.experimental.workerThreads,
isAppDirEnabled isAppDirEnabled
).then((resolved) => { ).then((resolved) => {
...@@ -425,7 +423,6 @@ export default async function build( ...@@ -425,7 +423,6 @@ export default async function build(
dir, dir,
eslintCacheDir, eslintCacheDir,
config.eslint?.dirs, config.eslint?.dirs,
config.experimental.cpus,
config.experimental.workerThreads, config.experimental.workerThreads,
telemetry, telemetry,
isAppDirEnabled && !!appDir isAppDirEnabled && !!appDir
......
...@@ -12,14 +12,13 @@ export async function verifyAndLint( ...@@ -12,14 +12,13 @@ export async function verifyAndLint(
dir: string, dir: string,
cacheLocation: string, cacheLocation: string,
configLintDirs: string[] | undefined, configLintDirs: string[] | undefined,
numWorkers: number | undefined,
enableWorkerThreads: boolean | undefined, enableWorkerThreads: boolean | undefined,
telemetry: Telemetry, telemetry: Telemetry,
hasAppDir: boolean hasAppDir: boolean
): Promise<void> { ): Promise<void> {
try { try {
const lintWorkers = new Worker(require.resolve('./eslint/runLintCheck'), { const lintWorkers = new Worker(require.resolve('./eslint/runLintCheck'), {
numWorkers, numWorkers: 1,
enableWorkerThreads, enableWorkerThreads,
maxRetries: 0, maxRetries: 0,
}) as Worker & { }) as Worker & {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment