Unverified Commit 97056e0d authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

Add a --webpack flag and default --turbopack to true (#84216)

Make Turbopack the default bundler for Next 🚀

## What

Add a `--webpack` flag so users can select webpack

Change behavior so if neither `--turbopack` or `--webpack` is set we default to Turbopack.

If we have defaulted the build to turbopack and we observe that the user has a `webpack` config in their next config but no turbopack config, then we issue a warning about this being a potential problem, if this happens during `next build` the warning becomes an error and fails the build.  The solution for users is just to explicitly set `--webpack` or `--turbopack`

There were a number of subtle issues
* some users directly set the `TURBOPACK` environment variable, this PR adds support for that though users should really be passing `--turbopack` so it will not be documented
* rspack is enabled via a plugin that sets an environment variable when loading next config, which means it happens way too late!
   * For builds this isn't too bad, we just have to recompute the `Bundler` after loading the config.  For `dev` the parent process can get out of sync with the child, but this is only really relevant for telemetry and for that we already load the config in the parent so just defer computing `isTurboSession`.

Most of this is about fixing package.json scripts and CI builds configs.

* For package.json 
   * i added aliases e.g. `test-dev` == `test-dev-webpack` but preserved the old names.  In the long run we will want to remove the unsuffixed aliases 
   * this required some modifications to existing configs to ensure everything was setting the correct env variables

* For ci, i added a new `IS_WEBPACK_TEST` variable to a number of tests but preserved all names.
   * Again, in the future it would make sense to rename ci jobs but that is deferred for right now.
   * This also makes it clear that a set of tests scenarios (e.g. ppr, experimental, test-new-tests-*) never run with turbopack. This is not addressed right now but should be in the future.

## Why

Today, the default bundler for next is webpack but with turbopack becoming stable it is time to just ship it.   Turbopack is already recommended for dev and builds.  Create Next App also steers users towards Turbopack.  According to telemetry we already have about 50% of all dev sessions and build adoption is growing.  So fundamentally why are we even asking users to make a decision?
parent f31d18d8
......@@ -502,6 +502,7 @@ jobs:
export NEXT_TEST_MODE=start
export NEXT_TEST_WASM=true
export IS_WEBPACK_TEST=1
node run-tests.js \
test/production/pages-dir/production/test/index.test.ts \
test/e2e/streaming-ssr/index.test.ts
......@@ -711,7 +712,7 @@ jobs:
secrets: inherit
test-dev:
test-dev: # TODO: rename to include webpack
name: test dev
needs: ['optimize-ci', 'changes', 'build-native', 'build-next']
if: ${{ needs.optimize-ci.outputs.skip == 'false' && needs.changes.outputs.docs-only == 'false' }}
......@@ -728,6 +729,7 @@ jobs:
uses: ./.github/workflows/build_reusable.yml
with:
afterBuild: |
export IS_WEBPACK_TEST=1
export NEXT_TEST_MODE=dev
export NEXT_TEST_REACT_VERSION="${{ matrix.react }}"
......@@ -752,8 +754,10 @@ jobs:
uses: ./.github/workflows/build_reusable.yml
with:
# Should this be using turbopack? a variation?
afterBuild: |
export NEXT_TEST_MODE=dev
export IS_WEBPACK_TEST=1
node run-tests.js \
test/e2e/app-dir/app/index.test.ts \
......@@ -780,6 +784,7 @@ jobs:
with:
nodeVersion: 20.9.0
afterBuild: |
export IS_WEBPACK_TEST=1
node run-tests.js \
--concurrency 4 \
test/production/pages-dir/production/test/index.test.ts \
......@@ -808,6 +813,7 @@ jobs:
with:
afterBuild: |
export NEXT_TEST_MODE=start
export IS_WEBPACK_TEST=1
node run-tests.js --type production \
test/e2e/app-dir/app/index.test.ts \
......@@ -836,6 +842,7 @@ jobs:
uses: ./.github/workflows/build_reusable.yml
with:
afterBuild: |
export IS_WEBPACK_TEST=1
export NEXT_TEST_MODE=start
export NEXT_TEST_REACT_VERSION="${{ matrix.react }}"
......@@ -875,6 +882,7 @@ jobs:
with:
nodeVersion: 20.9.0
afterBuild: |
export IS_WEBPACK_TEST=1
export NEXT_TEST_REACT_VERSION="${{ matrix.react }}"
node run-tests.js \
......@@ -896,6 +904,7 @@ jobs:
# these all run without concurrency because they're heavier
export TEST_CONCURRENCY=1
export IS_WEBPACK_TEST=1
BROWSER_NAME=firefox node run-tests.js \
test/production/pages-dir/production/test/index.test.ts
......@@ -923,6 +932,7 @@ jobs:
afterBuild: |
export __NEXT_EXPERIMENTAL_PPR=true
export NEXT_EXTERNAL_TESTS_FILTERS="test/ppr-tests-manifest.json"
export IS_WEBPACK_TEST=1
node run-tests.js \
--timings \
......@@ -945,6 +955,7 @@ jobs:
export __NEXT_EXPERIMENTAL_PPR=true
export NEXT_EXTERNAL_TESTS_FILTERS="test/ppr-tests-manifest.json"
export NEXT_TEST_MODE=dev
export IS_WEBPACK_TEST=1
node run-tests.js \
--timings \
......@@ -968,6 +979,7 @@ jobs:
export __NEXT_EXPERIMENTAL_PPR=true
export NEXT_EXTERNAL_TESTS_FILTERS="test/ppr-tests-manifest.json"
export NEXT_TEST_MODE=start
export IS_WEBPACK_TEST=1
node run-tests.js \
--timings \
......@@ -990,6 +1002,7 @@ jobs:
export __NEXT_EXPERIMENTAL_PPR=true # for compatibility with the existing tests
export __NEXT_EXPERIMENTAL_CACHE_COMPONENTS=true
export NEXT_EXTERNAL_TESTS_FILTERS="test/experimental-tests-manifest.json"
export IS_WEBPACK_TEST=1
node run-tests.js \
--timings \
......@@ -1014,6 +1027,7 @@ jobs:
export NEXT_EXTERNAL_TESTS_FILTERS="test/experimental-tests-manifest.json"
export NEXT_TEST_MODE=dev
export __NEXT_EXPERIMENTAL_ISOLATED_DEV_BUILD=true
export IS_WEBPACK_TEST=1
node run-tests.js \
--timings \
......@@ -1038,6 +1052,7 @@ jobs:
export __NEXT_EXPERIMENTAL_CACHE_COMPONENTS=true
export NEXT_EXTERNAL_TESTS_FILTERS="test/experimental-tests-manifest.json"
export NEXT_TEST_MODE=start
export IS_WEBPACK_TEST=1
node run-tests.js \
--timings \
......
......@@ -4,9 +4,9 @@
"private": true,
"scripts": {
"dev-turbopack": "next dev --turbopack",
"dev-webpack": "next dev",
"dev-webpack": "next dev --webpack",
"build-turbopack": "next build --turbopack",
"build-webpack": "next build",
"build-webpack": "next build --webpack",
"start-turbopack": "next start",
"start-webpack": "next start",
"build-application": "next build",
......
......@@ -3,10 +3,10 @@
"scripts": {
"prepare-bench": "node scripts/prepare-bench.mjs",
"benchmark": "node scripts/benchmark-runner.mjs",
"dev-webpack": "next dev",
"dev-turbopack": "next dev --turbo",
"build-webpack": "next build",
"build-turbopack": "next build --turbo",
"dev-webpack": "next dev --webpack",
"dev-turbopack": "next dev --turbopack",
"build-webpack": "next build --webpack",
"build-turbopack": "next build --turbopack",
"start": "next start"
},
"devDependencies": {
......
......@@ -17,29 +17,49 @@
"ci:publish": "tsx ./scripts/release/publish-npm.ts",
"test-types": "tsc",
"test-unit": "jest test/unit/ packages/next/ packages/font",
"test-dev": "cross-env NEXT_TEST_MODE=dev pnpm testheadless",
"test-dev-experimental": "cross-env NEXT_TEST_MODE=dev pnpm run with-experimental pnpm testheadless",
"test-dev-rspack": "pnpm run with-rspack pnpm run test-dev",
"test-dev-experimental-rspack": "pnpm run with-rspack pnpm run test-dev-experimental",
"test-dev-turbo": "cross-env NEXT_TEST_MODE=dev IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 pnpm testheadless",
"test-dev-experimental-turbo": "cross-env NEXT_TEST_MODE=dev IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 pnpm run with-experimental pnpm testheadless",
"test-start": "cross-env NEXT_TEST_MODE=start pnpm testheadless",
"test-start-experimental": "cross-env NEXT_TEST_MODE=start pnpm run with-experimental pnpm testheadless",
"test-start-rspack": "pnpm run with-rspack pnpm run test-start",
"test-start-experimental-rspack": "pnpm run with-rspack pnpm run test-start-experimental",
"test-start-turbo": "cross-env NEXT_TEST_MODE=start IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm testheadless",
"test-start-experimental-turbo": "cross-env NEXT_TEST_MODE=start IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm run with-experimental pnpm testheadless",
"test-deploy": "cross-env NEXT_TEST_MODE=deploy pnpm testheadless",
"testonly-dev": "cross-env NEXT_TEST_MODE=dev pnpm testonly",
"testonly-dev-rspack": "pnpm run with-rspack pnpm run testonly-dev",
"testonly-dev-turbo": "cross-env NEXT_TEST_MODE=dev IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 pnpm testonly",
"testonly-start": "cross-env NEXT_TEST_MODE=start pnpm testonly",
"testonly-start-rspack": "pnpm run with-rspack pnpm run testonly-start",
"testonly-start-turbo": "cross-env NEXT_TEST_MODE=start IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm testonly",
"testonly-deploy": "cross-env NEXT_TEST_MODE=deploy pnpm testonly",
"test": "pnpm testheadless",
"test-rspack": "pnpm run with-rspack pnpm run test",
"test-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 TURBOPACK_BUILD=1 pnpm testheadless",
"test-dev-inner": "cross-env NEXT_TEST_MODE=dev pnpm testheadless",
"test-dev": "pnpm run test-dev-webpack",
"test-dev-webpack": "pnpm run with-webpack pnpm test-dev-inner",
"test-dev-experimental-inner": "pnpm run with-experimental pnpm test-dev-inner",
"test-dev-experimental": "pnpm run test-dev-experimental-webpack",
"test-dev-experimental-webpack": "pnpm run with-webpack pnpm test-dev-experimental-inner",
"test-dev-rspack": "pnpm run with-rspack pnpm run test-dev-inner",
"test-dev-experimental-rspack": "pnpm run with-rspack pnpm run test-dev-experimental-inner",
"test-dev-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 pnpm test-dev-inner",
"test-dev-experimental-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 pnpm run with-experimental pnpm test-dev-inner",
"test-start-inner": "cross-env NEXT_TEST_MODE=start pnpm testheadless",
"test-start": "pnpm run test-start-webpack",
"test-start-webpack": "pnpm run with-webpack pnpm run test-start-inner",
"test-start-experimental-inner": "pnpm run with-experimental pnpm test-start-inner",
"test-start-experimental": "pnpm run test-start-experimental-webpack",
"test-start-experimental-webpack": "pnpm run with-webpack pnpm run test-start-experimental-inner",
"test-start-rspack": "pnpm run with-rspack pnpm run test-start-inner",
"test-start-experimental-rspack": "pnpm run with-rspack pnpm run test-start-experimental-inner",
"test-start-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm test-start-inner",
"test-start-experimental-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm run with-experimental pnpm test-start-experimental-inner",
"test-deploy-inner": "cross-env NEXT_TEST_MODE=deploy pnpm testheadless",
"test-deploy": "pnpm run test-deploy-webpack",
"test-deploy-webpack": "pnpm run with-webpack pnpm test-deploy-inner",
"test-deploy-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm test-deploy-inner",
"testonly-dev-inner": "cross-env NEXT_TEST_MODE=dev pnpm testonly",
"testonly-dev": "pnpm run testonly-dev-webpack",
"testonly-dev-webpack": "pnpm run with-webpack pnpm run testonly-dev-inner",
"testonly-dev-rspack": "pnpm run with-rspack pnpm run testonly-dev-inner",
"testonly-dev-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 pnpm testonly-dev-inner",
"testonly-start-inner": "cross-env NEXT_TEST_MODE=start pnpm testonly",
"testonly-start": "pnpm run testonly-start-webpack",
"testonly-start-webpack": "pnpm run with-webpack pnpm run testonly-start-inner",
"testonly-start-rspack": "pnpm run with-rspack pnpm run testonly-start-inner",
"testonly-start-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm testonly-start-inner",
"testonly-deploy-inner": "cross-env NEXT_TEST_MODE=deploy pnpm testonly",
"testonly-deploy": "pnpm run testonly-deploy-webpack",
"testonly-deploy-webpack": "pnpm run with-webpack pnpm run testonly-deploy-inner",
"testonly-deploy-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_BUILD=1 pnpm testonly-deploy-inner",
"test-inner": "pnpm testheadless",
"test": "pnpm test-webpack",
"test-webpack": "pnpm run with-webpack pnpm run test-inner",
"test-rspack": "pnpm run with-rspack pnpm run test-inner",
"test-turbo": "cross-env IS_TURBOPACK_TEST=1 TURBOPACK_DEV=1 TURBOPACK_BUILD=1 pnpm test-inner",
"testonly": "jest --runInBand",
"testheadless": "cross-env HEADLESS=true pnpm testonly",
"genstats": "cross-env LOCAL_STATS=true node .github/actions/next-stats-action/src/index.js",
......@@ -89,6 +109,7 @@
"build-storybook": "turbo run build-storybook",
"test-storybook": "turbo run test-storybook",
"with-rspack": "cross-env NEXT_RSPACK=1 NEXT_TEST_USE_RSPACK=1",
"with-webpack": "cross-env IS_WEBPACK_TEST=1",
"with-experimental": "cross-env __NEXT_EXPERIMENTAL_CACHE_COMPONENTS=true __NEXT_EXPERIMENTAL_PPR=true"
},
"devDependencies": {
......
module.exports = function withRspack(config) {
process.env.NEXT_RSPACK = 'true'
process.env.RSPACK_CONFIG_VALIDATE = 'loose-silent'
if (process.env.TURBOPACK === 'auto') {
// If next has defaulted to turbopack, override it.
delete process.env.TURBOPACK
} else if (process.env.TURBOPACK) {
console.error('Cannot call withRspack and pass the --turbopack flag.')
console.error('Please configure only one bundler.')
process.exit(1)
}
return config
}
......@@ -823,5 +823,6 @@
"822": "Stack frame resolver not initialized. This is a bug in Next.js.",
"823": "Timeout waiting for error state from frontend. The browser may not be responding to HMR messages.",
"824": "URL is required in MCP browser response. This is a bug in Next.js.",
"825": "Timeout waiting for response from frontend. The browser may not be responding to HMR messages."
"825": "Timeout waiting for response from frontend. The browser may not be responding to HMR messages.",
"826": "unknown bundler: %s"
}
......@@ -133,8 +133,9 @@ program
.option('--no-mangling', 'Disables mangling.')
.option('--profile', 'Enables production profiling for React.')
.option('--experimental-app-only', 'Builds only App Router routes.')
.option('--turbo', 'Starts development mode using Turbopack.')
.option('--turbopack', 'Starts development mode using Turbopack.')
.option('--turbo', 'Builds using Turbopack.')
.option('--turbopack', 'Builds using Turbopack.')
.option('--webpack', 'Builds using webpack.')
.addOption(
new Option(
'--experimental-build-mode [mode]',
......@@ -173,6 +174,7 @@ program
)
.option('--turbo', 'Starts development mode using Turbopack.')
.option('--turbopack', 'Starts development mode using Turbopack.')
.option('--webpack', 'Starts development mode using webpack.')
.addOption(
new Option(
'-p, --port <port>',
......
......@@ -83,6 +83,7 @@ import {
UNDERSCORE_GLOBAL_ERROR_ROUTE_ENTRY,
} from '../shared/lib/entry-constants'
import { isDynamicRoute } from '../shared/lib/router/utils'
import { Bundler, finalizeBundlerFromConfig } from '../lib/bundler'
import type { __ApiPreviewProps } from '../server/api-utils'
import loadConfig from '../server/config'
import type { BuildManifest } from '../server/get-page-files'
......@@ -901,7 +902,7 @@ export default async function build(
runLint = true,
noMangling = false,
appDirOnly = false,
isTurbopack = false,
bundler = Bundler.Turbopack,
experimentalBuildMode: 'default' | 'compile' | 'generate' | 'generate-env',
traceUploadUrl: string | undefined
): Promise<void> {
......@@ -914,7 +915,6 @@ export default async function build(
try {
const nextBuildSpan = trace('next-build', undefined, {
buildMode: experimentalBuildMode,
isTurboBuild: String(isTurbopack),
version: process.env.__NEXT_VERSION as string,
})
......@@ -949,6 +949,10 @@ export default async function build(
)
loadedConfig = config
// Reading the config can modify environment variables that influence the bundler selection.
bundler = finalizeBundlerFromConfig(bundler)
nextBuildSpan.setAttribute('bundler', getBundlerForTelemetry(bundler))
process.env.NEXT_DEPLOYMENT_ID = config.deploymentId || ''
NextBuildContext.config = config
......@@ -971,7 +975,7 @@ export default async function build(
NextBuildContext.buildId = buildId
if (experimentalBuildMode === 'generate-env') {
if (isTurbopack) {
if (bundler === Bundler.Turbopack) {
Log.warn('generate-env is not needed with turbopack')
process.exit(0)
}
......@@ -1398,7 +1402,7 @@ export default async function build(
})
// Turbopack already handles conflicting app and page routes.
if (!isTurbopack) {
if (bundler !== Bundler.Turbopack) {
const numConflictingAppPaths = conflictingAppPagePaths.length
if (mappedAppPages && numConflictingAppPaths > 0) {
Log.error(
......@@ -1681,7 +1685,7 @@ export default async function build(
let shutdownPromise = Promise.resolve()
if (!isGenerateMode) {
if (isTurbopack) {
if (bundler === Bundler.Turbopack) {
const {
duration: compilerDuration,
shutdownPromise: p,
......@@ -1796,7 +1800,7 @@ export default async function build(
telemetry.record(
eventBuildCompleted(pagesPaths, {
bundler: getBundlerForTelemetry(isTurbopack),
bundler: getBundlerForTelemetry(bundler),
durationInSeconds,
totalAppPagesCount,
})
......@@ -1812,7 +1816,7 @@ export default async function build(
telemetry.record(
eventBuildCompleted(pagesPaths, {
bundler: getBundlerForTelemetry(isTurbopack),
bundler: getBundlerForTelemetry(bundler),
durationInSeconds: compilerDuration,
totalAppPagesCount,
})
......@@ -2450,7 +2454,10 @@ export default async function build(
)
// If there's edge routes, append the edge instrumentation hook
// Turbopack generates this chunk with a hashed name and references it in middleware-manifest.
if (!isTurbopack && (edgeRuntimeAppCount || edgeRuntimePagesCount)) {
if (
bundler !== Bundler.Turbopack &&
(edgeRuntimeAppCount || edgeRuntimePagesCount)
) {
instrumentationHookEntryFiles.push(
path.join(
SERVER_DIRECTORY,
......@@ -2503,7 +2510,7 @@ export default async function build(
path.join(SERVER_DIRECTORY, FUNCTIONS_CONFIG_MANIFEST),
path.join(SERVER_DIRECTORY, MIDDLEWARE_MANIFEST),
path.join(SERVER_DIRECTORY, MIDDLEWARE_BUILD_MANIFEST + '.js'),
...(!isTurbopack
...(bundler !== Bundler.Turbopack
? [
path.join(
SERVER_DIRECTORY,
......@@ -2538,7 +2545,7 @@ export default async function build(
),
]
: []),
...(pagesDir && !isTurbopack
...(pagesDir && bundler !== Bundler.Turbopack
? [
DYNAMIC_CSS_MANIFEST + '.json',
path.join(SERVER_DIRECTORY, DYNAMIC_CSS_MANIFEST + '.js'),
......@@ -2606,7 +2613,7 @@ export default async function build(
],
}
if (isTurbopack) {
if (bundler === Bundler.Turbopack) {
await writeManifest(
path.join(
distDir,
......@@ -2622,7 +2629,11 @@ export default async function build(
await writeFunctionsConfigManifest(distDir, functionsConfigManifest)
if (!isTurbopack && !isGenerateMode && !buildTracesPromise) {
if (
bundler !== Bundler.Turbopack &&
!isGenerateMode &&
!buildTracesPromise
) {
buildTracesPromise = nextBuildSpan
.traceChild('collect-build-traces')
.traceAsyncFn(() => {
......@@ -2764,7 +2775,7 @@ export default async function build(
// we don't need to inline for turbopack build as
// it will handle it's own caching separate of compile
if (isGenerateMode && !isTurbopack) {
if (isGenerateMode && bundler !== Bundler.Turbopack) {
Log.info('Inlining static env ...')
await nextBuildSpan
......@@ -4200,7 +4211,7 @@ export default async function build(
if (telemetry) {
telemetry.record(
eventBuildFailed({
bundler: getBundlerForTelemetry(isTurbopack),
bundler: getBundlerForTelemetry(bundler),
errorCode: getErrorCodeForTelemetry(e),
durationInSeconds: Math.floor((Date.now() - buildStartTime) / 1000),
})
......@@ -4221,7 +4232,7 @@ export default async function build(
mode: 'build',
projectDir: dir,
distDir: loadedConfig.distDir,
isTurboSession: isTurbopack,
isTurboSession: bundler === Bundler.Turbopack,
sync: true,
})
}
......@@ -4235,16 +4246,17 @@ function errorFromUnsupportedSegmentConfig(): never {
process.exit(1)
}
function getBundlerForTelemetry(isTurbopack: boolean) {
if (isTurbopack) {
return 'turbopack'
function getBundlerForTelemetry(bundler: Bundler) {
switch (bundler) {
case Bundler.Turbopack:
return 'turbopack'
case Bundler.Rspack:
return 'rspack'
case Bundler.Webpack:
return 'webpack'
default:
throw new Error(`unknown bundler: ${bundler}`)
}
if (process.env.NEXT_RSPACK) {
return 'rspack'
}
return 'webpack'
}
function getErrorCodeForTelemetry(err: unknown) {
......
......@@ -10,6 +10,7 @@ import isError from '../lib/is-error'
import { getProjectDir } from '../lib/get-project-dir'
import { enableMemoryDebuggingMode } from '../lib/memory/startup'
import { disableMemoryDebuggingMode } from '../lib/memory/shutdown'
import { parseBundlerArgs } from '../lib/bundler'
export type NextBuildOptions = {
debug?: boolean
......@@ -19,6 +20,7 @@ export type NextBuildOptions = {
mangling: boolean
turbo?: boolean
turbopack?: boolean
webpack?: boolean
experimentalDebugMemoryUsage: boolean
experimentalAppOnly?: boolean
experimentalTurbo?: boolean
......@@ -82,12 +84,7 @@ const nextBuild = (options: NextBuildOptions, directory?: string) => {
printAndExit(`> No such directory exists as the project root: ${dir}`)
}
const isTurbopack = Boolean(
options.turbo || options.turbopack || process.env.IS_TURBOPACK_TEST
)
if (isTurbopack) {
process.env.TURBOPACK = '1'
}
const bundler = parseBundlerArgs(options)
return build(
dir,
......@@ -97,7 +94,7 @@ const nextBuild = (options: NextBuildOptions, directory?: string) => {
lint,
!mangling,
experimentalAppOnly,
isTurbopack,
bundler,
experimentalBuildMode,
traceUploadUrl
)
......
......@@ -38,11 +38,17 @@ import { once } from 'node:events'
import { clearTimeout } from 'timers'
import { flushAllTraces, trace } from '../trace'
import { traceId } from '../trace/shared'
import {
Bundler,
finalizeBundlerFromConfig,
parseBundlerArgs,
} from '../lib/bundler'
export type NextDevOptions = {
disableSourceMaps: boolean
turbo?: boolean
turbopack?: boolean
webpack?: boolean
port: number
hostname?: string
experimentalHttps?: boolean
......@@ -58,11 +64,11 @@ let dir: string
let child: undefined | ChildProcess
// The config in next-dev is only used to access config.distDir for telemetry and trace.
let config: NextConfigComplete
let isTurboSession = false
let bundler: Bundler
let traceUploadUrl: string
let sessionStopHandled = false
let sessionStarted = Date.now()
let sessionSpan = trace('next-dev')
const sessionStarted = Date.now()
const sessionSpan = trace('next-dev')
// How long should we wait for the child to cleanly exit after sending
// SIGINT/SIGTERM to the child process before sending SIGKILL?
......@@ -123,11 +129,13 @@ const handleSessionStop = async (signal: NodeJS.Signals | number | null) => {
new Telemetry({
distDir: path.join(dir, config.distDir),
})
// Reading the config can modify environment variables that influence the bundler selection.
bundler = finalizeBundlerFromConfig(bundler)
telemetry.record(
eventCliSessionStopped({
cliCommand: 'dev',
turboFlag: isTurboSession,
turboFlag: bundler === Bundler.Turbopack,
durationMilliseconds: Date.now() - sessionStarted,
pagesDir,
appDir,
......@@ -146,7 +154,7 @@ const handleSessionStop = async (signal: NodeJS.Signals | number | null) => {
mode: 'dev',
projectDir: dir,
distDir: config.distDir,
isTurboSession,
isTurboSession: bundler === Bundler.Turbopack,
})
}
......@@ -168,14 +176,7 @@ const nextDev = async (
portSource: PortSource,
directory?: string
) => {
const isTurbopack = Boolean(
options.turbo || options.turbopack || process.env.IS_TURBOPACK_TEST
)
if (isTurbopack) {
process.env.TURBOPACK = '1'
}
isTurboSession = isTurbopack
bundler = parseBundlerArgs(options)
dir = getProjectDir(process.env.NEXT_PRIVATE_DEV_DIR || directory)
......@@ -288,7 +289,9 @@ const nextDev = async (
stdio: 'inherit',
env: {
...defaultEnv,
...(isTurbopack ? { TURBOPACK: '1' } : undefined),
...(bundler === Bundler.Turbopack
? { TURBOPACK: process.env.TURBOPACK }
: undefined),
NEXT_PRIVATE_WORKER: '1',
NEXT_PRIVATE_TRACE_ID: traceId,
NODE_EXTRA_CA_CERTS: startServerOptions.selfSignedCertificate
......@@ -336,12 +339,13 @@ const nextDev = async (
(await loadConfig(PHASE_DEVELOPMENT_SERVER, dir, {
silent: true,
}))
bundler = finalizeBundlerFromConfig(bundler)
uploadTrace({
traceUploadUrl,
mode: 'dev',
projectDir: dir,
distDir: config.distDir,
isTurboSession,
isTurboSession: bundler === Bundler.Turbopack,
sync: true,
})
}
......
/// Utilties for configuring the bundler to use.
export enum Bundler {
Turbopack,
Webpack,
Rspack,
}
/**
* Parse the bundler arguments and potentially sets the `TURBOPACK` environment variable.
*
* NOTE: rspack is configured via next config which is chaotic so it is possible for this to be overridden later.
*
* @param options The options to parse.
* @returns The bundler that was configured
*/
export function parseBundlerArgs(options: {
turbo?: boolean
turbopack?: boolean
webpack?: boolean
}): Bundler {
const bundlerFlags = new Map<Bundler, string[]>()
const setBundlerFlag = (bundler: Bundler, flag: string) => {
bundlerFlags.set(bundler, (bundlerFlags.get(bundler) ?? []).concat(flag))
}
// What turbo flag was set? We allow multiple to be set, which is silly but not ambiguous, just pick the most relevant one.
if (options.turbopack) {
setBundlerFlag(Bundler.Turbopack, '--turbopack')
}
if (options.turbo) {
setBundlerFlag(Bundler.Turbopack, '--turbo')
} else if (process.env.TURBOPACK) {
// We don't really want to support this but it is trivial and not really confusing.
// If we don't support it and someone sets it, we would have inconsistent behavior
// since some parts of next would read the return value of this function and other
// parts will read the env variable.
setBundlerFlag(Bundler.Turbopack, 'TURBOPACK')
} else if (process.env.IS_TURBOPACK_TEST) {
setBundlerFlag(Bundler.Turbopack, 'IS_TURBOPACK_TEST')
}
if (options.webpack) {
setBundlerFlag(Bundler.Webpack, '--webpack')
}
if (process.env.IS_WEBPACK_TEST) {
setBundlerFlag(Bundler.Webpack, 'IS_WEBPACK_TEST')
}
// Mostly this is set via the NextConfig but it can also be set via the command line which is
// common for testing.
if (process.env.NEXT_RSPACK) {
setBundlerFlag(Bundler.Rspack, 'NEXT_RSPACK')
}
if (process.env.IS_RSPACK_TEST) {
setBundlerFlag(Bundler.Rspack, 'NEXT_TEST_USE_RSPACK')
}
if (bundlerFlags.size > 1) {
console.error(
`Multiple bundler flags set: ${Array.from(bundlerFlags.values()).flat().join(', ')}. Configure exactly one bundler.`
)
process.exit(1)
}
// The default is turbopack when nothing is configured.
if (bundlerFlags.size === 0) {
process.env.TURBOPACK = 'auto'
return Bundler.Turbopack
}
if (bundlerFlags.has(Bundler.Turbopack)) {
// Only conditionally assign to the environment variable, preserving already set values.
// If it was set to 'auto' because no flag was set and this function is called a second time we
// would upgrade to '1' but we don't really want that.
process.env.TURBOPACK ??= '1'
return Bundler.Turbopack
}
// Otherwise it is one of rspack or webpack. At this point there must be exactly one key in the map.
return bundlerFlags.keys().next().value!
}
/**
* Finalize the bundler based on the config.
*
* Rspack is configured via next config by setting an environment variable (yay, side effects)
* so this should only be called after parsing the config.
*/
export function finalizeBundlerFromConfig(fromOptions: Bundler) {
// Reading the next config can set NEXT_RSPACK environment variables.
if (process.env.NEXT_RSPACK) {
return Bundler.Rspack
}
return fromOptions
}
......@@ -47,11 +47,9 @@ const unsupportedTurbopackNextConfigOptions = [
]
// The following will need to be supported by `next build --turbopack`
const unsupportedProductionSpecificTurbopackNextConfigOptions: string[] = [
// TODO: Support disabling sourcemaps, currently they're always enabled.
// 'productionBrowserSourceMaps',
]
const unsupportedProductionSpecificTurbopackNextConfigOptions: string[] = []
/** */
export async function validateTurboNextConfig({
dir,
isDev,
......@@ -71,7 +69,7 @@ export async function validateTurboNextConfig({
let hasWebpackConfig = false
let hasTurboConfig = false
let unsupportedConfig: string[] = []
const unsupportedConfig: string[] = []
let rawNextConfig: NextConfig = {}
const phase = isDev ? PHASE_DEVELOPMENT_SERVER : PHASE_PRODUCTION_BUILD
......@@ -124,7 +122,7 @@ export async function validateTurboNextConfig({
const customKeys = flattenKeys(rawNextConfig)
let unsupportedKeys = isDev
const unsupportedKeys = isDev
? unsupportedTurbopackNextConfigOptions
: [
...unsupportedTurbopackNextConfigOptions,
......@@ -139,7 +137,7 @@ export async function validateTurboNextConfig({
hasTurboConfig = true
}
let isUnsupported =
const isUnsupported =
unsupportedKeys.some(
(unsupportedKey) =>
// Either the key matches (or is a more specific subkey) of
......@@ -162,13 +160,26 @@ export async function validateTurboNextConfig({
Log.error('Unexpected error occurred while checking config', e)
}
if (hasWebpackConfig && !hasTurboConfig) {
Log.warn(
`Webpack is configured while Turbopack is not, which may cause problems.`
// If the build was defaulted to Turbopack, we want to warn about possibly ignored webpack
// configuration. Otherwise the user explicitly picked turbopack and thus we expect that
// they have configured it correctly.
if (process.env.TURBOPACK === 'auto' && hasWebpackConfig && !hasTurboConfig) {
const logMethod = isDev ? Log.warn : Log.error
// In a production build with auto-detected Turbopack, we want to fail the build.
logMethod(
`Webpack is configured while Turbopack is not. This may be a mistake.`
)
Log.warn(
`See instructions if you need to configure Turbopack:\n https://nextjs.org/docs/app/api-reference/next-config-js/turbopack\n`
logMethod(
`To configure Turbopack, see:\n https://nextjs.org/docs/app/api-reference/next-config-js/turbopack`
)
logMethod(
`TIP: Silence this ${isDev ? 'warning' : 'error'} by passing the --turbopack or --webpack flag explicitly.`
)
// For production builds we want to simply fail to prevent accidental misconfiguration.
if (!isDev) {
process.exit(1)
}
}
if (unsupportedConfig.length) {
......
......@@ -153,6 +153,7 @@ async function main() {
NEXT_EXTERNAL_TESTS_FILTERS,
NEXT_TEST_MODE: testMode,
NEXT_TEST_VERSION: nextTestVersion,
IS_WEBPACK_TEST: '1',
},
})
}
......@@ -170,6 +171,7 @@ async function main() {
NEXT_TEST_VERSION: nextTestVersion,
IS_TURBOPACK_TEST: '1',
TURBOPACK_BUILD: testMode === 'start' ? '1' : undefined,
TURBOPACK_DEV: testMode === 'dev' ? '1' : undefined,
},
})
}
......
import { nextTestSetup } from 'e2e-utils'
describe('next-config-ts - turbopack', () => {
const { next } = nextTestSetup({
files: __dirname,
// explicitly ensure that turbopack is used
startCommand: 'pnpm next dev --turbo',
})
it('should work with Turbopack', async () => {
const $ = await next.render$('/')
expect($('p').text()).toBe('foo')
})
})
;(process.env.IS_TURBOPACK_TEST ? describe : describe.skip)(
'next-config-ts - turbopack',
() => {
const { next } = nextTestSetup({
files: __dirname,
// explicitly ensure that turbopack is used
startCommand: 'pnpm next dev --turbopack',
})
it('should work with Turbopack', async () => {
const $ = await next.render$('/')
expect($('p').text()).toBe('foo')
})
}
)
......@@ -5,40 +5,90 @@ const WARNING_MESSAGE = 'Webpack is configured while Turbopack is not'
const itif = (condition: boolean) => (condition ? it : it.skip)
const page = {
'app/page.js': `
export default function Page() {
return <p>hello world</p>
}
`,
}
const TURBOPACK_FLAG = process.env.TURBOPACK
describe('config-turbopack', () => {
describe('when webpack is configured but Turbopack is not', () => {
const { next, isTurbopack } = nextTestSetup({
files: {
'app/page.js': `
export default function Page() {
return <p>hello world</p>
}
`,
'next.config.js': `
describe('when turbopack is auto selected', () => {
if (process.env.IS_TURBOPACK_TEST) {
beforeAll(() => {
// This is hacky since it isn't a public API, but it's the only way to test this.
// If next started ignoring IS_TURBOPACK_TEST we could possibly change nextTestSetup to not set the turbopack flag.
process.env.TURBOPACK = 'auto'
})
afterAll(() => {
if (TURBOPACK_FLAG) {
process.env.TURBOPACK = TURBOPACK_FLAG
} else {
delete process.env.TURBOPACK
}
})
}
describe('when webpack is configured but Turbopack is not', () => {
const { next, isTurbopack, isNextDev, isNextStart } = nextTestSetup({
skipStart: Boolean(process.env.TURBOPACK_BUILD),
turbo: false,
files: {
...page,
'next.config.js': `
module.exports = {
webpack: (config) => {
return config
},
}
`,
},
})
},
})
itif(isTurbopack)('warns', async () => {
if (next) await next.render('/')
expect(next.cliOutput).toContain(WARNING_MESSAGE)
itif(isTurbopack && isNextDev)('warns', async () => {
if (next) await next.render('/')
expect(next.cliOutput).toContain(WARNING_MESSAGE)
})
itif(isTurbopack && isNextStart)('errors', async () => {
const { exitCode, cliOutput } = await next.build()
expect(exitCode).toBe(1)
expect(cliOutput).toContain(WARNING_MESSAGE)
})
})
// no warn cases work when auto selected too
noWarnCases()
})
describe('when webpack is configured and config.turbopack is set', () => {
const { next, isTurbopack } = nextTestSetup({
files: {
'app/page.js': `
export default function Page() {
return <p>hello world</p>
}
`,
'next.config.js': `
describe('when turbopack is explicitly configured', () => {
describe('when webpack is configured but Turbopack is not', () => {
const { next, isTurbopack } = nextTestSetup({
files: {
...page,
'next.config.js': `
module.exports = {
webpack: (config) => {
return config
},
}
`,
},
})
itif(isTurbopack)('does not warn', async () => {
if (next) await next.render('/')
expect(next.cliOutput).not.toContain(WARNING_MESSAGE)
})
})
noWarnCases()
})
/// These other cases don't warn because --turbopack is explicitly selected
function noWarnCases() {
describe('when webpack is configured and config.turbopack is set', () => {
const { next, isTurbopack } = nextTestSetup({
files: {
...page,
'next.config.js': `
module.exports = {
turbopack: {
rules: {
......@@ -52,24 +102,20 @@ describe('config-turbopack', () => {
},
}
`,
},
})
},
})
itif(isTurbopack)('does not warn', async () => {
if (next) await next.render('/')
expect(next.cliOutput).not.toContain(WARNING_MESSAGE)
itif(isTurbopack)('does not warn', async () => {
if (next) await next.render('/')
expect(next.cliOutput).not.toContain(WARNING_MESSAGE)
})
})
})
describe('when webpack is configured and config.experimental.turbo is set', () => {
const { next, isTurbopack } = nextTestSetup({
files: {
'app/page.js': `
export default function Page() {
return <p>hello world</p>
}
`,
'next.config.js': `
describe('when webpack is configured and config.experimental.turbo is set', () => {
const { next, isTurbopack } = nextTestSetup({
files: {
...page,
'next.config.js': `
module.exports = {
experimental: {
turbo: {
......@@ -85,12 +131,13 @@ describe('config-turbopack', () => {
},
}
`,
},
})
},
})
itif(isTurbopack)('does not warn', async () => {
if (next) await next.render('/')
expect(next.cliOutput).not.toContain(WARNING_MESSAGE)
itif(isTurbopack)('does not warn', async () => {
if (next) await next.render('/')
expect(next.cliOutput).not.toContain(WARNING_MESSAGE)
})
})
})
}
})
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment