Unverified Commit c11c0942 authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

[test] Run deploy tests against local JavaScript changes by default (#98353)

Deploy tests now use the checkout's built JavaScript packages when
`NEXT_TEST_VERSION` is unset. The harness reuses the isolated-test
packer, includes the tarballs in the deployment source, and installs
them through relative `file:` references.

The previous published-build workflow remains available through
`NEXT_TEST_VERSION`. For example, `NEXT_TEST_VERSION=canary` skips local
packing and tests the published canary. Exact versions and CI preview
tarball URLs remain supported. Existing CI deploy jobs already set
`NEXT_TEST_VERSION`, so their package selection does not change. The
harness skips local package preparation when it reuses an existing
deployment or invokes a custom deploy script, preserving those
workflows.

Staged monorepo peer dependencies use matching local versions so npm
accepts prerelease builds without conflicting direct-package overrides.
Fixture overrides retain their nested rules, including qualified
identity overrides with or without an explicit `.` entry. Configuration
updates preserve linked source files.

Deployments of local JavaScript packages use published SWC binaries
matching the local Next.js version. They do not include locally built
native binaries because those may target a different platform.

**Verification**

```
VERCEL_TURBOPACK_TEST_TEAM=vtest314-next-turbo-e2e-tests \
  pnpm test-deploy-turbo test/e2e/app-dir/third-parties/basic.test.ts
```

```
VERCEL_TEST_TEAM=vtest314-next-turbo-e2e-tests \
  pnpm test-deploy-webpack test/e2e/app-dir/third-parties/basic.test.ts
```

Both deploy modes passed. Additional temporary checks, removed
afterward, verified that:

- Deployments install the local JavaScript packages and use matching
local versions for monorepo peers.
- Qualified npm overrides preserve a nested `sharp@0.34.5` pin, both
with and without an explicit `.` entry. The check resolved `sharp` from
Next's own dependency context.
- Archive preparation leaves checkout manifests and tarballs unchanged.
Repacked archives preserve other package files, permissions, links, and
external peer declarations.
parent 662b0355
...@@ -152,25 +152,39 @@ and point it at your branch (this requires passing a custom tarball). ...@@ -152,25 +152,39 @@ and point it at your branch (this requires passing a custom tarball).
#### Running Deploy Tests Locally #### Running Deploy Tests Locally
You can run deploy tests locally against a specific commit using the `NEXT_TEST_VERSION` environment variable: **Local Changes**
By default, deploy tests use your checkout's JavaScript packages. Build the package outputs first, or keep the watch build running:
```sh ```sh
NEXT_TEST_VERSION=https://vercel-packages.vercel.app/next/commits/<commitSha>/next pnpm test-deploy <path-to-test> pnpm build
pnpm test-deploy-turbo test/e2e/app-dir/actions/
``` ```
For example, to test against commit `abc123`: The harness packs the outputs and includes the tarballs in the deployment source, with relative `file:` dependency references. The remote build uses published SWC binaries matching the local Next.js version. Locally built Rust binaries are not included.
**Version Overrides**
Set the `NEXT_TEST_VERSION` environment variable to use a published version, a dist-tag such as `canary`, or a CI preview tarball URL instead of local packages. This skips local packing:
```sh ```sh
NEXT_TEST_VERSION=https://vercel-packages.vercel.app/next/commits/abc123/next pnpm test-deploy test/e2e/app-dir/actions/ NEXT_TEST_VERSION=canary pnpm test-deploy-turbo test/e2e/app-dir/actions/
``` ```
This downloads a pre-built Next.js tarball from the specified commit and runs the deploy tests against it. For a specific commit, use its CI-published tarball:
```sh
NEXT_TEST_VERSION="https://vercel-packages.vercel.app/next/commits/<commitSha>/next" pnpm test-deploy-turbo test/e2e/app-dir/actions/
```
To test Rust changes, use a CI preview that includes the corresponding SWC binaries.
**Existing Deployments**
If you already have a deployment URL and want to skip the Vercel deploy step, Set `NEXT_TEST_DEPLOY_URL` to run against an existing deployment instead of creating one. This skips both local packing and deployment:
use `NEXT_TEST_DEPLOY_URL` instead:
```sh ```sh
NEXT_TEST_DEPLOY_URL=https://your-deployment.vercel.app pnpm test-deploy test/e2e/app-dir/actions/ NEXT_TEST_DEPLOY_URL=https://your-deployment.vercel.app pnpm test-deploy-turbo test/e2e/app-dir/actions/
``` ```
## Integration testing outside the repository with local builds ## Integration testing outside the repository with local builds
......
...@@ -140,6 +140,35 @@ async function applyWorkspaceOverrides(installDir, isolationRoot, overrides) { ...@@ -140,6 +140,35 @@ async function applyWorkspaceOverrides(installDir, isolationRoot, overrides) {
await fs.writeFile(workspaceFile, yaml.dump(workspaceConfig)) await fs.writeFile(workspaceFile, yaml.dump(workspaceConfig))
} }
/**
* @param {import('next/dist/trace').Span} parentSpan
* @returns {Promise<Map<string, string>>}
*/
async function packPackages(parentSpan) {
const repositoryDirectory = path.join(__dirname, '../..')
await parentSpan.traceChild('turbo-run-pack').traceAsyncFn(() =>
execa(
'pnpm',
[
'turbo',
'run',
'pack-for-isolated-tests',
'--output-logs',
'new-only',
'--ui',
'stream',
],
{
cwd: repositoryDirectory,
stdio: ['ignore', 'inherit', 'inherit'],
}
)
)
return parentSpan
.traceChild('linkPackages')
.traceAsyncFn(() => linkPackages({ repoDir: repositoryDirectory }))
}
/** /**
* *
* @param {object} param0 * @param {object} param0
...@@ -182,26 +211,7 @@ async function createNextInstall({ ...@@ -182,26 +211,7 @@ async function createNextInstall({
pkgPaths = new Map(JSON.parse(pkgPathsEnv)) pkgPaths = new Map(JSON.parse(pkgPathsEnv))
require('console').log('using provided pkg paths') require('console').log('using provided pkg paths')
} else { } else {
await rootSpan.traceChild('turbo-run-pack').traceAsyncFn(() => pkgPaths = await packPackages(rootSpan)
execa(
'pnpm',
[
'turbo',
'run',
'pack-for-isolated-tests',
'--output-logs',
'new-only',
// Jest tui can't handle Turborepo tui. But we're cutting off stdin
// so Turborepo's tui isn't interactive anyway.
'--ui',
'stream',
],
{
cwd: origRepoDir,
stdio: ['ignore', 'inherit', 'inherit'],
}
)
)
if (process.env.NEXT_TEST_WASM) { if (process.env.NEXT_TEST_WASM) {
const wasmPath = path.join(origRepoDir, 'crates', 'wasm', 'pkg') const wasmPath = path.join(origRepoDir, 'crates', 'wasm', 'pkg')
...@@ -235,12 +245,6 @@ async function createNextInstall({ ...@@ -235,12 +245,6 @@ async function createNextInstall({
swcNativeDirectory: process.env.NEXT_TEST_NATIVE_DIR, swcNativeDirectory: process.env.NEXT_TEST_NATIVE_DIR,
swcWasmDirectory: process.env.NEXT_TEST_WASM_DIR, swcWasmDirectory: process.env.NEXT_TEST_WASM_DIR,
}) })
pkgPaths = await rootSpan.traceChild('linkPackages').traceAsyncFn(() =>
linkPackages({
repoDir: origRepoDir,
})
)
} }
const combinedDependencies = { const combinedDependencies = {
...@@ -397,4 +401,5 @@ async function createNextInstall({ ...@@ -397,4 +401,5 @@ async function createNextInstall({
module.exports = { module.exports = {
createNextInstall, createNextInstall,
getPkgPaths: linkPackages, getPkgPaths: linkPackages,
packPackages,
} }
...@@ -3,12 +3,15 @@ import path from 'path' ...@@ -3,12 +3,15 @@ import path from 'path'
import dns from 'dns' import dns from 'dns'
import execa from 'execa' import execa from 'execa'
import fs from 'fs-extra' import fs from 'fs-extra'
import { load, dump } from 'js-yaml'
import * as tar from 'next/dist/compiled/tar'
import { NextInstance, type NextInstanceOpts } from './base' import { NextInstance, type NextInstanceOpts } from './base'
import * as projectEnv from '../../../scripts/reset-project.mjs' import * as projectEnv from '../../../scripts/reset-project.mjs'
import { Span } from 'next/dist/trace' import { Span } from 'next/dist/trace'
import { setTimeout } from 'timers/promises' import { setTimeout } from 'timers/promises'
import { FileRef } from '../e2e-utils' import { FileRef } from '../e2e-utils'
import { PROXY_HOST_MAP_ENV_KEY } from '../browsers/launch' import { PROXY_HOST_MAP_ENV_KEY } from '../browsers/launch'
import { packPackages } from '../create-next-install'
export class NextDeployInstance extends NextInstance { export class NextDeployInstance extends NextInstance {
private _cliOutput: string private _cliOutput: string
...@@ -327,6 +330,10 @@ export class NextDeployInstance extends NextInstance { ...@@ -327,6 +330,10 @@ export class NextDeployInstance extends NextInstance {
return return
} }
if (!process.env.NEXT_TEST_VERSION) {
await this.prepareLocalPackages(parentSpan)
}
// Original Vercel CLI deployment logic // Original Vercel CLI deployment logic
// ensure Vercel CLI is installed // ensure Vercel CLI is installed
try { try {
...@@ -531,6 +538,252 @@ export class NextDeployInstance extends NextInstance { ...@@ -531,6 +538,252 @@ export class NextDeployInstance extends NextInstance {
this.parseIdsFromCliOutput() this.parseIdsFromCliOutput()
} }
private async writeFixtureConfiguration(
filePath: string,
contents: string
): Promise<void> {
const temporaryDirectory = await fs.mkdtemp(
path.join(path.dirname(filePath), '.next-test-config-')
)
const temporaryPath = path.join(temporaryDirectory, path.basename(filePath))
try {
// Preserve permissions and replace links rather than their targets.
await fs.copyFile(filePath, temporaryPath, fs.constants.COPYFILE_EXCL)
await fs.writeFile(temporaryPath, contents)
await fs.rename(temporaryPath, filePath)
} finally {
await fs.remove(temporaryDirectory)
}
}
/**
* This method stages local JavaScript tarballs and rewrites the fixture's
* dependencies and overrides to relative `file:` references. It adds
* published SWC dependencies because locally built native binaries may target
* a different platform than the remote build.
*/
private async prepareLocalPackages(parentSpan: Span): Promise<void> {
const packagePaths = process.env.NEXT_TEST_PKG_PATHS
? new Map<string, string>(JSON.parse(process.env.NEXT_TEST_PKG_PATHS))
: await packPackages(parentSpan)
for (const name of ['next', '@next/env']) {
if (!packagePaths.has(name)) {
throw new Error(`Missing packed package for local deployment: ${name}`)
}
}
const directoryName = 'next-test-packages'
await fs.mkdir(path.join(this.testDir, directoryName))
const localPackages = new Map<string, string>()
for (const [name, source] of packagePaths) {
const relativePath = path.posix.join(directoryName, name, 'packed.tgz')
if (
!relativePath.startsWith(`${directoryName}/`) ||
name.includes('\\')
) {
throw new Error(`Invalid packed package name: ${name}`)
}
const destination = path.join(this.testDir, relativePath)
await fs.ensureDir(path.dirname(destination))
await fs.copyFile(source, destination, fs.constants.COPYFILE_EXCL)
// The staged packages use exact local peer versions so npm accepts
// prereleases without extra peer overrides.
const {
peerDependencies,
}: { peerDependencies?: Record<string, string> } = await fs.readJSON(
path.join(path.dirname(source), 'package.json')
)
const peerVersions = new Map<string, string>()
for (const peerName of Object.keys(peerDependencies ?? {})) {
const peerSource = packagePaths.get(peerName)
if (peerSource === undefined) {
continue
}
const { version } = await fs.readJSON(
path.join(path.dirname(peerSource), 'package.json')
)
if (typeof version !== 'string' || version.length === 0) {
throw new Error(
`Missing version for packed peer dependency: ${peerName}`
)
}
peerVersions.set(peerName, version)
}
if (peerVersions.size > 0) {
const temporaryDirectory = await fs.mkdtemp(
path.join(this.testDir, '.next-test-package-')
)
try {
await tar.x({ file: destination, cwd: temporaryDirectory })
const manifestPath = path.join(
temporaryDirectory,
'package/package.json'
)
const manifest = await fs.readJSON(manifestPath)
for (const [peerName, version] of peerVersions) {
if (manifest.peerDependencies?.[peerName] !== undefined) {
manifest.peerDependencies[peerName] = version
}
}
await fs.writeFile(
manifestPath,
JSON.stringify(manifest, null, 2) + '\n'
)
await tar.c(
{ file: destination, cwd: temporaryDirectory, gzip: true },
['package']
)
} finally {
await fs.remove(temporaryDirectory)
}
}
localPackages.set(name, `file:./${relativePath}`)
}
const packageJsonPath = path.join(this.testDir, 'package.json')
const packageJson: {
dependencies?: Record<string, string>
devDependencies?: Record<string, string>
optionalDependencies?: Record<string, string>
overrides?: Record<string, unknown>
resolutions?: Record<string, string>
pnpm?: { overrides?: Record<string, string> }
} = await fs.readJSON(packageJsonPath)
const dependencyFields = [
'dependencies',
'devDependencies',
'optionalDependencies',
] as const
for (const field of dependencyFields) {
const dependencies = packageJson[field]
if (dependencies === undefined) {
continue
}
for (const name of Object.keys(dependencies)) {
const localPackage = localPackages.get(name)
if (localPackage !== undefined) {
// npm requires identity overrides to match the new dependency.
if (packageJson.overrides !== undefined) {
for (const [key, override] of Object.entries(
packageJson.overrides
)) {
if (key !== name && !key.startsWith(`${name}@`)) {
continue
}
if (override === dependencies[name]) {
packageJson.overrides[key] = localPackage
} else if (
override !== null &&
typeof override === 'object' &&
('.' in override
? override['.'] === dependencies[name]
: key === `${name}@${dependencies[name]}`)
) {
Object.assign(override, { '.': localPackage })
}
}
}
dependencies[name] = localPackage
}
}
}
// Generated overrides follow fixture rules because npm uses the first
// matching rule.
packageJson.overrides ??= {}
const workspaceOverrides: Record<string, string> = {}
for (const [name, localPackage] of localPackages) {
if (
dependencyFields.some(
(field) => packageJson[field]?.[name] !== undefined
)
) {
continue
}
workspaceOverrides[name] = localPackage
if (packageJson.overrides[name] === undefined) {
packageJson.overrides[name] = localPackage
}
}
packageJson.resolutions = {
...workspaceOverrides,
...packageJson.resolutions,
}
packageJson.pnpm = {
...packageJson.pnpm,
overrides: {
...packageJson.resolutions,
...packageJson.pnpm?.overrides,
},
}
// Source tarballs omit the platform dependencies added during publication.
const version: string = require('next/package.json').version
const nativePackagesDirectory = path.join(
__dirname,
'../../../crates/next-napi-bindings/npm'
)
packageJson.optionalDependencies ??= {}
for (const platform of await fs.readdir(nativePackagesDirectory)) {
if (platform.startsWith('.')) {
continue
}
const { name }: { name: string } = await fs.readJSON(
path.join(nativePackagesDirectory, platform, 'package.json')
)
if (
packageJson.dependencies?.[name] === undefined &&
packageJson.devDependencies?.[name] === undefined &&
packageJson.optionalDependencies[name] === undefined
) {
packageJson.optionalDependencies[name] = version
}
}
await this.writeFixtureConfiguration(
packageJsonPath,
JSON.stringify(packageJson, null, 2) + '\n'
)
for (const filename of [
'pnpm-workspace.yaml',
'.vercelignore',
'.nowignore',
]) {
const filePath = path.join(this.testDir, filename)
try {
await fs.lstat(filePath)
} catch (error) {
if (error.code === 'ENOENT') {
continue
}
throw error
}
const contents = await fs.readFile(filePath, 'utf8')
if (filename === 'pnpm-workspace.yaml') {
const workspace = (load(contents) ?? {}) as {
overrides?: Record<string, string>
}
workspace.overrides = {
...packageJson.pnpm.overrides,
...workspace.overrides,
}
await this.writeFixtureConfiguration(filePath, dump(workspace))
} else if (contents !== '') {
const separator = contents.endsWith('\n') ? '' : '\n'
await this.writeFixtureConfiguration(
filePath,
`${contents}${separator}!/${directoryName}\n!/${directoryName}/**\n`
)
break
}
}
require('console').log(
`Deploying local JavaScript packages with published SWC ${version}. Local native binaries are not included.`
)
}
// When preview builds are private, the deploy build installs Next.js // When preview builds are private, the deploy build installs Next.js
// artifacts from an auth-protected route and needs credentials. The build // artifacts from an auth-protected route and needs credentials. The build
// authenticates with the Vercel OIDC token that Vercel automatically // authenticates with the Vercel OIDC token that Vercel automatically
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment