Unverified Commit e1138ce5 authored by Benjamin Woodruff ‮'s avatar Benjamin Woodruff Committed by GitHub

Turbopack: Add support for additionalRoots to deployment adapters (#99015)

Follow-up to #98003 so that this works with `vc deploy` (or anything
else using the adapters API, this isn't a breaking change).

Creates "synthetic" symlinks when it needs to rewrite a symlink target
(for cross-root links).

Potential follow-up: We can potentially extend the adapter API to let us
pass the full symlink information to adapters that support it, so that
we don't have to create a bunch of synthetic symlink files on disk.
parent 4701bf13
...@@ -92,9 +92,9 @@ Every name in a pragma must be declared in `test/lib/gate/conditions.ts` ...@@ -92,9 +92,9 @@ Every name in a pragma must be declared in `test/lib/gate/conditions.ts`
(typos fail the suite at collection). Two tiers: (typos fail the suite at collection). Two tiers:
- **static** — the run's shape: `dev`, `start`, `deploy`, `mode`, `turbopack`, - **static** — the run's shape: `dev`, `start`, `deploy`, `mode`, `turbopack`,
`rspack`, `webpack`, `bundler`, `react18`, `wasm`, `ci`; specialized CI `rspack`, `webpack`, `bundler`, `react18`, `wasm`, `linux`, `macos`,
variants `adapter`, `standaloneOutput`, `turbopackDev`, and `turbopackBuild`; `windows`, `ci`; specialized CI variants `adapter`, `standaloneOutput`,
plus the always-false `FIXME`/`TODO`. `turbopackDev`, and `turbopackBuild`; plus the always-false `FIXME`/`TODO`.
`prod` and `prefetching` are semantic aliases for `!dev` — prefer the name `prod` and `prefetching` are semantic aliases for `!dev` — prefer the name
that states _why_ the suite cannot run. that states _why_ the suite cannot run.
- **lazy** — a predicate over the fixture's _resolved_ `next.config` - **lazy** — a predicate over the fixture's _resolved_ `next.config`
......
...@@ -112,15 +112,19 @@ module.exports = { ...@@ -112,15 +112,19 @@ module.exports = {
} }
``` ```
To resolve files from linked dependencies outside the project root (via `npm link`, `yarn link`, `pnpm link`, etc.), you must either configure the `turbopack.root` to the parent directory of both the project and the linked dependencies or configure an additional root. To resolve files from linked dependencies outside the project root (e.g. [`npm link`]), you must either configure the `turbopack.root` to the parent directory of both the project and the linked dependencies or configure an additional root.
Do not extend the root directory too broadly (e.g. to your home directory). Turbopack may watch all files inside of roots, and configuring too broad of a root directory can cause significant performance problems. Use this option to configure the root when working in a monorepo. Do not extend the root directory too broadly (e.g. to your home directory). Turbopack may watch all files inside of roots, and configuring too broad of a root directory can cause significant performance problems.
[`npm link`]: https://docs.npmjs.com/cli/commands/npm-link
### Additional roots (experimental) ### Additional roots (experimental)
`experimental.turbopackAdditionalRoots` allows Turbopack to follow symbolic links whose targets are outside `turbopack.root`. It is useful for linked packages that live outside the project. `experimental.turbopackAdditionalRoots` allows Turbopack to follow symbolic links whose targets are outside `turbopack.root`.
It is useful for linked packages that live outside the project (e.g. [`npm link`](https://docs.npmjs.com/cli/commands/npm-link) or a [global virtual store](https://pnpm.io/global-virtual-store)).
Additional roots are read-only. Additional roots can only be crossed at symlink boundaries. Direct relative imports that escape a root are unsupported. Additional roots can only be crossed at symlink boundaries. Direct relative imports that escape a root are unsupported.
```js filename="next.config.js" ```js filename="next.config.js"
const path = require('path') const path = require('path')
...@@ -136,7 +140,13 @@ module.exports = { ...@@ -136,7 +140,13 @@ module.exports = {
Each key identifies one root. `path` can be absolute or relative to the directory where you run Next.js. Each key identifies one root. `path` can be absolute or relative to the directory where you run Next.js.
If a root path is not found when `next build` or `next dev` starts, a warning will be issued, unless `ignoreIfMissing` is set to `true` (defaults to false). > **Good to know**:
>
> - The symbolic link and its target must both be available on the build machine before `next dev` or `next build` starts.
> - Deployment tools and CI systems may copy your repository to a separate build machine. These tools may be unaware of the additional roots, and they may fail to copy required files.
> - On Windows, builds [require Developer Mode or elevated permissions to create symbolic links](https://blogs.windows.com/windowsdeveloper/2016/12/02/symlinks-windows-10/). Creating output artifacts may require creating symbolic links. In some cases, Turbopack can fall back to [directory junctions](https://learn.microsoft.com/en-us/windows/win32/fileio/hard-links-and-junctions#junctions) when symlink creation is not permitted, but these junctions use absolute paths, so the resulting output may not be relocatable.
If a root path is not found when `next build` or `next dev` starts, a warning will be issued, unless `ignoreIfMissing` is set to `true` (defaults to `false`).
To ensure portability of your configuration across a variety of platforms, including Windows, a root name must meet all of these requirements: To ensure portability of your configuration across a variety of platforms, including Windows, a root name must meet all of these requirements:
...@@ -145,9 +155,9 @@ To ensure portability of your configuration across a variety of platforms, inclu ...@@ -145,9 +155,9 @@ To ensure portability of your configuration across a variety of platforms, inclu
- It is not a Windows device name. - It is not a Windows device name.
- It does not duplicate an earlier name under ASCII case-insensitive comparison. - It does not duplicate an earlier name under ASCII case-insensitive comparison.
To avoid ambiguous references to files, roots may not contain overlapping paths. For example, configuring both `foo/` and `foo/bar/` as roots would produce a warning, and the last-configured value will be ignored. To avoid ambiguous references to files, additional roots may not overlap `turbopack.root` or one another. This means an additional root cannot be inside `turbopack.root` or contain it. For example, configuring both `foo/` and `foo/bar/` as additional roots would produce a warning, and the overlapping additional root will be ignored.
Additional roots changes the [`trace format`](/docs/app/api-reference/config/next-config-js/output#turbopack-nft-extensions) and requires support from the [build adapter](/docs/app/api-reference/adapters). Additional roots change Next.js's [`trace format`](/docs/app/api-reference/config/next-config-js/output#turbopack-nft-extensions).
When using [`output: "standalone"`](/docs/app/api-reference/config/next-config-js/output#automatically-copying-traced-files), unbundled files stored in these additional roots will be copied to a subdirectory of the output directory containing the configured name of the root. When using [`output: "standalone"`](/docs/app/api-reference/config/next-config-js/output#automatically-copying-traced-files), unbundled files stored in these additional roots will be copied to a subdirectory of the output directory containing the configured name of the root.
......
...@@ -63,6 +63,10 @@ import { resolveCacheHandlerPathToFilesystem } from '../../lib/format-dynamic-im ...@@ -63,6 +63,10 @@ import { resolveCacheHandlerPathToFilesystem } from '../../lib/format-dynamic-im
import { InvariantError } from '../../shared/lib/invariant-error' import { InvariantError } from '../../shared/lib/invariant-error'
import type { __ApiPreviewProps } from '../../server/api-utils' import type { __ApiPreviewProps } from '../../server/api-utils'
import { mapNftFileEntries, type NftJson } from '../nft' import { mapNftFileEntries, type NftJson } from '../nft'
import {
createAdapterSyntheticSymlinkDirectory,
type SyntheticSymlinkManager,
} from './synthetic-symlinks'
interface SharedRouteFields { interface SharedRouteFields {
/** /**
...@@ -654,6 +658,7 @@ export async function handleBuildComplete({ ...@@ -654,6 +658,7 @@ export async function handleBuildComplete({
) as NextAdapter ) as NextAdapter
if (typeof adapterMod.onBuildComplete === 'function') { if (typeof adapterMod.onBuildComplete === 'function') {
const syntheticSymlinks = createAdapterSyntheticSymlinkDirectory(distDir)
const outputs: AdapterOutputs = { const outputs: AdapterOutputs = {
pages: [], pages: [],
pagesApi: [], pagesApi: [],
...@@ -725,6 +730,7 @@ export async function handleBuildComplete({ ...@@ -725,6 +730,7 @@ export async function handleBuildComplete({
bundler, bundler,
hasInstrumentationHook, hasInstrumentationHook,
config, config,
syntheticSymlinks,
}) })
async function handleTraceFiles( async function handleTraceFiles(
...@@ -737,7 +743,9 @@ export async function handleBuildComplete({ ...@@ -737,7 +743,9 @@ export async function handleBuildComplete({
assets, assets,
assetsHashes, assetsHashes,
repoRoot, repoRoot,
`${entryFilePath}.nft.json` `${entryFilePath}.nft.json`,
syntheticSymlinks,
config.outputHashSalt || ''
) )
Object.assign( Object.assign(
assets, assets,
...@@ -2490,6 +2498,7 @@ async function getSharedNodeAssets({ ...@@ -2490,6 +2498,7 @@ async function getSharedNodeAssets({
requiredServerFiles, requiredServerFiles,
hasInstrumentationHook, hasInstrumentationHook,
config, config,
syntheticSymlinks,
}: { }: {
dir: string dir: string
bundler: Bundler bundler: Bundler
...@@ -2499,6 +2508,7 @@ async function getSharedNodeAssets({ ...@@ -2499,6 +2508,7 @@ async function getSharedNodeAssets({
requiredServerFiles: string[] requiredServerFiles: string[]
hasInstrumentationHook: boolean hasInstrumentationHook: boolean
config: NextConfigComplete config: NextConfigComplete
syntheticSymlinks: SyntheticSymlinkManager
}) { }) {
const sharedNodeAssets: Record<string, string> = {} const sharedNodeAssets: Record<string, string> = {}
const sharedNodeAssetsHashes: Record<string, string> = {} const sharedNodeAssetsHashes: Record<string, string> = {}
...@@ -2699,7 +2709,9 @@ async function getSharedNodeAssets({ ...@@ -2699,7 +2709,9 @@ async function getSharedNodeAssets({
sharedNodeAssets, sharedNodeAssets,
sharedNodeAssetsHashes, sharedNodeAssetsHashes,
repoRoot, repoRoot,
path.join(distDir, 'server', 'instrumentation.js.nft.json') path.join(distDir, 'server', 'instrumentation.js.nft.json'),
syntheticSymlinks,
salt
) )
const fileOutputPath = path.relative( const fileOutputPath = path.relative(
...@@ -2764,38 +2776,61 @@ async function loadNFT( ...@@ -2764,38 +2776,61 @@ async function loadNFT(
assets: Record<string, string>, assets: Record<string, string>,
assetsHashes: Record<string, string>, assetsHashes: Record<string, string>,
repoRoot: string, repoRoot: string,
traceFilePath: string traceFilePath: string,
syntheticSymlinks: SyntheticSymlinkManager,
salt: string
): Promise<{ entryHash?: string }> { ): Promise<{ entryHash?: string }> {
const nft = JSON.parse(await fs.readFile(traceFilePath, 'utf8')) as NftJson const nft = JSON.parse(await fs.readFile(traceFilePath, 'utf8')) as NftJson
// This call site only records source locations and hashes, so it does not need
// the mapped symlink targets.
for (const entry of mapNftFileEntries(nft, traceFilePath, repoRoot)) { for (const entry of mapNftFileEntries(nft, traceFilePath, repoRoot)) {
assets[entry.destination] = entry.source let source = entry.source
if (entry.hash) { let hash = entry.hash
assetsHashes[entry.destination] = entry.hash
if (entry.symlinkCrossesRoot) {
if (entry.symlinkTarget === undefined) {
throw new InvariantError(
`Expected cross-root symlink ${JSON.stringify(entry.destination)} to have a target`
)
}
const linkTarget =
path.relative(path.dirname(entry.destination), entry.symlinkTarget) ||
'.'
hash = hashLinkTarget(salt, linkTarget)
source = syntheticSymlinks.createLink(entry.source, linkTarget, hash)
}
assets[entry.destination] = source
if (hash) {
assetsHashes[entry.destination] = hash
} }
} }
return { entryHash: nft.entryHash } return { entryHash: nft.entryHash }
} }
async function hashFile(salt: string, filePath: string): Promise<string> { async function hashFile(salt: string, filePath: string): Promise<string> {
const hash = crypto.createHash('sha256')
hash.update(salt)
try { try {
// Try symlink first, since readFile just transparently resolves those (or fails if it's a // Try symlink first, since readFile just transparently resolves those (or fails if it's a
// directory symlink). // directory symlink).
const linkTarget = await fs.readlink(filePath) const linkTarget = await fs.readlink(filePath)
hash.update('link') return hashLinkTarget(salt, linkTarget)
hash.update(linkTarget)
} catch (e: any) { } catch (e: any) {
if (e.code === 'EINVAL') { if (e.code === 'EINVAL') {
// Not a symlink // Not a symlink
const hash = crypto.createHash('sha256')
hash.update(salt)
hash.update('file:') hash.update('file:')
hash.update(await fs.readFile(filePath)) hash.update(await fs.readFile(filePath))
return hash.digest('hex')
} else { } else {
throw e throw e
} }
} }
}
function hashLinkTarget(salt: string, linkTarget: string): string {
const hash = crypto.createHash('sha256')
hash.update(salt)
hash.update('link')
hash.update(linkTarget)
return hash.digest('hex') return hash.digest('hex')
} }
import fs from 'fs/promises'
import os from 'os'
import path from 'path'
import { createAdapterSyntheticSymlinkDirectory } from './synthetic-symlinks'
describe('adapter synthetic symlinks', () => {
// @force-gate !windows
describe('SyntheticSymlinkManager', () => {
let testDirectory: string
beforeEach(async () => {
testDirectory = await fs.mkdtemp(
path.join(os.tmpdir(), 'next-adapter-symlinks-')
)
})
afterEach(async () => {
await fs.rm(testDirectory, {
recursive: true,
force: true,
maxRetries: 3,
})
})
it('cleans stale staging and reuses the staged path', async () => {
const distDir = path.join(testDirectory, '.next')
const stagingRoot = path.join(distDir, 'adapter', 'synthetic_symlinks')
const staleFile = path.join(stagingRoot, 'stale')
await fs.mkdir(stagingRoot, { recursive: true })
await fs.writeFile(staleFile, 'stale')
const manager = createAdapterSyntheticSymlinkDirectory(distDir)
await expect(fs.access(staleFile)).rejects.toMatchObject({
code: 'ENOENT',
})
const source = path.join(testDirectory, 'source-link')
const linkTarget = path.relative(
path.join('functions', 'app', 'node_modules'),
path.join('next_additional_roots', 'packages', 'pkg')
)
const targetHash = 'a'.repeat(64)
const first = manager.createLink(source, linkTarget, targetHash)
const second = manager.createLink(
path.join(testDirectory, 'equivalent-source-link'),
linkTarget,
targetHash
)
expect(path.dirname(first)).toBe(stagingRoot)
expect(path.basename(first)).toBe('a'.repeat(32))
expect(second).toBe(first)
expect(await fs.readlink(first)).toBe(linkTarget)
})
})
})
import fs from 'fs'
import path from 'path'
type SymlinkTargetType = 'file' | 'dir'
/**
* When passing cross-root symlinks from Turbopack's additional roots feature to
* the adapter, we may need to rewrite the symlink target's relative path.
*
* Adapters receive a map of `{"destination": "source"}` file paths, and they
* copy symlink paths verbatim.
*
* We must write a "synthetic" symlink at a source path (inside
* `.next/adapter/synthetic_symlinks`) for the adapter to copy to its output
* artifact (e.g. a Lambda zip file).
*
* As a future optimization, we could pass symlink information directly to the
* adapter, if the adapter signals that it supports accepting that information.
* That would avoid a lot of small filesystem operations.
*/
export class SyntheticSymlinkManager {
private readonly stagedLinkNames = new Set<string>()
constructor(private readonly stagingRoot: string) {}
createLink(source: string, linkTarget: string, targetHash: string): string {
let targetType: SymlinkTargetType = 'file'
// Keep 128 bits of entropy while shortening the path to avoid Windows'
// 260-character path limit.
let stagedName = targetHash.slice(0, 32)
if (process.platform === 'win32') {
try {
targetType = fs.statSync(source).isDirectory() ? 'dir' : 'file'
} catch (error) {
const code = (error as NodeJS.ErrnoException).code
// We cannot determine the target type, so just create a file symlink
// ENOENT: Dangling link, preserve the dangling link as a file link
// ELOOP: Unresolvable link cycle, preserve any part of the cycle that
// was traced
if (code !== 'ENOENT' && code !== 'ELOOP') {
throw error
}
}
stagedName += `_${targetType}`
}
const stagedPath = path.join(this.stagingRoot, stagedName)
if (!this.stagedLinkNames.has(stagedName)) {
try {
fs.symlinkSync(linkTarget, stagedPath, targetType)
} catch (error) {
// This link may exist if `rmSync` (with `force: true`) failed to delete
// some files (can happen on Windows), but it's content-addressed, so
// we can safely ignore EEXIST.
if ((error as NodeJS.ErrnoException).code !== 'EEXIST') {
throw error
}
}
this.stagedLinkNames.add(stagedName)
}
return stagedPath
}
}
export function createAdapterSyntheticSymlinkDirectory(
distDir: string
): SyntheticSymlinkManager {
const stagingRoot = path.join(distDir, 'adapter', 'synthetic_symlinks')
fs.rmSync(stagingRoot, { recursive: true, force: true, maxRetries: 3 })
fs.mkdirSync(stagingRoot, { recursive: true })
return new SyntheticSymlinkManager(stagingRoot)
}
import path from 'path'
import { mapNftFileEntries, type NftJson } from './nft'
describe('mapNftFileEntries', () => {
it('marks only symlinks whose tuple specifies another root', () => {
const traceFilePath = path.join(
path.parse(process.cwd()).root,
'repo',
'.next',
'server',
'page.js.nft.json'
)
const repoRoot = path.join(path.parse(traceFilePath).root, 'repo')
const nft: NftJson = {
version: 1,
files: ['same-root-link', 'root-zero-link', 'base-root-link'],
symlinks: [
[0, 'same-root-target'],
[1, 'root-zero-target', 0],
[2, 'base-root-target', -1],
],
additionalRoots: [
{
name: 'root-zero',
path: '../../../external',
files: [],
symlinks: [],
},
],
}
const [sameRoot, rootZero, baseRoot] = mapNftFileEntries(
nft,
traceFilePath,
repoRoot
)
expect(sameRoot).toEqual(
expect.objectContaining({
symlinkTarget: path.join('.next', 'server', 'same-root-target'),
})
)
expect(sameRoot.symlinkCrossesRoot).toBe(false)
expect(rootZero).toEqual(
expect.objectContaining({
symlinkTarget: path.join(
'next_additional_roots',
'root-zero',
'root-zero-target'
),
symlinkCrossesRoot: true,
})
)
expect(baseRoot).toEqual(
expect.objectContaining({
symlinkTarget: path.join('.next', 'server', 'base-root-target'),
symlinkCrossesRoot: true,
})
)
})
})
...@@ -118,6 +118,8 @@ export interface MappedNftFileEntry { ...@@ -118,6 +118,8 @@ export interface MappedNftFileEntry {
* `undefined` value here means that the file is not a symlink. * `undefined` value here means that the file is not a symlink.
*/ */
symlinkTarget?: string symlinkTarget?: string
/** Whether the symlink target is stored relative to a different NFT root. */
symlinkCrossesRoot: boolean
} }
function invalid(message: string): never { function invalid(message: string): never {
...@@ -229,8 +231,10 @@ export function mapNftFileEntries( ...@@ -229,8 +231,10 @@ export function mapNftFileEntries(
} }
let symlinkTarget: string | undefined let symlinkTarget: string | undefined
let symlinkCrossesRoot = false
if (symlink !== undefined) { if (symlink !== undefined) {
const [, target, rootIndex] = symlink const [, target, rootIndex] = symlink
symlinkCrossesRoot = rootIndex !== undefined
const targetRootIndex = rootIndex ?? currentRootIndex const targetRootIndex = rootIndex ?? currentRootIndex
symlinkTarget = symlinkTarget =
targetRootIndex === -1 targetRootIndex === -1
...@@ -246,9 +250,11 @@ export function mapNftFileEntries( ...@@ -246,9 +250,11 @@ export function mapNftFileEntries(
} }
result.push({ result.push({
...mapped, source: mapped.source,
destination: mapped.destination,
hash: fileHashes?.[fileIndex], hash: fileHashes?.[fileIndex],
symlinkTarget, symlinkTarget,
symlinkCrossesRoot,
}) })
} }
} }
......
!fixtures/additional-root/node_modules
!fixtures/additional-root/node_modules/**
import fs from 'fs/promises'
import path from 'path'
/** @type {import('next').NextAdapter} */
const adapter = {
name: 'capture-additional-roots-adapter',
async onBuildComplete(context) {
await fs.writeFile(
path.join(context.projectDir, 'build-complete.json'),
JSON.stringify(context)
)
},
}
export default adapter
import linked from './linked'
export function register() {
if (!linked.value) {
throw new Error('Expected the linked package to be available')
}
}
const fs = require('node:fs')
const path = require('node:path')
const linkedPackage = fs.realpathSync(path.join(__dirname, 'linked'))
/** @type {import('next').NextConfig} */
const nextConfig = {
output: 'standalone',
outputHashSalt: 'adapter-symlink-test',
serverExternalPackages: ['sibling'],
experimental: {
turbopackAdditionalRoots: {
linkedPackages: {
path: path.resolve(linkedPackage, '../..'),
},
missingOptional: {
path: './missing-optional-root',
ignoreIfMissing: false,
},
},
},
}
if (
!process.env.NEXT_ADAPTER_PATH &&
process.env.NEXT_TEST_CAPTURE_ADAPTER === '1'
) {
nextConfig.adapterPath = require.resolve('./capture-adapter.mjs')
}
module.exports = nextConfig
import fs from 'node:fs/promises'
import os from 'node:os'
import path from 'node:path'
const projectRoot = process.cwd()
const source = path.join(projectRoot, 'fixtures/additional-root')
const temporaryDirectory = await fs.realpath(os.tmpdir())
const destination = await fs.mkdtemp(
path.join(temporaryDirectory, 'next-additional-root-')
)
const linkedPackage = path.join(destination, 'packages/linked')
const link = path.join(projectRoot, 'linked')
try {
await fs.cp(source, destination, { recursive: true })
await fs.rm(link, { recursive: true, force: true })
await fs.symlink(
process.platform === 'win32'
? linkedPackage
: path.relative(projectRoot, linkedPackage),
link,
'junction'
)
} catch (error) {
await fs.rm(destination, { recursive: true, force: true })
throw error
}
process.stdout.write(`${destination}\n`)
import { isNextStart, nextTestSetup } from 'e2e-utils' import crypto from 'crypto'
import execa from 'execa'
import { isNextDeploy, isNextStart, nextTestSetup } from 'e2e-utils'
import fs from 'fs-extra' import fs from 'fs-extra'
import os from 'os' import os from 'os'
import path from 'path' import path from 'path'
import type { NextAdapter } from 'next'
import { import {
fetchViaHTTP, fetchViaHTTP,
findPort, findPort,
...@@ -10,53 +13,48 @@ import { ...@@ -10,53 +13,48 @@ import {
retry, retry,
} from 'next-test-utils' } from 'next-test-utils'
// Deploy only uploads `project`, but this suite intentionally uses a sibling filesystem root. // Non-adapter deploys do not exercise Next.js' adapter asset contract.
// // @force-gate turbopack && (!deploy || adapter)
// @force-gate turbopack && !deploy
describe('turbopack additional roots', () => { describe('turbopack additional roots', () => {
const { next, isNextDev } = nextTestSetup({ const { next, isNextDev } = nextTestSetup({
files: __dirname, files: __dirname,
subDir: 'project', subDir: 'project',
nextConfig: { packageJson: {
output: 'standalone', scripts: {
serverExternalPackages: ['sibling'], prebuild: 'node prepare-deploy-additional-root.mjs',
experimental: {
turbopackAdditionalRoots: {
linkedPackages: { path: '../additional-root' },
missingOptional: {
path: './missing-optional-root',
ignoreIfMissing: false,
},
},
}, },
}, },
env: {
...(isNextStart && { NEXT_TEST_CAPTURE_ADAPTER: '1' }),
},
skipStart: true, skipStart: true,
}) })
let externalRoot: string let externalRoot: string | undefined
let linkedPackage: string
beforeAll(async () => { beforeAll(async () => {
externalRoot = path.resolve(next.testDir, '../additional-root') if (!isNextDeploy) {
linkedPackage = path.join(externalRoot, 'packages/linked') const result = await execa(
'node',
await fs.copy( ['prepare-deploy-additional-root.mjs'],
path.join(__dirname, 'fixtures/additional-root'), {
externalRoot cwd: next.testDir,
) }
)
await fs.symlink( externalRoot = result.stdout
linkedPackage, }
path.join(next.testDir, 'linked'),
'junction' // use a junction point on windows (this argument is ignored everywhere else)
)
await next.start() await next.start()
}) })
afterAll(async () => { afterAll(async () => {
await next.stop() try {
await fs.remove(externalRoot) await next.stop()
} finally {
if (!isNextDeploy && externalRoot) {
await fs.remove(externalRoot)
}
}
}) })
it('resolves a linked package, sibling dependency, and next/dist', async () => { it('resolves a linked package, sibling dependency, and next/dist', async () => {
...@@ -69,6 +67,7 @@ describe('turbopack additional roots', () => { ...@@ -69,6 +67,7 @@ describe('turbopack additional roots', () => {
it('reports initialization warnings when startup succeeds', () => { it('reports initialization warnings when startup succeeds', () => {
expect(next.cliOutput).toContain('Invalid Turbopack additional root') expect(next.cliOutput).toContain('Invalid Turbopack additional root')
expect(next.cliOutput).not.toContain('overlaps the project root')
}) })
if (isNextDev) { if (isNextDev) {
...@@ -76,7 +75,10 @@ describe('turbopack additional roots', () => { ...@@ -76,7 +75,10 @@ describe('turbopack additional roots', () => {
const browser = await next.browser('/') const browser = await next.browser('/')
await next.patchFile( await next.patchFile(
'../additional-root/packages/linked/index.js', path.relative(
next.testDir,
path.join(externalRoot!, 'packages/linked/index.js')
),
(content) => content.replace('linked-', 'updated-'), (content) => content.replace('linked-', 'updated-'),
async () => { async () => {
await retry(async () => { await retry(async () => {
...@@ -96,6 +98,9 @@ describe('turbopack additional roots', () => { ...@@ -96,6 +98,9 @@ describe('turbopack additional roots', () => {
'.next/server/app/page.js.nft.json' '.next/server/app/page.js.nft.json'
) )
const nft = await fs.readJson(nftPath) const nft = await fs.readJson(nftPath)
expect(
path.resolve(path.dirname(nftPath), nft.additionalRoots[0].path)
).toBe(externalRoot)
const crossRootSymlinks = nft.symlinks const crossRootSymlinks = nft.symlinks
.filter((symlink: [number, string, number?]) => symlink.length === 3) .filter((symlink: [number, string, number?]) => symlink.length === 3)
.map( .map(
...@@ -112,6 +117,7 @@ describe('turbopack additional roots', () => { ...@@ -112,6 +117,7 @@ describe('turbopack additional roots', () => {
const additionalRoots = nft.additionalRoots.map((root: any) => { const additionalRoots = nft.additionalRoots.map((root: any) => {
const copy = { ...root } const copy = { ...root }
delete copy.fileHashes delete copy.fileHashes
copy.path = '<temporary-root>'
return copy return copy
}) })
...@@ -132,15 +138,96 @@ describe('turbopack additional roots', () => { ...@@ -132,15 +138,96 @@ describe('turbopack additional roots', () => {
"node_modules/sibling/package.json", "node_modules/sibling/package.json",
], ],
"name": "linkedPackages", "name": "linkedPackages",
"path": "../../../../additional-root", "path": "<temporary-root>",
"symlinks": [], "symlinks": [],
}, },
] ]
`) `)
}) })
it('provides relocatable synthetic symlinks through adapter assets', async () => {
const buildComplete: Parameters<
NonNullable<NextAdapter['onBuildComplete']>
>[0] = await next.readJSON('build-complete.json')
const rootOutput = buildComplete.outputs.appPages.find(
(output) => output.pathname === '/'
)
expect(rootOutput).toBeDefined()
expect(rootOutput).not.toHaveProperty('assetSymlinks')
const crossRootAsset = Object.entries(rootOutput!.assets).find(
([destination]) => path.basename(destination).startsWith('sibling-')
)
expect(crossRootAsset).toBeDefined()
const [destination, source] = crossRootAsset!
const stagingRoot = path.join(
buildComplete.distDir,
'adapter',
'synthetic_symlinks'
)
expect(path.dirname(source)).toBe(stagingRoot)
expect(path.basename(source)).toMatch(/^[0-9a-f]{32}$/)
expect((await fs.lstat(source)).isSymbolicLink()).toBe(true)
const target = path.join(
'next_additional_roots',
'linkedPackages',
'node_modules',
'sibling'
)
const expectedPayload = path.relative(path.dirname(destination), target)
expect(await fs.readlink(source)).toBe(expectedPayload)
expect(
path.normalize(path.join(path.dirname(destination), expectedPayload))
).toBe(target)
const expectedHash = crypto
.createHash('sha256')
.update('adapter-symlink-test')
.update('link')
.update(expectedPayload)
.digest('hex')
expect(rootOutput!.assetsHashes[destination]).toBe(expectedHash)
const nft = await fs.readJson(
path.join(next.testDir, '.next/server/app/page.js.nft.json')
)
const [sourceFileIndex] = nft.symlinks.find(
(symlink: [number, string, number?]) => symlink.length === 3
)
expect(rootOutput!.assetsHashes[destination]).not.toBe(
nft.fileHashes[sourceFileIndex]
)
const instrumentationNft = await fs.readJson(
path.join(next.testDir, '.next/server/instrumentation.js.nft.json')
)
expect(
instrumentationNft.symlinks.some(
(symlink: [number, string, number?]) => symlink.length === 3
)
).toBe(true)
expect(await fs.readdir(stagingRoot)).toEqual([path.basename(source)])
const equivalentSources = buildComplete.outputs.appPages
.map((output) => output.assets[destination])
.filter(Boolean)
expect(equivalentSources.length).toBeGreaterThan(1)
expect(new Set(equivalentSources)).toEqual(new Set([source]))
})
it('runs after relocating standalone output away from the source root', async () => { it('runs after relocating standalone output away from the source root', async () => {
await next.stop() await next.stop()
const syntheticSymlinkRoot = path.join(
next.testDir,
'.next/adapter/synthetic_symlinks'
)
await fs.remove(syntheticSymlinkRoot)
delete next.env.NEXT_TEST_CAPTURE_ADAPTER
expect((await next.build()).exitCode).toBe(0)
expect(await fs.pathExists(syntheticSymlinkRoot)).toBe(false)
const temporaryDirectory = await fs.mkdtemp( const temporaryDirectory = await fs.mkdtemp(
path.join(os.tmpdir(), 'next-additional-roots-') path.join(os.tmpdir(), 'next-additional-roots-')
) )
......
...@@ -83,11 +83,11 @@ fails the whole suite at collection time rather than silently disabling the gate ...@@ -83,11 +83,11 @@ fails the whole suite at collection time rather than silently disabling the gate
There are two tiers: There are two tiers:
- **static** — the run's own shape (`dev`, `start`, `deploy`, `mode`, - **static** — the run's own shape (`dev`, `start`, `deploy`, `mode`,
`turbopack`, `rspack`, `webpack`, `bundler`, `react18`, `wasm`, `ci`), `turbopack`, `rspack`, `webpack`, `bundler`, `react18`, `wasm`, `linux`,
semantic aliases for `!dev` that state the reason rather than the mode `macos`, `windows`, `ci`), semantic aliases for `!dev` that state the reason
(`prod`, `prefetching`), specialized CI variants (`adapter`, rather than the mode (`prod`, `prefetching`), specialized CI variants
`standaloneOutput`, `turbopackDev`, `turbopackBuild`), plus `FIXME` / `TODO`, (`adapter`, `standaloneOutput`, `turbopackDev`, `turbopackBuild`), plus
which are always false. `FIXME` / `TODO`, which are always false.
- **lazy** — a predicate over the fixture's *resolved* `next.config` - **lazy** — a predicate over the fixture's *resolved* `next.config`
(`cacheComponents`, `ppr`, `prefetchInlining`, `output`, …), read the first (`cacheComponents`, `ppr`, `prefetchInlining`, `output`, …), read the first
time a gate asks for it. time a gate asks for it.
......
...@@ -13,8 +13,9 @@ ...@@ -13,8 +13,9 @@
* ## The two tiers * ## The two tiers
* *
* **`staticCondition`** — the value is known before any test runs (run mode, * **`staticCondition`** — the value is known before any test runs (run mode,
* bundler, React version). These are the only conditions `@force-gate` accepts, * bundler, operating system, React version). These are the only conditions
* because a real Jest skip has to be decided while tests are being collected. * `@force-gate` accepts, because a real Jest skip has to be decided while tests
* are being collected.
* *
* **`lazyCondition`** — a predicate over the *resolved* `next.config` of the * **`lazyCondition`** — a predicate over the *resolved* `next.config` of the
* fixture the suite booted (`NextInstance.getResolvedConfig()`). The value is * fixture the suite booted (`NextInstance.getResolvedConfig()`). The value is
...@@ -131,6 +132,18 @@ export const conditions: Record<string, Condition> = { ...@@ -131,6 +132,18 @@ export const conditions: Record<string, Condition> = {
'using the wasm SWC binary (`NEXT_TEST_WASM`)', 'using the wasm SWC binary (`NEXT_TEST_WASM`)',
() => getGateTestContext().wasm () => getGateTestContext().wasm
), ),
linux: staticCondition(
'running on Linux',
() => process.platform === 'linux'
),
macos: staticCondition(
'running on macOS',
() => process.platform === 'darwin'
),
windows: staticCondition(
'running on Windows',
() => process.platform === 'win32'
),
ci: staticCondition('running in CI (`NEXT_TEST_CI`)', () => ci: staticCondition('running in CI (`NEXT_TEST_CI`)', () =>
Boolean(process.env.NEXT_TEST_CI) Boolean(process.env.NEXT_TEST_CI)
), ),
......
...@@ -196,6 +196,12 @@ describe('@gate runtime', () => { ...@@ -196,6 +196,12 @@ describe('@gate runtime', () => {
expect(await gate((c) => c.mode === 'start' && c.webpack)).toBe(true) expect(await gate((c) => c.mode === 'start' && c.webpack)).toBe(true)
}) })
it('reports the host platform', async () => {
expect(await gate((c) => c.linux)).toBe(process.platform === 'linux')
expect(await gate((c) => c.macos)).toBe(process.platform === 'darwin')
expect(await gate((c) => c.windows)).toBe(process.platform === 'win32')
})
it('reads a lazy condition from the running fixture', async () => { it('reads a lazy condition from the running fixture', async () => {
const getResolvedConfig = fixtureWith({ cacheComponents: true }) const getResolvedConfig = fixtureWith({ cacheComponents: true })
expect(await gate((c) => c.cacheComponents)).toBe(true) expect(await gate((c) => c.cacheComponents)).toBe(true)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment