Unverified Commit 299c2f48 authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

[ci] Fix test axis fixtures never running their off state in CI (#100912)

parent fcd3fdeb
......@@ -144,7 +144,7 @@ start passing. Working example: `test/e2e/app-dir/concurrent-router-queue/`
pairs with a lazy `@force-gate` when the off state proves nothing —
`test/e2e/app-dir/use-offline/` — which skips the redundant axis run (build
included) instead of covering it. Axis `A` aliases `__NEXT_CACHE_COMPONENTS`
for now (see `scripts/run-jest.sh`) — fine, because these fixtures pin
for now (see `test/jest-global-setup.ts`) — fine, because these fixtures pin
`cacheComponents` explicitly, so that run's env default is a no-op for them.
**Keep exactly one flag varying per fixture.** A red shard must attribute to a
......
......@@ -8,6 +8,7 @@ const createJestConfig = nextJest()
const customJestConfig = {
displayName: process.env.IS_WEBPACK_TEST ? 'webpack' : 'Turbopack',
testMatch: ['**/*.test.js', '**/*.test.ts', '**/*.test.jsx', '**/*.test.tsx'],
globalSetup: '<rootDir>/jest-global-setup.ts',
setupFilesAfterEnv: ['<rootDir>/jest-setup-after-env.ts'],
verbose: true,
rootDir: 'test',
......
......@@ -56,35 +56,6 @@ while [ $# -gt 0 ]; do
shift
done
# `__NEXT_TEST_AXIS` names the alternate flag configurations of the test
# matrix. Axes are lettered (`A`, `B`, …) — a fixed enumeration a fixture opts
# into, not a boolean, and not one of the buckets test *sharding* splits a run
# into. CI runs the suites once plainly and once per axis, and a fixture keys
# an experimental flag on an axis to cover both states of the flag — enabled
# by default, disabled on that axis:
#
# experimental: {
# concurrentRouterQueue: process.env.__NEXT_TEST_AXIS !== 'A',
# }
#
# paired with a `// @gate concurrentRouterQueue` on the affected tests: a
# plain run — including a local run with no special env — exercises the
# feature, and the axis run covers the off state (see
# test/lib/gate/README.md).
#
# For now there is a single axis, `A`, and it is an alias for
# `__NEXT_CACHE_COMPONENTS` (the `--experimental` run) rather than a CI
# dimension of its own. That works because most experiments hard-code
# `cacheComponents: true` in their fixture anyway — the cache-components env
# default only applies to fixtures that don't set it themselves, so for these
# fixtures that run is free to double as the axis run. Setting either name
# implies the other.
if [ -n "${__NEXT_TEST_AXIS:-}" ]; then
export __NEXT_CACHE_COMPONENTS=true
elif [ "${__NEXT_CACHE_COMPONENTS:-}" = "true" ]; then
export __NEXT_TEST_AXIS=A
fi
# Resolves to `node_modules/.bin/jest` via `$PATH`. This relies on being
# invoked through pnpm (or another package runner), which prepends the
# workspace's `node_modules/.bin/` to `$PATH` before running the script.
......
export default function globalSetup() {
// `__NEXT_TEST_AXIS` names an alternate flag configuration of the test
// matrix. Axes are letters (`A`, `B`, …). A fixture can key an experimental
// flag on an axis to cover both states of the flag:
//
// ```js
// experimental: {
// concurrentRouterQueue: process.env.__NEXT_TEST_AXIS !== 'A',
// }
// ```
//
// A plain run enables the flag, and the axis run disables it. See
// `test/lib/gate/README.md`.
//
// The only axis today is `A`. It is an alias for the Cache Components run
// (`__NEXT_CACHE_COMPONENTS=true`), so CI has no separate job for it. Most
// fixtures that use the axis set `cacheComponents: true` themselves, so the
// env default of that run does not change them. Setting either variable sets
// the other.
if (process.env.__NEXT_TEST_AXIS) {
process.env.__NEXT_CACHE_COMPONENTS = 'true'
} else if (process.env.__NEXT_CACHE_COMPONENTS === 'true') {
process.env.__NEXT_TEST_AXIS = 'A'
}
}
......@@ -2,11 +2,16 @@ const deploymentTestFlags = [
'__NEXT_CACHE_COMPONENTS',
'__NEXT_PARTIAL_PREFETCHING',
'__NEXT_EXPERIMENTAL_CACHED_NAVIGATIONS',
'__NEXT_TEST_AXIS',
]
/**
* Capture test flags in the deployed config because Vercel environment variable
* names cannot start with an underscore. This runs in the local test harness.
*
* Callers insert the assignments before the config content. A fixture can read
* a flag while its config module loads, for example
* `process.env.__NEXT_TEST_AXIS !== 'A'`.
*/
export function getDeploymentTestEnvAssignments(): string {
let assignments = ''
......
......@@ -235,10 +235,10 @@ export class PatchedFileRef {
}
/**
* Return a PatchedFileRef that appends captured test env assignments when the
* harness copies the file in deploy mode. Other modes copy the content unchanged.
* Nested app configs need this because the deploy harness only patches the
* fixture-root config.
* Return a PatchedFileRef that prepends captured test env assignments when the
* harness copies the file in deploy mode. Other modes copy the content
* unchanged. Nested app configs need this because the deploy harness only
* patches the fixture-root config.
*
* TODO: Patch nested configs automatically in the deploy harness so fixtures no
* longer need these explicit overrides.
......@@ -248,7 +248,7 @@ export function patchFileWithDeployEnvAssignments(
path: string
): PatchedFileRef {
return new PatchedFileRef(path, (content) =>
isNextDeploy ? `${content}\n` + getDeploymentTestEnvAssignments() : content
isNextDeploy ? getDeploymentTestEnvAssignments() + content : content
)
}
......
......@@ -119,8 +119,8 @@ On top of the dimensions the test matrix already has (mode, bundler, React
version), the suites run once plainly and once per _test axis_ — a fixed,
lettered set of alternate flag configurations marked by `__NEXT_TEST_AXIS`.
(Today there is one axis, `A`, an alias for the `--experimental` /
`__NEXT_CACHE_COMPONENTS` run; see `scripts/run-jest.sh`.) A fixture can key
an experimental flag on an axis instead of pinning it — enabled by default,
`__NEXT_CACHE_COMPONENTS` run; see `test/jest-global-setup.ts`.) A fixture can
key an experimental flag on an axis instead of pinning it — enabled by default,
disabled on that axis:
```js
......@@ -149,8 +149,8 @@ dimension. The gate itself keeps working either way — lazy conditions read the
_resolved_ config, so they observe whatever the fixture decided, not how it
decided it.
To reproduce the disabled (axis) state locally, set the marker the same way
CI does:
To reproduce the disabled (axis) state locally, set the marker. CI sets
`__NEXT_CACHE_COMPONENTS=true` instead, which has the same effect:
```sh
__NEXT_TEST_AXIS=A NEXT_SKIP_ISOLATE=1 pnpm test-start-webpack test/e2e/app-dir/concurrent-router-queue/concurrent-router-queue.test.ts
......
......@@ -551,8 +551,8 @@ export class NextInstance {
await fs.writeFile(
fileName,
`${content}\n` +
getDeploymentTestEnvAssignments() +
getDeploymentTestEnvAssignments() +
`${content}\n` +
`
// alias __NEXT_TEST_MODE for next-deploy as "_" is not a valid
// env variable during deploy
......
......@@ -5,6 +5,7 @@ describe.each([
'__NEXT_CACHE_COMPONENTS',
'__NEXT_PARTIAL_PREFETCHING',
'__NEXT_EXPERIMENTAL_CACHED_NAVIGATIONS',
'__NEXT_TEST_AXIS',
])('deployment capture of %s', (flag) => {
beforeEach(() => {
jest.replaceProperty(process, 'env', {
......@@ -12,6 +13,7 @@ describe.each([
__NEXT_CACHE_COMPONENTS: '',
__NEXT_PARTIAL_PREFETCHING: '',
__NEXT_EXPERIMENTAL_CACHED_NAVIGATIONS: '',
__NEXT_TEST_AXIS: '',
})
})
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment