Unverified Commit 1aec1b2f authored by Sebastian "Sebbie" Silbermann's avatar Sebastian "Sebbie" Silbermann Committed by GitHub

[ci] Increase deploy test concurrency to 8 (#99770)

Increases test concurrency for the deploy test jobs to 8 (prev: 2 in the
full suite, 1 in new build-and-test jobs).

For deploy tests, the CPU-intensive work that should be as stable as
possible (the Next.js build and server) does not happen on the runner.
It happens remotely when the deployment is built at Vercel. The runner
mostly handles I/O and setup: pushing the deployment, waiting for it to
become ready, polling logs, and making requests against it. So the usual
reason to cap concurrency (runner CPU contention) barely applies.

Adjusted rate-limits:
- 10x `api-projects-get` (prev failure:
https://github.com/vercel/next.js/actions/runs/37617890690/job/112780953612#step:35:1159)

Trade-offs to watch:

- Peak concurrent Vercel deployments per run rises 4× (shards × 2 →
shards × 8; 64 → 256 concurrent test files at today's shard counts). If
we hit Vercel account deployment-concurrency limits, increase the limits
- maybe we still need a bit beefier runners if the concurrent
setup/tests starve runner CPU

## test plan

- [x] [deploy tests with debug logging on this branch takes 30 minutes
wall time without hitting Vercel rate
limits](https://github.com/vercel/next.js/actions/runs/37627918534);
usual flake on the 1st try (down from [97 minutes on
canary](https://github.com/vercel/next.js/actions/runs/37588790388/attempts/1))

---------
Co-authored-by: 's avatarClaude Code <noreply@anthropic.com>
parent c6f8ac72
...@@ -198,7 +198,7 @@ jobs: ...@@ -198,7 +198,7 @@ jobs:
NEXT_TEST_DEPLOY_SCRIPT_PATH="${{ github.event.inputs.deployScriptPath || '' }}" \ NEXT_TEST_DEPLOY_SCRIPT_PATH="${{ github.event.inputs.deployScriptPath || '' }}" \
NEXT_TEST_DEPLOY_LOGS_SCRIPT_PATH="${{ github.event.inputs.deployLogsScriptPath || '' }}" \ NEXT_TEST_DEPLOY_LOGS_SCRIPT_PATH="${{ github.event.inputs.deployLogsScriptPath || '' }}" \
NEXT_TEST_CLEANUP_SCRIPT_PATH="${{ github.event.inputs.cleanupScriptPath || '' }}" \ NEXT_TEST_CLEANUP_SCRIPT_PATH="${{ github.event.inputs.cleanupScriptPath || '' }}" \
node run-tests.js --timings --require-timings -g ${{ matrix.group }} -c 2 --type e2e node run-tests.js --timings --require-timings -g ${{ matrix.group }} -c 8 --type e2e
testTimingsArtifact: 'test-timings' testTimingsArtifact: 'test-timings'
skipNativeBuild: 'yes' skipNativeBuild: 'yes'
skipNativeInstall: 'no' skipNativeInstall: 'no'
...@@ -249,7 +249,7 @@ jobs: ...@@ -249,7 +249,7 @@ jobs:
NEXT_TEST_DEPLOY_SCRIPT_PATH="${{ github.event.inputs.deployScriptPath || '' }}" \ NEXT_TEST_DEPLOY_SCRIPT_PATH="${{ github.event.inputs.deployScriptPath || '' }}" \
NEXT_TEST_DEPLOY_LOGS_SCRIPT_PATH="${{ github.event.inputs.deployLogsScriptPath || '' }}" \ NEXT_TEST_DEPLOY_LOGS_SCRIPT_PATH="${{ github.event.inputs.deployLogsScriptPath || '' }}" \
NEXT_TEST_CLEANUP_SCRIPT_PATH="${{ github.event.inputs.cleanupScriptPath || '' }}" \ NEXT_TEST_CLEANUP_SCRIPT_PATH="${{ github.event.inputs.cleanupScriptPath || '' }}" \
node run-tests.js --timings --require-timings -g ${{ matrix.group }} -c 2 --type e2e node run-tests.js --timings --require-timings -g ${{ matrix.group }} -c 8 --type e2e
testTimingsArtifact: 'test-timings' testTimingsArtifact: 'test-timings'
skipNativeBuild: 'yes' skipNativeBuild: 'yes'
skipNativeInstall: 'no' skipNativeInstall: 'no'
...@@ -297,7 +297,7 @@ jobs: ...@@ -297,7 +297,7 @@ jobs:
VERCEL_CLI_VERSION="${{ github.event.inputs.vercelCliVersion || 'vercel@latest' }}" \ VERCEL_CLI_VERSION="${{ github.event.inputs.vercelCliVersion || 'vercel@latest' }}" \
VERCEL_FORCE_BUILD_IN_HIVE="${{ github.event.inputs.forceBuildInHive || '' }}" \ VERCEL_FORCE_BUILD_IN_HIVE="${{ github.event.inputs.forceBuildInHive || '' }}" \
VERCEL_BUILD_CONTAINER_VERSION="${{ github.event.inputs.buildContainerVersion || '' }}" \ VERCEL_BUILD_CONTAINER_VERSION="${{ github.event.inputs.buildContainerVersion || '' }}" \
node run-tests.js --timings --require-timings -g ${{ matrix.group }} -c 2 --type e2e node run-tests.js --timings --require-timings -g ${{ matrix.group }} -c 8 --type e2e
testTimingsArtifact: 'test-timings' testTimingsArtifact: 'test-timings'
skipNativeBuild: 'yes' skipNativeBuild: 'yes'
skipNativeInstall: 'no' skipNativeInstall: 'no'
......
...@@ -92,7 +92,16 @@ async function main() { ...@@ -92,7 +92,16 @@ async function main() {
return return
} }
const RUN_TESTS_ARGS = ['run-tests.js', '-c', '1', '--retries', '0'] // Deploy tests build remotely at Vercel; the runner mostly waits on I/O,
// so run more of them concurrently (same as the deploy test jobs in
// test_e2e_deploy_release.yml).
const RUN_TESTS_ARGS = [
'run-tests.js',
'-c',
testMode === 'deploy' ? '8' : '1',
'--retries',
'0',
]
// Only override the test version for deploy tests, as they need to run against // Only override the test version for deploy tests, as they need to run against
// the artifacts for the pull request. Otherwise, we don't need to specify this property, // the artifacts for the pull request. Otherwise, we don't need to specify this property,
// as tests will run against the local version of Next.js. // as tests will run against the local version of Next.js.
......
...@@ -417,6 +417,10 @@ export class NextDeployInstance extends NextInstance { ...@@ -417,6 +417,10 @@ export class NextDeployInstance extends NextInstance {
// the flag is enough to force plain-URL output for both link and deploy. // the flag is enough to force plain-URL output for both link and deploy.
vercelFlags.push('--non-interactive=false') vercelFlags.push('--non-interactive=false')
if (process.env.RUNNER_DEBUG === '1') {
vercelFlags.push('--debug')
}
// If the token is available in the environment, use it as the token in the // If the token is available in the environment, use it as the token in the
// environment. // environment.
if (TEST_TOKEN) { if (TEST_TOKEN) {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment