Unverified Commit 179eba81 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

[ci] Fix Rspack test manifest update workflow (#100052)

### What?

Fixes the scheduled `Update Rspack test manifest` workflow. Both of its
jobs, development and production, have failed every day, for example
[run
38217690831](https://github.com/vercel/next.js/actions/runs/38217690831).

### Why?

The workflow is supposed to keep
`test/rspack-{dev,build}-tests-manifest.json` up to date. Because it
keeps failing, those manifests are stale, and the release Rspack shards
run tests that are already known to fail (#100001 works around this by
hand).

There are two independent blockers:

1. **The GitHub CLI has no token.** #93455 moved these workflows to the
release GitHub App token, which is passed as `RELEASE_GITHUB_TOKEN`.
That removed the env var the GitHub CLI used to pick up.
`test/update-bundler-manifest.js` lists and downloads test-result
artifacts with `gh api`, so those calls now fail with "set the GH_TOKEN
environment variable".
2. **The runner path is out of date.** The updater strips a hardcoded
self-hosted runner prefix (`/root/actions-runner/...`) from test file
paths. Current artifacts come from GitHub-hosted runners
(`/home/runner/work/next.js/next.js/`). So even with auth fixed, the
script stops while processing the results.

### How?

- **Separate tokens for reads and writes.** The updater step now gets
the job's built-in token for the GitHub CLI, used only to read
artifacts. The release App token is unchanged and still does the work
that has to trigger CI: checkout, signed commits and PR creation. Its
permissions are not widened.
- **Least privilege.** Following `.github/AGENTS.md`, the workflow
defaults to no permissions, and each job grants only `actions: read`.
- **Path from the runner.** The updater derives its path prefix from
`GITHUB_WORKSPACE`, with a trailing separator so manifest keys stay
repo-relative (`test/...`). It assumes the jobs that produce and consume
the artifacts run on the same kind of GitHub-hosted runner, which is
true today. If the variable is missing it fails with a clear message, so
a manual run can't produce bad keys.

This follows the approach of @bgw's earlier draft #94835, applied fresh
on current canary. The obsolete Turbopack workflow that #94835 deleted
is already gone.

### Verification

- On unmodified canary, the updater downloads the artifact and then
fails on the old runner prefix.
- With this change, the dev and build updaters both succeed against the
latest real canary artifacts. All 1,535 dev keys and 1,606 build keys
are repo-relative. I discarded the generated manifests.
- Workflow permissions and token wiring were checked by parsing the
YAML. The updater's path handling was checked with zero, one and several
trailing slashes, and with the variable missing.
- Prettier and ESLint pass.
- **Not yet verified end to end in Actions.** Local runs used my own
GitHub CLI login, not the job-scoped token. Please dispatch the workflow
on this branch, or wait for the next scheduled run, to confirm `actions:
read` is enough. The first successful run will open a large manifest
refresh PR. Because the workflow uses `--override`, previously-passing
tests can move to failing in that refresh.

<!-- NEXT_JS_LLM -->


<!-- fleet 96f9352a-face-45ea-8233-1f0245d272b9 -->
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 6cc25f4f
# A recurring workflow which updates the passing/failing/skipped integration tests for Turbopack. # A recurring workflow which updates the passing/failing/skipped integration tests for Rspack.
name: Update Rspack test manifest name: Update Rspack test manifest
on: on:
...@@ -10,11 +10,17 @@ on: ...@@ -10,11 +10,17 @@ on:
env: env:
COREPACK_VERSION: 0.34.7 COREPACK_VERSION: 0.34.7
permissions: {}
jobs: jobs:
update_dev_manifest: update_dev_manifest:
name: Update and upload Rspack development test manifest name: Update and upload Rspack development test manifest
if: github.repository_owner == 'vercel' if: github.repository_owner == 'vercel'
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions:
# Used by `gh api` in test/update-bundler-manifest.js to list and download
# test result artifacts.
actions: read
steps: steps:
- name: Create GitHub App token - name: Create GitHub App token
id: release-app-token id: release-app-token
...@@ -63,6 +69,9 @@ jobs: ...@@ -63,6 +69,9 @@ jobs:
shell: bash shell: bash
run: node scripts/automated-update-workflow.js run: node scripts/automated-update-workflow.js
env: env:
# Used by `gh api` in test/update-bundler-manifest.js to download test
# result artifacts. Branch and PR creation use RELEASE_GITHUB_TOKEN.
GH_TOKEN: ${{ github.token }}
RELEASE_GITHUB_TOKEN: ${{ steps.release-app-token.outputs.token }} RELEASE_GITHUB_TOKEN: ${{ steps.release-app-token.outputs.token }}
RELEASE_GITHUB_APP_SLUG: ${{ steps.release-app-token.outputs.app-slug }} RELEASE_GITHUB_APP_SLUG: ${{ steps.release-app-token.outputs.app-slug }}
RELEASE_GITHUB_APP_USER_ID: ${{ steps.release-app-user.outputs.user-id }} RELEASE_GITHUB_APP_USER_ID: ${{ steps.release-app-user.outputs.user-id }}
...@@ -78,6 +87,10 @@ jobs: ...@@ -78,6 +87,10 @@ jobs:
name: Update and upload Rspack production test manifest name: Update and upload Rspack production test manifest
if: github.repository_owner == 'vercel' if: github.repository_owner == 'vercel'
runs-on: ubuntu-latest runs-on: ubuntu-latest
permissions:
# Used by `gh api` in test/update-bundler-manifest.js to list and download
# test result artifacts.
actions: read
steps: steps:
- name: Create GitHub App token - name: Create GitHub App token
id: release-app-token id: release-app-token
...@@ -126,6 +139,9 @@ jobs: ...@@ -126,6 +139,9 @@ jobs:
shell: bash shell: bash
run: node scripts/automated-update-workflow.js run: node scripts/automated-update-workflow.js
env: env:
# Used by `gh api` in test/update-bundler-manifest.js to download test
# result artifacts. Branch and PR creation use RELEASE_GITHUB_TOKEN.
GH_TOKEN: ${{ github.token }}
RELEASE_GITHUB_TOKEN: ${{ steps.release-app-token.outputs.token }} RELEASE_GITHUB_TOKEN: ${{ steps.release-app-token.outputs.token }}
RELEASE_GITHUB_APP_SLUG: ${{ steps.release-app-token.outputs.app-slug }} RELEASE_GITHUB_APP_SLUG: ${{ steps.release-app-token.outputs.app-slug }}
RELEASE_GITHUB_APP_USER_ID: ${{ steps.release-app-user.outputs.user-id }} RELEASE_GITHUB_APP_USER_ID: ${{ steps.release-app-user.outputs.user-id }}
......
...@@ -8,7 +8,16 @@ const { bold } = require('kleur') ...@@ -8,7 +8,16 @@ const { bold } = require('kleur')
const yargs = require('yargs/yargs') const yargs = require('yargs/yargs')
const { hideBin } = require('yargs/helpers') const { hideBin } = require('yargs/helpers')
const WORKING_PATH = '/root/actions-runner/_work/next.js/next.js/' // Test results contain absolute paths from the GitHub Actions job that ran
// the tests. GitHub-hosted runners use the same workspace path for that job
// and this one, so strip `GITHUB_WORKSPACE` to get repo-relative paths.
if (!process.env.GITHUB_WORKSPACE) {
console.error(
'missing GITHUB_WORKSPACE env, set it to the workspace path used by the CI job that produced the test results (e.g. /home/runner/work/next.js/next.js)'
)
process.exit(1)
}
const WORKING_PATH = process.env.GITHUB_WORKSPACE.replace(/\/*$/, '/')
const INITIALIZING_TEST_CASES = [ const INITIALIZING_TEST_CASES = [
'compile successfully', 'compile successfully',
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment