Unverified Commit a871df18 authored by Benjamin Woodruff ‮'s avatar Benjamin Woodruff Committed by GitHub

Copy pnpm store to tmpfs for test fixtures (#99960)

pnpm does not use the cache from the homedir if it's installing to a
different mountpoint, because hardlinking/reflinking wouldn't work. This
behavior is not new, but I only discovered this now because a warning
was recently added to pnpm (https://github.com/pnpm/pnpm/pull/15575).
This PR manually copies the cache to the `tmpfs` we use in CI.

If we switch to use pnpm's GVS for all of our tests, this should also
let us work around this problem, as symlinks can cross mountpoints.

The cost of copying the cache is pretty small (2-3 seconds, depending on
how you count the 1s that is run in parallel with downloading test
timings), and almost certainly pays for itself if it means we don't have
to download as much stuff from npm:

<img width="698" height="381" alt="Screenshot 2026-10-09 at 4 42 11 PM"
src="https://github.com/user-attachments/assets/34a2ece8-35c2-4efb-a989-cefa24020cf9"
/>
parent f5be3904
...@@ -407,6 +407,31 @@ jobs: ...@@ -407,6 +407,31 @@ jobs:
working-directory: turbopack/crates/turbopack-tracing/tests/node-file-trace working-directory: turbopack/crates/turbopack-tracing/tests/node-file-trace
run: pnpm install --recursive run: pnpm install --recursive
# Tests running in /tmp use a separate store for hard links, so we need to copy the cache
# there for it to be useful to tests. If we switch to using pnpm's global virtual store, we
# shouldn't need this, as symlinks can cross mountpoints.
- name: Copy pnpm cache to tmpfs
id: copy-pnpm-cache
background: true
if: ${{ runner.os == 'Linux' && inputs.skipInstallBuild != 'yes' }}
env:
PNPM_STORE_PATH: ${{ steps.get-store-path.outputs.STORE_PATH }}
run: |
PROBE_SOURCE=$(mktemp "$PNPM_STORE_PATH/.hardlink-probe.XXXXXX")
PROBE_DIR=$(mktemp -d /tmp/pnpm-hardlink-probe.XXXXXX)
if ln "$PROBE_SOURCE" "$PROBE_DIR/link"; then
rm -f "$PROBE_SOURCE"
rm -rf "$PROBE_DIR"
echo "Hard links from the pnpm store to /tmp work; skipping the fixture store copy."
else
rm -f "$PROBE_SOURCE"
rm -rf "$PROBE_DIR"
FIXTURE_STORE_PATH="/tmp/.pnpm-store"
mkdir -p "$FIXTURE_STORE_PATH"
# -a preserves hardlinks, which may be important for the pnpm store
cp -a "$PNPM_STORE_PATH" "$FIXTURE_STORE_PATH/"
fi
- name: Install Playwright browsers - name: Install Playwright browsers
background: true background: true
if: ${{ inputs.skipInstallBuild != 'yes' && inputs.needsPlaywright != 'no' }} if: ${{ inputs.skipInstallBuild != 'yes' && inputs.needsPlaywright != 'no' }}
...@@ -433,6 +458,9 @@ jobs: ...@@ -433,6 +458,9 @@ jobs:
fi fi
echo "Test timings loaded ($(wc -c < test-timings.json) bytes)" echo "Test timings loaded ($(wc -c < test-timings.json) bytes)"
- name: Wait for pnpm cache copy
wait: copy-pnpm-cache
- name: Fetch test timings via turbo - name: Fetch test timings via turbo
background: true background: true
if: ${{ inputs.testTimingsArtifact == '' }} if: ${{ inputs.testTimingsArtifact == '' }}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment