Unverified Commit 7ca8d66d authored by Will Binns-Smith's avatar Will Binns-Smith Committed by GitHub

Bundle Analyzer Agent Skill: Add browser initial-load eval (#99178)

## Summary

Add a click-gated CodeMirror eval that measures cold browser JavaScript
before and after optimization. Require lazy loading, preload on hover,
working editing, and a 200,000-byte initial-load budget.

Share measurement and setup helpers under `evals/lib/bundle-optimizer/`.
Require invocation of `next-bundle-optimizer` in the skills treatment.

## Verification

- Root and eval TypeScript checks pass on the rebased stack.
- Formatting, focused ESLint, and local setup/transcript checks pass.
- Remote agent evals not rerun after these updates.
- Bootstrap build blocked by a registry 401 while resolving
`@pnpm/exe@10.33.0`.

<!-- NEXT_JS_LLM -->

<!-- fleet d7bdbc2b-b0bc-4591-99eb-dccb0fa49c73 -->

<!-- fleet ea768377-d71c-4a15-90b2-f4d50d4f6a1c -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarWill Binns-Smith <755844+wbinnssmith@users.noreply.github.com>
parent bd824641
......@@ -82,13 +82,4 @@ CLAUDE.local.md
index.scip
# Evals
evals/.tarballs/
evals/experiments/
evals/results/
evals/agent-results.json
# Upgrade eval execution artifacts
evals/next-upgrade/.work/
evals/next-upgrade/results/
.env*
/.tarballs/
/experiments/
/results/
/agent-results.json
# Upgrade eval execution artifacts
/next-upgrade/.work/
/next-upgrade/results/
......@@ -34,7 +34,8 @@ Then edit three files:
**`PROMPT.md`** — what you'd type into the agent. Write it like a real user would: describe the symptom or goal, not the API. "Navigating from `/a` to `/b` is slow, fix it" is a good prompt. "Use `instant`" is not — you're testing whether the agent understands the feature well enough to reach for it, not whether it can pattern-match a name you handed it.
**`EVAL.ts`** — vitest assertions against files the agent wrote. Regex the source, don't run it.
**`EVAL.ts`** — vitest assertions against the app the agent wrote. Check source
for API contracts and use runtime assertions to verify rendered behavior.
```ts
import { expect, test } from 'vitest'
......@@ -48,6 +49,23 @@ test('exports instant', () => {
})
```
Runtime assertions can import `buildNextApp` and `startNextServer` from
`./__agent_eval__/next-test-utils.mjs`, which the runner copies into the sandbox.
Assertions run before the configured build script, so call `buildNextApp()` before
starting a production server. `startNextServer()` waits for a successful HTTP
response and returns `{ url, stop }`; call `stop()` in a `finally` block. Fixture
setup can use `{ mode: 'dev', port: 3100, detached: true, logFile: '/tmp/next-dev.log' }`
to leave a development server running for the agent.
Type-check the runner and assertions locally:
```bash
pnpm typescript --project evals/tsconfig.json
```
`evals/types` resolves sandbox-injected helpers to their source types. Each
fixture's application is checked separately during its build.
**`app/`** (or `pages/`) — the starting state. Give the agent something to edit, not a blank slate.
`package.json` needs a `build` script. `next.config.ts` and `tsconfig.json` stay unless your feature requires specific config.
......@@ -88,6 +106,37 @@ Browser-dependent fixtures declare `@playwright/test` in `dependencies` or
`devDependencies` in their `package.json`. The runner detects that dependency
and installs Chromium and its system libraries before the agent starts.
Fixtures opt into before/after JavaScript measurements with `"browserJs": true`
in `eval.config.json`. The runner adds measurement hooks only to experiments
containing those fixtures.
The dynamic-editor bundle-optimizer fixture measures cold Turbopack production
loads in headless Chromium before and after the agent runs. It uses
plain `next build`, disabled browser caching, and fresh browser contexts. Passing
requires fewer compressed JavaScript response-body bytes than the baseline and a
rounded byte budget of 200,000 bytes.
The editor must stay unloaded until interaction, preload on pointer hover, and
stay editable. CodeMirror responses are identified by its
`cm-content` runtime class.
`lib/next-test-utils.mjs` owns production builds and server cleanup.
Shared utilities in `lib/bundle-optimizer/browser-js.ts` handle browser cleanup,
cold-page measurement, byte summaries, and before/after eval results.
Content and interaction checks stay in each fixture's
`__eval__/measure-browser-js.ts`. `lib/bundle-optimizer/hooks.ts` runs the same assertions
before the agent starts and records their outcomes under `analysis.browserJs.before.checks`.
Expected optimization failures allow the agent run to proceed; build, browser,
and content failures stop setup. Per-request compressed and decoded sizes,
before/after totals, and bytes saved are recorded under `analysis.browserJs` in
`result.json`.
The SDK withholds `__eval__/` resources and `EVAL.ts` before creating the sandbox's
initial Git commit. The runner bundles the eval and its measurement imports into
one grader. Setup returns that grader and the baseline through validation-only
files and environment values; neither reaches the coding agent. The SDK restores
the grader after the agent completes, so app edits do not change validation.
A run takes ~2–5 min. To validate a fixture without executing:
```bash
......@@ -158,6 +207,7 @@ Full transcripts land in `evals/results/<variant>/<timestamp>/<eval>/run-1/`. Gr
evals/
├── eval.config.json # optional skill and timeout settings by fixture
├── evals/agent-*/ # fixtures
├── lib/bundle-optimizer/ # optimizer measurement and transcript hooks
├── lib/setup.ts # uploads tarball, writes AGENTS.md (shared by all evals)
├── experiments/ # generated per-run, gitignored
├── .tarballs/ # packed next, gitignored
......
......@@ -65,5 +65,11 @@
"agentFeedback": true,
"publish": false,
"timeout": 900
},
"agent-059-bundle-optimizer-next-dynamic": {
"skills": ["next-bundle-optimizer"],
"browserJs": true,
"publish": false,
"timeout": 1800
}
}
import { spawn } from 'node:child_process'
import { openSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { readFileSync, rmSync, writeFileSync } from 'node:fs'
import { startNextServer } from './__agent_eval__/next-test-utils.mjs'
const log = openSync('/tmp/agent-055-next-dev.log', 'a')
const dev = spawn('npm', ['run', 'dev'], {
const app = await startNextServer({
mode: 'dev',
port: 3100,
detached: true,
stdio: ['ignore', log, log],
logFile: '/tmp/agent-055-next-dev.log',
})
dev.unref()
const deadline = Date.now() + 30_000
while (Date.now() < deadline) {
try {
const response = await fetch('http://localhost:3100')
if (response.ok) break
} catch {}
await new Promise((resolve) => setTimeout(resolve, 250))
}
const response = await fetch('http://localhost:3100')
if (!response.ok) {
throw new Error(`next dev failed to start: ${response.status}`)
}
const reportsResponse = await fetch('http://localhost:3100/reports/acme')
const reportsResponse = await fetch(`${app.url}/reports/acme`)
if (!reportsResponse.ok) {
await app.stop()
throw new Error(`reports route failed to compile: ${reportsResponse.status}`)
}
......
import { spawn } from 'node:child_process'
import { openSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
import { readFileSync, rmSync, writeFileSync } from 'node:fs'
import { startNextServer } from './__agent_eval__/next-test-utils.mjs'
const log = openSync('/tmp/agent-055-next-dev.log', 'a')
const dev = spawn('npm', ['run', 'dev'], {
const app = await startNextServer({
mode: 'dev',
port: 3100,
detached: true,
stdio: ['ignore', log, log],
logFile: '/tmp/agent-057-next-dev.log',
})
dev.unref()
const deadline = Date.now() + 30_000
while (Date.now() < deadline) {
try {
const response = await fetch('http://localhost:3100')
if (response.ok) break
} catch {}
await new Promise((resolve) => setTimeout(resolve, 250))
}
const response = await fetch('http://localhost:3100')
if (!response.ok) {
throw new Error(`next dev failed to start: ${response.status}`)
}
const reportsResponse = await fetch('http://localhost:3100/reports/acme')
const reportsResponse = await fetch(`${app.url}/reports/acme`)
if (!reportsResponse.ok) {
await app.stop()
throw new Error(`reports route failed to compile: ${reportsResponse.status}`)
}
......
import { beforeAll, expect, test } from 'vitest'
import { readFileSync, readdirSync } from 'node:fs'
import { join } from 'node:path'
import { runBrowserJsEval } from '../../lib/bundle-optimizer/browser-js.js'
import { measureBrowserJs } from './__eval__/measure-browser-js.js'
import type {
BrowserMeasurement as Measurement,
JavaScriptSummary,
} from '../../lib/bundle-optimizer/browser-js.js'
type BrowserDetails = {
initial: JavaScriptSummary & {
editorLoaded: boolean
}
editor: { preloaded: boolean }
}
type BrowserMeasurement = Measurement<BrowserDetails>
let before: BrowserMeasurement
let after: BrowserMeasurement
const budgetBytes = 200_000
beforeAll(async () => {
const measured = await runBrowserJsEval<BrowserDetails>(
budgetBytes,
measureBrowserJs
)
before = measured.before
after = measured.after
}, 360_000)
test('builds and preserves a functioning editor', () => {
expect(after.editor).toBeDefined()
})
test('keeps initial browser JavaScript under 200,000 bytes', () => {
expect(after.initial.encodedBytes).toBeGreaterThan(0)
expect(after.initial.encodedBytes).toBeLessThanOrEqual(budgetBytes)
})
test('reduces initial browser JavaScript', () => {
expect(after.initial.encodedBytes).toBeLessThan(before.initial.encodedBytes)
})
test('does not load the editor before an interaction', () => {
expect(after.initial.editorLoaded).toBe(false)
})
test('preloads the editor on pointer hover', () => {
expect(after.editor?.preloaded).toBe(true)
})
test('imports next/dynamic for the editor async boundary', () => {
const sources = ['app', 'components']
.flatMap((directory) =>
readdirSync(join(process.cwd(), directory), {
recursive: true,
encoding: 'utf8',
})
.filter((path) => /\.[cm]?[jt]sx?$/.test(path))
.map((path) => join(directory, path))
)
.map((path) => readFileSync(join(process.cwd(), path), 'utf8'))
.join('\n')
expect(sources).toMatch(/from\s*['"]next\/dynamic['"]/)
})
The `/` route in this Next.js app has become slow on first load. Optimize its browser initial-load performance without changing the visible content or removing any features.
The formula editor must still open from the button, support JavaScript editing, and feel responsive when the user signals that they are about to open it by hovering over the button.
Use the tools and documentation available in the project, verify the optimization, and leave the app in a production-buildable state.
import assert from 'node:assert/strict'
import { expect } from 'playwright/test'
import type { Browser } from 'playwright/test'
import type { JavaScriptSummary } from '../../../lib/bundle-optimizer/browser-js.js'
import {
withProductionBrowser,
withMeasuredPage,
summarizeJavaScript,
} from '../../../lib/bundle-optimizer/browser-js.js'
async function measureEditor(
browser: Browser,
url: string
): Promise<{
initial: JavaScriptSummary & {
editorLoaded: boolean
}
editor: JavaScriptSummary & {
preloaded: boolean
}
}> {
return withMeasuredPage(
browser,
url,
async (page, snapshot) => {
await expect(
page.getByRole('heading', { name: 'Formula Workspace' })
).toBeVisible()
await expect(
page.getByText(
'Build and test a formula before adding it to the dashboard.',
{ exact: true }
)
).toBeVisible()
await expect(page.locator('.cm-content')).toHaveCount(0)
const loadedInitially = await snapshot()
const initial = {
...loadedInitially,
editorLoaded: loadedInitially.requests.some(
(request) => request.editor
),
}
const button = page.getByRole('button', { name: 'Open formula editor' })
const preload = page.waitForResponse(
(response) =>
response.ok() &&
/javascript/.test(response.headers()['content-type'] ?? ''),
{ timeout: 10_000 }
)
await button.hover()
await preload.catch((error) => {
if (error.name !== 'TimeoutError') throw error
})
const loaded = await snapshot()
const preloaded = loaded.requests
.slice(initial.requests.length)
.some((request) => request.editor)
await expect(page.locator('.cm-content')).toHaveCount(0)
await button.click()
const editor = page.locator('.cm-content')
await expect(editor).toBeVisible()
await expect(editor).toHaveText('revenue - costs')
await editor.fill('const profit = revenue - costs + 1')
await expect(editor).toHaveText('const profit = revenue - costs + 1')
await expect(
editor.locator('span').filter({ hasText: /^const$/ })
).toBeVisible()
const opened = await snapshot()
assert(
opened.requests.some((request) => request.editor),
'CodeMirror runtime was not identified'
)
return {
initial,
editor: {
...summarizeJavaScript(
opened.requests.slice(initial.requests.length)
),
preloaded,
},
}
},
(body) => ({
// CodeMirror's DOM class survives minification of its runtime.
editor: body.includes('cm-content'),
})
)
}
export function measureBrowserJs() {
return withProductionBrowser(measureEditor)
}
import type { ReactNode } from 'react'
export default function RootLayout({ children }: { children: ReactNode }) {
return (
<html lang="en">
<body>{children}</body>
</html>
)
}
import { Workspace } from '../components/workspace'
export default function Page() {
return (
<main>
<h1>Formula Workspace</h1>
<Workspace />
</main>
)
}
'use client'
import CodeMirror from '@uiw/react-codemirror'
import { javascript } from '@codemirror/lang-javascript'
export function HeavyEditor({
value,
onChange,
}: {
value: string
onChange: (value: string) => void
}) {
return (
<CodeMirror
value={value}
height="240px"
extensions={[javascript({ jsx: true })]}
onChange={onChange}
/>
)
}
'use client'
import { useState } from 'react'
import { HeavyEditor } from './heavy-editor'
export function Workspace() {
const [showEditor, setShowEditor] = useState(false)
const [formula, setFormula] = useState('revenue - costs')
return (
<section>
<p>Build and test a formula before adding it to the dashboard.</p>
<button onClick={() => setShowEditor(true)}>Open formula editor</button>
{showEditor ? (
<HeavyEditor value={formula} onChange={setFormula} />
) : null}
</section>
)
}
import type { NextConfig } from 'next'
const nextConfig: NextConfig = {}
export default nextConfig
{
"name": "agent-059-bundle-optimizer-next-dynamic",
"private": true,
"type": "module",
"scripts": {
"build": "next build",
"dev": "next dev",
"start": "next start"
},
"dependencies": {
"@codemirror/lang-javascript": "^6.2.4",
"@uiw/react-codemirror": "^4.25.4",
"next": "^16",
"react": "19.1.0",
"react-dom": "19.1.0"
},
"devDependencies": {
"@playwright/test": "1.61.0",
"@types/node": "^20",
"@types/react": "^19",
"@types/react-dom": "^19",
"typescript": "^5",
"vitest": "^3.1.3"
}
}
{
"compilerOptions": {
"target": "ES2017",
"lib": ["dom", "dom.iterable", "esnext"],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "react-jsx",
"incremental": true,
"plugins": [
{
"name": "next"
}
]
},
"include": [
"next-env.d.ts",
".next/types/**/*.ts",
"**/*.ts",
"**/*.tsx",
".next/dev/types/**/*.ts"
],
"exclude": ["node_modules", "EVAL.ts", "__eval__"]
}
import assert from 'node:assert/strict'
import { rm } from 'node:fs/promises'
import { chromium, expect } from 'playwright/test'
import type { Browser, Page, Request } from 'playwright/test'
import { buildNextApp, startNextServer } from '../next-test-utils.mjs'
export type ScriptRequest = {
url: string
encodedBytes: number
decodedBytes: number
} & Record<string, unknown>
export type JavaScriptSummary = {
encodedBytes: number
decodedBytes: number
requests: ScriptRequest[]
}
export type BaselineCheck = {
name: string
status: 'passed' | 'failed'
failures: string[]
}
export type BrowserMeasurement<
Details extends Record<string, unknown> = Record<string, unknown>,
> = {
initial: JavaScriptSummary
checks?: BaselineCheck[]
} & Details
export type BrowserJsEvalResult<
Details extends Record<string, unknown> = Record<string, unknown>,
> = {
before: BrowserMeasurement<Details>
after: BrowserMeasurement<Details>
budgetBytes: number
savedBytes: number
}
export async function withProductionBrowser<T>(
verify: (browser: Browser, url: string) => Promise<T>
): Promise<T> {
await rm('.next', { recursive: true, force: true })
await buildNextApp()
const server = await startNextServer()
let browser: Browser | undefined
try {
browser = await chromium.launch({ headless: true })
return await verify(browser, server.url)
} finally {
try {
await browser?.close()
} finally {
await server.stop()
}
}
}
export function summarizeJavaScript(
requests: ScriptRequest[]
): JavaScriptSummary {
return {
encodedBytes: requests.reduce((sum, item) => sum + item.encodedBytes, 0),
decodedBytes: requests.reduce((sum, item) => sum + item.decodedBytes, 0),
requests: [...requests],
}
}
export async function withMeasuredPage<T>(
browser: Browser,
url: string,
verify: (
page: Page,
snapshot: () => Promise<JavaScriptSummary>
) => Promise<T>,
classifyScript: (body: Buffer) => Record<string, boolean> = () => ({})
): Promise<T> {
const context = await browser.newContext({ serviceWorkers: 'block' })
try {
const page = await context.newPage()
const cdp = await context.newCDPSession(page)
await cdp.send('Network.enable')
await cdp.send('Network.setCacheDisabled', { cacheDisabled: true })
const requests: ScriptRequest[] = []
const pending = new Set<Promise<void>>()
const inFlight = new Set<Request>()
let lastActivity = Date.now()
const errors: string[] = []
page.on('pageerror', (error) => errors.push(error.message))
page.on('request', (request) => {
inFlight.add(request)
lastActivity = Date.now()
})
page.on('requestfinished', (request) => {
inFlight.delete(request)
lastActivity = Date.now()
})
page.on('requestfailed', (request) => {
inFlight.delete(request)
lastActivity = Date.now()
errors.push(request.url())
})
page.on('response', (response) => {
if (
response.request().resourceType() !== 'script' &&
!/javascript/.test(response.headers()['content-type'] ?? '')
)
return
const measured = (async () => {
assert.equal(await response.finished(), null, response.url())
assert(response.ok(), `Script failed: ${response.url()}`)
const sizes = await response.request().sizes()
const body = await response.body()
requests.push({
...classifyScript(body),
url: response.url(),
encodedBytes: sizes.responseBodySize,
decodedBytes: body.length,
})
})()
.catch((error) => {
errors.push(error.message)
})
.finally(() => {
pending.delete(measured)
})
pending.add(measured)
})
const response = await page.goto(url, { waitUntil: 'load' })
assert(response?.ok(), 'Initial navigation failed')
async function snapshot(): Promise<JavaScriptSummary> {
// Playwright's navigation load state stays satisfied after interactions.
await expect
.poll(
() =>
inFlight.size === 0 &&
pending.size === 0 &&
Date.now() - lastActivity >= 500,
{ timeout: 30_000 }
)
.toBe(true)
assert.deepEqual(errors, [])
return summarizeJavaScript(requests)
}
const result = await verify(page, snapshot)
await snapshot()
return result
} finally {
await context.close()
}
}
export function readBrowserJsMeasurement<
Details extends Record<string, unknown> = Record<string, unknown>,
>(output: string): BrowserMeasurement<Details> {
const match = output.match(/NEXT_EVAL_BROWSER_JS:(\{[^\r\n]+\})/)
assert(match, 'Missing browser JavaScript measurement')
const measurement: BrowserMeasurement<Details> = JSON.parse(match[1])
assert(
Number.isFinite(measurement.initial?.encodedBytes) &&
measurement.initial.encodedBytes > 0,
'Invalid compressed JavaScript bytes'
)
assert(
Number.isFinite(measurement.initial?.decodedBytes) &&
measurement.initial.decodedBytes > 0,
'Invalid decoded JavaScript bytes'
)
assert(measurement.initial.requests?.length, 'Missing JavaScript requests')
return measurement
}
export async function runBrowserJsEval<
Details extends Record<string, unknown> = Record<string, unknown>,
>(
budgetBytes: number,
measure: () => Promise<BrowserMeasurement<Details>>
): Promise<BrowserJsEvalResult<Details>> {
const baseline = process.env.NEXT_EVAL_BROWSER_JS_BEFORE
const isBaseline = process.env.NEXT_EVAL_BROWSER_JS_PHASE === 'before'
assert(isBaseline || baseline, 'Missing pre-agent browser measurement')
const after = await measure()
const before = isBaseline
? after
: readBrowserJsMeasurement<Details>('NEXT_EVAL_BROWSER_JS:' + baseline)
const measurement = {
before,
after,
budgetBytes,
savedBytes: before.initial.encodedBytes - after.initial.encodedBytes,
}
console.log('NEXT_EVAL_BROWSER_JS_RESULT:' + JSON.stringify(measurement))
return measurement
}
import { join } from 'node:path'
import { build } from 'esbuild'
import { readBrowserJsMeasurement } from './browser-js.js'
import type { BaselineCheck, BrowserJsEvalResult } from './browser-js.js'
import type {
EvalRunData,
RunCompleteHook,
RunCompleteContext,
Sandbox,
SetupFunction,
SetupResult,
} from '@vercel/agent-eval'
import { parseTranscript } from '@vercel/agent-eval'
export function createBrowserJsHooks(
fixtures: Record<string, string>,
skills: string[]
): { setup: SetupFunction; onRunComplete: RunCompleteHook } {
return {
setup: async (sandbox) => {
const pkg = JSON.parse(await sandbox.readFile('package.json'))
const fixture = Object.keys(fixtures).find(
(name) => fixtures[name] === pkg.name
)
if (fixture) return prepareBrowserJs(sandbox, fixture)
},
onRunComplete: (context) => {
if (!Object.hasOwn(fixtures, context.fixture.name)) return context.runData
const runData = skills.includes('next-bundle-optimizer')
? assertBundleOptimizerSkillInvoked(context)
: context.runData
return analyzeBrowserJs({ ...context, runData })
},
}
}
type VitestReport = {
numTotalTests: number
testResults: {
assertionResults: {
fullName: string
status: string
failureMessages: string[]
}[]
}[]
}
async function prepareBrowserJs(
sandbox: Sandbox,
fixture: string
): Promise<SetupResult> {
const bundled = await build({
entryPoints: [join(process.cwd(), 'evals', fixture, 'EVAL.ts')],
bundle: true,
packages: 'external',
platform: 'node',
format: 'esm',
target: 'node24',
write: false,
})
const validationFiles = { 'EVAL.ts': bundled.outputFiles[0].text }
const configPath = '__agent_eval__/browser-js-baseline.config.mjs'
const reportPath = '__agent_eval__/browser-js-baseline-results.json'
let baselineChecks: BaselineCheck[] = []
let before: BrowserJsEvalResult['before']
try {
await sandbox.writeFiles({
...validationFiles,
[configPath]: `export default { test: { include: ['EVAL.ts'], reporters: ['default', 'json'], outputFile: '${reportPath}', bail: 0 } }`,
})
const checks = await sandbox.runCommand(
'npx',
['vitest', 'run', '--config', configPath],
{
env: {
NEXT_EVAL_BROWSER_JS_PHASE: 'before',
},
}
)
if (checks.exitCode !== 0 && checks.exitCode !== 1) {
throw new Error(`Baseline assertions could not run:\n${checks.stderr}`)
}
const match = checks.stdout.match(
/NEXT_EVAL_BROWSER_JS_RESULT:(\{[^\r\n]+\})/
)
if (!match)
throw new Error(
`Browser JavaScript baseline failed:\n${checks.stdout}\n${checks.stderr}`
)
const measurement: BrowserJsEvalResult = JSON.parse(match[1])
before = readBrowserJsMeasurement(
'NEXT_EVAL_BROWSER_JS:' + JSON.stringify(measurement.before)
)
const report: VitestReport = JSON.parse(await sandbox.readFile(reportPath))
baselineChecks = report.testResults.flatMap((suite) =>
suite.assertionResults.map((check): BaselineCheck => {
if (check.status !== 'passed' && check.status !== 'failed') {
throw new Error('Baseline assertions did not complete')
}
return {
name: check.fullName,
status: check.status,
failures: check.failureMessages.map(
(message) => message.split('\n')[0]
),
}
})
)
before.checks = baselineChecks
if (
!baselineChecks.length ||
baselineChecks.length !== report.numTotalTests
) {
throw new Error('Baseline assertions did not complete')
}
} finally {
// EVAL.ts must be hidden again before the coding agent starts.
const cleanup = await sandbox.runCommand('node', [
'--input-type=module',
'--eval',
`import { rm } from 'node:fs/promises'; await Promise.all(${JSON.stringify(['EVAL.ts', configPath, reportPath])}.map(path => rm(path, { force: true })))`,
])
if (cleanup.exitCode !== 0)
throw new Error('Could not remove baseline assertion files')
}
console.log(
` Initial browser JavaScript: ${before.initial.encodedBytes} bytes`
)
console.log(
` Baseline assertions: ${baselineChecks.filter((check) => check.status === 'passed').length} passed, ${baselineChecks.filter((check) => check.status === 'failed').length} failed`
)
return {
validationFiles,
validationEnv: {
NEXT_EVAL_BROWSER_JS_BEFORE: JSON.stringify(before),
},
}
}
function assertBundleOptimizerSkillInvoked({
config,
runData,
}: RunCompleteContext): EvalRunData {
const invoked = runData.transcript
? parseTranscript(runData.transcript, config.agent).events.some(
(event) =>
event.type === 'tool_call' &&
event.tool?.originalName === 'Skill' &&
event.tool.args?.skill === 'next-bundle-optimizer'
)
: false
if (invoked) return runData
return {
...runData,
result: {
...runData.result,
status: 'failed',
error: [
runData.result.error,
'next-bundle-optimizer skill was not invoked',
]
.filter(Boolean)
.join('\n'),
},
}
}
function analyzeBrowserJs({ runData }: RunCompleteContext): EvalRunData {
const output = runData.outputContent?.eval ?? ''
const match = output.match(/NEXT_EVAL_BROWSER_JS_RESULT:(\{[^\r\n]+\})/)
if (!match) {
if (runData.result.status === 'failed') return runData
throw new Error('Missing browser JavaScript eval result')
}
const measurement: BrowserJsEvalResult = JSON.parse(match[1])
readBrowserJsMeasurement(
'NEXT_EVAL_BROWSER_JS:' + JSON.stringify(measurement.before)
)
readBrowserJsMeasurement(
'NEXT_EVAL_BROWSER_JS:' + JSON.stringify(measurement.after)
)
return {
...runData,
result: {
...runData.result,
analysis: {
...runData.result.analysis,
browserJs: measurement,
},
},
}
}
import { execFile, spawn } from 'node:child_process'
import { closeSync, openSync, readFileSync } from 'node:fs'
import { createRequire } from 'node:module'
import { join } from 'node:path'
import { setTimeout } from 'node:timers/promises'
import { promisify } from 'node:util'
function nextBin() {
return createRequire(join(process.cwd(), 'package.json')).resolve(
'next/dist/bin/next'
)
}
function nextEnv(mode) {
return {
...process.env,
NODE_ENV: mode === 'dev' ? 'development' : 'production',
NEXT_TELEMETRY_DISABLED: '1',
}
}
export async function buildNextApp() {
await promisify(execFile)(process.execPath, [nextBin(), 'build'], {
env: nextEnv('build'),
timeout: 180_000,
killSignal: 'SIGKILL',
maxBuffer: 10 * 1024 * 1024,
})
}
export async function startNextServer({
mode = 'start',
port = 0,
detached = false,
logFile = undefined,
} = {}) {
if (detached && (!port || !logFile)) {
throw new Error('Detached servers require a port and log file')
}
const log = logFile ? openSync(logFile, 'a') : undefined
const server = spawn(
process.execPath,
[nextBin(), mode, '--hostname', '127.0.0.1', '--port', String(port)],
{
env: nextEnv(mode),
detached,
stdio:
log === undefined ? ['ignore', 'pipe', 'pipe'] : ['ignore', log, log],
}
)
if (log !== undefined) closeSync(log)
let output = ''
server.stdout?.on('data', (chunk) => {
output += chunk
})
server.stderr?.on('data', (chunk) => {
output += chunk
})
let startupError
const stopped = new Promise((resolve) => {
server.once('exit', () => resolve())
server.once('error', (error) => {
startupError = error
resolve()
})
})
async function stop() {
if (server.pid && server.exitCode === null && server.signalCode === null) {
if (detached) process.kill(-server.pid, 'SIGKILL')
else server.kill('SIGKILL')
}
await stopped
}
try {
const deadline = Date.now() + 30_000
while (Date.now() < deadline) {
if (startupError) throw startupError
if (server.exitCode !== null || server.signalCode !== null) break
const url = port
? `http://127.0.0.1:${port}`
: output.match(/http:\/\/127\.0\.0\.1:\d+/)?.[0]
if (url) {
try {
const response = await fetch(url, {
signal: AbortSignal.timeout(2_000),
})
await response.arrayBuffer()
if (response.ok) {
if (detached) server.unref()
return { url, stop }
}
} catch {}
}
await setTimeout(100)
}
const logs = logFile ? readFileSync(logFile, 'utf8') : output
throw new Error(`Next.js ${mode} failed to become ready:\n${logs}`)
} catch (error) {
await stop()
throw error
}
}
......@@ -159,13 +159,26 @@ export async function installPlaywright(sandbox: Sandbox): Promise<void> {
* before the coding agent starts.
*/
export async function prepareFixture(sandbox: Sandbox): Promise<void> {
let pkg: { scripts?: Record<string, string> }
let pkg: {
scripts?: Record<string, string>
dependencies?: { next?: string }
devDependencies?: { next?: string }
}
try {
pkg = JSON.parse(await sandbox.readFile('package.json'))
} catch {
return
}
if (pkg.dependencies?.next ?? pkg.devDependencies?.next) {
await sandbox.writeFiles({
'__agent_eval__/next-test-utils.mjs': readFileSync(
join(REPO_ROOT, 'evals/lib/next-test-utils.mjs'),
'utf8'
),
})
}
if (!pkg.scripts?.['eval:setup']) return
const { exitCode, stderr } = await sandbox.runCommand('npm', [
......
......@@ -3,13 +3,18 @@
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"allowJs": true,
"strict": true,
"skipLibCheck": true,
"noEmit": true,
"types": ["node"]
"types": ["node", "vitest"]
},
"include": [
"lib",
"types",
"evals/*/EVAL.ts",
"evals/*/__eval__/measure-browser-js.ts",
"evals/*/eval-setup.mjs",
"experiments",
"next-upgrade/runner",
"next-upgrade/verifier/verify.ts"
......
declare module '*__agent_eval__/next-test-utils.mjs' {
export const buildNextApp: typeof import('../lib/next-test-utils.mjs').buildNextApp
export const startNextServer: typeof import('../lib/next-test-utils.mjs').startNextServer
}
......@@ -186,6 +186,7 @@
"dd-trace": "4.12.0",
"dotenv": "16.4.7",
"es5-ext": "0.10.53",
"esbuild": "0.25.9",
"escape-string-regexp": "2.0.0",
"eslint": "9.37.0",
"eslint-config-next": "workspace:*",
......@@ -290,6 +291,7 @@
"typescript": "6.0.2",
"typescript-eslint": "8.46.0",
"unfetch": "4.2.0",
"vitest": "3.2.7",
"wait-port": "0.2.2",
"webpack": "5.98.0",
"webpack-bundle-analyzer": "4.7.0",
......
diff --git a/dist/index.d.ts b/dist/index.d.ts
index 2469dbb..32d17d5 100644
--- a/dist/index.d.ts
+++ b/dist/index.d.ts
@@ -3,7 +3,7 @@
*
* Framework for testing AI coding agents in isolated sandboxes.
*/
-export type { AgentType, BuiltInAgentType, ModelTier, ModelPolicy, EvalFilter, ValidationMode, BrandConfig, Sandbox, SetupFunction, RunCompleteContext, RunCompleteHook, ExperimentConfig, ResolvedExperimentConfig, EvalFixture, EvalRunResult, EvalRunData, EvalSummary, ProgressEvent, ExperimentResults, FailureType, Classification, } from './lib/types.js';
+export type { AgentType, BuiltInAgentType, ModelTier, ModelPolicy, EvalFilter, ValidationMode, BrandConfig, Sandbox, SetupFunction, SetupResult, RunCompleteContext, RunCompleteHook, ExperimentConfig, ResolvedExperimentConfig, EvalFixture, EvalRunResult, EvalRunData, EvalSummary, ProgressEvent, ExperimentResults, FailureType, Classification, } from './lib/types.js';
export { REQUIRED_EVAL_FILES, EXCLUDED_FILES, NATIVE_DEFAULT_MODEL } from './lib/types.js';
export { CONFIG_DEFAULTS, validateConfig, resolveConfig, loadConfig, resolveEvalNames, } from './lib/config.js';
export { FixtureValidationError, discoverFixtures, validateFixtureFiles, validatePackageJson, loadFixture, loadAllFixtures, getFixtureFiles, readFixtureFiles, copyFixtureFiles, } from './lib/fixture.js';
diff --git a/dist/lib/agents/plugin/orchestrator.js b/dist/lib/agents/plugin/orchestrator.js
index a2c32ee70e7..fd0f3c2a3d8 100644
index a2c32ee..840763f 100644
--- a/dist/lib/agents/plugin/orchestrator.js
+++ b/dist/lib/agents/plugin/orchestrator.js
@@ -87,11 +87,28 @@ export function resolveJudgeRuntime(def, options) {
......@@ -311,7 +324,7 @@ index a2c32ee70e7..fd0f3c2a3d8 100644
if (aborted) {
return {
success: false,
@@ -313,12 +524,13 @@ async function runOnce(def, fixturePath, options) {
@@ -313,14 +524,18 @@ async function runOnce(def, fixturePath, options) {
modelRepair,
};
}
......@@ -322,11 +335,16 @@ index a2c32ee70e7..fd0f3c2a3d8 100644
// By default the judge is the codegen agent+model; options.judge pins a fixed one
// (judgeRuntime was resolved at step 4b so its CLI could be installed).
- const validationEnv = { ...judgeRuntime.authEnv, ...neutralWorkspace.env };
+ const validationEnv = { ...judgeRuntime.authEnv, ...neutralWorkspace.env, ...setupResult?.env };
+ const validationEnv = { ...judgeRuntime.authEnv, ...neutralWorkspace.env, ...setupResult?.env, ...setupResult?.validationEnv };
if (options.validation !== 'none') {
await sandbox.uploadFiles(testFiles);
+ if (setupResult?.validationFiles) {
+ await sandbox.writeFiles(setupResult.validationFiles);
+ }
await createVitestConfig(sandbox);
@@ -337,6 +549,7 @@ async function runOnce(def, fixturePath, options) {
await injectTranscriptContext(sandbox, transcript, def.o11yAgentName, options.model);
// Judge runtime: ship the eval helper, materialize the raw transcript as a
@@ -337,6 +552,7 @@ async function runOnce(def, fixturePath, options) {
await sandbox.writeFiles(judgeFiles);
}
const validationResults = await runValidation(sandbox, options.scripts ?? [], options.validation, validationEnv);
......@@ -334,7 +352,7 @@ index a2c32ee70e7..fd0f3c2a3d8 100644
// 11. Capture generated/deleted files (git diff).
const { generatedFiles, deletedFiles } = await captureGeneratedFiles(sandbox);
return {
@@ -380,9 +593,15 @@ async function runOnce(def, fixturePath, options) {
@@ -380,9 +596,15 @@ async function runOnce(def, fixturePath, options) {
if (options.signal) {
options.signal.removeEventListener('abort', abortHandler);
}
......@@ -353,7 +371,7 @@ index a2c32ee70e7..fd0f3c2a3d8 100644
}
}
diff --git a/dist/lib/runner.js b/dist/lib/runner.js
index b0c6a3672925853e19a5091ad4155c8086e8c946..02764f9d2a879e3437b3d46ea40cf47f3f976ea6 100644
index b0c6a36..02764f9 100644
--- a/dist/lib/runner.js
+++ b/dist/lib/runner.js
@@ -76,6 +76,17 @@ export async function runExperiment(options) {
......@@ -453,7 +471,7 @@ index b0c6a3672925853e19a5091ad4155c8086e8c946..02764f9d2a879e3437b3d46ea40cf47f
const resultsByFixture = new Map();
for (const fixture of fixtures) {
diff --git a/dist/lib/sandbox.d.ts b/dist/lib/sandbox.d.ts
index 8803497ab16a5f707b519e899d2d641481c43f37..10a1202622e4ec01b62246d1346a7af2457197e0 100644
index 8803497..10a1202 100644
--- a/dist/lib/sandbox.d.ts
+++ b/dist/lib/sandbox.d.ts
@@ -2,7 +2,7 @@
......@@ -486,7 +504,7 @@ index 8803497ab16a5f707b519e899d2d641481c43f37..10a1202622e4ec01b62246d1346a7af2
/**
* Stop and clean up the sandbox.
diff --git a/dist/lib/sandbox.js b/dist/lib/sandbox.js
index 26591c4d91b378deaae40ee173c8c493b512a13f..f23babdfb47b292bf9a55172227c546058e71879 100644
index 26591c4..12da89d 100644
--- a/dist/lib/sandbox.js
+++ b/dist/lib/sandbox.js
@@ -94,9 +94,12 @@ export class SandboxManager {
......@@ -521,17 +539,39 @@ index 26591c4d91b378deaae40ee173c8c493b512a13f..f23babdfb47b292bf9a55172227c5460
});
}
/**
@@ -374,7 +381,7 @@ export function splitTestFiles(files) {
const testFiles = [];
for (const file of files) {
const name = file.path.split('/').pop() ?? file.path;
- if (isTestFilePattern(name)) {
+ if (isTestFilePattern(name) || file.path.split('/').includes('__eval__')) {
testFiles.push(file);
}
else {
@@ -387,7 +394,7 @@ export function splitTestFiles(files) {
* Verify that no test files exist in the sandbox.
*/
export async function verifyNoTestFiles(sandbox) {
- const result = await sandbox.runShell("find . -path './node_modules' -prune -o -name 'EVAL.ts' -print");
+ const result = await sandbox.runShell("find . -path './node_modules' -prune -o \\( -name 'EVAL.ts' -o -name 'EVAL.tsx' -o -name '__eval__' \\) -print");
const foundTests = result.stdout.trim();
if (foundTests) {
throw new Error(`Test files found in sandbox before agent run: ${foundTests}`);
diff --git a/dist/lib/types.d.ts b/dist/lib/types.d.ts
index c6e6e93a3a48ab39faf93ded7b0efb1ca4e22e7f..66ac1c45b7cd451ba6f66c6ffa8c0d9d5cfaccce 100644
index c6e6e93..909b2c6 100644
--- a/dist/lib/types.d.ts
+++ b/dist/lib/types.d.ts
@@ -65,7 +65,10 @@ export interface Sandbox {
@@ -65,7 +65,14 @@ export interface Sandbox {
* Setup function that runs before the agent starts.
* Receives a sandbox instance for pre-configuration.
*/
-export type SetupFunction = (sandbox: Sandbox) => Promise<void>;
+export interface SetupResult {
+ env: Record<string, string>;
+ env?: Record<string, string>;
+ /** Files restored after the coding agent completes, before validation. */
+ validationFiles?: Record<string, string>;
+ /** Environment values passed only to validation. */
+ validationEnv?: Record<string, string>;
+}
+export type SetupFunction = (sandbox: Sandbox) => Promise<SetupResult | void>;
export interface RunCompleteContext {
......
......@@ -187,7 +187,7 @@ patchedDependencies:
'@modelcontextprotocol/sdk': 680fe4edb7abd1de29d08cdf217a22506e815a8cc1c2282201d5d47aa3e5da12
'@rspack/core@1.6.7': 4cf28ea116b0e27c7c80b09035905f9d16a7b18d1f2b7d312fc80d42cd57068d
'@types/node@20.17.7': fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab
'@vercel/agent-eval@2.2.1': c453f42ffcf4e7e070b8cba54135cd4788d4df5fdb829e0065f453fd1646e5c1
'@vercel/agent-eval@2.2.1': f89d8e50c6627841d89b2307f56a606d7a8c179061e145d95c3213f51e2d8b82
'@vercel/blob': cb53bfa5effb15b3a361e176003df667cadf757b27b7c9b458c2f0fce86ea893
'@vercel/og@1.0.1': 3d59af991a9f85bbc368d2891b4e965f10dd11c86b9786e79fc4a362bb01401b
minizlib@3.1.0: 587688821244aaee46680929cd869947377a284ac1b30c9ae8660a5ea7d2be67
......@@ -323,7 +323,7 @@ importers:
version: 1.1.0
'@testing-library/jest-dom':
specifier: 6.1.2
version: 6.1.2(@jest/globals@29.7.0(supports-color@8.1.1))(@types/jest@29.5.5)(jest@29.7.0(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(babel-plugin-macros@3.1.0)(supports-color@8.1.1))
version: 6.1.2(@jest/globals@29.7.0(supports-color@8.1.1))(@types/jest@29.5.5)(jest@29.7.0(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(babel-plugin-macros@3.1.0)(supports-color@8.1.1))(vitest@3.2.7(@edge-runtime/vm@5.0.0)(@types/debug@4.1.13)(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(jsdom@20.0.3(supports-color@8.1.1))(lightningcss@1.30.1)(sass@1.54.0)(supports-color@8.1.1)(terser@5.39.0)(tsx@4.19.2))
'@testing-library/react':
specifier: ^15.0.5
version: 15.0.7(@types/react@19.2.18)(react-dom@19.3.0-canary-dc9aed36-20261027(react@19.3.0-canary-dc9aed36-20261027))(react@19.3.0-canary-dc9aed36-20261027)
......@@ -380,7 +380,7 @@ importers:
version: 16.0.11
'@vercel/agent-eval':
specifier: 2.2.1
version: 2.2.1(patch_hash=c453f42ffcf4e7e070b8cba54135cd4788d4df5fdb829e0065f453fd1646e5c1)(supports-color@8.1.1)
version: 2.2.1(patch_hash=f89d8e50c6627841d89b2307f56a606d7a8c179061e145d95c3213f51e2d8b82)(supports-color@8.1.1)
'@vercel/blob':
specifier: 2.3.2
version: 2.3.2(patch_hash=cb53bfa5effb15b3a361e176003df667cadf757b27b7c9b458c2f0fce86ea893)
......@@ -447,6 +447,9 @@ importers:
es5-ext:
specifier: 0.10.53
version: 0.10.53
esbuild:
specifier: 0.25.9
version: 0.25.9
escape-string-regexp:
specifier: 2.0.0
version: 2.0.0
......@@ -759,6 +762,9 @@ importers:
unfetch:
specifier: 4.2.0
version: 4.2.0
vitest:
specifier: 3.2.7
version: 3.2.7(@edge-runtime/vm@5.0.0)(@types/debug@4.1.13)(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(jsdom@20.0.3(supports-color@8.1.1))(lightningcss@1.30.1)(sass@1.54.0)(supports-color@8.1.1)(terser@5.39.0)(tsx@4.19.2)
wait-port:
specifier: 0.2.2
version: 0.2.2(supports-color@8.1.1)
......@@ -3276,6 +3282,12 @@ packages:
cpu: [ppc64]
os: [aix]
'@esbuild/aix-ppc64@0.28.2':
resolution: {integrity: sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==}
engines: {node: '>=18'}
cpu: [ppc64]
os: [aix]
'@esbuild/android-arm64@0.23.1':
resolution: {integrity: sha512-xw50ipykXcLstLeWH7WRdQuysJqejuAGPd30vd1i5zSyKK3WE+ijzHmLKxdiCMtH1pHz78rOg0BKSYOSB/2Khw==}
engines: {node: '>=18'}
......@@ -3288,6 +3300,12 @@ packages:
cpu: [arm64]
os: [android]
'@esbuild/android-arm64@0.28.2':
resolution: {integrity: sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==}
engines: {node: '>=18'}
cpu: [arm64]
os: [android]
'@esbuild/android-arm@0.23.1':
resolution: {integrity: sha512-uz6/tEy2IFm9RYOyvKl88zdzZfwEfKZmnX9Cj1BHjeSGNuGLuMD1kR8y5bteYmwqKm1tj8m4cb/aKEorr6fHWQ==}
engines: {node: '>=18'}
......@@ -3300,6 +3318,12 @@ packages:
cpu: [arm]
os: [android]
'@esbuild/android-arm@0.28.2':
resolution: {integrity: sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==}
engines: {node: '>=18'}
cpu: [arm]
os: [android]
'@esbuild/android-x64@0.23.1':
resolution: {integrity: sha512-nlN9B69St9BwUoB+jkyU090bru8L0NA3yFvAd7k8dNsVH8bi9a8cUAUSEcEEgTp2z3dbEDGJGfP6VUnkQnlReg==}
engines: {node: '>=18'}
......@@ -3312,6 +3336,12 @@ packages:
cpu: [x64]
os: [android]
'@esbuild/android-x64@0.28.2':
resolution: {integrity: sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==}
engines: {node: '>=18'}
cpu: [x64]
os: [android]
'@esbuild/darwin-arm64@0.23.1':
resolution: {integrity: sha512-YsS2e3Wtgnw7Wq53XXBLcV6JhRsEq8hkfg91ESVadIrzr9wO6jJDMZnCQbHm1Guc5t/CdDiFSSfWP58FNuvT3Q==}
engines: {node: '>=18'}
......@@ -3324,6 +3354,12 @@ packages:
cpu: [arm64]
os: [darwin]
'@esbuild/darwin-arm64@0.28.2':
resolution: {integrity: sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [darwin]
'@esbuild/darwin-x64@0.23.1':
resolution: {integrity: sha512-aClqdgTDVPSEGgoCS8QDG37Gu8yc9lTHNAQlsztQ6ENetKEO//b8y31MMu2ZaPbn4kVsIABzVLXYLhCGekGDqw==}
engines: {node: '>=18'}
......@@ -3336,6 +3372,12 @@ packages:
cpu: [x64]
os: [darwin]
'@esbuild/darwin-x64@0.28.2':
resolution: {integrity: sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==}
engines: {node: '>=18'}
cpu: [x64]
os: [darwin]
'@esbuild/freebsd-arm64@0.23.1':
resolution: {integrity: sha512-h1k6yS8/pN/NHlMl5+v4XPfikhJulk4G+tKGFIOwURBSFzE8bixw1ebjluLOjfwtLqY0kewfjLSrO6tN2MgIhA==}
engines: {node: '>=18'}
......@@ -3348,6 +3390,12 @@ packages:
cpu: [arm64]
os: [freebsd]
'@esbuild/freebsd-arm64@0.28.2':
resolution: {integrity: sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [freebsd]
'@esbuild/freebsd-x64@0.23.1':
resolution: {integrity: sha512-lK1eJeyk1ZX8UklqFd/3A60UuZ/6UVfGT2LuGo3Wp4/z7eRTRYY+0xOu2kpClP+vMTi9wKOfXi2vjUpO1Ro76g==}
engines: {node: '>=18'}
......@@ -3360,6 +3408,12 @@ packages:
cpu: [x64]
os: [freebsd]
'@esbuild/freebsd-x64@0.28.2':
resolution: {integrity: sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==}
engines: {node: '>=18'}
cpu: [x64]
os: [freebsd]
'@esbuild/linux-arm64@0.23.1':
resolution: {integrity: sha512-/93bf2yxencYDnItMYV/v116zff6UyTjo4EtEQjUBeGiVpMmffDNUyD9UN2zV+V3LRV3/on4xdZ26NKzn6754g==}
engines: {node: '>=18'}
......@@ -3372,6 +3426,12 @@ packages:
cpu: [arm64]
os: [linux]
'@esbuild/linux-arm64@0.28.2':
resolution: {integrity: sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==}
engines: {node: '>=18'}
cpu: [arm64]
os: [linux]
'@esbuild/linux-arm@0.23.1':
resolution: {integrity: sha512-CXXkzgn+dXAPs3WBwE+Kvnrf4WECwBdfjfeYHpMeVxWE0EceB6vhWGShs6wi0IYEqMSIzdOF1XjQ/Mkm5d7ZdQ==}
engines: {node: '>=18'}
......@@ -3384,6 +3444,12 @@ packages:
cpu: [arm]
os: [linux]
'@esbuild/linux-arm@0.28.2':
resolution: {integrity: sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==}
engines: {node: '>=18'}
cpu: [arm]
os: [linux]
'@esbuild/linux-ia32@0.23.1':
resolution: {integrity: sha512-VTN4EuOHwXEkXzX5nTvVY4s7E/Krz7COC8xkftbbKRYAl96vPiUssGkeMELQMOnLOJ8k3BY1+ZY52tttZnHcXQ==}
engines: {node: '>=18'}
......@@ -3396,6 +3462,12 @@ packages:
cpu: [ia32]
os: [linux]
'@esbuild/linux-ia32@0.28.2':
resolution: {integrity: sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==}
engines: {node: '>=18'}
cpu: [ia32]
os: [linux]
'@esbuild/linux-loong64@0.23.1':
resolution: {integrity: sha512-Vx09LzEoBa5zDnieH8LSMRToj7ir/Jeq0Gu6qJ/1GcBq9GkfoEAoXvLiW1U9J1qE/Y/Oyaq33w5p2ZWrNNHNEw==}
engines: {node: '>=18'}
......@@ -3408,6 +3480,12 @@ packages:
cpu: [loong64]
os: [linux]
'@esbuild/linux-loong64@0.28.2':
resolution: {integrity: sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==}
engines: {node: '>=18'}
cpu: [loong64]
os: [linux]
'@esbuild/linux-mips64el@0.23.1':
resolution: {integrity: sha512-nrFzzMQ7W4WRLNUOU5dlWAqa6yVeI0P78WKGUo7lg2HShq/yx+UYkeNSE0SSfSure0SqgnsxPvmAUu/vu0E+3Q==}
engines: {node: '>=18'}
......@@ -3420,6 +3498,12 @@ packages:
cpu: [mips64el]
os: [linux]
'@esbuild/linux-mips64el@0.28.2':
resolution: {integrity: sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==}
engines: {node: '>=18'}
cpu: [mips64el]
os: [linux]
'@esbuild/linux-ppc64@0.23.1':
resolution: {integrity: sha512-dKN8fgVqd0vUIjxuJI6P/9SSSe/mB9rvA98CSH2sJnlZ/OCZWO1DJvxj8jvKTfYUdGfcq2dDxoKaC6bHuTlgcw==}
engines: {node: '>=18'}
......@@ -3432,6 +3516,12 @@ packages:
cpu: [ppc64]
os: [linux]
'@esbuild/linux-ppc64@0.28.2':
resolution: {integrity: sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==}
engines: {node: '>=18'}
cpu: [ppc64]
os: [linux]
'@esbuild/linux-riscv64@0.23.1':
resolution: {integrity: sha512-5AV4Pzp80fhHL83JM6LoA6pTQVWgB1HovMBsLQ9OZWLDqVY8MVobBXNSmAJi//Csh6tcY7e7Lny2Hg1tElMjIA==}
engines: {node: '>=18'}
......@@ -3444,6 +3534,12 @@ packages:
cpu: [riscv64]
os: [linux]
'@esbuild/linux-riscv64@0.28.2':
resolution: {integrity: sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==}
engines: {node: '>=18'}
cpu: [riscv64]
os: [linux]
'@esbuild/linux-s390x@0.23.1':
resolution: {integrity: sha512-9ygs73tuFCe6f6m/Tb+9LtYxWR4c9yg7zjt2cYkjDbDpV/xVn+68cQxMXCjUpYwEkze2RcU/rMnfIXNRFmSoDw==}
engines: {node: '>=18'}
......@@ -3456,6 +3552,12 @@ packages:
cpu: [s390x]
os: [linux]
'@esbuild/linux-s390x@0.28.2':
resolution: {integrity: sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==}
engines: {node: '>=18'}
cpu: [s390x]
os: [linux]
'@esbuild/linux-x64@0.23.1':
resolution: {integrity: sha512-EV6+ovTsEXCPAp58g2dD68LxoP/wK5pRvgy0J/HxPGB009omFPv3Yet0HiaqvrIrgPTBuC6wCH1LTOY91EO5hQ==}
engines: {node: '>=18'}
......@@ -3468,12 +3570,24 @@ packages:
cpu: [x64]
os: [linux]
'@esbuild/linux-x64@0.28.2':
resolution: {integrity: sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==}
engines: {node: '>=18'}
cpu: [x64]
os: [linux]
'@esbuild/netbsd-arm64@0.25.9':
resolution: {integrity: sha512-9jNJl6FqaUG+COdQMjSCGW4QiMHH88xWbvZ+kRVblZsWrkXlABuGdFJ1E9L7HK+T0Yqd4akKNa/lO0+jDxQD4Q==}
engines: {node: '>=18'}
cpu: [arm64]
os: [netbsd]
'@esbuild/netbsd-arm64@0.28.2':
resolution: {integrity: sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==}
engines: {node: '>=18'}
cpu: [arm64]
os: [netbsd]
'@esbuild/netbsd-x64@0.23.1':
resolution: {integrity: sha512-aevEkCNu7KlPRpYLjwmdcuNz6bDFiE7Z8XC4CPqExjTvrHugh28QzUXVOZtiYghciKUacNktqxdpymplil1beA==}
engines: {node: '>=18'}
......@@ -3486,6 +3600,12 @@ packages:
cpu: [x64]
os: [netbsd]
'@esbuild/netbsd-x64@0.28.2':
resolution: {integrity: sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==}
engines: {node: '>=18'}
cpu: [x64]
os: [netbsd]
'@esbuild/openbsd-arm64@0.23.1':
resolution: {integrity: sha512-3x37szhLexNA4bXhLrCC/LImN/YtWis6WXr1VESlfVtVeoFJBRINPJ3f0a/6LV8zpikqoUg4hyXw0sFBt5Cr+Q==}
engines: {node: '>=18'}
......@@ -3498,6 +3618,12 @@ packages:
cpu: [arm64]
os: [openbsd]
'@esbuild/openbsd-arm64@0.28.2':
resolution: {integrity: sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==}
engines: {node: '>=18'}
cpu: [arm64]
os: [openbsd]
'@esbuild/openbsd-x64@0.23.1':
resolution: {integrity: sha512-aY2gMmKmPhxfU+0EdnN+XNtGbjfQgwZj43k8G3fyrDM/UdZww6xrWxmDkuz2eCZchqVeABjV5BpildOrUbBTqA==}
engines: {node: '>=18'}
......@@ -3510,12 +3636,24 @@ packages:
cpu: [x64]
os: [openbsd]
'@esbuild/openbsd-x64@0.28.2':
resolution: {integrity: sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==}
engines: {node: '>=18'}
cpu: [x64]
os: [openbsd]
'@esbuild/openharmony-arm64@0.25.9':
resolution: {integrity: sha512-4Xd0xNiMVXKh6Fa7HEJQbrpP3m3DDn43jKxMjxLLRjWnRsfxjORYJlXPO4JNcXtOyfajXorRKY9NkOpTHptErg==}
engines: {node: '>=18'}
cpu: [arm64]
os: [openharmony]
'@esbuild/openharmony-arm64@0.28.2':
resolution: {integrity: sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==}
engines: {node: '>=18'}
cpu: [arm64]
os: [openharmony]
'@esbuild/sunos-x64@0.23.1':
resolution: {integrity: sha512-RBRT2gqEl0IKQABT4XTj78tpk9v7ehp+mazn2HbUeZl1YMdaGAQqhapjGTCe7uw7y0frDi4gS0uHzhvpFuI1sA==}
engines: {node: '>=18'}
......@@ -3528,6 +3666,12 @@ packages:
cpu: [x64]
os: [sunos]
'@esbuild/sunos-x64@0.28.2':
resolution: {integrity: sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==}
engines: {node: '>=18'}
cpu: [x64]
os: [sunos]
'@esbuild/win32-arm64@0.23.1':
resolution: {integrity: sha512-4O+gPR5rEBe2FpKOVyiJ7wNDPA8nGzDuJ6gN4okSA1gEOYZ67N8JPk58tkWtdtPeLz7lBnY6I5L3jdsr3S+A6A==}
engines: {node: '>=18'}
......@@ -3540,6 +3684,12 @@ packages:
cpu: [arm64]
os: [win32]
'@esbuild/win32-arm64@0.28.2':
resolution: {integrity: sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==}
engines: {node: '>=18'}
cpu: [arm64]
os: [win32]
'@esbuild/win32-ia32@0.23.1':
resolution: {integrity: sha512-BcaL0Vn6QwCwre3Y717nVHZbAa4UBEigzFm6VdsVdT/MbZ38xoj1X9HPkZhbmaBGUD1W8vxAfffbDe8bA6AKnQ==}
engines: {node: '>=18'}
......@@ -3552,6 +3702,12 @@ packages:
cpu: [ia32]
os: [win32]
'@esbuild/win32-ia32@0.28.2':
resolution: {integrity: sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==}
engines: {node: '>=18'}
cpu: [ia32]
os: [win32]
'@esbuild/win32-x64@0.23.1':
resolution: {integrity: sha512-BHpFFeslkWrXWyUPnbKm+xYYVYruCinGcftSBaa8zoF9hZO4BcSCFUvHVTtzpIY6YzUnYtuEhZ+C9iEXjxnasg==}
engines: {node: '>=18'}
......@@ -3564,6 +3720,12 @@ packages:
cpu: [x64]
os: [win32]
'@esbuild/win32-x64@0.28.2':
resolution: {integrity: sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==}
engines: {node: '>=18'}
cpu: [x64]
os: [win32]
'@eslint-community/eslint-utils@4.9.0':
resolution: {integrity: sha512-ayVFHdtZ+hsq1t2Dy24wCmGXGe4q9Gu3smhLYALJrr473ZH27MsnSL+LKUlimp4BWJqMDMLmPpx/Q9R3OAlL4g==}
engines: {node: ^12.22.0 || ^14.17.0 || >=16.0.0}
......@@ -6141,6 +6303,144 @@ packages:
rollup:
optional: true
'@rollup/rollup-android-arm-eabi@4.65.0':
resolution: {integrity: sha512-t2AnVba25baFdq7gczzS13JyVcdubLVbFX1X04mxrggDrTQPUXNv3XM0k9H1hr5tqvg7AVvtQ720QHAj0iYqhg==}
cpu: [arm]
os: [android]
'@rollup/rollup-android-arm64@4.65.0':
resolution: {integrity: sha512-dNONUaEwPGhl/nuO2KV8VABWlcjYyU6KtZGN8dQv5OS5JU7JSCxPijVNKXbRJd0iuTzTiJ4TjPt/4bbvDRNiMQ==}
cpu: [arm64]
os: [android]
'@rollup/rollup-darwin-arm64@4.65.0':
resolution: {integrity: sha512-zWQqUH+9zavr2Bxuzfpuv2SIF3X31cr58TvSQC9o1GhpOvkqRwr60GkZdQcudyT/BEgdpdSafjgd2I13MbTL7g==}
cpu: [arm64]
os: [darwin]
'@rollup/rollup-darwin-x64@4.65.0':
resolution: {integrity: sha512-RQYreBFWtO4//hIuOaZkYUnF/lPU4lz35dveDimDshWCJvVisVtJ/shmT9bkXK8wFJY+FJF89X6F3Cq1qI3LCw==}
cpu: [x64]
os: [darwin]
'@rollup/rollup-freebsd-arm64@4.65.0':
resolution: {integrity: sha512-A0NfMj6Cuh/9BE5k3awJKv6HoxFVXIcLAb6SZqAMoxUHPA1ZNNK4RsGuUPD/vL7nEAm9LG+ZzXWWTrnC0XtkpA==}
cpu: [arm64]
os: [freebsd]
'@rollup/rollup-freebsd-x64@4.65.0':
resolution: {integrity: sha512-BI6/H7q3aP0I7S7GOrjyxTjgEJlSnovXE38+bdxBpuKN6z+d//uF+T+VQ3Q7ltJL/71p17ufartARX9Ja5gOGQ==}
cpu: [x64]
os: [freebsd]
'@rollup/rollup-linux-arm-gnueabihf@4.65.0':
resolution: {integrity: sha512-W1vUprf09RWaVWnsFP1vlUkEWJzTZlpObOEAYZXt9Fq0UVAOYEkeGr7wBX7McKqqIQskXdBnpiOMgwPujordgA==}
cpu: [arm]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-arm-musleabihf@4.65.0':
resolution: {integrity: sha512-s8QTcZWgrl9haFzEGIqyE5BWyo89pFDGLMBFZ752WYxk8DJwNU/O1X6pZgPw829d9/sm6CCHyrMP3aurt76p1Q==}
cpu: [arm]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-arm64-gnu@4.65.0':
resolution: {integrity: sha512-ke1f6dLRzkqveyjjF2fkEwQYZvxHtNNwjegY8nJrwcO0Bdqr8CzFO/pKE0Bns13cx0qz2Ba2Y0Hs3dSwMdnHxA==}
cpu: [arm64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-arm64-musl@4.65.0':
resolution: {integrity: sha512-926SMeUcy4y18Rz/w2aJiUKOnyAnm6czg72bXkQEaeemsll+Wr0hnsjb2BGRpHvWgwAMYy5ACJCotbvfLCfj0A==}
cpu: [arm64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-loong64-gnu@4.65.0':
resolution: {integrity: sha512-6T7JNRP1hc4hqF2dUsEoVLOt9K8tmODoCLldtQRYnrddGGQY8Gqe2pM/qMMUo7EMowpP7oa2N95k0/8sYK/ipw==}
cpu: [loong64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-loong64-musl@4.65.0':
resolution: {integrity: sha512-SImGFl1Je5kpyAgOCPONVT7h6ASV8mAUwGRjY7eZvVjPg//PfHOXgI+P241RF/+HqsA4PP3qD527ysWFKNQyhQ==}
cpu: [loong64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-ppc64-gnu@4.65.0':
resolution: {integrity: sha512-XeDTbmPF1Yu5asfXGPrMkWoxTDhgryQbmw5Q1+R582O51VNCk23J6f2pDrsAp5YkneBVBwtExx6ZGOb9D2aQ6A==}
cpu: [ppc64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-ppc64-musl@4.65.0':
resolution: {integrity: sha512-VS55JUI/1nsljmV/jfqNkLZXFw8uMO8nNlfoHmSevNf6fO3neuhyDvx9dv3c/Ytt8r/431SL8Qax1dpYvOMZTg==}
cpu: [ppc64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-riscv64-gnu@4.65.0':
resolution: {integrity: sha512-Vblw84TbX7RS4la1Y4+yQ7aore/oSyaNtFbppuBr5Vs/4LfBeXKq9dIfoExw6zOvRERqd2rRAjnZRaMhH/0x5w==}
cpu: [riscv64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-riscv64-musl@4.65.0':
resolution: {integrity: sha512-7+gHjtONvKgwR9InTKFaLq53f6YzBJkgeMRBENdWTsQr+spahGqQm8HF5aXpq8i4JaRrOqYyv6Zn1WltDTxaZQ==}
cpu: [riscv64]
os: [linux]
libc: [musl]
'@rollup/rollup-linux-s390x-gnu@4.65.0':
resolution: {integrity: sha512-KvNDC0J6s4vVHhtJ6KrDXU0Er0au/hmRIBq/+1maCV8TseGupeKizq0bWcrbp2HE1T8xoknvN0Q/KZ/YF0qX3w==}
cpu: [s390x]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-x64-gnu@4.65.0':
resolution: {integrity: sha512-GExdD15WPvu/x/drELjVam7vX8EsPgvTMOa/iJL/p/4m/oOqzbb7JXZg8fXR4hlXtg/sF2zyTZ3bR0AEtEMlhQ==}
cpu: [x64]
os: [linux]
libc: [glibc]
'@rollup/rollup-linux-x64-musl@4.65.0':
resolution: {integrity: sha512-8sT9PemIZfIMED4Vd4KGoOXVDj5CMSWNcGdbheRUx9TYS03yOWdT2Q0lXPyEyNBB9YTilwewIQKNIBk6Nxi0Kg==}
cpu: [x64]
os: [linux]
libc: [musl]
'@rollup/rollup-openbsd-x64@4.65.0':
resolution: {integrity: sha512-pO6Ehi0jo5yE0R6PltKhJhv9mlIiyzBJk5N5NUF2OF6xtVKI+1bdXxj/rb0jGXy3hck1ELVT0QPY3tzkAm360w==}
cpu: [x64]
os: [openbsd]
'@rollup/rollup-openharmony-arm64@4.65.0':
resolution: {integrity: sha512-vo7N3/+cjjPDYr8vtxC+qSNgWa9sDvJ7g8N8R5SWqFY4WiNH1ULcAU70SKVyHeiIZOYYwNP9pTipmhN5sl1NPw==}
cpu: [arm64]
os: [openharmony]
'@rollup/rollup-win32-arm64-msvc@4.65.0':
resolution: {integrity: sha512-ArwvgFAbjKcDM7qqotZ4OVYf+UMT9xH2YzkCzK4+AuZSDihMlz1Ekoe4INwBVXzn2mnJzuIYOcSx07iTBijGRQ==}
cpu: [arm64]
os: [win32]
'@rollup/rollup-win32-ia32-msvc@4.65.0':
resolution: {integrity: sha512-QGMmK4/cxKE/36FoLMSHQBCyymv/eFTnsCvvsBF0rgmOvVbvX78VWZRerL3dHa77f4pM6Xcc2iGDhJ10w/svKQ==}
cpu: [ia32]
os: [win32]
'@rollup/rollup-win32-x64-gnu@4.65.0':
resolution: {integrity: sha512-XfKYFWHIPs4HPODIMsGF1IKlS3H1e3WokUZIgpmn4+uNnZBlnRDDHEylNyazbQXwhMKZre68OjtRYbKqb5+j8w==}
cpu: [x64]
os: [win32]
'@rollup/rollup-win32-x64-msvc@4.65.0':
resolution: {integrity: sha512-wPWi++K4Bs1VR4XihOhfGi4IGX6Cub7FlerTf8X2v+2BtDHddgSqWCXwGD+h2yAe4/niIhNDwAltioOC+wEogQ==}
cpu: [x64]
os: [win32]
'@rspack/binding-darwin-arm64@1.6.7':
resolution: {integrity: sha512-QiIAP8JTAtht0j8/xZZEQTJRB9e+KrOm9c7JJm73CewVg55rDWRrwopiVfBNlTu1coem1ztUHJYdQhg2uXfqww==}
cpu: [arm64]
......@@ -6891,6 +7191,9 @@ packages:
'@types/bytes@3.1.1':
resolution: {integrity: sha512-lOGyCnw+2JVPKU3wIV0srU0NyALwTBJlVSx5DfMQOFuuohA8y9S8orImpuIQikZ0uIQ8gehrRjxgQC1rLRi11w==}
'@types/chai@5.2.3':
resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==}
'@types/cheerio@0.22.16':
resolution: {integrity: sha512-bSbnU/D4yzFdzLpp3+rcDj0aQQMIRUBNJU7azPxdqMpnexjUSvGJyDuOBQBHeOZh1mMKgsJm6Dy+LLh80Ew4tQ==}
......@@ -6939,6 +7242,9 @@ packages:
'@types/debug@4.1.5':
resolution: {integrity: sha512-Q1y515GcOdTHgagaVFhHnIFQ38ygs/kmxdNpvpou+raI9UO3YZcHDngBSYKQklcKlvA7iuQlmIKbzvmxcOE9CQ==}
'@types/deep-eql@4.0.2':
resolution: {integrity: sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==}
'@types/doctrine@0.0.9':
resolution: {integrity: sha512-eOIHzCUSH7SMfonMG1LsC2f8vxBFtho6NGBznK41R84YzPuvSBzrhEps33IsQiOW9+VL6NQ9DbjQJznk/S4uRA==}
......@@ -6963,6 +7269,9 @@ packages:
'@types/estree@1.0.8':
resolution: {integrity: sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==}
'@types/estree@1.0.9':
resolution: {integrity: sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==}
'@types/events@3.0.0':
resolution: {integrity: sha512-EaObqwIvayI5a8dCzhFrjKzVwKLxjoG9T6Ppd5CEo07LRKfQ8Yokw54r5+Wq7FaBQ+yXRvQAYPrHwya1/UFt9g==}
......@@ -7561,21 +7870,50 @@ packages:
'@vitest/expect@2.0.5':
resolution: {integrity: sha512-yHZtwuP7JZivj65Gxoi8upUN2OzHTi3zVfjwdpu2WrvCZPLwsJ2Ey5ILIPccoW23dd/zQBlJ4/dhi7DWNyXCpA==}
'@vitest/expect@3.2.7':
resolution: {integrity: sha512-E8eBXaKibuvH2pSZErOjdVb5vF4PbKYcrnluBTYxEk1l/VhhwZg1kZQsdtjq+CsF5CFydf2Rdkz7jDHKSisi3w==}
'@vitest/mocker@3.2.7':
resolution: {integrity: sha512-Trr0hYO9CM3Wj6ksWHRhK9IZpIY6wTMO5u/MqXurMxT57sWBaOPEtP3Oq60ihZuh5JsiagKfz95OcxdEP6dBrA==}
peerDependencies:
msw: ^2.4.9
vite: ^5.0.0 || ^6.0.0 || ^7.0.0-0
peerDependenciesMeta:
msw:
optional: true
vite:
optional: true
'@vitest/pretty-format@2.0.5':
resolution: {integrity: sha512-h8k+1oWHfwTkyTkb9egzwNMfJAEx4veaPSnMeKbVSjp4euqGSbQlm5+6VHwTr7u4FJslVVsUG5nopCaAYdOmSQ==}
'@vitest/pretty-format@2.1.8':
resolution: {integrity: sha512-9HiSZ9zpqNLKlbIDRWOnAWqgcA7xu+8YxXSekhr0Ykab7PAYFkhkwoqVArPOtJhPmYeE2YHgKZlj3CP36z2AJQ==}
'@vitest/pretty-format@3.2.7':
resolution: {integrity: sha512-KUHlwqVu0sRlhCdyPdQ/wBoTfRahjUky1MubOmYw9fWfIZy1gNoHpuaaQBPAaMaVYdQYHJLurzj8ECCj5OwTqA==}
'@vitest/runner@3.2.7':
resolution: {integrity: sha512-sB9y4ovltoQP+WaUPwmSxO9WIg9Ig694Di5PalVPsYHklAdE027mehpWF2SQSVq+k6sFgaivbTjTJwZLSHbedA==}
'@vitest/snapshot@3.2.7':
resolution: {integrity: sha512-7C+MwShwtBSI5Buwoyg3s/iY1eHL9PKAf+O1wVh/TdnjXUtkoL/9YQtre90i4MtNXM6edP1wJ2zOBpfCyhIS7g==}
'@vitest/spy@2.0.5':
resolution: {integrity: sha512-c/jdthAhvJdpfVuaexSrnawxZz6pywlTPe84LUB2m/4t3rl2fTo9NFGBG4oWgaD+FTgDDV8hJ/nibT7IfH3JfA==}
'@vitest/spy@3.2.7':
resolution: {integrity: sha512-Q2eQGI6d2L/hBtZ0qNuKcAGid68XK6cv1xsoaIma6PaJhHPoqcEJhYpXZ/5myCMqkNgtP6UKuBhbc0nHKnrkuQ==}
'@vitest/utils@2.0.5':
resolution: {integrity: sha512-d8HKbqIcya+GR67mkZbrzhS5kKhtp8dQLcmRZLGTscGVg7yImT82cIrhtn2L8+VujWcy6KZweApgNmPsTAO/UQ==}
'@vitest/utils@2.1.8':
resolution: {integrity: sha512-dwSoui6djdwbfFmIgbIjX2ZhIoG7Ex/+xpxyiEgIGzjliY8xGkcpITKTlp6B4MgtGkF2ilvm97cPM96XZaAgcA==}
'@vitest/utils@3.2.7':
resolution: {integrity: sha512-x6BDOd7dyo3PFLY3I9/HJ25X/6OurhGXk2/B9gOZNPF7XDVjeBK4k01lQE5uvDpbuheErh91qYuE1E2OEjK3Rw==}
'@webassemblyjs/ast@1.14.1':
resolution: {integrity: sha512-nuBEDgQfm1ccRp/8bCQrx1frohyufl4JlbMMZ4P1wpeOfDhF6FQkxZJ1b/e+PLwr6X1Nhw6OLme5usuBWYBvuQ==}
......@@ -8442,6 +8780,10 @@ packages:
resolution: {integrity: sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==}
engines: {node: '>= 0.8'}
cac@6.7.14:
resolution: {integrity: sha512-b6Ilus+c3RrdDk+JhLKUAQfzzgLEPy6wcXqS7f/xe1EETvsDP6GORG7SFuOs6cID5YkqchW/LXZbX5bc8j7ZcQ==}
engines: {node: '>=8'}
cacheable-request@6.1.0:
resolution: {integrity: sha512-Oj3cAGPCqOZX7Rz64Uny2GYAZNliQSqfbePrgAQ1wKAihYmCUnraBtJtKcGR4xz7wF+LoJC+ssFZvv5BgF9Igg==}
engines: {node: '>=8'}
......@@ -10144,6 +10486,11 @@ packages:
engines: {node: '>=18'}
hasBin: true
esbuild@0.28.2:
resolution: {integrity: sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==}
engines: {node: '>=18'}
hasBin: true
escalade@3.1.1:
resolution: {integrity: sha512-k0er2gUkLf8O0zKJiAhmkTnJlTvINGv7ygDNPbeIsX/TJjGJZHuh9B2UxbsaEkmlEo9MfhrSzmhIlhRlI2GXnw==}
engines: {node: '>=6'}
......@@ -10551,6 +10898,10 @@ packages:
expect-type@0.14.2:
resolution: {integrity: sha512-ed3+tr5ujbIYXZ8Pl/VgIphwJQ0q5tBLGGdn7Zvwt1WyPBRX83xjT5pT77P/GkuQbctx0K2ZNSSan7eruJqTCQ==}
expect-type@1.4.0:
resolution: {integrity: sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==}
engines: {node: '>=12.0.0'}
expect@29.5.0:
resolution: {integrity: sha512-yM7xqUrCO2JdpFo4XpM82t+PJBFybdqoQuJLDGeDX2ij8NZzqRHyu3Hp188/JX7SWqud+7t4MUdvcgGBICMHZg==}
engines: {node: ^14.15.0 || ^16.10.0 || >=18.0.0}
......@@ -12345,6 +12696,9 @@ packages:
js-tokens@4.0.0:
resolution: {integrity: sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==}
js-tokens@9.0.1:
resolution: {integrity: sha512-mxa9E9ITFOt0ban3j6L5MpjwegGz6lBQmM1IJkWeBZGcMxto50+eWdjC/52xDbS2vy0k7vIMK0Fe2wfL9OQSpQ==}
js-yaml@3.14.1:
resolution: {integrity: sha512-okMH7OXXJ7YrN9Ok3/SXrnu4iX9yOk+25nqX4imS2npuvTYDmo/QEZoqwZkYaIDk3jVvBOTOIEgEhaLOynBS9g==}
hasBin: true
......@@ -14235,6 +14589,9 @@ packages:
resolution: {integrity: sha512-gDKb8aZMDeD/tZWs9P6+q0J9Mwkdl6xMV8TjnGP3qJVJ06bdMgkbBlLU8IdfOsIsFz2BW1rNVT3XuNEl8zPAvw==}
engines: {node: '>=8'}
pathe@2.0.3:
resolution: {integrity: sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==}
pathval@2.0.0:
resolution: {integrity: sha512-vE7JKRyES09KiunauX7nd2Q9/L7lhok4smP9RZTDeD4MVs72Dp2qNFVz39Nz5a0FVEW0BJR6C0DYrq6unoziZA==}
engines: {node: '>= 14.16'}
......@@ -15846,6 +16203,11 @@ packages:
engines: {node: '>=10.0.0'}
hasBin: true
rollup@4.65.0:
resolution: {integrity: sha512-QVCvQUTGDN6JRwi3jVGCY3H4aTHvi9rX0LyoBMkGUvdh/Z6O51CzcLz85BWZaTCDGhl6ySGqYZs+xI1jmPqGcw==}
engines: {node: '>=18.0.0', npm: '>=8.0.0'}
hasBin: true
router@2.2.0:
resolution: {integrity: sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ==}
engines: {node: '>= 18'}
......@@ -16177,6 +16539,9 @@ packages:
resolution: {integrity: sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw==}
engines: {node: '>= 0.4'}
siginfo@2.0.0:
resolution: {integrity: sha512-ybx0WO1/8bSBLEWXZvEd7gMW3Sn3JFlW3TvX1nREbDLRNQNaeNN8WK0meBwPdAaOI7TtRRRJn/Es1zhrrCHu7g==}
signal-exit@3.0.3:
resolution: {integrity: sha512-VUJ49FC8U1OxwZLxIbTTrDvLnf/6TDgxZcK8wxR8zs13xpx7xbG60ndBlhNrFi2EMuFRoeDoJO7wthSLq42EjA==}
......@@ -16383,6 +16748,9 @@ packages:
resolution: {integrity: sha512-XlkWvfIm6RmsWtNJx+uqtKLS8eqFbxUg0ZzLXqY0caEy9l7hruX8IpiDnjsLavoBgqCCR71TqWO8MaXYheJ3RQ==}
engines: {node: '>=10'}
stackback@0.0.2:
resolution: {integrity: sha512-1XMJE5fQo1jGH6Y/7ebnwPOBEkIEnT4QF32d5R1+VXdXveM0IBMJt8zfaxX1P3QhVwrYe+576+jkANtSS2mBbw==}
stacktrace-parser@0.1.10:
resolution: {integrity: sha512-KJP1OCML99+8fhOHxwwzyWrlUuVX5GQ0ZpJTd1DFXhdkrvg1szxfHhawXUZ3g9TkXORQd4/WG68jMlQZ2p8wlg==}
engines: {node: '>=6'}
......@@ -16398,6 +16766,9 @@ packages:
resolution: {integrity: sha512-RwNA9Z/7PrK06rYLIzFMlaF+l73iwpzsqRIFgbMLbTcLD6cOao82TaWefPXQvB2fOC4AjuYSEndS7N/mTCbkdQ==}
engines: {node: '>= 0.8'}
std-env@3.10.0:
resolution: {integrity: sha512-5GS12FdOZNliM5mAOxFRg7Ir0pWz8MdpYm6AY6VPkGpbA7ZzmbzNcBJQ0GPvvyWgcY7QAhCgf9Uy89I03faLkg==}
stop-iteration-iterator@1.1.0:
resolution: {integrity: sha512-eLoXW/DHyl62zxY4SCaIgnRhuMr6ri4juEYARS8E6sCEqzKpOiE521Ucofdx+KnDZl5xmvGYaaKCk5FEOxJCoQ==}
engines: {node: '>= 0.4'}
......@@ -16579,6 +16950,9 @@ packages:
resolution: {integrity: sha512-6fPc+R4ihwqP6N/aIv2f1gMH8lOVtWQHoqC4yK6oSDVVocumAsfCqjkXnqiYMhmMwS/mEHLp7Vehlt3ql6lEig==}
engines: {node: '>=8'}
strip-literal@3.1.0:
resolution: {integrity: sha512-8r3mkIM/2+PpjHoOtiAW8Rg3jJLHaV7xPwG+YRGrv6FP0wwk/toTpATxWYOW0BKdWwl82VT2tFYi5DlROa0Mxg==}
stubborn-fs@1.2.5:
resolution: {integrity: sha512-H2N9c26eXjzL/S/K+i/RHHcFanE74dptvvjM8iwzwbVcWY/zjBbgRqF3K0DY4+OD+uTTASTBvDoxPDaPN02D7g==}
......@@ -16881,22 +17255,40 @@ packages:
tiny-invariant@1.3.3:
resolution: {integrity: sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==}
tinybench@2.9.0:
resolution: {integrity: sha512-0+DUvqWMValLmha6lr4kD8iAMK1HzV0/aKnCtWb9v9641TnP/MFb7Pc2bxoxQjTXAErryXVgUOfv2YqNllqGeg==}
tinydate@1.3.0:
resolution: {integrity: sha512-7cR8rLy2QhYHpsBDBVYnnWXm8uRTr38RoZakFSW7Bs7PzfMPNZthuMLkwqZv7MTu8lhQ91cOFYS5a7iFj2oR3w==}
engines: {node: '>=4'}
tinyexec@0.3.2:
resolution: {integrity: sha512-KQQR9yN7R5+OSwaK0XQoj22pwHoTlgYqmUscPYoknOoWCWfj/5/ABTMRi69FrKU5ffPVh5QcFikpWJI/P1ocHA==}
tinyglobby@0.2.15:
resolution: {integrity: sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==}
engines: {node: '>=12.0.0'}
tinypool@1.1.1:
resolution: {integrity: sha512-Zba82s87IFq9A9XmjiX5uZA/ARWDrB03OHlq+Vw1fSdt0I+4/Kutwy8BP4Y/y/aORMo61FQ0vIb5j44vSo5Pkg==}
engines: {node: ^18.0.0 || >=20.0.0}
tinyrainbow@1.2.0:
resolution: {integrity: sha512-weEDEq7Z5eTHPDh4xjX789+fHfF+P8boiFB+0vbWzpbnbsEr/GRaohi/uMKxg8RZMXnl1ItAi/IUHWMsjDV7kQ==}
engines: {node: '>=14.0.0'}
tinyrainbow@2.0.0:
resolution: {integrity: sha512-op4nsTR47R6p0vMUUoYl/a+ljLFVtlfaXkLQmqfLR1qHma1h/ysYk4hEXZ880bf2CYgTskvTa/e196Vd5dDQXw==}
engines: {node: '>=14.0.0'}
tinyspy@3.0.2:
resolution: {integrity: sha512-n1cw8k1k0x4pgA2+9XrOkFydTerNcJ1zWCO5Nn9scWHTD+5tp8dghT2x1uduQePZTZgd3Tupf+x9BxJjeJi77Q==}
engines: {node: '>=14.0.0'}
tinyspy@4.0.6:
resolution: {integrity: sha512-u8KszXvGfU68hVcZpRHKG28T0krMuv2G5nDhiHaMLen/gIuFEgIJhaJuO69qjnXg5paSrbPMFfx3brNuN8eVSg==}
engines: {node: '>=14.0.0'}
title-case@3.0.3:
resolution: {integrity: sha512-e1zGYRvbffpcHIrnuqT0Dh+gEJtDaxDSoG4JAIpq4oDFyooziLBIiYQv0GBT4FUAnUop5uZ1hiIAj7oAF6sOCA==}
......@@ -17647,6 +18039,79 @@ packages:
vfile@6.0.3:
resolution: {integrity: sha512-KzIbH/9tXat2u30jf+smMwFCsno4wHVdNmzFyL+T/L3UGqqk6JKfVqOFOZEpZSHADH1k40ab6NUIXZq422ov3Q==}
vite-node@3.2.4:
resolution: {integrity: sha512-EbKSKh+bh1E1IFxeO0pg1n4dvoOTt0UDiXMd/qn++r98+jPO1xtJilvXldeuQ8giIB5IkpjCgMleHMNEsGH6pg==}
engines: {node: ^18.0.0 || ^20.0.0 || >=22.0.0}
hasBin: true
vite@7.3.7:
resolution: {integrity: sha512-AHEzTORMbi7/7aXp7w7KZ5+DCUdSeLsJz0grY8xHpc9DHdYb0vQu6DqJfm2tOw3gDExNgPgnJTXxM6syYFHaCA==}
engines: {node: ^20.19.0 || >=22.12.0}
hasBin: true
peerDependencies:
'@types/node': 20.17.7
jiti: '>=1.21.0'
less: ^4.0.0
lightningcss: ^1.21.0
sass: ^1.70.0
sass-embedded: ^1.70.0
stylus: '>=0.54.8'
sugarss: ^5.0.0
terser: ^5.16.0
tsx: ^4.8.1
yaml: ^2.4.2
peerDependenciesMeta:
'@types/node':
optional: true
jiti:
optional: true
less:
optional: true
lightningcss:
optional: true
sass:
optional: true
sass-embedded:
optional: true
stylus:
optional: true
sugarss:
optional: true
terser:
optional: true
tsx:
optional: true
yaml:
optional: true
vitest@3.2.7:
resolution: {integrity: sha512-KrxIJ62Fd89gfysR4WotlgZABiz2dqFPgqGzX7s+CwsqLFomRH7777ZcrOD6+WVAh7khPQP41A+BKbpcJFrdEg==}
engines: {node: ^18.0.0 || ^20.0.0 || >=22.0.0}
hasBin: true
peerDependencies:
'@edge-runtime/vm': '*'
'@types/debug': ^4.1.12
'@types/node': 20.17.7
'@vitest/browser': 3.2.7
'@vitest/ui': 3.2.7
happy-dom: '*'
jsdom: '*'
peerDependenciesMeta:
'@edge-runtime/vm':
optional: true
'@types/debug':
optional: true
'@types/node':
optional: true
'@vitest/browser':
optional: true
'@vitest/ui':
optional: true
happy-dom:
optional: true
jsdom:
optional: true
vm-browserify@1.1.2:
resolution: {integrity: sha512-2ham8XPWTONajOR0ohOKOHXkm3+gaBmGut3SRuu75xLd/RRaY6vqgh8NBYYk7+RW3u5AtzPQZG8F10LHkl0lAQ==}
......@@ -17860,6 +18325,11 @@ packages:
engines: {node: ^16.13.0 || >=18.0.0}
hasBin: true
why-is-node-running@2.3.0:
resolution: {integrity: sha512-hUrmaWBdVDcxvYqnyh09zunKzROWjbZTiNy8dBEjkS7ehEDQibXJ7XvlmtbwuTclUiIyN+CyXQD4Vmko8fNm8w==}
engines: {node: '>=8'}
hasBin: true
widest-line@3.1.0:
resolution: {integrity: sha512-NsmoXalsWVDMGupxZ5R08ka9flZjjiLvHVAWYOKtiKM8ujtZWr9cRffak+uSE48+Ob8ObalXpwyeUiyDD6QFgg==}
engines: {node: '>=8'}
......@@ -19602,150 +20072,228 @@ snapshots:
'@esbuild/aix-ppc64@0.25.9':
optional: true
'@esbuild/aix-ppc64@0.28.2':
optional: true
'@esbuild/android-arm64@0.23.1':
optional: true
'@esbuild/android-arm64@0.25.9':
optional: true
'@esbuild/android-arm64@0.28.2':
optional: true
'@esbuild/android-arm@0.23.1':
optional: true
'@esbuild/android-arm@0.25.9':
optional: true
'@esbuild/android-arm@0.28.2':
optional: true
'@esbuild/android-x64@0.23.1':
optional: true
'@esbuild/android-x64@0.25.9':
optional: true
'@esbuild/android-x64@0.28.2':
optional: true
'@esbuild/darwin-arm64@0.23.1':
optional: true
'@esbuild/darwin-arm64@0.25.9':
optional: true
'@esbuild/darwin-arm64@0.28.2':
optional: true
'@esbuild/darwin-x64@0.23.1':
optional: true
'@esbuild/darwin-x64@0.25.9':
optional: true
'@esbuild/darwin-x64@0.28.2':
optional: true
'@esbuild/freebsd-arm64@0.23.1':
optional: true
'@esbuild/freebsd-arm64@0.25.9':
optional: true
'@esbuild/freebsd-arm64@0.28.2':
optional: true
'@esbuild/freebsd-x64@0.23.1':
optional: true
'@esbuild/freebsd-x64@0.25.9':
optional: true
'@esbuild/freebsd-x64@0.28.2':
optional: true
'@esbuild/linux-arm64@0.23.1':
optional: true
'@esbuild/linux-arm64@0.25.9':
optional: true
'@esbuild/linux-arm64@0.28.2':
optional: true
'@esbuild/linux-arm@0.23.1':
optional: true
'@esbuild/linux-arm@0.25.9':
optional: true
'@esbuild/linux-arm@0.28.2':
optional: true
'@esbuild/linux-ia32@0.23.1':
optional: true
'@esbuild/linux-ia32@0.25.9':
optional: true
'@esbuild/linux-ia32@0.28.2':
optional: true
'@esbuild/linux-loong64@0.23.1':
optional: true
'@esbuild/linux-loong64@0.25.9':
optional: true
'@esbuild/linux-loong64@0.28.2':
optional: true
'@esbuild/linux-mips64el@0.23.1':
optional: true
'@esbuild/linux-mips64el@0.25.9':
optional: true
'@esbuild/linux-mips64el@0.28.2':
optional: true
'@esbuild/linux-ppc64@0.23.1':
optional: true
'@esbuild/linux-ppc64@0.25.9':
optional: true
'@esbuild/linux-ppc64@0.28.2':
optional: true
'@esbuild/linux-riscv64@0.23.1':
optional: true
'@esbuild/linux-riscv64@0.25.9':
optional: true
'@esbuild/linux-riscv64@0.28.2':
optional: true
'@esbuild/linux-s390x@0.23.1':
optional: true
'@esbuild/linux-s390x@0.25.9':
optional: true
'@esbuild/linux-s390x@0.28.2':
optional: true
'@esbuild/linux-x64@0.23.1':
optional: true
'@esbuild/linux-x64@0.25.9':
optional: true
'@esbuild/linux-x64@0.28.2':
optional: true
'@esbuild/netbsd-arm64@0.25.9':
optional: true
'@esbuild/netbsd-arm64@0.28.2':
optional: true
'@esbuild/netbsd-x64@0.23.1':
optional: true
'@esbuild/netbsd-x64@0.25.9':
optional: true
'@esbuild/netbsd-x64@0.28.2':
optional: true
'@esbuild/openbsd-arm64@0.23.1':
optional: true
'@esbuild/openbsd-arm64@0.25.9':
optional: true
'@esbuild/openbsd-arm64@0.28.2':
optional: true
'@esbuild/openbsd-x64@0.23.1':
optional: true
'@esbuild/openbsd-x64@0.25.9':
optional: true
'@esbuild/openbsd-x64@0.28.2':
optional: true
'@esbuild/openharmony-arm64@0.25.9':
optional: true
'@esbuild/openharmony-arm64@0.28.2':
optional: true
'@esbuild/sunos-x64@0.23.1':
optional: true
'@esbuild/sunos-x64@0.25.9':
optional: true
'@esbuild/sunos-x64@0.28.2':
optional: true
'@esbuild/win32-arm64@0.23.1':
optional: true
'@esbuild/win32-arm64@0.25.9':
optional: true
'@esbuild/win32-arm64@0.28.2':
optional: true
'@esbuild/win32-ia32@0.23.1':
optional: true
'@esbuild/win32-ia32@0.25.9':
optional: true
'@esbuild/win32-ia32@0.28.2':
optional: true
'@esbuild/win32-x64@0.23.1':
optional: true
'@esbuild/win32-x64@0.25.9':
optional: true
'@esbuild/win32-x64@0.28.2':
optional: true
'@eslint-community/eslint-utils@4.9.0(eslint@9.37.0(jiti@2.6.1)(supports-color@8.1.1))':
dependencies:
eslint: 9.37.0(jiti@2.6.1)(supports-color@8.1.1)
......@@ -22549,6 +23097,81 @@ snapshots:
optionalDependencies:
rollup: 2.35.1
'@rollup/rollup-android-arm-eabi@4.65.0':
optional: true
'@rollup/rollup-android-arm64@4.65.0':
optional: true
'@rollup/rollup-darwin-arm64@4.65.0':
optional: true
'@rollup/rollup-darwin-x64@4.65.0':
optional: true
'@rollup/rollup-freebsd-arm64@4.65.0':
optional: true
'@rollup/rollup-freebsd-x64@4.65.0':
optional: true
'@rollup/rollup-linux-arm-gnueabihf@4.65.0':
optional: true
'@rollup/rollup-linux-arm-musleabihf@4.65.0':
optional: true
'@rollup/rollup-linux-arm64-gnu@4.65.0':
optional: true
'@rollup/rollup-linux-arm64-musl@4.65.0':
optional: true
'@rollup/rollup-linux-loong64-gnu@4.65.0':
optional: true
'@rollup/rollup-linux-loong64-musl@4.65.0':
optional: true
'@rollup/rollup-linux-ppc64-gnu@4.65.0':
optional: true
'@rollup/rollup-linux-ppc64-musl@4.65.0':
optional: true
'@rollup/rollup-linux-riscv64-gnu@4.65.0':
optional: true
'@rollup/rollup-linux-riscv64-musl@4.65.0':
optional: true
'@rollup/rollup-linux-s390x-gnu@4.65.0':
optional: true
'@rollup/rollup-linux-x64-gnu@4.65.0':
optional: true
'@rollup/rollup-linux-x64-musl@4.65.0':
optional: true
'@rollup/rollup-openbsd-x64@4.65.0':
optional: true
'@rollup/rollup-openharmony-arm64@4.65.0':
optional: true
'@rollup/rollup-win32-arm64-msvc@4.65.0':
optional: true
'@rollup/rollup-win32-ia32-msvc@4.65.0':
optional: true
'@rollup/rollup-win32-x64-gnu@4.65.0':
optional: true
'@rollup/rollup-win32-x64-msvc@4.65.0':
optional: true
'@rspack/binding-darwin-arm64@1.6.7':
optional: true
......@@ -23256,7 +23879,7 @@ snapshots:
lz-string: 1.5.0
pretty-format: 27.5.1
'@testing-library/jest-dom@6.1.2(@jest/globals@29.7.0(supports-color@8.1.1))(@types/jest@29.5.5)(jest@29.7.0(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(babel-plugin-macros@3.1.0)(supports-color@8.1.1))':
'@testing-library/jest-dom@6.1.2(@jest/globals@29.7.0(supports-color@8.1.1))(@types/jest@29.5.5)(jest@29.7.0(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(babel-plugin-macros@3.1.0)(supports-color@8.1.1))(vitest@3.2.7(@edge-runtime/vm@5.0.0)(@types/debug@4.1.13)(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(jsdom@20.0.3(supports-color@8.1.1))(lightningcss@1.30.1)(sass@1.54.0)(supports-color@8.1.1)(terser@5.39.0)(tsx@4.19.2))':
dependencies:
'@adobe/css-tools': 4.4.1
'@babel/runtime': 7.27.0
......@@ -23270,6 +23893,7 @@ snapshots:
'@jest/globals': 29.7.0(supports-color@8.1.1)
'@types/jest': 29.5.5
jest: 29.7.0(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(babel-plugin-macros@3.1.0)(supports-color@8.1.1)
vitest: 3.2.7(@edge-runtime/vm@5.0.0)(@types/debug@4.1.13)(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(jsdom@20.0.3(supports-color@8.1.1))(lightningcss@1.30.1)(sass@1.54.0)(supports-color@8.1.1)(terser@5.39.0)(tsx@4.19.2)
'@testing-library/jest-dom@6.5.0':
dependencies:
......@@ -23383,6 +24007,11 @@ snapshots:
'@types/bytes@3.1.1': {}
'@types/chai@5.2.3':
dependencies:
'@types/deep-eql': 4.0.2
assertion-error: 2.0.1
'@types/cheerio@0.22.16':
dependencies:
'@types/node': 20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab)
......@@ -23432,6 +24061,8 @@ snapshots:
'@types/debug@4.1.5': {}
'@types/deep-eql@4.0.2': {}
'@types/doctrine@0.0.9': {}
'@types/eslint-plugin-jsx-a11y@6.10.1(jiti@2.6.1)(supports-color@9.4.0)':
......@@ -23461,6 +24092,8 @@ snapshots:
'@types/estree@1.0.8': {}
'@types/estree@1.0.9': {}
'@types/events@3.0.0': {}
'@types/express-serve-static-core@4.17.33':
......@@ -24107,7 +24740,7 @@ snapshots:
dependencies:
uncrypto: 0.1.3
'@vercel/agent-eval@2.2.1(patch_hash=c453f42ffcf4e7e070b8cba54135cd4788d4df5fdb829e0065f453fd1646e5c1)(supports-color@8.1.1)':
'@vercel/agent-eval@2.2.1(patch_hash=f89d8e50c6627841d89b2307f56a606d7a8c179061e145d95c3213f51e2d8b82)(supports-color@8.1.1)':
dependencies:
'@ai-sdk/anthropic': 1.2.12(zod@3.25.76)
'@vercel/sandbox': 1.8.0
......@@ -24206,6 +24839,22 @@ snapshots:
chai: 5.3.3
tinyrainbow: 1.2.0
'@vitest/expect@3.2.7':
dependencies:
'@types/chai': 5.2.3
'@vitest/spy': 3.2.7
'@vitest/utils': 3.2.7
chai: 5.3.3
tinyrainbow: 2.0.0
'@vitest/mocker@3.2.7(vite@7.3.7(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(terser@5.39.0)(tsx@4.19.2))':
dependencies:
'@vitest/spy': 3.2.7
estree-walker: 3.0.3
magic-string: 0.30.19
optionalDependencies:
vite: 7.3.7(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(terser@5.39.0)(tsx@4.19.2)
'@vitest/pretty-format@2.0.5':
dependencies:
tinyrainbow: 1.2.0
......@@ -24214,10 +24863,30 @@ snapshots:
dependencies:
tinyrainbow: 1.2.0
'@vitest/pretty-format@3.2.7':
dependencies:
tinyrainbow: 2.0.0
'@vitest/runner@3.2.7':
dependencies:
'@vitest/utils': 3.2.7
pathe: 2.0.3
strip-literal: 3.1.0
'@vitest/snapshot@3.2.7':
dependencies:
'@vitest/pretty-format': 3.2.7
magic-string: 0.30.19
pathe: 2.0.3
'@vitest/spy@2.0.5':
dependencies:
tinyspy: 3.0.2
'@vitest/spy@3.2.7':
dependencies:
tinyspy: 4.0.6
'@vitest/utils@2.0.5':
dependencies:
'@vitest/pretty-format': 2.0.5
......@@ -24231,6 +24900,12 @@ snapshots:
loupe: 3.2.1
tinyrainbow: 1.2.0
'@vitest/utils@3.2.7':
dependencies:
'@vitest/pretty-format': 3.2.7
loupe: 3.2.1
tinyrainbow: 2.0.0
'@webassemblyjs/ast@1.14.1':
dependencies:
'@webassemblyjs/helper-numbers': 1.13.2
......@@ -25291,6 +25966,8 @@ snapshots:
bytes@3.1.2: {}
cac@6.7.14: {}
cacheable-request@6.1.0:
dependencies:
clone-response: 1.0.2
......@@ -27426,6 +28103,35 @@ snapshots:
'@esbuild/win32-ia32': 0.25.9
'@esbuild/win32-x64': 0.25.9
esbuild@0.28.2:
optionalDependencies:
'@esbuild/aix-ppc64': 0.28.2
'@esbuild/android-arm': 0.28.2
'@esbuild/android-arm64': 0.28.2
'@esbuild/android-x64': 0.28.2
'@esbuild/darwin-arm64': 0.28.2
'@esbuild/darwin-x64': 0.28.2
'@esbuild/freebsd-arm64': 0.28.2
'@esbuild/freebsd-x64': 0.28.2
'@esbuild/linux-arm': 0.28.2
'@esbuild/linux-arm64': 0.28.2
'@esbuild/linux-ia32': 0.28.2
'@esbuild/linux-loong64': 0.28.2
'@esbuild/linux-mips64el': 0.28.2
'@esbuild/linux-ppc64': 0.28.2
'@esbuild/linux-riscv64': 0.28.2
'@esbuild/linux-s390x': 0.28.2
'@esbuild/linux-x64': 0.28.2
'@esbuild/netbsd-arm64': 0.28.2
'@esbuild/netbsd-x64': 0.28.2
'@esbuild/openbsd-arm64': 0.28.2
'@esbuild/openbsd-x64': 0.28.2
'@esbuild/openharmony-arm64': 0.28.2
'@esbuild/sunos-x64': 0.28.2
'@esbuild/win32-arm64': 0.28.2
'@esbuild/win32-ia32': 0.28.2
'@esbuild/win32-x64': 0.28.2
escalade@3.1.1: {}
escalade@3.2.0: {}
......@@ -28134,6 +28840,8 @@ snapshots:
expect-type@0.14.2: {}
expect-type@1.4.0: {}
expect@29.5.0:
dependencies:
'@jest/expect-utils': 29.7.0
......@@ -30762,6 +31470,8 @@ snapshots:
js-tokens@4.0.0: {}
js-tokens@9.0.1: {}
js-yaml@3.14.1:
dependencies:
argparse: 1.0.10
......@@ -33262,6 +33972,8 @@ snapshots:
path-type@4.0.0: {}
pathe@2.0.3: {}
pathval@2.0.0: {}
pause-stream@0.0.11:
......@@ -35067,6 +35779,38 @@ snapshots:
optionalDependencies:
fsevents: 2.1.3
rollup@4.65.0:
dependencies:
'@types/estree': 1.0.9
optionalDependencies:
'@napi-rs/lzma-linux-x64-gnu': 1.5.1
'@rollup/rollup-android-arm-eabi': 4.65.0
'@rollup/rollup-android-arm64': 4.65.0
'@rollup/rollup-darwin-arm64': 4.65.0
'@rollup/rollup-darwin-x64': 4.65.0
'@rollup/rollup-freebsd-arm64': 4.65.0
'@rollup/rollup-freebsd-x64': 4.65.0
'@rollup/rollup-linux-arm-gnueabihf': 4.65.0
'@rollup/rollup-linux-arm-musleabihf': 4.65.0
'@rollup/rollup-linux-arm64-gnu': 4.65.0
'@rollup/rollup-linux-arm64-musl': 4.65.0
'@rollup/rollup-linux-loong64-gnu': 4.65.0
'@rollup/rollup-linux-loong64-musl': 4.65.0
'@rollup/rollup-linux-ppc64-gnu': 4.65.0
'@rollup/rollup-linux-ppc64-musl': 4.65.0
'@rollup/rollup-linux-riscv64-gnu': 4.65.0
'@rollup/rollup-linux-riscv64-musl': 4.65.0
'@rollup/rollup-linux-s390x-gnu': 4.65.0
'@rollup/rollup-linux-x64-gnu': 4.65.0
'@rollup/rollup-linux-x64-musl': 4.65.0
'@rollup/rollup-openbsd-x64': 4.65.0
'@rollup/rollup-openharmony-arm64': 4.65.0
'@rollup/rollup-win32-arm64-msvc': 4.65.0
'@rollup/rollup-win32-ia32-msvc': 4.65.0
'@rollup/rollup-win32-x64-gnu': 4.65.0
'@rollup/rollup-win32-x64-msvc': 4.65.0
fsevents: 2.3.3
router@2.2.0(supports-color@9.4.0):
dependencies:
debug: 4.4.3(supports-color@9.4.0)
......@@ -35534,6 +36278,8 @@ snapshots:
side-channel-map: 1.0.1
side-channel-weakmap: 1.0.2
siginfo@2.0.0: {}
signal-exit@3.0.3: {}
signal-exit@3.0.7: {}
......@@ -35764,6 +36510,8 @@ snapshots:
dependencies:
escape-string-regexp: 2.0.0
stackback@0.0.2: {}
stacktrace-parser@0.1.10(patch_hash=ff70befa8ec032ce77713d8cf9b1cef70ccb047d635eddb4072bc72a6c744522):
dependencies:
type-fest: 0.7.1
......@@ -35774,6 +36522,8 @@ snapshots:
statuses@2.0.1: {}
std-env@3.10.0: {}
stop-iteration-iterator@1.1.0:
dependencies:
es-errors: 1.3.0
......@@ -35995,6 +36745,10 @@ snapshots:
strip-json-comments@3.1.1: {}
strip-literal@3.1.0:
dependencies:
js-tokens: 9.0.1
stubborn-fs@1.2.5: {}
style-inject@0.3.0: {}
......@@ -36374,17 +37128,27 @@ snapshots:
tiny-invariant@1.3.3: {}
tinybench@2.9.0: {}
tinydate@1.3.0: {}
tinyexec@0.3.2: {}
tinyglobby@0.2.15:
dependencies:
fdir: 6.5.0(picomatch@4.0.3)
picomatch: 4.0.3
tinypool@1.1.1: {}
tinyrainbow@1.2.0: {}
tinyrainbow@2.0.0: {}
tinyspy@3.0.2: {}
tinyspy@4.0.6: {}
title-case@3.0.3:
dependencies:
tslib: 2.8.1
......@@ -37235,6 +37999,88 @@ snapshots:
'@types/unist': 3.0.3
vfile-message: 4.0.2
vite-node@3.2.4(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(supports-color@8.1.1)(terser@5.39.0)(tsx@4.19.2):
dependencies:
cac: 6.7.14
debug: 4.4.3(supports-color@8.1.1)
es-module-lexer: 1.7.0
pathe: 2.0.3
vite: 7.3.7(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(terser@5.39.0)(tsx@4.19.2)
transitivePeerDependencies:
- '@types/node'
- jiti
- less
- lightningcss
- sass
- sass-embedded
- stylus
- sugarss
- supports-color
- terser
- tsx
- yaml
vite@7.3.7(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(terser@5.39.0)(tsx@4.19.2):
dependencies:
esbuild: 0.28.2
fdir: 6.5.0(picomatch@4.0.3)
picomatch: 4.0.3
postcss: 8.5.23
rollup: 4.65.0
tinyglobby: 0.2.15
optionalDependencies:
'@types/node': 20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab)
fsevents: 2.3.3
jiti: 2.6.1
lightningcss: 1.30.1
sass: 1.54.0
terser: 5.39.0
tsx: 4.19.2
vitest@3.2.7(@edge-runtime/vm@5.0.0)(@types/debug@4.1.13)(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(jsdom@20.0.3(supports-color@8.1.1))(lightningcss@1.30.1)(sass@1.54.0)(supports-color@8.1.1)(terser@5.39.0)(tsx@4.19.2):
dependencies:
'@types/chai': 5.2.3
'@vitest/expect': 3.2.7
'@vitest/mocker': 3.2.7(vite@7.3.7(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(terser@5.39.0)(tsx@4.19.2))
'@vitest/pretty-format': 3.2.7
'@vitest/runner': 3.2.7
'@vitest/snapshot': 3.2.7
'@vitest/spy': 3.2.7
'@vitest/utils': 3.2.7
chai: 5.3.3
debug: 4.4.3(supports-color@8.1.1)
expect-type: 1.4.0
magic-string: 0.30.19
pathe: 2.0.3
picomatch: 4.0.3
std-env: 3.10.0
tinybench: 2.9.0
tinyexec: 0.3.2
tinyglobby: 0.2.15
tinypool: 1.1.1
tinyrainbow: 2.0.0
vite: 7.3.7(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(terser@5.39.0)(tsx@4.19.2)
vite-node: 3.2.4(@types/node@20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab))(jiti@2.6.1)(lightningcss@1.30.1)(sass@1.54.0)(supports-color@8.1.1)(terser@5.39.0)(tsx@4.19.2)
why-is-node-running: 2.3.0
optionalDependencies:
'@edge-runtime/vm': 5.0.0
'@types/debug': 4.1.13
'@types/node': 20.17.7(patch_hash=fe6957869a9b616ec526ededb72bbe846b863bd2349204218772df5c1e62d6ab)
jsdom: 20.0.3(supports-color@8.1.1)
transitivePeerDependencies:
- jiti
- less
- lightningcss
- msw
- sass
- sass-embedded
- stylus
- sugarss
- supports-color
- terser
- tsx
- yaml
vm-browserify@1.1.2: {}
w3c-xmlserializer@4.0.0:
......@@ -37565,6 +38411,11 @@ snapshots:
dependencies:
isexe: 3.1.1
why-is-node-running@2.3.0:
dependencies:
siginfo: 2.0.0
stackback: 0.0.2
widest-line@3.1.0:
dependencies:
string-width: 4.2.3
......@@ -36,7 +36,7 @@ const EXPERIMENTS_DIR = path.join(EVALS_DIR, 'experiments')
const TARBALL_DIR = path.join(EVALS_DIR, '.tarballs')
const TARBALL = path.join(TARBALL_DIR, 'next.tgz')
/** @typedef {{ skills?: string[], timeout?: number, agentFeedback?: boolean, publish?: boolean }} EvalConfig */
/** @typedef {{ skills?: string[], timeout?: number, agentFeedback?: boolean, publish?: boolean, browserJs?: boolean }} EvalConfig */
/** @type {Record<string, EvalConfig>} */
const EVAL_CONFIG = JSON.parse(fs.readFileSync(EVAL_CONFIG_PATH, 'utf-8'))
......@@ -70,8 +70,30 @@ function writeExperiments(selectedEvals, variants, timeout, runs) {
const evalsField = variantEvals
? `\n evals: ${JSON.stringify(variantEvals.length === 1 ? variantEvals[0] : variantEvals)},`
: ''
const browserJsFixtures = Object.fromEntries(
(variantEvals ?? listEvals())
.filter((name) => readFixtureConfig(name).browserJs)
.map((name) => {
const pkg = JSON.parse(
fs.readFileSync(
path.join(FIXTURES_DIR, name, 'package.json'),
'utf8'
)
)
return [name, pkg.name]
})
)
const measuresBrowserJs = Object.keys(browserJsFixtures).length > 0
let onRunComplete = v.onRunComplete
if (measuresBrowserJs) {
const runData = onRunComplete
? `${onRunComplete}(context)`
: 'context.runData'
onRunComplete = `(context) => browserJs.onRunComplete({ ...context, runData: ${runData} })`
}
const body = `import type { ExperimentConfig } from '@vercel/agent-eval'
${v.imports}
${measuresBrowserJs ? `import { createBrowserJsHooks } from '../lib/bundle-optimizer/hooks.js'\n\nconst browserJs = createBrowserJsHooks(${JSON.stringify(browserJsFixtures)}, ${JSON.stringify(v.skills ?? [])})\n` : ''}
const config: ExperimentConfig = {
// Via the Vercel AI Gateway, so the OIDC token from \`vc env pull\` is the only
......@@ -86,9 +108,10 @@ const config: ExperimentConfig = {
earlyExit: ${runs === 1},
timeout: ${timeout},
sandbox: 'auto',
${v.onRunComplete ? `onRunComplete: ${v.onRunComplete},` : ''}
${onRunComplete ? `onRunComplete: ${onRunComplete},` : ''}
setup: async (sandbox) => {
${v.setup}
${measuresBrowserJs ? 'return browserJs.setup(sandbox)' : ''}
},
}
......@@ -176,10 +199,16 @@ function readFixtureConfig(evalName) {
`${EVAL_CONFIG_PATH}: ${evalName}.agentFeedback must be a boolean`
)
}
if (config.browserJs !== undefined && typeof config.browserJs !== 'boolean') {
throw new Error(
`${EVAL_CONFIG_PATH}: ${evalName}.browserJs must be a boolean`
)
}
return {
skills: skillNames,
timeout: config.timeout ?? 720,
agentFeedback: config.agentFeedback ?? false,
browserJs: config.browserJs ?? false,
}
}
......@@ -208,6 +237,7 @@ function getExperimentSettings(selectedEvals) {
suffix: multipleSkillGroups ? `skills-${skills.join('-')}` : 'skills',
imports: `import { installLocalSkills, installNextJs, installPlaywright, prepareFixture } from '../lib/setup.js'`,
setup: `await installNextJs(sandbox)\n await installPlaywright(sandbox)\n await prepareFixture(sandbox)\n await installLocalSkills(sandbox, ${JSON.stringify(skills)})`,
skills,
evals,
}))
......
import { EventEmitter } from 'node:events'
import { spawnSync } from 'node:child_process'
import { dirname, join } from 'node:path'
import { pathToFileURL } from 'node:url'
import type { Browser, Page, Request } from 'playwright/test'
import { withMeasuredPage } from '../../../evals/lib/bundle-optimizer/browser-js'
function deferred<T>() {
let resolve!: (value: T) => void
const promise = new Promise<T>((done) => {
resolve = done
})
return { promise, resolve }
}
function measuredBrowser() {
const page = Object.assign(new EventEmitter(), {
goto: jest.fn(async () => ({ ok: () => true })),
})
const close = jest.fn()
const context = {
newPage: async () => page as unknown as Page,
newCDPSession: async () => ({ send: async () => {} }),
close,
}
const browser = { newContext: async () => context } as unknown as Browser
return { browser, page, close }
}
function script(url: string, body = 'void 0') {
const finished = deferred<null>()
const decoded = deferred<Buffer>()
const request = {
resourceType: () => 'script',
url: () => url,
sizes: async () => ({ responseBodySize: 10 }),
} as unknown as Request
const response = {
request: () => request,
url: () => url,
headers: () => ({ 'content-type': 'application/javascript' }),
ok: () => true,
finished: () => finished.promise,
body: () => decoded.promise,
}
return { request, response, finished, decoded, body }
}
describe('browser JavaScript snapshots', () => {
beforeEach(() => jest.useFakeTimers())
afterEach(() => jest.useRealTimers())
it('waits for headers, bodies, and scripts requested by a loader', async () => {
const { browser, page, close } = measuredBrowser()
const loader = script('http://app/loader.js')
const editor = script('http://app/editor.js', 'cm-content')
let settled = false
const ready = deferred<void>()
const result = withMeasuredPage(
browser,
'http://app',
async (_, snapshot) => {
page.emit('request', loader.request)
ready.resolve()
const summary = await snapshot()
settled = true
return summary
},
(body) => ({ editor: body.includes('cm-content') })
)
await ready.promise
await jest.advanceTimersByTimeAsync(1000)
expect(settled).toBe(false)
page.emit('response', loader.response)
loader.finished.resolve(null)
page.emit('requestfinished', loader.request)
await jest.advanceTimersByTimeAsync(1000)
expect(settled).toBe(false)
page.emit('request', editor.request)
loader.decoded.resolve(Buffer.from(loader.body))
await jest.advanceTimersByTimeAsync(1000)
expect(settled).toBe(false)
page.emit('response', editor.response)
editor.finished.resolve(null)
editor.decoded.resolve(Buffer.from(editor.body))
page.emit('requestfinished', editor.request)
await jest.advanceTimersByTimeAsync(2000)
expect(await result).toMatchObject({
encodedBytes: 20,
requests: [
{ url: 'http://app/loader.js' },
{ url: 'http://app/editor.js', editor: true },
],
})
expect(close).toHaveBeenCalledTimes(1)
})
it('reports failed requests and closes the context', async () => {
const { browser, page, close } = measuredBrowser()
const failed = script('http://app/broken.js')
const ready = deferred<void>()
const result = withMeasuredPage(
browser,
'http://app',
async (_, snapshot) => {
page.emit('request', failed.request)
page.emit('requestfailed', failed.request)
ready.resolve()
return snapshot()
}
)
const rejected = result.catch((error) => error)
await ready.promise
await jest.advanceTimersByTimeAsync(2000)
await expect(rejected).resolves.toMatchObject({
message: expect.stringContaining('http://app/broken.js'),
})
expect(close).toHaveBeenCalledTimes(1)
})
})
it('withholds grader directories and rejects leaked resources before the agent runs', () => {
const sdk = dirname(require.resolve('@vercel/agent-eval/package.json'))
const result = spawnSync(
process.execPath,
[
'--input-type=module',
'--eval',
`
import assert from 'node:assert/strict'
import { splitTestFiles, verifyNoTestFiles } from ${JSON.stringify(pathToFileURL(join(sdk, 'dist/lib/sandbox.js')).href)}
const files = ['app/page.tsx', 'EVAL.ts', '__eval__/measure.ts', '__eval__/nested/helper.ts', 'app/__eval__/helper.ts', 'app/__eval__-visible.ts'].map(path => ({ path, content: Buffer.from('') }))
const { workspaceFiles, testFiles } = splitTestFiles(files)
assert.deepEqual(workspaceFiles.map(file => file.path), ['app/page.tsx', 'app/__eval__-visible.ts'])
assert.equal(testFiles.length, 4)
await assert.rejects(verifyNoTestFiles({ runShell: async () => ({ stdout: './__eval__', exitCode: 0 }) }), /Test files found/)
`,
],
{ encoding: 'utf8' }
)
expect(result.status).toBe(0)
expect(result.stderr).toBe('')
})
it('restores trusted graders and scopes browser measurement hooks to configured fixtures', () => {
const result = spawnSync(
process.execPath,
[join(__dirname, 'sdk-lifecycle.cjs')],
{
cwd: join(__dirname, '../../..'),
encoding: 'utf8',
}
)
expect(result.stderr).toBe('')
expect(result.status).toBe(0)
})
const fs = require('node:fs')
const path = require('node:path')
const { tmpdir } = require('node:os')
const { pathToFileURL } = require('node:url')
const { createRequire } = require('node:module')
const assert = require('node:assert/strict')
const vm = require('node:vm')
const taskRequire = createRequire(process.cwd() + '/package.json')
const { build } = taskRequire('esbuild')
const sdk = path.dirname(taskRequire.resolve('@vercel/agent-eval/package.json'))
const directory = fs.mkdtempSync(
path.join(tmpdir(), 'next-eval-sdk-lifecycle-')
)
const fixture = path.join(directory, 'fixture')
const modulePath = path.join(directory, 'agents/plugin/orchestrator.mjs')
const sandboxUrl = pathToFileURL(path.join(sdk, 'dist/lib/sandbox.js')).href
const state = {
snapshots: new Map(),
initial: [],
agentRuns: 0,
validations: 0,
installs: 0,
captures: 0,
beforeValidations: 0,
afterValidations: 0,
}
globalThis.__evalSdkTest = state
function makeSandbox(options) {
const files = new Map(
options.snapshotId ? state.snapshots.get(options.snapshotId) : []
)
return {
files,
sandboxId: 'local-test',
getWorkingDirectory: () => fixture,
setWorkingDirectory: () => {},
uploadFiles: async (inputs) => {
for (const file of inputs) files.set(file.path, file.content.toString())
},
writeFiles: async (inputs) => {
for (const [name, source] of Object.entries(inputs))
files.set(name, source)
},
readFile: async (name) => files.get(name),
stop: async () => {},
snapshot: async () => {
assert(!files.has('EVAL.ts'))
assert(![...files.keys()].some((name) => name.includes('__eval__/')))
state.snapshots.set('prepared', new Map(files))
return {
snapshotId: 'prepared',
delete: async () => state.snapshots.delete('prepared'),
}
},
runCommand: async (cmd, args, options = {}) => {
assert.equal(options.env.VALIDATION_FLAG, undefined)
if (cmd === 'install-check') {
state.installs++
return { exitCode: 0 }
}
assert.equal(args[0], '__agent_eval__/run.mjs')
assert.equal(options.env.PUBLIC_FLAG, '1')
assert(!files.has('EVAL.ts'))
assert(![...files.keys()].some((name) => name.includes('__eval__/')))
state.agentRuns++
files.set('EVAL.ts', 'tampered')
files.set('__eval__/measurement.ts', 'tampered')
files.set(
'__agent_eval__/agent-result.json',
JSON.stringify({ ok: true, output: 'stub', transcript: '' })
)
return { exitCode: 0, stdout: '', stderr: '' }
},
}
}
state.makeSandbox = makeSandbox
const mocks = {
'../../sandbox.js': `import { collectLocalFiles, splitTestFiles } from ${JSON.stringify(sandboxUrl)};
export { collectLocalFiles, splitTestFiles };
export const createSandbox = async options => globalThis.__evalSdkTest.makeSandbox(options);
export const verifyNoTestFiles = async sandbox => { if ([...sandbox.files.keys()].some(name => name === 'EVAL.ts' || name.includes('__eval__/'))) throw Error('Leaked grader'); };`,
'../shared.js': `import assert from 'node:assert/strict';
export const initGitAndCommit = async sandbox => { const files = [...sandbox.files.keys()]; assert.deepEqual(files, ['app/page.tsx']); globalThis.__evalSdkTest.initial.push(files); };
export const prepareNeutralWorkspace = async sandbox => ({ cwd: sandbox.getWorkingDirectory(), env: {} });
export const createVitestConfig = async sandbox => { assert.equal(sandbox.files.get('EVAL.ts'), 'trusted bundled grader'); assert.equal(sandbox.files.get('__eval__/measurement.ts'), 'trusted original measurement'); };
export const runValidation = async (sandbox, scripts, mode, env) => { assert.equal(env.VALIDATION_FLAG, '1'); assert.equal(env.PUBLIC_FLAG, '1'); assert.equal(sandbox.files.get('__agent_eval__/baseline.json'), 'trusted baseline'); globalThis.__evalSdkTest.validations++; return { allPassed: true, scripts: {} }; };
export const captureGeneratedFiles = async () => ({ generatedFiles: {}, deletedFiles: [] });
export const injectTranscriptContext = async () => {};
export const resolveAgentApiKey = () => undefined;
export const EVAL_HELPER_PATH = '__agent_eval__/eval-helper.mjs';
export const JUDGE_TRANSCRIPT_FILE = '__agent_eval__/transcript';
export const JUDGE_CONFIG_PATH = '__agent_eval__/judge-config';
export const JUDGE_RUNNER_PATH = '__agent_eval__/judge-runner';`,
'../redact.js': 'export const redactRunResult = result => result;',
'../registry.js':
'export const getAgent = () => { throw Error("Unexpected judge"); };',
'./contract.js':
'export const assertBundledSkillsControl = () => {}; export const assertCrossAgentJudgeSupport = () => {}; export const assertWebResearchControl = () => {};',
}
;(async () => {
fs.mkdirSync(path.join(fixture, 'app'), { recursive: true })
fs.mkdirSync(path.join(fixture, '__eval__'))
fs.writeFileSync(path.join(fixture, 'app/page.tsx'), 'app')
fs.writeFileSync(path.join(fixture, 'EVAL.ts'), 'original grader')
fs.writeFileSync(
path.join(fixture, '__eval__/measurement.ts'),
'trusted original measurement'
)
fs.mkdirSync(path.dirname(modulePath), { recursive: true })
fs.writeFileSync(path.join(directory, 'agents/eval-helper.mjs'), '')
fs.writeFileSync(path.join(directory, 'runner.mjs'), '')
await build({
entryPoints: [path.join(sdk, 'dist/lib/agents/plugin/orchestrator.js')],
outfile: modulePath,
bundle: true,
platform: 'node',
format: 'esm',
packages: 'external',
external: [sandboxUrl],
plugins: [
{
name: 'offline-sdk-boundaries',
setup(builder) {
builder.onResolve({ filter: /./ }, (args) =>
args.importer
.replace(/\\/g, '/')
.endsWith('/plugin/orchestrator.js') && mocks[args.path]
? { path: args.path, namespace: 'sdk-test' }
: undefined
)
builder.onLoad({ filter: /./, namespace: 'sdk-test' }, (args) => ({
contents: mocks[args.path],
loader: 'js',
}))
},
},
],
})
const { runWithDefinition } = await import(pathToFileURL(modulePath).href)
const def = {
name: 'offline',
displayName: 'offline',
o11yAgentName: 'offline',
runnerPath: path.join(directory, 'runner.mjs'),
install: () => [{ kind: 'command', cmd: 'install-check', args: [] }],
configFiles: () => [],
authEnv: () => ({}),
}
const options = {
model: 'offline',
prompt: 'offline',
validation: 'vitest',
agentOptions: {
afterAgent: async (sandbox) => {
assert.equal(sandbox.files.get('EVAL.ts'), 'tampered')
state.captures++
},
beforeValidation: async (sandbox) => {
assert.equal(sandbox.files.get('EVAL.ts'), 'tampered')
state.beforeValidations++
},
afterValidation: async (sandbox) => {
assert.equal(sandbox.files.get('EVAL.ts'), 'trusted bundled grader')
state.afterValidations++
},
},
setup: async () => ({
env: { PUBLIC_FLAG: '1' },
validationFiles: {
'EVAL.ts': 'trusted bundled grader',
'__agent_eval__/baseline.json': 'trusted baseline',
},
validationEnv: { VALIDATION_FLAG: '1' },
}),
}
delete process.env.AGENT_EVAL_PREPARE_FIXTURE_ONCE
const regular = await runWithDefinition(def, fixture, options)
assert.equal(regular.success, true, regular.error)
process.env.AGENT_EVAL_PREPARE_FIXTURE_ONCE = '1'
process.env.AGENT_EVAL_PREPARED_FIXTURE_CONSUMERS = '1'
const prepared = await runWithDefinition(def, fixture, options)
assert.equal(prepared.success, true, prepared.error)
assert.equal(state.agentRuns, 2)
assert.equal(state.validations, 2)
assert.equal(state.initial.length, 2)
assert.equal(state.installs, 1)
assert.equal(state.captures, 2)
assert.equal(state.beforeValidations, 2)
assert.equal(state.afterValidations, 2)
const experiments = new Map()
const runner = { exports: {} }
vm.runInNewContext(
fs
.readFileSync(path.join(process.cwd(), 'run-evals.js'), 'utf8')
.replace(
/\nmain\(\)\s*$/,
'\nmodule.exports = { readEvalSuites, getExperimentSettings, writeExperiments }'
),
{
__dirname: process.cwd(),
module: runner,
require: (name) =>
name === 'fs'
? {
...fs,
rmSync: () => {},
mkdirSync: () => {},
writeFileSync: (name, content) =>
experiments.set(path.basename(name), content),
}
: taskRequire(name),
}
)
runner.exports.readEvalSuites()
const selectedEvals = [
'agent-059-bundle-optimizer-next-dynamic',
'agent-046-adopt-partial-prefetching',
]
const { variants, timeout } =
runner.exports.getExperimentSettings(selectedEvals)
runner.exports.writeExperiments(selectedEvals, variants, timeout, 1)
assert.match(experiments.get('baseline.ts'), /createBrowserJsHooks/)
assert.match(
experiments.get('skills-next-bundle-optimizer.ts'),
/agent-059-bundle-optimizer-next-dynamic/
)
assert.doesNotMatch(
experiments.get('skills-next-partial-prefetching-adoption.ts'),
/createBrowserJsHooks|agent-059-bundle-optimizer-next-dynamic/
)
fs.symlinkSync(
path.join(process.cwd(), 'node_modules'),
path.join(directory, 'node_modules'),
'junction'
)
const hooksPath = path.join(directory, 'browser-js-hooks.mjs')
await build({
entryPoints: [
path.join(process.cwd(), 'evals/lib/bundle-optimizer/hooks.ts'),
],
outfile: hooksPath,
bundle: true,
packages: 'external',
platform: 'node',
format: 'esm',
})
const { createBrowserJsHooks } = await import(pathToFileURL(hooksPath).href)
const measurement = {
before: {
initial: {
encodedBytes: 200,
decodedBytes: 400,
requests: [{ url: '/before.js' }],
},
},
after: {
initial: {
encodedBytes: 100,
decodedBytes: 200,
requests: [{ url: '/after.js' }],
},
},
budgetBytes: 150,
savedBytes: 100,
}
const evalName = 'agent-059-bundle-optimizer-next-dynamic'
const hooks = createBrowserJsHooks({ [evalName]: 'renamed-package' }, [])
const files = new Map()
const sandbox = {
readFile: async (name) =>
name === 'package.json'
? JSON.stringify({ name: 'renamed-package' })
: JSON.stringify({
numTotalTests: 1,
testResults: [
{
assertionResults: [
{
fullName: 'byte budget',
status: 'failed',
failureMessages: ['Over budget'],
},
],
},
],
}),
writeFiles: async (inputs) => {
for (const [name, content] of Object.entries(inputs))
files.set(name, content)
},
runCommand: async (command, args, options) => {
if (command === 'npx') {
assert.equal(options.env.NEXT_EVAL_BROWSER_JS_PHASE, 'before')
assert(files.get('EVAL.ts').includes('editorLoaded'))
return {
exitCode: 1,
stdout: 'NEXT_EVAL_BROWSER_JS_RESULT:' + JSON.stringify(measurement),
}
}
assert.equal(command, 'node')
assert(args.includes('--eval'))
files.clear()
return { exitCode: 0 }
},
}
const cwd = process.cwd()
try {
process.chdir(path.join(cwd, 'evals'))
const prepared = await hooks.setup(sandbox)
assert.equal(files.size, 0)
assert(prepared.validationFiles['EVAL.ts'].includes('editorLoaded'))
const before = JSON.parse(
prepared.validationEnv.NEXT_EVAL_BROWSER_JS_BEFORE
)
assert.equal(before.initial.encodedBytes, 200)
assert.equal(before.checks[0].status, 'failed')
} finally {
process.chdir(cwd)
}
assert.equal(
await hooks.setup({ readFile: async () => '{"name":"unrelated"}' }),
undefined
)
const runData = {
result: { status: 'passed' },
outputContent: {
eval: 'NEXT_EVAL_BROWSER_JS_RESULT:' + JSON.stringify(measurement),
},
}
const context = { fixture: { name: evalName }, runData }
const analyzed = await hooks.onRunComplete(context)
assert.deepEqual(analyzed.result.analysis.browserJs, measurement)
assert.equal(analyzed.result.status, 'passed')
assert.equal(
await hooks.onRunComplete({ fixture: { name: 'unrelated' }, runData }),
runData
)
const required = createBrowserJsHooks({ [evalName]: 'renamed-package' }, [
'next-bundle-optimizer',
])
const missingSkill = await required.onRunComplete(context)
assert.equal(missingSkill.result.status, 'failed')
assert.match(missingSkill.result.error, /skill was not invoked/)
console.log(
'SDK grader isolation, prepared-snapshot validation, and configured browser measurement hooks pass'
)
})()
.catch((error) => {
console.error(error.message)
process.exitCode = 1
})
.finally(() => fs.rmSync(directory, { recursive: true, force: true }))
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment