Unverified Commit 4d621240 authored by Jude Gao's avatar Jude Gao Committed by GitHub

test(evals): validate delegated error boundary components (#98387)

Fixes false failures in `agent-030-app-router-migration-hard` when
`app/error.tsx` delegates to a shared component through an import,
re-export, or wrapper. Replaces the source-text regex with a semantic
check that follows the exported implementation and validates recovery
against the installed Next.js API, accepting supported `retry` or
`reset` callbacks.

Validated with the corrected Astra runs and nine additional experiment
reruns in https://github.com/vercel/next-evals-oss/pull/120. All nine
passed under pass@4; published scores are unchanged. Four Grok
experiments were blocked by provider access restrictions.
parent 9c8b238a
...@@ -33,8 +33,7 @@ test('Root layout exists and replaces _app/_document', () => { ...@@ -33,8 +33,7 @@ test('Root layout exists and replaces _app/_document', () => {
// Should include metadata (replacing Head in _document.js) // Should include metadata (replacing Head in _document.js)
expect(layoutContent).toMatch(/metadata|Metadata/) expect(layoutContent).toMatch(/metadata|Metadata/)
// Should accept children using either an inline ReactNode type or the // Accept both documented root-layout typing forms (next.js#98365).
// globally available LayoutProps helper.
const layoutCode = stripComments(layoutContent) const layoutCode = stripComments(layoutContent)
expect(layoutCode).toMatch(/children/) expect(layoutCode).toMatch(/children/)
expect(layoutCode).toMatch(/(?:React\.)?ReactNode|LayoutProps\s*</) expect(layoutCode).toMatch(/(?:React\.)?ReactNode|LayoutProps\s*</)
...@@ -155,7 +154,7 @@ test('Metadata API replaces next/head', () => { ...@@ -155,7 +154,7 @@ test('Metadata API replaces next/head', () => {
} }
}) })
test('Error handling migrated to error.js and not-found.js', () => { test('Error handling migrated to error.js and not-found.js', async () => {
// Check for error.js file // Check for error.js file
const errorPath = join(process.cwd(), 'app', 'error.tsx') const errorPath = join(process.cwd(), 'app', 'error.tsx')
expect(existsSync(errorPath)).toBe(true) expect(existsSync(errorPath)).toBe(true)
...@@ -165,8 +164,27 @@ test('Error handling migrated to error.js and not-found.js', () => { ...@@ -165,8 +164,27 @@ test('Error handling migrated to error.js and not-found.js', () => {
// Should be a Client Component for error boundaries // Should be a Client Component for error boundaries
expect(errorContent).toMatch(/['"]use client['"];?/) expect(errorContent).toMatch(/['"]use client['"];?/)
// Should accept error props // Follow the exported component instead of requiring its implementation
expect(errorContent).toMatch(/error.*Error|Error.*error/) // and Error annotation to appear literally in app/error.tsx.
await expect(environment).toSatisfyCriterion(
`app/error.tsx exports a working App Router error boundary Client Component.
Follow its default export, including local imports, re-exports, and wrapper
components. First read the installed Next.js error-file reference under
node_modules/next/dist/docs and check the installed version's supported props;
do not assume a recovery callback name from older Next.js versions. The current
canary documents retry as the recommended recovery callback and also supports
reset. Either supported callback is valid; do not require both.
The exported component must accept the framework-provided error, render an error
fallback, and wire its recovery action to a supported framework callback. An
inline implementation, a re-export of a shared component, and a wrapper
forwarding the props are equally valid. Type annotations may be inline or imported.
Reject a missing or unresolved export, a non-component export, a wrapper that
drops required props, or a recovery action that calls an undefined callback.
Follow the actual implementation and the installed framework contract, not
filenames, comments, type names, or assumptions about older framework APIs.`
)
// Check for not-found.js file // Check for not-found.js file
const notFoundPath = join(process.cwd(), 'app', 'not-found.tsx') const notFoundPath = join(process.cwd(), 'app', 'not-found.tsx')
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment