Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
N
next.js
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Alex
next.js
Commits
58d44064
Commit
58d44064
authored
Mar 24, 2026
by
devjiwonchoi
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Add agent-041-catch-error eval
parent
6c3295f9
Hide whitespace changes
Inline
Side-by-side
Showing
8 changed files
with
189 additions
and
0 deletions
+189
-0
EVAL.ts
evals/evals/agent-041-catch-error/EVAL.ts
+90
-0
PROMPT.md
evals/evals/agent-041-catch-error/PROMPT.md
+1
-0
layout.tsx
evals/evals/agent-041-catch-error/app/layout.tsx
+17
-0
page.tsx
evals/evals/agent-041-catch-error/app/page.tsx
+10
-0
widget.tsx
evals/evals/agent-041-catch-error/app/widget.tsx
+16
-0
next.config.ts
evals/evals/agent-041-catch-error/next.config.ts
+5
-0
package.json
evals/evals/agent-041-catch-error/package.json
+23
-0
tsconfig.json
evals/evals/agent-041-catch-error/tsconfig.json
+27
-0
No files found.
evals/evals/agent-041-catch-error/EVAL.ts
0 → 100644
View file @
58d44064
/**
* Catch Error
*
* Tests whether the agent uses `unstable_catchError` from `next/error` to create
* a component-level error boundary instead of reaching for `error.js` or a raw
* React error boundary class.
*/
import
{
expect
,
test
}
from
'vitest'
import
{
readFileSync
,
existsSync
,
readdirSync
,
statSync
}
from
'fs'
import
{
join
}
from
'path'
type
SourceFile
=
{
path
:
string
;
content
:
string
}
const
IGNORE_DIRS
=
new
Set
([
'.git'
,
'.next'
,
'node_modules'
,
'dist'
,
'build'
,
'coverage'
,
])
const
IGNORE_FILES
=
new
Set
([
'EVAL.ts'
,
'PROMPT.md'
])
function
readSourceFiles
(
dir
:
string
):
SourceFile
[]
{
if
(
!
existsSync
(
dir
))
return
[]
const
files
:
SourceFile
[]
=
[]
for
(
const
entry
of
readdirSync
(
dir
))
{
if
(
IGNORE_DIRS
.
has
(
entry
))
continue
const
fullPath
=
join
(
dir
,
entry
)
const
stats
=
statSync
(
fullPath
)
if
(
stats
.
isDirectory
())
{
files
.
push
(...
readSourceFiles
(
fullPath
))
continue
}
if
(
IGNORE_FILES
.
has
(
entry
))
continue
if
(
/
\.(
ts|tsx|js|jsx
)
$/
.
test
(
entry
))
{
files
.
push
({
path
:
fullPath
,
content
:
readFileSync
(
fullPath
,
'utf-8'
),
})
}
}
return
files
}
const
sourceFiles
=
readSourceFiles
(
process
.
cwd
())
const
allSource
=
sourceFiles
.
map
((
f
)
=>
f
.
content
).
join
(
'
\
n'
)
test
(
'uses unstable_catchError from next/error'
,
()
=>
{
expect
(
allSource
).
toMatch
(
/from
\s
+
[
'"
]
next
\/
error
[
'"
]
/
)
expect
(
allSource
).
toMatch
(
/unstable_catchError/
)
})
test
(
"file with catchError has 'use client' directive"
,
()
=>
{
const
catchErrorFile
=
sourceFiles
.
find
((
f
)
=>
f
.
content
.
includes
(
'unstable_catchError'
)
)
expect
(
catchErrorFile
).
toBeDefined
()
expect
(
catchErrorFile
!
.
content
).
toMatch
(
/
[
'"
]
use client
[
'"
]
/
)
})
test
(
'provides retry functionality via unstable_retry'
,
()
=>
{
expect
(
allSource
).
toMatch
(
/unstable_retry/
)
})
test
(
'does not use error.js file convention'
,
()
=>
{
const
errorFile
=
sourceFiles
.
find
((
f
)
=>
/error
\.(
tsx
?
|jsx
?)
$/
.
test
(
f
.
path
))
expect
(
errorFile
).
toBeUndefined
()
})
test
(
'renders the error message'
,
()
=>
{
const
catchErrorFile
=
sourceFiles
.
find
((
f
)
=>
f
.
content
.
includes
(
'unstable_catchError'
)
)
expect
(
catchErrorFile
).
toBeDefined
()
expect
(
catchErrorFile
!
.
content
).
toMatch
(
/error
\.
message/i
)
})
test
(
'does not use raw React error boundary class'
,
()
=>
{
expect
(
allSource
).
not
.
toMatch
(
/componentDidCatch/
)
expect
(
allSource
).
not
.
toMatch
(
/getDerivedStateFromError/
)
})
evals/evals/agent-041-catch-error/PROMPT.md
0 → 100644
View file @
58d44064
The Widget sometimes crashes and takes down the whole page. Make it so the rest of the page keeps working when the Widget fails. Show the error message and a button to retry.
evals/evals/agent-041-catch-error/app/layout.tsx
0 → 100644
View file @
58d44064
import
type
{
Metadata
}
from
'next'
export
const
metadata
:
Metadata
=
{
title
:
'App'
,
}
export
default
function
RootLayout
({
children
,
}:
Readonly
<
{
children
:
React
.
ReactNode
}
>
)
{
return
(
<
html
lang=
"en"
>
<
body
>
{
children
}
</
body
>
</
html
>
)
}
evals/evals/agent-041-catch-error/app/page.tsx
0 → 100644
View file @
58d44064
import
Widget
from
'./widget'
export
default
function
Home
()
{
return
(
<
main
>
<
h1
>
Dashboard
</
h1
>
<
Widget
/>
</
main
>
)
}
evals/evals/agent-041-catch-error/app/widget.tsx
0 → 100644
View file @
58d44064
'use client'
import
{
useState
}
from
'react'
export
default
function
Widget
()
{
const
[
clicked
,
setClicked
]
=
useState
(
false
)
if
(
clicked
)
{
throw
new
Error
(
'Widget failed to load'
)
}
return
(
<
div
>
<
h2
>
Widget
</
h2
>
<
button
onClick=
{
()
=>
setClicked
(
true
)
}
>
Load data
</
button
>
</
div
>
)
}
evals/evals/agent-041-catch-error/next.config.ts
0 → 100644
View file @
58d44064
import
type
{
NextConfig
}
from
'next'
const
nextConfig
:
NextConfig
=
{}
export
default
nextConfig
evals/evals/agent-041-catch-error/package.json
0 → 100644
View file @
58d44064
{
"private"
:
true
,
"type"
:
"module"
,
"scripts"
:
{
"dev"
:
"next dev"
,
"build"
:
"next build"
,
"start"
:
"next start"
},
"dependencies"
:
{
"next"
:
"^16"
,
"react"
:
"19.1.0"
,
"react-dom"
:
"19.1.0"
},
"devDependencies"
:
{
"@types/node"
:
"^20"
,
"@types/react"
:
"^19"
,
"@types/react-dom"
:
"^19"
,
"@vitejs/plugin-react"
:
"^4.4.1"
,
"typescript"
:
"^5"
,
"vite-tsconfig-paths"
:
"^5.1.4"
,
"vitest"
:
"^3.1.3"
}
}
evals/evals/agent-041-catch-error/tsconfig.json
0 → 100644
View file @
58d44064
{
"compilerOptions"
:
{
"lib"
:
[
"dom"
,
"dom.iterable"
,
"esnext"
],
"allowJs"
:
true
,
"skipLibCheck"
:
true
,
"strict"
:
true
,
"noEmit"
:
true
,
"esModuleInterop"
:
true
,
"module"
:
"esnext"
,
"moduleResolution"
:
"bundler"
,
"resolveJsonModule"
:
true
,
"isolatedModules"
:
true
,
"jsx"
:
"preserve"
,
"incremental"
:
true
,
"plugins"
:
[
{
"name"
:
"next"
}
],
"paths"
:
{
"@/*"
:
[
"./*"
]
},
"target"
:
"ES2017"
},
"include"
:
[
"next-env.d.ts"
,
"**/*.ts"
,
"**/*.tsx"
,
".next/types/**/*.ts"
],
"exclude"
:
[
"node_modules"
]
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment