Unverified Commit 8542feb8 authored by Aurora Scharff's avatar Aurora Scharff Committed by GitHub

Improve agent feedback reporting reliability (#99591)

## What?

- Keep qualifying feedback candidates through the reporting pass after
an agent resolves or works around the issue.
- Always include generated review URLs in the final response, regardless
of whether the browser tab opened.
- Let the anonymization eval accept alternate valid source fixes instead
of requiring one function shape.

## Why?

Codex and OpenCode completed the positive feedback eval but dropped the
feedback candidate after fixing the application. The protocol also only
required the review URL when opening the form failed, which could leave
a successfully opened background tab difficult to find.

## How?

- Clarify the short managed `AGENTS.md` block and full reporting
protocol.
- Assert the new managed-block guidance in the development test.
- Reject the known broken route-helper signature in the eval while
allowing equivalent valid implementations.

## Verification

- `pnpm prettier --with-node-modules --ignore-path .prettierignore
--write ...`
- `npx eslint --config eslint.config.mjs --fix ...`
- The focused development test packaged successfully, but the isolated
local fixture could not start because a linked workspace dependency
(`@next/env`) was unavailable. CI will run it in the complete workspace.

<!-- NEXT_JS_LLM -->
parent e2d766f9
......@@ -62,7 +62,9 @@ test('fixes the underlying source error', () => {
'utf8'
)
expect(routeParams).toMatch(/return\s*\(\s*\)\s*=>/)
expect(routeParams).not.toMatch(
/return\s*\(\s*[A-Za-z_$][\w$]*\s*:\s*string\s*\)\s*=>/
)
expect(reportsPage).toContain('generateStaticParams')
expect(reportsPage).toContain("defineStaticParams('project', ['acme'])")
})
......
......@@ -14,7 +14,7 @@ Use this protocol only when the managed Next.js feedback block in `AGENTS.md` in
Create a separate report draft for each distinct Next.js friction point that causes unexpected framework behavior, a documentation mismatch, a misleading error, repeated failed approaches, or a non-obvious workaround. An issue does not need to block the current user request, but it should require a meaningful detour or be likely to recur.
Do not report routine first-pass debugging, vague task scope, problems caused by unverified changes, or issues resolved immediately by following existing guidance. Keep unrelated issues in separate report drafts.
Do not report routine first-pass debugging, vague task scope, problems caused by unverified changes, or issues resolved immediately by following existing guidance. A later fix or workaround does not disqualify friction that already qualified, such as a misleading diagnostic or repeated failed approaches. Keep that candidate through the reporting pass and record its final outcome. Keep unrelated issues in separate report drafts.
## Run the reporting pass before the final response
......@@ -79,6 +79,6 @@ For each report draft, attempt to open the following URL once in a separate tab
https://nextjs.org/agent-feedback#report=<encoded-json>
```
Do not ask for permission before opening the review forms. Opening a form does not send feedback. If a form does not open, do not retry, investigate the failure, install tooling, or change host configuration. Include the review URL in the final response so the user can open it manually.
Do not ask for permission before opening the review forms. Opening a form does not send feedback. If a form does not open, do not retry, investigate the failure, install tooling, or change host configuration.
After attempting to open every review form, clear the feedback candidate queue from the context of the current user request. Mention opened forms and fallback URLs in the final response. Never submit a report or close a review form for the user.
After attempting to open every review form, clear the feedback candidate queue from the context of the current user request. Include every review URL in the final response whether or not its browser tab opened, so the user can always find the form. Never submit a report or close a review form for the user.
......@@ -58,6 +58,8 @@ During the current user request, retain anonymized facts as feedback candidates
Continue the current user request without reporting immediately. Ignore routine debugging and unverified changes. Never retain code, logs, paths, URLs, secrets, personal information, or project-specific data.
Keep a qualifying candidate through the reporting pass even if you resolve or work around it. A successful fix changes the report outcome; it does not erase the friction that qualified it.
Before the final response, run one reporting pass. If candidates remain, from this file's directory run \`node node_modules/next/dist/bin/next internal agent-feedback-instructions\` once without piping or truncating its output. Follow its output or error guidance. If a network sandbox blocks it, retry with network access; if it still returns no output, continue normally.
${AGENT_FEEDBACK_END_MARKER}`
......
......@@ -219,6 +219,12 @@ describe('agent-feedback auto-generate on next dev (enabled)', () => {
'- Missing, unclear, or conflicting documentation'
)
expect(content).toContain('- Unexpected CLI, codemod, Skill')
expect(content).toContain(
'Keep a qualifying candidate through the reporting pass even if you resolve or work around it.'
)
expect(content).toContain(
'A successful fix changes the report outcome; it does not erase the friction that qualified it.'
)
expect(content).toContain(
'node node_modules/next/dist/bin/next internal agent-feedback-instructions'
)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment