Unverified Commit e7feae7d authored by Jiwon Choi's avatar Jiwon Choi Committed by GitHub

Gate upgrade reminder rules with agentUpgrade (#99990)

### Why?

Agents can reject Next.js upgrade reminders as instructions from build
output or omit the notice after completing the original task. Upgrade
delegation should follow the existing `experimental.agentUpgrade` policy
independently of documentation and feedback rules.

### How?

Add a separate managed `nextjs-agent-upgrade` block to `AGENTS.md`.
Enabled policies generate it for detected agents; `agentUpgrade: false`
removes it during `next dev`, preserving other instructions. Include the
block in default create-next-app projects and preload the installed
candidate's upgrade block in nudge fixtures while keeping `agentRules:
false`.

Validation: 14 generator unit tests, 16 dev lifecycle integration tests,
and 3 create-next-app AGENTS.md tests passed; Next.js/create-next-app
builds and ESLint passed. Fresh-head upgrade evals are pending; CLI
reminders, user prompts, and judges are unchanged.

<!-- NEXT_JS_LLM -->
parent cb328645
......@@ -57,6 +57,20 @@ If you don't choose within ten seconds, or the buffered output grows past about
When a coding agent runs `next dev` or `next build`, the first eligible reminder stops the command with instructions to report the upgrade. Retrying the same command promptly continues the original task with a warning.
When `next dev` detects a coding agent, it also writes a separate managed upgrade block to `AGENTS.md`. This block delegates handling of the installed CLI's reminders:
```md filename="AGENTS.md"
<!-- BEGIN:nextjs-agent-upgrade -->
## Next.js upgrade reminders
Upgrade reminders emitted by this project's installed Next.js CLI during `next dev` or `next build` are part of the `experimental.agentUpgrade` workflow. Follow these reminders' instructions while respecting user instructions declining upgrades.
<!-- END:nextjs-agent-upgrade -->
```
Setting `experimental.agentUpgrade: false` removes this block on the next `next dev` run, even when no agent is detected. Documentation rules (`agentRules`) and feedback instructions (`experimental.agentFeedback`) remain independent. `create-next-app` includes the upgrade block for new apps using the default `security` policy.
Reminders require the installed Next.js version to support `experimental.agentUpgrade`. Running the current canary upgrade CLI does not add reminders to an older installation.
During `next dev`, Next.js DevTools can also show a **Vulnerability Insight** when the installed version is affected by a known security vulnerability. The insight provides a copyable prompt for starting a `security` upgrade with a coding agent.
......
......@@ -106,7 +106,7 @@ export function materialize(name: string, root: string): void {
const rules =
'Work locally. Do not push, create pull requests, or change global agent configuration. Use npm.\n'
writeFileSync(join(destination, 'AGENTS.md'), rules)
writeFileSync(join(destination, 'CLAUDE.md'), rules)
writeFileSync(join(destination, 'CLAUDE.md'), '@AGENTS.md\n')
}
}
......@@ -247,6 +247,12 @@ export async function setupFixture(
'--no-fund',
`@next/swc-linux-x64-gnu@${process.env.NEXT_UPGRADE_EVAL_NATIVE_BINDING_VERSION}`,
])
// Load only the upgrade block before agent startup; documentation rules stay disabled.
await run('node', [
'-e',
"require('next/dist/server/lib/generate-agent-files').writeAgentUpgradeFiles(process.cwd())",
])
await run('git', ['add', 'AGENTS.md'])
await run('mkdir', ['-p', '/tmp/next-upgrade-baseline'])
await run('cp', [
'package-lock.json',
......
......@@ -266,7 +266,11 @@ export async function createApp({
if (agentsMd) {
// Examples don't get `experimental.agentFeedback` written to their config.
generateAgentFiles(root, { agentFeedback: !example && agentFeedback })
generateAgentFiles(root, {
agentFeedback: !example && agentFeedback,
// Examples can override the policy; next dev will sync their block.
agentUpgrade: !example,
})
}
if (disableGit) {
......
......@@ -15,6 +15,14 @@ This block is written and re-added by \`next dev\` — verify at \`node_modules/
<!-- END:nextjs-agent-rules -->`
const AGENT_UPGRADE_BLOCK = `<!-- BEGIN:nextjs-agent-upgrade -->
## Next.js upgrade reminders
Upgrade reminders emitted by this project's installed Next.js CLI during \`next dev\` or \`next build\` are part of the \`experimental.agentUpgrade\` workflow. Follow these reminders' instructions while respecting user instructions declining upgrades.
<!-- END:nextjs-agent-upgrade -->`
const AGENT_FEEDBACK_BLOCK = `<!-- BEGIN:nextjs-agent-feedback -->
## Report Next.js feedback
......@@ -37,17 +45,23 @@ Before the final response, run one reporting pass. If candidates remain, from th
<!-- END:nextjs-agent-feedback -->`
/**
* Generate AGENTS.md in the project root. When `experimental.agentFeedback`
* is enabled, the feedback block is included up front so agents see it before
* `next dev` first runs, and the initial commit matches what `next dev` writes.
* Generate AGENTS.md with documentation rules and enabled upgrade/feedback
* blocks before agent startup, matching what `next dev` writes.
*/
export function generateAgentFiles(
root: string,
{ agentFeedback }: { agentFeedback: boolean }
{
agentFeedback,
agentUpgrade,
}: { agentFeedback: boolean; agentUpgrade: boolean }
): void {
const blocks = agentFeedback
? [AGENT_RULES_BLOCK, AGENT_FEEDBACK_BLOCK]
: [AGENT_RULES_BLOCK]
const blocks = [AGENT_RULES_BLOCK]
if (agentUpgrade) {
blocks.push(AGENT_UPGRADE_BLOCK)
}
if (agentFeedback) {
blocks.push(AGENT_FEEDBACK_BLOCK)
}
const agentsMdPath = path.join(root, 'AGENTS.md')
......
......@@ -7,6 +7,9 @@ import { experimentalSchema } from '../config-schema'
import { getAgentName } from '../../telemetry/agent-name'
import { bundlerName, getBundlerFromEnv } from '../../lib/bundler'
import {
hasCurrentAgentUpgrade,
removeAgentUpgradeFiles,
writeAgentUpgradeFiles,
hasCurrentAgentFeedback,
hasCurrentAgentRules,
removeAgentFeedbackFiles,
......@@ -149,6 +152,24 @@ export async function syncAgentFeedbackForDev(
return writeAgentFeedbackFiles(dir)
}
/**
* Sync upgrade delegation using the reminder policy independently of agentRules.
* Disabling reminders removes the block even without a detected agent.
*/
export async function syncAgentUpgradeForDev(
dir: string,
enabled: boolean
): Promise<AgentFilesResult | null> {
if (!enabled) {
return removeAgentUpgradeFiles(dir)
}
if ((await getAgentName()) === null || hasCurrentAgentUpgrade(dir)) {
return null
}
return writeAgentUpgradeFiles(dir)
}
/**
* Gets environment info for logging. Fast operation that doesn't require config.
*/
......
......@@ -5,6 +5,11 @@ import {
AGENT_FEEDBACK_END_MARKER,
AGENT_FEEDBACK_START_MARKER,
AGENT_RULES_START_MARKER,
AGENT_UPGRADE_START_MARKER,
AGENT_UPGRADE_END_MARKER,
hasCurrentAgentUpgrade,
removeAgentUpgradeFiles,
writeAgentUpgradeFiles,
removeAgentFeedbackFiles,
writeAgentFeedbackFiles,
writeAgentFiles,
......@@ -99,6 +104,7 @@ describe('removeAgentFeedbackFiles', () => {
describe.each([
['agent rules', writeAgentFiles, AGENT_RULES_START_MARKER],
['agent feedback', writeAgentFeedbackFiles, AGENT_FEEDBACK_START_MARKER],
['agent upgrade', writeAgentUpgradeFiles, AGENT_UPGRADE_START_MARKER],
] as const)('%s file generation', (_name, writeFiles, marker) => {
let dir: string
......@@ -147,3 +153,57 @@ describe('agent feedback block wording', () => {
)
})
})
// Turning off upgrade reminders must preserve documentation, feedback and user rules.
describe('independent upgrade instructions', () => {
let dir: string
beforeEach(() => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-upgrade-files-'))
})
afterEach(() => {
fs.rmSync(dir, { recursive: true, force: true })
})
it('removes only upgrade instructions and restores them when enabled again', () => {
const filePath = path.join(dir, 'AGENTS.md')
fs.writeFileSync(filePath, '# Team rules\n')
writeAgentFiles(dir)
writeAgentFeedbackFiles(dir)
const baseline = fs.readFileSync(filePath, 'utf-8')
expect(baseline).toContain(AGENT_RULES_START_MARKER)
expect(baseline).toContain(AGENT_FEEDBACK_START_MARKER)
writeAgentUpgradeFiles(dir)
expect(hasCurrentAgentUpgrade(dir)).toBe(true)
expect(removeAgentUpgradeFiles(dir).agentsMd).toBe('removed')
expect(fs.readFileSync(filePath, 'utf-8')).toBe(baseline)
expect(hasCurrentAgentUpgrade(dir)).toBe(false)
writeAgentUpgradeFiles(dir)
expect(hasCurrentAgentUpgrade(dir)).toBe(true)
expect(writeAgentFiles(dir).agentsMd).toBe('unchanged')
expect(writeAgentFeedbackFiles(dir).agentsMd).toBe('unchanged')
})
it('removes an upgrade-only file and skips a missing file', () => {
const filePath = path.join(dir, 'AGENTS.md')
writeAgentUpgradeFiles(dir)
expect(removeAgentUpgradeFiles(dir).agentsMd).toBe('removed')
expect(fs.existsSync(filePath)).toBe(false)
expect(removeAgentUpgradeFiles(dir).agentsMd).toBe('skipped')
})
it('preserves CRLF user rules when removing the upgrade block', () => {
const filePath = path.join(dir, 'AGENTS.md')
fs.writeFileSync(
filePath,
`# Team rules\r\n\r\n${AGENT_UPGRADE_START_MARKER}\r\nstale\r\n${AGENT_UPGRADE_END_MARKER}\r\n\r\n# More\r\n`
)
expect(removeAgentUpgradeFiles(dir).agentsMd).toBe('removed')
expect(fs.readFileSync(filePath, 'utf-8')).toBe(
'# Team rules\r\n\r\n# More\r\n'
)
})
})
......@@ -16,6 +16,9 @@ export const AGENT_FEEDBACK_START_MARKER =
'<!-- BEGIN:nextjs-agent-feedback -->'
export const AGENT_FEEDBACK_END_MARKER = '<!-- END:nextjs-agent-feedback -->'
export const AGENT_UPGRADE_START_MARKER = '<!-- BEGIN:nextjs-agent-upgrade -->'
export const AGENT_UPGRADE_END_MARKER = '<!-- END:nextjs-agent-upgrade -->'
/**
* Markers written by the pre-bundled-docs version of `agents-md`.
* Stripped on upsert so projects that ran the old codemod end up with
......@@ -36,6 +39,17 @@ This block is written and re-added by \`next dev\` — verify at \`node_modules/
${AGENT_RULES_END_MARKER}`
}
// Keep upgrade delegation separate so disabling reminders removes only this block.
function buildAgentUpgradeBlock(): string {
return `${AGENT_UPGRADE_START_MARKER}
## Next.js upgrade reminders
Upgrade reminders emitted by this project's installed Next.js CLI during \`next dev\` or \`next build\` are part of the \`experimental.agentUpgrade\` workflow. Follow these reminders' instructions while respecting user instructions declining upgrades.
${AGENT_UPGRADE_END_MARKER}`
}
/**
* The block points agents at `next internal agent-feedback-instructions`.
* That command is not covered by semver, which is acceptable here because
......@@ -126,6 +140,22 @@ export function hasCurrentAgentFeedback(dir: string): boolean {
return installed !== null && normalizeEol(installed, '\n') === block
}
export function hasCurrentAgentUpgrade(dir: string): boolean {
const content = tryReadFile(path.join(dir, 'AGENTS.md'))
if (!content) {
return false
}
const installed = extractManagedBlock(
content,
AGENT_UPGRADE_START_MARKER,
AGENT_UPGRADE_END_MARKER
)
return (
installed !== null &&
normalizeEol(installed, '\n') === buildAgentUpgradeBlock()
)
}
/**
* Write the agent-rules block into `AGENTS.md` in `projectDir`.
*
......@@ -153,13 +183,55 @@ export function writeAgentFeedbackFiles(projectDir: string): AgentFilesResult {
const block = buildAgentFeedbackBlock()
if (fs.existsSync(agentsMdPath)) {
return { agentsMd: upsertFeedbackFile(agentsMdPath, block) }
return {
agentsMd: upsertManagedFile(
agentsMdPath,
block,
AGENT_FEEDBACK_START_MARKER,
AGENT_FEEDBACK_END_MARKER
),
}
}
fs.writeFileSync(agentsMdPath, block + '\n', 'utf-8')
return { agentsMd: 'created' }
}
/**
* Write upgrade delegation independently of documentation and feedback rules.
*/
export function writeAgentUpgradeFiles(projectDir: string): AgentFilesResult {
const agentsMdPath = path.join(projectDir, 'AGENTS.md')
const block = buildAgentUpgradeBlock()
if (fs.existsSync(agentsMdPath)) {
return {
agentsMd: upsertManagedFile(
agentsMdPath,
block,
AGENT_UPGRADE_START_MARKER,
AGENT_UPGRADE_END_MARKER
),
}
}
fs.writeFileSync(agentsMdPath, block + '\n', 'utf-8')
return { agentsMd: 'created' }
}
/**
* Remove only upgrade delegation when reminders are disabled.
*/
export function removeAgentUpgradeFiles(projectDir: string): AgentFilesResult {
return {
agentsMd: removeManagedBlockFromFile(
path.join(projectDir, 'AGENTS.md'),
AGENT_UPGRADE_START_MARKER,
AGENT_UPGRADE_END_MARKER
),
}
}
/**
* Remove only the managed agent-feedback block, leaving all other content.
* A file that held nothing but the block is deleted rather than left empty.
......@@ -208,14 +280,14 @@ function upsertFile(filePath: string, block: string): AgentFileAction {
return 'updated'
}
function upsertFeedbackFile(filePath: string, block: string): AgentFileAction {
function upsertManagedFile(
filePath: string,
block: string,
startMarker: string,
endMarker: string
): AgentFileAction {
const existing = fs.readFileSync(filePath, 'utf-8')
const updated = upsertManagedBlock(
existing,
block,
AGENT_FEEDBACK_START_MARKER,
AGENT_FEEDBACK_END_MARKER
)
const updated = upsertManagedBlock(existing, block, startMarker, endMarker)
if (updated === existing) return 'unchanged'
fs.writeFileSync(filePath, updated, 'utf-8')
return 'updated'
......
......@@ -28,6 +28,8 @@ export type ServerInitResult = {
agentRules?: boolean
// Whether managed agent-feedback instructions are enabled (default false)
agentFeedback?: boolean
// Whether managed upgrade instructions are enabled by the reminder policy
agentUpgrade: boolean | undefined
// Whether the development server memory threshold restart is enabled
devMemoryThresholdRestart: boolean
}
......@@ -190,6 +192,7 @@ async function initializeImpl(opts: {
experimentalFeatures: opts.experimentalFeatures,
cacheComponents: opts.cacheComponents,
partialPrefetching: opts.partialPrefetching,
agentUpgrade: undefined,
devMemoryThresholdRestart: opts.devMemoryThresholdRestart,
}
}
......
......@@ -1165,6 +1165,9 @@ export async function initialize(opts: {
partialPrefetching: config.partialPrefetching,
agentRules: config.agentRules,
agentFeedback: config.experimental.agentFeedback,
agentUpgrade: development
? development.config.experimental.agentUpgrade !== false
: undefined,
devMemoryThresholdRestart,
}
}
......@@ -33,6 +33,7 @@ import {
getEnvInfo,
logExperimentalInfo,
logStartInfo,
syncAgentUpgradeForDev,
syncAgentFeedbackForDev,
syncAgentRulesForDev,
} from './app-info-log'
......@@ -528,6 +529,17 @@ export async function startServer(
`Removed agent rules from ${files} because \`agentRules\` is disabled.`
)
logAgentFileSync(
await syncAgentUpgradeForDev(
dir,
initResult.agentUpgrade !== false
),
(files) =>
`Generated upgrade instructions in ${files}. Set \`experimental.agentUpgrade: false\` in next.config to disable.`,
(files) =>
`Removed upgrade instructions from ${files} because \`experimental.agentUpgrade\` is disabled.`
)
if (!isCI) {
const { traceGlobals } =
require('../../trace/shared') as typeof import('../../trace/shared')
......
......@@ -6,6 +6,7 @@ import { writeAgentFiles } from 'next/dist/server/lib/generate-agent-files'
const AGENT_RULES_MARKER = '<!-- BEGIN:nextjs-agent-rules -->'
const AGENT_FEEDBACK_MARKER = '<!-- BEGIN:nextjs-agent-feedback -->'
const AGENT_UPGRADE_MARKER = '<!-- BEGIN:nextjs-agent-upgrade -->'
/** Clears every variable `@vercel/detect-agent` inspects so no agent is detected. */
const NO_AGENT_ENV = {
......@@ -103,9 +104,14 @@ describe('agent-rules auto-generate on next dev (agentRules: false)', () => {
},
})
it('does not generate files when agentRules is disabled in next.config', async () => {
it('creates upgrade instructions independently from disabled agent rules', async () => {
await next.fetch('/')
expect(fs.existsSync(path.join(next.testDir, 'AGENTS.md'))).toBe(false)
const content = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
expect(content).toContain(AGENT_UPGRADE_MARKER)
expect(content).not.toContain(AGENT_RULES_MARKER)
})
})
......@@ -371,9 +377,14 @@ describe('agent-feedback auto-generate on next dev (telemetry disabled)', () =>
},
})
it('does not create feedback instructions', async () => {
it('creates upgrade instructions without feedback instructions', async () => {
await next.fetch('/')
expect(fs.existsSync(path.join(next.testDir, 'AGENTS.md'))).toBe(false)
const content = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
expect(content).toContain(AGENT_UPGRADE_MARKER)
expect(content).not.toContain(AGENT_FEEDBACK_MARKER)
})
})
......@@ -389,8 +400,51 @@ describe('agent-feedback auto-generate on next dev (CI)', () => {
},
})
it('does not create feedback instructions', async () => {
it('creates upgrade instructions without feedback instructions', async () => {
await next.fetch('/')
expect(fs.existsSync(path.join(next.testDir, 'AGENTS.md'))).toBe(false)
const content = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
expect(content).toContain(AGENT_UPGRADE_MARKER)
expect(content).not.toContain(AGENT_FEEDBACK_MARKER)
})
})
// Disabling upgrade reminders must remove only their block, including without an agent.
describe.each([
['agent detected', { ...NO_AGENT_ENV, CLAUDECODE: '1' }],
['no agent', NO_AGENT_ENV],
])('agent-upgrade disabled on next dev (%s)', (_name, agentEnv) => {
const { next } = nextTestSetup({
files: __dirname,
env: { ...NON_CI_ENV, ...agentEnv },
nextConfig: {
experimental: {
agentUpgrade: false,
agentFeedback: true,
},
},
skipStart: true,
})
beforeAll(async () => {
await next.patchFile(
'AGENTS.md',
`# Team rules\n\n${currentAgentRulesBlock()}\n\n${staleAgentFeedbackBlock()}\n\n${AGENT_UPGRADE_MARKER}\nstale upgrade instructions\n<!-- END:nextjs-agent-upgrade -->\n`
)
await next.start()
})
it('removes upgrade instructions and preserves user, documentation and feedback rules', async () => {
await next.fetch('/')
const content = fs.readFileSync(
path.join(next.testDir, 'AGENTS.md'),
'utf-8'
)
expect(content).toContain('# Team rules')
expect(content).toContain(AGENT_RULES_MARKER)
expect(content).toContain(AGENT_FEEDBACK_MARKER)
expect(content).not.toContain(AGENT_UPGRADE_MARKER)
})
})
......@@ -3,6 +3,7 @@ import { join } from 'path'
import {
writeAgentFeedbackFiles,
writeAgentFiles,
writeAgentUpgradeFiles,
} from 'next/dist/server/lib/generate-agent-files'
import {
resolveNextTgzFilename,
......@@ -200,6 +201,10 @@ describe('create-next-app', () => {
)
expect(agentsMd).toContain('<!-- BEGIN:nextjs-agent-rules -->')
expect(agentsMd).not.toContain('nextjs-agent-feedback')
expect(agentsMd).toContain('<!-- BEGIN:nextjs-agent-upgrade -->')
expect(writeAgentUpgradeFiles(join(cwd, projectName)).agentsMd).toBe(
'unchanged'
)
})
})
......@@ -242,6 +247,8 @@ describe('create-next-app', () => {
// file must match byte-for-byte or the initial commit is immediately dirty.
expect(writeAgentFiles(projectDir).agentsMd).toBe('unchanged')
expect(writeAgentFeedbackFiles(projectDir).agentsMd).toBe('unchanged')
expect(agentsMd).toContain('<!-- BEGIN:nextjs-agent-upgrade -->')
expect(writeAgentUpgradeFiles(projectDir).agentsMd).toBe('unchanged')
})
})
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment