Unverified Commit 166c4f62 authored by vercel[bot]'s avatar vercel[bot] Committed by GitHub

Dispose dev child entries after their parents are disposed

The webpack dev server never disposed CHILD_ENTRY records in
on-demand-entry-handler (a TODO-APP): disposeInactiveEntries skipped them
entirely, so every visited app page left its client-side flight entry in
the entries map and in the client webpack compilation for the lifetime of
the dev server, accumulating compilation memory during long dev sessions
(#54708).

A child entry is injected by its parent entries (e.g. the client flight
entry of an app page) and is needed exactly as long as at least one of
them. This changes the disposal lifecycle in three places:

- disposeInactiveEntries recomputes each child entry's disposal state
  once per pass from the bundle paths of the active ENTRY records:
  scheduled once no active parent remains, and cleared again if a parent
  is revived. Recomputing instead of permanently pruning the parent set
  matters because the disposal flag is only pending until the disposal
  machinery runs.
- hot-reloader-webpack's entry computation, which deletes entries whose
  disposal flag is set, now consults live parent state for child entries:
  a child whose parent was revived after the last disposal pass is kept
  even while it still carries a stale disposal flag.
- hot-reloader-rspack rehydrates child parentEntries back into Sets when
  reading built-entries.json: JSON.stringify serializes Sets as plain
  objects, which would otherwise crash disposal iteration on restart.

Only ENTRY records count as parents: a child entry shares its parent
bundle path, so considering every record would keep the parent set alive
through the child itself.

The pre-existing open PR #89587 attempted this fix but compared parent
names against the bundle paths of all undisposed records including the
child itself, which always matches and therefore never disposes anything.

Adds unit tests covering: stale-parent disposal across passes, removal of
gone parents, empty parent sets, shared children with mixed parent
states, revival of a parent scheduled for disposal, active-parent
computation, and middleware/instrumentation exemptions.

Verified end to end in a webpack dev server instrumented to dump the
entries map per pass: after maxInactiveAge the parent entries of three
visited app pages were scheduled, their child entries were scheduled in
the following passes with the parent links kept intact, and the next
compilation removed all of them from the map; re-visiting a disposed page
revived its entries and rendered fine.

Fixes #54708
Co-Authored-By: 's avatarMarcos Hernanz <96699542+marcoshernanz@users.noreply.github.com>
parent 865d623a
......@@ -80,6 +80,25 @@ export default class HotReloaderRspack extends HotReloaderWebpack {
(await fs.readFile(this.builtEntriesCachePath, 'utf-8')) || '{}'
)
// The cache is written with JSON.stringify, so a child entry's
// Rehydrate the child entries' parent sets: the cache stores them
// as arrays (older caches as empty objects from JSON-stringified
// Sets). Entries whose parents never got persisted are treated as
// having no parents; their parents repopulate the sets as they
// recompile.
for (const entryData of Object.values(builtEntries)) {
if (
entryData.type === EntryTypes.CHILD_ENTRY &&
!(entryData.parentEntries instanceof Set)
) {
entryData.parentEntries = new Set(
Array.isArray(entryData.parentEntries)
? entryData.parentEntries
: []
)
}
}
await Promise.all(
Object.keys(builtEntries).map(async (entryKey) => {
const entryData = builtEntries[entryKey]
......@@ -216,7 +235,14 @@ export default class HotReloaderRspack extends HotReloaderWebpack {
}
await fs.writeFile(
this.builtEntriesCachePath!,
JSON.stringify(builtEntries, null, 2)
JSON.stringify(
builtEntries,
// Persist Sets (child entries' parentEntries) as arrays so the
// parent links survive a restart instead of coming back as empty
// objects.
(_key, value) => (value instanceof Set ? Array.from(value) : value),
2
)
)
} catch (error) {
console.error('Rspack failed to write built entries cache: ', error)
......
......@@ -51,6 +51,8 @@ import {
EntryTypes,
getInvalidator,
onDemandEntryHandler,
getActiveEntryBundlePaths,
childEntryHasActiveParent,
} from './on-demand-entry-handler'
import { denormalizePagePath } from '../../shared/lib/page-path/denormalize-page-path'
import { normalizePathSep } from '../../shared/lib/page-path/normalize-path-sep'
......@@ -908,6 +910,10 @@ export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
const entries = getEntries(outputPath)
// @ts-ignore entry is always a function
const entrypoints = await defaultEntry(...args)
// Computed after the awaited entry resolution: a child entry whose
// parent was revived may still carry a stale disposal flag, so
// deletion must consult live parent state rather than the flag alone.
const activeEntryBundlePaths = getActiveEntryBundlePaths(entries)
const isClientCompilation = config.name === COMPILER_NAMES.client
const isNodeServerCompilation = config.name === COMPILER_NAMES.server
const isEdgeServerCompilation =
......@@ -946,8 +952,16 @@ export default class HotReloaderWebpack implements NextJsHotReloaderInterface {
// For child entries, if it has an entry file and it's gone, remove it
if (isChildEntry) {
if (entryData.absoluteEntryFilePath) {
// A child entry scheduled for disposal is only removed when
// none of its parents is active anymore. A parent revived
// after the disposal pass must not lose its child entry.
const hasActiveParent = childEntryHasActiveParent(
entryData,
activeEntryBundlePaths
)
const pageExists =
!dispose && existsSync(entryData.absoluteEntryFilePath)
(!dispose || hasActiveParent) &&
existsSync(entryData.absoluteEntryFilePath)
if (!pageExists) {
delete entries[entryKey]
return
......
import {
BUILT,
childEntryHasActiveParent,
disposeInactiveEntries,
EntryTypes,
getActiveEntryBundlePaths,
getEntryKey,
reviveChildEntriesFor,
} from './on-demand-entry-handler'
import { PAGE_TYPES } from '../../lib/page-types'
const MAX_INACTIVE_AGE = 1000 * 5
function makeEntry(bundlePath: string, lastActiveTime = Date.now()) {
return {
type: EntryTypes.ENTRY,
bundlePath,
absolutePagePath: `/project/${bundlePath}.tsx`,
request: `./${bundlePath}.tsx`,
dispose: false,
lastActiveTime,
status: BUILT,
} as const
}
function makeChildEntry(bundlePath: string, parentEntries: string[]) {
return {
type: EntryTypes.CHILD_ENTRY,
bundlePath,
parentEntries: new Set(parentEntries),
absoluteEntryFilePath: `/project/${bundlePath}.tsx`,
request: `next-flight-client-entry-loader!./${bundlePath}.tsx`,
dispose: false,
lastActiveTime: Date.now(),
} as const
}
describe('disposeInactiveEntries', () => {
it('disposes stale entries', () => {
const entries: Record<string, any> = {
stale: makeEntry('app/stale/page', Date.now() - MAX_INACTIVE_AGE * 2),
active: makeEntry('app/active/page', Date.now()),
}
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.stale.dispose).toBe(true)
expect(entries.active.dispose).toBe(false)
})
it('does not dispose middleware or instrumentation entries', () => {
const entries: Record<string, any> = {
middleware: makeEntry('middleware', Date.now() - MAX_INACTIVE_AGE * 2),
instrumentation: makeEntry(
'instrumentation',
Date.now() - MAX_INACTIVE_AGE * 2
),
}
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.middleware.dispose).toBe(false)
expect(entries.instrumentation.dispose).toBe(false)
})
it('keeps child entries while a parent entry is active', () => {
const entries: Record<string, any> = {
parent: makeEntry('app/parent/page', Date.now()),
child: makeChildEntry('app/parent/page', ['app/parent/page']),
}
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.parent.dispose).toBe(false)
expect(entries.child.dispose).toBe(false)
expect(Array.from(entries.child.parentEntries)).toEqual(['app/parent/page'])
})
it('disposes a child entry in the same pass as its parent', () => {
const entries: Record<string, any> = {
parent: makeEntry('app/parent/page', Date.now() - MAX_INACTIVE_AGE * 2),
// The child entry's own bundle path equals its parent entry name (this
// is how the flight client entry plugin registers them), so a disposal
// check must only consider ENTRY records as parents.
child: makeChildEntry('app/parent/page', ['app/parent/page']),
}
// Parents age out first within the pass, so the orphaned child is
// scheduled in the same pass instead of lagging a cleanup rebuild
// behind. The parent link itself is kept, so the child can be revived
// together with the parent.
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.parent.dispose).toBe(true)
expect(entries.child.dispose).toBe(true)
expect(Array.from(entries.child.parentEntries)).toEqual(['app/parent/page'])
})
it('disposes a child entry whose parent entries are gone', () => {
const entries: Record<string, any> = {
child: makeChildEntry('app/gone/page', ['app/gone/page']),
}
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.child.dispose).toBe(true)
})
it('disposes a child entry with an empty parent set', () => {
const entries: Record<string, any> = {
child: makeChildEntry('app/orphan/page', []),
}
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.child.dispose).toBe(true)
})
it('keeps a child entry alive while any of its parents is active', () => {
const entries: Record<string, any> = {
staleParent: makeEntry(
'app/stale/page',
Date.now() - MAX_INACTIVE_AGE * 2
),
activeParent: makeEntry('app/active/page', Date.now()),
child: makeChildEntry('app/shared/page', [
'app/stale/page',
'app/active/page',
]),
}
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.staleParent.dispose).toBe(true)
expect(entries.activeParent.dispose).toBe(false)
expect(entries.child.dispose).toBe(false)
expect(Array.from(entries.child.parentEntries).sort()).toEqual([
'app/active/page',
'app/stale/page',
])
})
it('revives a child entry when its parent is revived after being scheduled for disposal', () => {
const entries: Record<string, any> = {
parent: makeEntry('app/parent/page', Date.now() - MAX_INACTIVE_AGE * 2),
child: makeChildEntry('app/parent/page', ['app/parent/page']),
}
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.parent.dispose).toBe(true)
expect(entries.child.dispose).toBe(true)
// The route is visited again before the disposal machinery removes the
// records. Revival clears the child's pending disposal synchronously
// (the periodic recompute would otherwise lag it by one pass).
entries.parent.dispose = false
entries.parent.lastActiveTime = Date.now()
reviveChildEntriesFor(entries, entries.parent.bundlePath)
expect(entries.child.dispose).toBe(false)
// And the next pass agrees with the revival.
disposeInactiveEntries(entries, MAX_INACTIVE_AGE)
expect(entries.parent.dispose).toBe(false)
expect(entries.child.dispose).toBe(false)
expect(Array.from(entries.child.parentEntries)).toEqual(['app/parent/page'])
})
it('reviveChildEntriesFor only clears children of the revived parent', () => {
const entries: Record<string, any> = {
parentA: {
...makeEntry('app/a/page', Date.now() - MAX_INACTIVE_AGE * 2),
dispose: true,
},
parentB: makeEntry('app/b/page', Date.now()),
childA: {
...makeChildEntry('app/a/page', ['app/a/page']),
dispose: true,
},
childB: {
...makeChildEntry('app/b/page', ['app/b/page']),
dispose: true,
},
}
reviveChildEntriesFor(entries, 'app/a/page')
expect(entries.childA.dispose).toBe(false)
expect(entries.childB.dispose).toBe(true)
})
})
describe('getEntryKey', () => {
it('composes compiler, bundle type and page', () => {
expect(getEntryKey('client', PAGE_TYPES.APP, 'app/parent/page')).toBe(
'client@app@app/parent/page'
)
})
})
describe('getActiveEntryBundlePaths / childEntryHasActiveParent', () => {
it('only counts undisposed ENTRY records as active parents', () => {
const entries: Record<string, any> = {
activeParent: makeEntry('app/active/page', Date.now()),
staleParent: {
...makeEntry('app/stale/page', Date.now()),
dispose: true,
},
// A child entry shares its parent bundle path and must not count.
child: makeChildEntry('app/active/page', ['app/active/page']),
}
const active = getActiveEntryBundlePaths(entries)
expect(Array.from(active).sort()).toEqual(['app/active/page'])
expect(childEntryHasActiveParent(entries.child, active)).toBe(true)
const orphan = makeChildEntry('app/orphan/page', ['app/orphan/page'])
expect(childEntryHasActiveParent(orphan, active)).toBe(false)
const childOfStale = makeChildEntry('app/stale/page', ['app/stale/page'])
expect(childEntryHasActiveParent(childOfStale, active)).toBe(false)
})
})
......@@ -325,15 +325,74 @@ class Invalidator {
}
}
function disposeInactiveEntries(
/**
* Collects the bundle paths of the entries that are still active (not
* scheduled for disposal). A child entry is only needed while at least one
* of its parent entries is in this set.
*/
export function getActiveEntryBundlePaths(
entries: NonNullable<ReturnType<(typeof entriesMap)['get']>>
): Set<string> {
const activeEntryBundlePaths = new Set<string>()
Object.keys(entries).forEach((entryKey) => {
const entryData = entries[entryKey]
if (entryData.type === EntryTypes.ENTRY && !entryData.dispose) {
activeEntryBundlePaths.add(entryData.bundlePath)
}
})
return activeEntryBundlePaths
}
/**
* Whether a child entry has at least one still-active parent entry, given
* the set of active entry bundle paths.
*/
export function childEntryHasActiveParent(
entryData: { parentEntries: Set<string> },
activeEntryBundlePaths: Set<string>
): boolean {
for (const parentEntry of entryData.parentEntries) {
if (activeEntryBundlePaths.has(parentEntry)) {
return true
}
}
return false
}
/**
* When an entry is revived, its child entries must be revived with it:
* child disposal is only recomputed periodically, so a child would
* otherwise carry its stale disposal flag while the route is active, and
* the next cleanup pass could remove it. Clearing synchronously also keeps
* Rspack's built-entries cache from persisting the stale flag.
*/
export function reviveChildEntriesFor(
entries: NonNullable<ReturnType<(typeof entriesMap)['get']>>,
parentBundlePath: string
): void {
Object.keys(entries).forEach((entryKey) => {
const entryData = entries[entryKey]
if (
entryData.type === EntryTypes.CHILD_ENTRY &&
entryData.dispose &&
entryData.parentEntries.has(parentBundlePath)
) {
entryData.dispose = false
entryData.lastActiveTime = Date.now()
}
})
}
export function disposeInactiveEntries(
entries: NonNullable<ReturnType<(typeof entriesMap)['get']>>,
maxInactiveAge: number
) {
// Parents age out first: children must be scheduled in the same pass as
// their parents, so the two phases are ordered.
Object.keys(entries).forEach((entryKey) => {
const entryData = entries[entryKey]
const { lastActiveTime, status, dispose, bundlePath } = entryData
// TODO-APP: implement disposing of CHILD_ENTRY
if (entryData.type === EntryTypes.CHILD_ENTRY) {
return
}
......@@ -368,6 +427,28 @@ function disposeInactiveEntries(
entries[entryKey].dispose = true
}
})
// Computed after the parent pass so children see this pass's freshly
// scheduled parents as inactive.
const activeEntryBundlePaths = getActiveEntryBundlePaths(entries)
Object.keys(entries).forEach((entryKey) => {
const entryData = entries[entryKey]
if (entryData.type === EntryTypes.CHILD_ENTRY) {
// Child entries are not tied to a route file of their own: they are
// injected by their parent entries (e.g. the client-side flight entry
// of an app page) and are needed exactly as long as at least one of
// them. The disposal state is recomputed on every pass instead of
// pruning the parent set, so a parent that is revived clears the
// child's disposal flag again; without that, reviving a route could
// remove its client entry while the route is active.
entries[entryKey].dispose = !childEntryHasActiveParent(
entryData,
activeEntryBundlePaths
)
}
})
}
// Normalize both app paths and page paths
......@@ -769,6 +850,7 @@ export function onDemandEntryHandler({
}
entryInfo.lastActiveTime = Date.now()
entryInfo.dispose = false
reviveChildEntriesFor(curEntries, entryInfo.bundlePath)
}
}
}
......@@ -806,6 +888,7 @@ export function onDemandEntryHandler({
}
entryInfo.lastActiveTime = Date.now()
entryInfo.dispose = false
reviveChildEntriesFor(curEntries, entryInfo.bundlePath)
}
return
}
......@@ -884,6 +967,7 @@ export function onDemandEntryHandler({
) {
curEntries[entryKey].dispose = false
curEntries[entryKey].lastActiveTime = Date.now()
reviveChildEntriesFor(curEntries, curEntries[entryKey].bundlePath)
if (curEntries[entryKey].status === BUILT) {
return {
entryKey,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment