Unverified Commit 466cdfaa authored by Will Binns-Smith's avatar Will Binns-Smith Committed by GitHub

Turbopack server hmr: avoid complete `clear` on graph changes (#95546)

Currently, importing a new module causes a complete eviction (`clear()`)
and re-evaluation of server chunks, both in the Turbopack runtime's
module cache and Node's `require.cache`.

Here's what happens today:

- A new module is imported into a chunk's graph
- This changes its `availability_info`, which in dev for non-entry
chunks is encoded into the chunk's filepath
- `VersionedContentMap` works entirely on chunk paths. When we construct
instructions to transition a chunk into its new state, we fail to find
the prior state and fall back to `clear()` as described above.

An ideal version of this is a refactor that implements
`VersionedContentMap` on a per-module basis, not a per-chunk one. This
PR doesn't do that, but achieves consistent module-level updates when a
chunk's availability info changes.

It does this by implementing chunk lists for server entry chunks the
same way the client hmr implementation does: an entry chunk's version
aggregates the versions of its dependent chunks (including dynamically
imported ones), keyed by merger rather than path. Entry chunks do not
encode availability info in their paths, so they are insulated from
missing versions. Updates are applied in Node.js through the same shared
merged-update machinery in the unified hmr runtime that the client
already uses, plus a new `ChunkListUpdate` branch in the server hmr
client to unwrap the merged updates.

Details:

- The merged-update wire format (`EcmascriptMergedUpdate` etc.) moves
from turbopack-browser into shared `turbopack_ecmascript::chunk_list`,
along with runtime-agnostic `ChunkListVersion` and `update_chunk_list`
implementations.
- `turbopack-nodejs` gains a chunk content merger mirroring the browser
one. `EcmascriptBuildNodeEntryChunk`'s versioned content is now a
chunk-list content over its sync and async chunks. It is version-only
and never emitted; the entry chunk still inlines its loader calls.
- The aggregate server hmr subscription now tracks only entry chunks.
Shared chunks under `server/chunks/` ride the entry's `ChunkListUpdate`
as module deltas, so their content-hash paths no longer matter.
- The Node hmr client applies `ChunkListUpdate` by feeding each nested
merged update through the shared apply path. Modules that appear in an
"added" chunk but already exist in the module cache were moved by a
chunk rename and are treated as modified.
- On a successful partial apply, the hot reloader clears the manifest
cache for updated chunks and notifies browsers to refetch RSC, without
clearing `require.cache`.

### Test Plan

Adds a series of e2e tests to catch the described manual test plan in
both entry and dynamic chunks.
parent b340bc74
...@@ -3,10 +3,12 @@ use std::sync::Arc; ...@@ -3,10 +3,12 @@ use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use rustc_hash::FxHashMap; use rustc_hash::FxHashMap;
use turbo_rcstr::RcStr; use turbo_rcstr::RcStr;
use turbo_tasks::{FxIndexMap, ReadRef, ResolvedVc, TraitRef, TryJoinIterExt, Vc}; use turbo_tasks::{FxIndexMap, FxIndexSet, ReadRef, ResolvedVc, TraitRef, TryJoinIterExt, Vc};
use turbo_tasks_fs::FileSystemPath; use turbo_tasks_fs::FileSystemPath;
use turbo_tasks_hash::{Xxh3Hash64Hasher, encode_base64}; use turbo_tasks_hash::{Xxh3Hash64Hasher, encode_base64};
use turbopack_browser::ecmascript::list::content::EcmascriptDevChunkListContent;
use turbopack_core::version::{PartialUpdate, Update, Version, VersionState, VersionedContent}; use turbopack_core::version::{PartialUpdate, Update, Version, VersionState, VersionedContent};
use turbopack_nodejs::ecmascript::node::entry::chunk_list_content::EcmascriptBuildNodeChunkListContent;
use crate::versioned_content_map::VersionedContentMap; use crate::versioned_content_map::VersionedContentMap;
...@@ -15,6 +17,17 @@ pub struct HmrChunkWithContent { ...@@ -15,6 +17,17 @@ pub struct HmrChunkWithContent {
pub content: ResolvedVc<Box<dyn VersionedContent>>, pub content: ResolvedVc<Box<dyn VersionedContent>>,
} }
/// Whether `content` is a chunk list, i.e. an entry point of the chunk graph that
/// an HMR subscription can be anchored on.
///
/// Note this must enumerate every chunk list content type. A new chunking context
/// that introduces one has to be added here, otherwise its chunks silently drop
/// out of the HMR subscription.
pub fn is_entry_chunk_list_content(content: ResolvedVc<Box<dyn VersionedContent>>) -> bool {
ResolvedVc::try_downcast_type::<EcmascriptBuildNodeChunkListContent>(content).is_some()
|| ResolvedVc::try_downcast_type::<EcmascriptDevChunkListContent>(content).is_some()
}
/// Per-chunk versions keyed by path /// Per-chunk versions keyed by path
#[turbo_tasks::value(serialization = "skip", shared)] #[turbo_tasks::value(serialization = "skip", shared)]
pub struct AggregateHmrVersion { pub struct AggregateHmrVersion {
...@@ -80,60 +93,71 @@ impl AggregateHmrVersion { ...@@ -80,60 +93,71 @@ impl AggregateHmrVersion {
} }
} }
pub fn merge_ecmascript_merged_update( /// Aggregates per-entry HMR instructions into a single combined `ChunkListUpdate`.
combined_entries: &mut FxHashMap<String, serde_json::Value>, #[derive(Default)]
combined_chunks: &mut FxHashMap<String, serde_json::Value>, pub struct ChunkListUpdateBuilder {
instruction: &serde_json::Value, chunks: FxHashMap<String, serde_json::Value>,
) { merged: FxIndexSet<serde_json::Value>,
let Some(obj) = instruction.as_object() else { }
return;
}; impl ChunkListUpdateBuilder {
if let Some(entries) = obj.get("entries").and_then(|v| v.as_object()) { pub fn add_instruction(&mut self, instruction: &serde_json::Value) {
for (k, v) in entries { let Some(obj) = instruction.as_object() else {
combined_entries.insert(k.clone(), v.clone()); return;
};
match obj.get("type").and_then(|v| v.as_str()) {
Some("ChunkListUpdate") => {
if let Some(chunks) = obj.get("chunks").and_then(|v| v.as_object()) {
for (k, v) in chunks {
self.chunks.insert(k.clone(), v.clone());
}
}
if let Some(merged) = obj.get("merged").and_then(|v| v.as_array()) {
for update in merged {
self.push_merged(update);
}
}
}
Some("EcmascriptMergedUpdate") => {
self.push_merged(instruction);
}
// Unknown instruction shapes are ignored; the caller already
// escalates `Total`/`Missing` updates to a full restart.
_ => {}
} }
} }
if let Some(chunks) = obj.get("chunks").and_then(|v| v.as_object()) {
for (k, v) in chunks { fn push_merged(&mut self, update: &serde_json::Value) {
combined_chunks.insert(k.clone(), v.clone()); self.merged.insert(update.clone());
}
} }
}
/// Builds an `Update::Partial` whose instruction is a combined pub fn is_empty(&self) -> bool {
/// `EcmascriptMergedUpdate` covering `entries` and `chunks`. Empty maps are self.chunks.is_empty() && self.merged.is_empty()
/// omitted so an empty `entries`/`chunks` field never appears in the payload.
///
/// Passing empty maps produces an instruction with only `type:
/// "EcmascriptMergedUpdate"`, used to advance `VersionState` to `to` without
/// the JS consumer applying anything: it sees a `partial` event with nothing
/// to apply and short-circuits.
pub fn merged_partial_update(
to: TraitRef<Box<dyn Version>>,
entries: FxHashMap<String, serde_json::Value>,
chunks: FxHashMap<String, serde_json::Value>,
) -> Update {
let mut instruction = serde_json::Map::new();
instruction.insert(
"type".to_string(),
serde_json::Value::String("EcmascriptMergedUpdate".to_string()),
);
if !entries.is_empty() {
instruction.insert(
"entries".to_string(),
serde_json::Value::Object(entries.into_iter().collect()),
);
} }
if !chunks.is_empty() {
pub fn build(self, to: TraitRef<Box<dyn Version>>) -> Update {
let mut instruction = serde_json::Map::new();
instruction.insert( instruction.insert(
"chunks".to_string(), "type".to_string(),
serde_json::Value::Object(chunks.into_iter().collect()), serde_json::Value::String("ChunkListUpdate".to_string()),
); );
if !self.chunks.is_empty() {
instruction.insert(
"chunks".to_string(),
serde_json::Value::Object(self.chunks.into_iter().collect()),
);
}
if !self.merged.is_empty() {
instruction.insert(
"merged".to_string(),
serde_json::Value::Array(self.merged.into_iter().collect()),
);
}
Update::Partial(PartialUpdate {
to,
instruction: Arc::new(serde_json::Value::Object(instruction)),
})
} }
Update::Partial(PartialUpdate {
to,
instruction: Arc::new(serde_json::Value::Object(instruction)),
})
} }
/// Per-chunk [`Update`]s computed against an `AggregateHmrVersion` snapshot. /// Per-chunk [`Update`]s computed against an `AggregateHmrVersion` snapshot.
......
...@@ -1363,12 +1363,45 @@ impl AppEndpoint { ...@@ -1363,12 +1363,45 @@ impl AppEndpoint {
{ {
client_assets.extend(assets.all_assets().await?.iter().copied()); client_assets.extend(assets.all_assets().await?.iter().copied());
} }
let mut ssr_client_reference_chunks: Vec<ResolvedVc<Box<dyn OutputAsset>>> = vec![];
for &assets in client_references_chunks_ref for &assets in client_references_chunks_ref
.client_component_ssr_chunks .client_component_ssr_chunks
.values() .values()
{ {
// TODO(alexkirsz) In which manifest does this go? // TODO(alexkirsz) In which manifest does this go?
server_assets.extend(assets.all_assets().await?.iter().copied()); let all = assets.all_assets().await?;
server_assets.extend(all.iter().copied());
ssr_client_reference_chunks.extend(all.iter().copied());
}
// In development, register a server-side HMR chunk list that owns all
// client-component SSR chunks. This is a bit of a hack to provide a single subscription
// point for these chunks, which aren't normally reachable from the rest of the server-side
// chunk lists.
//
// TODO: This anchor can go away once edges to chunk references are represented explicitly
// in the chunk graph (rsc chunk -> ssr chunk), which would make the SSR chunks reachable
// from the RSC chunk lists directly.
if is_app_page
&& runtime == NextRuntime::NodeJs
&& project
.client_compile_time_info()
.await?
.hot_module_replacement_enabled
{
let ssr_hmr_chunk_list_path = server_path.join(&format!(
"app{original_name}/client-components-ssr.js",
original_name = app_entry.original_name
))?;
let ssr_hmr_chunks = project
.server_chunking_context(process_client_assets)
.server_hmr_chunk_list(
ssr_hmr_chunk_list_path,
Vc::cell(ssr_client_reference_chunks),
)
.to_resolved()
.await?;
server_assets.insert(ssr_hmr_chunks);
} }
// In development, register a page-specific HMR chunk list that owns all client // In development, register a page-specific HMR chunk list that owns all client
......
...@@ -96,10 +96,7 @@ use turbopack_node::worker_threads_backend; ...@@ -96,10 +96,7 @@ use turbopack_node::worker_threads_backend;
use turbopack_nodejs::NodeJsChunkingContext; use turbopack_nodejs::NodeJsChunkingContext;
use crate::{ use crate::{
aggregate_hmr::{ aggregate_hmr::{AggregateHmrVersion, ChunkListUpdateBuilder, DiffResult, diff_chunks_against},
AggregateHmrVersion, DiffResult, diff_chunks_against, merge_ecmascript_merged_update,
merged_partial_update,
},
app::{AppProject, OptionAppProject}, app::{AppProject, OptionAppProject},
empty::EmptyEndpoint, empty::EmptyEndpoint,
entrypoints::Entrypoints, entrypoints::Entrypoints,
...@@ -2478,6 +2475,17 @@ impl Project { ...@@ -2478,6 +2475,17 @@ impl Project {
}) })
} }
#[turbo_tasks::function]
async fn aggregate_hmr_root_path(
self: Vc<Self>,
target: HmrTarget,
) -> Result<Vc<FileSystemPath>> {
match target {
HmrTarget::Client => bail!("aggregate HMR is not implemented for the client"),
HmrTarget::Server => Ok(self.node_root().await?.join("server/app")?.cell()),
}
}
/// Get HMR content by chunk_name for the specified target. /// Get HMR content by chunk_name for the specified target.
#[turbo_tasks::function] #[turbo_tasks::function]
async fn hmr_content( async fn hmr_content(
...@@ -2585,7 +2593,7 @@ impl Project { ...@@ -2585,7 +2593,7 @@ impl Project {
let Some(map) = this.await?.versioned_content_map else { let Some(map) = this.await?.versioned_content_map else {
bail!("must be in dev mode to hmr") bail!("must be in dev mode to hmr")
}; };
let root = this.hmr_root_path(target).owned().await?; let root = this.aggregate_hmr_root_path(target).owned().await?;
AggregateHmrVersion::from_map(*map, &root).await AggregateHmrVersion::from_map(*map, &root).await
} }
let version_op = aggregate_hmr_version_operation(self, target); let version_op = aggregate_hmr_version_operation(self, target);
...@@ -2602,8 +2610,13 @@ impl Project { ...@@ -2602,8 +2610,13 @@ impl Project {
} }
/// Aggregate counterpart to [`Self::hmr_update`]: a single `Update` whose /// Aggregate counterpart to [`Self::hmr_update`]: a single `Update` whose
/// `EcmascriptMergedUpdate` is the union of per-chunk diffs under /// combined `ChunkListUpdate` is the union of the per-entry-chunk diffs
/// `target`'s root. /// under `target`'s root.
///
/// Each tracked entry chunk's own update is a `ChunkListUpdate` (carrying
/// the module deltas for its shared chunks via the merger) or a bare
/// `EcmascriptMergedUpdate`; both are folded into one `ChunkListUpdate` that
/// the runtime applies exactly as it would a single chunk list.
/// ///
/// All-or-nothing restart: any chunk needing `Total`/`Missing` escalates /// All-or-nothing restart: any chunk needing `Total`/`Missing` escalates
/// the whole batch to `Total` (the runtime can't partially restart). New /// the whole batch to `Total` (the runtime can't partially restart). New
...@@ -2622,7 +2635,7 @@ impl Project { ...@@ -2622,7 +2635,7 @@ impl Project {
let Some(map) = self.await?.versioned_content_map else { let Some(map) = self.await?.versioned_content_map else {
bail!("must be in dev mode to hmr") bail!("must be in dev mode to hmr")
}; };
let root = self.hmr_root_path(target).owned().await?; let root = self.aggregate_hmr_root_path(target).owned().await?;
let chunks_versioned_content = map.hmr_chunks_in_path(&root).await?; let chunks_versioned_content = map.hmr_chunks_in_path(&root).await?;
// No chunks to diff yet (e.g. before any endpoints have been written). // No chunks to diff yet (e.g. before any endpoints have been written).
...@@ -2648,13 +2661,10 @@ impl Project { ...@@ -2648,13 +2661,10 @@ impl Project {
// the *next* change produces a real diff; returning `Total` instead would // the *next* change produces a real diff; returning `Total` instead would
// force a needless full re-evaluation. // force a needless full re-evaluation.
if chunk_updates.is_empty() && !has_new_chunks { if chunk_updates.is_empty() && !has_new_chunks {
return Ok( return Ok(ChunkListUpdateBuilder::default().build(to_ref).cell());
merged_partial_update(to_ref, FxHashMap::default(), FxHashMap::default()).cell(),
);
} }
let mut combined_entries: FxHashMap<String, serde_json::Value> = FxHashMap::default(); let mut builder = ChunkListUpdateBuilder::default();
let mut combined_chunks: FxHashMap<String, serde_json::Value> = FxHashMap::default();
for (_path, update) in chunk_updates { for (_path, update) in chunk_updates {
match &*update { match &*update {
Update::None => {} Update::None => {}
...@@ -2662,20 +2672,16 @@ impl Project { ...@@ -2662,20 +2672,16 @@ impl Project {
return Ok(Update::Total(TotalUpdate { to: to_ref }).cell()); return Ok(Update::Total(TotalUpdate { to: to_ref }).cell());
} }
Update::Partial(PartialUpdate { instruction, .. }) => { Update::Partial(PartialUpdate { instruction, .. }) => {
merge_ecmascript_merged_update( builder.add_instruction(instruction);
&mut combined_entries,
&mut combined_chunks,
instruction,
);
} }
} }
} }
if combined_entries.is_empty() && combined_chunks.is_empty() && !has_new_chunks { if builder.is_empty() && !has_new_chunks {
return Ok(Update::None.cell()); return Ok(Update::None.cell());
} }
Ok(merged_partial_update(to_ref, combined_entries, combined_chunks).cell()) Ok(builder.build(to_ref).cell())
} }
/// Gets a list of all HMR chunk names that can be subscribed to for the /// Gets a list of all HMR chunk names that can be subscribed to for the
......
...@@ -15,7 +15,7 @@ use turbopack_core::{ ...@@ -15,7 +15,7 @@ use turbopack_core::{
version::OptionVersionedContent, version::OptionVersionedContent,
}; };
use crate::aggregate_hmr::HmrChunkWithContent; use crate::aggregate_hmr::{HmrChunkWithContent, is_entry_chunk_list_content};
#[derive( #[derive(
Clone, TraceRawVcs, PartialEq, Eq, ValueDebugFormat, Debug, NonLocalValue, Encode, Decode, Clone, TraceRawVcs, PartialEq, Eq, ValueDebugFormat, Debug, NonLocalValue, Encode, Decode,
...@@ -90,16 +90,18 @@ impl VersionedContentMap { ...@@ -90,16 +90,18 @@ impl VersionedContentMap {
.resolved_cell() .resolved_cell()
} }
/// Lists every HMR-eligible chunk under `root` with its [`VersionedContent`], /// Lists the aggregate-HMR *entry* chunks under `root` with their
/// sorted by path. /// [`VersionedContent`], sorted by path. Only entry-chunk-list content is
/// returned (see [`is_entry_chunk_list_content`]). Callers scope which
/// entries are included by narrowing `root` (e.g. the aggregate server-HMR
/// subscription passes `server/app` to include App Router entries only).
/// ///
/// `map_path_to_op` is an `FxHashMap`, whose iteration order /// `map_path_to_op` is an `FxHashMap`, whose iteration order depends on
/// depends on bucket layout rather than insertion order, so the same set of /// bucket layout rather than insertion order, so the same set of paths can
/// paths can come out in a different order across calls. /// come out in a different order across calls. Since this map contains
/// /// entries that span server and client contexts, changes for one context
/// Since this map contains entries that span server and client contexts, changes for one /// can shift the internals of the map, making iteration order different
/// context can shift the internals of the map, making iteration order different for the same /// for the same set of paths.
/// set of paths.
pub async fn hmr_chunks_in_path( pub async fn hmr_chunks_in_path(
self: Vc<Self>, self: Vc<Self>,
root: &FileSystemPath, root: &FileSystemPath,
...@@ -116,10 +118,6 @@ impl VersionedContentMap { ...@@ -116,10 +118,6 @@ impl VersionedContentMap {
.into_iter() .into_iter()
.filter_map(|path| { .filter_map(|path| {
let rel = root.get_path_to(&path)?; let rel = root.get_path_to(&path)?;
// Skip source maps: they aren't HMR-eligible chunks.
if rel.ends_with(".map") {
return None;
}
Some((RcStr::from(rel), path)) Some((RcStr::from(rel), path))
}) })
.map(|(name, path)| async move { .map(|(name, path)| async move {
...@@ -132,6 +130,14 @@ impl VersionedContentMap { ...@@ -132,6 +130,14 @@ impl VersionedContentMap {
return Ok(None); return Ok(None);
} }
let content = asset.versioned_content().to_resolved().await?; let content = asset.versioned_content().to_resolved().await?;
// *Important*: only chunk lists are subscribed to. Individual chunks are already
// covered by the chunk list that owns them, so including them here
// would produce duplicate updates for the same change.
if !is_entry_chunk_list_content(content) {
return Ok(None);
}
Ok(Some(HmrChunkWithContent { Ok(Some(HmrChunkWithContent {
path: name, path: name,
content, content,
......
...@@ -2400,15 +2400,14 @@ async fn get_source_map_rope( ...@@ -2400,15 +2400,14 @@ async fn get_source_map_rope(
return Ok(FileContent::NotFound.cell()); return Ok(FileContent::NotFound.cell());
}; };
let Some(chunk_base_unix) = project.node_root().await?.get_path_to(&fs_path) else { let node_root = project.node_root().await?;
let Some(chunk_base_unix) = node_root.get_path_to(&fs_path).map(ToOwned::to_owned) else {
// The path is not within the dist dir // The path is not within the dist dir
return Ok(FileContent::NotFound.cell()); return Ok(FileContent::NotFound.cell());
}; };
let client_path = project let client_relative_path = project.client_relative_path().await?;
.client_relative_path() let client_path = client_relative_path.join(&chunk_base_unix)?;
.await?
.join(chunk_base_unix)?;
// `fs_path` is the server path: it's inside `node_root`, and `output_fs` is the filesystem // `fs_path` is the server path: it's inside `node_root`, and `output_fs` is the filesystem
// that `node_root` uses. // that `node_root` uses.
...@@ -2421,6 +2420,24 @@ async fn get_source_map_rope( ...@@ -2421,6 +2420,24 @@ async fn get_source_map_rope(
// chunks. // chunks.
map = container.get_source_map(client_path, module.clone()); map = container.get_source_map(client_path, module.clone());
if !map.await?.is_content() { if !map.await?.is_content() {
// An older revision of a chunk's sourcemap may be requested by an HMR client. We
// remove stale entries from the VersionStateMap but a client may be holding onto a
// reference to a stale chunk and requesting its sourcmemap via the error
// overlay.
//
// This exists because we don't have logic for the server hmr client to mark which
// chunks it's no longer using.
//
// Fall back to reading from the filesystem.
let map_relative = format!("{chunk_base_unix}.map");
let server_map = node_root.join(&map_relative)?.read();
if server_map.await?.is_content() {
return Ok(server_map);
}
let client_map = client_relative_path.join(&map_relative)?.read();
if client_map.await?.is_content() {
return Ok(client_map);
}
bail!("chunk/module {sys_path:?} (module: {module:?}) is missing a sourcemap"); bail!("chunk/module {sys_path:?} (module: {module:?}) is missing a sourcemap");
} }
} }
......
...@@ -1456,5 +1456,7 @@ ...@@ -1456,5 +1456,7 @@
"1455": "Page \"%s\" returned incomplete params from \"generateStaticParams()\". With \"output: export\", every params object must include all dynamic route parameters. Missing: %s. See more info here: https://nextjs.org/docs/messages/generate-static-params", "1455": "Page \"%s\" returned incomplete params from \"generateStaticParams()\". With \"output: export\", every params object must include all dynamic route parameters. Missing: %s. See more info here: https://nextjs.org/docs/messages/generate-static-params",
"1456": "Expected an HTML size for prerendered app route \"%s\"", "1456": "Expected an HTML size for prerendered app route \"%s\"",
"1457": "Expected prerender classification data for PPR route \"%s\"", "1457": "Expected prerender classification data for PPR route \"%s\"",
"1458": "Expected complete prerender classification for route \"%s\"" "1458": "Expected complete prerender classification for route \"%s\"",
"1459": "[Server HMR] unreachable: unknown HMR instruction type %s",
"1460": "[Server HMR] unreachable: unexpected update instruction type %s"
} }
...@@ -234,20 +234,28 @@ export type Update = IssuesUpdate | PartialUpdate ...@@ -234,20 +234,28 @@ export type Update = IssuesUpdate | PartialUpdate
* The runtime file cannot import from this ES module without triggering module semantics, * The runtime file cannot import from this ES module without triggering module semantics,
* so we maintain a copy there. Please keep both definitions in sync. * so we maintain a copy there. Please keep both definitions in sync.
*/ */
export interface NodeJsEcmascriptMergedUpdate {
type: 'EcmascriptMergedUpdate'
entries?: Record<
string,
{ code: string; url: string; map?: string | undefined }
>
chunks?: Record<
string,
| { type: 'added' | 'deleted'; modules?: string[] }
| { type: 'partial'; added?: string[]; deleted?: string[] }
>
}
export interface NodeJsChunkListUpdate {
type: 'ChunkListUpdate'
merged?: NodeJsEcmascriptMergedUpdate[]
chunks?: Record<string, { type: 'added' | 'deleted' | 'total' | 'partial' }>
}
export interface NodeJsPartialHmrUpdate extends BaseUpdate { export interface NodeJsPartialHmrUpdate extends BaseUpdate {
type: 'partial' type: 'partial'
instruction: { instruction: NodeJsEcmascriptMergedUpdate | NodeJsChunkListUpdate
type: 'EcmascriptMergedUpdate'
entries?: Record<
string,
{ code: string; url: string; map?: string | undefined }
>
chunks?: Record<
string,
| { type: 'added' | 'deleted'; modules?: string[] }
| { type: 'partial'; added?: string[]; deleted?: string[] }
>
}
} }
export interface NodeJsRestartHmrUpdate { export interface NodeJsRestartHmrUpdate {
......
...@@ -38,6 +38,7 @@ import { ...@@ -38,6 +38,7 @@ import {
} from './middleware-turbopack' } from './middleware-turbopack'
import { PageNotFoundError } from '../../shared/lib/utils' import { PageNotFoundError } from '../../shared/lib/utils'
import { debounce } from '../utils' import { debounce } from '../utils'
import { clearManifestCache } from '../load-manifest.external'
import { deleteCache } from './require-cache' import { deleteCache } from './require-cache'
import { import {
clearAllModuleContexts, clearAllModuleContexts,
...@@ -169,12 +170,45 @@ declare global { ...@@ -169,12 +170,45 @@ declare global {
var __turbopack_server_hmr_handlers__: Map<string, unknown> | undefined var __turbopack_server_hmr_handlers__: Map<string, unknown> | undefined
} }
/**
* Collects the output chunk paths touched by a partial HMR update. Both
* single-chunk `EcmascriptMergedUpdate`s and `ChunkListUpdate`s (which nest
* per-chunk deltas inside `merged`) are flattened so the manifest cache can be
* invalidated for every affected chunk after a successful apply.
*/
function collectUpdatedChunkPaths(
instruction: NodeJsPartialHmrUpdate['instruction']
): string[] {
const paths = new Set<string>()
if (instruction.type === 'EcmascriptMergedUpdate') {
for (const chunkPath of Object.keys(instruction.chunks ?? {})) {
paths.add(chunkPath)
}
} else if (instruction.type === 'ChunkListUpdate') {
for (const chunkPath of Object.keys(instruction.chunks ?? {})) {
paths.add(chunkPath)
}
for (const merged of instruction.merged ?? []) {
for (const chunkPath of Object.keys(merged.chunks ?? {})) {
paths.add(chunkPath)
}
}
} else {
throw new Error(
`[Server HMR] unreachable: unknown HMR instruction type ${(instruction as { type: string }).type}`
)
}
return Array.from(paths)
}
function setupServerHmr( function setupServerHmr(
project: Project, project: Project,
{ {
reEvaluateAllModulesExpensive, reEvaluateAllModulesExpensive,
onApplied,
}: { }: {
reEvaluateAllModulesExpensive: () => void | Promise<void> reEvaluateAllModulesExpensive: () => void | Promise<void>
onApplied: (chunkPaths: string[]) => void | Promise<void>
} }
) { ) {
async function runSubscription() { async function runSubscription() {
...@@ -205,8 +239,14 @@ function setupServerHmr( ...@@ -205,8 +239,14 @@ function setupServerHmr(
// knows how to apply; `ChunkListUpdate` is browser-only. Anything else is // knows how to apply; `ChunkListUpdate` is browser-only. Anything else is
// unknown to us, so ignore it rather than evicting the module cache. // unknown to us, so ignore it rather than evicting the module cache.
const instruction = update.instruction const instruction = update.instruction
if (!instruction || instruction.type !== 'EcmascriptMergedUpdate') { if (
continue !instruction ||
(instruction.type !== 'EcmascriptMergedUpdate' &&
instruction.type !== 'ChunkListUpdate')
) {
throw new Error(
`[Server HMR] unreachable: unexpected update instruction type ${(instruction as { type: string }).type}`
)
} }
// No handler registered yet (before first request, or right after // No handler registered yet (before first request, or right after
...@@ -222,8 +262,18 @@ function setupServerHmr( ...@@ -222,8 +262,18 @@ function setupServerHmr(
__turbopack_server_hmr_apply__(update) __turbopack_server_hmr_apply__(update)
} catch { } catch {
// A matching runtime tried the apply and threw. Evict require.cache // A matching runtime tried the apply and threw. Evict require.cache
// so the next request loads fresh. (A no-match update is a no-op.) // so the next request loads fresh, then skip onApplied. (A no-match
// update is a no-op and does not throw.)
await reEvaluateAllModulesExpensive() await reEvaluateAllModulesExpensive()
continue
}
const updatedChunkPaths = collectUpdatedChunkPaths(instruction)
// An empty partial only advances the version state (e.g. the seed
// transition or a new endpoint); nothing changed on disk, so don't
// invalidate manifests or ping browsers to refetch RSC.
if (updatedChunkPaths.length > 0) {
await onApplied(updatedChunkPaths)
} }
} else { } else {
await reEvaluateAllModulesExpensive() await reEvaluateAllModulesExpensive()
...@@ -584,9 +634,11 @@ export async function createHotReloaderTurbopack( ...@@ -584,9 +634,11 @@ export async function createHotReloaderTurbopack(
} else { } else {
// Figure out if the server files have changed // Figure out if the server files have changed
let hasChange = false let hasChange = false
const currentPaths = new Set<string>()
for (const { path, contentHash } of writtenEndpoint.serverPaths) { for (const { path, contentHash } of writtenEndpoint.serverPaths) {
// We ignore source maps // We ignore source maps
if (path.endsWith('.map')) continue if (path.endsWith('.map')) continue
currentPaths.add(path)
const localKey = `${key}:${path}` const localKey = `${key}:${path}`
const localHash = serverPathState.get(localKey) const localHash = serverPathState.get(localKey)
const globalHash = serverPathState.get(path) const globalHash = serverPathState.get(path)
...@@ -607,11 +659,27 @@ export async function createHotReloaderTurbopack( ...@@ -607,11 +659,27 @@ export async function createHotReloaderTurbopack(
} }
} }
const localKeyPrefix = `${key}:`
for (const pathKey of serverPathState.keys()) {
if (
pathKey.startsWith(localKeyPrefix) &&
!currentPaths.has(pathKey.slice(localKeyPrefix.length))
) {
serverPathState.delete(pathKey)
hasChange = true
}
}
if (!hasChange) { if (!hasChange) {
return false return false
} }
} }
// Edge does not participate in server HMR.
if (writtenEndpoint.type === 'edge') {
void clearAllModuleContexts()
}
const serverPaths = writtenEndpoint.serverPaths.map(({ path: p }) => const serverPaths = writtenEndpoint.serverPaths.map(({ path: p }) =>
join(distDir, p) join(distDir, p)
) )
...@@ -752,16 +820,23 @@ export async function createHotReloaderTurbopack( ...@@ -752,16 +820,23 @@ export async function createHotReloaderTurbopack(
} }
} }
function sendEnqueuedMessages() { function hasCompilationErrors() {
for (const [, issueMap] of currentEntryIssues) { for (const [, issueMap] of currentEntryIssues) {
if ( if (
[...issueMap.values()].filter((i) => i.severity !== 'warning').length > [...issueMap.values()].filter((i) => i.severity !== 'warning').length >
0 0
) { ) {
// During compilation errors we want to delay the HMR events until errors are fixed return true
return
} }
} }
return false
}
function sendEnqueuedMessages() {
if (hasCompilationErrors()) {
// During compilation errors we want to delay the HMR events until errors are fixed
return
}
for (const client of [ for (const client of [
...clientsWithoutHtmlRequestId, ...clientsWithoutHtmlRequestId,
...@@ -1871,6 +1946,7 @@ export async function createHotReloaderTurbopack( ...@@ -1871,6 +1946,7 @@ export async function createHotReloaderTurbopack(
force: forceDeleteCache, force: forceDeleteCache,
}) })
}, },
serverFastRefresh,
}, },
}) })
} finally { } finally {
...@@ -2019,6 +2095,18 @@ export async function createHotReloaderTurbopack( ...@@ -2019,6 +2095,18 @@ export async function createHotReloaderTurbopack(
process.exit(1) process.exit(1)
}) })
// Tell browsers to refetch RSC (soft refresh, not full page reload).
// Skip while there are outstanding compilation errors: an RSC refetch would
// 500 and force a full-page navigation, losing client state (e.g. recovering
// from a syntax error). A subsequent successful compile/apply fires this
// again to refresh.
function notifyServerComponentChanges() {
if (hasCompilationErrors()) return
hotReloader.send({
type: HMR_MESSAGE_SENT_TO_BROWSER.SERVER_COMPONENT_CHANGES,
})
}
if (serverFastRefresh) { if (serverFastRefresh) {
setupServerHmr(project, { setupServerHmr(project, {
reEvaluateAllModulesExpensive: async () => { reEvaluateAllModulesExpensive: async () => {
...@@ -2045,10 +2133,18 @@ export async function createHotReloaderTurbopack( ...@@ -2045,10 +2133,18 @@ export async function createHotReloaderTurbopack(
resetFetch() resetFetch()
// Tell browsers to refetch RSC (soft refresh, not full page reload) notifyServerComponentChanges()
hotReloader.send({ },
type: HMR_MESSAGE_SENT_TO_BROWSER.SERVER_COMPONENT_CHANGES, onApplied: (chunkPaths: string[]) => {
}) // Clear the evalManifest() shared cache for each updated chunk so the
// next RSC render picks up the HMR-applied module changes. Unlike
// a full restart, this does NOT clear require.cache — the HMR-applied
// modules in devModuleCache must persist for dep preservation.
for (const chunkPath of chunkPaths) {
clearManifestCache(join(distDir, chunkPath))
}
notifyServerComponentChanges()
}, },
}) })
} }
......
...@@ -142,6 +142,14 @@ type HandleRouteTypeHooks = { ...@@ -142,6 +142,14 @@ type HandleRouteTypeHooks = {
handleWrittenEndpoint: HandleWrittenEndpoint handleWrittenEndpoint: HandleWrittenEndpoint
subscribeToChanges: StartChangeSubscription subscribeToChanges: StartChangeSubscription
handleServerComponentChanges?: () => void handleServerComponentChanges?: () => void
// When Turbopack server fast refresh is enabled, the aggregate server-HMR
// subscription (setupServerHmr `onApplied` in hot-reloader-turbopack.ts)
// owns the browser refresh signal for app-page RSC changes and only fires
// after the server module cache is refreshed. In that mode the per-page
// `rscHmrEndpoint` subscription must NOT also send SERVER_COMPONENT_CHANGES,
// or every edit triggers two RSC refetches (the first immediately
// superseded).
serverFastRefresh?: boolean
} }
export async function handleRouteType({ export async function handleRouteType({
...@@ -365,6 +373,21 @@ export async function handleRouteType({ ...@@ -365,6 +373,21 @@ export async function handleRouteType({
} }
// Report the next compilation again // Report the next compilation again
readyIds?.delete(pathname) readyIds?.delete(pathname)
// When server fast refresh is enabled, the aggregate server-HMR
// subscription sends SERVER_COMPONENT_CHANGES after applying the
// update in-process. Sending here too would double the refresh.
//
// But the aggregate subscription only fires when there is a live
// server-HMR handler registered (i.e. the page has rendered at
// least once). When recovering from a build error the page never
// rendered, so no handler exists, the aggregate stays silent, and
// this per-page send is the only thing that clears the redbox.
// Only suppress when a handler is actually live to own the refresh.
const hasLiveServerHmrHandler =
(globalThis.__turbopack_server_hmr_handlers__?.size ?? 0) > 0
if (hooks?.serverFastRefresh && hasLiveServerHmrHandler) {
return
}
hooks?.handleServerComponentChanges?.() hooks?.handleServerComponentChanges?.()
}, },
(e) => { (e) => {
......
...@@ -403,7 +403,7 @@ describe('ReactRefreshLogBox app', () => { ...@@ -403,7 +403,7 @@ describe('ReactRefreshLogBox app', () => {
"stack": [ "stack": [
"ClickCount.render Child.js (4:11)", "ClickCount.render Child.js (4:11)",
"Home index.js (6:7)", "Home index.js (6:7)",
"<FIXME-file-protocol>", "Page app/page.js (4:10)",
], ],
} }
`) `)
......
...@@ -528,7 +528,7 @@ describe('Error recovery app', () => { ...@@ -528,7 +528,7 @@ describe('Error recovery app', () => {
"stack": [ "stack": [
"Child child.js (3:9)", "Child child.js (3:9)",
"Index index.js (6:7)", "Index index.js (6:7)",
"<FIXME-file-protocol>", "Page app/client/page.js (4:10)",
], ],
} }
`) `)
...@@ -828,7 +828,7 @@ describe('Error recovery app', () => { ...@@ -828,7 +828,7 @@ describe('Error recovery app', () => {
"stack": [ "stack": [
"Foo Foo.js (3:3)", "Foo Foo.js (3:3)",
"FunctionDefault index.js (4:10)", "FunctionDefault index.js (4:10)",
"<FIXME-file-protocol>", "Page app/page.js (4:10)",
], ],
} }
`) `)
......
'use client'
export function ClientGreeting({ text }: { text: string }) {
return <p id="client-greeting">{text}</p>
}
// An always-imported dependency that is never modified during the test.
// Its evaluation timestamp detects whether clear() fired.
export const depEvalTime = Date.now()
import { depEvalTime } from './dep'
const evaluatedAt = Date.now()
export default function Page() {
return (
<div>
<p id="greeting">hello world</p>
<p id="module-eval-time">Module Evaluated At: {evaluatedAt}</p>
<p id="dep-eval-time">Dep Evaluated At: {depEvalTime}</p>
<p id="client-component">not imported yet</p>
</div>
)
}
// An always-imported (synchronous) dependency that is never modified during the
// test. Its evaluation timestamp is used to detect whether clear() fired:
// clear() wipes require.cache and re-evaluates every module, which would change
// this timestamp. A partial HMR apply leaves unmodified modules untouched.
export const depEvalTime = Date.now()
// Added to lazy.ts's import graph during the test. It starts out unreferenced,
// then the test adds an import of it inside the dynamically-imported module.
// This changes the dynamic chunk's content hash and thus its path.
export const lazyNewModuleValue = 'from-lazy-new-module'
// This module is loaded via dynamic import() on the server, so it lands in its
// own chunk under server/chunks/ rather than the entry chunk. Editing it (or
// adding a new import to it) changes that chunk's content hash and therefore its
// path — the exact "availability info / chunk rename" scenario, but for a
// dynamically-imported chunk instead of the synchronous entry chunk graph.
export const lazyValue = 'lazy-v0'
import { depEvalTime } from './dep'
const evaluatedAt = Date.now()
export default async function Page() {
// Load a module in its own chunk via a server-side dynamic import(). The
// chunk lives under server/chunks/ and is not part of the entry chunk's
// synchronous chunk list, so its updates are surfaced through the async
// loader module rather than the entry chunk list.
const { lazyValue } = await import('./lazy')
return (
<div>
<p id="greeting">hello world</p>
<p id="module-eval-time">Module Evaluated At: {evaluatedAt}</p>
<p id="dep-eval-time">Dep Evaluated At: {depEvalTime}</p>
<p id="lazy-value">{lazyValue}</p>
<p id="lazy-new-module-value">not imported yet</p>
</div>
)
}
// An always-imported dependency that is never modified during the test.
// Its evaluation timestamp is used to detect whether clear() fired: clear()
// wipes require.cache and re-evaluates every module, which would change this
// timestamp. A partial HMR apply leaves unmodified modules untouched.
export const depEvalTime = Date.now()
// This module is dynamically imported by the page during HMR tests.
// It starts not being imported, then the test adds the import.
export const newModuleValue = 'from-new-module'
export const newModuleEvalTime = Date.now()
import { depEvalTime } from './dep'
const pageLoadId = Math.random().toString(36).slice(2)
const evaluatedAt = Date.now()
export default function Page() {
return (
<div>
<p id="greeting">hello world</p>
<p id="page-load-id">Page Load ID: {pageLoadId}</p>
<p id="module-eval-time">Module Evaluated At: {evaluatedAt}</p>
<p id="dep-eval-time">Dep Evaluated At: {depEvalTime}</p>
<p id="new-module-value">not imported yet</p>
</div>
)
}
...@@ -124,7 +124,6 @@ const STATS_RELATIVE_PATH = '.next/warm-restart-task-stats.json' ...@@ -124,7 +124,6 @@ const STATS_RELATIVE_PATH = '.next/warm-restart-task-stats.json'
expect(missed).toMatchInlineSnapshot(` expect(missed).toMatchInlineSnapshot(`
[ [
"<turbopack_browser::ecmascript::list::content::EcmascriptDevChunkListContent as dyn turbopack_core::version::VersionedContent>::update", "<turbopack_browser::ecmascript::list::content::EcmascriptDevChunkListContent as dyn turbopack_core::version::VersionedContent>::update",
"<turbopack_core::version::FileHashVersion as dyn turbopack_core::version::Version>::id",
"next_api::project::Project::hmr_update", "next_api::project::Project::hmr_update",
"next_api::project::Project::hmr_version_state", "next_api::project::Project::hmr_version_state",
"next_napi_bindings::next_api::project::hmr_update_with_issues_operation", "next_napi_bindings::next_api::project::hmr_update_with_issues_operation",
......
...@@ -38,7 +38,7 @@ enum CurrentChunkMethodWithData { ...@@ -38,7 +38,7 @@ enum CurrentChunkMethodWithData {
/// Contents of an [`EcmascriptDevChunkList`]. /// Contents of an [`EcmascriptDevChunkList`].
#[turbo_tasks::value] #[turbo_tasks::value]
pub(super) struct EcmascriptDevChunkListContent { pub struct EcmascriptDevChunkListContent {
current_chunk_method: CurrentChunkMethodWithData, current_chunk_method: CurrentChunkMethodWithData,
#[bincode(with = "turbo_bincode::indexmap")] #[bincode(with = "turbo_bincode::indexmap")]
pub(super) chunks_contents: FxIndexMap<String, ResolvedVc<Box<dyn VersionedContent>>>, pub(super) chunks_contents: FxIndexMap<String, ResolvedVc<Box<dyn VersionedContent>>>,
......
pub(crate) mod asset; pub(crate) mod asset;
pub(crate) mod content; pub mod content;
pub(crate) mod chunk; pub(crate) mod chunk;
pub(crate) mod content; pub(crate) mod content;
pub(crate) mod evaluate; pub(crate) mod evaluate;
pub(crate) mod list; pub mod list;
pub(crate) mod merged; pub(crate) mod merged;
pub(crate) mod update; pub(crate) mod update;
pub(crate) mod version; pub(crate) mod version;
......
...@@ -57,10 +57,20 @@ if (handlers.size === 0) { ...@@ -57,10 +57,20 @@ if (handlers.size === 0) {
): void => { ): void => {
const registry: Map<string, HmrHandlerEntry> = const registry: Map<string, HmrHandlerEntry> =
globalThis.__turbopack_server_hmr_handlers__ ?? new Map() globalThis.__turbopack_server_hmr_handlers__ ?? new Map()
const updateChunkPaths = Object.keys(update.instruction?.chunks ?? {})
// Chunk paths can appear either directly on the instruction (single-chunk
// updates) or nested inside `merged` entries (chunks covered by a
// merger). Collect both so routing isn't skipped just because a mergeable
// chunk's update only reports its paths inside `merged`.
const updateChunkPaths = new Set<string>([
...Object.keys(update.instruction?.chunks ?? {}),
...(update.instruction?.merged ?? []).flatMap((merged) =>
Object.keys(merged.chunks ?? {})
),
])
const toCall: HmrHandlerEntry[] = [] const toCall: HmrHandlerEntry[] = []
if (updateChunkPaths.length === 0) { if (updateChunkPaths.size === 0) {
for (const entry of registry.values()) toCall.push(entry) for (const entry of registry.values()) toCall.push(entry)
} else { } else {
const seen = new Set<string>() const seen = new Set<string>()
......
...@@ -10,7 +10,7 @@ type NodeJsHmrPayload = { ...@@ -10,7 +10,7 @@ type NodeJsHmrPayload = {
} }
issues: Issue[] issues: Issue[]
type: 'partial' type: 'partial'
instruction: EcmascriptMergedUpdate instruction: NodeJsEcmascriptMergedUpdate | NodeJsChunkListUpdate
} }
/** /**
...@@ -79,38 +79,79 @@ function handleNodejsUpdate( ...@@ -79,38 +79,79 @@ function handleNodejsUpdate(
} }
const instruction = msg.instruction const instruction = msg.instruction
if (instruction.type !== 'EcmascriptMergedUpdate') {
return
}
try { try {
const { entries = {}, chunks = {} } = instruction if (instruction.type === 'ChunkListUpdate') {
// All node ecmascript chunks are mergeable, so a `total`/`partial` here
// means a non-mergeable asset changed in an unsupported way. Escalate
// to a full clear() rather than leave stale factories in memory.
for (const [chunkPath, chunkUpdate] of Object.entries(
instruction.chunks ?? {}
)) {
if (chunkUpdate.type === 'total' || chunkUpdate.type === 'partial') {
throw new Error(
`unsupported '${chunkUpdate.type}' update for chunk ${chunkPath}`
)
}
}
const evalModuleEntry = (entry: EcmascriptModuleEntry) => { if (instruction.merged) {
// eslint-disable-next-line no-eval for (const merged of instruction.merged) {
return (0, eval)(entry.map ? inlineSourcemaps(entry) : entry.code) applyEcmascriptMergedUpdate(merged, moduleFactories, devModuleCache)
}
}
return
} }
const { added, modified } = computeChangedModules( if (instruction.type === 'EcmascriptMergedUpdate') {
entries, applyEcmascriptMergedUpdate(instruction, moduleFactories, devModuleCache)
chunks, return
undefined // no chunkModulesMap for Node.js }
)
// Use shared HMR update implementation
applyEcmascriptMergedUpdateShared({
added,
modified,
disposedModules: [], // no disposedModules for Node.js (no chunk management)
evalModuleEntry,
instantiateModule,
applyModuleFactoryName: () => {}, // Node doesn't use this
moduleFactories,
devModuleCache,
autoAcceptRootModules: true,
})
} catch (e) { } catch (e) {
console.error('[Server HMR] Update failed, full reload needed:', e) console.error('[Server HMR] Update failed, full reload needed:', e)
throw e throw e
} }
} }
function applyEcmascriptMergedUpdate(
instruction: NodeJsEcmascriptMergedUpdate,
moduleFactories: ModuleFactories,
devModuleCache: ModuleCache<HotModule>
): void {
const { entries = {}, chunks = {} } = instruction
const evalModuleEntry = (entry: EcmascriptModuleEntry) => {
const code = entry.map ? inlineSourcemaps(entry) : entry.code
// eslint-disable-next-line no-eval
return (0, eval)(`(require) => ${code}`)(require)
}
const { added, modified } = computeChangedModules(
entries,
chunks,
undefined // no chunkModulesMap for Node.js
)
// Modules that appear in an "added" chunk but already exist in the cache
// were moved to a renamed chunk. Treat them as modified so the dependency
// walk runs and they get re-instantiated with the new factory.
for (const [moduleId, entry] of added) {
if (entry != null && devModuleCache[moduleId] != null) {
added.delete(moduleId)
modified.set(moduleId, entry)
}
}
// Use shared HMR update implementation
applyEcmascriptMergedUpdateShared({
added,
modified,
disposedModules: [], // no disposedModules for Node.js (no chunk management)
evalModuleEntry,
instantiateModule,
applyModuleFactoryName: () => {}, // Node doesn't use this
moduleFactories,
devModuleCache,
autoAcceptRootModules: true,
})
}
...@@ -7,16 +7,28 @@ ...@@ -7,16 +7,28 @@
* Keep NodeJsPartialHmrUpdate in sync with the copy in packages/next/src/build/swc/types.ts. * Keep NodeJsPartialHmrUpdate in sync with the copy in packages/next/src/build/swc/types.ts.
*/ */
interface NodeJsEcmascriptMergedUpdate {
type: 'EcmascriptMergedUpdate'
entries?: Record<
string,
{ code: string; url: string; map?: string | undefined }
>
chunks?: Record<
string,
| { type: 'added' | 'deleted'; modules?: string[] }
| { type: 'partial'; added?: string[]; deleted?: string[] }
>
}
interface NodeJsChunkListUpdate {
type: 'ChunkListUpdate'
merged?: NodeJsEcmascriptMergedUpdate[]
chunks?: Record<string, { type: 'added' | 'deleted' | 'total' | 'partial' }>
}
interface NodeJsPartialHmrUpdate { interface NodeJsPartialHmrUpdate {
type: 'partial' type: 'partial'
instruction: { instruction: NodeJsEcmascriptMergedUpdate | NodeJsChunkListUpdate
type: 'EcmascriptMergedUpdate'
entries: Record<
string,
{ code: string; url: string; map?: string | undefined }
>
chunks?: Record<string, { type: 'partial' }>
}
} }
interface NodeJsRestartHmrUpdate { interface NodeJsRestartHmrUpdate {
......
...@@ -35,7 +35,8 @@ use turbopack_ecmascript::{ ...@@ -35,7 +35,8 @@ use turbopack_ecmascript::{
use turbopack_ecmascript_runtime::RuntimeType; use turbopack_ecmascript_runtime::RuntimeType;
use crate::ecmascript::node::{ use crate::ecmascript::node::{
chunk::EcmascriptBuildNodeChunk, entry::chunk::EcmascriptBuildNodeEntryChunk, chunk::EcmascriptBuildNodeChunk,
entry::{chunk::EcmascriptBuildNodeEntryChunk, chunk_list::EcmascriptBuildNodeChunkList},
}; };
/// A builder for [`Vc<NodeJsChunkingContext>`]. /// A builder for [`Vc<NodeJsChunkingContext>`].
...@@ -313,6 +314,31 @@ impl NodeJsChunkingContext { ...@@ -313,6 +314,31 @@ impl NodeJsChunkingContext {
pub fn asset_prefix(&self) -> Vc<Option<RcStr>> { pub fn asset_prefix(&self) -> Vc<Option<RcStr>> {
Vc::cell(self.asset_prefix.clone()) Vc::cell(self.asset_prefix.clone())
} }
/// Creates a standalone server-HMR tracking anchor at `path` covering
/// `chunks`, without producing an evaluate chunk.
///
/// Unlike the browser's `hmr_chunk_list`, the caller supplies an explicit
/// output `path` so the anchor can be placed alongside the App Router
/// entries it belongs to (under `server/app/`). This is what lets the
/// aggregate server-HMR subscription scope tracking to App Router: the
/// anchor for client-component SSR chunks (which are physically emitted
/// under the shared `server/chunks/ssr/`) is registered under the app
/// entry's directory so it rides the same App Router scope.
#[turbo_tasks::function]
pub async fn server_hmr_chunk_list(
self: ResolvedVc<Self>,
path: FileSystemPath,
chunks: Vc<OutputAssets>,
) -> Result<Vc<Box<dyn OutputAsset>>> {
#[cfg(debug_assertions)]
if !matches!(*self.runtime_type().await?, RuntimeType::Development) {
bail!("server_hmr_chunk_list can only be used in development");
}
Ok(Vc::upcast(EcmascriptBuildNodeChunkList::new(
*self, path, chunks,
)))
}
} }
impl NodeJsChunkingContext { impl NodeJsChunkingContext {
......
...@@ -7,7 +7,9 @@ use turbopack_core::{ ...@@ -7,7 +7,9 @@ use turbopack_core::{
code_builder::{Code, CodeBuilder}, code_builder::{Code, CodeBuilder},
output::OutputAsset, output::OutputAsset,
source_map::{GenerateSourceMap, SourceMapAsset}, source_map::{GenerateSourceMap, SourceMapAsset},
version::{Update, Version, VersionedContent}, version::{
MergeableVersionedContent, Update, Version, VersionedContent, VersionedContentMerger,
},
}; };
use turbopack_ecmascript::{ use turbopack_ecmascript::{
chunk::{EcmascriptChunkContent, EcmascriptChunkContentEntries}, chunk::{EcmascriptChunkContent, EcmascriptChunkContentEntries},
...@@ -16,8 +18,8 @@ use turbopack_ecmascript::{ ...@@ -16,8 +18,8 @@ use turbopack_ecmascript::{
}; };
use super::{ use super::{
chunk::EcmascriptBuildNodeChunk, update::update_node_chunk, chunk::EcmascriptBuildNodeChunk, merged::merger::EcmascriptBuildNodeChunkContentMerger,
version::EcmascriptBuildNodeChunkVersion, update::update_node_chunk, version::EcmascriptBuildNodeChunkVersion,
}; };
use crate::NodeJsChunkingContext; use crate::NodeJsChunkingContext;
...@@ -140,3 +142,11 @@ impl VersionedContent for EcmascriptBuildNodeChunkContent { ...@@ -140,3 +142,11 @@ impl VersionedContent for EcmascriptBuildNodeChunkContent {
Ok(update_node_chunk(self, from_version).await?.cell()) Ok(update_node_chunk(self, from_version).await?.cell())
} }
} }
#[turbo_tasks::value_impl]
impl MergeableVersionedContent for EcmascriptBuildNodeChunkContent {
#[turbo_tasks::function]
fn get_merger(&self) -> Vc<Box<dyn VersionedContentMerger>> {
Vc::upcast(EcmascriptBuildNodeChunkContentMerger::new())
}
}
...@@ -14,11 +14,15 @@ use turbopack_core::{ ...@@ -14,11 +14,15 @@ use turbopack_core::{
OutputAssetsWithReferenced, OutputAssetsWithReferenced,
}, },
source_map::{GenerateSourceMap, SourceMapAsset}, source_map::{GenerateSourceMap, SourceMapAsset},
version::VersionedContent,
}; };
use turbopack_ecmascript::{chunk::EcmascriptChunkPlaceable, utils::StringifyJs}; use turbopack_ecmascript::{chunk::EcmascriptChunkPlaceable, utils::StringifyJs};
use turbopack_ecmascript_runtime::RuntimeType; use turbopack_ecmascript_runtime::RuntimeType;
use super::runtime::EcmascriptBuildNodeRuntimeChunk; use super::{
chunk_list_content::EcmascriptBuildNodeChunkListContent,
runtime::EcmascriptBuildNodeRuntimeChunk,
};
use crate::NodeJsChunkingContext; use crate::NodeJsChunkingContext;
/// An Ecmascript chunk that loads a list of parallel chunks, then instantiates /// An Ecmascript chunk that loads a list of parallel chunks, then instantiates
...@@ -223,6 +227,23 @@ impl Asset for EcmascriptBuildNodeEntryChunk { ...@@ -223,6 +227,23 @@ impl Asset for EcmascriptBuildNodeEntryChunk {
FileContent::Content(File::from(code.source_code().clone())).cell(), FileContent::Content(File::from(code.source_code().clone())).cell(),
)) ))
} }
#[turbo_tasks::function]
fn versioned_content(self: Vc<Self>) -> Vc<Box<dyn VersionedContent>> {
Vc::upcast(self.chunk_list_content())
}
}
#[turbo_tasks::value_impl]
impl EcmascriptBuildNodeEntryChunk {
#[turbo_tasks::function]
fn chunk_list_content(&self) -> Vc<EcmascriptBuildNodeChunkListContent> {
EcmascriptBuildNodeChunkListContent::new(
*self.chunking_context,
*self.other_chunks,
*self.references,
)
}
} }
#[turbo_tasks::value_impl] #[turbo_tasks::value_impl]
......
use turbo_rcstr::rcstr;
use turbo_tasks::{ResolvedVc, ValueToString, Vc};
use turbo_tasks_fs::{File, FileContent, FileSystemPath};
use turbopack_core::{
asset::{Asset, AssetContent},
output::{OutputAsset, OutputAssets, OutputAssetsReference, OutputAssetsWithReferenced},
version::VersionedContent,
};
use super::chunk_list_content::EcmascriptBuildNodeChunkListContent;
use crate::NodeJsChunkingContext;
/// A server-side HMR tracking anchor for a set of chunks built outside any
/// entry's own module graph (e.g. client-component SSR chunks produced via
/// separate `chunk_group(IsolatedMerged)` calls).
///
/// It is never `require()`d at runtime — it exists only so the aggregate
/// server-HMR subscription has a *stable-ident* [`VersionedContent`]
/// ([`EcmascriptBuildNodeChunkListContent`], whose version is a
/// `ChunkListVersion`) covering these chunks. The stable ident means adding,
/// removing, or renaming a covered chunk doesn't force a `TotalUpdate`
/// (restart); edits ride the merged `ChunkListUpdate` just like the entry's own
/// chunk list.
#[turbo_tasks::value(shared)]
#[derive(ValueToString)]
#[value_to_string("Ecmascript Build Node Chunk List")]
pub(crate) struct EcmascriptBuildNodeChunkList {
pub(super) chunking_context: ResolvedVc<NodeJsChunkingContext>,
/// Explicit output path for this tracking anchor. The caller controls
/// placement so the anchor can live alongside the entries it belongs to
/// (e.g. under `server/app/`), which is how the aggregate server-HMR
/// subscription scopes what it tracks.
pub(super) path: FileSystemPath,
pub(super) chunks: ResolvedVc<OutputAssets>,
}
#[turbo_tasks::value_impl]
impl EcmascriptBuildNodeChunkList {
#[turbo_tasks::function]
pub fn new(
chunking_context: ResolvedVc<NodeJsChunkingContext>,
path: FileSystemPath,
chunks: ResolvedVc<OutputAssets>,
) -> Vc<Self> {
EcmascriptBuildNodeChunkList {
chunking_context,
path,
chunks,
}
.cell()
}
#[turbo_tasks::function]
fn own_content(&self) -> Vc<EcmascriptBuildNodeChunkListContent> {
EcmascriptBuildNodeChunkListContent::new_from_chunks(*self.chunking_context, *self.chunks)
}
}
#[turbo_tasks::value_impl]
impl OutputAssetsReference for EcmascriptBuildNodeChunkList {
#[turbo_tasks::function]
fn references(&self) -> Vc<OutputAssetsWithReferenced> {
OutputAssetsWithReferenced::from_assets(*self.chunks)
}
}
#[turbo_tasks::value_impl]
impl OutputAsset for EcmascriptBuildNodeChunkList {
#[turbo_tasks::function]
fn path(&self) -> Vc<FileSystemPath> {
// Use the explicit path from the caller. The path deliberately does not
// encode the covered chunks' idents, so it stays stable when a chunk is
// added/removed/renamed (which is what insulates the aggregate HMR
// version from `TotalUpdate`s).
self.path.clone().cell()
}
}
#[turbo_tasks::value_impl]
impl Asset for EcmascriptBuildNodeChunkList {
#[turbo_tasks::function]
fn content(self: Vc<Self>) -> Vc<AssetContent> {
// Never loaded at runtime, but this can't `bail!`: as a real
// `OutputAsset` in `server_assets` it flows through `all_asset_paths`,
// which hashes every asset's content to build the server paths list.
// Erroring here fails the whole endpoint write. Emit an empty stub
// instead.
AssetContent::file(FileContent::Content(File::from(rcstr!(""))).cell())
}
#[turbo_tasks::function]
fn versioned_content(self: Vc<Self>) -> Vc<Box<dyn VersionedContent>> {
Vc::upcast(self.own_content())
}
}
use anyhow::{Result, bail};
use turbo_tasks::{FxIndexMap, ResolvedVc, TryJoinIterExt, Vc};
use turbo_tasks_fs::FileSystemPath;
use turbopack_core::{
asset::{Asset, AssetContent},
chunk::ChunkingContext,
output::{
ExpandOutputAssetsInput, OutputAsset, OutputAssets, OutputAssetsReferences,
expand_output_assets,
},
version::{Update, Version, VersionedContent},
};
use turbopack_ecmascript::chunk_list::{
update::update_chunk_list,
version::{ChunkListVersion, compute_chunk_list_version},
};
use crate::NodeJsChunkingContext;
/// Maps each chunk to its `output_root`-relative path and versioned content.
async fn collect_chunks_contents(
output_root: &FileSystemPath,
chunks: impl Iterator<Item = ResolvedVc<Box<dyn OutputAsset>>>,
) -> Result<FxIndexMap<String, ResolvedVc<Box<dyn VersionedContent>>>> {
chunks
.map(async |chunk| {
let chunk_path = chunk.path().await?;
let Some(path) = output_root.get_path_to(&chunk_path) else {
bail!("chunk {chunk_path} is not inside the output root {output_root}");
};
Ok((
path.to_string(),
chunk.versioned_content().to_resolved().await?,
))
})
.try_join()
.await
.map(FxIndexMap::from_iter)
}
/// Never emitted as an asset — the entry chunk already inlines the `R.c(...)`
/// calls for its shared chunks. This exists purely to give the entry a
/// *stable* [`VersionedContent`] identity keyed off a chunk-list ident, so
/// adding/removing/renaming a shared chunk doesn't force a `TotalUpdate`.
///
/// Tracks both synchronous chunks and chunks reachable via async-loader
/// references (dynamic `import()`), so an edit inside a lazy-loaded module
/// still rides the merged `ChunkListUpdate` instead of being missed.
#[turbo_tasks::value]
pub struct EcmascriptBuildNodeChunkListContent {
#[bincode(with = "turbo_bincode::indexmap")]
pub(super) chunks_contents: FxIndexMap<String, ResolvedVc<Box<dyn VersionedContent>>>,
}
#[turbo_tasks::value_impl]
impl EcmascriptBuildNodeChunkListContent {
#[turbo_tasks::function]
pub async fn new(
chunking_context: ResolvedVc<NodeJsChunkingContext>,
chunks: ResolvedVc<OutputAssets>,
references: ResolvedVc<OutputAssetsReferences>,
) -> Result<Vc<Self>> {
let output_root = chunking_context.output_root().owned().await?;
// Expand async-loader references transitively to reach dynamically
// imported chunks. `inner=false`: only follow Reference edges (async
// loaders), not Asset-adjacent files like source maps that aren't part
// of the module graph and can't be hot-reloaded.
let async_chunks = expand_output_assets(
references
.await?
.iter()
.copied()
.map(ExpandOutputAssetsInput::Reference),
false,
)
.await?;
let chunks_contents = collect_chunks_contents(
&output_root,
chunks.await?.iter().copied().chain(async_chunks),
)
.await?;
Ok(EcmascriptBuildNodeChunkListContent { chunks_contents }.cell())
}
/// Builds a chunk list content directly from a fixed set of `chunks`,
/// without expanding async-loader references. Used by
/// [`super::chunk_list::EcmascriptBuildNodeChunkList`] to track chunks
/// (e.g. client-component SSR chunks) that are already fully enumerated by
/// the caller.
#[turbo_tasks::function]
pub async fn new_from_chunks(
chunking_context: ResolvedVc<NodeJsChunkingContext>,
chunks: Vc<OutputAssets>,
) -> Result<Vc<Self>> {
let output_root = chunking_context.output_root().owned().await?;
let chunks_contents =
collect_chunks_contents(&output_root, chunks.await?.iter().copied()).await?;
Ok(EcmascriptBuildNodeChunkListContent { chunks_contents }.cell())
}
#[turbo_tasks::function]
pub async fn version(&self) -> Result<Vc<ChunkListVersion>> {
compute_chunk_list_version(&self.chunks_contents).await
}
}
#[turbo_tasks::value_impl]
impl VersionedContent for EcmascriptBuildNodeChunkListContent {
#[turbo_tasks::function]
fn content(self: Vc<Self>) -> Result<Vc<AssetContent>> {
bail!("EcmascriptBuildNodeChunkListContent does not have content")
}
#[turbo_tasks::function]
fn version(self: Vc<Self>) -> Vc<Box<dyn Version>> {
Vc::upcast(self.version())
}
#[turbo_tasks::function]
async fn update(
self: ResolvedVc<Self>,
from_version: ResolvedVc<Box<dyn Version>>,
) -> Result<Vc<Update>> {
let this = self.await?;
let to_version = self.version();
update_chunk_list(&this.chunks_contents, to_version, from_version).await
}
}
pub(crate) mod chunk; pub(crate) mod chunk;
pub(crate) mod chunk_list;
pub mod chunk_list_content;
pub(crate) mod runtime; pub(crate) mod runtime;
use anyhow::{Result, bail};
use turbo_tasks::{ResolvedVc, TryJoinIterExt, Vc};
use turbopack_core::version::{Update, Version, VersionedContent};
use super::{
update::update_ecmascript_merged_chunk, version::EcmascriptBuildNodeMergedChunkVersion,
};
use crate::ecmascript::node::content::EcmascriptBuildNodeChunkContent;
/// Composite [`VersionedContent`] that is the result of merging multiple
/// node ecmascript chunk contents together through the
/// [`EcmascriptBuildNodeChunkContentMerger`].
#[turbo_tasks::value(serialization = "skip", shared)]
pub(crate) struct EcmascriptBuildNodeMergedChunkContent {
pub(super) contents: Vec<ResolvedVc<EcmascriptBuildNodeChunkContent>>,
}
#[turbo_tasks::value_impl]
impl EcmascriptBuildNodeMergedChunkContent {
#[turbo_tasks::function]
pub async fn version(&self) -> Result<Vc<EcmascriptBuildNodeMergedChunkVersion>> {
Ok(EcmascriptBuildNodeMergedChunkVersion {
versions: self
.contents
.iter()
.map(|content| async move { content.own_version().await })
.try_join()
.await?,
}
.cell())
}
}
#[turbo_tasks::value_impl]
impl VersionedContent for EcmascriptBuildNodeMergedChunkContent {
#[turbo_tasks::function]
fn content(self: Vc<Self>) -> Result<Vc<turbopack_core::asset::AssetContent>> {
bail!("EcmascriptBuildNodeMergedChunkContent does not have content")
}
#[turbo_tasks::function]
fn version(self: Vc<Self>) -> Vc<Box<dyn Version>> {
Vc::upcast(self.version())
}
#[turbo_tasks::function]
async fn update(
self: Vc<Self>,
from_version: ResolvedVc<Box<dyn Version>>,
) -> Result<Vc<Update>> {
Ok(update_ecmascript_merged_chunk(self, from_version)
.await?
.cell())
}
}
use anyhow::{Result, bail};
use turbo_tasks::{ResolvedVc, TryJoinIterExt, Vc};
use turbopack_core::version::{VersionedContent, VersionedContentMerger, VersionedContents};
use crate::ecmascript::node::{
content::EcmascriptBuildNodeChunkContent,
merged::content::EcmascriptBuildNodeMergedChunkContent,
};
/// Merges multiple [`EcmascriptBuildNodeChunkContent`] into a single
/// [`EcmascriptBuildNodeMergedChunkContent`]. This allows the chunk list to
/// produce a single `EcmascriptMergedUpdate` for multiple chunks updating at
/// the same time.
#[turbo_tasks::value]
pub(crate) struct EcmascriptBuildNodeChunkContentMerger;
#[turbo_tasks::value_impl]
impl EcmascriptBuildNodeChunkContentMerger {
#[turbo_tasks::function]
pub fn new() -> Vc<Self> {
Self::cell(EcmascriptBuildNodeChunkContentMerger)
}
}
#[turbo_tasks::value_impl]
impl VersionedContentMerger for EcmascriptBuildNodeChunkContentMerger {
#[turbo_tasks::function]
async fn merge(
&self,
contents: Vc<VersionedContents>,
) -> Result<Vc<Box<dyn VersionedContent>>> {
let contents = contents
.await?
.iter()
.map(|content| async move {
if let Some(content) =
ResolvedVc::try_downcast_type::<EcmascriptBuildNodeChunkContent>(*content)
{
Ok(content)
} else {
bail!("expected Vc<EcmascriptBuildNodeChunkContent>")
}
})
.try_join()
.await?;
Ok(Vc::upcast(
EcmascriptBuildNodeMergedChunkContent { contents }.cell(),
))
}
}
pub(crate) mod content;
pub(crate) mod merger;
pub(crate) mod update;
pub(crate) mod version;
use std::sync::Arc;
use anyhow::Result;
use turbo_tasks::{FxIndexMap, ReadRef, ResolvedVc, TryJoinIterExt, Vc};
use turbopack_core::{
chunk::{ChunkingContext, ModuleId},
output::OutputAsset,
version::{PartialUpdate, TotalUpdate, Update, Version},
};
use turbopack_ecmascript::chunk_list::merged_update::{
EcmascriptMergedChunkAdded, EcmascriptMergedChunkDeleted, EcmascriptMergedChunkPartial,
EcmascriptMergedChunkUpdate, EcmascriptMergedUpdate, EcmascriptModuleEntry,
};
use crate::ecmascript::node::{
merged::{
content::EcmascriptBuildNodeMergedChunkContent,
version::EcmascriptBuildNodeMergedChunkVersion,
},
update::{NodeChunkUpdate, update_ecmascript_node_chunk_content},
version::EcmascriptBuildNodeChunkVersion,
};
struct MergedModuleMap {
versions: Vec<ReadRef<EcmascriptBuildNodeChunkVersion>>,
}
impl MergedModuleMap {
fn new(versions: Vec<ReadRef<EcmascriptBuildNodeChunkVersion>>) -> Self {
Self { versions }
}
fn get(&self, id: &ModuleId) -> Option<u128> {
for version in &self.versions {
if let Some(hash) = version.entries_hashes.get(id) {
return Some(*hash);
}
}
None
}
}
pub(crate) async fn update_ecmascript_merged_chunk(
content: Vc<EcmascriptBuildNodeMergedChunkContent>,
from_version: ResolvedVc<Box<dyn Version>>,
) -> Result<Update> {
let to_merged_version = content.version();
let from_merged_version = if let Some(from) =
ResolvedVc::try_downcast_type::<EcmascriptBuildNodeMergedChunkVersion>(from_version)
{
from
} else {
return Ok(Update::Total(TotalUpdate {
to: Vc::upcast::<Box<dyn Version>>(to_merged_version)
.into_trait_ref()
.await?,
}));
};
let to = to_merged_version.await?;
let from = from_merged_version.await?;
if from.ptr_eq(&to) {
return Ok(Update::None);
}
let mut from_versions_by_chunk_path: FxIndexMap<_, _> = from
.versions
.iter()
.map(|version| (&*version.chunk_path, version))
.collect();
let merged_module_map = MergedModuleMap::new(from.versions.to_vec());
let content = content.await?;
let to_contents = content
.contents
.iter()
.map(|content| async move {
let entries = content.entries().await?;
let content_ref = content.await?;
let output_root = content_ref.chunking_context.output_root().await?;
let path = content_ref.chunk.path().await?;
Ok((*content, entries, output_root, path))
})
.try_join()
.await?;
let mut merged_update = EcmascriptMergedUpdate::default();
for (content, entries, output_root, path) in &to_contents {
let Some(chunk_path) = output_root.get_path_to(path) else {
continue;
};
let chunk_update = if let Some(from_version) =
from_versions_by_chunk_path.swap_remove(chunk_path)
{
// Reuse the single-chunk diff so the merged path stays in sync with
// the standalone chunk update path.
let to_version = content.own_version().await?;
match update_ecmascript_node_chunk_content(**content, &to_version, from_version).await?
{
NodeChunkUpdate::None => continue,
NodeChunkUpdate::Partial {
added,
modified,
deleted,
} => {
let mut partial = EcmascriptMergedChunkPartial::default();
for (module_id, (module_hash, module_code)) in added {
partial.added.insert(module_id.clone());
// Only ship the code if no other chunk in the group
// already provides this module at the same hash.
if merged_module_map.get(&module_id) != Some(module_hash) {
let entry = EcmascriptModuleEntry::from_code(
&module_id,
module_code,
chunk_path,
)
.await?;
merged_update.entries.insert(module_id, entry);
}
}
partial.deleted.extend(deleted.into_keys());
for (module_id, module_code) in modified {
let entry =
EcmascriptModuleEntry::from_code(&module_id, module_code, chunk_path)
.await?;
merged_update.entries.insert(module_id, entry);
}
EcmascriptMergedChunkUpdate::Partial(partial)
}
}
} else {
let mut added = EcmascriptMergedChunkAdded::default();
for (id, entry) in entries {
let hash = *entry.hash.await?;
added.modules.insert(id.clone());
if merged_module_map.get(id) != Some(hash) {
let entry =
EcmascriptModuleEntry::from_code(id, *entry.code, chunk_path).await?;
merged_update.entries.insert(id.clone(), entry);
}
}
EcmascriptMergedChunkUpdate::Added(added)
};
merged_update.chunks.insert(chunk_path, chunk_update);
}
for (chunk_path, chunk_version) in from_versions_by_chunk_path {
let hashes = &chunk_version.entries_hashes;
merged_update.chunks.insert(
chunk_path,
EcmascriptMergedChunkUpdate::Deleted(EcmascriptMergedChunkDeleted {
modules: hashes.keys().cloned().collect(),
}),
);
}
let update = if merged_update.is_empty() {
Update::None
} else {
Update::Partial(PartialUpdate {
to: Vc::upcast::<Box<dyn Version>>(to_merged_version)
.into_trait_ref()
.await?,
instruction: Arc::new(serde_json::to_value(&merged_update)?),
})
};
Ok(update)
}
use anyhow::Result;
use turbo_rcstr::RcStr;
use turbo_tasks::{ReadRef, TryJoinIterExt, Vc};
use turbo_tasks_hash::{Xxh3Hash64Hasher, encode_base64};
use turbopack_core::version::Version;
use crate::ecmascript::node::version::EcmascriptBuildNodeChunkVersion;
/// The version of a [`super::content::EcmascriptBuildNodeMergedChunkContent`].
#[turbo_tasks::value(serialization = "skip", shared)]
pub(crate) struct EcmascriptBuildNodeMergedChunkVersion {
pub(super) versions: Vec<ReadRef<EcmascriptBuildNodeChunkVersion>>,
}
#[turbo_tasks::value_impl]
impl Version for EcmascriptBuildNodeMergedChunkVersion {
#[turbo_tasks::function]
async fn id(&self) -> Result<Vc<RcStr>> {
let mut hasher = Xxh3Hash64Hasher::new();
hasher.write_value(self.versions.len());
let sorted_ids = {
let mut sorted_ids = self
.versions
.iter()
.map(|version| ReadRef::cell(version.clone()).id())
.try_join()
.await?;
sorted_ids.sort();
sorted_ids
};
for id in sorted_ids {
hasher.write_value(id);
}
let hash = hasher.finish();
let hash = encode_base64(hash);
Ok(Vc::cell(hash.into()))
}
}
pub(crate) mod chunk; pub(crate) mod chunk;
pub(crate) mod content; pub(crate) mod content;
pub(crate) mod entry; pub mod entry;
pub(crate) mod merged;
pub(crate) mod update; pub(crate) mod update;
pub(crate) mod version; pub(crate) mod version;
use std::sync::Arc; use std::sync::Arc;
use anyhow::Result; use anyhow::Result;
use serde::Serialize; use turbo_tasks::{FxIndexMap, ReadRef, ResolvedVc, Vc};
use turbo_tasks::{FxIndexMap, FxIndexSet, ReadRef, ResolvedVc, Vc};
use turbo_tasks_fs::rope::Rope;
use turbopack_core::{ use turbopack_core::{
chunk::ModuleId, chunk::ModuleId,
code_builder::Code, code_builder::Code,
source_map::GenerateSourceMap,
version::{PartialUpdate, TotalUpdate, Update, Version}, version::{PartialUpdate, TotalUpdate, Update, Version},
}; };
use turbopack_ecmascript::chunk_list::merged_update::{
EcmascriptMergedChunkPartial, EcmascriptMergedChunkUpdate, EcmascriptMergedUpdate,
EcmascriptModuleEntry,
};
use super::{content::EcmascriptBuildNodeChunkContent, version::EcmascriptBuildNodeChunkVersion}; use super::{content::EcmascriptBuildNodeChunkContent, version::EcmascriptBuildNodeChunkVersion};
#[derive(Serialize, Default)]
#[serde(
tag = "type",
rename = "EcmascriptMergedUpdate",
rename_all = "camelCase"
)]
struct EcmascriptMergedUpdate<'a> {
/// A map from module id to latest module entry.
#[serde(skip_serializing_if = "FxIndexMap::is_empty")]
entries: FxIndexMap<ModuleId, EcmascriptModuleEntry>,
/// A map from chunk path to the chunk update.
#[serde(skip_serializing_if = "FxIndexMap::is_empty")]
chunks: FxIndexMap<&'a str, EcmascriptMergedChunkUpdate>,
}
impl EcmascriptMergedUpdate<'_> {
fn is_empty(&self) -> bool {
self.entries.is_empty() && self.chunks.is_empty()
}
}
#[derive(Serialize)]
#[serde(tag = "type", rename_all = "camelCase")]
enum EcmascriptMergedChunkUpdate {
Partial(EcmascriptMergedChunkPartial),
}
#[derive(Serialize, Default)]
#[serde(rename_all = "camelCase")]
struct EcmascriptMergedChunkPartial {
#[serde(skip_serializing_if = "FxIndexSet::is_empty")]
added: FxIndexSet<ModuleId>,
#[serde(skip_serializing_if = "FxIndexSet::is_empty")]
deleted: FxIndexSet<ModuleId>,
}
#[derive(Serialize)]
struct EcmascriptModuleEntry {
#[serde(with = "turbo_tasks_fs::rope::ser_as_string")]
code: Rope,
url: String,
#[serde(with = "turbo_tasks_fs::rope::ser_option_as_string")]
map: Option<Rope>,
}
impl EcmascriptModuleEntry {
async fn from_code(id: &ModuleId, code: Vc<Code>, chunk_path: &str) -> Result<Self> {
let map = &*code.generate_source_map().await?;
let map = map.as_content().map(|f| f.content().clone());
/// serde_qs can't serialize a lone enum when it's [serde::untagged].
#[derive(Serialize)]
struct Id<'a> {
id: &'a ModuleId,
}
let id = serde_qs::to_string(&Id { id }).unwrap();
Ok(EcmascriptModuleEntry {
// Cloning a rope is cheap.
code: code.await?.source_code().clone(),
url: format!("{}?{}", chunk_path, id),
map,
})
}
}
pub(super) async fn update_node_chunk( pub(super) async fn update_node_chunk(
content: Vc<EcmascriptBuildNodeChunkContent>, content: Vc<EcmascriptBuildNodeChunkContent>,
from_version: ResolvedVc<Box<dyn Version>>, from_version: ResolvedVc<Box<dyn Version>>,
...@@ -121,7 +56,7 @@ pub(super) async fn update_node_chunk( ...@@ -121,7 +56,7 @@ pub(super) async fn update_node_chunk(
} => { } => {
let mut partial = EcmascriptMergedChunkPartial::default(); let mut partial = EcmascriptMergedChunkPartial::default();
for (module_id, module_code) in added { for (module_id, (_hash, module_code)) in added {
partial.added.insert(module_id.clone()); partial.added.insert(module_id.clone());
let entry = let entry =
...@@ -161,16 +96,18 @@ pub(super) async fn update_node_chunk( ...@@ -161,16 +96,18 @@ pub(super) async fn update_node_chunk(
Ok(update) Ok(update)
} }
enum NodeChunkUpdate { pub(crate) enum NodeChunkUpdate {
None, None,
Partial { Partial {
added: FxIndexMap<ModuleId, Vc<Code>>, /// Added modules, keyed by id, with their content hash (used by the
/// merged-chunk path to dedup code emission across chunks) and code.
added: FxIndexMap<ModuleId, (u128, Vc<Code>)>,
modified: FxIndexMap<ModuleId, Vc<Code>>, modified: FxIndexMap<ModuleId, Vc<Code>>,
deleted: FxIndexMap<ModuleId, u128>, deleted: FxIndexMap<ModuleId, u128>,
}, },
} }
async fn update_ecmascript_node_chunk_content( pub(super) async fn update_ecmascript_node_chunk_content(
content: Vc<EcmascriptBuildNodeChunkContent>, content: Vc<EcmascriptBuildNodeChunkContent>,
to: &ReadRef<EcmascriptBuildNodeChunkVersion>, to: &ReadRef<EcmascriptBuildNodeChunkVersion>,
from: &ReadRef<EcmascriptBuildNodeChunkVersion>, from: &ReadRef<EcmascriptBuildNodeChunkVersion>,
...@@ -205,14 +142,14 @@ async fn update_ecmascript_node_chunk_content( ...@@ -205,14 +142,14 @@ async fn update_ecmascript_node_chunk_content(
} }
// Check for added modules // Check for added modules
for (id, _hash) in &to.entries_hashes { for (id, hash) in &to.entries_hashes {
if !from.entries_hashes.contains_key(id) { if !from.entries_hashes.contains_key(id) {
let entries = match &entries_ref { let entries = match &entries_ref {
Some(entries) => entries, Some(entries) => entries,
None => entries_ref.insert(content.entries().await?), None => entries_ref.insert(content.entries().await?),
}; };
if let Some(entry) = entries.get(id) { if let Some(entry) = entries.get(id) {
added.insert(id.clone(), *entry.code); added.insert(id.clone(), (*hash, *entry.code));
} }
} }
} }
......
...@@ -2,6 +2,6 @@ ...@@ -2,6 +2,6 @@
#![feature(arbitrary_self_types_pointers)] #![feature(arbitrary_self_types_pointers)]
pub(crate) mod chunking_context; pub(crate) mod chunking_context;
pub(crate) mod ecmascript; pub mod ecmascript;
pub use chunking_context::{NodeJsChunkingContext, NodeJsChunkingContextBuilder}; pub use chunking_context::{NodeJsChunkingContext, NodeJsChunkingContextBuilder};
...@@ -1671,34 +1671,63 @@ function initializeServerHmr(moduleFactories, devModuleCache) { ...@@ -1671,34 +1671,63 @@ function initializeServerHmr(moduleFactories, devModuleCache) {
return; return;
} }
const instruction = msg.instruction; const instruction = msg.instruction;
if (instruction.type !== 'EcmascriptMergedUpdate') {
return;
}
try { try {
const { entries = {}, chunks = {} } = instruction; if (instruction.type === 'ChunkListUpdate') {
const evalModuleEntry = (entry)=>{ // All node ecmascript chunks are mergeable, so a `total`/`partial` here
// eslint-disable-next-line no-eval // means a non-mergeable asset changed in an unsupported way. Escalate
return (0, eval)(entry.map ? inlineSourcemaps(entry) : entry.code); // to a full clear() rather than leave stale factories in memory.
}; for (const [chunkPath, chunkUpdate] of Object.entries(instruction.chunks ?? {})){
const { added, modified } = computeChangedModules(entries, chunks, undefined // no chunkModulesMap for Node.js if (chunkUpdate.type === 'total' || chunkUpdate.type === 'partial') {
); throw new Error(`unsupported '${chunkUpdate.type}' update for chunk ${chunkPath}`);
// Use shared HMR update implementation }
applyEcmascriptMergedUpdateShared({ }
added, if (instruction.merged) {
modified, for (const merged of instruction.merged){
disposedModules: [], applyEcmascriptMergedUpdate(merged, moduleFactories, devModuleCache);
evalModuleEntry, }
instantiateModule, }
applyModuleFactoryName: ()=>{}, return;
moduleFactories, }
devModuleCache, if (instruction.type === 'EcmascriptMergedUpdate') {
autoAcceptRootModules: true applyEcmascriptMergedUpdate(instruction, moduleFactories, devModuleCache);
}); return;
}
} catch (e) { } catch (e) {
console.error('[Server HMR] Update failed, full reload needed:', e); console.error('[Server HMR] Update failed, full reload needed:', e);
throw e; throw e;
} }
} }
function applyEcmascriptMergedUpdate(instruction, moduleFactories, devModuleCache) {
const { entries = {}, chunks = {} } = instruction;
const evalModuleEntry = (entry)=>{
const code = entry.map ? inlineSourcemaps(entry) : entry.code;
// eslint-disable-next-line no-eval
return (0, eval)(`(require) => ${code}`)(require);
};
const { added, modified } = computeChangedModules(entries, chunks, undefined // no chunkModulesMap for Node.js
);
// Modules that appear in an "added" chunk but already exist in the cache
// were moved to a renamed chunk. Treat them as modified so the dependency
// walk runs and they get re-instantiated with the new factory.
for (const [moduleId, entry] of added){
if (entry != null && devModuleCache[moduleId] != null) {
added.delete(moduleId);
modified.set(moduleId, entry);
}
}
// Use shared HMR update implementation
applyEcmascriptMergedUpdateShared({
added,
modified,
disposedModules: [],
evalModuleEntry,
instantiateModule,
applyModuleFactoryName: ()=>{},
moduleFactories,
devModuleCache,
autoAcceptRootModules: true
});
}
/// <reference path="../../shared/runtime/dev-protocol.d.ts" /> /// <reference path="../../shared/runtime/dev-protocol.d.ts" />
/// <reference path="./hmr-client.ts" /> /// <reference path="./hmr-client.ts" />
/** /**
...@@ -1731,9 +1760,16 @@ if (handlers.size === 0) { ...@@ -1731,9 +1760,16 @@ if (handlers.size === 0) {
// First registration in this generation: install the routing dispatcher. // First registration in this generation: install the routing dispatcher.
globalThis.__turbopack_server_hmr_apply__ = (update)=>{ globalThis.__turbopack_server_hmr_apply__ = (update)=>{
const registry = globalThis.__turbopack_server_hmr_handlers__ ?? new Map(); const registry = globalThis.__turbopack_server_hmr_handlers__ ?? new Map();
const updateChunkPaths = Object.keys(update.instruction?.chunks ?? {}); // Chunk paths can appear either directly on the instruction (single-chunk
// updates) or nested inside `merged` entries (chunks covered by a
// merger). Collect both so routing isn't skipped just because a mergeable
// chunk's update only reports its paths inside `merged`.
const updateChunkPaths = new Set([
...Object.keys(update.instruction?.chunks ?? {}),
...(update.instruction?.merged ?? []).flatMap((merged)=>Object.keys(merged.chunks ?? {}))
]);
const toCall = []; const toCall = [];
if (updateChunkPaths.length === 0) { if (updateChunkPaths.size === 0) {
for (const entry of registry.values())toCall.push(entry); for (const entry of registry.values())toCall.push(entry);
} else { } else {
const seen = new Set(); const seen = new Set();
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment