Unverified Commit 21182268 authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

[turbopack] Reduce the size of module graph datastructures (#79706)

## Refactor module graph to reduce memory usage

### What?
This PR refactors the module graph implementation to reduce memory usage by:
1. Changing `graph_idx` from `usize` to `u32` to reduce memory footprint
2. Removing unused fields from `SingleModuleGraphModuleNode` (layer and issues)
3. Moving layer resolution to the point of use in dynamic imports and server actions

This is a small progression in runtime:

![image.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/AwJ29EfoPcPdLSwCZxAz/d669023b-9ac4-44d5-ac9f-54e42b71d341.png)

### Why?
These changes optimize memory usage in the module graph, which is particularly important for large applications. By removing unused fields and using more compact types, we can reduce the memory footprint of the graph.

Notably, SingleModuleGraphNode is now 24 bytes instead of  48.  To make it smaller still we would need to reduce the size of visited moduel, perhaps by storing a reference to the graph that did the original visit?  This would be tricky from a lifetimes perspective but much more optimal.
parent 6e80e56b
...@@ -125,12 +125,12 @@ pub async fn map_next_dynamic(graph: Vc<SingleModuleGraph>) -> Result<Vc<Dynamic ...@@ -125,12 +125,12 @@ pub async fn map_next_dynamic(graph: Vc<SingleModuleGraph>) -> Result<Vc<Dynamic
.await? .await?
.iter_nodes() .iter_nodes()
.map(|node| async move { .map(|node| async move {
let SingleModuleGraphModuleNode { module, layer, .. } = node; let SingleModuleGraphModuleNode { module } = node;
let layer = match module.ident().await?.layer {
if layer Some(l) => Some(l.await?),
.as_ref() None => None,
.is_some_and(|layer| &**layer == "app-client" || &**layer == "client") };
{ if layer.is_some_and(|layer| *layer == "app-client" || *layer == "client") {
if let Some(dynamic_entry_module) = if let Some(dynamic_entry_module) =
ResolvedVc::try_downcast_type::<NextDynamicEntryModule>(*module) ResolvedVc::try_downcast_type::<NextDynamicEntryModule>(*module)
{ {
......
...@@ -435,14 +435,17 @@ pub async fn map_server_actions(graph: Vc<SingleModuleGraph>) -> Result<Vc<AllMo ...@@ -435,14 +435,17 @@ pub async fn map_server_actions(graph: Vc<SingleModuleGraph>) -> Result<Vc<AllMo
.iter_nodes() .iter_nodes()
.map(|node| { .map(|node| {
async move { async move {
let SingleModuleGraphModuleNode { module, layer, .. } = node; let SingleModuleGraphModuleNode { module } = node;
let layer = match module.ident().await?.layer {
Some(l) => Some(l.await?),
None => None,
};
// TODO: compare module contexts instead? // TODO: compare module contexts instead?
let layer = match &layer { let layer = match layer {
Some(layer) if &**layer == "app-rsc" || &**layer == "app-edge-rsc" => { Some(layer) if *layer == "app-rsc" || *layer == "app-edge-rsc" => {
ActionLayer::Rsc ActionLayer::Rsc
} }
Some(layer) if &**layer == "app-client" => ActionLayer::ActionBrowser, Some(layer) if *layer == "app-client" => ActionLayer::ActionBrowser,
// TODO really ignore SSR? // TODO really ignore SSR?
_ => return Ok(None), _ => return Ok(None),
}; };
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment