Unverified Commit 1b77dba6 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Turbopack: Fix unsound IntoIterator for ReadRef<T> (#94122)

### What?

Replaces the unsound by-value `IntoIterator` impl for `ReadRef<T>` in
`turbo-tasks` with a sound clone-free variant, and adapts the callers
across `turbopack-*` and `next-api`.

### Why?

The previous impl used `transmute_copy` to fabricate `&'static`-typed
items so it could expose them through the standard `Iterator` trait.
Those references were only really valid as long as the `ReadRef` inside
the iterator stayed alive — but `Iterator::Item` is a fixed associated
type, so once the items were stashed in futures, `Vec`s, or `serde_json`
map keys, the lifetime was completely unenforced.

This produced a latent use-after-free whenever something else
(turbo-tasks cell eviction, an intermediate `Drop`, etc.) released the
underlying storage between the iteration site and the next dereference.
The observed symptom was a panic in `RcStr::as_str` during JSON
serialization of `AssetHashesManifestAsset`'s manifest:

```
thread 'tokio-rt-worker' panicked at turbopack/crates/turbo-rcstr/src/lib.rs:132:52:
range end index 13 out of range for slice of length 7
```

The byte read at the inline-length position was junk left over from
freed/reused memory — `len = 13` is unreachable for any
legitimately-constructed inline `RcStr` (max inline length is 7 on
64-bit). The bug site was
`crates/next-api/src/project_asset_hashes_manifest.rs`, which consumed
an `OutputAssetsWithPaths` `ReadRef`, kept `&RcStr` references in
`asset_paths` past the `try_join` that dropped the iterator, then
serialized them.

### How?

**`turbopack/crates/turbo-tasks/src/read_ref.rs`** — new by-value impl:

```rust
pub struct ReadRefIter<T, I, J>
where
    T: VcValueType,
    I: Copy + 'static,
    J: Iterator<Item = &'static I>,
{
    iter: J,
    _read_ref: ReadRef<T>,
}

impl<T, I, J> Iterator for ReadRefIter<T, I, J> /* … */ {
    type Item = I;
    fn next(&mut self) -> Option<I> { self.iter.next().copied() }
}

impl<T, I, J> IntoIterator for ReadRef<T>
where
    T: VcValueType,
    I: Copy + 'static,
    J: Iterator<Item = &'static I> + 'static,
    &'static VcReadTarget<T>: IntoIterator<Item = &'static I, IntoIter = J>,
{
    type Item = I;
    type IntoIter = ReadRefIter<T, I, J>;
    fn into_iter(self) -> Self::IntoIter {
        let r: &VcReadTarget<T> = &self;
        // SAFETY: the fabricated `&'static` reference is only stored inside
        // `iter`, which lives inside the returned `ReadRefIter` alongside
        // the `ReadRef` that owns the data. `next()` only ever yields
        // `Copy`-ed-out values — no reference (with the fake `'static`
        // lifetime or otherwise) ever leaves the iterator. Struct-field drop
        // order (`iter` then `_read_ref`) drops the borrow before the
        // backing storage.
        let r = unsafe { std::mem::transmute::<&VcReadTarget<T>, &'static VcReadTarget<T>>(r) };
        ReadRefIter { iter: r.into_iter(), _read_ref: self }
    }
}
```

Key properties:
- **No cloning.** Setup is one borrow + `transmute`; `next()` is
`Option::copied()` (bitwise copy via the `Copy` bound), not
`Clone::clone`. Nothing in the iterator clones the backing collection or
its elements.
- **Contained `unsafe`.** The fake `'static` reference never leaves
`ReadRefIter`. `Iterator::next` yields `I` by value, so the lifetime
never escapes into futures, `Vec`s, or other persistence outside the
iterator.
- **Drop order safe.** Struct fields drop in declaration order: `iter`
(and any borrows it holds) drops before `_read_ref` (the backing `Arc`).
- **`Copy` bound.** The impl is restricted to element types that are
`Copy` — `ResolvedVc<_>`, integer ids, owned-tuple-of-`Copy`, etc. For
non-`Copy` element types (`RcStr`, `FileSystemPath`, `PatternMatch`,
`(String, _)`, `(ModuleId, ReadRef<_>)`, …) callers iterate by reference
via the existing `IntoIterator for &'a ReadRef<T>` impl (`for x in
&read_ref` or `read_ref.iter()`).

The original buggy site in `project_asset_hashes_manifest.rs` now uses
`output_assets.iter()` and keeps `&'a RcStr` references in the manifest
struct. The borrow checker now enforces the lifetime that used to be
faked via `transmute` — `output_assets` outlives the references because
nothing consumes it, and there are no clones at the call site either.

**Caller adjustments.** Touching the impl forced a sweep of all call
sites that were implicitly leaning on the unsound shape (yielding
`&'static`-typed items as a stand-in for owned items). The fixes fall
into a small number of categories:

- Drop redundant `.copied()` / `.cloned()` / `|&x| f(x)` patterns after
`into_iter()` (items are owned `Copy` values now, no need to
deref-and-copy).
- Switch non-`Copy` element iteration to `&read_ref` / `read_ref.iter()`
(e.g. `PatternMatches`, `CodeAndIds`, `UnresolvedUrlReferences`,
`GraphEntries`, `Vec<RcStr>`).
- Reshape `crates/next-api/src/paths.rs` helpers from `impl
IntoIterator<Item = &ResolvedVc<_>>` to `impl IntoIterator<Item =
ResolvedVc<_>>` — `ResolvedVc` is `Copy`, so by-value is the natural
shape and it composes directly with the new by-value
`ReadRef::into_iter`. Callers in `app.rs`, `pages.rs`, `middleware.rs`,
`instrumentation.rs`, `font.rs` updated to match (either passing the
`ReadRef`/`Vec` directly, or `.iter().copied()` for borrowed sources).
- A few small follow-ups: `for (key, EndpointGroup { primary, .. }) in
&entrypoint_groups` in `routes_hashes_manifest.rs` (with a borrowed `&'l
str` key in the manifest); `compute_async_module_info_single(graph,
result)` (no `*graph`, it's already `Copy`); `&(ty, batch)` → `(ty,
batch)` destructures in `chunking/mod.rs`.

### Testing

- `cac` clean across the workspace.
- `ca clippy --all-targets` clean.
- `ca test -p turbo-tasks-backend` — all unit + integration tests pass.
- `ca test -p turbopack-tests --tests` — execution snapshot suite (218
passed, 0 failed, 1 ignored) and snapshot suite (87 passed, 0 failed).

Closes NEXT-
Fixes #
parent c7a5a095
...@@ -376,7 +376,7 @@ pub async fn analyze_output_assets(output_assets: Vc<OutputAssets>) -> Result<Vc ...@@ -376,7 +376,7 @@ pub async fn analyze_output_assets(output_assets: Vc<OutputAssets>) -> Result<Vc
// Process the output assets and extract chunk parts. // Process the output assets and extract chunk parts.
// Also creates sources for the chunk parts. // Also creates sources for the chunk parts.
for &asset in output_assets.await? { for asset in output_assets.await? {
let filename = asset.path().to_string().owned().await?; let filename = asset.path().to_string().owned().await?;
if filename.ends_with(".map") || filename.ends_with(".nft.json") { if filename.ends_with(".map") || filename.ends_with(".nft.json") {
// Skip source maps. // Skip source maps.
...@@ -385,7 +385,7 @@ pub async fn analyze_output_assets(output_assets: Vc<OutputAssets>) -> Result<Vc ...@@ -385,7 +385,7 @@ pub async fn analyze_output_assets(output_assets: Vc<OutputAssets>) -> Result<Vc
let output_file_index = builder.add_output_file(AnalyzeOutputFile { filename }); let output_file_index = builder.add_output_file(AnalyzeOutputFile { filename });
let chunk_parts = split_output_asset_into_parts(*asset).await?; let chunk_parts = split_output_asset_into_parts(*asset).await?;
for chunk_part in chunk_parts { for chunk_part in &chunk_parts {
let decoded_source = urlencoding::decode(&chunk_part.source)?; let decoded_source = urlencoding::decode(&chunk_part.source)?;
let source = if let Some(stripped) = decoded_source.strip_prefix(&prefix) { let source = if let Some(stripped) = decoded_source.strip_prefix(&prefix) {
Cow::Borrowed(stripped) Cow::Borrowed(stripped)
...@@ -439,7 +439,7 @@ pub async fn analyze_module_graphs(module_graphs: Vc<ModuleGraphs>) -> Result<Vc ...@@ -439,7 +439,7 @@ pub async fn analyze_module_graphs(module_graphs: Vc<ModuleGraphs>) -> Result<Vc
let mut all_modules = FxIndexSet::default(); let mut all_modules = FxIndexSet::default();
let mut all_edges = FxIndexSet::default(); let mut all_edges = FxIndexSet::default();
let mut all_async_edges = FxIndexSet::default(); let mut all_async_edges = FxIndexSet::default();
for &module_graph in module_graphs.await? { for module_graph in module_graphs.await? {
let module_graph = module_graph.await?; let module_graph = module_graph.await?;
module_graph.traverse_edges_unordered(|parent, node| { module_graph.traverse_edges_unordered(|parent, node| {
if let Some((parent_node, reference)) = parent { if let Some((parent_node, reference)) = parent {
......
...@@ -902,7 +902,7 @@ impl AppProject { ...@@ -902,7 +902,7 @@ impl AppProject {
let client_shared_entries = client_shared_entries let client_shared_entries = client_shared_entries
.await? .await?
.into_iter() .into_iter()
.map(|m| ResolvedVc::upcast(*m)) .map(ResolvedVc::upcast)
.collect(); .collect();
// SEGMENT: client_shared_entries and server utils shared by the layout segments // SEGMENT: client_shared_entries and server utils shared by the layout segments
...@@ -1471,7 +1471,7 @@ impl AppEndpoint { ...@@ -1471,7 +1471,7 @@ impl AppEndpoint {
) )
.to_resolved() .to_resolved()
.await?; .await?;
server_assets.extend(app_entry_chunks.all_assets().await?.into_iter().copied()); server_assets.extend(app_entry_chunks.all_assets().await?);
let app_entry_chunk_group_ref = app_entry_chunks.await?; let app_entry_chunk_group_ref = app_entry_chunks.await?;
let app_entry_chunks = app_entry_chunk_group_ref.assets; let app_entry_chunks = app_entry_chunk_group_ref.assets;
let app_entry_chunks_ref = app_entry_chunks.await?; let app_entry_chunks_ref = app_entry_chunks.await?;
...@@ -1553,20 +1553,26 @@ impl AppEndpoint { ...@@ -1553,20 +1553,26 @@ impl AppEndpoint {
let node_root_value = node_root.clone(); let node_root_value = node_root.clone();
file_paths_from_root file_paths_from_root.extend(
.extend(get_js_paths_from_root(&node_root_value, &middleware_assets).await?); get_js_paths_from_root(&node_root_value, middleware_assets.iter().copied())
file_paths_from_root .await?,
.extend(get_js_paths_from_root(&node_root_value, &app_entry_chunks_ref).await?); );
file_paths_from_root.extend(
get_js_paths_from_root(&node_root_value, app_entry_chunks_ref.iter().copied())
.await?,
);
let all_output_assets = all_assets_from_entries(*app_entry_chunks).await?; let all_output_assets = all_assets_from_entries(*app_entry_chunks).await?;
wasm_paths_from_root wasm_paths_from_root
.extend(get_wasm_paths_from_root(&node_root_value, &middleware_assets).await?); .extend(get_wasm_paths_from_root(&node_root_value, middleware_assets).await?);
wasm_paths_from_root wasm_paths_from_root.extend(
.extend(get_wasm_paths_from_root(&node_root_value, &all_output_assets).await?); get_wasm_paths_from_root(&node_root_value, all_output_assets.iter().copied())
.await?,
);
let all_assets = let all_assets =
get_asset_paths_from_root(&node_root_value, &all_output_assets).await?; get_asset_paths_from_root(&node_root_value, all_output_assets).await?;
let entry_file = rcstr!("app-edge-has-no-entrypoint"); let entry_file = rcstr!("app-edge-has-no-entrypoint");
...@@ -1595,7 +1601,7 @@ impl AppEndpoint { ...@@ -1595,7 +1601,7 @@ impl AppEndpoint {
server_assets.extend(loadable_manifest_output.iter().copied()); server_assets.extend(loadable_manifest_output.iter().copied());
file_paths_from_root.extend( file_paths_from_root.extend(
get_js_paths_from_root(&node_root_value, &loadable_manifest_output).await?, get_js_paths_from_root(&node_root_value, loadable_manifest_output).await?,
); );
} }
if emit_manifests != EmitManifests::None { if emit_manifests != EmitManifests::None {
......
...@@ -59,7 +59,7 @@ impl Asset for FontManifest { ...@@ -59,7 +59,7 @@ impl Asset for FontManifest {
// `_next` gets added again later, so we "strip" it here via // `_next` gets added again later, so we "strip" it here via
// `get_font_paths_from_root`. // `get_font_paths_from_root`.
let font_paths: Vec<String> = let font_paths: Vec<String> =
get_font_paths_from_root(client_root, &all_client_output_assets) get_font_paths_from_root(client_root, all_client_output_assets)
.await? .await?
.iter() .iter()
.filter_map(|p| p.split("_next/").last().map(|f| f.to_string())) .filter_map(|p| p.split("_next/").last().map(|f| f.to_string()))
......
...@@ -148,7 +148,7 @@ impl InstrumentationEndpoint { ...@@ -148,7 +148,7 @@ impl InstrumentationEndpoint {
let node_root_value = node_root.clone(); let node_root_value = node_root.clone();
let file_paths_from_root = let file_paths_from_root =
get_js_paths_from_root(&node_root_value, &edge_chunk_group.await?.assets.await?) get_js_paths_from_root(&node_root_value, edge_chunk_group.await?.assets.await?)
.await?; .await?;
let mut output_assets = edge_chunk_group.all_assets().owned().await?; let mut output_assets = edge_chunk_group.all_assets().owned().await?;
......
...@@ -263,7 +263,7 @@ impl MiddlewareEndpoint { ...@@ -263,7 +263,7 @@ impl MiddlewareEndpoint {
let edge_assets = edge_chunk_group_ref.assets.await?; let edge_assets = edge_chunk_group_ref.assets.await?;
let file_paths_from_root = let file_paths_from_root =
get_js_paths_from_root(&node_root_value, &edge_assets).await?; get_js_paths_from_root(&node_root_value, edge_assets.iter().copied()).await?;
let entrypoint_asset = *edge_assets let entrypoint_asset = *edge_assets
.last() .last()
.context("expected assets for edge middleware endpoint")?; .context("expected assets for edge middleware endpoint")?;
...@@ -278,7 +278,7 @@ impl MiddlewareEndpoint { ...@@ -278,7 +278,7 @@ impl MiddlewareEndpoint {
get_wasm_paths_from_root(&node_root_value, edge_all_assets.await?).await?; get_wasm_paths_from_root(&node_root_value, edge_all_assets.await?).await?;
let all_assets = let all_assets =
get_asset_paths_from_root(&node_root_value, &edge_all_assets.await?).await?; get_asset_paths_from_root(&node_root_value, edge_all_assets.await?).await?;
let regions = if let Some(regions) = config.preferred_region.as_ref() { let regions = if let Some(regions) = config.preferred_region.as_ref() {
if regions.len() == 1 { if regions.len() == 1 {
......
...@@ -107,7 +107,6 @@ impl NextDynamicGraphs { ...@@ -107,7 +107,6 @@ impl NextDynamicGraphs {
.get_next_dynamic_imports_for_endpoint(entry) .get_next_dynamic_imports_for_endpoint(entry)
.await? .await?
.into_iter() .into_iter()
.map(|(k, v)| (*k, *v))
// TODO remove this collect and return an iterator instead // TODO remove this collect and return an iterator instead
.collect::<Vec<_>>()) .collect::<Vec<_>>())
}) })
......
...@@ -215,7 +215,7 @@ impl ServerNftJsonAsset { ...@@ -215,7 +215,7 @@ impl ServerNftJsonAsset {
// These are used by packages/next/src/server/require-hook.ts // These are used by packages/next/src/server/require-hook.ts
let shared_entries = ["styled-jsx", "styled-jsx/style", "styled-jsx/style.js"]; let shared_entries = ["styled-jsx", "styled-jsx/style", "styled-jsx/style.js"];
let cache_handler_entries = cache_handler.into_iter().chain(cache_handlers).map(|f| { let cache_handler_entries = cache_handler.iter().chain(cache_handlers.iter()).map(|f| {
asset_context asset_context
.process( .process(
Vc::upcast(FileSource::new(f.clone())), Vc::upcast(FileSource::new(f.clone())),
......
...@@ -1430,8 +1430,9 @@ impl PageEndpoint { ...@@ -1430,8 +1430,9 @@ impl PageEndpoint {
let pages_structure = this.pages_structure.await?; let pages_structure = this.pages_structure.await?;
if pages_structure.should_create_pages_entries { if pages_structure.should_create_pages_entries {
server_assets.extend(assets_ref.iter().copied()); server_assets.extend(assets_ref.iter().copied());
file_paths_from_root file_paths_from_root.extend(
.extend(get_js_paths_from_root(&node_root, &assets_ref).await?); get_js_paths_from_root(&node_root, assets_ref.iter().copied()).await?,
);
} }
if emit_manifests == EmitManifests::Full { if emit_manifests == EmitManifests::Full {
...@@ -1445,28 +1446,30 @@ impl PageEndpoint { ...@@ -1445,28 +1446,30 @@ impl PageEndpoint {
if pages_structure.should_create_pages_entries { if pages_structure.should_create_pages_entries {
server_assets.extend(loadable_manifest_output.iter().copied()); server_assets.extend(loadable_manifest_output.iter().copied());
file_paths_from_root.extend( file_paths_from_root.extend(
get_js_paths_from_root(&node_root, &loadable_manifest_output) get_js_paths_from_root(&node_root, loadable_manifest_output)
.await?, .await?,
); );
} }
} }
let (wasm_paths_from_root, all_assets) = let (wasm_paths_from_root, all_assets) = if pages_structure
if pages_structure.should_create_pages_entries { .should_create_pages_entries
let all_output_assets = all_assets_from_entries(all_assets).await?; {
let all_output_assets = all_assets_from_entries(all_assets).await?;
let mut wasm_paths_from_root = fxindexset![]; let mut wasm_paths_from_root = fxindexset![];
wasm_paths_from_root.extend( wasm_paths_from_root.extend(
get_wasm_paths_from_root(&node_root, &all_output_assets).await?, get_wasm_paths_from_root(&node_root, all_output_assets.iter().copied())
); .await?,
);
let all_assets = let all_assets =
get_asset_paths_from_root(&node_root, &all_output_assets).await?; get_asset_paths_from_root(&node_root, all_output_assets).await?;
(wasm_paths_from_root, all_assets) (wasm_paths_from_root, all_assets)
} else { } else {
(fxindexset![], vec![]) (fxindexset![], vec![])
}; };
let named_regex = get_named_middleware_regex(&this.pathname).into(); let named_regex = get_named_middleware_regex(&this.pathname).into();
let matchers = ProxyMatcher { let matchers = ProxyMatcher {
......
...@@ -96,7 +96,7 @@ pub async fn all_paths_in_root( ...@@ -96,7 +96,7 @@ pub async fn all_paths_in_root(
assets: Vc<OutputAssets>, assets: Vc<OutputAssets>,
root: FileSystemPath, root: FileSystemPath,
) -> Result<Vc<Vec<RcStr>>> { ) -> Result<Vc<Vec<RcStr>>> {
let all_assets = &*all_assets_from_entries(assets).await?; let all_assets = all_assets_from_entries(assets).await?;
Ok(Vc::cell( Ok(Vc::cell(
get_paths_from_root(&root, all_assets, |_| true).await?, get_paths_from_root(&root, all_assets, |_| true).await?,
...@@ -105,12 +105,12 @@ pub async fn all_paths_in_root( ...@@ -105,12 +105,12 @@ pub async fn all_paths_in_root(
pub(crate) async fn get_paths_from_root( pub(crate) async fn get_paths_from_root(
root: &FileSystemPath, root: &FileSystemPath,
output_assets: impl IntoIterator<Item = &ResolvedVc<Box<dyn OutputAsset>>>, output_assets: impl IntoIterator<Item = ResolvedVc<Box<dyn OutputAsset>>>,
filter: impl FnOnce(&str) -> bool + Copy, filter: impl FnOnce(&str) -> bool + Copy,
) -> Result<Vec<RcStr>> { ) -> Result<Vec<RcStr>> {
output_assets output_assets
.into_iter() .into_iter()
.map(move |&file| async move { .map(move |file| async move {
let path = &*file.path().await?; let path = &*file.path().await?;
let Some(relative) = root.get_path_to(path) else { let Some(relative) = root.get_path_to(path) else {
return Ok(None); return Ok(None);
...@@ -128,18 +128,18 @@ pub(crate) async fn get_paths_from_root( ...@@ -128,18 +128,18 @@ pub(crate) async fn get_paths_from_root(
pub(crate) async fn get_js_paths_from_root( pub(crate) async fn get_js_paths_from_root(
root: &FileSystemPath, root: &FileSystemPath,
output_assets: impl IntoIterator<Item = &ResolvedVc<Box<dyn OutputAsset>>>, output_assets: impl IntoIterator<Item = ResolvedVc<Box<dyn OutputAsset>>>,
) -> Result<Vec<RcStr>> { ) -> Result<Vec<RcStr>> {
get_paths_from_root(root, output_assets, |path| path.ends_with(".js")).await get_paths_from_root(root, output_assets, |path| path.ends_with(".js")).await
} }
pub(crate) async fn get_wasm_paths_from_root( pub(crate) async fn get_wasm_paths_from_root(
root: &FileSystemPath, root: &FileSystemPath,
output_assets: impl IntoIterator<Item = &ResolvedVc<Box<dyn OutputAsset>>>, output_assets: impl IntoIterator<Item = ResolvedVc<Box<dyn OutputAsset>>>,
) -> Result<Vec<(RcStr, ResolvedVc<Box<dyn OutputAsset>>)>> { ) -> Result<Vec<(RcStr, ResolvedVc<Box<dyn OutputAsset>>)>> {
output_assets output_assets
.into_iter() .into_iter()
.map(move |&file| async move { .map(move |file| async move {
let path = &*file.path().await?; let path = &*file.path().await?;
let Some(relative) = root.get_path_to(path) else { let Some(relative) = root.get_path_to(path) else {
return Ok(None); return Ok(None);
...@@ -157,7 +157,7 @@ pub(crate) async fn get_wasm_paths_from_root( ...@@ -157,7 +157,7 @@ pub(crate) async fn get_wasm_paths_from_root(
pub(crate) async fn get_asset_paths_from_root( pub(crate) async fn get_asset_paths_from_root(
root: &FileSystemPath, root: &FileSystemPath,
output_assets: impl IntoIterator<Item = &ResolvedVc<Box<dyn OutputAsset>>>, output_assets: impl IntoIterator<Item = ResolvedVc<Box<dyn OutputAsset>>>,
) -> Result<Vec<RcStr>> { ) -> Result<Vec<RcStr>> {
get_paths_from_root(root, output_assets, |path| { get_paths_from_root(root, output_assets, |path| {
!path.ends_with(".js") && !path.ends_with(".map") && !path.ends_with(".wasm") !path.ends_with(".js") && !path.ends_with(".map") && !path.ends_with(".wasm")
...@@ -167,7 +167,7 @@ pub(crate) async fn get_asset_paths_from_root( ...@@ -167,7 +167,7 @@ pub(crate) async fn get_asset_paths_from_root(
pub(crate) async fn get_font_paths_from_root( pub(crate) async fn get_font_paths_from_root(
root: &FileSystemPath, root: &FileSystemPath,
output_assets: impl IntoIterator<Item = &ResolvedVc<Box<dyn OutputAsset>>>, output_assets: impl IntoIterator<Item = ResolvedVc<Box<dyn OutputAsset>>>,
) -> Result<Vec<RcStr>> { ) -> Result<Vec<RcStr>> {
get_paths_from_root(root, output_assets, |path| { get_paths_from_root(root, output_assets, |path| {
path.ends_with(".woff") path.ends_with(".woff")
......
...@@ -2696,10 +2696,7 @@ async fn any_output_changed( ...@@ -2696,10 +2696,7 @@ async fn any_output_changed(
server: bool, server: bool,
) -> Result<Vc<Completion>> { ) -> Result<Vc<Completion>> {
let all_assets = expand_output_assets( let all_assets = expand_output_assets(
roots roots.await?.into_iter().map(ExpandOutputAssetsInput::Asset),
.await?
.into_iter()
.map(|&a| ExpandOutputAssetsInput::Asset(a)),
true, true,
) )
.await?; .await?;
......
...@@ -68,11 +68,7 @@ pub async fn endpoints_outputs(endpoints: Vc<Endpoints>) -> Result<Vc<OutputAsse ...@@ -68,11 +68,7 @@ pub async fn endpoints_outputs(endpoints: Vc<Endpoints>) -> Result<Vc<OutputAsse
.map(async |endpoint| endpoint.output().await?.output_assets.await) .map(async |endpoint| endpoint.output().await?.output_assets.await)
.try_join() .try_join()
.await?; .await?;
let set = all_outputs let set = all_outputs.into_iter().flatten().collect::<FxIndexSet<_>>();
.into_iter()
.flatten()
.copied()
.collect::<FxIndexSet<_>>();
Ok(Vc::cell(set.into_iter().collect())) Ok(Vc::cell(set.into_iter().collect()))
} }
...@@ -105,7 +101,7 @@ pub async fn expand_outputs( ...@@ -105,7 +101,7 @@ pub async fn expand_outputs(
.await? .await?
.into_iter() .into_iter()
.flatten() .flatten()
.map(|asset| ExpandOutputAssetsInput::Asset(*asset)), .map(ExpandOutputAssetsInput::Asset),
true, true,
) )
.await?; .await?;
...@@ -138,7 +134,7 @@ impl Asset for AssetHashesManifestAsset { ...@@ -138,7 +134,7 @@ impl Asset for AssetHashesManifestAsset {
let output_assets = expand_outputs(*self.project, self.asset_root.clone()).await?; let output_assets = expand_outputs(*self.project, self.asset_root.clone()).await?;
let asset_paths = output_assets let asset_paths = output_assets
.into_iter() .iter()
.map(async |(asset, path)| { .map(async |(asset, path)| {
Ok(( Ok((
path, path,
......
...@@ -176,7 +176,6 @@ async fn module_graphs_of_endpoints( ...@@ -176,7 +176,6 @@ async fn module_graphs_of_endpoints(
.try_flat_join() .try_flat_join()
.await? .await?
.into_iter() .into_iter()
.copied()
.collect::<FxIndexSet<_>>() .collect::<FxIndexSet<_>>()
.into_iter() .into_iter()
.collect::<Vec<_>>(); .collect::<Vec<_>>();
......
...@@ -57,11 +57,7 @@ pub async fn endpoints_outputs(endpoints: Vc<Endpoints>) -> Result<Vc<OutputAsse ...@@ -57,11 +57,7 @@ pub async fn endpoints_outputs(endpoints: Vc<Endpoints>) -> Result<Vc<OutputAsse
.map(async |endpoint| Ok(endpoint.output().await?.output_assets.await?)) .map(async |endpoint| Ok(endpoint.output().await?.output_assets.await?))
.try_join() .try_join()
.await?; .await?;
let set = all_outputs let set = all_outputs.into_iter().flatten().collect::<FxIndexSet<_>>();
.into_iter()
.flatten()
.copied()
.collect::<FxIndexSet<_>>();
Ok(Vc::cell(set.into_iter().collect())) Ok(Vc::cell(set.into_iter().collect()))
} }
...@@ -71,7 +67,7 @@ pub async fn outputs_hash(outputs: Vc<OutputAssets>) -> Result<Vc<u64>> { ...@@ -71,7 +67,7 @@ pub async fn outputs_hash(outputs: Vc<OutputAssets>) -> Result<Vc<u64>> {
outputs outputs
.await? .await?
.into_iter() .into_iter()
.map(|asset| ExpandOutputAssetsInput::Asset(*asset)), .map(ExpandOutputAssetsInput::Asset),
true, true,
) )
.await?; .await?;
...@@ -89,12 +85,11 @@ pub async fn endpoint_entry_modules( ...@@ -89,12 +85,11 @@ pub async fn endpoint_entry_modules(
base_module_graph: Vc<ModuleGraph>, base_module_graph: Vc<ModuleGraph>,
endpoint: Vc<Box<dyn Endpoint>>, endpoint: Vc<Box<dyn Endpoint>>,
) -> Result<Vc<Modules>> { ) -> Result<Vc<Modules>> {
let entries = endpoint.entries(); let entries = endpoint.entries().await?;
let additional_entries = endpoint.additional_entries(base_module_graph); let additional_entries = endpoint.additional_entries(base_module_graph).await?;
let modules = entries let modules = entries
.await? .iter()
.into_iter() .chain(additional_entries.iter())
.chain(additional_entries.await?)
.flat_map(|e| e.entries()) .flat_map(|e| e.entries())
.collect::<FxIndexSet<_>>(); .collect::<FxIndexSet<_>>();
Ok(Vc::cell(modules.into_iter().collect())) Ok(Vc::cell(modules.into_iter().collect()))
...@@ -116,11 +111,11 @@ pub async fn endpoints_entry_modules( ...@@ -116,11 +111,11 @@ pub async fn endpoints_entry_modules(
.try_join() .try_join()
.await?; .await?;
let modules = entries_and_additional_entries let modules = entries_and_additional_entries
.into_iter() .iter()
.flat_map(|(entries, additional_entries)| { .flat_map(|(entries, additional_entries)| {
entries entries
.into_iter() .iter()
.chain(additional_entries) .chain(additional_entries.iter())
.flat_map(|e| e.entries()) .flat_map(|e| e.entries())
}) })
.collect::<FxIndexSet<_>>(); .collect::<FxIndexSet<_>>();
...@@ -136,7 +131,7 @@ pub async fn sources_hash(module_graph: Vc<ModuleGraph>, modules: Vc<Modules>) - ...@@ -136,7 +131,7 @@ pub async fn sources_hash(module_graph: Vc<ModuleGraph>, modules: Vc<Modules>) -
let module_graph = module_graph.await?; let module_graph = module_graph.await?;
module_graph.traverse_nodes_dfs( module_graph.traverse_nodes_dfs(
modules.into_iter().copied(), modules,
&mut all_modules, &mut all_modules,
|module, all_modules| { |module, all_modules| {
all_modules.insert(*module); all_modules.insert(*module);
...@@ -196,7 +191,7 @@ impl Asset for RoutesHashesManifestAsset { ...@@ -196,7 +191,7 @@ impl Asset for RoutesHashesManifestAsset {
let entrypoint_groups = self.project.get_all_endpoint_groups(false).await?; let entrypoint_groups = self.project.get_all_endpoint_groups(false).await?;
for (key, EndpointGroup { primary, .. }) in entrypoint_groups { for (key, EndpointGroup { primary, .. }) in &entrypoint_groups {
let entry = if let &[entry] = &primary.as_slice() { let entry = if let &[entry] = &primary.as_slice() {
( (
sources_hash( sources_hash(
......
...@@ -305,14 +305,16 @@ async fn hmr( ...@@ -305,14 +305,16 @@ async fn hmr(
.await?; .await?;
let start = Instant::now(); let start = Instant::now();
for ident in idents { for ident in &idents {
if !ident.ends_with(".js") { if !ident.ends_with(".js") {
continue; continue;
} }
let session = session.clone(); let session = session.clone();
let start = Instant::now(); let start = Instant::now();
let ident_for_task = ident.clone();
let task = tt.spawn_root_task(move || { let task = tt.spawn_root_task(move || {
let session = session.clone(); let session = session.clone();
let ident = ident_for_task.clone();
async move { async move {
let project = project.project(); let project = project.project();
let state = project.hmr_version_state(ident.clone(), HmrTarget::Client, session); let state = project.hmr_version_state(ident.clone(), HmrTarget::Client, session);
......
...@@ -53,7 +53,6 @@ impl OutputAssetsReference for BuildManifest { ...@@ -53,7 +53,6 @@ impl OutputAssetsReference for BuildManifest {
let references = chunks let references = chunks
.into_iter() .into_iter()
.flatten() .flatten()
.copied()
.chain(root_main_files) .chain(root_main_files)
.chain(self.polyfill_files.iter().copied()) .chain(self.polyfill_files.iter().copied())
.collect(); .collect();
......
...@@ -3,7 +3,6 @@ use std::{ ...@@ -3,7 +3,6 @@ use std::{
fmt::{self, Debug, Display}, fmt::{self, Debug, Display},
hash::{Hash, Hasher}, hash::{Hash, Hasher},
marker::PhantomData, marker::PhantomData,
mem::transmute_copy,
ops::Deref, ops::Deref,
}; };
...@@ -141,6 +140,7 @@ where ...@@ -141,6 +140,7 @@ where
} }
} }
/// Iterate by reference over a [`ReadRef`].
impl<'a, T, I, J: Iterator<Item = I>> IntoIterator for &'a ReadRef<T> impl<'a, T, I, J: Iterator<Item = I>> IntoIterator for &'a ReadRef<T>
where where
T: VcValueType, T: VcValueType,
...@@ -155,44 +155,63 @@ where ...@@ -155,44 +155,63 @@ where
} }
} }
impl<T, I: 'static, J: Iterator<Item = I>> IntoIterator for ReadRef<T> impl<T, I, J> IntoIterator for ReadRef<T>
where where
T: VcValueType, T: VcValueType,
&'static VcReadTarget<T>: IntoIterator<Item = I, IntoIter = J>, I: Copy + 'static,
J: Iterator<Item = &'static I> + 'static,
&'static VcReadTarget<T>: IntoIterator<Item = &'static I, IntoIter = J>,
{ {
type Item = I; type Item = I;
type IntoIter = ReadRefIter<T, I, J>; type IntoIter = ReadRefIter<T, I, J>;
fn into_iter(self) -> Self::IntoIter { fn into_iter(self) -> Self::IntoIter {
let r = &*self; let r: &VcReadTarget<T> = &self;
// # Safety // SAFETY: The `&'static` reference fabricated here is only stored in
// The reference will we valid as long as the ReadRef is valid. // `iter`, which lives inside the returned `ReadRefIter` alongside the
let r = unsafe { transmute_copy::<&'_ VcReadTarget<T>, &'static VcReadTarget<T>>(&r) }; // `ReadRef` that owns the data. The public `Iterator::next` only
// returns `Copy`-ed-out values — no reference (with the fake `'static`
// lifetime or otherwise) ever leaves the iterator. Struct-field drop
// order (`iter` then `_read_ref`) drops any references still held by
// `iter` before the backing storage.
let r = unsafe { std::mem::transmute::<&VcReadTarget<T>, &'static VcReadTarget<T>>(r) };
ReadRefIter { ReadRefIter {
read_ref: self,
iter: r.into_iter(), iter: r.into_iter(),
_read_ref: self,
} }
} }
} }
pub struct ReadRefIter<T, I: 'static, J: Iterator<Item = I>> /// Consuming iteration over a [`ReadRef`], yielding items by **copy**.
///
/// `Iterator::Item` is a fixed associated type — it cannot borrow from
/// `&mut self`.
///
/// The iterator owns the original [`ReadRef`], borrows into the underlying value, and
/// `Iterator::next` simply copies each element out of that borrow. This restricts the impl to
/// element types that are [`Copy`] — typically `ResolvedVc<_>`, integer ids, etc. For
/// non-`Copy` element types (or if you want zero-copy iteration over
/// borrows), iterate by reference instead: `for item in &read_ref { ... }`.
pub struct ReadRefIter<T, I, J>
where where
T: VcValueType, T: VcValueType,
I: Copy + 'static,
J: Iterator<Item = &'static I>,
{ {
iter: J, iter: J,
#[allow(dead_code)] _read_ref: ReadRef<T>,
read_ref: ReadRef<T>,
} }
impl<T, I: 'static, J: Iterator<Item = I>> Iterator for ReadRefIter<T, I, J> impl<T, I, J> Iterator for ReadRefIter<T, I, J>
where where
T: VcValueType, T: VcValueType,
I: Copy + 'static,
J: Iterator<Item = &'static I>,
{ {
type Item = I; type Item = I;
fn next(&mut self) -> Option<Self::Item> { fn next(&mut self) -> Option<I> {
self.iter.next() self.iter.next().copied()
} }
} }
......
...@@ -119,9 +119,9 @@ impl EcmascriptBrowserChunkContent { ...@@ -119,9 +119,9 @@ impl EcmascriptBrowserChunkContent {
let content = this.content.await?; let content = this.content.await?;
let chunk_items = content.chunk_item_code_and_ids().await?; let chunk_items = content.chunk_item_code_and_ids().await?;
for item in chunk_items { for item in &chunk_items {
for (id, item_code) in item { for (id, item_code) in &**item {
write!(code, "\n{}, ", StringifyJs(&id))?; write!(code, "\n{}, ", StringifyJs(id))?;
code.push_code(item_code); code.push_code(item_code);
write!(code, ",")?; write!(code, ",")?;
} }
......
...@@ -107,7 +107,7 @@ impl AvailableModules { ...@@ -107,7 +107,7 @@ impl AvailableModules {
match idents { match idents {
IdentStrings::Single(ident) => hasher.write_value(ident), IdentStrings::Single(ident) => hasher.write_value(ident),
IdentStrings::Multiple(idents) => { IdentStrings::Multiple(idents) => {
for ident in idents { for ident in &idents {
hasher.write_value(ident); hasher.write_value(ident);
} }
} }
......
...@@ -232,7 +232,7 @@ async fn batch_chunk_items_with_info( ...@@ -232,7 +232,7 @@ async fn batch_chunk_items_with_info(
) -> Result<Vc<BatchChunkItemsWithInfo>> { ) -> Result<Vc<BatchChunkItemsWithInfo>> {
let split_batch_group = batch_group.split_by_chunk_type().await?; let split_batch_group = batch_group.split_by_chunk_type().await?;
if split_batch_group.len() == 1 { if split_batch_group.len() == 1 {
let &(ty, batch) = split_batch_group.into_iter().next().unwrap(); let (ty, batch) = split_batch_group.into_iter().next().unwrap();
Ok(batch_chunk_items_with_info_with_type( Ok(batch_chunk_items_with_info_with_type(
*batch, *batch,
*ty, *ty,
...@@ -241,9 +241,7 @@ async fn batch_chunk_items_with_info( ...@@ -241,9 +241,7 @@ async fn batch_chunk_items_with_info(
} else { } else {
let maps = split_batch_group let maps = split_batch_group
.into_iter() .into_iter()
.map(|&(ty, batch)| { .map(|(ty, batch)| batch_chunk_items_with_info_with_type(*batch, *ty, chunking_context))
batch_chunk_items_with_info_with_type(*batch, *ty, chunking_context)
})
.try_join() .try_join()
.await?; .await?;
Ok(Vc::cell( Ok(Vc::cell(
......
...@@ -197,13 +197,11 @@ impl ChunkGroupResult { ...@@ -197,13 +197,11 @@ impl ChunkGroupResult {
.await? .await?
.into_iter() .into_iter()
.chain(self.referenced_assets.await?) .chain(self.referenced_assets.await?)
.copied()
.map(ExpandOutputAssetsInput::Asset) .map(ExpandOutputAssetsInput::Asset)
.chain( .chain(
self.references self.references
.await? .await?
.into_iter() .into_iter()
.copied()
.map(ExpandOutputAssetsInput::Reference), .map(ExpandOutputAssetsInput::Reference),
), ),
false, false,
...@@ -226,13 +224,11 @@ impl ChunkGroupResult { ...@@ -226,13 +224,11 @@ impl ChunkGroupResult {
self.referenced_assets self.referenced_assets
.await? .await?
.into_iter() .into_iter()
.copied()
.map(ExpandOutputAssetsInput::Asset) .map(ExpandOutputAssetsInput::Asset)
.chain( .chain(
self.references self.references
.await? .await?
.into_iter() .into_iter()
.copied()
.map(ExpandOutputAssetsInput::Reference), .map(ExpandOutputAssetsInput::Reference),
), ),
false, false,
......
...@@ -25,7 +25,7 @@ pub async fn compute_async_module_info( ...@@ -25,7 +25,7 @@ pub async fn compute_async_module_info(
// Layout segment optimization, we can individually compute the async modules for each graph. // Layout segment optimization, we can individually compute the async modules for each graph.
let mut result = None; let mut result = None;
for graph in graphs.iter_graphs().await? { for graph in graphs.iter_graphs().await? {
result = Some(compute_async_module_info_single(*graph, result)); result = Some(compute_async_module_info_single(graph, result));
} }
Ok(result Ok(result
.context("There must be at least one single graph in the module graph")? .context("There must be at least one single graph in the module graph")?
......
...@@ -138,12 +138,12 @@ impl OutputAssetsWithReferenced { ...@@ -138,12 +138,12 @@ impl OutputAssetsWithReferenced {
.await? .await?
.into_iter() .into_iter()
.chain(self.referenced_assets.await?) .chain(self.referenced_assets.await?)
.map(|&asset| ExpandOutputAssetsInput::Asset(asset)) .map(ExpandOutputAssetsInput::Asset)
.chain( .chain(
self.references self.references
.await? .await?
.into_iter() .into_iter()
.map(|&reference| ExpandOutputAssetsInput::Reference(reference)), .map(ExpandOutputAssetsInput::Reference),
), ),
inner_output_assets, inner_output_assets,
) )
...@@ -224,13 +224,11 @@ impl OutputAssetsWithReferenced { ...@@ -224,13 +224,11 @@ impl OutputAssetsWithReferenced {
self.referenced_assets self.referenced_assets
.await? .await?
.into_iter() .into_iter()
.copied()
.map(ExpandOutputAssetsInput::Asset) .map(ExpandOutputAssetsInput::Asset)
.chain( .chain(
self.references self.references
.await? .await?
.into_iter() .into_iter()
.copied()
.map(ExpandOutputAssetsInput::Reference), .map(ExpandOutputAssetsInput::Reference),
), ),
false, false,
...@@ -259,12 +257,12 @@ async fn get_referenced_assets( ...@@ -259,12 +257,12 @@ async fn get_referenced_assets(
.await? .await?
.into_iter() .into_iter()
.chain(refs.referenced_assets.await?) .chain(refs.referenced_assets.await?)
.map(|&asset| ExpandOutputAssetsInput::Asset(asset)) .map(ExpandOutputAssetsInput::Asset)
.chain( .chain(
refs.references refs.references
.await? .await?
.into_iter() .into_iter()
.map(|&reference| ExpandOutputAssetsInput::Reference(reference)), .map(ExpandOutputAssetsInput::Reference),
); );
Ok(Either::Right(assets)) Ok(Either::Right(assets))
} }
......
...@@ -258,7 +258,7 @@ pub async fn all_assets_from_entries( ...@@ -258,7 +258,7 @@ pub async fn all_assets_from_entries(
entries entries
.await? .await?
.into_iter() .into_iter()
.map(|&asset| ExpandOutputAssetsInput::Asset(asset)), .map(ExpandOutputAssetsInput::Asset),
true, true,
) )
.await?, .await?,
......
...@@ -1470,7 +1470,7 @@ async fn find_package( ...@@ -1470,7 +1470,7 @@ async fn find_package(
let matches = let matches =
read_matches(dir.clone(), rcstr!(""), true, package_name_with_extensions) read_matches(dir.clone(), rcstr!(""), true, package_name_with_extensions)
.await?; .await?;
for m in matches { for m in &matches {
if let PatternMatch::File(_, package_file) = m { if let PatternMatch::File(_, package_file) = m {
packages.push(FindPackageItem::PackageFile { packages.push(FindPackageItem::PackageFile {
name: get_package_name(dir, package_file)?, name: get_package_name(dir, package_file)?,
......
...@@ -226,7 +226,7 @@ impl OutputAssetsReference for CssModuleChunkItem { ...@@ -226,7 +226,7 @@ impl OutputAssetsReference for CssModuleChunkItem {
async fn references(&self) -> Result<Vc<OutputAssetsWithReferenced>> { async fn references(&self) -> Result<Vc<OutputAssetsWithReferenced>> {
let mut references = Vec::new(); let mut references = Vec::new();
if let ParseCssResult::Ok { url_references, .. } = &*self.module.parse_css().await? { if let ParseCssResult::Ok { url_references, .. } = &*self.module.parse_css().await? {
for (_, reference) in url_references.await? { for (_, reference) in &*url_references.await? {
if let ReferencedAsset::Some(asset) = *reference if let ReferencedAsset::Some(asset) = *reference
.get_referenced_asset(*self.chunking_context) .get_referenced_asset(*self.chunking_context)
.await? .await?
......
...@@ -137,7 +137,7 @@ async fn expand( ...@@ -137,7 +137,7 @@ async fn expand(
.await? .await?
.into_iter() .into_iter()
.chain(refs.referenced_assets.await?); .chain(refs.referenced_assets.await?);
for &asset in ref_assets { for asset in ref_assets {
if assets_set.insert(asset) { if assets_set.insert(asset) {
let path = asset.path().await?; let path = asset.path().await?;
if let Some(sub_path) = root_path.get_path_to(&path) { if let Some(sub_path) = root_path.get_path_to(&path) {
......
...@@ -146,9 +146,9 @@ async fn resolve_reference_from_dir( ...@@ -146,9 +146,9 @@ async fn resolve_reference_from_dir(
}; };
let matches = abs_matches let matches = abs_matches
.into_iter() .iter()
.flatten() .flatten()
.chain(rel_matches.into_iter().flatten()); .chain(rel_matches.iter().flatten());
let mut affecting_sources = Vec::new(); let mut affecting_sources = Vec::new();
let mut results = Vec::new(); let mut results = Vec::new();
...@@ -168,7 +168,7 @@ async fn resolve_reference_from_dir( ...@@ -168,7 +168,7 @@ async fn resolve_reference_from_dir(
results.push(( results.push((
RequestKey::new(matched_path.clone()), RequestKey::new(matched_path.clone()),
ResolvedVc::upcast( ResolvedVc::upcast(
RawModule::new(Vc::upcast(FileSource::new(path))) RawModule::new(Vc::upcast(FileSource::new(path.clone())))
.to_resolved() .to_resolved()
.await?, .await?,
), ),
......
...@@ -52,7 +52,7 @@ pub async fn node_file_trace( ...@@ -52,7 +52,7 @@ pub async fn node_file_trace(
} }
println!("FILELIST:"); println!("FILELIST:");
for a in result { for a in &result {
println!("{a}"); println!("{a}");
} }
......
...@@ -69,9 +69,9 @@ impl EcmascriptBuildNodeChunkContent { ...@@ -69,9 +69,9 @@ impl EcmascriptBuildNodeChunkContent {
let content = self.content.await?; let content = self.content.await?;
let chunk_items = content.chunk_item_code_and_ids().await?; let chunk_items = content.chunk_item_code_and_ids().await?;
for item in chunk_items { for item in &chunk_items {
for (id, item_code) in item { for (id, item_code) in &**item {
write!(code, "\n{}, ", StringifyJs(&id))?; write!(code, "\n{}, ", StringifyJs(id))?;
code.push_code(item_code); code.push_code(item_code);
write!(code, ",")?; write!(code, ",")?;
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment