Unverified Commit a72565ab authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

turbo-persistence: parallelize large batch_get reads (#100620)

### What?

Parallelize large `TurboPersistence::batch_get` calls in
`turbo-persistence`.

- Batches with **more than 2048 keys** (`PARALLEL_BATCH_GET_THRESHOLD`)
are split into chunks and run on the database's existing
`ParallelScheduler`. Smaller batches run serially, one chunk per shard.
- `split_into_batch_chunks` aims for **4 chunks per task the scheduler
can run at the same time** (like `turbo_tasks::util::good_chunk_size`).
Every chunk stays inside _one shard_ of the family's current shard
index, so small shards produce smaller chunks.
- New `ParallelScheduler` methods:
- `parallelism()`: the number of tasks the scheduler can run at once.
The turbo-tasks scheduler returns
`turbo_tasks::parallel::available_parallelism()`, `SerialScheduler`
returns 1.
- `try_parallel_for_each_task_mut`: runs exactly one task per (already
chunked) item.
- Less memory traffic in `batch_get`:
  - The sort is over compact `(u64 hash, u32 index)` cells.
- Values are materialized as soon as an SST lookup finds them and
written directly into the output `Vec`. There is no final loop to
re-order the results.
- Parallel chunks write disjoint indices concurrently through a small
`ConcurrentSlice`.
- Each chunk tracks the keys already found in newer SST files with a
`turbo_bitset::Bitset` backed by a `SmallVec`, plus a counter. The
bitset is preallocated for the chunk's positions (`Bitset::with_len`)
and its `SmallVec` stores up to 256 positions inline, so typical chunks
don't heap-allocate and larger chunks allocate their known size once.
  - `MetaFile::batch_lookup_entry` was adapted to match.
- `batch_get` now requires `K: QueryKey + Sync`. All existing callers
already satisfy it.

### Why?

Warm builds restore task data with a few very large batches: up to about
68K keys, ~24–27 calls over 2048 keys per build. Those reads are
expensive serially.

### Results

Warm-build benchmark from #100570
(`bench/nested-deps-app-router-many-pages`, `bench-warm-build.mjs
--files=1000`) on a 32 vCPU Linux VM with release native builds.

The first version of this PR (fixed 256-key chunks), interleaved on the
same machine against canary and against itself with parallelism forced
off (benchmark-only env override). 12 iterations with 3 warm-up, i.e. 9
timed builds per run:

| warm compile, median | canary | this PR, serial | this PR, 2048 |
|---|---|---|---|
| pass A | 7.70 s | 7.70 s | **6.80 s** |
| pass B | 8.20 s | 8.20 s | **7.20 s** |

- That is about **0.9–1.0 s (~12%)** faster compile with 1,000 changed
files. Earlier rounds on other machines point the same way (0.6–1.3 s).
- The serial restructuring on its own is neutral, so the gain comes from
parallelism.

Switching from fixed 256-key chunks to parallelism-based chunk counts
(and to `turbo_bitset::Bitset`), 3 interleaved passes with 6 timed
builds each, on one machine:

| warm compile / wall, median | 256-key chunks | parallelism-based |
|---|---|---|
| pass 1 | 7.10 / 13.32 s | 7.75 / 13.89 s |
| pass 2 | 6.85 / 12.57 s | 6.65 / 11.96 s |
| pass 3 | 7.25 / 13.15 s | 6.85 / 12.87 s |

No consistent difference in warm builds. In a micro benchmark (Rayon, 32
threads, two passes) 4K-key batches get ~21–38% faster, 16K-key batches
are about the same, and 64K-key batches are mixed (up to ~10–16% slower
in some cases).

Per-call profile of a `--files=1000` warm build (~148K `batch_get`
calls, ~1.84M keys):
- calls with more than 2048 keys carry ~36% of the keys;
- their summed elapsed time drops from **1.9–2.4 s** (serial) to
**0.21–0.25 s** (parallel);
- calls with 128 keys or fewer are 99.96% of all calls and are not
affected by this change.

Much smaller thresholds were also tried but there was no benefit.

<!-- NEXT_JS_LLM -->


<!-- fleet 2e9e7c99-0cc2-4e7b-9e62-627c0627638b -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: 's avatarLuke Sandberg <210140+lukesandberg@users.noreply.github.com>
parent 34bfd9bb
...@@ -9181,6 +9181,7 @@ dependencies = [ ...@@ -9181,6 +9181,7 @@ dependencies = [
"tempfile", "tempfile",
"thread_local", "thread_local",
"tracing", "tracing",
"turbo-bitset",
"turbo-tasks-malloc", "turbo-tasks-malloc",
"xxhash-rust", "xxhash-rust",
"zerocopy", "zerocopy",
......
...@@ -37,6 +37,7 @@ serde_json = { workspace = true } ...@@ -37,6 +37,7 @@ serde_json = { workspace = true }
smallvec = { workspace = true } smallvec = { workspace = true }
thread_local = { workspace = true } thread_local = { workspace = true }
tracing = { workspace = true } tracing = { workspace = true }
turbo-bitset = { workspace = true }
xxhash-rust = { workspace = true } xxhash-rust = { workspace = true }
# Compression is skipped under Miri. # Compression is skipped under Miri.
......
use std::cell::UnsafeCell;
/// A mutably borrowed slice whose elements can be written from multiple threads, as long as each
/// element is written at most once while the slice is shared.
pub(crate) struct ConcurrentSlice<'l, T> {
data: &'l [UnsafeCell<T>],
}
// SAFETY: Through a shared `ConcurrentSlice`, other threads move `T` values into the slice and drop
// the values they replace, which requires `T: Send`. It never hands out references to elements, so
// it doesn't require `T: Sync`. Callers of `write` guarantee that no element is accessed by two
// threads. Distinct elements are distinct memory locations, so writes from different threads to
// different elements never race or tear, even when `T` is a single byte. This is the same
// guarantee that makes sending the halves of `split_at_mut` to different threads sound.
unsafe impl<T: Send> Send for ConcurrentSlice<'_, T> {}
// SAFETY: See above.
unsafe impl<T: Send> Sync for ConcurrentSlice<'_, T> {}
impl<'l, T> ConcurrentSlice<'l, T> {
pub(crate) fn new(slice: &'l mut [T]) -> Self {
let len = slice.len();
// SAFETY: `UnsafeCell<T>` has the same memory layout as `T` (it is `#[repr(transparent)]`),
// and the pointer comes from a mutable borrow of `len` elements that lasts for `'l`, so no
// other access to the elements exists while `data` is alive.
let data =
unsafe { std::slice::from_raw_parts(slice.as_mut_ptr().cast::<UnsafeCell<T>>(), len) };
Self { data }
}
/// Replaces the element at `index` with `value`, dropping the previous value. Panics if
/// `index` is out of bounds.
///
/// # Safety
///
/// Each `index` must be written at most once while `self` is shared, so no two threads access
/// the same element.
pub(crate) unsafe fn write(&self, index: usize, value: T) {
let cell = &self.data[index];
// SAFETY: The caller guarantees that no other access to this element happens concurrently.
unsafe { *cell.get() = value };
}
}
#[cfg(test)]
mod tests {
use crate::concurrent_slice::ConcurrentSlice;
#[test]
fn writes_from_multiple_threads() {
const LEN: usize = 64;
// The previous values are boxed, so leaking or double-dropping them is detected by Miri.
let mut values: Vec<Option<Box<usize>>> = vec![Some(Box::new(usize::MAX)); LEN];
let writer = ConcurrentSlice::new(&mut values);
std::thread::scope(|scope| {
for thread in 0..4 {
let writer = &writer;
scope.spawn(move || {
for index in (thread..LEN).step_by(4) {
// SAFETY: Each thread writes a distinct residue class of indices.
unsafe { writer.write(index, Some(Box::new(index))) };
}
});
}
});
for (index, value) in values.iter().enumerate() {
assert_eq!(value.as_deref(), Some(&index));
}
}
#[test]
fn writes_adjacent_bytes_from_multiple_threads() {
const LEN: usize = 64;
let mut values = [0u8; LEN];
let writer = ConcurrentSlice::new(&mut values);
std::thread::scope(|scope| {
for thread in 0..4 {
let writer = &writer;
scope.spawn(move || {
// Adjacent bytes are written by different threads.
for index in (thread..LEN).step_by(4) {
// SAFETY: Each thread writes a distinct residue class of indices.
unsafe { writer.write(index, index as u8 + 1) };
}
});
}
});
for (index, &value) in values.iter().enumerate() {
assert_eq!(value, index as u8 + 1);
}
}
#[test]
#[should_panic]
fn write_out_of_bounds_panics() {
let mut values = [0u32; 4];
let writer = ConcurrentSlice::new(&mut values);
// SAFETY: No other write happens.
unsafe { writer.write(4, 1) };
}
}
...@@ -15,6 +15,7 @@ mod collector; ...@@ -15,6 +15,7 @@ mod collector;
mod collector_entry; mod collector_entry;
mod compaction; mod compaction;
mod compression; mod compression;
mod concurrent_slice;
mod constants; mod constants;
mod db; mod db;
mod key; mod key;
......
...@@ -20,6 +20,7 @@ use zerocopy::{FromBytes, Immutable, IntoBytes, KnownLayout, Ref, big_endian as ...@@ -20,6 +20,7 @@ use zerocopy::{FromBytes, Immutable, IntoBytes, KnownLayout, Ref, big_endian as
use crate::mmap_helper::advise_mmap_for_persistence; use crate::mmap_helper::advise_mmap_for_persistence;
use crate::{ use crate::{
AccessMode, Compression, FamilyConfig, QueryKey, AccessMode, Compression, FamilyConfig, QueryKey,
db::BatchGetResolved,
lookup_entry::LookupValue, lookup_entry::LookupValue,
shard::ShardBits, shard::ShardBits,
static_sorted_file::{BlockCache, SstLookupResult, StaticSortedFile, StaticSortedFileMetaData}, static_sorted_file::{BlockCache, SstLookupResult, StaticSortedFile, StaticSortedFileMetaData},
...@@ -614,26 +615,32 @@ impl MetaFile { ...@@ -614,26 +615,32 @@ impl MetaFile {
} }
/// Looks up the keys of `cells` in the SST file of the entry `entry_index`, whose hash range is /// Looks up the keys of `cells` in the SST file of the entry `entry_index`, whose hash range is
/// `range`. Only cells without a result are looked up. `cells` must be sorted by key hash. /// `range`. `cells` holds `(key hash, key index)` pairs and must be sorted by key hash. Only
/// cells whose position is not in `resolved` yet are looked up. When a key is found, its
/// position is added to `resolved`, `resolved_count` is incremented and `on_found` is called
/// with the position of the cell and the value. `resolved_count` must be the number of
/// positions in `resolved`.
#[allow(clippy::too_many_arguments)] #[allow(clippy::too_many_arguments)]
pub(crate) fn batch_lookup_entry<K: QueryKey>( pub(crate) fn batch_lookup_entry<K: QueryKey>(
&self, &self,
entry_index: u32, entry_index: u32,
range: &StaticSortedFileRange, range: &StaticSortedFileRange,
keys: &[K], keys: &[K],
cells: &mut [(u64, usize, Option<LookupValue>)], cells: &[(u64, u32)],
empty_cells: &mut usize, resolved: &mut BatchGetResolved,
resolved_count: &mut usize,
key_block_cache: &BlockCache, key_block_cache: &BlockCache,
value_block_cache: &BlockCache, value_block_cache: &BlockCache,
mut on_found: impl FnMut(usize, LookupValue) -> Result<()>,
) -> Result<MetaBatchLookupResult> { ) -> Result<MetaBatchLookupResult> {
debug_assert!( debug_assert!(
cells.is_sorted_by_key(|(hash, _, _)| *hash), cells.is_sorted_by_key(|(hash, _)| *hash),
"Cells must be sorted by key hash" "Cells must be sorted by key hash"
); );
#[allow(unused_mut, reason = "It's used when stats are enabled")] #[allow(unused_mut, reason = "It's used when stats are enabled")]
let mut lookup_result = MetaBatchLookupResult::default(); let mut lookup_result = MetaBatchLookupResult::default();
let start_index = cells let start_index = cells
.binary_search_by(|(hash, _, _)| hash.cmp(&range.min_hash).then(Ordering::Greater)) .binary_search_by(|(hash, _)| hash.cmp(&range.min_hash).then(Ordering::Greater))
.err() .err()
.unwrap(); .unwrap();
if start_index >= cells.len() { if start_index >= cells.len() {
...@@ -644,7 +651,7 @@ impl MetaFile { ...@@ -644,7 +651,7 @@ impl MetaFile {
return Ok(lookup_result); return Ok(lookup_result);
} }
let end_index = cells let end_index = cells
.binary_search_by(|(hash, _, _)| hash.cmp(&range.max_hash).then(Ordering::Less)) .binary_search_by(|(hash, _)| hash.cmp(&range.max_hash).then(Ordering::Less))
.err() .err()
.unwrap(); .unwrap();
if start_index >= end_index { if start_index >= end_index {
...@@ -655,12 +662,15 @@ impl MetaFile { ...@@ -655,12 +662,15 @@ impl MetaFile {
return Ok(lookup_result); return Ok(lookup_result);
} }
let entry = &self.entries[entry_index as usize]; let entry = &self.entries[entry_index as usize];
for (hash, index, result) in &mut cells[start_index..end_index] { for (position, &(hash, index)) in cells.iter().enumerate().take(end_index).skip(start_index)
debug_assert!(range.contains(*hash), "Key hash out of range"); {
if result.is_some() { debug_assert!(range.contains(hash), "Key hash out of range");
// `batch_get` has at most `u32::MAX` keys, so positions fit in a `u32`.
let bit = position as u32;
if resolved.contains(bit) {
continue; continue;
} }
if !entry.amqf.contains_fingerprint(*hash) { if !entry.amqf.contains_fingerprint(hash) {
#[cfg(feature = "stats")] #[cfg(feature = "stats")]
{ {
lookup_result.quick_filter_misses += 1; lookup_result.quick_filter_misses += 1;
...@@ -668,8 +678,8 @@ impl MetaFile { ...@@ -668,8 +678,8 @@ impl MetaFile {
continue; continue;
} }
let sst_result = entry.sst(self)?.lookup::<_, false>( let sst_result = entry.sst(self)?.lookup::<_, false>(
*hash, hash,
&keys[*index], &keys[index as usize],
key_block_cache, key_block_cache,
value_block_cache, value_block_cache,
)?; )?;
...@@ -679,13 +689,15 @@ impl MetaFile { ...@@ -679,13 +689,15 @@ impl MetaFile {
let Some(value) = values.pop() else { let Some(value) = values.pop() else {
unreachable!() unreachable!()
}; };
*result = Some(value); let inserted = resolved.insert(bit);
*empty_cells -= 1; debug_assert!(inserted);
*resolved_count += 1;
#[cfg(feature = "stats")] #[cfg(feature = "stats")]
{ {
lookup_result.hits += 1; lookup_result.hits += 1;
} }
if *empty_cells == 0 { on_found(position, value)?;
if *resolved_count == cells.len() {
return Ok(lookup_result); return Ok(lookup_result);
} }
} else { } else {
......
pub trait ParallelScheduler: Clone + Sync + Send { pub trait ParallelScheduler: Clone + Sync + Send {
/// The number of tasks this scheduler can run at the same time. Always at least 1.
fn parallelism(&self) -> usize;
fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R
where where
R: Send; R: Send;
...@@ -25,6 +28,18 @@ pub trait ParallelScheduler: Clone + Sync + Send { ...@@ -25,6 +28,18 @@ pub trait ParallelScheduler: Clone + Sync + Send {
T: Send + Sync, T: Send + Sync,
E: Send + 'static; E: Send + 'static;
/// Like [`ParallelScheduler::try_parallel_for_each_mut`], but `items` are already chunked by
/// the caller: every item runs as its own task, and implementations must not group items
/// into larger chunks.
fn try_parallel_for_each_task_mut<T, E>(
&self,
items: &mut [T],
f: impl (Fn(&mut T) -> Result<(), E>) + Send + Sync,
) -> Result<(), E>
where
T: Send + Sync,
E: Send + 'static;
fn try_parallel_for_each_owned<T, E>( fn try_parallel_for_each_owned<T, E>(
&self, &self,
items: Vec<T>, items: Vec<T>,
...@@ -59,6 +74,10 @@ pub trait ParallelScheduler: Clone + Sync + Send { ...@@ -59,6 +74,10 @@ pub trait ParallelScheduler: Clone + Sync + Send {
pub struct SerialScheduler; pub struct SerialScheduler;
impl ParallelScheduler for SerialScheduler { impl ParallelScheduler for SerialScheduler {
fn parallelism(&self) -> usize {
1
}
fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R
where where
R: Send, R: Send,
...@@ -105,6 +124,21 @@ impl ParallelScheduler for SerialScheduler { ...@@ -105,6 +124,21 @@ impl ParallelScheduler for SerialScheduler {
Ok(()) Ok(())
} }
fn try_parallel_for_each_task_mut<T, E>(
&self,
items: &mut [T],
f: impl (Fn(&mut T) -> Result<(), E>) + Send + Sync,
) -> Result<(), E>
where
T: Send + Sync,
E: Send,
{
for item in items {
f(item)?;
}
Ok(())
}
fn try_parallel_for_each_owned<T, E>( fn try_parallel_for_each_owned<T, E>(
&self, &self,
items: Vec<T>, items: Vec<T>,
......
...@@ -4,7 +4,7 @@ use std::{fs, path::Path}; ...@@ -4,7 +4,7 @@ use std::{fs, path::Path};
use anyhow::Result; use anyhow::Result;
#[cfg(not(miri))] #[cfg(not(miri))]
use rayon::iter::{IntoParallelIterator, ParallelIterator}; use rayon::iter::{IndexedParallelIterator, IntoParallelIterator, ParallelIterator};
use rstest::rstest; use rstest::rstest;
// Miri rejects a process that exits while rayon's global worker threads are still parked, so // Miri rejects a process that exits while rayon's global worker threads are still parked, so
...@@ -31,6 +31,10 @@ struct RayonParallelScheduler; ...@@ -31,6 +31,10 @@ struct RayonParallelScheduler;
#[cfg(not(miri))] #[cfg(not(miri))]
impl ParallelScheduler for RayonParallelScheduler { impl ParallelScheduler for RayonParallelScheduler {
fn parallelism(&self) -> usize {
rayon::current_num_threads()
}
fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R
where where
R: Send, R: Send,
...@@ -69,6 +73,18 @@ impl ParallelScheduler for RayonParallelScheduler { ...@@ -69,6 +73,18 @@ impl ParallelScheduler for RayonParallelScheduler {
items.into_par_iter().try_for_each(f) items.into_par_iter().try_for_each(f)
} }
fn try_parallel_for_each_task_mut<T, E>(
&self,
items: &mut [T],
f: impl (Fn(&mut T) -> Result<(), E>) + Send + Sync,
) -> Result<(), E>
where
T: Send + Sync,
E: Send,
{
items.into_par_iter().with_max_len(1).try_for_each(f)
}
fn try_parallel_for_each_owned<T, E>( fn try_parallel_for_each_owned<T, E>(
&self, &self,
items: Vec<T>, items: Vec<T>,
...@@ -3289,3 +3305,97 @@ fn commits_with_more_shards_than_collectors_split_files_at_shard_boundaries( ...@@ -3289,3 +3305,97 @@ fn commits_with_more_shards_than_collectors_split_files_at_shard_boundaries(
} }
Ok(()) Ok(())
} }
// Writes 10K keys and runs the parallel `batch_get` path on Rayon, which is too slow under Miri.
#[cfg(not(miri))]
#[rstest]
#[case::mmap(true)]
#[case::file(false)]
fn batch_get_large_batches(#[case] mmap: bool) -> Result<()> {
const KEYS: u32 = 10_000;
let value_of = |key: u32, version: u8| -> Vec<u8> {
let mut value = vec![version; 8];
value[..4].copy_from_slice(&key.to_le_bytes());
value
};
let expected = |key: u32| -> Option<Vec<u8>> {
if key >= KEYS || key.is_multiple_of(11) {
None
} else if key.is_multiple_of(7) {
Some(value_of(key, 2))
} else {
Some(value_of(key, 1))
}
};
let tempdir = tempfile::tempdir()?;
let db = open_db_with_config::<1>(tempdir.path(), config_with_mmap::<1>(mmap))?;
let batch = db.write_batch()?;
for key in 0..KEYS {
batch.put(0, key.to_be_bytes().to_vec(), value_of(key, 1).into())?;
}
db.commit_write_batch(batch)?;
// A second commit adds a newer layer of SST files with overwrites and tombstones.
let batch = db.write_batch()?;
for key in 0..KEYS {
if key.is_multiple_of(11) {
batch.delete(0, key.to_be_bytes().to_vec())?;
} else if key.is_multiple_of(7) {
batch.put(0, key.to_be_bytes().to_vec(), value_of(key, 2).into())?;
}
}
db.commit_write_batch(batch)?;
// Batches at and around the parallel threshold, and well above it. Keys are spread over the
// key space in a scrambled order, include misses (keys >= KEYS) and duplicates.
for batch_size in [2048usize, 2049, 6000] {
let extra_keys = [0u32, 0, 1000, KEYS];
let keys: Vec<Vec<u8>> = (0..(batch_size - extra_keys.len()) as u32)
.map(|i| {
(i.wrapping_mul(7919) % (KEYS + KEYS / 10))
.to_be_bytes()
.to_vec()
})
.chain(extra_keys.map(|key| key.to_be_bytes().to_vec()))
.collect();
assert_eq!(keys.len(), batch_size);
#[cfg(feature = "stats")]
let before = db.statistics();
let results = db.batch_get(0, &keys)?;
assert_eq!(results.len(), batch_size);
#[cfg(feature = "stats")]
{
let after = db.statistics();
assert_eq!(
(after.hits + after.misses) - (before.hits + before.misses),
batch_size as u64,
"every key of the batch is counted as exactly one hit or miss"
);
}
for (key, result) in keys.iter().zip(&results) {
let key_num = u32::from_be_bytes(key[..].try_into().unwrap());
assert_eq!(
result.as_deref(),
expected(key_num).as_deref(),
"batch_get mismatch for key {key_num} in a batch of {batch_size}"
);
assert_eq!(result.as_deref(), db.get(0, key)?.as_deref());
}
}
// Duplicates of a key have the same hash, so they are adjacent and span several chunks.
let mut keys = vec![3u32.to_be_bytes().to_vec(); 3000];
keys.extend(vec![11u32.to_be_bytes().to_vec(); 600]);
let results = db.batch_get(0, &keys)?;
assert_eq!(results.len(), keys.len());
let (hits, deleted) = results.split_at(3000);
assert!(
hits.iter()
.all(|r| r.as_deref() == Some(&value_of(3, 1)[..]))
);
assert!(deleted.iter().all(Option::is_none));
db.shutdown();
Ok(())
}
use turbo_persistence::ParallelScheduler; use turbo_persistence::ParallelScheduler;
use turbo_tasks::{block_in_place, parallel}; use turbo_tasks::{block_in_place, parallel, scope_bounded::scope_bounded};
#[derive(Clone, Copy, Default)] #[derive(Clone, Copy, Default)]
pub struct TurboTasksParallelScheduler; pub struct TurboTasksParallelScheduler;
impl ParallelScheduler for TurboTasksParallelScheduler { impl ParallelScheduler for TurboTasksParallelScheduler {
fn parallelism(&self) -> usize {
// Same fallback as `turbo_tasks::util::good_chunk_size` when the parallelism is unknown.
parallel::available_parallelism().map_or(4, |count| count.get())
}
fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R fn block_in_place<R>(&self, f: impl FnOnce() -> R + Send) -> R
where where
R: Send, R: Send,
...@@ -43,6 +48,32 @@ impl ParallelScheduler for TurboTasksParallelScheduler { ...@@ -43,6 +48,32 @@ impl ParallelScheduler for TurboTasksParallelScheduler {
parallel::try_for_each_mut(items, f) parallel::try_for_each_mut(items, f)
} }
fn try_parallel_for_each_task_mut<T, E>(
&self,
items: &mut [T],
f: impl (Fn(&mut T) -> Result<(), E>) + Send + Sync,
) -> Result<(), E>
where
T: Send + Sync,
E: Send + 'static,
{
// The items are already chunked by the caller, so spawn one job per item instead of
// regrouping them with `parallel::try_for_each_mut`'s chunking heuristic.
match items {
[] => Ok(()),
[item] => f(item),
items => {
let f = &f;
scope_bounded(items.len(), |scope| {
for item in items.iter_mut() {
scope.spawn(move || f(item));
}
})
.collect::<Result<(), E>>()
}
}
}
fn try_parallel_for_each_owned<T, E>( fn try_parallel_for_each_owned<T, E>(
&self, &self,
items: Vec<T>, items: Vec<T>,
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment