turbo-persistence: parallelize large batch_get reads (#100620)
### What? Parallelize large `TurboPersistence::batch_get` calls in `turbo-persistence`. - Batches with **more than 2048 keys** (`PARALLEL_BATCH_GET_THRESHOLD`) are split into chunks and run on the database's existing `ParallelScheduler`. Smaller batches run serially, one chunk per shard. - `split_into_batch_chunks` aims for **4 chunks per task the scheduler can run at the same time** (like `turbo_tasks::util::good_chunk_size`). Every chunk stays inside _one shard_ of the family's current shard index, so small shards produce smaller chunks. - New `ParallelScheduler` methods: - `parallelism()`: the number of tasks the scheduler can run at once. The turbo-tasks scheduler returns `turbo_tasks::parallel::available_parallelism()`, `SerialScheduler` returns 1. - `try_parallel_for_each_task_mut`: runs exactly one task per (already chunked) item. - Less memory traffic in `batch_get`: - The sort is over compact `(u64 hash, u32 index)` cells. - Values are materialized as soon as an SST lookup finds them and written directly into the output `Vec`. There is no final loop to re-order the results. - Parallel chunks write disjoint indices concurrently through a small `ConcurrentSlice`. - Each chunk tracks the keys already found in newer SST files with a `turbo_bitset::Bitset` backed by a `SmallVec`, plus a counter. The bitset is preallocated for the chunk's positions (`Bitset::with_len`) and its `SmallVec` stores up to 256 positions inline, so typical chunks don't heap-allocate and larger chunks allocate their known size once. - `MetaFile::batch_lookup_entry` was adapted to match. - `batch_get` now requires `K: QueryKey + Sync`. All existing callers already satisfy it. ### Why? Warm builds restore task data with a few very large batches: up to about 68K keys, ~24–27 calls over 2048 keys per build. Those reads are expensive serially. ### Results Warm-build benchmark from #100570 (`bench/nested-deps-app-router-many-pages`, `bench-warm-build.mjs --files=1000`) on a 32 vCPU Linux VM with release native builds. The first version of this PR (fixed 256-key chunks), interleaved on the same machine against canary and against itself with parallelism forced off (benchmark-only env override). 12 iterations with 3 warm-up, i.e. 9 timed builds per run: | warm compile, median | canary | this PR, serial | this PR, 2048 | |---|---|---|---| | pass A | 7.70 s | 7.70 s | **6.80 s** | | pass B | 8.20 s | 8.20 s | **7.20 s** | - That is about **0.9–1.0 s (~12%)** faster compile with 1,000 changed files. Earlier rounds on other machines point the same way (0.6–1.3 s). - The serial restructuring on its own is neutral, so the gain comes from parallelism. Switching from fixed 256-key chunks to parallelism-based chunk counts (and to `turbo_bitset::Bitset`), 3 interleaved passes with 6 timed builds each, on one machine: | warm compile / wall, median | 256-key chunks | parallelism-based | |---|---|---| | pass 1 | 7.10 / 13.32 s | 7.75 / 13.89 s | | pass 2 | 6.85 / 12.57 s | 6.65 / 11.96 s | | pass 3 | 7.25 / 13.15 s | 6.85 / 12.87 s | No consistent difference in warm builds. In a micro benchmark (Rayon, 32 threads, two passes) 4K-key batches get ~21–38% faster, 16K-key batches are about the same, and 64K-key batches are mixed (up to ~10–16% slower in some cases). Per-call profile of a `--files=1000` warm build (~148K `batch_get` calls, ~1.84M keys): - calls with more than 2048 keys carry ~36% of the keys; - their summed elapsed time drops from **1.9–2.4 s** (serial) to **0.21–0.25 s** (parallel); - calls with 128 keys or fewer are 99.96% of all calls and are not affected by this change. Much smaller thresholds were also tried but there was no benefit. <!-- NEXT_JS_LLM --> <!-- fleet 2e9e7c99-0cc2-4e7b-9e62-627c0627638b --> --------- Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Showing
This diff is collapsed.
Please
register
or
sign in
to comment