turbo-tasks-backend: fan out large aggregation update job and removal batches
### What?
An aggregation update queue now runs large batches in parallel instead of one
by one:
- **Jobs:** when it holds at least 4096 `jobs`, all of them run in parallel.
- **Edge removals:** when at least 4096 edge removals are due, all of them run
in parallel. Removals are due when no job, aggregation number update or
balancing is pending; this is the same rule #100892 uses to process
removals one by one.
How the parallel batch runs:
- Each chunk runs on a child execution context and pushes the work its jobs
generate into a private buffer queue. Buffers are never processed.
- After all chunks finish, every buffer is merged back into the queue:
- generated jobs are appended to `jobs`;
- generated removals are appended to the deferred removals, FIFO;
- aggregation number updates go through the same done-map-aware merging as
regular pushes;
- balance and find-and-schedule work stays deduplicated;
- scheduled tasks keep their highest priority;
- optimizations respect the queue's cap and budget. Refused optimizations
set `optimization_pending`, as before.
- GC queues, fan-out buffers and contexts with a GC collectible callback never
fan out.
Stacked on #100889 (on #100892 and #100873).
### Why?
One queue runs on one thread, but a single update can cascade into a very
large number of jobs, e.g. when a task with many followers becomes an
aggregating node, or loses many followers. This spreads such a cascade over
the available threads.
Every job of a batch completes on its own, which is what `scope_bounded`
requires. A removal that runs before its addition records follower debt
(#100873) instead of waiting. Additions that pay that debt come back through
the merge and run to completion with the queue.
This changes one ordering rule from #100892, and only for removal batches at
or above the threshold. Processed one by one, the work a removal generates
runs before the next removal. In a parallel batch, it runs after the whole
batch, with the normal priorities. A batch still only starts when no job,
aggregation number update or balancing is pending, so removals still run
after the additions queued before them. Below the threshold, nothing changes.
This is a parallelization change. It is not a fix for the aggregation edge
removal panic or memory runaways; that is #100873.
### How?
- `JOB_FAN_OUT_THRESHOLD = 4096`. `AggregationUpdateQueue::fan_out_kind` picks
`FanOutKind::Jobs`, `FanOutKind::Removals` or none.
- `fan_out` takes all queued jobs of that kind and runs them via
`turbo_tasks::parallel::map_collect_chunked_owned`. Jobs and removals share
one implementation.
- `ChildExecuteContext` is now `Copy`. Its priority is still resolved on the
calling thread.
- New: `ExecuteContext::has_gc_collectible`.
- Aggregation number merging moved into `push_aggregation_number_update`, so
`push` and the buffer merge share it.
- The test hook type is now `Send + Sync`. Buffers never inherit hooks.
- Tests cover:
- the threshold and the exclusions, for both batch kinds;
- that no removal runs, and no removal batch starts, while other work is
pending;
- every buffer merge rule, including removals;
- work generated by a removal batch, with the final edges and collectible
counts;
- activation with a test-only threshold: additions, deferred removals and
then an invalidation. They check that both batch kinds fanned out, and
compare the graph and data against sequential processing, with 1 and 4
runtime workers.
### Verification
On top of #100889 `e44ad7f0`:
- `cargo test -p turbo-tasks-backend` passes in default,
`verify_aggregation_graph` and `aggregation_update_no_batch` modes (400
tests each). This includes the deferred removal, follower debt and held
dirty state tests from the PRs below, unchanged.
- clippy `--all-targets -D warnings` (default and
`verify_aggregation_graph`), `cargo fmt --check`, and a `cargo check` with
`trace_aggregation_update_stats,trace_aggregation_update_queue` pass.
- Locally forced threshold of 4 (not committed): the full suite passes in
default and `verify_aggregation_graph` modes with no tests excluded, with
about 24k jobs fan-outs and 10k removal fan-outs per mode.
- `turbo_tasks_backend_priority_contention/realistic_warm/128x512` (the
realistic contention bench from #100561, added locally, not part of this
PR; Criterion defaults; 32 CPUs), 6 interleaved runs on one boot. All runs
completed, with peak RSS around 0.26 GB:
| | Runs | ok | Median ms per run |
|---|---:|---:|---|
| #100889 | 3 | 3 | 67.05, 67.05, 69.79 |
| This PR | 3 | 3 | 67.81, 67.49, 66.97 |
No clear difference; this bench shows no measured speed-up.
<!-- NEXT_JS_LLM -->
Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
Showing
This diff is collapsed.
Please
register
or
sign in
to comment