turbo-tasks-gc: Defer most aggregation graph management until after GC has settled (#98591)
Fix AggregationGraph updates during GC.
During GC we aremoving a lot of edges so this can trigger a lot of
aggregation graph churn. Because of the highly parallel nature of GC
this can end up with multiple queues fighting over updates which can
exceed various retry limits and lead to things like trying to turn nodes
into `followers` that are actively being deleted.
Instead we just don't rebalance the aggregation graph during GC, instead
we buffer every `rebalance` request and set the `optimize` budget to
zero. Then after GC settles we process all the rebalances at once,
taking care to filter out any tasks that have been since deleted.
Because GC is only removing edges it can lead to inefficient graphs (too
many interior nodes) but not violate graph invariants.
Alternatives considered:
The initial idea was to ensure GC had finished with all 'children' of
the node before processing rebalance requests, but this was only a
partial solution (and complex!), since it didn't handle the case of an
aggregating node having multiple children become collectible in the
initial scan, this could still trigger 'optimization contention' and
turning dead nodes into followers.
Testing:
The test/benchmark are enhanced to produce non-trivial aggregation
graphs which improves test coverage (though the issue of racing deletes
was trivially found by CI also)
---------
Co-authored-by:
Claude Opus 5 (1M context) <noreply@anthropic.com>
Showing
This diff is collapsed.
Please
register
or
sign in
to comment