Unverified Commit 0fc57c31 authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

Don't optimize the aggregation graph during `connect_child` (#100147)

If a task completion enqueues too much optimization work we record an
`optimization_pending` flag so that it can be processed in a later
attempt.

However `connect_child` calls `increase_active_count` which attempted to
flush pending optimizations. This could greatly slow down execution by
enqueueing graph management in the hot `connect_child` path. But also it
could be self defeating, since we would optimize a task, and then add it
as a child, which would (on task_completion) potentially add a new
`upper` which could trigger more optimizations.

For `Module::ident` this could be pathological since a lot of large
tasks call it for all modules (e.g. to compute module ids), each of
these could enqueue optimization work and then the next task (e.g. parts
of asset tracing) could both complete optimizations and enqueue new
ones.

This severely punished `project_feature_usage` which was single threaded
and thus did a lot of expensive work.
parent 6f6ea204
......@@ -1034,11 +1034,11 @@ impl AggregationUpdateQueue {
/// update handlers that already hold a task guard. The flag itself is cleared later,
/// when `optimize_task` actually runs for this task.
///
/// Intentionally not called from `find_and_schedule_dirty_internal`: that function
/// schedules dirty tasks but doesn't perform aggregation work that materially benefits
/// from up-to-date optimization, so we leave its tasks for one of the other recovery
/// hooks to pick up. There is currently no dedicated unit test for the drop/recovery
/// paths; coverage relies on integration tests that exercise the aggregation graph.
/// Intentionally not called from `find_and_schedule_dirty_internal` or increase_active_count:
/// these functions do not modify the aggregation graph or use it in a way that benefits from
/// up-to-date optimization, so we leave its tasks for one of the other recovery hooks to
/// pick up. These functions are also in the crticial path of execution, it is better to defer
/// optimization work to where it is required.
fn check_optimization_pending(&mut self, task: &TaskGuard<'_>) {
if !task.optimization_pending() {
// Common case: nothing pending, no work to do.
......@@ -3162,7 +3162,6 @@ impl AggregationUpdateQueue {
);
// Revive the task if GC soft-deleted it.
let mut task = resurrect_deleted(task, task_id, self, ctx);
self.check_optimization_pending(&task);
let state = task.get_activeness_mut_or_insert_with(|| ActivenessState::new(task_id));
let is_new = state.is_empty();
let is_positive_now = state.increment_active_counter();
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment