Unverified Commit 0fc57c31 authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

Don't optimize the aggregation graph during `connect_child` (#100147)

If a task completion enqueues too much optimization work we record an
`optimization_pending` flag so that it can be processed in a later
attempt.

However `connect_child` calls `increase_active_count` which attempted to
flush pending optimizations. This could greatly slow down execution by
enqueueing graph management in the hot `connect_child` path. But also it
could be self defeating, since we would optimize a task, and then add it
as a child, which would (on task_completion) potentially add a new
`upper` which could trigger more optimizations.

For `Module::ident` this could be pathological since a lot of large
tasks call it for all modules (e.g. to compute module ids), each of
these could enqueue optimization work and then the next task (e.g. parts
of asset tracing) could both complete optimizations and enqueue new
ones.

This severely punished `project_feature_usage` which was single threaded
and thus did a lot of expensive work.
parent 6f6ea204
...@@ -1034,11 +1034,11 @@ impl AggregationUpdateQueue { ...@@ -1034,11 +1034,11 @@ impl AggregationUpdateQueue {
/// update handlers that already hold a task guard. The flag itself is cleared later, /// update handlers that already hold a task guard. The flag itself is cleared later,
/// when `optimize_task` actually runs for this task. /// when `optimize_task` actually runs for this task.
/// ///
/// Intentionally not called from `find_and_schedule_dirty_internal`: that function /// Intentionally not called from `find_and_schedule_dirty_internal` or increase_active_count:
/// schedules dirty tasks but doesn't perform aggregation work that materially benefits /// these functions do not modify the aggregation graph or use it in a way that benefits from
/// from up-to-date optimization, so we leave its tasks for one of the other recovery /// up-to-date optimization, so we leave its tasks for one of the other recovery hooks to
/// hooks to pick up. There is currently no dedicated unit test for the drop/recovery /// pick up. These functions are also in the crticial path of execution, it is better to defer
/// paths; coverage relies on integration tests that exercise the aggregation graph. /// optimization work to where it is required.
fn check_optimization_pending(&mut self, task: &TaskGuard<'_>) { fn check_optimization_pending(&mut self, task: &TaskGuard<'_>) {
if !task.optimization_pending() { if !task.optimization_pending() {
// Common case: nothing pending, no work to do. // Common case: nothing pending, no work to do.
...@@ -3162,7 +3162,6 @@ impl AggregationUpdateQueue { ...@@ -3162,7 +3162,6 @@ impl AggregationUpdateQueue {
); );
// Revive the task if GC soft-deleted it. // Revive the task if GC soft-deleted it.
let mut task = resurrect_deleted(task, task_id, self, ctx); let mut task = resurrect_deleted(task, task_id, self, ctx);
self.check_optimization_pending(&task);
let state = task.get_activeness_mut_or_insert_with(|| ActivenessState::new(task_id)); let state = task.get_activeness_mut_or_insert_with(|| ActivenessState::new(task_id));
let is_new = state.is_empty(); let is_new = state.is_empty();
let is_positive_now = state.increment_active_counter(); let is_positive_now = state.increment_active_counter();
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment