Unverified Commit 74dc549c authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

[turbo-tasks-backend] Clean up task state before publishing execution completion (#99729)

## What?

Task completion now cleans up the task's per-execution state (removing
cells that no longer exist, `cleanup_after_execution`) under the same
task lock that finishes the execution, before the completion is
published (in-progress state removed, `done_event` notified).

Previously the cleanup ran afterwards, under a second lock acquisition.
A waiter could invalidate the task in that gap and its next execution
could start. The old cleanup would then remove that execution's state,
e.g. the `outdated_*_dependencies` it was just given, and could also
remove cells it had created.

## Why?

`collectibles_outdated_dependency` fails locally on canary (5/5 runs
here) and passes in CI only by timing. Tracing showed the old cleanup
observing the next collector execution already running with populated
`outdated_collectibles_dependencies`, then clearing them. Later
collectible emissions then made the waiting collector stale, so it
executed an extra time. The optimization from #99386 assumes that
bookkeeping is intact.

There is a second, independent reason the test can see an extra
execution: `State::set` invalidates its readers one at a time. The
collector can then read the source before the source was invalidated,
see the old collectibles, and legitimately re-execute when the new ones
arrive.

## How?

- `task_execution_completed_cleanup` now takes the held `TaskGuard`.
`task_execution_completed_finish` calls it after setting output and
immutability, before dropping the guard and notifying `done_event`. The
canceled path still cleans up, and stale reschedules (including
`verify_determinism` reruns) still skip cleanup, as before. Removed cell
data is still dropped outside the critical section.
- Tests:
- The existing test is unchanged. Its docs now say it is
timing-dependent and also covers this cleanup race.
- A new gated variant stages the overlap explicitly: the collector waits
until the new emitter execution has started, polls its strongly
consistent source read to `Pending`, and only then lets the emitter
emit. This deterministically covers the dirty-not-stale optimization,
independent of `State` invalidation order.

## Verification

- With the fix: both tests pass 30/30 locally, and `cargo test -p
turbo-tasks-backend` passes.
- Mutant (always stale on collectible change): the gated test fails
10/10.
- Known limitation: there is no deterministic regression for the cleanup
race itself without a backend test hook. With canary's ordering
restored, the ungated test failed intermittently, between 0/20 and 8/20
depending on test layout and invocation. The fix was confirmed by
tracing and by an experiment that widened the old gap.
- Clippy (default and `verify_serialization,print_cache_item_size`), a
`verify_determinism` check, and `cargo test -p turbo-tasks-macros-tests`
pass.

<!-- NEXT_JS_LLM -->


<!-- fleet 98a896e8-00fa-4f81-97f0-48bd49e2a9a2 -->
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: 's avatarLuke Sandberg <210140+lukesandberg@users.noreply.github.com>
parent ccf8c15a
......@@ -2180,12 +2180,13 @@ impl TurboTasksBackend {
has_invalidator: bool,
turbo_tasks: &TurboTasks<TurboTasksBackend>,
) -> Option<TaskPriority> {
// Task completion is a 4 step process:
// Task completion is a 3 step process:
// 1. Remove old edges (dependencies, collectibles, children, cells) and update the
// aggregation number of the task and the new children.
// 2. Connect the new children to the task (and do the relevant aggregation updates).
// 3. Remove dirty flag (and propagate that to uppers) and remove the in-progress state.
// 4. Shrink the task memory to reduce footprint of the task.
// 3. Remove the in-progress state and the dirty flag (and propagate that to uppers), remove
// no longer existing cells and shrink the task memory. The changes to the task itself
// happen under one lock, so the next execution can't start before the cleanup is done.
// The snapshot waits for this entire completion path, including its scoped child work,
// before it observes the graph.
......@@ -2279,15 +2280,19 @@ impl TurboTasksBackend {
return Some(stale_priority);
}
let (stale_priority, in_progress_cells) = self.task_execution_completed_finish(
&mut ctx,
task_id,
#[cfg(feature = "verify_determinism")]
no_output_set,
new_output,
is_now_immutable,
is_session_dependent,
);
let (stale_priority, in_progress_cells, removed_data) = self
.task_execution_completed_finish(
&mut ctx,
task_id,
#[cfg(feature = "verify_determinism")]
no_output_set,
new_output,
is_now_immutable,
is_session_dependent,
cell_counters,
is_error,
is_recomputation,
);
if let Some(stale_priority) = stale_priority {
// Task was stale and has been rescheduled
#[cfg(feature = "trace_task_details")]
......@@ -2295,14 +2300,6 @@ impl TurboTasksBackend {
return Some(stale_priority);
}
let removed_data = self.task_execution_completed_cleanup(
&mut ctx,
task_id,
cell_counters,
is_error,
is_recomputation,
);
// Drop data outside of critical sections
drop(removed_data);
drop(in_progress_cells);
......@@ -2809,11 +2806,15 @@ impl TurboTasksBackend {
new_output: Option<OutputValue>,
is_now_immutable: bool,
is_session_dependent: bool,
cell_counters: &AutoMap<ValueTypeId, u32, BuildHasherDefault<FxHasher>, 8>,
is_error: bool,
is_recomputation: bool,
) -> (
Option<TaskPriority>,
Option<
auto_hash_map::AutoMap<CellId, InProgressCellState, BuildHasherDefault<FxHasher>, 1>,
>,
Vec<SharedReference>,
) {
let mut task = ctx.task(task_id, TaskDataCategory::All);
let Some(in_progress) = task.take_in_progress() else {
......@@ -2821,7 +2822,13 @@ impl TurboTasksBackend {
};
if matches!(in_progress, InProgressState::Canceled) {
// Task was canceled in the meantime, so we don't finish it
return (None, None);
let removed_data = Self::task_execution_completed_cleanup(
&mut task,
cell_counters,
is_error,
is_recomputation,
);
return (None, None, removed_data);
}
let InProgressState::InProgress(InProgressStateInner {
done_event,
......@@ -2843,7 +2850,7 @@ impl TurboTasksBackend {
reason: TaskExecutionReason::Stale,
});
debug_assert!(old.is_none(), "InProgress already exists");
return (Some(stale_priority), None);
return (Some(stale_priority), None, Vec::new());
}
// Set the output if it has changed
......@@ -2885,19 +2892,29 @@ impl TurboTasksBackend {
.then(TaskPriority::leaf);
#[cfg(not(feature = "verify_determinism"))]
let stale_priority: Option<TaskPriority> = None;
if stale_priority.is_some() {
let removed_data = if stale_priority.is_some() {
let old = task.set_in_progress(InProgressState::Scheduled {
done_event,
reason: TaskExecutionReason::Stale,
});
debug_assert!(old.is_none(), "InProgress already exists");
drop(task);
Vec::new()
} else {
// Clean up before the completion becomes visible: once the guard is dropped, the next
// execution can start, and cleaning up after that would remove its state.
let removed_data = Self::task_execution_completed_cleanup(
&mut task,
cell_counters,
is_error,
is_recomputation,
);
drop(task);
// Notify dependent tasks that are waiting for this task to finish
done_event.notify(usize::MAX);
}
removed_data
};
drop(old_content);
......@@ -2906,18 +2923,24 @@ impl TurboTasksBackend {
}
// We return so the data can be dropped outside of critical sections
(stale_priority, in_progress_cells)
(stale_priority, in_progress_cells, removed_data)
}
/// Removes the cells this execution no longer has and frees the per-execution bookkeeping
/// (`cleanup_after_execution`).
///
/// Runs under the same task guard that completes the execution, before the completion is
/// published (`in_progress` removed and `done_event` notified). Otherwise a waiter could start
/// the next execution in between, and this cleanup would remove that execution's state, e.g.
/// the outdated dependencies it was just given.
///
/// Returns the removed cell data, so it can be dropped outside of the critical section.
fn task_execution_completed_cleanup(
&self,
ctx: &mut ExecuteContext<'_>,
task_id: TaskId,
task: &mut TaskGuard<'_>,
cell_counters: &AutoMap<ValueTypeId, u32, BuildHasherDefault<FxHasher>, 8>,
is_error: bool,
is_recomputation: bool,
) -> Vec<SharedReference> {
let mut task = ctx.task(task_id, TaskDataCategory::All);
let mut removed_cell_data = Vec::new();
// An error is potentially caused by a eventual consistency, so we avoid updating cells
// after an error as it is likely transient and we want to keep the dependent tasks
......@@ -2965,8 +2988,6 @@ impl TurboTasksBackend {
// Free memory now that execution is complete.
task.cleanup_after_execution();
drop(task);
// Return so we can drop outside of critical sections
removed_cell_data
}
......
......@@ -8,11 +8,27 @@
//! Until the collector reads the collectibles again, its dependency on them is left over from the
//! previous execution. Collectibles emitted in the meantime will be seen when the read happens, so
//! they must make the collector dirty without making the running execution stale.
//!
//! There are two variants:
//! - [`collector_is_not_re_executed_for_collectibles_it_has_not_read_yet`] lets the input change
//! race freely. It re-executes the collector right after its previous execution completed, which
//! also covers that the completion's cleanup can't remove the left-over dependencies of the next
//! execution. Its overlap depends on timing: a `State` invalidates its readers one by one, so the
//! collector could in principle read the source before the source was invalidated, see the old
//! collectibles, and then legitimately re-execute when the new ones arrive.
//! - [`gated_collector_is_not_re_executed_for_collectibles_it_has_not_read_yet`] stages the overlap
//! explicitly with [`Gates`], so it doesn't depend on that order.
use std::sync::atomic::{AtomicUsize, Ordering};
use std::{
future::{Future, poll_fn},
pin::pin,
sync::atomic::{AtomicBool, AtomicUsize, Ordering},
task::Poll,
};
use anyhow::Result;
use futures::future::try_join_all;
use tokio::sync::watch;
use turbo_rcstr::RcStr;
use turbo_tasks::{
CollectiblesSource, NonLocalValue, ReadRef, ResolvedVc, State, TransientInstance,
......@@ -117,3 +133,129 @@ async fn collector_is_not_re_executed_for_collectibles_it_has_not_read_yet() {
.await
.unwrap()
}
/// Shared between the collector and the emitter of one gated test case.
#[derive(NonLocalValue)]
struct Gates {
executions: AtomicUsize,
/// Set once the input has changed; only then do the gates below apply.
armed: AtomicBool,
/// The emitter's execution after the input change has started.
#[turbo_tasks(unsafe_ignore)] // holds no `Vc`
emitter_started: watch::Sender<bool>,
/// The collector's source read is waiting for the source.
#[turbo_tasks(unsafe_ignore)] // holds no `Vc`
collector_waiting: watch::Sender<bool>,
}
impl Default for Gates {
fn default() -> Self {
Self {
executions: AtomicUsize::new(0),
armed: AtomicBool::new(false),
emitter_started: watch::Sender::new(false),
collector_waiting: watch::Sender::new(false),
}
}
}
async fn wait_for(gate: &watch::Sender<bool>) {
gate.subscribe()
.wait_for(|open| *open)
.await
.expect("the sender is alive while it is borrowed");
}
/// Like [`emit_all`], but after the input change it only emits once the collector is waiting for
/// the source.
#[turbo_tasks::function]
async fn gated_emit_all(
input: ResolvedVc<Input>,
gates: TransientInstance<Gates>,
) -> Result<Vc<()>> {
let count = *input.await?.count.get();
if gates.armed.load(Ordering::Acquire) {
gates.emitter_started.send_replace(true);
wait_for(&gates.collector_waiting).await;
}
try_join_all((0..count).map(|i| emit_one(i).into_future())).await?;
Ok(Vc::cell(()))
}
#[turbo_tasks::function(operation, root)]
async fn gated_source(input: ResolvedVc<Input>, gates: TransientInstance<Gates>) -> Result<Vc<()>> {
gated_emit_all(*input, gates).await?;
Ok(Vc::cell(()))
}
/// Like [`collector`], but after the input change it only reads the source once the source was
/// invalidated by it (the emitter's new execution has started).
#[turbo_tasks::function(operation, root)]
async fn gated_collector(
input: ResolvedVc<Input>,
take: bool,
gates: TransientInstance<Gates>,
) -> Result<Vc<u32>> {
gates.executions.fetch_add(1, Ordering::AcqRel);
let _ = *input.await?.count.get();
let armed = gates.armed.load(Ordering::Acquire);
if armed {
wait_for(&gates.emitter_started).await;
}
let source = gated_source(input, gates.clone());
if armed {
let mut read = pin!(source.read_strongly_consistent());
poll_fn(|cx| match read.as_mut().poll(cx) {
Poll::Ready(_) => panic!("the source can't settle before it is allowed to emit"),
Poll::Pending => {
// The source emits while the read is waiting for it.
gates.collector_waiting.send_replace(true);
Poll::Ready(())
}
})
.await;
read.await?;
} else {
source.read_strongly_consistent().await?;
}
let collectibles = if take {
source.take_collectibles::<Box<dyn ValueToString>>()
} else {
source.peek_collectibles::<Box<dyn ValueToString>>()
};
Ok(Vc::cell(collectibles.len() as u32))
}
async fn gated_executions_after_change(take: bool) -> Result<usize> {
let input = ReadRef::new_owned(Input {
count: State::new(5),
});
let input_vc = ReadRef::resolved_cell(input.clone());
let gates = TransientInstance::new(Gates::default());
let collector = gated_collector(input_vc, take, gates.clone());
assert_eq!(*collector.read_strongly_consistent().await?, 5);
let initial = gates.executions.load(Ordering::Acquire);
assert_eq!(initial, 1, "take={take}: initial executions");
gates.armed.store(true, Ordering::Release);
input.count.set(200);
assert_eq!(*collector.read_strongly_consistent().await?, 200);
Ok(gates.executions.load(Ordering::Acquire) - initial)
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn gated_collector_is_not_re_executed_for_collectibles_it_has_not_read_yet() {
run_once(&REGISTRATION, async || {
for take in [false, true] {
assert_eq!(
gated_executions_after_change(take).await?,
1,
"take={take}: executions after the input change"
);
}
anyhow::Ok(())
})
.await
.unwrap()
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment