[turbo-tasks-backend] Clean up task state before publishing execution completion (#99729)
## What? Task completion now cleans up the task's per-execution state (removing cells that no longer exist, `cleanup_after_execution`) under the same task lock that finishes the execution, before the completion is published (in-progress state removed, `done_event` notified). Previously the cleanup ran afterwards, under a second lock acquisition. A waiter could invalidate the task in that gap and its next execution could start. The old cleanup would then remove that execution's state, e.g. the `outdated_*_dependencies` it was just given, and could also remove cells it had created. ## Why? `collectibles_outdated_dependency` fails locally on canary (5/5 runs here) and passes in CI only by timing. Tracing showed the old cleanup observing the next collector execution already running with populated `outdated_collectibles_dependencies`, then clearing them. Later collectible emissions then made the waiting collector stale, so it executed an extra time. The optimization from #99386 assumes that bookkeeping is intact. There is a second, independent reason the test can see an extra execution: `State::set` invalidates its readers one at a time. The collector can then read the source before the source was invalidated, see the old collectibles, and legitimately re-execute when the new ones arrive. ## How? - `task_execution_completed_cleanup` now takes the held `TaskGuard`. `task_execution_completed_finish` calls it after setting output and immutability, before dropping the guard and notifying `done_event`. The canceled path still cleans up, and stale reschedules (including `verify_determinism` reruns) still skip cleanup, as before. Removed cell data is still dropped outside the critical section. - Tests: - The existing test is unchanged. Its docs now say it is timing-dependent and also covers this cleanup race. - A new gated variant stages the overlap explicitly: the collector waits until the new emitter execution has started, polls its strongly consistent source read to `Pending`, and only then lets the emitter emit. This deterministically covers the dirty-not-stale optimization, independent of `State` invalidation order. ## Verification - With the fix: both tests pass 30/30 locally, and `cargo test -p turbo-tasks-backend` passes. - Mutant (always stale on collectible change): the gated test fails 10/10. - Known limitation: there is no deterministic regression for the cleanup race itself without a backend test hook. With canary's ordering restored, the ungated test failed intermittently, between 0/20 and 8/20 depending on test layout and invocation. The fix was confirmed by tracing and by an experiment that widened the old gap. - Clippy (default and `verify_serialization,print_cache_item_size`), a `verify_determinism` check, and `cargo test -p turbo-tasks-macros-tests` pass. <!-- NEXT_JS_LLM --> <!-- fleet 98a896e8-00fa-4f81-97f0-48bd49e2a9a2 --> Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Showing
Please
register
or
sign in
to comment