Unverified Commit 86d92b94 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

turbo-tasks-backend: check access in track_modification and only track Data on…

turbo-tasks-backend: check access in track_modification and only track Data on serialization invalidation (#99581)

### What?

Turbo-tasks backend: when a task guard marks a category (Data or Meta)
of a persistent task as modified, it now checks that the guard was
acquired with that category. Serialization invalidation now marks only
Data as modified.

### Why?

Marking a category as modified means the next snapshot writes the
in-memory copy of that category to disk. If the category was never
restored, the in-memory copy is empty, and the snapshot overwrites the
real persisted data.

`invalidate_serialization` hit exactly this case:

- The backend acquires the task with Data access only.
- The guard then marked both Data and Meta as modified, bypassing any
access check.
- After an eviction, mutating a `State` stored in a cell therefore
persisted an empty Meta.
- On the next restore the task's output was gone, so the task silently
re-executed.

### How?

- **Access check.** `TaskGuardImpl::track_modification` runs the
existing debug access assertion before delegating for non-transient
tasks. Transient tasks are still skipped.
- Every caller is now covered, including the macro-generated accessors
and the hand-written cell-data mutators.
- Any future mismatch fails loudly in debug builds and tests instead of
corrupting persisted state.
- **Data only for serialization invalidation.** Serialization
invalidation concerns cell data (for example a `State` mutated in
place), so it now marks only Data, through the checked guard path rather
than the raw storage guard.
- **Audit of the remaining callers.** None of them track a category they
don't hold:
  - GC tombstoning and resurrection acquire `All`.
  - The cell-data mutators check Data.
  - New-task initialization restores `All`.
- The generated accessors derive the check and the tracked category from
the same field category.
- **Regression test.** `tests/invalidate_serialization.rs` runs a task
whose cell holds a `State` and snapshots/evicts it (asserting that both
Data and Meta were evicted). It then mutates the state, snapshots again,
and asserts the task is served from cache without re-executing.
- The test deliberately has no readers of the state. A re-executing
dependent would restore the task's Meta before the snapshot and hide the
bug.
  - With the old tracking it fails on every run; with the fix it passes.

Verified with `cargo test -p turbo-tasks-backend` (debug build, so the
access assertions are active), plus clippy and rustfmt.

<!-- NEXT_JS_LLM -->


<!-- fleet c78c9059-116c-46e3-8ab3-5adc412156f1 -->
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 59644380
......@@ -1708,15 +1708,12 @@ impl TaskGuard for TaskGuardImpl<'_> {
fn invalidate_serialization(&mut self) {
// TODO this causes race conditions, since we never know when a value is changed. We can't
// "snapshot" the value correctly.
if !self.task_id.is_transient() {
// Unconditional track (no mutation to detect a no-op against): always mark dirty.
let _ = self
.task
.track_modification(SpecificTaskDataCategory::Data, "invalidate_serialization");
let _ = self
.task
.track_modification(SpecificTaskDataCategory::Meta, "invalidate_serialization");
}
// Serialization invalidation is about cell data (e.g. a `State` mutated in place), so only
// the Data category is marked modified. Tracking Meta here would be wrong: callers only
// restore Data, and persisting an unrestored Meta would overwrite the stored Meta with an
// empty one. `track_modification` skips transient tasks and checks the access category.
// Unconditional track (no mutation to detect a no-op against): always mark dirty.
let _ = self.track_modification(SpecificTaskDataCategory::Data, "invalidate_serialization");
}
fn prefetch(&mut self) -> Option<FxIndexMap<TaskId, TaskDataCategory>> {
......@@ -1778,6 +1775,9 @@ impl TaskStorageAccessors for TaskGuardImpl<'_> {
// Transient tasks are never persisted, so there is nothing to track or undo.
TrackOutcome::NoChange
} else {
// Marking a category as modified persists it on the next snapshot, so the guard must
// have been acquired with (i.e. restored) that category.
self.check_access(category);
self.task.track_modification(category, name)
}
}
......
......@@ -837,7 +837,7 @@ impl StorageWriteGuard<'_> {
#[cfg(feature = "trace_task_modification")] name: &str,
) -> TrackOutcome {
// Transient tasks are never persisted, so tracking modifications is meaningless.
// All callers (TaskGuard, invalidate_serialization) already
// All callers (TaskGuard, initialize_new_task) already
// guard against this, but we enforce it here as defense-in-depth.
debug_assert!(
!self.inner.key().is_transient(),
......
#![feature(arbitrary_self_types)]
#![feature(arbitrary_self_types_pointers)]
#![allow(clippy::needless_return)] // tokio macro-generated code doesn't respect this
//! Serialization invalidation (e.g. mutating a `State` stored in a cell) on a task whose Meta
//! category is not restored must not persist that unrestored (empty) Meta over the real one.
mod util;
use std::sync::atomic::{AtomicU32, Ordering};
use turbo_tasks::{State, Vc};
use turbo_tasks_backend::TestSnapshotOutcome;
use crate::util::create_tt;
static CREATE_STATE_EXECUTIONS: AtomicU32 = AtomicU32::new(0);
#[turbo_tasks::value(transparent)]
struct Step(State<u32>);
#[turbo_tasks::function(operation, root)]
fn create_state(initial: u32) -> Vc<Step> {
CREATE_STATE_EXECUTIONS.fetch_add(1, Ordering::SeqCst);
Step(State::new(initial)).cell()
}
/// 1. Run `create_state`, then snapshot and evict it, so it loses its restored Meta and Data.
/// 2. `state.set` calls `invalidate_serialization` on `create_state`, which only restores Data.
/// 3. Snapshot again. If Meta was (wrongly) marked modified while unrestored, the empty in-memory
/// Meta overwrites the persisted one (losing e.g. the task output).
/// 4. Reading `create_state` again restores Meta from disk and must not re-execute it.
///
/// Nothing reads the state's value (no `State::get`), so `set` invalidates no dependents. A
/// re-executing dependent would read `create_state`'s cell, restoring its Meta before the snapshot
/// and hiding the bug.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn invalidate_serialization_does_not_persist_unrestored_meta() {
let (tt, _persistence_dir) =
create_tt("invalidate_serialization_does_not_persist_unrestored_meta");
let state = turbo_tasks::run_once(tt.clone(), async move {
let state = create_state(1).read_strongly_consistent().await?;
assert_eq!(CREATE_STATE_EXECUTIONS.load(Ordering::SeqCst), 1);
anyhow::Ok(state)
})
.await
.unwrap();
// Outside of the root task `create_state` is no longer active, so it can be evicted.
let TestSnapshotOutcome {
had_new_data,
eviction_counts,
..
} = tt.backend().snapshot_and_evict_for_testing(&tt);
assert!(had_new_data, "snapshot should have persisted data");
assert_eq!(
eviction_counts.data_and_meta, 1,
"create_state must lose its restored Data and Meta: {eviction_counts:?}"
);
// Mutating the state invalidates the serialization of `create_state`.
turbo_tasks::run_once(tt.clone(), async move {
state.set(2);
anyhow::Ok(())
})
.await
.unwrap();
let TestSnapshotOutcome { had_new_data, .. } = tt.backend().snapshot_and_evict_for_testing(&tt);
assert!(
had_new_data,
"invalidate_serialization must persist create_state's Data"
);
let result = turbo_tasks::run_once(tt.clone(), async move {
// Restores `create_state`'s Meta from disk. With corrupted Meta the output is gone and
// the task re-executes.
create_state(1).read_strongly_consistent().await?;
assert_eq!(
CREATE_STATE_EXECUTIONS.load(Ordering::SeqCst),
1,
"create_state must not re-execute: its persisted Meta must survive \
invalidate_serialization"
);
anyhow::Ok(())
})
.await;
tt.stop_and_wait().await;
result.unwrap();
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment