Unverified Commit a0e452b9 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

turbo-tasks: batch invalidation via Invalidator::invalidate_all (#100485)

### What?

Adds a batch invalidation API and uses it wherever many tasks are
invalidated at once:

- `Invalidator::invalidate_all`, `invalidate_all_with_reason` and
`invalidate_all_with_reasons` invalidate a set of invalidators as one
batch.
  - Duplicates are fine: task ids are deduplicated once.
  - Empty input does nothing, and records no reasons.
- `TurboTasksApi::invalidate_many` / `invalidate_many_with_reasons`
(implemented by `TurboTasks`) make one `Backend::invalidate_tasks` call,
which previously had no in-tree callers.
  - Reasons are recorded once, before any work starts.
- `TurboTasksBackend::invalidate_tasks` splits large batches into chunks
and invalidates them in parallel, using the same sizing as
`parallel::for_each_owned` (via the now-public
`parallel::for_each_chunk`). Batched invalidations report the
`Invalidator` dirty cause, like single ones.
- Converted call sites:
- File watcher: all invalidators of one event batch go in one call, with
one `WatchChange` reason per path. There is now one `file change` span
per batch instead of one per invalidator. Its `paths` field lists the
changed paths, comma separated, capped at 20 with `, ... (N more
paths)`. The call still happens under the existing invalidation lock.
- `DiskFileSystem`: whole-filesystem, subtree and after-write
invalidation. These replace the per-invalidator
`parallel::for_each_owned` calls and per-invalidator reason clones.
  - `State::set`: all dependents in one call.
- A new `state_fan_out` criterion bench in
`turbo-tasks-backend/benches`.

### Why?

Every multi-task producer called `Invalidator::invalidate` once per
task. Each call went through `Backend::invalidate_task` →
`invalidate(smallvec![id])`, which runs its own operation and
scheduling. So none of these producers got the batched scheduling from
#100446 (one scheduler queue lock per batch instead of per task), or the
batched restores of paged-out tasks from #100465, which this PR builds
on.

### How?

- **Dirty priority across helper threads.** When an invalidation comes
from inside a task (e.g. `State::set` in a task), the dirtied tasks get
that task's priority as their parent priority, which the backend reads
from the task-local task state. The parallel helper threads don't have
it. So `invalidate_tasks` reads the priority once, before parallelizing,
and gives every chunk an `ExecuteContext` with that priority
(`execute_context_with_priority`;
`ExecuteContext::get_current_task_priority` returns it instead of the
task-local). Nothing else of the task state is needed, and other
contexts are unchanged.
- **Parallel work inside `with_turbo_tasks`.** `scope_bounded` wrapped
every job, including jobs it runs on the calling thread, in
`turbo_tasks_scope`, which panics while `with_turbo_tasks` holds a
borrow of the instance. Jobs now only enter the scope when the thread
doesn't already have that same instance. This lets
`State::notify_mutated` keep using `with_turbo_tasks`.
- **Unique ids at the backend.** `invalidate_many*` requires unique ids.
The `Invalidator` helpers dedupe, because the watcher and the filesystem
can yield the same task for several paths. Debug builds assert
uniqueness for the whole batch before chunking and again in `invalidate`
for each chunk; release builds pass the ids through without collecting
them.

### Benchmark

```
cargo bench -p turbo-tasks-backend --bench mod -- state_fan_out
```

- **Setup:** one `State` read by N dependent tasks. Each iteration times
`State::set` plus a strongly consistent read of the root sum, and
asserts the result.
- **Variants:** `resident`, and `evicted`, where everything is
snapshotted and evicted before each iteration. That eviction step is not
timed, and it asserts that at least N tasks were evicted.
- **Comparison:** A/B against this PR's parent on `canary`, with the
same bench copied in. Six paired runs on one 8-vCPU VM, alternating
which side runs first.
- **Revision:** measured on an earlier revision of this PR, before it
was rebased onto #100465 (batched restores in `invalidate`), and while
the chunking still lived in `TurboTasks`. The A/B wasn't re-run on the
current revision, so the numbers don't include #100465's effect on
either side.
- **Statistics:** the change is the geometric mean of the per-pair
ratios, with a 95% t-interval.

| variant / N | median base | median this PR | change (95% CI) |
|---|---|---|---|
| resident / 1 | 0.284 ms | 0.288 ms | −7.6% [−32.0%, +25.5%] |
| resident / 1,000 | 5.278 ms | 5.023 ms | −1.3% [−5.7%, +3.2%] |
| resident / 10,000 | 61.045 ms | 58.029 ms | −4.7% [−8.3%, −1.0%] |
| evicted / 1 | 0.258 ms | 0.265 ms | +3.3% [−0.2%, +7.0%] |
| evicted / 1,000 | 8.295 ms | 6.760 ms | **−19.4%** [−21.6%, −17.2%] |
| evicted / 10,000 | 91.967 ms | 69.775 ms | **−21.8%** [−25.1%, −18.4%]
|

- **Evicted:** a clear improvement when the dependents have to be
restored, from invalidating the chunks in parallel and batching their
scheduling (measured without #100465's batched restores).
- **Resident:** a small win at 10k, and no reliable difference at 1k or
for a single task.
- **Limits:** the intervals reflect repeatability on one VM, not across
machines. This is a `State` fan-out microbenchmark, not an end-to-end
filesystem measurement. The OS page cache was warm.

### Tests

- **`tests/invalidate_all.rs`:**
- a 2,000-task batch, large enough to chunk, re-executes every task
exactly once
  - duplicates in one batch
  - reasons show up in the update info
  - empty batches are no-ops
- `State::set` inside a task with 2,000 dependents, and
`parallel::for_each` inside `with_turbo_tasks` (both panic without the
`scope_bounded` change)
- a 1,999-task batch invalidated from inside a task leaves every task
dirty with the same parent priority as one task invalidated on its own
(fails without the explicit priority: tasks of the helper-thread chunks
get `Initial`). Re-executions are held back so the dirty state can be
inspected.
- **Watcher:** the `paths` span formatting, including truncation.

<!-- NEXT_JS_LLM -->


<!-- fleet d21ddb86-de88-41df-82c7-aa6a29a6165e -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 952b5afb
...@@ -7,12 +7,13 @@ pub(crate) mod cell_fan_out; ...@@ -7,12 +7,13 @@ pub(crate) mod cell_fan_out;
pub(crate) mod gc; pub(crate) mod gc;
pub(crate) mod overhead; pub(crate) mod overhead;
pub(crate) mod scope_stress; pub(crate) mod scope_stress;
pub(crate) mod state_fan_out;
pub(crate) mod stress; pub(crate) mod stress;
criterion_group!( criterion_group!(
name = turbo_tasks_backend_stress; name = turbo_tasks_backend_stress;
config = Criterion::default(); config = Criterion::default();
targets = stress::fibonacci, scope_stress::scope_stress, overhead::overhead, gc::gc, targets = stress::fibonacci, scope_stress::scope_stress, overhead::overhead, gc::gc,
cell_fan_out::cell_fan_out cell_fan_out::cell_fan_out, state_fan_out::state_fan_out
); );
criterion_main!(turbo_tasks_backend_stress); criterion_main!(turbo_tasks_backend_stress);
//! Benchmarks invalidating many tasks from outside: one `State` is read directly by `width`
//! dependent tasks, so `State::set` invalidates all of them through their invalidators
//! (`Invalidator` → `TurboTasksApi` → `Backend`).
//!
//! The graph is built once per configuration. Each iteration sets the state and reads the root
//! strongly consistently; the timing covers invalidation, scheduling and re-execution of all
//! dependents. In the `evicted` variant all tasks are snapshotted and evicted before each
//! iteration (not timed), so the invalidation also has to restore the dependents from disk.
//!
//! ```bash
//! cargo bench -p turbo-tasks-backend --bench mod -- state_fan_out
//! ```
use std::{
sync::Arc,
time::{Duration, Instant},
};
use anyhow::Result;
use criterion::{BenchmarkId, Criterion, Throughput};
use turbo_tasks::{OperationVc, ResolvedVc, State, TryJoinIterExt, TurboTasks, Vc};
use turbo_tasks_backend::{
BackendOptions, BackingStorageOptions, EvictionMode, GitVersionInfo, StorageMode,
TurboTasksBackend,
};
#[turbo_tasks::value(transparent)]
struct Input(State<u32>);
#[turbo_tasks::function(operation, root)]
fn create_input() -> Vc<Input> {
Input(State::new(0)).cell()
}
/// Reads the state directly, so it is one of the state's invalidators.
#[turbo_tasks::function]
async fn dependent(input: ResolvedVc<Input>, index: u32) -> Result<Vc<u32>> {
Ok(Vc::cell(input.await?.get().wrapping_add(index)))
}
#[turbo_tasks::function(operation, root)]
async fn fan_out_root(input: ResolvedVc<Input>, width: u32) -> Result<Vc<u32>> {
let values = (0..width)
.map(|index| dependent(*input, index))
.try_join()
.await?;
Ok(Vc::cell(
values
.iter()
.fold(0u32, |sum, value| sum.wrapping_add(**value)),
))
}
/// The sum `fan_out_root` computes when the state is `value`.
fn expected_sum(value: u32, width: u32) -> u32 {
(0..width).fold(0u32, |sum, index| {
sum.wrapping_add(value.wrapping_add(index))
})
}
fn create_tt() -> (Arc<TurboTasks<TurboTasksBackend>>, tempfile::TempDir) {
let parent = std::path::PathBuf::from(format!("{}/.cache", env!("CARGO_TARGET_TMPDIR")));
std::fs::create_dir_all(&parent).unwrap();
let dir = tempfile::Builder::new()
.prefix("state-fan-out-bench-")
.tempdir_in(&parent)
.unwrap();
let tt = TurboTasks::new(TurboTasksBackend::new(
BackendOptions {
// Snapshots are only taken explicitly (`snapshot_and_evict_for_testing`).
storage_mode: Some(StorageMode::ReadWriteOnShutdown),
eviction_mode: EvictionMode::Full,
..Default::default()
},
turbo_tasks_backend::turbo_backing_storage(
dir.path(),
&GitVersionInfo {
describe: "bench-unversioned",
dirty: false,
},
BackingStorageOptions {
is_ci: false,
is_short_session: true,
skip_compaction: true,
},
)
.unwrap()
.0,
));
(tt, dir)
}
async fn set_and_read(input_op: OperationVc<Input>, width: u32, value: u32) -> Result<Duration> {
let input = input_op.resolve().strongly_consistent().await?;
let state = input_op.read_strongly_consistent().await?;
let start = Instant::now();
state.set(value);
let sum = *fan_out_root(input, width)
.read_strongly_consistent()
.await?;
let elapsed = start.elapsed();
assert_eq!(sum, expected_sum(value, width));
Ok(elapsed)
}
pub fn state_fan_out(c: &mut Criterion) {
let mut group = c.benchmark_group("turbo_tasks_backend_state_fan_out");
group.sample_size(20);
group.measurement_time(Duration::from_secs(10));
for evict in [false, true] {
let variant = if evict { "evicted" } else { "resident" };
for width in [1u32, 1_000, 10_000] {
group.throughput(Throughput::Elements(width as u64));
group.bench_with_input(BenchmarkId::new(variant, width), &width, |b, &width| {
let rt = tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.unwrap();
// `TurboTasks::new` needs the runtime context.
let (tt, _dir) = rt.block_on(async { create_tt() });
// Build the graph once (not timed).
let input_op = rt
.block_on(tt.run_once(async move {
let input_op = create_input();
let input = input_op.resolve().strongly_consistent().await?;
let sum = *fan_out_root(input, width)
.read_strongly_consistent()
.await?;
assert_eq!(sum, expected_sum(0, width));
Ok(input_op)
}))
.unwrap();
let mut next_value = 1u32;
b.iter_custom(|iters| {
let mut total = Duration::ZERO;
for _ in 0..iters {
if evict {
let _guard = rt.enter();
let counts = tt
.backend()
.snapshot_and_evict_for_testing(&tt)
.eviction_counts;
// Every dependent must have been evicted, so the invalidation has to
// restore it.
let evicted = counts.full + counts.data_and_meta + counts.data_only;
assert!(
evicted >= width as usize,
"expected at least {width} evicted tasks, got {counts:?}"
);
}
let value = next_value;
next_value = next_value.wrapping_add(1);
total += rt
.block_on(tt.run_once(async move {
set_and_read(input_op, width, value).await
}))
.unwrap();
}
total
});
rt.block_on(async { tt.stop_and_wait().await });
});
}
}
group.finish();
}
...@@ -29,7 +29,7 @@ pub use gc::{GcPassResult, GcStats, TtlCounter}; ...@@ -29,7 +29,7 @@ pub use gc::{GcPassResult, GcStats, TtlCounter};
use hashbrown::hash_table::Entry; use hashbrown::hash_table::Entry;
use parking_lot::{Mutex, RwLock}; use parking_lot::{Mutex, RwLock};
use rustc_hash::{FxHashMap, FxHashSet, FxHasher}; use rustc_hash::{FxHashMap, FxHashSet, FxHasher};
use smallvec::{SmallVec, smallvec}; use smallvec::SmallVec;
use tokio::time::{Duration, Instant}; use tokio::time::{Duration, Instant};
use tracing::{Span, field::display, trace_span}; use tracing::{Span, field::display, trace_span};
use turbo_bincode::TurboBincodeBuffer; use turbo_bincode::TurboBincodeBuffer;
...@@ -47,6 +47,7 @@ use turbo_tasks::{ ...@@ -47,6 +47,7 @@ use turbo_tasks::{
event::{Event, EventDescription, EventListener}, event::{Event, EventDescription, EventListener},
macro_helpers::NativeFunction, macro_helpers::NativeFunction,
message_queue::{DiagnosticEvent, Severity, TimingEvent, TraceEvent}, message_queue::{DiagnosticEvent, Severity, TimingEvent, TraceEvent},
parallel,
registry::get_value_type, registry::get_value_type,
scope_bounded::scope_bounded, scope_bounded::scope_bounded,
task_statistics::TaskStatisticsApi, task_statistics::TaskStatisticsApi,
...@@ -390,6 +391,17 @@ impl TurboTasksBackend { ...@@ -390,6 +391,17 @@ impl TurboTasksBackend {
ExecuteContext::new(self, turbo_tasks) ExecuteContext::new(self, turbo_tasks)
} }
/// Like [`TurboTasksBackend::execute_context`], but with an explicit current task priority
/// instead of the one from the task-local task state. For work that runs on another thread
/// than the calling task.
fn execute_context_with_priority<'a>(
&'a self,
turbo_tasks: &'a TurboTasks<TurboTasksBackend>,
current_task_priority: TaskPriority,
) -> ExecuteContext<'a> {
ExecuteContext::new(self, turbo_tasks).with_current_task_priority(current_task_priority)
}
/// Like [`TurboTasksBackend::execute_context`], but refuses to hand out a context once /// Like [`TurboTasksBackend::execute_context`], but refuses to hand out a context once
/// shutdown has begun, and blocks shutdown for as long as the returned context is alive. /// shutdown has begun, and blocks shutdown for as long as the returned context is alive.
/// ///
...@@ -465,6 +477,24 @@ impl TurboTasksBackend { ...@@ -465,6 +477,24 @@ impl TurboTasksBackend {
self.storage.resident_task_count_for_testing() self.storage.resident_task_count_for_testing()
} }
/// The parent priority a dirty task was made dirty with, or `None` if it doesn't exist or is
/// not dirty. Test-only hook for verifying invalidation priorities.
#[doc(hidden)]
pub fn dirty_parent_priority_for_testing(
&self,
task: TaskId,
turbo_tasks: &TurboTasks<TurboTasksBackend>,
) -> Option<TaskPriority> {
let mut ctx = self.execute_context(turbo_tasks);
let task = ctx.try_task(task, TaskDataCategory::All)?;
match task.get_dirty() {
Some(Dirtyness::Dirty {
parent_priority, ..
}) => Some(*parent_priority),
_ => None,
}
}
/// The persistent `parent_count` of a resident task (0 if absent or not resident). Test-only /// The persistent `parent_count` of a resident task (0 if absent or not resident). Test-only
/// hook for verifying incremental refcount maintenance. /// hook for verifying incremental refcount maintenance.
#[doc(hidden)] #[doc(hidden)]
...@@ -1846,7 +1876,7 @@ impl TurboTasksBackend { ...@@ -1846,7 +1876,7 @@ impl TurboTasksBackend {
panic!("Dependency tracking is disabled so invalidation is not allowed"); panic!("Dependency tracking is disabled so invalidation is not allowed");
} }
invalidate( invalidate(
smallvec![task_id], [task_id],
#[cfg(feature = "task_dirty_cause")] #[cfg(feature = "task_dirty_cause")]
TaskDirtyCause::Invalidator, TaskDirtyCause::Invalidator,
self.execute_context(turbo_tasks), self.execute_context(turbo_tasks),
...@@ -1857,12 +1887,25 @@ impl TurboTasksBackend { ...@@ -1857,12 +1887,25 @@ impl TurboTasksBackend {
if !self.should_track_dependencies() { if !self.should_track_dependencies() {
panic!("Dependency tracking is disabled so invalidation is not allowed"); panic!("Dependency tracking is disabled so invalidation is not allowed");
} }
// `invalidate` checks each chunk for duplicates, but a duplicate split across two chunks
// would only be seen here.
debug_assert!(
tasks.len() <= 1 || tasks.iter().collect::<FxHashSet<_>>().len() == tasks.len(),
"invalidate_tasks() must not be called with duplicate task ids: {tasks:?}"
);
// Large batches are invalidated in parallel chunks. The chunks may run on other threads,
// which don't have the calling task's state, so its priority (the parent priority of the
// dirty tasks) is read once here and passed to every chunk.
let current_task_priority = turbo_tasks.get_current_task_priority();
parallel::for_each_chunk(tasks, |chunk| {
invalidate( invalidate(
tasks.iter().copied().collect(), chunk.iter().copied(),
// Called by `TurboTasksApi::invalidate_many*`, i.e. batched `Invalidator`s.
#[cfg(feature = "task_dirty_cause")] #[cfg(feature = "task_dirty_cause")]
TaskDirtyCause::Unknown, TaskDirtyCause::Invalidator,
self.execute_context(turbo_tasks), self.execute_context_with_priority(turbo_tasks, current_task_priority),
); );
});
} }
fn invalidate_tasks_set( fn invalidate_tasks_set(
...@@ -1874,7 +1917,7 @@ impl TurboTasksBackend { ...@@ -1874,7 +1917,7 @@ impl TurboTasksBackend {
panic!("Dependency tracking is disabled so invalidation is not allowed"); panic!("Dependency tracking is disabled so invalidation is not allowed");
} }
invalidate( invalidate(
tasks.iter().copied().collect(), tasks.iter().copied(),
#[cfg(feature = "task_dirty_cause")] #[cfg(feature = "task_dirty_cause")]
TaskDirtyCause::Unknown, TaskDirtyCause::Unknown,
self.execute_context(turbo_tasks), self.execute_context(turbo_tasks),
......
...@@ -19,19 +19,26 @@ use crate::{ ...@@ -19,19 +19,26 @@ use crate::{
/// Invalidates `task_ids`. Callers must not pass duplicate ids (the bulk task access expects /// Invalidates `task_ids`. Callers must not pass duplicate ids (the bulk task access expects
/// unique ids, and deduplicating here would cost every invalidation). /// unique ids, and deduplicating here would cost every invalidation).
pub fn invalidate( pub fn invalidate(
task_ids: SmallVec<[TaskId; 4]>, task_ids: impl IntoIterator<Item = TaskId>,
#[cfg(feature = "task_dirty_cause")] cause: TaskDirtyCause, #[cfg(feature = "task_dirty_cause")] cause: TaskDirtyCause,
mut ctx: ExecuteContext<'_>, mut ctx: ExecuteContext<'_>,
) { ) {
// Debug builds collect the ids to check they are unique. Release builds stream them.
#[cfg(debug_assertions)]
let task_ids = {
let task_ids: Vec<TaskId> = task_ids.into_iter().collect();
debug_assert!( debug_assert!(
task_ids.len() <= 1 task_ids.len() <= 1
|| task_ids.iter().collect::<rustc_hash::FxHashSet<_>>().len() == task_ids.len(), || task_ids.iter().collect::<rustc_hash::FxHashSet<_>>().len() == task_ids.len(),
"invalidate() must not be called with duplicate task ids: {task_ids:?}" "invalidate() must not be called with duplicate task ids: {task_ids:?}"
); );
task_ids
};
let task_ids = task_ids.into_iter();
let mut queue = AggregationUpdateQueue::new(); let mut queue = AggregationUpdateQueue::new();
// The tasks to schedule are collected and handed to the scheduler as one batch, which takes // The tasks to schedule are collected and handed to the scheduler as one batch, which takes
// the scheduler's queue lock once instead of once per task. // the scheduler's queue lock once instead of once per task.
let mut to_schedule: SmallVec<[_; 4]> = SmallVec::with_capacity(task_ids.len()); let mut to_schedule: SmallVec<[_; 4]> = SmallVec::with_capacity(task_ids.size_hint().0);
// Tasks that no longer exist are skipped, there is nothing to invalidate. Tasks that are not // Tasks that no longer exist are skipped, there is nothing to invalidate. Tasks that are not
// resident are restored from persistent storage in batches. // resident are restored from persistent storage in batches.
ctx.for_each_try_task_all(task_ids, "invalidate", |mut task, ctx| { ctx.for_each_try_task_all(task_ids, "invalidate", |mut task, ctx| {
......
...@@ -111,6 +111,10 @@ pub struct ExecuteContext<'e> { ...@@ -111,6 +111,10 @@ pub struct ExecuteContext<'e> {
/// teardown in `stop()` waits for this context to be dropped. /// teardown in `stop()` waits for this context to be dropped.
_shutdown_guard: Option<RwLockReadGuard<'e, bool>>, _shutdown_guard: Option<RwLockReadGuard<'e, bool>>,
task_lock_counter: TaskLockCounter, task_lock_counter: TaskLockCounter,
/// Overrides the current task priority, which is otherwise read from the task-local task
/// state. Set for work that runs off the calling task's thread (see
/// [`ExecuteContext::with_current_task_priority`]).
current_task_priority: Option<TaskPriority>,
} }
impl<'e> ExecuteContext<'e> { impl<'e> ExecuteContext<'e> {
...@@ -125,9 +129,18 @@ impl<'e> ExecuteContext<'e> { ...@@ -125,9 +129,18 @@ impl<'e> ExecuteContext<'e> {
_operation_guard: backend.start_operation(), _operation_guard: backend.start_operation(),
_shutdown_guard: None, _shutdown_guard: None,
task_lock_counter: TaskLockCounter::new(), task_lock_counter: TaskLockCounter::new(),
current_task_priority: None,
} }
} }
/// Makes [`ExecuteContext::get_current_task_priority`] return `priority` instead of reading
/// it from the task-local task state. Used when the work runs on another thread than the
/// calling task, where that task-local is not available.
pub(super) fn with_current_task_priority(mut self, priority: TaskPriority) -> Self {
self.current_task_priority = Some(priority);
self
}
/// Like [`ExecuteContext::new`], but owns the shutdown read guard that /// Like [`ExecuteContext::new`], but owns the shutdown read guard that
/// [`TurboTasksBackend::try_execute_context`] acquired, keeping `stop()` out until this /// [`TurboTasksBackend::try_execute_context`] acquired, keeping `stop()` out until this
/// context is dropped. /// context is dropped.
...@@ -143,6 +156,7 @@ impl<'e> ExecuteContext<'e> { ...@@ -143,6 +156,7 @@ impl<'e> ExecuteContext<'e> {
_operation_guard: backend.start_operation(), _operation_guard: backend.start_operation(),
_shutdown_guard: Some(shutdown_guard), _shutdown_guard: Some(shutdown_guard),
task_lock_counter: TaskLockCounter::new(), task_lock_counter: TaskLockCounter::new(),
current_task_priority: None,
} }
} }
...@@ -165,6 +179,7 @@ impl<'e> ExecuteContext<'e> { ...@@ -165,6 +179,7 @@ impl<'e> ExecuteContext<'e> {
_operation_guard: None, _operation_guard: None,
_shutdown_guard: None, _shutdown_guard: None,
task_lock_counter: TaskLockCounter::new(), task_lock_counter: TaskLockCounter::new(),
current_task_priority: None,
} }
} }
...@@ -940,6 +955,7 @@ impl<'e> ExecuteContext<'e> { ...@@ -940,6 +955,7 @@ impl<'e> ExecuteContext<'e> {
ChildExecuteContext { ChildExecuteContext {
backend: self.backend, backend: self.backend,
turbo_tasks: self.turbo_tasks, turbo_tasks: self.turbo_tasks,
current_task_priority: self.current_task_priority,
} }
} }
...@@ -1153,7 +1169,8 @@ impl<'e> ExecuteContext<'e> { ...@@ -1153,7 +1169,8 @@ impl<'e> ExecuteContext<'e> {
} }
pub fn get_current_task_priority(&self) -> TaskPriority { pub fn get_current_task_priority(&self) -> TaskPriority {
self.turbo_tasks.get_current_task_priority() self.current_task_priority
.unwrap_or_else(|| self.turbo_tasks.get_current_task_priority())
} }
/// Record `task` as a GC candidate **if it is in fact collectible**. /// Record `task` as a GC candidate **if it is in fact collectible**.
...@@ -1228,6 +1245,7 @@ impl<'e> ExecuteContext<'e> { ...@@ -1228,6 +1245,7 @@ impl<'e> ExecuteContext<'e> {
pub struct ChildExecuteContext<'e> { pub struct ChildExecuteContext<'e> {
backend: &'e TurboTasksBackend, backend: &'e TurboTasksBackend,
turbo_tasks: &'e TurboTasks<TurboTasksBackend>, turbo_tasks: &'e TurboTasks<TurboTasksBackend>,
current_task_priority: Option<TaskPriority>,
} }
impl<'e> ChildExecuteContext<'e> { impl<'e> ChildExecuteContext<'e> {
...@@ -1241,6 +1259,7 @@ impl<'e> ChildExecuteContext<'e> { ...@@ -1241,6 +1259,7 @@ impl<'e> ChildExecuteContext<'e> {
// already waits for, so it needs no shutdown guard of its own. // already waits for, so it needs no shutdown guard of its own.
_shutdown_guard: None, _shutdown_guard: None,
task_lock_counter: TaskLockCounter::new(), task_lock_counter: TaskLockCounter::new(),
current_task_priority: self.current_task_priority,
} }
} }
} }
......
This diff is collapsed.
...@@ -25,9 +25,9 @@ use tokio::{ ...@@ -25,9 +25,9 @@ use tokio::{
use tracing::Instrument; use tracing::Instrument;
use turbo_rcstr::{RcStr, rcstr}; use turbo_rcstr::{RcStr, rcstr};
use turbo_tasks::{ use turbo_tasks::{
CapturedEffect, Effect, EffectExt, EffectStateStorage, InvalidationReason, NonLocalValue, CapturedEffect, Effect, EffectExt, EffectStateStorage, InvalidationReason, Invalidator,
OperationVc, ReadRef, ResolvedVc, TurboTasksApi, ValueToString, Vc, debug::ValueDebugFormat, NonLocalValue, OperationVc, ReadRef, ResolvedVc, TurboTasksApi, ValueToString, Vc,
parallel, turbo_tasks_weak, turbobail, debug::ValueDebugFormat, turbo_tasks_weak, turbobail,
}; };
use turbo_tasks_hash::{hash_xxh3_hash64, hash_xxh3_hash128}; use turbo_tasks_hash::{hash_xxh3_hash64, hash_xxh3_hash128};
use turbo_unix_path::{normalize_path, sys_to_unix, unix_to_sys}; use turbo_unix_path::{normalize_path, sys_to_unix, unix_to_sys};
...@@ -309,9 +309,7 @@ impl DiskFileSystemInner { ...@@ -309,9 +309,7 @@ impl DiskFileSystemInner {
let reason = Write { let reason = Write {
path: full_path.to_string_lossy().into_owned(), path: full_path.to_string_lossy().into_owned(),
}; };
for invalidator in invalidators { Invalidator::invalidate_all_with_reason(invalidators, &*turbo_tasks, reason);
invalidator.invalidate_with_reason(&*turbo_tasks, reason.clone());
}
} }
} }
...@@ -345,19 +343,16 @@ impl DiskFileSystemInner { ...@@ -345,19 +343,16 @@ impl DiskFileSystemInner {
let invalidators = invalidator_map let invalidators = invalidator_map
.into_iter() .into_iter()
.chain(dir_invalidator_map) .chain(dir_invalidator_map)
.flat_map(|(_, invalidators)| invalidators.into_iter()) .flat_map(|(_, invalidators)| invalidators.into_iter());
.collect::<Vec<_>>(); Invalidator::invalidate_all(invalidators, &*turbo_tasks);
parallel::for_each_owned(invalidators, |invalidator| {
invalidator.invalidate(&*turbo_tasks)
});
} }
/// Invalidates every tracked file in the filesystem. /// Invalidates every tracked file in the filesystem.
/// ///
/// Calls the given `reason` closure to find the [`InvalidationReason`]. /// Calls the given `reason` closure to find the [`InvalidationReason`].
pub(crate) fn invalidate_with_reason<R: InvalidationReason + Clone>( pub(crate) fn invalidate_with_reason<R: InvalidationReason>(
&self, &self,
reason: impl Fn(&Path) -> R + Sync, reason: impl Fn(&Path) -> R,
) { ) {
let _span = tracing::info_span!("invalidate filesystem", name = &*self.root).entered(); let _span = tracing::info_span!("invalidate filesystem", name = &*self.root).entered();
let Some(turbo_tasks) = self.turbo_tasks.upgrade() else { let Some(turbo_tasks) = self.turbo_tasks.upgrade() else {
...@@ -367,28 +362,32 @@ impl DiskFileSystemInner { ...@@ -367,28 +362,32 @@ impl DiskFileSystemInner {
let invalidator_map = take(&mut *self.invalidator_map.lock().unwrap()); let invalidator_map = take(&mut *self.invalidator_map.lock().unwrap());
let dir_invalidator_map = take(&mut *self.dir_invalidator_map.lock().unwrap()); let dir_invalidator_map = take(&mut *self.dir_invalidator_map.lock().unwrap());
let invalidators = invalidator_map let path_invalidator_counts = || {
.into_iter() invalidator_map
.chain(dir_invalidator_map) .values()
.flat_map(|(path, invalidators)| { .chain(dir_invalidator_map.values())
let reason_for_path = reason(&path); .map(|path_invalidators| path_invalidators.len())
invalidators .filter(|&count| count > 0)
.into_iter() };
.map(move |i| (reason_for_path.clone(), i)) let mut invalidators = Vec::with_capacity(path_invalidator_counts().sum());
}) let mut reasons = Vec::with_capacity(path_invalidator_counts().count());
.collect::<Vec<_>>(); for (path, path_invalidators) in invalidator_map.into_iter().chain(dir_invalidator_map) {
parallel::for_each_owned(invalidators, |(reason, invalidator)| { if path_invalidators.is_empty() {
invalidator.invalidate_with_reason(&*turbo_tasks, reason) continue;
}); }
reasons.push(reason(&path));
invalidators.extend(path_invalidators);
}
Invalidator::invalidate_all_with_reasons(invalidators, &*turbo_tasks, reasons);
} }
/// Invalidates tracked files/directories for `paths` and their children. /// Invalidates tracked files/directories for `paths` and their children.
/// Also invalidates tracked directory reads for all parent directories to /// Also invalidates tracked directory reads for all parent directories to
/// account for file creations/deletions under the deferred subtree. /// account for file creations/deletions under the deferred subtree.
fn invalidate_path_and_children_with_reason<R: InvalidationReason + Clone>( fn invalidate_path_and_children_with_reason<R: InvalidationReason>(
&self, &self,
paths: impl IntoIterator<Item = PathBuf>, paths: impl IntoIterator<Item = PathBuf>,
reason: impl Fn(&Path) -> R + Sync, reason: impl Fn(&Path) -> R,
) { ) {
let _span = let _span =
tracing::info_span!("invalidate filesystem paths", name = &*self.root).entered(); tracing::info_span!("invalidate filesystem paths", name = &*self.root).entered();
...@@ -397,9 +396,18 @@ impl DiskFileSystemInner { ...@@ -397,9 +396,18 @@ impl DiskFileSystemInner {
}; };
let _guard = self.tokio_handle.enter(); let _guard = self.tokio_handle.enter();
let paths = paths.into_iter();
let path_count = paths.size_hint().0;
let mut invalidator_map = self.invalidator_map.lock().unwrap(); let mut invalidator_map = self.invalidator_map.lock().unwrap();
let mut dir_invalidator_map = self.dir_invalidator_map.lock().unwrap(); let mut dir_invalidator_map = self.dir_invalidator_map.lock().unwrap();
let mut invalidators = Vec::new(); let mut invalidators = Vec::with_capacity(path_count);
let mut reasons = Vec::with_capacity(path_count);
let mut add = |path: &Path, path_invalidators: FxHashSet<Invalidator>| {
if !path_invalidators.is_empty() {
reasons.push(reason(path));
invalidators.extend(path_invalidators);
}
};
let mut parent_dirs_to_invalidate = FxHashSet::default(); let mut parent_dirs_to_invalidate = FxHashSet::default();
for path in paths { for path in paths {
...@@ -412,43 +420,26 @@ impl DiskFileSystemInner { ...@@ -412,43 +420,26 @@ impl DiskFileSystemInner {
for (invalidated_path, path_invalidators) in for (invalidated_path, path_invalidators) in
invalidator_map.extract_path_with_children(&path) invalidator_map.extract_path_with_children(&path)
{ {
let reason_for_path = reason(&invalidated_path); add(&invalidated_path, path_invalidators);
invalidators.extend(
path_invalidators
.into_iter()
.map(|invalidator| (reason_for_path.clone(), invalidator)),
);
} }
for (invalidated_path, path_invalidators) in for (invalidated_path, path_invalidators) in
dir_invalidator_map.extract_path_with_children(&path) dir_invalidator_map.extract_path_with_children(&path)
{ {
let reason_for_path = reason(&invalidated_path); add(&invalidated_path, path_invalidators);
invalidators.extend(
path_invalidators
.into_iter()
.map(|invalidator| (reason_for_path.clone(), invalidator)),
);
} }
} }
for path in parent_dirs_to_invalidate { for path in parent_dirs_to_invalidate {
if let Some(path_invalidators) = dir_invalidator_map.remove(path.as_path()) { if let Some(path_invalidators) = dir_invalidator_map.remove(path.as_path()) {
let reason_for_path = reason(&path); add(&path, path_invalidators);
invalidators.extend(
path_invalidators
.into_iter()
.map(|invalidator| (reason_for_path.clone(), invalidator)),
);
} }
} }
drop(invalidator_map); drop(invalidator_map);
drop(dir_invalidator_map); drop(dir_invalidator_map);
parallel::for_each_owned(invalidators, |(reason, invalidator)| { Invalidator::invalidate_all_with_reasons(invalidators, &*turbo_tasks, reasons);
invalidator.invalidate_with_reason(&*turbo_tasks, reason)
});
} }
} }
......
...@@ -754,18 +754,26 @@ impl DiskWatcher { ...@@ -754,18 +754,26 @@ impl DiskWatcher {
let _guard = fs.tokio_handle().enter(); let _guard = fs.tokio_handle().enter();
let _lock = fs.invalidation_lock().blocking_write(); let _lock = fs.invalidation_lock().blocking_write();
// All invalidators of the batch are invalidated together, with one reason per path.
let mut invalidators = Vec::new();
// The changed paths, for the span and (when enabled) the invalidation reasons.
let mut changed_paths: Vec<PathBuf> = Vec::new();
batch.execute( batch.execute(
fs.invalidator_map(), fs.invalidator_map(),
fs.dir_invalidator_map(), fs.dir_invalidator_map(),
|invalidation_reason_path, invalidator| { |invalidation_reason_path, invalidator| {
invalidators.push(invalidator);
if changed_paths.last().map(|p| p.as_path()) != Some(invalidation_reason_path) {
changed_paths.push(invalidation_reason_path.to_path_buf());
}
},
);
invalidate( invalidate(
&*fs, &*fs,
&*turbo_tasks, &*turbo_tasks,
report_invalidation_reason, report_invalidation_reason,
invalidation_reason_path, &changed_paths,
invalidator, invalidators,
)
},
); );
} }
} }
...@@ -1097,7 +1105,7 @@ impl BatchedInvalidations { ...@@ -1097,7 +1105,7 @@ impl BatchedInvalidations {
&mut self, &mut self,
invalidator_map: &InvalidatorMap, invalidator_map: &InvalidatorMap,
dir_invalidator_map: &InvalidatorMap, dir_invalidator_map: &InvalidatorMap,
invalidate: impl Fn(&Path, Invalidator), mut invalidate: impl FnMut(&Path, Invalidator),
) { ) {
for (map, exact_flag, recursive_flag) in [ for (map, exact_flag, recursive_flag) in [
( (
...@@ -1132,28 +1140,57 @@ impl BatchedInvalidations { ...@@ -1132,28 +1140,57 @@ impl BatchedInvalidations {
} }
} }
/// The maximum number of paths listed in the `file change` span.
const MAX_SPAN_PATHS: usize = 20;
/// Formats paths for the `file change` span: comma separated, at most [`MAX_SPAN_PATHS`], with
/// `, ... (N more paths)` appended when there are more.
struct SpanPaths<'a>(&'a [PathBuf]);
impl std::fmt::Display for SpanPaths<'_> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
for (index, path) in self.0.iter().take(MAX_SPAN_PATHS).enumerate() {
if index > 0 {
f.write_str(", ")?;
}
write!(f, "{}", path.display())?;
}
if self.0.len() > MAX_SPAN_PATHS {
write!(f, ", ... ({} more paths)", self.0.len() - MAX_SPAN_PATHS)?;
}
Ok(())
}
}
#[instrument( #[instrument(
parent = None, parent = None,
level = "info", level = "info",
name = "file change", name = "file change",
skip_all, skip_all,
fields(name = %invalidation_reason_path.display()) fields(
paths = %SpanPaths(invalidation_reason_paths),
invalidators = invalidators.len(),
)
)] )]
fn invalidate( fn invalidate(
inner: &impl DiskFileSystemWatcherApi, inner: &impl DiskFileSystemWatcherApi,
turbo_tasks: &dyn TurboTasksApi, turbo_tasks: &dyn TurboTasksApi,
report_invalidation_reason: bool, report_invalidation_reason: bool,
invalidation_reason_path: &Path, invalidation_reason_paths: &[PathBuf],
invalidator: Invalidator, invalidators: Vec<Invalidator>,
) { ) {
if report_invalidation_reason if invalidators.is_empty() {
&& let Some(path) =
format_absolute_fs_path(invalidation_reason_path, inner.name(), inner.root_path())
{
invalidator.invalidate_with_reason(turbo_tasks, WatchChange { path });
return; return;
} }
invalidator.invalidate(turbo_tasks); if report_invalidation_reason {
let reasons = invalidation_reason_paths
.iter()
.filter_map(|path| format_absolute_fs_path(path, inner.name(), inner.root_path()))
.map(|path| WatchChange { path });
Invalidator::invalidate_all_with_reasons(invalidators, turbo_tasks, reasons);
} else {
Invalidator::invalidate_all(invalidators, turbo_tasks);
}
} }
/// Invalidation was caused by a watcher rescan event. This will likely invalidate *every* watched /// Invalidation was caused by a watcher rescan event. This will likely invalidate *every* watched
...@@ -1221,6 +1258,30 @@ mod tests { ...@@ -1221,6 +1258,30 @@ mod tests {
use super::*; use super::*;
use crate::watcher::mock_fs_api::MockFileSystem; use crate::watcher::mock_fs_api::MockFileSystem;
#[test]
fn span_paths_are_truncated() {
let paths = |n: usize| {
(0..n)
.map(|i| PathBuf::from(format!("p{i}")))
.collect::<Vec<_>>()
};
assert_eq!(SpanPaths(&paths(0)).to_string(), "");
assert_eq!(SpanPaths(&paths(2)).to_string(), "p0, p1");
let twenty = (0..20)
.map(|i| format!("p{i}"))
.collect::<Vec<_>>()
.join(", ");
assert_eq!(SpanPaths(&paths(20)).to_string(), twenty);
assert_eq!(
SpanPaths(&paths(21)).to_string(),
format!("{twenty}, ... (1 more paths)")
);
assert_eq!(
SpanPaths(&paths(25)).to_string(),
format!("{twenty}, ... (5 more paths)")
);
}
#[cfg(not(miri))] #[cfg(not(miri))]
#[tokio::test] #[tokio::test]
async fn batches_only_tracked_paths() { async fn batches_only_tracked_paths() {
......
...@@ -2,6 +2,7 @@ use std::{fmt::Display, mem::replace, sync::Arc}; ...@@ -2,6 +2,7 @@ use std::{fmt::Display, mem::replace, sync::Arc};
use bincode::{Decode, Encode}; use bincode::{Decode, Encode};
use indexmap::map::Entry; use indexmap::map::Entry;
use rustc_hash::FxHashSet;
use turbo_dyn_eq_hash::{ use turbo_dyn_eq_hash::{
DynEq, DynHash, impl_eq_for_dyn, impl_hash_for_dyn, impl_partial_eq_for_dyn, DynEq, DynHash, impl_eq_for_dyn, impl_hash_for_dyn, impl_partial_eq_for_dyn,
}; };
...@@ -46,6 +47,59 @@ impl Invalidator { ...@@ -46,6 +47,59 @@ impl Invalidator {
(Arc::new(reason) as Arc<dyn InvalidationReason>).into(), (Arc::new(reason) as Arc<dyn InvalidationReason>).into(),
); );
} }
/// Invalidates all `invalidators` together, as one batch. Duplicates are fine, every task is
/// invalidated once. Large batches may be split into chunks that are invalidated in parallel.
pub fn invalidate_all(
invalidators: impl IntoIterator<Item = Invalidator>,
turbo_tasks: &dyn TurboTasksApi,
) {
let tasks = Self::unique_tasks(invalidators);
if !tasks.is_empty() {
turbo_tasks.invalidate_many(&tasks);
}
}
/// Like [`Invalidator::invalidate_all`], recording `reason` for the batch.
pub fn invalidate_all_with_reason<T: InvalidationReason>(
invalidators: impl IntoIterator<Item = Invalidator>,
turbo_tasks: &dyn TurboTasksApi,
reason: T,
) {
Self::invalidate_all_with_reasons(invalidators, turbo_tasks, [reason]);
}
/// Like [`Invalidator::invalidate_all`], recording all `reasons` for the batch. The reasons
/// are not tied to individual tasks, they are only reported in the aggregated update info.
pub fn invalidate_all_with_reasons<T: InvalidationReason>(
invalidators: impl IntoIterator<Item = Invalidator>,
turbo_tasks: &dyn TurboTasksApi,
reasons: impl IntoIterator<Item = T>,
) {
let tasks = Self::unique_tasks(invalidators);
if tasks.is_empty() {
return;
}
let reasons = reasons
.into_iter()
.map(|reason| (Arc::new(reason) as Arc<dyn InvalidationReason>).into())
.collect();
turbo_tasks.invalidate_many_with_reasons(&tasks, reasons);
}
/// The backend requires unique task ids in a batch.
fn unique_tasks(invalidators: impl IntoIterator<Item = Invalidator>) -> Vec<TaskId> {
let invalidators = invalidators.into_iter();
let capacity = invalidators.size_hint().0;
let mut seen = FxHashSet::with_capacity_and_hasher(capacity, Default::default());
let mut tasks = Vec::with_capacity(capacity);
for invalidator in invalidators {
if seen.insert(invalidator.task) {
tasks.push(invalidator.task);
}
}
tasks
}
} }
unsafe impl OperationValue for Invalidator {} unsafe impl OperationValue for Invalidator {}
......
...@@ -125,6 +125,15 @@ pub trait TurboTasksCallApi: Sync + Send { ...@@ -125,6 +125,15 @@ pub trait TurboTasksCallApi: Sync + Send {
pub trait TurboTasksApi: TurboTasksCallApi + Sync + Send { pub trait TurboTasksApi: TurboTasksCallApi + Sync + Send {
fn invalidate(&self, task: TaskId); fn invalidate(&self, task: TaskId);
fn invalidate_with_reason(&self, task: TaskId, reason: StaticOrArc<dyn InvalidationReason>); fn invalidate_with_reason(&self, task: TaskId, reason: StaticOrArc<dyn InvalidationReason>);
/// Invalidates all `tasks` as one batch. `tasks` must not contain duplicates. The backend may
/// split large batches into chunks that are invalidated in parallel.
fn invalidate_many(&self, tasks: &[TaskId]);
/// Like [`TurboTasksApi::invalidate_many`], recording all `reasons` once for the whole batch.
fn invalidate_many_with_reasons(
&self,
tasks: &[TaskId],
reasons: Vec<StaticOrArc<dyn InvalidationReason>>,
);
fn invalidate_serialization(&self, task: TaskId); fn invalidate_serialization(&self, task: TaskId);
...@@ -1873,6 +1882,33 @@ impl<B: Backend + 'static> TurboTasksApi for TurboTasks<B> { ...@@ -1873,6 +1882,33 @@ impl<B: Backend + 'static> TurboTasksApi for TurboTasks<B> {
self.backend.invalidate_task(task, self); self.backend.invalidate_task(task, self);
} }
#[instrument(level = "info", skip_all, name = "invalidate many", fields(count = tasks.len()))]
fn invalidate_many(&self, tasks: &[TaskId]) {
if tasks.is_empty() {
return;
}
self.backend.invalidate_tasks(tasks, self);
}
#[instrument(level = "info", skip_all, name = "invalidate many", fields(count = tasks.len()))]
fn invalidate_many_with_reasons(
&self,
tasks: &[TaskId],
reasons: Vec<StaticOrArc<dyn InvalidationReason>>,
) {
if tasks.is_empty() {
return;
}
// Recorded once, before any task is invalidated.
{
let (_, reason_set) = &mut *self.aggregated_update.lock().unwrap();
for reason in reasons {
reason_set.insert(reason);
}
}
self.backend.invalidate_tasks(tasks, self);
}
fn invalidate_serialization(&self, task: TaskId) { fn invalidate_serialization(&self, task: TaskId) {
self.backend.invalidate_serialization(task, self); self.backend.invalidate_serialization(task, self);
} }
...@@ -2252,6 +2288,20 @@ pub fn turbo_tasks_scope<T>(tt: Arc<dyn TurboTasksApi>, f: impl FnOnce() -> T) - ...@@ -2252,6 +2288,20 @@ pub fn turbo_tasks_scope<T>(tt: Arc<dyn TurboTasksApi>, f: impl FnOnce() -> T) -
TURBO_TASKS.sync_scope(tt, f) TURBO_TASKS.sync_scope(tt, f)
} }
/// Like [`turbo_tasks_scope`], but runs `f` directly when `tt` is already the current turbo-tasks
/// instance. Entering the scope again would be redundant, and panics while the current one is
/// borrowed (e.g. inside [`with_turbo_tasks`]).
pub(crate) fn ensure_turbo_tasks_scope<T>(tt: Arc<dyn TurboTasksApi>, f: impl FnOnce() -> T) -> T {
let is_current = TURBO_TASKS
.try_with(|current| std::ptr::addr_eq(Arc::as_ptr(current), Arc::as_ptr(&tt)))
.unwrap_or(false);
if is_current {
f()
} else {
TURBO_TASKS.sync_scope(tt, f)
}
}
pub fn turbo_tasks_future_scope<T>( pub fn turbo_tasks_future_scope<T>(
tt: Arc<dyn TurboTasksApi>, tt: Arc<dyn TurboTasksApi>,
f: impl Future<Output = T>, f: impl Future<Output = T>,
......
...@@ -87,6 +87,33 @@ where ...@@ -87,6 +87,33 @@ where
}); });
} }
/// Like [`for_each`], but calls `f` once per chunk instead of once per item. A slice too small to
/// be chunked is passed to `f` as a whole on the calling thread, and an empty slice doesn't call
/// `f`. Uses the same chunk sizing as [`for_each`].
pub fn for_each_chunk<'l, T, F>(items: &'l [T], f: F)
where
T: Sync,
F: Fn(&'l [T]) + Send + Sync,
{
if items.is_empty() {
return;
}
let Some(Chunked {
chunk_size,
chunk_count,
}) = get_chunked(items.len())
else {
f(items);
return;
};
let f = &f;
let _results = scope_bounded(chunk_count, |scope| {
for chunk in items.chunks(chunk_size) {
scope.spawn(move || f(chunk))
}
});
}
pub fn for_each_owned<T>(items: Vec<T>, f: impl Fn(T) + Send + Sync) pub fn for_each_owned<T>(items: Vec<T>, f: impl Fn(T) + Send + Sync)
where where
T: Send + Sync, T: Send + Sync,
......
...@@ -26,7 +26,10 @@ use parking_lot::Mutex; ...@@ -26,7 +26,10 @@ use parking_lot::Mutex;
use tokio::{runtime::Handle, task::block_in_place}; use tokio::{runtime::Handle, task::block_in_place};
use tracing::{Span, info_span}; use tracing::{Span, info_span};
use crate::{TurboTasksApi, manager::try_turbo_tasks, turbo_tasks_scope}; use crate::{
TurboTasksApi,
manager::{ensure_turbo_tasks_scope, try_turbo_tasks},
};
/// A job placed on the work queue: its result-slot index and the closure to run. /// A job placed on the work queue: its result-slot index and the closure to run.
type WorkQueueJob = (usize, Box<dyn FnOnce() + Send + 'static>); type WorkQueueJob = (usize, Box<dyn FnOnce() + Send + 'static>);
...@@ -171,8 +174,10 @@ impl<'scope, 'env: 'scope, R: Send + 'env> Scope<'scope, 'env, R> { ...@@ -171,8 +174,10 @@ impl<'scope, 'env: 'scope, R: Send + 'env> Scope<'scope, 'env, R> {
let f: Box<dyn FnOnce() + Send + 'scope> = Box::new(|| { let f: Box<dyn FnOnce() + Send + 'scope> = Box::new(|| {
let result = { let result = {
if let Some(turbo_tasks) = turbo_tasks { if let Some(turbo_tasks) = turbo_tasks {
// Ensure that the turbo tasks context is maintained across the job. // Ensure that the turbo tasks context is maintained across the job. Jobs run
turbo_tasks_scope(turbo_tasks, f) // on the calling thread already have it (possibly borrowed, e.g. inside
// `with_turbo_tasks`), so it's only entered when needed.
ensure_turbo_tasks_scope(turbo_tasks, f)
} else { } else {
// If no turbo tasks context is available, just run the job. // If no turbo tasks context is available, just run the job.
f() f()
......
...@@ -84,9 +84,7 @@ fn notify_mutated( ...@@ -84,9 +84,7 @@ fn notify_mutated(
} }
let _span = trace_span!("state value changed").entered(); let _span = trace_span!("state value changed").entered();
with_turbo_tasks(|tt| { with_turbo_tasks(|tt| {
for invalidator in invalidators { Invalidator::invalidate_all(invalidators, &**tt);
invalidator.invalidate(&**tt);
}
if let Some(serialization_invalidator) = serialization_invalidator { if let Some(serialization_invalidator) = serialization_invalidator {
tt.invalidate_serialization(serialization_invalidator.task()); tt.invalidate_serialization(serialization_invalidator.task());
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment