turbo-tasks: Reduce allocations on cache hits (#92756)
### What? Reduce heap allocations when turbo-tasks functions get cache hits (~85% of calls). ### Why? Every turbo-tasks function call (generated by `#[turbo_tasks::function]`) was boxing its arguments into `Box<dyn MagicAny>` before looking up the task cache. This allocation is wasted on cache hits, which are the overwhelmingly common case. ### How? **Deferred boxing via `StackMagicAny` trait object:** Introduce a `StackMagicAny` trait that abstracts over a stack-resident `Option<T>`: - `as_ref(&self) -> &dyn MagicAny` — borrow the argument for hash/equality (cache lookup) - `take_box(&mut self) -> Box<dyn MagicAny>` — move the value to the heap (zero clones) - `as_any_mut(&mut self) -> &mut dyn Any` — downcast to concrete type without boxing The data flow: 1. **Callsite** (macro-generated): creates `StackMagicAnySlot::new((args...))` on the stack, calls `dynamic_call(..., &mut arg)` 2. **`dynamic_call`**: checks resolution via `arg.as_ref()`, routes to `native_call` (resolved) or boxes via `arg.take_box()` for `LocalTaskSpec` (unresolved) 3. **Backend `get_or_create_task_inner`**: does a read-only `raw_get` lookup using `hash_from_components` + `eq_components` with the borrowed `&dyn MagicAny`. On cache hit (~85%), returns immediately — **zero heap allocation**. On cache miss, re-checks under write lock using the same borrowed reference, and only calls `arg.take_box()` in the vacant entry case (true cache miss). Boxing is now deferred past all of these: - **Memory cache hit** — the common case, no allocation at all - **Backing storage hit** — found in persistent storage, no allocation needed - **Lost race under write lock** — another thread inserted while we upgraded; we use their task_id, still no allocation - **Trait method dispatch (no filtering)** — `filter_owned` is now `Option<FilterOwnedArgsFunctor>`; when `None` (the common case where all args are used), the original `&mut dyn StackMagicAny` passes straight through to `dynamic_call` without boxing **Optimized `filter_owned` for traits:** When trait methods *do* need argument filtering (unused `_`-prefixed parameters), the old path did `take_box()` → `downcast_args_owned()` → dereference → repack. This is an unnecessary heap round-trip. The new `downcast_stack_args_owned()` function uses `as_any_mut()` to downcast directly to `&mut StackMagicAnySlot<T>` and calls `take()` on the inner `Option`, skipping the intermediate `Box` entirely. **Additional changes:** - `Backend::get_or_create_*_task` now takes decomposed parameters (`native_fn`, `this`, `&mut dyn StackMagicAny`) instead of a pre-constructed `CachedTaskType` - Persistent and transient task creation merged into a shared `get_or_create_task_inner(transient: bool)` - `connect_child` uses eagerly-set `persistent_task_type` from `initialize_new_task` - `OwnedMagicAny` adapter wraps already-boxed args (from async resolution tasks) to fit the `StackMagicAny` interface - Both `dynamic_call` and `trait_call` take `&mut dyn StackMagicAny` (trait dispatch also benefits) - `CachedTaskType::hash_encode` now delegates to `hash_encode_components` (deduplicated) - Removed `try_native_call`, `native_call_if_consistent`, `try_get_or_create_*` — the deferred boxing approach subsumes these ### Binary size Binary size is neutral (linux-x86_64, `--release`, stripped + gzipped: 30.9 MB on both canary and this branch). ### Overhead benchmark (turbo-tasks-backend, median, lower is better) Measured on an isolated Firecracker microVM (linux-x86_64). Variance is nontrivial on this environment, but the direction is consistently positive across all turbo-tasks benchmarks. | Benchmark | Canary | This PR | Delta | |---|---|---|---| | `turbo-cached-same-keys/1` | 512.6 ns | 490.2 ns | -4.4% | | `turbo-cached-same-keys/10` | 493.4 ns | 483.8 ns | -2.0% | | `turbo-cached-same-keys/100` | 502.1 ns | 484.9 ns | -3.4% | | `turbo-cached-same-keys/1000` | 1.31 µs | 774.2 ns | -41.0% | | `turbo-cached-different-keys/1` | 1.02 µs | 1.00 µs | -1.4% | | `turbo-cached-different-keys/10` | 1.13 µs | 1.11 µs | -1.8% | | `turbo-cached-different-keys/100` | 1.29 µs | 1.24 µs | -3.8% | | `turbo-cached-different-keys/1000` | 2.37 µs | 2.32 µs | -2.5% | | `turbo-uncached/1` | 23.43 µs | 20.47 µs | -12.6% | | `turbo-uncached/10` | 32.23 µs | 29.71 µs | -7.8% | | `turbo-uncached/100` | 126.01 µs | 124.95 µs | -0.8% | | `turbo-uncached/1000` | 1.079 ms | 1.048 ms | -2.9% | | `turbo-uncached-parallel/1` | 6.16 µs | 5.67 µs | -8.0% | | `turbo-uncached-parallel/10` | 5.47 µs | 5.12 µs | -6.4% | | `turbo-uncached-parallel/100` | 14.65 µs | 14.55 µs | -0.7% | | `turbo-uncached-parallel/1000` | 132.09 µs | 129.13 µs | -2.2% | <!-- NEXT_JS_LLM_PR -->
Showing
This diff is collapsed.
This diff is collapsed.
Please
register
or
sign in
to comment