Unverified Commit 4074ede0 authored by Benjamin Woodruff's avatar Benjamin Woodruff Committed by GitHub

docs(turbopack): Better document the Vc type, with references to ResolvedVc and…

docs(turbopack): Better document the Vc type, with references to ResolvedVc and VcOperation (#72524)

![Screenshot 2024-11-25 at 16-42-42 Vc in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/759f1e16-90b0-45e1-bfbf-19cbfd238748.png)

![Screenshot 2024-11-25 at 16-43-09 OperationVc in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/6b36e3aa-6fa2-4a9d-baf9-7f24987acf9f.png)

![Screenshot 2024-11-25 at 16-43-25 ResolvedVc in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/ed5794bb-0f5b-491e-ae6e-9ccf9878e609.png)

![Screenshot 2024-11-25 at 16-43-41 State in turbo_tasks - Rust.png](https://graphite-user-uploaded-assets-prod.s3.amazonaws.com/HAZVitxRNnZz8QMiPn4a/4a79d97f-f33b-44ca-b26e-930c2352089f.png)
parent eecc5f1c
...@@ -2,25 +2,29 @@ ...@@ -2,25 +2,29 @@
//! execution. //! execution.
//! //!
//! It defines 4 primitives: //! It defines 4 primitives:
//! - functions: Unit of execution, invalidation and reexecution. //! - **[Functions][macro@crate::function]:** Units of execution, invalidation, and reexecution.
//! - values: Data created, stored and returned by functions. //! - **[Values][macro@crate::value]:** Data created, stored, and returned by functions.
//! - traits: Traits that define a set of functions on values. //! - **[Traits][macro@crate::value_trait]:** Traits that define a set of functions on values.
//! - collectibles: Values emitted in functions that bubble up the call graph and can be collected //! - **[Collectibles][crate::TurboTasks::emit_collectible]:** Values emitted in functions that
//! in parent functions. //! bubble up the call graph and can be collected in parent functions.
//! //!
//! It also defines some derived elements from that: //! It also defines some derived elements from that:
//! - cells: The locations in functions where values are stored. The content of a cell can change //! - **[Tasks][book-tasks]:** An instance of a function together with its arguments.
//! after the reexecution of a function. //! - **[Cells][book-cells]:** The locations associated with tasks where values are stored. The
//! - Vcs: A reference to a cell in a function or a return value of a function. //! contents of a cell can change after the reexecution of a function.
//! - task: An instance of a function together with its arguments. //! - **[`Vc`s ("Value Cells")][Vc]:** A reference to a cell or a return value of a function.
//! //!
//! A Vc can be read to get a read-only reference to the stored data. //! A [`Vc`] can be read to get [a read-only reference][ReadRef] to the stored data, representing a
//! snapshot of that cell at that point in time.
//! //!
//! On execution of functions, turbo-tasks will track which Vcs are read. Once //! On execution of functions, `turbo-tasks` will track which [`Vc`]s are read. Once any of these
//! any of these change, turbo-tasks will invalidate the task created from the //! change, `turbo-tasks` will invalidate the task created from the function's execution and it will
//! function's execution and it will eventually be scheduled and reexecuted. //! eventually be scheduled and reexecuted.
//! //!
//! Collectibles go through a similar process. //! Collectibles go through a similar process.
//!
//! [book-cells]: https://turbopack-rust-docs.vercel.sh/turbo-engine/cells.html
//! [book-tasks]: https://turbopack-rust-docs.vercel.sh/turbo-engine/tasks.html
#![feature(trivial_bounds)] #![feature(trivial_bounds)]
#![feature(min_specialization)] #![feature(min_specialization)]
......
...@@ -66,7 +66,7 @@ macro_rules! stringify_path { ...@@ -66,7 +66,7 @@ macro_rules! stringify_path {
/// implement [`ShrinkToFit`]. /// implement [`ShrinkToFit`].
/// ///
/// This is used by the derive macro for [`ShrinkToFit`], which is called by the /// This is used by the derive macro for [`ShrinkToFit`], which is called by the
/// [turbo_tasks::value][crate::value] macro. /// [turbo_tasks::value][macro@crate::value] macro.
/// ///
/// [autoderef]: http://lukaskalbertodt.github.io/2019/12/05/generalized-autoref-based-specialization.html /// [autoderef]: http://lukaskalbertodt.github.io/2019/12/05/generalized-autoref-based-specialization.html
pub struct ShrinkToFitDerefSpecialization<'a, T> { pub struct ShrinkToFitDerefSpecialization<'a, T> {
......
...@@ -54,6 +54,15 @@ impl Display for CellId { ...@@ -54,6 +54,15 @@ impl Display for CellId {
} }
} }
/// A type-erased representation of [`Vc`].
///
/// Type erasure reduces the [monomorphization] (and therefore binary size and compilation time)
/// required to support `Vc`.
///
/// This type is heavily used within the [`Backend`][crate::backend::Backend] trait, but should
/// otherwise be treated as an internal implementation detail of `turbo-tasks`.
///
/// [monomorphization]: https://doc.rust-lang.org/book/ch10-01-syntax.html#performance-of-code-using-generics
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum RawVc { pub enum RawVc {
TaskOutput(TaskId), TaskOutput(TaskId),
......
...@@ -96,6 +96,25 @@ impl<T> Drop for StateRef<'_, T> { ...@@ -96,6 +96,25 @@ impl<T> Drop for StateRef<'_, T> {
} }
} }
/// An [internally-mutable] type, similar to [`RefCell`][std::cell::RefCell] or [`Mutex`] that can
/// be stored inside a [`VcValueType`].
///
/// **[`State`] should only be used with [`OperationVc`] and types that implement
/// [`OperationValue`]**.
///
/// Setting values inside a [`State`] bypasses the normal argument and return value tracking
/// that's tracks child function calls and re-runs tasks until their values settled. That system is
/// needed for [strong consistency]. [`OperationVc`] ensures that function calls are reconnected
/// with the parent/child call graph.
///
/// When reading a `State` with [`State::get`], the state itself (though not any values inside of
/// it) is marked as a dependency of the current task.
///
/// [internally-mutable]: https://doc.rust-lang.org/book/ch15-05-interior-mutability.html
/// [`VcValueType`]: crate::VcValueType
/// [strong consistency]: crate::Vc::strongly_consistent
/// [`OperationVc`]: crate::OperationVc
/// [`OperationValue`]: crate::OperationValue
#[derive(Serialize, Deserialize)] #[derive(Serialize, Deserialize)]
pub struct State<T> { pub struct State<T> {
serialization_invalidator: SerializationInvalidator, serialization_invalidator: SerializationInvalidator,
......
...@@ -38,18 +38,127 @@ use crate::{ ...@@ -38,18 +38,127 @@ use crate::{
CellId, CollectiblesSource, RawVc, ResolveTypeError, SharedReference, ShrinkToFit, CellId, CollectiblesSource, RawVc, ResolveTypeError, SharedReference, ShrinkToFit,
}; };
/// A Value Cell (`Vc` for short) is a reference to a memoized computation /// A "Value Cell" (`Vc` for short) is a reference to a memoized computation result stored on the
/// result stored on the heap or in persistent cache, depending on the /// heap or in persistent cache, depending on the Turbo Engine backend implementation.
/// Turbo Engine backend implementation.
/// ///
/// In order to get a reference to the pointed value, you need to `.await` the /// In order to get a reference to the pointed value, you need to `.await` the [`Vc<T>`] to get a
/// [`Vc<T>`] to get a [`ReadRef<T>`][crate::ReadRef]: /// [`ReadRef<T>`][`ReadRef`]:
/// ///
/// ``` /// ```
/// let some_vc: Vc<T>; /// let some_vc: Vc<T>;
/// let some_ref: ReadRef<T> = some_vc.await?; /// let some_ref: ReadRef<T> = some_vc.await?;
/// some_ref.some_method_on_t(); /// some_ref.some_method_on_t();
/// ``` /// ```
///
/// `Vc`s are similar to a [`Future`] or a Promise with a few key differences:
///
/// - The value pointed to by a `Vc` can be invalidated by changing dependencies or cache evicted,
/// meaning that `await`ing a `Vc` multiple times can give different results. A [`ReadRef`] is
/// snapshot of the underlying cell at a point in time.
///
/// - Reading (`await`ing) `Vc`s causes the current task to be tracked a dependent of the `Vc`'s
/// task or task cell. When the read task or task cell changes, the current task may be
/// re-executed.
///
/// - `Vc` types are always [`Copy`]. Most [`Future`]s are not. This works because `Vc`s are
/// represented as a few ids or indicies into data structures managed by the `turbo-tasks`
/// framework. `Vc` types are not reference counted, but do support [tracing] for a hypothetical
/// (unimplemented) garbage collector.
///
/// - Unlike futures (but like promises), the work that a `Vc` represents [begins execution even if
/// the `Vc` is not `await`ed](#execution-model).
///
/// For a more in-depth explanation of the concepts behind value cells, [refer to the Turbopack
/// book][book-cells].
///
///
/// ## Subtypes
///
/// There are a couple of "subtypes" of `Vc`. These can both be cheaply converted back into a `Vc`.
///
/// - **[`ResolvedVc`]:** A reference to a cell constructed within a task, as part of a [`Vc::cell`]
/// or `value_type.cell()` constructor. As the cell has been constructed at least once, the
/// concrete type of the cell is known (allowing [downcasting][ResolvedVc::try_downcast]). This is
/// stored as a combination of a task id, a type id, and a cell id.
///
/// - **[`OperationVc`]:** The synchronous return value of a [`turbo_tasks::function`]. Internally,
/// this is stored using a task id. Exact type information of trait types (i.e. `Vc<Box<dyn
/// Trait>>`) is not known because the function may not have finished execution yet. Operations
/// must first be [`connected`][OperationVc::connect]ed before being read.
///
/// [`ResolvedVc`] is almost always preferred over the more awkward [`OperationVc`] API, but
/// [`OperationVc`] can be useful inside of [`State`] or when dealing with [collectibles].
///
/// In addition to these potentially-explicit representations of a `Vc`, there's another internal
/// representation of a `Vc`, known as a "Local `Vc`".
///
/// - **Local Operation or Cell:** Same as [`ResolvedVc`] or [`OperationVc`], but these values are
/// stored in task-local state that is freed after their parent non-local task exits. These values
/// are sometimes created when calling a [`turbo_tasks::function`] as an optimization. [Converting
/// a local `Vc` to a `ResolvedVc`][Vc::to_resolved] will construct a new
/// [non-local][NonLocalValue] cell.
///
/// These many representations are stored internally using a type-erased [`RawVc`]. Type erasure
/// reduces the [monomorphization] (and therefore binary size and compilation time) required to
/// support `Vc` and its subtypes.
///
/// <div class="warning">
/// <p>
/// Local <code>Vc</code>s are not valid outside of their parent task, so they must be implicitly
/// (e.g. as an argument or return type) or explicitly (e.g. via <a
/// href="#method.to_resolved"><code>Vc::to_resolved</code></a>) be converted to a non-local <a
/// href="struct.ResolvedVc.html"><code>ResolvedVc</code></a> or <a
/// href="struct.VcOperation.html"><code>VcOperation</code></a> before crossing task boundaries.
/// </p>
/// <p>
/// For this reason, <code>Vc</code> types (which are potentially local) will be disallowed as
/// fields in <a href="attr.value.html"><code>turbo_tasks::value</code></a>s in the future.
/// </p>
/// </div>
///
/// | | Representation? | [Non-Local?] | Equality? | Can be Downcast? |
/// |-----------------|-----------------------------|--------------|-------------------------|----------------------------|
/// | [`Vc`] | One of many | ⚠️ Maybe | ❌ Not recommended | ⚠️ After resolution |
/// | [`ResolvedVc`] | Task Id + Type Id + Cell Id | ✅ Yes | ✅ Yes, [see docs][rvc] | ✅ [Yes, cheaply][resolve] |
/// | [`OperationVc`] | Task Id | ✅ Yes | ✅ Yes, [see docs][ovc] | ⚠️ After resolution |
///
/// [Non-Local]: NonLocalValue
/// [rvc]: ResolvedVc
/// [ovc]: ResolvedVc
/// [resolve]: ResolvedVc::try_downcast
///
/// See the documentation for [`ResolvedVc`] and [`OperationVc`] for more details about these
/// subtypes.
///
///
/// ## Execution Model
///
/// While task functions are expected to be side-effect free, their execution behavior is still
/// important for performance reasons, or to code using [collectibles] to represent issues or
/// side-effects.
///
/// Function calls are neither "eager", nor "lazy". Even if not awaited, they are guaranteed to
/// execute (potentially emitting collectibles) before the root task finishes or before the
/// completion of any strongly consistent read containing their call. However, the exact point when
/// that execution begins is an implementation detail. Functions may execute more than once due to
/// dirty task invalidation.
///
///
/// ## Equality & Hashing
///
/// Because `Vc`s can be equivalent but have different representation, it's not recommended to
/// compare `Vc`s by equality. Instead, you should convert a `Vc` to an explicit subtype first
/// (likely [`ResolvedVc`]). Future versions of `Vc` may not implement [`Eq`], [`PartialEq`], or
/// [`Hash`].
///
///
/// [tracing]: crate::trace::TraceRawVcs
/// [`ReadRef`]: crate::ReadRef
/// [`turbo_tasks::function`]: crate::function
/// [monomorphization]: https://doc.rust-lang.org/book/ch10-01-syntax.html#performance-of-code-using-generics
/// [`State`]: crate::State
/// [book-cells]: https://turbopack-rust-docs.vercel.sh/turbo-engine/cells.html
/// [collectibles]: CollectiblesSource
#[must_use] #[must_use]
#[derive(Serialize, Deserialize)] #[derive(Serialize, Deserialize)]
#[serde(transparent, bound = "")] #[serde(transparent, bound = "")]
......
...@@ -8,6 +8,43 @@ use crate::{ ...@@ -8,6 +8,43 @@ use crate::{
Upcast, Vc, VcValueTrait, Upcast, Vc, VcValueTrait,
}; };
/// A "subtype" (can be converted via [`.connect()`]) of [`Vc`] that
/// represents a specific call (with arguments) to [a task][macro@crate::function].
///
/// Unlike [`Vc`], `OperationVc`:
///
/// - Does not potentially refer to task-local information, meaning that it implements
/// [`NonLocalValue`], and can be used in any [`#[turbo_tasks::value]`][macro@crate::value].
///
/// - Has only one potential internal representation, meaning that it has a saner equality
/// definition.
///
/// - Can be [reconnected][OperationVc::connect] to the strongly-consistent compilation graph after
/// being placed inside of a [`State`].
///
/// - Makes sense with [collectibles][`CollectiblesSource`], as it represents a function call, and
/// only function calls can have issues or side-effects.
///
///
/// ## Equality & Hashing
///
/// Equality between two `OperationVc`s means that both have an identical in-memory representation
/// and point to the same task function call. The implementation of [`Hash`] has similar behavior.
///
/// If [connected] and then `.await`ed at the same time, both would likely resolve to the same
/// [`ReadRef`], though it is possible that they may not if the task or cell is invalidated between
/// `.await`s.
///
/// Because equality is a synchronous operation that cannot read the cell contents, even if the
/// `OperationVc`s are not equal, it is possible that if `.await`ed, both `OperationVc`s could point
/// to the same or equal values.
///
/// [`.connect()`]: OperationVc::connect
/// [reconnected]: OperationVc::connect
/// [connected]: OperationVc::connect
/// [`NonLocalValue`]: crate::NonLocalValue
/// [`State`]: crate::State
/// [`ReadRef`]: crate::ReadRef
#[must_use] #[must_use]
pub struct OperationVc<T> pub struct OperationVc<T>
where where
...@@ -21,6 +58,9 @@ impl<T: ?Sized> OperationVc<T> { ...@@ -21,6 +58,9 @@ impl<T: ?Sized> OperationVc<T> {
/// ///
/// The caller must ensure that the `Vc` is not a local task and it points to a a single /// The caller must ensure that the `Vc` is not a local task and it points to a a single
/// operation. /// operation.
///
/// **This API is a placeholder and will likely be removed soon** in favor of a future API that
/// uses macros and static (compile-time) assertions in place of runtime assertions.
pub fn new(node: Vc<T>) -> Self { pub fn new(node: Vc<T>) -> Self {
// TODO to avoid this runtime check, we should mark functions with `(operation)` and return // TODO to avoid this runtime check, we should mark functions with `(operation)` and return
// a OperationVc directly // a OperationVc directly
...@@ -32,7 +72,7 @@ impl<T: ?Sized> OperationVc<T> { ...@@ -32,7 +72,7 @@ impl<T: ?Sized> OperationVc<T> {
} }
/// Marks this operation's underlying function call as a child of the current task, and returns /// Marks this operation's underlying function call as a child of the current task, and returns
/// a dereferenced [`Vc`] that can be [resolved][Vc::to_resolved] or read with `.await?`. /// a [`Vc`] that can be [resolved][Vc::to_resolved] or read with `.await?`.
/// ///
/// By marking this function call as a child of the current task, turbo-tasks will re-run tasks /// By marking this function call as a child of the current task, turbo-tasks will re-run tasks
/// as-needed to achieve strong consistency at the root of the function call tree. This explicit /// as-needed to achieve strong consistency at the root of the function call tree. This explicit
......
...@@ -16,6 +16,52 @@ use crate::{ ...@@ -16,6 +16,52 @@ use crate::{
ResolveTypeError, Upcast, VcRead, VcTransparentRead, VcValueTrait, VcValueType, ResolveTypeError, Upcast, VcRead, VcTransparentRead, VcValueTrait, VcValueType,
}; };
/// A "subtype" (via [`Deref`]) of [`Vc`] that represents a specific [`Vc::cell`]/`.cell()` or
/// [`ResolvedVc::cell`]/`.resolved_cell()` constructor call within [a task][macro@crate::function].
///
/// Unlike [`Vc`], `ResolvedVc`:
///
/// - Does not potentially refer to task-local information, meaning that it implements
/// [`NonLocalValue`], and can be used in any [`#[turbo_tasks::value]`][macro@crate::value].
///
/// - Has only one potential internal representation, meaning that it has a saner equality
/// definition.
///
/// - Points to a concrete value with a type, and is therefore [cheap to
/// downcast][ResolvedVc::try_downcast].
///
///
/// ## Construction
///
/// There are a few ways to construct a `ResolvedVc`, in order of preference:
///
/// 1. Given a [value][VcValueType], construct a `ResolvedVc` using [`ResolvedVc::cell`] (for
/// "transparent" values) or by calling the generated `.resolved_cell()` constructor on the value
/// type.
///
/// 2. Given an argument to a function using the [`#[turbo_tasks::function]`][macro@crate::function]
/// macro, change the argument's type to a `ResolvedVc`. The [rewritten external signature] will
/// still use [`Vc`], but when the function is called, the [`Vc`] will be resolved.
///
/// 3. Given a [`Vc`], use [`.to_resolved().await?`][Vc::to_resolved].
///
///
/// ## Equality & Hashing
///
/// Equality between two `ResolvedVc`s means that both have an identical in-memory representation
/// and point to the same cell. The implementation of [`Hash`] has similar behavior.
///
/// If `.await`ed at the same time, both would likely resolve to the same [`ReadRef`], though it is
/// possible that they may not if the cell is invalidated between `.await`s.
///
/// Because equality is a synchronous operation that cannot read the cell contents, even if the
/// `ResolvedVc`s are not equal, it is possible that if `.await`ed, both `ResolvedVc`s could point
/// to the same or equal values.
///
///
/// [`NonLocalValue`]: crate::NonLocalValue
/// [rewritten external signature]: https://turbopack-rust-docs.vercel.sh/turbo-engine/tasks.html#external-signature-rewriting
/// [`ReadRef`]: crate::ReadRef
#[derive(Serialize, Deserialize)] #[derive(Serialize, Deserialize)]
#[serde(transparent, bound = "")] #[serde(transparent, bound = "")]
pub struct ResolvedVc<T> pub struct ResolvedVc<T>
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment