Unverified Commit cdec4740 authored by Benjamin Woodruff's avatar Benjamin Woodruff Committed by GitHub

docs(turbopack-core): Copy "layers" documentation from mdbook to rustdoc (#91350)

This is implemented as a dummy module that includes a markdown file. Clap does something similar in their rustdocs.

I had claude review the content for accuracy. It made a few modifications. I added rustdoc links by hand.

I converted the mermaid diagram to excalidraw and uploaded the image to vercel blob.

Copied from: https://turbopack-rust-docs.vercel.sh/turbopack/layers.html

Rendered:

![Screenshot 2026-03-13 at 17-13-58 turbopack_core _layers - Rust.png](https://app.graphite.com/user-attachments/assets/d237a05e-da03-405a-8f36-e17e0a351b14.png)
parent 9b263e98
...@@ -3,6 +3,8 @@ ...@@ -3,6 +3,8 @@
"attacks", "attacks",
"color", "color",
"dead", "dead",
"deno",
"digitalocean",
"dirty", "dirty",
"execute", "execute",
"executed", "executed",
...@@ -14,19 +16,18 @@ ...@@ -14,19 +16,18 @@
"failures", "failures",
"fire", "fire",
"fires", "fires",
"flightcontrol",
"fly.io",
"github",
"hook", "hook",
"hooks", "hooks",
"host-hostess", "host-hostess",
"invalid", "invalid",
"railway",
"remains", "remains",
"simple",
"special", "special",
"white", "sst",
"deno", "white"
"digitalocean",
"flightcontrol",
"fly.io",
"github",
"railway",
"sst"
] ]
} }
Turbopack models the user's code as it travels through Turbopack in multiple ways, each of which can be thought of as a layer in a larger system.
```text
┌───────────────┬──────────────────────────────┐
│ Sources │ │
├───────────────┘ │
│ What "the user wrote" as │
│ application code │
└──────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────┐
│ Modules │ │
├───────────────┘ │
│ The "compiler's understanding" of the │
│ application code │
└──────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────┐
│ Output assets │ │
├───────────────┘ │
│ What the "target environment understands" as │
│ executable application │
└──────────────────────────────────────────────┘
```
Each layer builds on the previous one: [`Source`]s are processed into [`Module`]s, and [`Module`]s are transformed into [`OutputAsset`]s. For a given set of entry points, the full module graph is discovered and analyzed before chunking into [`OutputAsset`]s.
[`Source`]: crate::source::Source
[`Module`]: crate::module::Module
[`OutputAsset`]: crate::output::OutputAsset
## Sources
[`Source`]s are content of code or files before they are analyzed and converted into [`Module`]s. They might be the original source file the user has written, or a virtual source code that was generated. They might also be transformed from other [`Source`] types, e.g. when using a preprocessor like Sass or webpack loaders.
Sources do **not** model references (the relationships between files like through `import`, `sourceMappingURL`, etc.).
Each Source has an identifier composed of file path, query, fragment, and other modifiers.
## Modules
[`Module`]s are the result of parsing, transforming and analyzing [`Source`]s. They include references to other modules as analyzed.
References can be followed to traverse a subgraph of the module graph. They implicitly form the module graph by exposing references.
Each [`Module`] has an identifier composed of file path, query, fragment, and other modifiers.
## Output Assets
[`OutputAsset`]s are artifacts that are understood by the target environment. They include references to other output assets.
[`OutputAsset`]s are usually the result of transforming one or more modules to a given output format. This can be a very simple transformation like copying the [`Source`] content (like with static assets like images), or a complex transformation like chunking and bundling modules (like with JavaScript or CSS).
[`OutputAsset`]s can be emitted to disk or served from the dev server.
Each [`OutputAsset`] has a file path.
## Common Supertrait: Asset
[`Source`]s and [`OutputAsset`]s both implement the [`Asset`] trait, which provides access to file content. [`Module`]s do not implement [`Asset`], as they represent the compiler's parsed understanding of code rather than raw file content.
[`Asset`]: crate::asset::Asset
## Example
<figure style="display: flex; flex-direction: column; justify-content: center;">
<img alt="An example of sources, modules, and output assets and how they refer to each other" width="800px" src="https://h8dxkfmaphn8o0p3.public.blob.vercel-storage.com/rustdoc-images/layers.excalidraw.png">
<!-- https://excalidraw.com/#json=5Ll5jG5OIcBUPrCdq3Zxu,Iw_HDeXFXuU8Pne3TAEIMA -->
</figure>
...@@ -52,6 +52,9 @@ pub mod virtual_fs { ...@@ -52,6 +52,9 @@ pub mod virtual_fs {
pub use turbo_tasks_fs::VirtualFileSystem; pub use turbo_tasks_fs::VirtualFileSystem;
} }
#[doc = include_str!("../layers.md")]
pub mod _layers {}
pub const PROJECT_FILESYSTEM_NAME: &str = "project"; pub const PROJECT_FILESYSTEM_NAME: &str = "project";
pub const SOURCE_URL_PROTOCOL: &str = "turbopack:"; pub const SOURCE_URL_PROTOCOL: &str = "turbopack:";
......
...@@ -34,6 +34,9 @@ pub enum ModuleSideEffects { ...@@ -34,6 +34,9 @@ pub enum ModuleSideEffects {
} }
/// A module. This usually represents parsed source code, which has references to other modules. /// A module. This usually represents parsed source code, which has references to other modules.
///
/// For documentation about where this is used and how it fits into the rest of Turbopack, see
/// [`crate::_layers`].
#[turbo_tasks::value_trait] #[turbo_tasks::value_trait]
pub trait Module { pub trait Module {
/// The identifier of the [`Module`]. It's expected to be unique and capture all properties of /// The identifier of the [`Module`]. It's expected to be unique and capture all properties of
......
...@@ -26,8 +26,10 @@ pub trait OutputAssetsReference { ...@@ -26,8 +26,10 @@ pub trait OutputAssetsReference {
} }
} }
/// An asset that should be outputted, e. g. written to disk or served from a /// An asset that should be outputted, e. g. written to disk or served from a server.
/// server. ///
/// For documentation about where this is used and how it fits into the rest of Turbopack, see
/// [`crate::_layers`].
#[turbo_tasks::value_trait] #[turbo_tasks::value_trait]
pub trait OutputAsset: Asset + OutputAssetsReference { pub trait OutputAsset: Asset + OutputAssetsReference {
/// The identifier of the [OutputAsset]. It's expected to be unique and /// The identifier of the [OutputAsset]. It's expected to be unique and
......
...@@ -6,6 +6,9 @@ use crate::{asset::Asset, ident::AssetIdent}; ...@@ -6,6 +6,9 @@ use crate::{asset::Asset, ident::AssetIdent};
/// Unparsed input source code. Source code is processed into [`Module`]s by the [`AssetContext`]. /// Unparsed input source code. Source code is processed into [`Module`]s by the [`AssetContext`].
/// All `Source`s have content and an identifier. /// All `Source`s have content and an identifier.
/// ///
/// For documentation about where this is used and how it fits into the rest of Turbopack, see
/// [`crate::_layers`].
///
/// [`Module`]: crate::module::Module /// [`Module`]: crate::module::Module
/// [`AssetContext`]: crate::context::AssetContext /// [`AssetContext`]: crate::context::AssetContext
#[turbo_tasks::value_trait] #[turbo_tasks::value_trait]
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment