Unverified Commit 3214020b authored by Benjamin Woodruff's avatar Benjamin Woodruff Committed by GitHub

Turbopack: Fix missing canonicalization of paths and always use verbatim paths…

Turbopack: Fix missing canonicalization of paths and always use verbatim paths internally for Windows (#95668)

I was seeing failures on https://github.com/vercel/next.js/pull/95628
and went down this rabbit hole.

- We were (incorrectly) not canonicalizing the `DiskFileSystem` root
dir. This happened to work out for us because pnpm also wasn't
canonicalizing junction point targets, and so the root dir prefix
matched. However after `#95628`, pnpm started canonicalizing junction
point targets because we added `pnpm-workspace.yaml` files. This was
just coincidence and indicative of much deeper problems.

- `dunce` isn't a good fit for us. It picks a win32 or verbatim
representation based on path length, which would mean that long paths
inside of a short root path would never work because we'd never be able
to strip the prefix, since we'd be looking for a win32 path prefix.
Omnipath is better for what we want to do:
https://docs.rs/omnipath/latest/omnipath/windows/trait.WinPathExt.html

- The windows verbatim path format works with long paths (>260
characters) and behaves a lot closer to cross-platform unix paths, which
is what we want. E.g. if somebody tries to import a JS module from a
Windows 8.3 short path, it should fail. This will fail with verbatim
paths (good!). This PR *always* uses the verbatim path format internally
within `DiskFileSystem`.

- `read_link` was re-inventing `try_from_sys_path` but badly. Just call
`try_from_sys_path`.

- `turbopack/crates/turbo-tasks-fs/src/embed/file.rs` was dead code.

- `validate_path_length_inner` was incorrectly referring to verbatim
paths as UNC paths. These are two different things.

- Canonicalize any absolute symlink targets outside of the
`DiskFileSystem` root path, resolving any symlinks they may have,
normalizing case-insensitive paths, and handling Windows 8.3 short name
format.

## Some Context About Windows Path Formats:

- win32 paths: These are your normal user-friendly `C:\foo\blah` paths.
When calling most filesystem APIs, these are limited to 260 characters
unless the user has enabled long paths on the system, and avoiding the
limit with these paths also requires a [`longPathAware` application
manifest](https://learn.microsoft.com/en-us/windows/win32/sbscs/application-manifests#longPathAware).
You can mix `/` and `\` in win32 paths, and Windows will normalize the
path separator for you.

- verbatim paths: This is a rust-specific name (used by the stdlib) for
the "extended length" prefixed paths. These are paths starting with
`\\?\`. These paths can be up to ~32,767 characters long. They do not
support Windows 8.3 shortnames. They do not normalize between `/` and
`\`, and do not support `.` or `..` components, however
[`Path::join`](https://doc.rust-lang.org/std/path/struct.Path.html#method.join)
will do this normalization if given a verbatim path.

- Windows 8.3 shortnames: A legacy format where paths can be represented
by 8 characters, followed by a tilde and a digit (or hash in some
extreme cases where there are many collisions), followed by a 3
character file extension. NTFS supports this,
[ReFS](https://learn.microsoft.com/en-us/windows-server/storage/refs/refs-overview)
(used by [Windows Dev
Drive](https://learn.microsoft.com/en-us/windows/dev-drive/)) does not.

- UNC ("Universal Naming Convention"): This is a specific representation
of paths used for remote files on network servers.

- `file://` URIs: These must be formed from [win32 style
paths](https://docs.rs/url/latest/url/struct.Url.html#method.from_file_path),
though the normal 260 character limit typically does not apply here.

- canonicalize: Corresponds to
[`std::fs::canonicalize`](https://doc.rust-lang.org/stable/std/fs/fn.canonicalize.html).
This involves actual fs operations (that we can't easily track), and
recursively resolves symlinks. It will fail if the file path does not
exist. On windows, this always returns a verbatim path.

- wide format: Windows paths are utf-16. Generally we don't support
paths that can't be encoded to utf-8, but we should try to fail as
obviously as we can in these cases. We don't have good filesystem issue
handling yet though.

- case sensitivity: Generally paths are not case-sensitive on Windows or
macOS, but are case-sensitive on most Linux filesystems. The exact
definition of how case sensitivity works with unicode paths is highly
platform and filesystem dependent.

- junction points: The windows equivalent of symlinks (symlinks are also
supported, but require developer mode to be turned on). These can
contain an arbitrary path reference in any format (win32, verbatim, etc)
with any case and with windows 8.3 short names. Junction points are
always target absolute paths to directories.
parent 3ed60553
...@@ -4971,6 +4971,7 @@ dependencies = [ ...@@ -4971,6 +4971,7 @@ dependencies = [
"turbo-rcstr", "turbo-rcstr",
"turbo-tasks", "turbo-tasks",
"turbo-tasks-backend", "turbo-tasks-backend",
"turbo-tasks-fs",
"turbo-tasks-malloc", "turbo-tasks-malloc",
"turbopack-node", "turbopack-node",
"turbopack-trace-utils", "turbopack-trace-utils",
...@@ -5413,6 +5414,12 @@ dependencies = [ ...@@ -5413,6 +5414,12 @@ dependencies = [
"memchr", "memchr",
] ]
[[package]]
name = "omnipath"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "80adb31078122c880307e9cdfd4e3361e6545c319f9b9dcafcb03acd3b51a575"
[[package]] [[package]]
name = "once_cell" name = "once_cell"
version = "1.21.3" version = "1.21.3"
...@@ -10188,13 +10195,14 @@ dependencies = [ ...@@ -10188,13 +10195,14 @@ dependencies = [
"codspeed-criterion-compat", "codspeed-criterion-compat",
"concurrent-queue", "concurrent-queue",
"dashmap 6.1.0", "dashmap 6.1.0",
"dunce", "fs-err",
"futures", "futures",
"include_dir", "include_dir",
"indexmap 2.13.0", "indexmap 2.13.0",
"jsonc-parser", "jsonc-parser",
"mime", "mime",
"notify", "notify",
"omnipath",
"parking_lot", "parking_lot",
"rand 0.10.1", "rand 0.10.1",
"regex", "regex",
...@@ -10204,6 +10212,7 @@ dependencies = [ ...@@ -10204,6 +10212,7 @@ dependencies = [
"serde_json", "serde_json",
"serde_path_to_error", "serde_path_to_error",
"sha2", "sha2",
"smallvec",
"tempfile", "tempfile",
"thiserror 1.0.69", "thiserror 1.0.69",
"tokio", "tokio",
...@@ -10216,6 +10225,7 @@ dependencies = [ ...@@ -10216,6 +10225,7 @@ dependencies = [
"turbo-tasks-hash", "turbo-tasks-hash",
"turbo-tasks-testing", "turbo-tasks-testing",
"turbo-unix-path", "turbo-unix-path",
"url",
"urlencoding", "urlencoding",
] ]
...@@ -10419,7 +10429,6 @@ dependencies = [ ...@@ -10419,7 +10429,6 @@ dependencies = [
"clap", "clap",
"codspeed-criterion-compat", "codspeed-criterion-compat",
"console-subscriber", "console-subscriber",
"dunce",
"futures", "futures",
"owo-colors", "owo-colors",
"regex", "regex",
...@@ -10915,7 +10924,6 @@ version = "0.1.0" ...@@ -10915,7 +10924,6 @@ version = "0.1.0"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"bincode 2.0.1", "bincode 2.0.1",
"dunce",
"rustc-hash 2.1.1", "rustc-hash 2.1.1",
"serde", "serde",
"serde_json", "serde_json",
......
use std::time::Duration; use std::{path::Path, time::Duration};
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use async_trait::async_trait; use async_trait::async_trait;
...@@ -45,9 +45,10 @@ use turbo_tasks::{ ...@@ -45,9 +45,10 @@ use turbo_tasks::{
}; };
use turbo_tasks_env::{EnvMap, ProcessEnv}; use turbo_tasks_env::{EnvMap, ProcessEnv};
use turbo_tasks_fs::{ use turbo_tasks_fs::{
DiskFileSystem, FileContent, FileSystem, FileSystemPath, VirtualFileSystem, invalidation, DiskFileSystem, FileContent, FileSystem, FileSystemPath, VirtualFileSystem,
canonicalize_to_rcstr, invalidation,
}; };
use turbo_unix_path::{join_path, unix_to_sys}; use turbo_unix_path::join_path;
use turbopack::{ use turbopack::{
ModuleAssetContext, evaluate_context::node_build_environment, externals_tracing_module_context, ModuleAssetContext, evaluate_context::node_build_environment, externals_tracing_module_context,
global_module_ids::get_global_module_id_strategy, transition::TransitionOptions, global_module_ids::get_global_module_id_strategy, transition::TransitionOptions,
...@@ -694,7 +695,7 @@ impl ProjectContainer { ...@@ -694,7 +695,7 @@ impl ProjectContainer {
.context("ProjectContainer need to be initialized with initialize()")?; .context("ProjectContainer need to be initialized with initialize()")?;
if let Some(root_path) = root_path { if let Some(root_path) = root_path {
new_options.root_path = root_path; new_options.root_path = canonicalize_to_rcstr(Path::new(&*root_path))?;
} }
if let Some(project_path) = project_path { if let Some(project_path) = project_path {
new_options.project_path = project_path; new_options.project_path = project_path;
...@@ -1094,23 +1095,6 @@ impl Project { ...@@ -1094,23 +1095,6 @@ impl Project {
DiskFileSystem::new(rcstr!("output"), *self.root_path) DiskFileSystem::new(rcstr!("output"), *self.root_path)
} }
#[turbo_tasks::function]
pub async fn dist_dir_absolute(&self) -> Result<Vc<RcStr>> {
let root_path = self.root_path.await?;
Ok(Vc::cell(
format!(
"{}{}{}",
root_path,
std::path::MAIN_SEPARATOR,
unix_to_sys(
&join_path(&self.project_path, &self.dist_dir)
.context("expected project_path to be inside of root_path")?
)
)
.into(),
))
}
#[turbo_tasks::function] #[turbo_tasks::function]
pub async fn node_root(self: Vc<Self>) -> Result<Vc<FileSystemPath>> { pub async fn node_root(self: Vc<Self>) -> Result<Vc<FileSystemPath>> {
let this = self.await?; let this = self.await?;
......
...@@ -25,6 +25,7 @@ tracing-subscriber = "0.3" ...@@ -25,6 +25,7 @@ tracing-subscriber = "0.3"
turbo-rcstr = { workspace = true } turbo-rcstr = { workspace = true }
turbo-tasks = { workspace = true } turbo-tasks = { workspace = true }
turbo-tasks-backend = { workspace = true } turbo-tasks-backend = { workspace = true }
turbo-tasks-fs = { workspace = true }
turbo-tasks-malloc = { workspace = true } turbo-tasks-malloc = { workspace = true }
turbopack-trace-utils = { workspace = true } turbopack-trace-utils = { workspace = true }
turbopack-node = { workspace = true, default-features = false, features = ["process_pool"]} turbopack-node = { workspace = true, default-features = false, features = ["process_pool"]}
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
#![feature(arbitrary_self_types)] #![feature(arbitrary_self_types)]
#![feature(arbitrary_self_types_pointers)] #![feature(arbitrary_self_types_pointers)]
use std::{str::FromStr, time::Instant}; use std::{path::Path, str::FromStr, time::Instant};
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use futures_util::{StreamExt, TryStreamExt}; use futures_util::{StreamExt, TryStreamExt};
...@@ -17,6 +17,7 @@ use turbo_tasks::{ ...@@ -17,6 +17,7 @@ use turbo_tasks::{
read_strongly_consistent_and_apply_effects, take_effects, read_strongly_consistent_and_apply_effects, take_effects,
}; };
use turbo_tasks_backend::TurboTasksBackend; use turbo_tasks_backend::TurboTasksBackend;
use turbo_tasks_fs::canonicalize_to_rcstr;
use turbo_tasks_malloc::TurboMalloc; use turbo_tasks_malloc::TurboMalloc;
pub async fn main_inner( pub async fn main_inner(
...@@ -31,6 +32,7 @@ pub async fn main_inner( ...@@ -31,6 +32,7 @@ pub async fn main_inner(
.with_context(|| format!("loading file at {}", path.display()))?; .with_context(|| format!("loading file at {}", path.display()))?;
let mut options: ProjectOptions = serde_json::from_reader(&mut file)?; let mut options: ProjectOptions = serde_json::from_reader(&mut file)?;
options.root_path = canonicalize_to_rcstr(Path::new(&*options.root_path))?;
if matches!(strategy, Strategy::Development { .. }) { if matches!(strategy, Strategy::Development { .. }) {
options.dev = true; options.dev = true;
......
...@@ -265,7 +265,7 @@ pub fn create_turbo_tasks( ...@@ -265,7 +265,7 @@ pub fn create_turbo_tasks(
let describe = cache_describe(next_version); let describe = cache_describe(next_version);
let version_info = git_version_info(&describe); let version_info = git_version_info(&describe);
let (backing_storage, cache_state) = turbo_backing_storage( let (backing_storage, cache_state) = turbo_backing_storage(
&output_path.join("cache/turbopack"), &output_path.join("cache").join("turbopack"),
&version_info, &version_info,
is_ci, is_ci,
is_short_session, is_short_session,
......
...@@ -31,7 +31,7 @@ bitflags = "1.3.2" ...@@ -31,7 +31,7 @@ bitflags = "1.3.2"
bytes = { workspace = true } bytes = { workspace = true }
concurrent-queue = { workspace = true } concurrent-queue = { workspace = true }
dashmap = { workspace = true } dashmap = { workspace = true }
dunce = { workspace = true } fs-err = { workspace = true }
futures = { workspace = true } futures = { workspace = true }
include_dir = { version = "0.7.3", features = ["nightly"] } include_dir = { version = "0.7.3", features = ["nightly"] }
indexmap = { workspace = true } indexmap = { workspace = true }
...@@ -44,6 +44,7 @@ rustc-hash = { workspace = true } ...@@ -44,6 +44,7 @@ rustc-hash = { workspace = true }
serde = { workspace = true, features = ["rc"] } serde = { workspace = true, features = ["rc"] }
serde_json = { workspace = true } serde_json = { workspace = true }
serde_path_to_error = { workspace = true } serde_path_to_error = { workspace = true }
smallvec = { workspace = true }
thiserror = { workspace = true } thiserror = { workspace = true }
tokio = { workspace = true } tokio = { workspace = true }
tracing = { workspace = true } tracing = { workspace = true }
...@@ -53,8 +54,12 @@ turbo-rcstr = { workspace = true } ...@@ -53,8 +54,12 @@ turbo-rcstr = { workspace = true }
turbo-tasks = { workspace = true } turbo-tasks = { workspace = true }
turbo-tasks-hash = { workspace = true } turbo-tasks-hash = { workspace = true }
turbo-unix-path = { workspace = true } turbo-unix-path = { workspace = true }
url = { workspace = true }
urlencoding = { workspace = true } urlencoding = { workspace = true }
[target.'cfg(windows)'.dependencies]
omnipath = "0.1.6"
[dev-dependencies] [dev-dependencies]
criterion = { workspace = true, features = ["async_tokio"] } criterion = { workspace = true, features = ["async_tokio"] }
rand = { workspace = true } rand = { workspace = true }
......
use std::path::Path;
pub use ::include_dir::{ pub use ::include_dir::{
include_dir, {self}, include_dir, {self},
}; };
...@@ -5,14 +7,18 @@ use anyhow::Result; ...@@ -5,14 +7,18 @@ use anyhow::Result;
use turbo_rcstr::RcStr; use turbo_rcstr::RcStr;
use turbo_tasks::Vc; use turbo_tasks::Vc;
use crate::{DiskFileSystem, FileSystem, embed::EmbeddedFileSystem}; use crate::{DiskFileSystem, FileSystem, canonicalize_to_rcstr, embed::EmbeddedFileSystem};
#[turbo_tasks::function] #[turbo_tasks::function]
pub async fn directory_from_relative_path( pub async fn directory_from_relative_path(
name: RcStr, name: RcStr,
path: RcStr, path: RcStr,
) -> Result<Vc<Box<dyn FileSystem>>> { ) -> Result<Vc<Box<dyn FileSystem>>> {
let disk_fs = DiskFileSystem::new(name, Vc::cell(path)); // note: canonicalizing inside of a turbo-tasks function causes an untracked read of the root
// directory path (e.g. if it's a symlink), but it's okay because this codepath is only used in
// development and the root directory is unlikely to move.
let root = canonicalize_to_rcstr(Path::new(&*path))?;
let disk_fs = DiskFileSystem::new(name, Vc::cell(root));
disk_fs.await?.start_watching(None).await?; disk_fs.await?.start_watching(None).await?;
Ok(Vc::upcast(disk_fs)) Ok(Vc::upcast(disk_fs))
......
use std::path::PathBuf;
use anyhow::{Context, Result};
use dunce::canonicalize;
use turbo_rcstr::RcStr;
use turbo_tasks::Vc;
use crate::{DiskFileSystem, File, FileContent, FileSystem};
#[turbo_tasks::function]
pub async fn content_from_relative_path(
package_path: RcStr,
path: RcStr,
) -> Result<Vc<FileContent>> {
let package_path = PathBuf::from(package_path);
let resolved_path = package_path.join(path);
let resolved_path =
canonicalize(&resolved_path).context("failed to canonicalize embedded file path")?;
let root_path = resolved_path.parent().unwrap();
let path = resolved_path.file_name().unwrap().to_str().unwrap();
let disk_fs = DiskFileSystem::new(
root_path.to_string_lossy().into(),
Vc::cell(root_path.to_string_lossy().into()),
);
disk_fs.await?.start_watching(None).await?;
let fs_path = disk_fs.root().await?.join(path)?;
Ok(fs_path.read())
}
#[turbo_tasks::function]
pub fn content_from_str(string: RcStr) -> Vc<FileContent> {
FileContent::Content(File::from(string)).cell()
}
/// Loads a file's content from disk and invalidates on change (debug builds).
///
/// In production, this will embed a file's content into the binary directly.
#[cfg(feature = "dynamic_embed_contents")]
#[macro_export]
macro_rules! embed_file {
($path:expr) => {{
// check that the file exists at compile time
let _ = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/", $path));
turbo_tasks_fs::embed::content_from_relative_path(
env!("CARGO_MANIFEST_DIR").into(),
$path.into(),
)
}};
}
/// Embeds a file's content into the binary (production).
#[cfg(not(feature = "dynamic_embed_contents"))]
#[macro_export]
macro_rules! embed_file {
($path:expr) => {
turbo_tasks_fs::embed::content_from_str(
include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/", $path)).into(),
)
};
}
pub(crate) mod dir; pub(crate) mod dir;
pub(crate) mod file;
pub(crate) mod fs; pub(crate) mod fs;
pub use self::{dir::*, file::*, fs::*}; pub use self::{dir::*, fs::*};
This diff is collapsed.
use std::{ use std::{
io::{self, ErrorKind}, io::{self, ErrorKind},
path::{Path, PathBuf}, path::Path,
}; };
use anyhow::{Context, Result, anyhow}; use anyhow::{Context, Result, anyhow};
use turbo_tasks::ResolvedVc; use turbo_tasks::ResolvedVc;
use url::Url;
use crate::{DiskFileSystem, FileSystemPath}; use crate::{DiskFileSystem, FileSystemPath};
...@@ -31,36 +32,10 @@ pub async fn uri_from_file(root: FileSystemPath, path: Option<&str>) -> Result<S ...@@ -31,36 +32,10 @@ pub async fn uri_from_file(root: FileSystemPath, path: Option<&str>) -> Result<S
None => root, None => root,
}; };
Ok(uri_from_path_buf(root_fs.to_sys_path(&path))) // `to_sys_path` returns a win32 path on Windows. `Url::from_file_path` can also handle
} // verbatim (`\\?\`-prefixed) disk and UNC paths, in case that conversion failed.
let sys_path = root_fs.to_sys_path(&path);
#[cfg(not(target_os = "windows"))] Ok(String::from(Url::from_file_path(&sys_path).map_err(
pub fn uri_from_path_buf(sys_path: PathBuf) -> String { |_| anyhow!("path {sys_path:?} cannot be converted to a file:// URI"),
use turbo_unix_path::sys_to_unix; )?))
let sys_path = sys_path.to_string_lossy();
format!(
"file://{}",
sys_to_unix(&sys_path)
.split('/')
.map(|s| urlencoding::encode(s))
.collect::<Vec<_>>()
.join("/")
)
}
#[cfg(target_os = "windows")]
pub fn uri_from_path_buf(sys_path: PathBuf) -> String {
let raw_path = sys_path.to_string_lossy().to_string();
let normalized_path = raw_path.replace('\\', "/");
let mut segments = normalized_path.split('/');
let first = segments.next().unwrap_or_default(); // e.g., "C:"
let encoded_path = std::iter::once(first.to_string()) // keep "C:" intact
.chain(segments.map(|s| urlencoding::encode(s).into_owned()))
.collect::<Vec<_>>()
.join("/");
format!("file:///{}", encoded_path)
} }
use std::path::{Path, PathBuf};
/// Converts `path` into the verbatim, drive-letter-case-folded representation used internally by
/// [`crate::DiskFileSystem`] on Windows.
///
/// This is the purely lexical counterpart to [`std::fs::canonicalize`]: `to_verbatim`
/// (`GetFullPathNameW`) makes the path absolute and verbatim (`\\?\`-prefixed) but, unlike
/// `canonicalize`, does not touch the disk — it resolves neither symlinks nor 8.3 short names. The
/// drive letter is then upper-cased to match the form `GetFinalPathNameByHandle` (and thus
/// `canonicalize`) produces.
///
/// No-op on non-Windows platforms (returns `path` unchanged).
pub fn to_verbatim_with_case_folded_disk(path: &Path) -> std::io::Result<PathBuf> {
#[cfg(windows)]
{
use std::{
ffi::OsString,
os::windows::ffi::{OsStrExt, OsStringExt},
path::{Component, Prefix},
};
use omnipath::WinPathExt;
// `to_verbatim` guarantees an absolute, verbatim path from here on, so there's no
// non-verbatim case to guard against below.
let path = path.to_verbatim()?;
// Only `\\?\C:\...` (`VerbatimDisk`) paths carry a drive letter; verbatim UNC
// (`\\?\UNC\...`) and other verbatim device paths (`\\?\prefix`) don't, so there's nothing
// to case-fold for those.
//
// We can't read the letter from `VerbatimDisk(disk)` because that value is already
// normalized to uppercase, so it wouldn't tell us whether the underlying path needs
// rewriting.
let is_verbatim_disk = matches!(
path.components().next(),
Some(Component::Prefix(prefix)) if matches!(prefix.kind(), Prefix::VerbatimDisk(_))
);
if !is_verbatim_disk {
return Ok(path);
}
// The layout is `\\?\C:\...`, so the drive letter is the 5th UTF-16 code unit (index 4)
// and is guaranteed to be a-z or A-Z.
if let Some(disk) = path.as_os_str().encode_wide().nth(4).map(|disk| disk as u8)
&& disk.is_ascii_lowercase()
{
// we must encode/decode because OsString's internal encoding is opaque/unstable
let mut wide: Vec<u16> = path.as_os_str().encode_wide().collect();
wide[4] = u16::from(disk.to_ascii_uppercase());
return Ok(PathBuf::from(OsString::from_wide(&wide)));
}
Ok(path)
}
#[cfg(not(windows))]
Ok(path.to_path_buf())
}
#[cfg(all(test, windows))]
mod tests {
use std::path::Path;
use super::*;
fn fold(path: &str) -> String {
to_verbatim_with_case_folded_disk(Path::new(path))
.unwrap()
.to_str()
.unwrap()
.to_owned()
}
#[test]
fn upper_cases_lower_drive_letter() {
// `to_verbatim` passes an already-verbatim path through unchanged, so these exercise the
// drive-letter case-folding in isolation.
assert_eq!(fold(r"\\?\c:\foo\bar"), r"\\?\C:\foo\bar");
assert_eq!(fold(r"\\?\z:\"), r"\\?\Z:\");
}
#[test]
fn ignores_verbatim_paths_without_a_drive() {
assert_eq!(
fold(r"\\?\UNC\server\share\foo"),
r"\\?\UNC\server\share\foo"
);
}
}
...@@ -10,6 +10,7 @@ pub fn memory_pressure() -> Option<u8> { ...@@ -10,6 +10,7 @@ pub fn memory_pressure() -> Option<u8> {
platform::memory_pressure() platform::memory_pressure()
} }
#[allow(dead_code)]
fn clamp_percent(value: f64) -> u8 { fn clamp_percent(value: f64) -> u8 {
if !value.is_finite() { if !value.is_finite() {
return 0; return 0;
......
...@@ -41,7 +41,6 @@ anyhow = { workspace = true } ...@@ -41,7 +41,6 @@ anyhow = { workspace = true }
bincode = { workspace = true } bincode = { workspace = true }
clap = { workspace = true, features = ["derive", "env"] } clap = { workspace = true, features = ["derive", "env"] }
console-subscriber = { workspace = true, optional = true } console-subscriber = { workspace = true, optional = true }
dunce = { workspace = true }
futures = { workspace = true } futures = { workspace = true }
owo-colors = { workspace = true } owo-colors = { workspace = true }
rustc-hash = { workspace = true } rustc-hash = { workspace = true }
......
...@@ -2,10 +2,9 @@ use std::{env::current_dir, path::PathBuf}; ...@@ -2,10 +2,9 @@ use std::{env::current_dir, path::PathBuf};
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use bincode::{Decode, Encode}; use bincode::{Decode, Encode};
use dunce::canonicalize;
use turbo_rcstr::{RcStr, rcstr}; use turbo_rcstr::{RcStr, rcstr};
use turbo_tasks::{Vc, trace::TraceRawVcs}; use turbo_tasks::{Vc, trace::TraceRawVcs};
use turbo_tasks_fs::{DiskFileSystem, FileSystem}; use turbo_tasks_fs::{DiskFileSystem, FileSystem, canonicalize_to_rcstr};
#[turbo_tasks::task_input] #[turbo_tasks::task_input]
#[derive(Clone, Debug, Hash, PartialEq, Eq, TraceRawVcs, Encode, Decode)] #[derive(Clone, Debug, Hash, PartialEq, Eq, TraceRawVcs, Encode, Decode)]
...@@ -27,21 +26,14 @@ pub fn normalize_dirs( ...@@ -27,21 +26,14 @@ pub fn normalize_dirs(
project_dir: &Option<PathBuf>, project_dir: &Option<PathBuf>,
root_dir: &Option<PathBuf>, root_dir: &Option<PathBuf>,
) -> Result<NormalizedDirs> { ) -> Result<NormalizedDirs> {
let project_dir: RcStr = project_dir let project_dir = match project_dir.as_ref() {
.as_ref() Some(dir) => canonicalize_to_rcstr(dir),
.map(canonicalize) None => canonicalize_to_rcstr(&current_dir().context("current directory can't be found")?),
.unwrap_or_else(current_dir) }
.context("project directory can't be found")? .context("project directory can't be found")?;
.to_str()
.context("project directory contains invalid characters")?
.into();
let root_dir = match root_dir.as_ref() { let root_dir = match root_dir.as_ref() {
Some(root) => canonicalize(root) Some(root) => canonicalize_to_rcstr(root).context("root directory can't be found")?,
.context("root directory can't be found")?
.to_str()
.context("root directory contains invalid characters")?
.into(),
None => project_dir.clone(), None => project_dir.clone(),
}; };
......
...@@ -7,9 +7,7 @@ use serde::{Deserialize, Serialize}; ...@@ -7,9 +7,7 @@ use serde::{Deserialize, Serialize};
use serde_json::value::RawValue; use serde_json::value::RawValue;
use turbo_rcstr::RcStr; use turbo_rcstr::RcStr;
use turbo_tasks::{ResolvedVc, turbofmt}; use turbo_tasks::{ResolvedVc, turbofmt};
use turbo_tasks_fs::{ use turbo_tasks_fs::{DiskFileSystem, FileContent, FileSystemPath, rope::Rope};
DiskFileSystem, FileContent, FileSystemPath, rope::Rope, util::uri_from_path_buf,
};
use url::Url; use url::Url;
use crate::SOURCE_URL_PROTOCOL_STR; use crate::SOURCE_URL_PROTOCOL_STR;
...@@ -281,7 +279,14 @@ pub async fn absolute_fileify_source_map( ...@@ -281,7 +279,14 @@ pub async fn absolute_fileify_source_map(
transform_relative_files(map, &context_path, |context_fs, src_rest| { transform_relative_files(map, &context_path, |context_fs, src_rest| {
let path = context_path.join(src_rest)?; let path = context_path.join(src_rest)?;
Ok(uri_from_path_buf(context_fs.to_sys_path(&path))) // `to_sys_path` returns a win32 path on Windows. `Url::from_file_path` can also handle
// verbatim (`\\?\`-prefixed) disk and UNC paths, in case that conversion failed.
let sys_path = context_fs.to_sys_path(&path);
Ok(Url::from_file_path(&sys_path)
.map_err(|()| {
anyhow::anyhow!("path {sys_path:?} cannot be converted to a file:// URI")
})?
.into())
}) })
.await .await
} }
......
...@@ -7,6 +7,7 @@ use clap::Parser; ...@@ -7,6 +7,7 @@ use clap::Parser;
use tracing_subscriber::{Registry, layer::SubscriberExt, util::SubscriberInitExt}; use tracing_subscriber::{Registry, layer::SubscriberExt, util::SubscriberInitExt};
use turbo_tasks::TurboTasks; use turbo_tasks::TurboTasks;
use turbo_tasks_backend::{BackendOptions, TurboTasksBackend, noop_backing_storage}; use turbo_tasks_backend::{BackendOptions, TurboTasksBackend, noop_backing_storage};
use turbo_tasks_fs::canonicalize_to_rcstr;
use turbo_tasks_malloc::TurboMalloc; use turbo_tasks_malloc::TurboMalloc;
use turbopack_nft::nft::node_file_trace; use turbopack_nft::nft::node_file_trace;
use turbopack_trace_utils::{ use turbopack_trace_utils::{
...@@ -89,9 +90,10 @@ async fn main_inner(args: Arguments) -> Result<()> { ...@@ -89,9 +90,10 @@ async fn main_inner(args: Arguments) -> Result<()> {
noop_backing_storage(), noop_backing_storage(),
)); ));
let project_root = canonicalize_to_rcstr(&current_dir()?)?;
tt.run_once(async move { tt.run_once(async move {
node_file_trace( node_file_trace(
current_dir()?.to_str().unwrap().into(), project_root,
args.entry.into(), args.entry.into(),
args.graph, args.graph,
args.show_issues, args.show_issues,
......
...@@ -19,7 +19,6 @@ turbopack-node = { workspace = true, default-features = false, features = ["proc ...@@ -19,7 +19,6 @@ turbopack-node = { workspace = true, default-features = false, features = ["proc
[dev-dependencies] [dev-dependencies]
anyhow = { workspace = true } anyhow = { workspace = true }
bincode = { workspace = true } bincode = { workspace = true }
dunce = { workspace = true }
rustc-hash = { workspace = true } rustc-hash = { workspace = true }
serde = { workspace = true } serde = { workspace = true }
serde_json = { workspace = true } serde_json = { workspace = true }
......
...@@ -5,11 +5,10 @@ ...@@ -5,11 +5,10 @@
mod util; mod util;
use std::{env, path::PathBuf}; use std::{env, fs::canonicalize, path::PathBuf};
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use bincode::{Decode, Encode}; use bincode::{Decode, Encode};
use dunce::canonicalize;
use serde::Deserialize; use serde::Deserialize;
use tracing_subscriber::{Registry, layer::SubscriberExt, util::SubscriberInitExt}; use tracing_subscriber::{Registry, layer::SubscriberExt, util::SubscriberInitExt};
use turbo_rcstr::{RcStr, rcstr}; use turbo_rcstr::{RcStr, rcstr};
......
...@@ -3,10 +3,9 @@ ...@@ -3,10 +3,9 @@
mod util; mod util;
use std::{collections::VecDeque, fs, io, path::PathBuf}; use std::{collections::VecDeque, fs, fs::canonicalize, io, path::PathBuf};
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use dunce::canonicalize;
use rustc_hash::FxHashSet; use rustc_hash::FxHashSet;
use serde::Deserialize; use serde::Deserialize;
use serde_json::json; use serde_json::json;
......
use std::{path::PathBuf, sync::LazyLock}; use std::{fs::canonicalize, path::PathBuf, sync::LazyLock};
use dunce::canonicalize;
use turbo_rcstr::RcStr; use turbo_rcstr::RcStr;
/// The turbo repo root. Should be used as the root when building with turbopack /// The turbo repo root. Should be used as the root when building with turbopack
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment