Unverified Commit 729ff712 authored by Niklas Mischkulnig's avatar Niklas Mischkulnig Committed by GitHub

refactor(turbopack): drop the generic on `new_atom`, and stop it copying an owned string (#98520)

### What?

Follow-up to the review comments on #98497.

`new_atom` loses its generic parameter and takes a `Cow<'_, str>`, which
as a side effect stops it
copying a string it was handed ownership of. `FileSystemPath`'s
`get_relative_request_to` stops
hand-rolling a conversion the standard `From` impl already does, and the
two tests added for it in
`path.rs` are replaced by one that covers what that layer actually adds.

### Why?

@bgw noted that `get_relative_request_to` was matching on the `Cow` it
gets back only to call
`.into()` in both arms, and wondered whether `impl From<Cow<'_, str>>
for RcStr` was missing a fast
path for the owned case.

It was not missing one, and it never needed one — but an owned string
*was* being copied, for a
different reason. `new_atom` was generic over `T: AsRef<str> +
Into<String>` and opened with
`let text = text.as_ref();`, which shadows the owned value. From that
point on only a `&str` was in
scope, so building the `Box<str>` allocated a fresh buffer and copied
into it, dropping the `String`
that a `Cow::Owned` had just handed over.

So this is not a fast path being added. It is an accidental
pessimisation being removed: the generic
plus the `as_ref()` were exactly what stopped the standard conversion
from doing the right thing on
its own. Whether that copy ever mattered in practice is a separate
question, and one this does not
attempt to answer.

The other two comments were about the tests: they re-ran the case table
that already lives in
`turbo-unix-path`, which is where the path computation itself is
implemented and tested — "60 LOC to
test a 3 LOC branch".

### How?

`new_atom` takes a `Cow<'_, str>`. All five of its callers already
passed one, so nothing else
changes, and ownership now survives to where the `Box<str>` is built —
`Box<str>`'s own
`From<Cow<'_, str>>` moves an owned buffer in and copies only a borrowed
one. The inline-atom path is
untouched, since it reads through a `&str` either way and does not
allocate.

Contents-based assertions cannot distinguish a moved buffer from a
copied one, so the test compares
the data pointer before and after the conversion. It uses a string
longer than `MAX_INLINE_LEN`,
because an inline atom stores its bytes in the tagged value by design
and could never preserve a
pointer, and it asserts the atom really is dynamic so it cannot pass
vacuously if that boundary ever
moves. It fails on `canary` today.

`get_relative_request_to` collapses to a single `.into()`. Its sibling
`get_relative_path_to` keeps
its match on purpose: one arm uses `std::ptr::eq` to hand back a clone
of the `RcStr` the path
already holds, which the blanket conversion cannot do. That arm predates
#98497 and is an
optimisation rather than the redundancy that was flagged.

For the tests, `turbo-unix-path` remains the place where the relative
path computation is covered,
untouched. What is left here is only what this layer adds on top: that
each method reaches for the
form it names, and that neither returns a path across filesystems.
Keeping the first of those is a
judgement call — a method delegating to the wrong free function is
precisely the bug #98497 existed
to fix, and it is invisible to the lower-level table — so it is asserted
with one descendant pair
rather than a table. Happy to drop it entirely if you would rather.

To check that the smaller test still earns its place,
`get_relative_request_to` was temporarily
pointed at the plain-path function; the test fails as it should.

<!-- NEXT_JS_LLM -->

<!-- fleet 3acc7e63-52a8-4472-89cd-b1bbb8ad59fa -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent c2e4f05f
use std::{num::NonZeroU8, ptr::NonNull}; use std::{borrow::Cow, num::NonZeroU8, ptr::NonNull};
use triomphe::Arc; use triomphe::Arc;
...@@ -47,9 +47,11 @@ pub unsafe fn restore_arc(v: TaggedValue) -> Arc<DynamicPrehashedString> { ...@@ -47,9 +47,11 @@ pub unsafe fn restore_arc(v: TaggedValue) -> Arc<DynamicPrehashedString> {
/// This can create any kind of [Atom], although this lives in the `dynamic` /// This can create any kind of [Atom], although this lives in the `dynamic`
/// module. /// module.
pub(crate) fn new_atom<T: AsRef<str> + Into<String>>(text: T) -> RcStr { ///
let text = text.as_ref(); /// Takes a [`Cow`] rather than a `&str` so that an already-owned string can be moved into the atom
if is_atom_inlineable(text) { /// instead of being copied into a new allocation.
pub(crate) fn new_atom(text: Cow<'_, str>) -> RcStr {
if is_atom_inlineable(&text) {
let len = text.len(); let len = text.len();
// INLINE_TAG ensures this is never zero // INLINE_TAG ensures this is never zero
let tag = INLINE_TAG_INIT | ((len as u8) << LEN_OFFSET); let tag = INLINE_TAG_INIT | ((len as u8) << LEN_OFFSET);
...@@ -64,7 +66,8 @@ pub(crate) fn new_atom<T: AsRef<str> + Into<String>>(text: T) -> RcStr { ...@@ -64,7 +66,8 @@ pub(crate) fn new_atom<T: AsRef<str> + Into<String>>(text: T) -> RcStr {
let prehashed = DynamicPrehashedString { let prehashed = DynamicPrehashedString {
// NOTE: This will capture as a Box<str> which will essentially // NOTE: This will capture as a Box<str> which will essentially
// `shrink_to_fit` the bytes. // `shrink_to_fit` the bytes. `Box<str>`'s own `From<Cow<'_, str>>` impl already moves an
// owned string's buffer in rather than copying it.
value: text.into(), value: text.into(),
hash, hash,
}; };
......
...@@ -718,6 +718,37 @@ mod tests { ...@@ -718,6 +718,37 @@ mod tests {
use super::*; use super::*;
/// An `RcStr` built from an owned string takes over its buffer instead of copying it. Only a
/// non-inline string can show this: an inline atom stores its bytes in the tagged value, so it
/// has no buffer to take over.
#[test]
fn from_owned_cow_takes_over_the_buffer() {
let mut string = "a string far too long to be stored inline".to_string();
// So that `String::into_boxed_str` has no reason to reallocate.
string.shrink_to_fit();
let ptr = string.as_ptr();
let rc_str = RcStr::from(Cow::Owned(string));
assert!(
rc_str.tag() == DYNAMIC_TAG,
"the test string must not be inlineable"
);
assert_eq!(
rc_str.as_str().as_ptr(),
ptr,
"the owned buffer should have been taken over, not copied"
);
}
/// There is no buffer to take over here, so this only pins the contents.
#[test]
fn from_borrowed_cow_copies() {
let string = "a string far too long to be stored inline";
let rc_str = RcStr::from(Cow::Borrowed(string));
assert_eq!(rc_str.as_str(), string);
}
#[test] #[test]
fn test_refcount() { fn test_refcount() {
fn refcount(str: &RcStr) -> usize { fn refcount(str: &RcStr) -> usize {
......
...@@ -131,10 +131,7 @@ impl FileSystemPath { ...@@ -131,10 +131,7 @@ impl FileSystemPath {
return None; return None;
} }
Some(match get_relative_request_to(&self.path, &other.path) { Some(get_relative_request_to(&self.path, &other.path).into())
Cow::Borrowed(path) => path.into(),
Cow::Owned(path) => path.into(),
})
} }
/// Returns the final component of the FileSystemPath, or an empty string /// Returns the final component of the FileSystemPath, or an empty string
...@@ -794,63 +791,11 @@ mod tests { ...@@ -794,63 +791,11 @@ mod tests {
use super::*; use super::*;
use crate::VirtualFileSystem; use crate::VirtualFileSystem;
/// Builds two paths on the same filesystem and returns them. /// `turbo-unix-path` covers how the relative path itself is computed, so this only pins what
fn paths_on_one_fs( /// this layer adds: that each method reaches for the form it names, and that neither crosses
fs: ResolvedVc<Box<dyn FileSystem>>, /// between filesystems.
from: &str,
target: &str,
) -> (FileSystemPath, FileSystemPath) {
(
FileSystemPath::new_normalized_unchecked(fs, from.into()),
FileSystemPath::new_normalized_unchecked(fs, target.into()),
)
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_relative_path_to() {
let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new(
BackendOptions::default(),
noop_backing_storage(),
));
tt.run_once(async move {
let fs = Vc::upcast::<Box<dyn FileSystem>>(VirtualFileSystem::new())
.to_resolved()
.await?;
for (from, target, expected) in [
("a/b/c", "a/b/c", "."),
("a/c/d", "a/b/c", "../../b/c"),
("", "a/b/c", "a/b/c"),
("a/b", "a/b/c", "c"),
("a/b/c", "", "../../.."),
("a/b/c", "c/b/a", "../../../c/b/a"),
] {
let (from_path, target_path) = paths_on_one_fs(fs, from, target);
assert_eq!(
from_path.get_relative_path_to(&target_path).as_deref(),
Some(expected),
"{from:?} -> {target:?}"
);
}
// A path on another filesystem is not reachable relatively.
let (from_path, _) = paths_on_one_fs(fs, "a/b", "a/b/c");
let other_fs = Vc::upcast::<Box<dyn FileSystem>>(VirtualFileSystem::new())
.to_resolved()
.await?;
let on_other_fs = FileSystemPath::new_normalized_unchecked(other_fs, rcstr!("a/b/c"));
assert_eq!(from_path.get_relative_path_to(&on_other_fs), None);
anyhow::Ok(())
})
.await
.unwrap();
}
/// The cases this covers are the ones `get_relative_path_to` was asserted against before it
/// stopped prefixing `./`, so they pin that the request form still produces them.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)] #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_relative_request_to() { async fn get_relative_to() {
let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new(
BackendOptions::default(), BackendOptions::default(),
noop_backing_storage(), noop_backing_storage(),
...@@ -859,30 +804,21 @@ mod tests { ...@@ -859,30 +804,21 @@ mod tests {
let fs = Vc::upcast::<Box<dyn FileSystem>>(VirtualFileSystem::new()) let fs = Vc::upcast::<Box<dyn FileSystem>>(VirtualFileSystem::new())
.to_resolved() .to_resolved()
.await?; .await?;
let dir = FileSystemPath::new_normalized_unchecked(fs, rcstr!("a/b"));
let file = FileSystemPath::new_normalized_unchecked(fs, rcstr!("a/b/c.js"));
for (from, target, expected) in [ assert_eq!(dir.get_relative_path_to(&file).as_deref(), Some("c.js"));
("a/b/c", "a/b/c", "."), assert_eq!(
("a/c/d", "a/b/c", "../../b/c"), dir.get_relative_request_to(&file).as_deref(),
("", "a/b/c", "./a/b/c"), Some("./c.js")
("a/b", "a/b/c", "./c"), );
("a/b/c", "", "../../.."),
("a/b/c", "c/b/a", "../../../c/b/a"),
] {
let (from_path, target_path) = paths_on_one_fs(fs, from, target);
assert_eq!(
from_path.get_relative_request_to(&target_path).as_deref(),
Some(expected),
"{from:?} -> {target:?}"
);
}
// A path on another filesystem is not reachable relatively.
let (from_path, _) = paths_on_one_fs(fs, "a/b", "a/b/c");
let other_fs = Vc::upcast::<Box<dyn FileSystem>>(VirtualFileSystem::new()) let other_fs = Vc::upcast::<Box<dyn FileSystem>>(VirtualFileSystem::new())
.to_resolved() .to_resolved()
.await?; .await?;
let on_other_fs = FileSystemPath::new_normalized_unchecked(other_fs, rcstr!("a/b/c")); let elsewhere = FileSystemPath::new_normalized_unchecked(other_fs, rcstr!("a/b/c.js"));
assert_eq!(from_path.get_relative_request_to(&on_other_fs), None); assert_eq!(dir.get_relative_path_to(&elsewhere), None);
assert_eq!(dir.get_relative_request_to(&elsewhere), None);
anyhow::Ok(()) anyhow::Ok(())
}) })
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment