Unverified Commit e648495f authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Replace lzzzz with lz4_flex (#98799)

## What?

Replace the direct `lzzzz` dependency used by Turbopack persistence with
`lz4_flex` 0.14. The migration covers persisted blocks, the SST
inspection utility, and optional compressed-size diagnostics in the task
backend.

## Why?

This removes the direct C/liblz4-backed codec dependency from these
paths in favor of a Rust implementation. Persistence caches are scoped
to the Next.js and Turbopack version, so this does not require a
compatibility path for caches written by earlier versions.

## How?

Persistence continues to use raw LZ4 blocks and the existing compression
selection and checksum layout.

A large thread-local `CompressTable` is reused across blocks. Starting
with the large table costs about 8 KiB more per compression thread than
the small table, but improves compression speed and produces LZ4 streams
that are faster for `lz4_flex` to decode.

The compressor retains the existing caller-owned buffer API. It clears
and reserves the output vector, lets `lz4_flex` write directly into the
uninitialized backing allocation, and exposes only the initialized byte
count returned by the compressor. This avoids both per-call output
allocations and pre-zeroing the maximum output size. Zstandard behavior
is unchanged.

### Benchmark vs `lzzzz`

The final PR configuration was benchmarked against `lzzzz` 2.0.0 using
the repository-pinned Rust nightly. The benchmark used source-code
blocks sampled from this repository, pinned each process to one physical
CPU, measured thread CPU time, and alternated codec order within every
paired round. The table reports the median across four physical cores
and 20 paired rounds per process (480 paired rounds total).

Each decoder read the output produced by its own encoder, matching how a
same-version persistence cache is written and later read.

| Source corpus | Block size | Compression | Decompression | Compressed
size |
| --- | ---: | ---: | ---: | ---: |
| Mixed | 12 KiB | **7.1% faster** | 9.9% slower | 0.08% larger |
| Rust | 12 KiB | **5.5% faster** | 10.4% slower | 0.09% larger |
| TypeScript | 12 KiB | **7.4% faster** | 7.4% slower | 0.23% larger |
| Mixed | 16 KiB | **5.9% faster** | 8.8% slower | 0.17% smaller |
| Rust | 16 KiB | **4.6% faster** | 10.0% slower | 0.12% smaller |
| TypeScript | 16 KiB | **6.7% faster** | 6.1% slower | 0.02% smaller |

For these 12–16 KiB source blocks, `lz4_flex` compression was **4.6–7.4%
faster**, decompression was **6.1–10.4% slower**, and compressed size
changed by **-0.17% to +0.23%**.

The benchmark was stable enough to support those codec-level ranges:
wall time divided by thread CPU time was 1.000073 at the median and
1.001883 at p99. The median difference between the two codec orders was
0.38 percentage points for compression and 1.45 points for
decompression.

A separate decoder-only comparison fed identical `lzzzz`-encoded source
blocks to both decoders; there, `lz4_flex` was 17–21% slower. The
smaller regression in the table above comes from the large Flex
compression table producing a different valid LZ4 token stream. The PR
already uses `lz4_flex`'s supported unsafe pointer implementation with
validation; a local experiment that removed validation recovered only
5–6% and remained slower than liblz4, so this PR does not weaken decoder
safety.

These are codec microbenchmarks, not an end-to-end Turbopack speedup
claim.

## Verification

- `cargo fmt --all -- --check`
- `cargo clippy -p turbo-persistence --all-targets -- -D warnings`
- `cargo test -p turbo-persistence -- --test-threads=1` (110 passed)
- `cargo check -p turbo-tasks-backend --features
print_cache_item_size_with_compressed`
- Pinned-nightly paired codec benchmark: 24 processes / 480 paired
rounds

<!-- NEXT_JS_LLM -->

<!-- fleet 8f69f876-cb41-4fa2-ab60-dc8ccea88055 -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarBenjamin Woodruff ‮ <180404+bgw@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent d0df62e8
...@@ -3795,13 +3795,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index" ...@@ -3795,13 +3795,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154" checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154"
[[package]] [[package]]
name = "lzzzz" name = "lz4_flex"
version = "2.0.0" version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb42ba2edf1f0491b2e2c85fd49cf2a7adad3cab92ea0e29bff82d6fdafcefda" checksum = "ecbdfe44b1bd960b68170b417450a628c43f7cf56bb3c5317e61cb230ee7f226"
dependencies = [
"cc",
]
[[package]] [[package]]
name = "mach2" name = "mach2"
...@@ -9085,7 +9082,7 @@ dependencies = [ ...@@ -9085,7 +9082,7 @@ dependencies = [
"either", "either",
"fs-err", "fs-err",
"jiff", "jiff",
"lzzzz", "lz4_flex",
"memmap2", "memmap2",
"nohash-hasher", "nohash-hasher",
"parking_lot", "parking_lot",
...@@ -9205,7 +9202,7 @@ dependencies = [ ...@@ -9205,7 +9202,7 @@ dependencies = [
"indexmap 2.14.0", "indexmap 2.14.0",
"indoc", "indoc",
"jiff", "jiff",
"lzzzz", "lz4_flex",
"parking_lot", "parking_lot",
"rand 0.10.1", "rand 0.10.1",
"regex", "regex",
......
...@@ -291,7 +291,10 @@ lightningcss = { version = "1.0.0-alpha.72", features = [ ...@@ -291,7 +291,10 @@ lightningcss = { version = "1.0.0-alpha.72", features = [
"into_owned", "into_owned",
"browserslist", "browserslist",
] } ] }
lzzzz = "2.0.0" lz4_flex = { version = "0.14.0", default-features = false, features = [
"std",
"checked-decode",
] }
markdown = "1.0.0-alpha.18" markdown = "1.0.0-alpha.18"
memchr = "2.8.0" memchr = "2.8.0"
mime = "0.3.16" mime = "0.3.16"
......
...@@ -32,9 +32,9 @@ setup_wasi_env() { ...@@ -32,9 +32,9 @@ setup_wasi_env() {
rustup target add wasm32-wasip1-threads || return 1 rustup target add wasm32-wasip1-threads || return 1
# `lzzzz` (LZ4, via turbo-persistence) and `zstd-sys` have C build scripts, so a WASI clang and # `zstd-sys` has a C build script, so a WASI clang and sysroot are required on top of the Rust
# sysroot are required on top of the Rust target. The SDK build must match the host running the # target. The SDK build must match the host running the compiler; an x86_64 clang on arm64 fails
# compiler; an x86_64 clang on arm64 fails with "Exec format error". # with "Exec format error".
local wasi_sdk_version=33.0 wasi_sdk_arch wasi_sdk_sha256 local wasi_sdk_version=33.0 wasi_sdk_arch wasi_sdk_sha256
case "$(uname -m)" in case "$(uname -m)" in
x86_64 | amd64) x86_64 | amd64)
......
...@@ -22,7 +22,7 @@ dashmap = { workspace = true} ...@@ -22,7 +22,7 @@ dashmap = { workspace = true}
either = { workspace = true } either = { workspace = true }
fs-err = { workspace = true } fs-err = { workspace = true }
jiff = { version = "0.2.10", features = ["serde"] } jiff = { version = "0.2.10", features = ["serde"] }
lzzzz = { workspace = true } lz4_flex = { workspace = true }
memmap2 = "0.9.5" memmap2 = "0.9.5"
nohash-hasher = { workspace = true } nohash-hasher = { workspace = true }
parking_lot = { workspace = true } parking_lot = { workspace = true }
......
...@@ -17,7 +17,7 @@ use std::{ ...@@ -17,7 +17,7 @@ use std::{
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use byteorder::{BE, ReadBytesExt}; use byteorder::{BE, ReadBytesExt};
use fs_err::{self as fs, File}; use fs_err::{self as fs, File};
use lzzzz::lz4::decompress; use lz4_flex::block::decompress_into;
use memmap2::Mmap; use memmap2::Mmap;
use turbo_persistence::{ use turbo_persistence::{
BLOCK_HEADER_SIZE, Compression, MAX_INLINE_VALUE_SIZE, checksum_block, BLOCK_HEADER_SIZE, Compression, MAX_INLINE_VALUE_SIZE, checksum_block,
...@@ -349,7 +349,7 @@ fn read_block( ...@@ -349,7 +349,7 @@ fn read_block(
let mut buffer = vec![0u8; uncompressed_length as usize]; let mut buffer = vec![0u8; uncompressed_length as usize];
let bytes_written = match compression { let bytes_written = match compression {
Compression::Lz4 => { Compression::Lz4 => {
decompress(compressed_data, &mut buffer).context("LZ4 decompression failed")? decompress_into(compressed_data, &mut buffer).context("LZ4 decompression failed")?
} }
Compression::Zstd3 => zstd::bulk::decompress_to_buffer(compressed_data, &mut buffer) Compression::Zstd3 => zstd::bulk::decompress_to_buffer(compressed_data, &mut buffer)
.context("zstd decompression failed")?, .context("zstd decompression failed")?,
......
use std::{cell::RefCell, mem::MaybeUninit, rc::Rc, sync::Arc}; use std::{cell::RefCell, mem::MaybeUninit, rc::Rc, sync::Arc};
use anyhow::{Context, Result, ensure}; use anyhow::{Context, Result, ensure};
use lzzzz::lz4::{self, decompress}; use lz4_flex::block::{
CompressTable, compress_into_with_table, decompress_into, get_maximum_output_size,
};
/// Compression algorithm used for a family's SST blocks and blob values. /// Compression algorithm used for a family's SST blocks and blob values.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] #[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
...@@ -15,6 +17,10 @@ pub enum Compression { ...@@ -15,6 +17,10 @@ pub enum Compression {
} }
thread_local! { thread_local! {
/// Reuse lz4_flex's large hash table across independent blocks. Starting large improves
/// compression speed and produces faster-to-decode streams for typical persistence blocks.
static LZ4_COMPRESS_TABLE: RefCell<CompressTable> = RefCell::new(CompressTable::large());
/// Zstd decompression contexts are reusable and relatively expensive to create. Keep one per /// Zstd decompression contexts are reusable and relatively expensive to create. Keep one per
/// worker thread to avoid allocation on every block read without a global lock. /// worker thread to avoid allocation on every block read without a global lock.
static ZSTD_DECOMPRESSOR: RefCell<zstd::bulk::Decompressor<'static>> = RefCell::new( static ZSTD_DECOMPRESSOR: RefCell<zstd::bulk::Decompressor<'static>> = RefCell::new(
...@@ -35,7 +41,7 @@ fn decompress_block( ...@@ -35,7 +41,7 @@ fn decompress_block(
zero-copy mmap path" zero-copy mmap path"
); );
let bytes_written = match compression { let bytes_written = match compression {
Compression::Lz4 => decompress(block, dest).map_err(anyhow::Error::from), Compression::Lz4 => decompress_into(block, dest).map_err(anyhow::Error::from),
Compression::Zstd3 => ZSTD_DECOMPRESSOR.with_borrow_mut(|decompressor| { Compression::Zstd3 => ZSTD_DECOMPRESSOR.with_borrow_mut(|decompressor| {
decompressor decompressor
.decompress_to_buffer(block, dest) .decompress_to_buffer(block, dest)
...@@ -116,16 +122,30 @@ impl Compressor { ...@@ -116,16 +122,30 @@ impl Compressor {
Ok(Self { compression, zstd }) Ok(Self { compression, zstd })
} }
/// Compresses `block` into reusable storage, replacing its contents.
#[tracing::instrument(level = "trace", skip_all)] #[tracing::instrument(level = "trace", skip_all)]
pub(crate) fn compress_into_buffer( pub(crate) fn compress_into_buffer(
&mut self, &mut self,
block: &[u8], block: &[u8],
buffer: &mut Vec<u8>, buffer: &mut Vec<u8>,
) -> Result<()> { ) -> Result<()> {
buffer.clear();
match self.compression { match self.compression {
Compression::Lz4 => { Compression::Lz4 => {
lz4::compress_to_vec(block, buffer, lz4::ACC_LEVEL_DEFAULT) let max_output_size = get_maximum_output_size(block.len());
buffer.reserve(max_output_size);
// SAFETY: `reserve` guarantees at least `max_output_size` writable bytes from
// `as_mut_ptr`. lz4_flex is built without `safe-encode`; its `SliceSink` explicitly
// supports possibly uninitialized output and initializes every byte before
// advancing the returned length. The Vec remains logically empty until compression
// succeeds, then `set_len` exposes exactly that initialized prefix.
let output =
unsafe { std::slice::from_raw_parts_mut(buffer.as_mut_ptr(), max_output_size) };
let compressed_len = LZ4_COMPRESS_TABLE
.with_borrow_mut(|table| compress_into_with_table(block, output, table))
.context("LZ4 compression failed")?; .context("LZ4 compression failed")?;
// SAFETY: `compress_into_with_table` initialized this many bytes in `buffer` above.
unsafe { buffer.set_len(compressed_len) };
} }
Compression::Zstd3 => { Compression::Zstd3 => {
buffer.reserve(zstd::zstd_safe::compress_bound(block.len())); buffer.reserve(zstd::zstd_safe::compress_bound(block.len()));
......
...@@ -411,7 +411,7 @@ fn write_block_to_file( ...@@ -411,7 +411,7 @@ fn write_block_to_file(
compressor.compress_into_buffer(block, compress_buffer)?; compressor.compress_into_buffer(block, compress_buffer)?;
// Same threshold as LevelDB/RocksDB: require at least 12.5% savings. // Same threshold as LevelDB/RocksDB: require at least 12.5% savings.
if compress_buffer.len() < block.len() - (block.len() / 8) { if compress_buffer.len() < block.len() - (block.len() / 8) {
(block.len().try_into().unwrap(), compress_buffer.as_slice()) (block.len().try_into().unwrap(), compress_buffer)
} else { } else {
(0, block) (0, block)
} }
...@@ -422,15 +422,13 @@ fn write_block_to_file( ...@@ -422,15 +422,13 @@ fn write_block_to_file(
// Checksum is computed on the on-disk data (after compression). // Checksum is computed on the on-disk data (after compression).
let checksum = checksum_block(data_to_write); let checksum = checksum_block(data_to_write);
let result = write_raw_block_to_file( write_raw_block_to_file(
file, file,
block_offsets, block_offsets,
uncompressed_size, uncompressed_size,
checksum, checksum,
data_to_write, data_to_write,
); )
compress_buffer.clear();
result
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
...@@ -2034,7 +2032,7 @@ mod tests { ...@@ -2034,7 +2032,7 @@ mod tests {
body.to_vec() body.to_vec()
} else { } else {
let mut out = vec![0u8; uncompressed_size]; let mut out = vec![0u8; uncompressed_size];
lzzzz::lz4::decompress(body, &mut out)?; lz4_flex::block::decompress_into(body, &mut out)?;
out out
}) })
} }
......
...@@ -14,7 +14,7 @@ workspace = true ...@@ -14,7 +14,7 @@ workspace = true
[features] [features]
default = [] default = []
print_cache_item_size_with_compressed = ["print_cache_item_size", "dep:lzzzz"] print_cache_item_size_with_compressed = ["print_cache_item_size", "dep:lz4_flex"]
print_cache_item_size = [] print_cache_item_size = []
no_fast_stale = [] no_fast_stale = []
aggregation_update_no_batch = [] aggregation_update_no_batch = []
...@@ -47,7 +47,7 @@ fs-err = { workspace = true } ...@@ -47,7 +47,7 @@ fs-err = { workspace = true }
hashbrown = { workspace = true } hashbrown = { workspace = true }
indexmap = { workspace = true } indexmap = { workspace = true }
jiff = "0.2.10" jiff = "0.2.10"
lzzzz = { workspace = true, optional = true } lz4_flex = { workspace = true, optional = true }
parking_lot = { workspace = true } parking_lot = { workspace = true }
rand = { workspace = true } rand = { workspace = true }
ringmap = { workspace = true, features = ["serde"] } ringmap = { workspace = true, features = ["serde"] }
......
...@@ -1256,19 +1256,15 @@ impl TurboTasksBackend { ...@@ -1256,19 +1256,15 @@ impl TurboTasksBackend {
#[cfg(feature = "print_cache_item_size")] #[cfg(feature = "print_cache_item_size")]
impl TaskCacheStats { impl TaskCacheStats {
#[cfg(feature = "print_cache_item_size_with_compressed")] #[cfg(feature = "print_cache_item_size_with_compressed")]
fn compressed_size(data: &[u8]) -> Result<usize> { fn compressed_size(data: &[u8]) -> usize {
Ok(lzzzz::lz4::Compressor::new()?.next_to_vec( lz4_flex::block::compress(data).len()
data,
&mut Vec::new(),
lzzzz::lz4::ACC_LEVEL_DEFAULT,
)?)
} }
fn add_data(&mut self, data: &[u8]) { fn add_data(&mut self, data: &[u8]) {
self.data += data.len(); self.data += data.len();
#[cfg(feature = "print_cache_item_size_with_compressed")] #[cfg(feature = "print_cache_item_size_with_compressed")]
{ {
self.data_compressed += Self::compressed_size(data).unwrap_or(0); self.data_compressed += Self::compressed_size(data);
} }
self.data_count += 1; self.data_count += 1;
} }
...@@ -1277,7 +1273,7 @@ impl TurboTasksBackend { ...@@ -1277,7 +1273,7 @@ impl TurboTasksBackend {
self.meta += data.len(); self.meta += data.len();
#[cfg(feature = "print_cache_item_size_with_compressed")] #[cfg(feature = "print_cache_item_size_with_compressed")]
{ {
self.meta_compressed += Self::compressed_size(data).unwrap_or(0); self.meta_compressed += Self::compressed_size(data);
} }
self.meta_count += 1; self.meta_count += 1;
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment