Unverified Commit d52877ef authored by Luke Sandberg's avatar Luke Sandberg Committed by GitHub

turbo-persistence: drop key compression dictionary from SST files (#90608)

## Summary

- Remove the zstd key compression dictionary from SST files, simplifying
the on-disk format
- Key blocks now use plain LZ4 compression (same as value blocks),
eliminating the zstd dependency
- Update the turbo-persistence README to reflect the format changes

## Why

The key compression dictionary added complexity (zstd dependency,
dictionary computation, dual decompression paths) with diminishing
returns. Plain LZ4 is already used for value blocks and provides
sufficient compression for key blocks. This simplifies both the SST file
format and the compaction/inspection tooling.

This main goal is to unblock streaming SST writes.

## What Changed

**On-disk format (breaking):**
- SST files no longer start with a serialized key compression dictionary
- Meta file entries no longer include the
`key_compression_dictionary_length` field

**Code:**
- Removed `zstd` dependency from `turbo-persistence`
- Removed dictionary computation in `StaticSortedFileBuilder`
- Simplified `sst_inspect` tool (no more dictionary-aware decompression)
- Removed `key_compression_dictionary_length` from
`StaticSortedFileMetaData`, `MetaEntry`, and `MetaFileBuilder`

**Docs:**
- Updated README meta file format (removed dictionary length field)
- Updated README SST file format (removed dictionary region)
- Updated compression description to reference LZ4 directly

## Test Plan

- Existing turbo-persistence tests and benchmarks pass with the format
changes
- `sst_inspect` tool updated to work without dictionary logic

---------
Co-authored-by: 's avatarTobias Koppers <tobias.koppers@googlemail.com>
parent 9d13b676
......@@ -9388,7 +9388,6 @@ dependencies = [
"turbo-bincode",
"turbo-tasks-malloc",
"twox-hash 2.1.2",
"zstd",
]
[[package]]
......
......@@ -35,7 +35,6 @@ fn main() -> Result<()> {
amqf_entries,
sst_size,
flags,
key_compression_dictionary_size,
block_count,
} in meta_file.entries
{
......@@ -46,11 +45,9 @@ fn main() -> Result<()> {
);
println!(" AMQF {amqf_entries} entries = {} KiB", amqf_size / 1024);
println!(
" {} KiB = {} kiB key compression dict + {block_count} blocks (avg {} \
bytes/block)",
" {} KiB = {block_count} blocks (avg {} bytes/block)",
sst_size / 1024,
key_compression_dictionary_size / 1024,
(sst_size - key_compression_dictionary_size as u64) / block_count as u64
sst_size / block_count as u64
);
}
if !meta_file.obsolete_sst_files.is_empty() {
......
......@@ -33,7 +33,6 @@ thread_local = { workspace = true }
tracing = { workspace = true }
turbo-bincode= { workspace = true }
twox-hash = { workspace = true }
zstd = { version = "0.13.2", features = ["zdict_builder"] }
[dev-dependencies]
criterion = { workspace = true }
......
......@@ -44,7 +44,7 @@ Therefore there are these value types:
| Storage overhead | 0 | 8 B in key block + 8 B per ~8 kB in block table | 2 B in key block + 8 B in block table | 4 B in key block + 4 B in blob header |
| Compaction | re-compressed | re-compressed | copied compressed | pointer copied |
Small value blocks are emitted once they accumulate at least `MIN_SMALL_VALUE_BLOCK_SIZE` (8 kB) of data. This means actual block sizes range from 8 kB up to 8 kB + `MAX_SMALL_VALUE_SIZE` (4 kB) = 12 kB. This provides a good balance between compression efficiency (blocks ≥ 4 kB don't need a compression dictionary) and access cost (only ~8–12 kB needs to be decompressed per lookup).
Small value blocks are emitted once they accumulate at least `MIN_SMALL_VALUE_BLOCK_SIZE` (8 kB) of data. This means actual block sizes range from 8 kB up to 8 kB + `MAX_SMALL_VALUE_SIZE` (4 kB) = 12 kB. This provides a good balance between compression efficiency (blocks ≥ 4 kB compress well with LZ4) and access cost (only ~8–12 kB needs to be decompressed per lookup).
### Meta file
......@@ -59,7 +59,6 @@ A meta file can contain metadata about multiple SST files. The metadata is store
- 4 bytes count of described SST files
- foreach described SST file
- 4 bytes sequence number of the SST file
- 2 bytes key Compression Dictionary length
- 2 bytes block count
- 8 bytes min hash
- 8 bytes max hash
......@@ -77,7 +76,6 @@ A meta file can contain metadata about multiple SST files. The metadata is store
The SST file contains only data without any header.
- serialized key Compression Dictionary
- foreach block
- 4 bytes block header (uncompressed length or sentinel)
- block data (compressed or uncompressed)
......
......@@ -788,20 +788,12 @@ fn bench_static_sorted_file_lookup(c: &mut Criterion) {
// Create temp directory and write SST file
let tempdir = tempfile::tempdir().unwrap();
let sst_path = tempdir.path().join("00000001.sst");
let total_key_size = entry_count * 8;
let (meta, _file) = write_static_stored_file(
&entries,
total_key_size,
&sst_path,
MetaEntryFlags::FRESH,
)
.unwrap();
let (meta, _file) =
write_static_stored_file(&entries, &sst_path, MetaEntryFlags::FRESH).unwrap();
// Open the SST file
let sst_meta = StaticSortedFileMetaData {
sequence_number: 1,
key_compression_dictionary_length: meta.key_compression_dictionary_length,
block_count: meta.block_count,
};
let sst = StaticSortedFile::open(tempdir.path(), sst_meta).unwrap();
......
......@@ -19,7 +19,7 @@ use std::{
use anyhow::{Context, Result, bail};
use byteorder::{BE, ReadBytesExt};
use lzzzz::lz4::{decompress, decompress_with_dict};
use lzzzz::lz4::decompress;
use memmap2::Mmap;
use turbo_persistence::meta_file::MetaFile;
// Import shared constants from the crate
......@@ -80,8 +80,6 @@ struct SstStats {
/// Value block sizes (small values)
value_blocks: BlockSizeInfo,
/// Key compression dictionary size
key_dict_size: u64,
/// Block directory size (block_count * 4 bytes at end of file)
block_directory_size: u64,
......@@ -105,7 +103,6 @@ impl SstStats {
self.index_blocks.merge(&other.index_blocks);
self.key_blocks.merge(&other.key_blocks);
self.value_blocks.merge(&other.value_blocks);
self.key_dict_size += other.key_dict_size;
self.block_directory_size += other.block_directory_size;
self.inline_value_bytes += other.inline_value_bytes;
self.small_value_refs += other.small_value_refs;
......@@ -119,7 +116,6 @@ impl SstStats {
/// Information about an SST file from the meta file
struct SstInfo {
sequence_number: u32,
key_compression_dictionary_length: u16,
block_count: u16,
}
......@@ -201,7 +197,6 @@ fn collect_sst_info(db_path: &Path) -> Result<BTreeMap<u32, Vec<SstInfo>>> {
for entry in meta_file.entries() {
family_sst_info.entry(family).or_default().push(SstInfo {
sequence_number: entry.sequence_number(),
key_compression_dictionary_length: entry.key_compression_dictionary_length(),
block_count: entry.block_count(),
});
}
......@@ -212,22 +207,14 @@ fn collect_sst_info(db_path: &Path) -> Result<BTreeMap<u32, Vec<SstInfo>>> {
/// Decompress a block, respecting the optional compression protocol.
/// When uncompressed_length is 0, the block is stored uncompressed.
fn decompress_block(
compressed: &[u8],
uncompressed_length: u32,
dictionary: Option<&[u8]>,
) -> Result<Arc<[u8]>> {
fn decompress_block(compressed: &[u8], uncompressed_length: u32) -> Result<Arc<[u8]>> {
// Sentinel: uncompressed_length = 0 means block is stored uncompressed
if uncompressed_length == 0 {
return Ok(Arc::from(compressed));
}
let mut buffer = vec![0u8; uncompressed_length as usize];
let bytes_written = if let Some(dict) = dictionary {
decompress_with_dict(compressed, &mut buffer, dict)?
} else {
decompress(compressed, &mut buffer)?
};
let bytes_written = decompress(compressed, &mut buffer)?;
assert_eq!(
bytes_written, uncompressed_length as usize,
"Decompressed length does not match expected"
......@@ -245,7 +232,6 @@ fn analyze_sst_file(db_path: &Path, info: &SstInfo) -> Result<SstStats> {
let mmap = unsafe { Mmap::map(&file)? };
let mut stats = SstStats {
key_dict_size: info.key_compression_dictionary_length as u64,
block_directory_size: info.block_count as u64 * 4,
file_size,
..Default::default()
......@@ -253,14 +239,7 @@ fn analyze_sst_file(db_path: &Path, info: &SstInfo) -> Result<SstStats> {
// Calculate offsets
let block_offsets_start = mmap.len() - (info.block_count as usize * 4);
let blocks_start = info.key_compression_dictionary_length as usize;
// Get key compression dictionary if present
let key_dict = if info.key_compression_dictionary_length > 0 {
Some(&mmap[0..info.key_compression_dictionary_length as usize])
} else {
None
};
let blocks_start = 0;
// Iterate through all blocks
for block_index in 0..info.block_count {
......@@ -287,27 +266,14 @@ fn analyze_sst_file(db_path: &Path, info: &SstInfo) -> Result<SstStats> {
uncompressed_length as u64
};
// Try to decompress with key dictionary first (for key/index blocks)
let decompressed = match decompress_block(compressed_data, uncompressed_length, key_dict) {
let decompressed = match decompress_block(compressed_data, uncompressed_length) {
Ok(data) => data,
Err(_) => {
// If that fails, try without dictionary (value blocks)
match decompress_block(compressed_data, uncompressed_length, None) {
Ok(_) => {
// This is a value block
stats
.value_blocks
.add(compressed_size, actual_size, was_compressed);
continue; // Value blocks don't have entry type headers
}
Err(e) => {
eprintln!(
"Warning: Failed to decompress block {} in {:08}.sst: {}",
block_index, info.sequence_number, e
);
continue;
}
}
Err(e) => {
eprintln!(
"Warning: Failed to decompress block {} in {:08}.sst: {}",
block_index, info.sequence_number, e
);
continue;
}
};
......@@ -559,7 +525,7 @@ fn print_sst_details(seq_num: u32, stats: &SstStats) {
);
// Per-file overhead
let overhead = stats.key_dict_size + stats.block_directory_size;
let overhead = stats.block_directory_size;
let overhead_pct = if stats.file_size > 0 {
(overhead as f64 / stats.file_size as f64) * 100.0
} else {
......@@ -571,10 +537,6 @@ fn print_sst_details(seq_num: u32, stats: &SstStats) {
format_bytes(overhead),
overhead_pct
);
println!(
" │ Key compression dictionary: {}",
format_bytes(stats.key_dict_size)
);
println!(
" │ Block directory: {}",
format_bytes(stats.block_directory_size)
......@@ -634,7 +596,7 @@ fn print_family_summary(family: u32, sst_count: usize, stats: &SstStats) {
}
// Per-file overhead
let total_overhead = stats.key_dict_size + stats.block_directory_size;
let total_overhead = stats.block_directory_size;
let overhead_pct = if stats.file_size > 0 {
(total_overhead as f64 / stats.file_size as f64) * 100.0
} else {
......@@ -646,16 +608,6 @@ fn print_family_summary(family: u32, sst_count: usize, stats: &SstStats) {
format_bytes(total_overhead),
overhead_pct
);
println!(
" Key compression dictionaries: {}",
format_bytes(stats.key_dict_size)
);
if sst_count > 0 {
println!(
" Average per file: {}",
format_bytes(stats.key_dict_size / sst_count as u64)
);
}
println!(
" Block directories: {}",
format_bytes(stats.block_directory_size)
......
use std::{mem::MaybeUninit, sync::Arc};
use anyhow::{Context, Result};
use lzzzz::lz4::{ACC_LEVEL_DEFAULT, decompress, decompress_with_dict};
use lzzzz::lz4::{self, decompress};
/// Decompresses a block into an Arc allocation.
///
/// The caller must ensure `uncompressed_length > 0` (i.e., the block is actually compressed).
/// Uncompressed blocks should be handled via zero-copy mmap slices before calling this.
pub fn decompress_into_arc(
uncompressed_length: u32,
block: &[u8],
compression_dictionary: Option<&[u8]>,
_long_term: bool,
) -> Result<Arc<[u8]>> {
pub fn decompress_into_arc(uncompressed_length: u32, block: &[u8]) -> Result<Arc<[u8]>> {
debug_assert!(
uncompressed_length > 0,
"decompress_into_arc called with uncompressed_length=0; uncompressed blocks should use \
......@@ -26,11 +21,7 @@ pub fn decompress_into_arc(
let mut buffer = unsafe { buffer.assume_init() };
// We just created this Arc so refcount is 1; get_mut always succeeds.
let decompressed = Arc::get_mut(&mut buffer).expect("Arc refcount should be 1");
let bytes_written = if let Some(dict) = compression_dictionary {
decompress_with_dict(block, decompressed, dict)?
} else {
decompress(block, decompressed)?
};
let bytes_written = decompress(block, decompressed)?;
assert_eq!(
bytes_written, uncompressed_length as usize,
"Decompressed length does not match expected length"
......@@ -39,21 +30,7 @@ pub fn decompress_into_arc(
}
#[tracing::instrument(level = "trace", skip_all)]
pub fn compress_into_buffer(
block: &[u8],
dict: Option<&[u8]>,
_long_term: bool,
buffer: &mut Vec<u8>,
) -> Result<()> {
let mut compressor = if let Some(dict) = dict {
lzzzz::lz4::Compressor::with_dict(dict)
} else {
lzzzz::lz4::Compressor::new()
}
.context("LZ4 compressor creation failed")?;
let acc_factor = ACC_LEVEL_DEFAULT;
compressor
.next_to_vec(block, buffer, acc_factor)
.context("Compression failed")?;
pub fn compress_into_buffer(block: &[u8], buffer: &mut Vec<u8>) -> Result<()> {
lz4::compress_to_vec(block, buffer, lz4::ACC_LEVEL_DEFAULT).context("Compression failed")?;
Ok(())
}
......@@ -391,7 +391,7 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
let mut compressed = &mmap[..];
let uncompressed_length = compressed.read_u32::<BE>()?;
let buffer = decompress_into_arc(uncompressed_length, compressed, None, true)?;
let buffer = decompress_into_arc(uncompressed_length, compressed)?;
Ok(ArcBytes::from(buffer))
}
......@@ -1012,8 +1012,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
min_hash: entry.min_hash(),
max_hash: entry.max_hash(),
amqf,
key_compression_dictionary_length: entry
.key_compression_dictionary_length(),
block_count: entry.block_count(),
size: entry.size(),
flags: entry.flags(),
......@@ -1028,7 +1026,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
fn create_sst_file<S: ParallelScheduler>(
parallel_scheduler: &S,
entries: &[LookupEntry],
total_key_size: usize,
path: &Path,
seq: u32,
flags: MetaEntryFlags,
......@@ -1039,7 +1036,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
let (meta, file) = parallel_scheduler.block_in_place(|| {
write_static_stored_file(
entries,
total_key_size,
&path.join(format!("{seq:08}.sst")),
flags,
)
......@@ -1130,8 +1126,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
.track(is_medium, small_size);
if collector.is_full() {
let selected_total_key_size =
collector.last_entries_total_key_size;
swap(
&mut collector.entries,
&mut collector.last_entries,
......@@ -1156,7 +1150,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
collector.new_sst_files.push(create_sst_file(
&self.parallel_scheduler,
&collector.entries,
selected_total_key_size,
path,
seq,
flags,
......@@ -1184,7 +1177,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
collector.new_sst_files.push(create_sst_file(
&self.parallel_scheduler,
&collector.last_entries,
collector.last_entries_total_key_size,
path,
seq,
flags,
......@@ -1230,7 +1222,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
collector.new_sst_files.push(create_sst_file(
&self.parallel_scheduler,
&collector.entries,
collector.total_key_size,
path,
seq,
flags,
......@@ -1258,8 +1249,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
collector.new_sst_files.push(create_sst_file(
&self.parallel_scheduler,
part1,
// We don't know the exact sizes so we estimate them
collector.last_entries_total_key_size / 2,
path,
seq1,
flags,
......@@ -1269,7 +1258,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
collector.new_sst_files.push(create_sst_file(
&self.parallel_scheduler,
part2,
collector.last_entries_total_key_size / 2,
path,
seq2,
flags,
......@@ -1636,8 +1624,6 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
flags: entry.flags(),
amqf_size: entry.amqf_size(),
amqf_entries: amqf.len(),
key_compression_dictionary_size: entry
.key_compression_dictionary_length(),
block_count: entry.block_count(),
}
})
......@@ -1699,6 +1685,5 @@ pub struct MetaFileEntryInfo {
pub amqf_entries: usize,
pub sst_size: u64,
pub flags: MetaEntryFlags,
pub key_compression_dictionary_size: u16,
pub block_count: u16,
}
......@@ -155,10 +155,6 @@ impl MetaEntry {
self.max_hash
}
pub fn key_compression_dictionary_length(&self) -> u16 {
self.sst_data.key_compression_dictionary_length
}
pub fn block_count(&self) -> u16 {
self.sst_data.block_count
}
......@@ -265,7 +261,6 @@ impl MetaFile {
let entry = MetaEntry {
sst_data: StaticSortedFileMetaData {
sequence_number: file.read_u32::<BE>()?,
key_compression_dictionary_length: file.read_u16::<BE>()?,
block_count: file.read_u16::<BE>()?,
},
family,
......
......@@ -65,7 +65,6 @@ impl<'a> MetaFileBuilder<'a> {
let mut amqf_offset = 0;
for (sequence_number, sst) in &self.entries {
file.write_u32::<BE>(*sequence_number)?;
file.write_u16::<BE>(sst.key_compression_dictionary_length)?;
file.write_u16::<BE>(sst.block_count)?;
file.write_u64::<BE>(sst.min_hash)?;
file.write_u64::<BE>(sst.max_hash)?;
......
......@@ -2,7 +2,6 @@ use std::{
cmp::Ordering,
fs::File,
hash::BuildHasherDefault,
ops::Range,
path::{Path, PathBuf},
sync::Arc,
};
......@@ -121,8 +120,6 @@ impl ValueBlockCache for &mut Option<(u16, ArcBytes)> {
pub struct StaticSortedFileMetaData {
/// The sequence number of this file.
pub sequence_number: u32,
/// The length of the key compression dictionary.
pub key_compression_dictionary_length: u16,
/// The number of blocks in the SST file.
pub block_count: u16,
}
......@@ -132,17 +129,6 @@ impl StaticSortedFileMetaData {
let bc: usize = self.block_count.into();
sst_len - (bc * size_of::<u32>())
}
pub fn blocks_start(&self) -> usize {
let k: usize = self.key_compression_dictionary_length.into();
k
}
pub fn key_compression_dictionary_range(&self) -> Range<usize> {
let start = 0;
let end: usize = self.key_compression_dictionary_length.into();
start..end
}
}
/// A memory mapped SST file.
......@@ -393,31 +379,22 @@ impl StaticSortedFile {
/// Reads a key block from the file.
fn read_key_block(&self, block_index: u16) -> Result<ArcBytes> {
self.read_block(
block_index,
Some(&self.mmap[self.meta.key_compression_dictionary_range()]),
false,
)
self.read_block(block_index)
}
/// Reads a value block from the file.
/// Reads a small value block from the file.
fn read_small_value_block(&self, block_index: u16) -> Result<ArcBytes> {
self.read_block(block_index, None, false)
self.read_block(block_index)
}
/// Reads a value block from the file.
fn read_value_block(&self, block_index: u16) -> Result<ArcBytes> {
self.read_block(block_index, None, true)
self.read_block(block_index)
}
/// Reads a block from the file.
#[tracing::instrument(level = "info", name = "reading database block", skip_all)]
fn read_block(
&self,
block_index: u16,
compression_dictionary: Option<&[u8]>,
long_term: bool,
) -> Result<ArcBytes> {
fn read_block(&self, block_index: u16) -> Result<ArcBytes> {
let (uncompressed_length, block) = self.get_raw_block_slice(block_index)?;
// 0 means the block was not compressed, return the mmap-backed ArcBytes directly
......@@ -436,12 +413,7 @@ impl StaticSortedFile {
block.len(),
);
let buffer = decompress_into_arc(
uncompressed_length,
block,
compression_dictionary,
long_term,
)?;
let buffer = decompress_into_arc(uncompressed_length, block)?;
Ok(ArcBytes::from(buffer))
}
......@@ -465,13 +437,11 @@ impl StaticSortedFile {
#[cfg(feature = "strict_checks")]
if block_index >= self.meta.block_count {
bail!(
"Corrupted file seq:{} block:{} > number of blocks {} (block_offsets: {:x}, \
blocks: {:x})",
"Corrupted file seq:{} block:{} > number of blocks {} (block_offsets: {:x})",
self.meta.sequence_number,
block_index,
self.meta.block_count,
self.meta.block_offsets_start(self.mmap.len()),
self.meta.blocks_start()
);
}
let offset = self.meta.block_offsets_start(self.mmap.len()) + block_index as usize * 4;
......@@ -479,34 +449,30 @@ impl StaticSortedFile {
if offset + 4 > self.mmap.len() {
bail!(
"Corrupted file seq:{} block:{} block offset locations {} + 4 bytes > file end {} \
(block_offsets: {:x}, blocks: {:x})",
(block_offsets: {:x})",
self.meta.sequence_number,
block_index,
offset,
self.mmap.len(),
self.meta.block_offsets_start(self.mmap.len()),
self.meta.blocks_start()
);
}
let block_start = if block_index == 0 {
self.meta.blocks_start()
0
} else {
self.meta.blocks_start() + (&self.mmap[offset - 4..offset]).read_u32::<BE>()? as usize
(&self.mmap[offset - 4..offset]).read_u32::<BE>()? as usize
};
let block_end =
self.meta.blocks_start() + (&self.mmap[offset..offset + 4]).read_u32::<BE>()? as usize;
let block_end = (&self.mmap[offset..offset + 4]).read_u32::<BE>()? as usize;
#[cfg(feature = "strict_checks")]
if block_end > self.mmap.len() || block_start > self.mmap.len() {
bail!(
"Corrupted file seq:{} block:{} block {} - {} > file end {} (block_offsets: {:x}, \
blocks: {:x})",
"Corrupted file seq:{} block:{} block {} - {} > file end {} (block_offsets: {:x})",
self.meta.sequence_number,
block_index,
block_start,
block_end,
self.mmap.len(),
self.meta.block_offsets_start(self.mmap.len()),
self.meta.blocks_start()
);
}
let uncompressed_length =
......
use std::{
borrow::Cow,
cmp::min,
fs::File,
io::{BufWriter, Seek, Write},
path::Path,
......@@ -40,18 +39,6 @@ const MAX_SMALL_VALUE_BLOCK_ENTRIES: usize = MIN_SMALL_VALUE_BLOCK_SIZE;
/// The aimed false positive rate for the AMQF
const AMQF_FALSE_POSITIVE_RATE: f64 = 0.01;
/// The maximum compression dictionary size for key and index blocks
const KEY_COMPRESSION_DICTIONARY_SIZE: usize = 64 * 1024 - 1;
/// The maximum bytes that should be selected as key samples to create a compression dictionary
const KEY_COMPRESSION_SAMPLES_SIZE: usize = 256 * 1024;
/// The minimum bytes that should be selected as keys samples. Below that no compression dictionary
/// is used.
const MIN_KEY_COMPRESSION_SAMPLES_SIZE: usize = 1024;
/// The bytes that are used per key entry for a sample.
const COMPRESSION_DICTIONARY_SAMPLE_PER_ENTRY: usize = 100;
/// The minimum bytes that are used per key entry for a sample.
const MIN_COMPRESSION_DICTIONARY_SAMPLE_PER_ENTRY: usize = 16;
/// Determines whether to store the hash per entry based on max key length.
fn use_hash(max_key_len: usize) -> bool {
max_key_len > 32
......@@ -100,8 +87,6 @@ pub struct StaticSortedFileBuilderMeta<'a> {
pub max_hash: u64,
/// The AMQF data
pub amqf: Cow<'a, [u8]>,
/// The key compression dictionary
pub key_compression_dictionary_length: u16,
/// The number of blocks in the SST file
pub block_count: u16,
/// The file size of the SST file
......@@ -114,7 +99,6 @@ pub struct StaticSortedFileBuilderMeta<'a> {
pub fn write_static_stored_file<E: Entry>(
entries: &[E],
total_key_size: usize,
file: &Path,
flags: MetaEntryFlags,
) -> Result<(StaticSortedFileBuilderMeta<'static>, File)> {
......@@ -122,12 +106,8 @@ pub fn write_static_stored_file<E: Entry>(
let mut file = BufWriter::new(File::create(file)?);
let capacity = get_compression_buffer_capacity(total_key_size);
// We use a shared buffer for all operations to avoid excessive allocations
let mut buffer = Vec::with_capacity(capacity);
let key_dict = compute_key_compression_dictionary(entries, total_key_size, &mut buffer)?;
file.write_all(&key_dict)?;
let mut buffer = Vec::new();
let mut block_writer = BlockWriter::new(&mut file, &mut buffer);
......@@ -142,7 +122,6 @@ pub fn write_static_stored_file<E: Entry>(
let amqf = write_key_blocks_and_compute_amqf(
entries,
&value_locations,
&key_dict,
&mut block_writer,
&mut buffer,
)
......@@ -159,7 +138,6 @@ pub fn write_static_stored_file<E: Entry>(
min_hash,
max_hash,
amqf: Cow::Owned(amqf.into_vec()),
key_compression_dictionary_length: key_dict.len().try_into().unwrap(),
block_count,
size: file.stream_position()?,
flags,
......@@ -168,70 +146,9 @@ pub fn write_static_stored_file<E: Entry>(
Ok((meta, file.into_inner()?))
}
fn get_compression_buffer_capacity(total_key_size: usize) -> usize {
let mut size = 0;
if total_key_size >= MIN_KEY_COMPRESSION_SAMPLES_SIZE {
let key_compression_samples_size = min(KEY_COMPRESSION_SAMPLES_SIZE, total_key_size / 16);
size = key_compression_samples_size;
}
size
}
/// Computes compression dictionaries from keys of all entries
#[tracing::instrument(level = "trace", skip(entries))]
fn compute_key_compression_dictionary<E: Entry>(
entries: &[E],
total_key_size: usize,
buffer: &mut Vec<u8>,
) -> Result<Vec<u8>> {
if total_key_size < MIN_KEY_COMPRESSION_SAMPLES_SIZE {
return Ok(Vec::new());
}
let key_compression_samples_size = min(KEY_COMPRESSION_SAMPLES_SIZE, total_key_size / 16);
let mut sample_sizes = Vec::new();
// Limit the number of iterations to avoid infinite loops
let max_iterations = total_key_size / COMPRESSION_DICTIONARY_SAMPLE_PER_ENTRY * 2;
for i in 0..max_iterations {
let entry = &entries[i % entries.len()];
let key_remaining = key_compression_samples_size - buffer.len();
if key_remaining < MIN_COMPRESSION_DICTIONARY_SAMPLE_PER_ENTRY {
break;
}
let len = entry.key_len();
if len >= MIN_COMPRESSION_DICTIONARY_SAMPLE_PER_ENTRY {
let used_len = min(key_remaining, COMPRESSION_DICTIONARY_SAMPLE_PER_ENTRY);
if len <= used_len {
sample_sizes.push(len);
entry.write_key_to(buffer);
} else {
let mut temp = Vec::with_capacity(len);
entry.write_key_to(&mut temp);
debug_assert!(temp.len() == len);
let p = buffer.len() % (len - used_len);
sample_sizes.push(used_len);
buffer.extend_from_slice(&temp[p..p + used_len]);
}
}
}
debug_assert!(buffer.len() == sample_sizes.iter().sum::<usize>());
let result = if buffer.len() > MIN_KEY_COMPRESSION_SAMPLES_SIZE && sample_sizes.len() > 5 {
zstd::dict::from_continuous(buffer, &sample_sizes, KEY_COMPRESSION_DICTIONARY_SIZE)
.context("Key dictionary creation failed")?
} else {
Vec::new()
};
buffer.clear();
Ok(result)
}
enum CompressionConfig<'a> {
enum CompressionConfig {
/// Attempt compression; use the result only if it's smaller than the original.
TryCompress {
dict: Option<&'a [u8]>,
long_term: bool,
},
TryCompress,
/// Write the block uncompressed.
Uncompressed,
}
......@@ -266,15 +183,9 @@ impl<'l> BlockWriter<'l> {
}
#[tracing::instrument(level = "trace", skip_all)]
fn write_key_block(&mut self, block: &[u8], dict: &[u8]) -> Result<()> {
self.write_block(
block,
CompressionConfig::TryCompress {
dict: Some(dict),
long_term: false,
},
)
.context("Failed to write key block")
fn write_key_block(&mut self, block: &[u8]) -> Result<()> {
self.write_block(block, CompressionConfig::TryCompress)
.context("Failed to write key block")
}
#[tracing::instrument(level = "trace", skip_all)]
......@@ -286,32 +197,20 @@ impl<'l> BlockWriter<'l> {
#[tracing::instrument(level = "trace", skip_all)]
fn write_small_value_block(&mut self, block: &[u8]) -> Result<()> {
self.write_block(
block,
CompressionConfig::TryCompress {
dict: None,
long_term: false,
},
)
.context("Failed to write small value block")
self.write_block(block, CompressionConfig::TryCompress)
.context("Failed to write small value block")
}
#[tracing::instrument(level = "trace", skip_all)]
fn write_value_block(&mut self, block: &[u8]) -> Result<()> {
self.write_block(
block,
CompressionConfig::TryCompress {
dict: None,
long_term: true,
},
)
.context("Failed to write value block")
self.write_block(block, CompressionConfig::TryCompress)
.context("Failed to write value block")
}
fn write_block(&mut self, block: &[u8], compression: CompressionConfig<'_>) -> Result<()> {
fn write_block(&mut self, block: &[u8], compression: CompressionConfig) -> Result<()> {
let (uncompressed_size, data_to_write): (u32, &[u8]) = match compression {
CompressionConfig::TryCompress { dict, long_term } => {
self.compress_block_into_buffer(block, dict, long_term)?;
CompressionConfig::TryCompress => {
self.compress_block_into_buffer(block)?;
// Same threshold as LevelDB/RocksDB: require at least 12.5% savings to store
// compressed.
// See https://github.com/google/leveldb/blob/ac691084fdc5546421a55b25e7653d450e5a25fb/table/table_builder.cc#L164
......@@ -369,14 +268,9 @@ impl<'l> BlockWriter<'l> {
Ok(())
}
/// Compresses a block with a compression dictionary.
fn compress_block_into_buffer(
&mut self,
block: &[u8],
dict: Option<&[u8]>,
long_term: bool,
) -> Result<()> {
compress_into_buffer(block, dict, long_term, self.buffer)
/// Compresses a block using LZ4.
fn compress_block_into_buffer(&mut self, block: &[u8]) -> Result<()> {
compress_into_buffer(block, self.buffer)
}
}
......@@ -457,7 +351,6 @@ fn write_value_blocks(
fn write_key_blocks_and_compute_amqf(
entries: &[impl Entry],
value_locations: &[(u16, u32)],
key_compression_dictionary: &[u8],
writer: &mut BlockWriter<'_>,
buffer: &mut Vec<u8>,
) -> Result<TurboBincodeBuffer> {
......@@ -531,7 +424,7 @@ fn write_key_blocks_and_compute_amqf(
writer.next_block_index(),
));
block.finish();
writer.write_key_block(buffer, key_compression_dictionary)?;
writer.write_key_block(buffer)?;
buffer.clear();
current_block_size = 0;
current_block_max_key_len = 0;
......@@ -557,7 +450,7 @@ fn write_key_blocks_and_compute_amqf(
writer.next_block_index(),
));
block.finish();
writer.write_key_block(buffer, key_compression_dictionary)?;
writer.write_key_block(buffer)?;
buffer.clear();
}
......
......@@ -414,7 +414,7 @@ impl<K: StoreKey + Send + Sync, S: ParallelScheduler, const FAMILIES: usize>
let seq = self.current_sequence_number.fetch_add(1, Ordering::SeqCst) + 1;
let mut buffer = Vec::new();
buffer.write_u32::<BE>(value.len() as u32)?;
compress_into_buffer(value, None, true, &mut buffer)
compress_into_buffer(value, &mut buffer)
.context("Compression of value for blob file failed")?;
let file = self.db_path.join(format!("{seq:08}.blob"));
......@@ -433,15 +433,13 @@ impl<K: StoreKey + Send + Sync, S: ParallelScheduler, const FAMILIES: usize>
family: u32,
collector_data: (&[CollectorEntry<K>], usize),
) -> Result<(u32, File)> {
let (entries, total_key_size) = collector_data;
let (entries, _total_key_size) = collector_data;
let seq = self.current_sequence_number.fetch_add(1, Ordering::SeqCst) + 1;
let path = self.db_path.join(format!("{seq:08}.sst"));
let (meta, file) = self
.parallel_scheduler
.block_in_place(|| {
write_static_stored_file(entries, total_key_size, &path, MetaEntryFlags::FRESH)
})
.block_in_place(|| write_static_stored_file(entries, &path, MetaEntryFlags::FRESH))
.with_context(|| format!("Unable to write SST file {seq:08}.sst"))?;
#[cfg(feature = "verify_sst_content")]
......@@ -463,7 +461,6 @@ impl<K: StoreKey + Send + Sync, S: ParallelScheduler, const FAMILIES: usize>
&self.db_path,
StaticSortedFileMetaData {
sequence_number: seq,
key_compression_dictionary_length: meta.key_compression_dictionary_length,
block_count: meta.block_count,
},
)?;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment