Unverified Commit ea56922a authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

turbo-persistence: add CRC32 block checksums (#90754)

### What?

Add a 4-byte CRC32 checksum to every block in turbo-persistence SST
files.

### Why?

Detect on-disk cache corruption early with a clear error message, rather
than silently returning wrong data or hitting confusing LZ4
decompression failures.

### How?

**New on-disk block format:**
```
[4B uncompressed_length] [4B CRC32 checksum] [block data]
```
(Previously the checksum field did not exist — header was just the 4B
uncompressed_length.)

- The checksum is computed on the **uncompressed** block data using
`crc32fast` (hardware-accelerated CRC32C on modern CPUs).
- On write, the checksum is computed before compression and stored in
the block header.
- On read, the checksum is verified after decompression (or directly for
uncompressed blocks). A mismatch returns an `anyhow` error with a
message like: `Cache corruption detected: checksum mismatch in block N
of SST file seq:M`.
- During compaction, medium values are copied as raw compressed blocks
without decompression — the checksum is carried through the
`MediumRaw`/`LazyLookupValue::Medium` path unchanged.
- A `BLOCK_HEADER_SIZE` constant (= 8) replaces magic numbers across the
write, read, and inspect code.

**Files changed:**
- `compression.rs` — `checksum_block()` helper wrapping
`crc32fast::hash()`
- `static_sorted_file_builder.rs` — write path
(`write_raw_block_to_file`, `write_block_to_file`, `close()` index
block), `EntryValue::MediumRaw` carries checksum
- `static_sorted_file.rs` — read path (`get_raw_block_slice`,
`read_block` with `verify_checksum()` helper)
- `lookup_entry.rs` — `LazyLookupValue::Medium` carries checksum for
compaction
- `bin/sst_inspect.rs` — updated block parsing offsets
- `README.md` — documented new format

**Tests:**
- All 48 existing tests pass (checksums are transparent to correct data)
- 2 new corruption tests: one corrupts a compressed block's checksum,
the other corrupts an uncompressed block's data — both verify the error
is returned
parent 82db9449
...@@ -9431,6 +9431,7 @@ dependencies = [ ...@@ -9431,6 +9431,7 @@ dependencies = [
"bitfield", "bitfield",
"byteorder", "byteorder",
"codspeed-criterion-compat", "codspeed-criterion-compat",
"crc32fast",
"dashmap 6.1.0", "dashmap 6.1.0",
"either", "either",
"jiff", "jiff",
......
...@@ -380,6 +380,7 @@ clap = { version = "4.5.2", features = ["derive"] } ...@@ -380,6 +380,7 @@ clap = { version = "4.5.2", features = ["derive"] }
concurrent-queue = "2.5.0" concurrent-queue = "2.5.0"
console-subscriber = "0.4.1" console-subscriber = "0.4.1"
const_format = "0.2.30" const_format = "0.2.30"
crc32fast = "1.4.2"
criterion = { package = "codspeed-criterion-compat", version = "4.3.0" } criterion = { package = "codspeed-criterion-compat", version = "4.3.0" }
crossbeam-channel = "0.5.8" crossbeam-channel = "0.5.8"
dashmap = "6.1.0" dashmap = "6.1.0"
......
...@@ -17,6 +17,7 @@ anyhow = { workspace = true } ...@@ -17,6 +17,7 @@ anyhow = { workspace = true }
bincode = { workspace = true } bincode = { workspace = true }
bitfield = { workspace = true } bitfield = { workspace = true }
byteorder = { workspace = true } byteorder = { workspace = true }
crc32fast = { workspace = true }
dashmap = { workspace = true} dashmap = { workspace = true}
either = { workspace = true } either = { workspace = true }
jiff = "0.2.10" jiff = "0.2.10"
......
...@@ -41,7 +41,7 @@ Therefore there are these value types: ...@@ -41,7 +41,7 @@ Therefore there are these value types:
| Compression unit | key block | shared value block (≥ 8 kB) | dedicated value block | separate file | | Compression unit | key block | shared value block (≥ 8 kB) | dedicated value block | separate file |
| Compression unit size | ≤ 16 kB | 8 kB .. 12 kB | 4 kB .. 64 MB | > 64 MB | | Compression unit size | ≤ 16 kB | 8 kB .. 12 kB | 4 kB .. 64 MB | > 64 MB |
| Access cost | no extra overhead | decompress shared block (~8 kB) | decompress value size | open separate file, decompress value size | | Access cost | no extra overhead | decompress shared block (~8 kB) | decompress value size | open separate file, decompress value size |
| Storage overhead | 0 | 8 B in key block + 8 B per ~8 kB in block table | 2 B in key block + 8 B in block table | 4 B in key block + 4 B in blob header | | Storage overhead | 0 | 8 B in key block + 8 B per ~8 kB in block table | 2 B in key block + 8 B in block table | 4 B in key block + 8 B in blob header |
| Compaction | re-compressed | re-compressed | copied compressed | pointer copied | | Compaction | re-compressed | re-compressed | copied compressed | pointer copied |
Small value blocks are emitted once they accumulate at least `MIN_SMALL_VALUE_BLOCK_SIZE` (8 kB) of data. This means actual block sizes range from 8 kB up to 8 kB + `MAX_SMALL_VALUE_SIZE` (4 kB) = 12 kB. This provides a good balance between compression efficiency (blocks ≥ 4 kB compress well with LZ4) and access cost (only ~8–12 kB needs to be decompressed per lookup). Small value blocks are emitted once they accumulate at least `MIN_SMALL_VALUE_BLOCK_SIZE` (8 kB) of data. This means actual block sizes range from 8 kB up to 8 kB + `MAX_SMALL_VALUE_SIZE` (4 kB) = 12 kB. This provides a good balance between compression efficiency (blocks ≥ 4 kB compress well with LZ4) and access cost (only ~8–12 kB needs to be decompressed per lookup).
...@@ -78,6 +78,7 @@ The SST file contains only data without any header. ...@@ -78,6 +78,7 @@ The SST file contains only data without any header.
- foreach block - foreach block
- 4 bytes block header (uncompressed length or sentinel) - 4 bytes block header (uncompressed length or sentinel)
- 4 bytes checksum (CRC32 of uncompressed block data)
- block data (compressed or uncompressed) - block data (compressed or uncompressed)
- foreach block - foreach block
- 4 bytes end of block offset relative to start of all blocks - 4 bytes end of block offset relative to start of all blocks
...@@ -89,6 +90,10 @@ Blocks can be stored compressed (LZ4) or uncompressed. The 4-byte header disting ...@@ -89,6 +90,10 @@ Blocks can be stored compressed (LZ4) or uncompressed. The 4-byte header disting
- **Header > 0**: Block is LZ4 compressed. Header value is the uncompressed length. - **Header > 0**: Block is LZ4 compressed. Header value is the uncompressed length.
- **Header = 0**: Block is stored uncompressed. Actual length is derived from block offsets. - **Header = 0**: Block is stored uncompressed. Actual length is derived from block offsets.
#### Block Checksum
Each block stores a 4-byte CRC32 checksum (big-endian) computed on the **on-disk** block data (i.e. after compression). On read, the checksum is verified **before** decompression so that on-disk damage is caught before passing data to LZ4. A checksum mismatch returns an error indicating that the cached data is damaged.
#### Index Block #### Index Block
- 1 byte block type (0: index block) - 1 byte block type (0: index block)
...@@ -162,7 +167,13 @@ Future: ...@@ -162,7 +167,13 @@ Future:
### Blob file ### Blob file
The plain value compressed with dynamic compression. The plain value compressed with dynamic compression. Each blob file has an 8-byte header:
- 4 bytes: uncompressed length (u32 big-endian)
- 4 bytes: CRC32 checksum of the compressed data (u32 big-endian)
- remaining bytes: LZ4-compressed value data
The checksum is verified on the compressed data **before** decompression when the blob is read.
## Reading ## Reading
......
...@@ -21,13 +21,13 @@ use anyhow::{Context, Result, bail}; ...@@ -21,13 +21,13 @@ use anyhow::{Context, Result, bail};
use byteorder::{BE, ReadBytesExt}; use byteorder::{BE, ReadBytesExt};
use lzzzz::lz4::decompress; use lzzzz::lz4::decompress;
use memmap2::Mmap; use memmap2::Mmap;
use turbo_persistence::meta_file::MetaFile;
// Import shared constants from the crate // Import shared constants from the crate
use turbo_persistence::static_sorted_file::{ use turbo_persistence::static_sorted_file::{
BLOCK_TYPE_INDEX, BLOCK_TYPE_KEY_NO_HASH, BLOCK_TYPE_KEY_WITH_HASH, KEY_BLOCK_ENTRY_TYPE_BLOB, BLOCK_TYPE_INDEX, BLOCK_TYPE_KEY_NO_HASH, BLOCK_TYPE_KEY_WITH_HASH, KEY_BLOCK_ENTRY_TYPE_BLOB,
KEY_BLOCK_ENTRY_TYPE_DELETED, KEY_BLOCK_ENTRY_TYPE_INLINE_MIN, KEY_BLOCK_ENTRY_TYPE_MEDIUM, KEY_BLOCK_ENTRY_TYPE_DELETED, KEY_BLOCK_ENTRY_TYPE_INLINE_MIN, KEY_BLOCK_ENTRY_TYPE_MEDIUM,
KEY_BLOCK_ENTRY_TYPE_SMALL, KEY_BLOCK_ENTRY_TYPE_SMALL,
}; };
use turbo_persistence::{BLOCK_HEADER_SIZE, checksum_block, meta_file::MetaFile};
/// Block size information /// Block size information
#[derive(Default, Debug, Clone)] #[derive(Default, Debug, Clone)]
...@@ -252,9 +252,10 @@ fn analyze_sst_file(db_path: &Path, info: &SstInfo) -> Result<SstStats> { ...@@ -252,9 +252,10 @@ fn analyze_sst_file(db_path: &Path, info: &SstInfo) -> Result<SstStats> {
}; };
let block_end = blocks_start + (&mmap[offset..offset + 4]).read_u32::<BE>()? as usize; let block_end = blocks_start + (&mmap[offset..offset + 4]).read_u32::<BE>()? as usize;
// Read uncompressed length and compressed data // Read block header (uncompressed length + checksum) and block data
let uncompressed_length = (&mmap[block_start..block_start + 4]).read_u32::<BE>()?; let uncompressed_length = (&mmap[block_start..block_start + 4]).read_u32::<BE>()?;
let compressed_data = &mmap[block_start + 4..block_end]; let expected_checksum = (&mmap[block_start + 4..block_start + 8]).read_u32::<BE>()?;
let compressed_data = &mmap[block_start + BLOCK_HEADER_SIZE..block_end];
let compressed_size = compressed_data.len() as u64; let compressed_size = compressed_data.len() as u64;
// Determine if block was compressed (uncompressed_length > 0 means it was compressed) // Determine if block was compressed (uncompressed_length > 0 means it was compressed)
...@@ -266,6 +267,19 @@ fn analyze_sst_file(db_path: &Path, info: &SstInfo) -> Result<SstStats> { ...@@ -266,6 +267,19 @@ fn analyze_sst_file(db_path: &Path, info: &SstInfo) -> Result<SstStats> {
uncompressed_length as u64 uncompressed_length as u64
}; };
// Verify checksum on the raw on-disk data before decompression.
let actual_checksum = checksum_block(compressed_data);
if actual_checksum != expected_checksum {
bail!(
"Cache corruption detected: checksum mismatch in block {} of {:08}.sst (expected \
{:08x}, got {:08x})",
block_index,
info.sequence_number,
expected_checksum,
actual_checksum
);
}
let decompressed = match decompress_block(compressed_data, uncompressed_length) { let decompressed = match decompress_block(compressed_data, uncompressed_length) {
Ok(data) => data, Ok(data) => data,
Err(e) => { Err(e) => {
......
...@@ -29,6 +29,11 @@ pub fn decompress_into_arc(uncompressed_length: u32, block: &[u8]) -> Result<Arc ...@@ -29,6 +29,11 @@ pub fn decompress_into_arc(uncompressed_length: u32, block: &[u8]) -> Result<Arc
Ok(buffer) Ok(buffer)
} }
/// Computes a CRC32 checksum of a byte slice.
pub fn checksum_block(data: &[u8]) -> u32 {
crc32fast::hash(data)
}
#[tracing::instrument(level = "trace", skip_all)] #[tracing::instrument(level = "trace", skip_all)]
pub fn compress_into_buffer(block: &[u8], buffer: &mut Vec<u8>) -> Result<()> { pub fn compress_into_buffer(block: &[u8], buffer: &mut Vec<u8>) -> Result<()> {
lz4::compress_to_vec(block, buffer, lz4::ACC_LEVEL_DEFAULT).context("Compression failed")?; lz4::compress_to_vec(block, buffer, lz4::ACC_LEVEL_DEFAULT).context("Compression failed")?;
......
...@@ -24,7 +24,7 @@ use crate::{ ...@@ -24,7 +24,7 @@ use crate::{
DbConfig, FamilyKind, QueryKey, DbConfig, FamilyKind, QueryKey,
arc_bytes::ArcBytes, arc_bytes::ArcBytes,
compaction::selector::{Compactable, get_merge_segments}, compaction::selector::{Compactable, get_merge_segments},
compression::decompress_into_arc, compression::{checksum_block, decompress_into_arc},
constants::{ constants::{
DATA_THRESHOLD_PER_COMPACTED_FILE, KEY_BLOCK_AVG_SIZE, KEY_BLOCK_CACHE_SIZE, DATA_THRESHOLD_PER_COMPACTED_FILE, KEY_BLOCK_AVG_SIZE, KEY_BLOCK_CACHE_SIZE,
MAX_ENTRIES_PER_COMPACTED_FILE, VALUE_BLOCK_AVG_SIZE, VALUE_BLOCK_CACHE_SIZE, MAX_ENTRIES_PER_COMPACTED_FILE, VALUE_BLOCK_AVG_SIZE, VALUE_BLOCK_CACHE_SIZE,
...@@ -411,10 +411,23 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES> ...@@ -411,10 +411,23 @@ impl<S: ParallelScheduler, const FAMILIES: usize> TurboPersistence<S, FAMILIES>
mmap.advise(memmap2::Advice::DontFork)?; mmap.advise(memmap2::Advice::DontFork)?;
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
mmap.advise(memmap2::Advice::Unmergeable)?; mmap.advise(memmap2::Advice::Unmergeable)?;
let mut compressed = &mmap[..]; let mut reader = &mmap[..];
let uncompressed_length = compressed.read_u32::<BE>()?; let uncompressed_length = reader.read_u32::<BE>()?;
let expected_checksum = reader.read_u32::<BE>()?;
let buffer = decompress_into_arc(uncompressed_length, compressed)?; // Verify checksum on the compressed on-disk data before decompression.
let actual_checksum = checksum_block(reader);
if actual_checksum != expected_checksum {
bail!(
"Cache corruption detected: checksum mismatch in blob file {:08}.blob (expected \
{:08x}, got {:08x})",
seq,
expected_checksum,
actual_checksum
);
}
let buffer = decompress_into_arc(uncompressed_length, reader)?;
Ok(ArcBytes::from(buffer)) Ok(ArcBytes::from(buffer))
} }
......
...@@ -26,6 +26,7 @@ mod write_batch; ...@@ -26,6 +26,7 @@ mod write_batch;
mod tests; mod tests;
pub use arc_bytes::ArcBytes; pub use arc_bytes::ArcBytes;
pub use compression::checksum_block;
pub use db::{CompactConfig, MetaFileEntryInfo, MetaFileInfo, TurboPersistence}; pub use db::{CompactConfig, MetaFileEntryInfo, MetaFileInfo, TurboPersistence};
#[derive(Clone, Copy, Debug, PartialEq, Eq)] #[derive(Clone, Copy, Debug, PartialEq, Eq)]
...@@ -73,7 +74,7 @@ pub use static_sorted_file::{ ...@@ -73,7 +74,7 @@ pub use static_sorted_file::{
BlockCache, BlockWeighter, SstLookupResult, StaticSortedFile, StaticSortedFileMetaData, BlockCache, BlockWeighter, SstLookupResult, StaticSortedFile, StaticSortedFileMetaData,
}; };
pub use static_sorted_file_builder::{ pub use static_sorted_file_builder::{
Entry, EntryValue, StreamingSstWriter, write_static_stored_file, BLOCK_HEADER_SIZE, Entry, EntryValue, StreamingSstWriter, write_static_stored_file,
}; };
pub use value_buf::ValueBuffer; pub use value_buf::ValueBuffer;
pub use write_batch::WriteBatch; pub use write_batch::WriteBatch;
...@@ -24,6 +24,7 @@ pub enum LazyLookupValue { ...@@ -24,6 +24,7 @@ pub enum LazyLookupValue {
/// A medium sized value that is still compressed. /// A medium sized value that is still compressed.
Medium { Medium {
uncompressed_size: u32, uncompressed_size: u32,
checksum: u32,
block: ArcBytes, block: ArcBytes,
}, },
} }
...@@ -38,6 +39,7 @@ impl LazyLookupValue { ...@@ -38,6 +39,7 @@ impl LazyLookupValue {
LazyLookupValue::Medium { LazyLookupValue::Medium {
uncompressed_size, uncompressed_size,
block, block,
..
} => { } => {
if *uncompressed_size == 0 { if *uncompressed_size == 0 {
block.len() block.len()
...@@ -114,9 +116,11 @@ impl Entry for LookupEntry { ...@@ -114,9 +116,11 @@ impl Entry for LookupEntry {
}, },
LazyLookupValue::Medium { LazyLookupValue::Medium {
uncompressed_size, uncompressed_size,
checksum,
block, block,
} => EntryValue::MediumRaw { } => EntryValue::MediumRaw {
uncompressed_size: *uncompressed_size, uncompressed_size: *uncompressed_size,
checksum: *checksum,
block: block.as_ref(), block: block.as_ref(),
}, },
} }
......
...@@ -16,9 +16,10 @@ use smallvec::SmallVec; ...@@ -16,9 +16,10 @@ use smallvec::SmallVec;
use crate::{ use crate::{
QueryKey, QueryKey,
arc_bytes::ArcBytes, arc_bytes::ArcBytes,
compression::decompress_into_arc, compression::{checksum_block, decompress_into_arc},
constants::MAX_INLINE_VALUE_SIZE, constants::MAX_INLINE_VALUE_SIZE,
lookup_entry::{LazyLookupValue, LookupEntry, LookupValue}, lookup_entry::{LazyLookupValue, LookupEntry, LookupValue},
static_sorted_file_builder::BLOCK_HEADER_SIZE,
}; };
/// The block header for an index block. /// The block header for an index block.
...@@ -441,10 +442,33 @@ impl StaticSortedFile { ...@@ -441,10 +442,33 @@ impl StaticSortedFile {
self.read_block(block_index) self.read_block(block_index)
} }
/// Reads a block from the file. /// Verifies the CRC32 checksum of on-disk block data. Returns an error on mismatch.
fn verify_checksum(&self, data: &[u8], expected: u32, block_index: u16) -> Result<()> {
let actual = checksum_block(data);
if actual != expected {
bail!(
"Cache corruption detected: checksum mismatch in block {} of {:08}.sst (expected \
{:08x}, got {:08x})",
block_index,
self.meta.sequence_number,
expected,
actual
);
}
Ok(())
}
/// Reads a block from the file, decompressing if needed, and verifies its checksum.
///
/// The checksum is verified on the raw on-disk data **before** decompression, so
/// corruption is caught before passing data to LZ4.
#[tracing::instrument(level = "info", name = "reading database block", skip_all)] #[tracing::instrument(level = "info", name = "reading database block", skip_all)]
fn read_block(&self, block_index: u16) -> Result<ArcBytes> { fn read_block(&self, block_index: u16) -> Result<ArcBytes> {
let (uncompressed_length, block) = self.get_raw_block_slice(block_index)?; let (uncompressed_length, expected_checksum, block) =
self.get_raw_block_slice(block_index)?;
// Verify checksum on the raw on-disk data before decompression.
self.verify_checksum(block, expected_checksum, block_index)?;
// 0 means the block was not compressed, return the mmap-backed ArcBytes directly // 0 means the block was not compressed, return the mmap-backed ArcBytes directly
if uncompressed_length == 0 { if uncompressed_length == 0 {
...@@ -469,9 +493,13 @@ impl StaticSortedFile { ...@@ -469,9 +493,13 @@ impl StaticSortedFile {
/// Returns `(uncompressed_length, block_data)` as an owned `ArcBytes` backed by /// Returns `(uncompressed_length, block_data)` as an owned `ArcBytes` backed by
/// the mmap. Only use this when the block data needs to outlive the current borrow /// the mmap. Only use this when the block data needs to outlive the current borrow
/// (e.g. medium values stored in `LookupEntry`). /// (e.g. medium values stored in `LookupEntry`).
fn get_raw_block(&self, block_index: u16) -> Result<(u32, ArcBytes)> { fn get_raw_block(&self, block_index: u16) -> Result<(u32, u32, ArcBytes)> {
let (uncompressed_length, block) = self.get_raw_block_slice(block_index)?; let (uncompressed_length, checksum, block) = self.get_raw_block_slice(block_index)?;
Ok((uncompressed_length, self.mmap_slice_to_arc_bytes(block))) Ok((
uncompressed_length,
checksum,
self.mmap_slice_to_arc_bytes(block),
))
} }
/// Promotes a mmap subslice to an owned `ArcBytes`. This clones the `Arc<Mmap>`. /// Promotes a mmap subslice to an owned `ArcBytes`. This clones the `Arc<Mmap>`.
...@@ -482,7 +510,7 @@ impl StaticSortedFile { ...@@ -482,7 +510,7 @@ impl StaticSortedFile {
/// Gets the raw block slice directly from the memory mapped file, without /// Gets the raw block slice directly from the memory mapped file, without
/// cloning the `Arc<Mmap>`. The returned slice borrows from the mmap. /// cloning the `Arc<Mmap>`. The returned slice borrows from the mmap.
fn get_raw_block_slice(&self, block_index: u16) -> Result<(u32, &[u8])> { fn get_raw_block_slice(&self, block_index: u16) -> Result<(u32, u32, &[u8])> {
#[cfg(feature = "strict_checks")] #[cfg(feature = "strict_checks")]
if block_index >= self.meta.block_count { if block_index >= self.meta.block_count {
bail!( bail!(
...@@ -526,8 +554,9 @@ impl StaticSortedFile { ...@@ -526,8 +554,9 @@ impl StaticSortedFile {
} }
let uncompressed_length = let uncompressed_length =
u32::from_be_bytes(self.mmap[block_start..block_start + 4].try_into()?); u32::from_be_bytes(self.mmap[block_start..block_start + 4].try_into()?);
let block = &self.mmap[block_start + 4..block_end]; let checksum = u32::from_be_bytes(self.mmap[block_start + 4..block_start + 8].try_into()?);
Ok((uncompressed_length, block)) let block = &self.mmap[block_start + BLOCK_HEADER_SIZE..block_end];
Ok((uncompressed_length, checksum, block))
} }
} }
...@@ -626,9 +655,10 @@ impl StaticSortedFileIter { ...@@ -626,9 +655,10 @@ impl StaticSortedFileIter {
let value = if ty == KEY_BLOCK_ENTRY_TYPE_MEDIUM { let value = if ty == KEY_BLOCK_ENTRY_TYPE_MEDIUM {
let mut val = val; let mut val = val;
let block = val.read_u16::<BE>()?; let block = val.read_u16::<BE>()?;
let (uncompressed_size, block) = self.this.get_raw_block(block)?; let (uncompressed_size, checksum, block) = self.this.get_raw_block(block)?;
LazyLookupValue::Medium { LazyLookupValue::Medium {
uncompressed_size, uncompressed_size,
checksum,
block, block,
} }
} else { } else {
......
...@@ -11,7 +11,7 @@ use byteorder::{BE, ByteOrder, WriteBytesExt}; ...@@ -11,7 +11,7 @@ use byteorder::{BE, ByteOrder, WriteBytesExt};
use turbo_bincode::turbo_bincode_encode; use turbo_bincode::turbo_bincode_encode;
use crate::{ use crate::{
compression::compress_into_buffer, compression::{checksum_block, compress_into_buffer},
constants::{MAX_INLINE_VALUE_SIZE, MAX_SMALL_VALUE_SIZE, MIN_SMALL_VALUE_BLOCK_SIZE}, constants::{MAX_INLINE_VALUE_SIZE, MAX_SMALL_VALUE_SIZE, MIN_SMALL_VALUE_BLOCK_SIZE},
meta_file::{AmqfBincodeWrapper, MetaEntryFlags}, meta_file::{AmqfBincodeWrapper, MetaEntryFlags},
static_sorted_file::{ static_sorted_file::{
...@@ -21,6 +21,9 @@ use crate::{ ...@@ -21,6 +21,9 @@ use crate::{
}, },
}; };
/// Size of the per-block header on disk: 4 bytes uncompressed_size + 4 bytes CRC32 checksum.
pub const BLOCK_HEADER_SIZE: usize = 8;
/// The maximum number of entries that should go into a single key block /// The maximum number of entries that should go into a single key block
const MAX_KEY_BLOCK_ENTRIES: usize = MAX_KEY_BLOCK_SIZE / KEY_BLOCK_ENTRY_META_OVERHEAD; const MAX_KEY_BLOCK_ENTRIES: usize = MAX_KEY_BLOCK_SIZE / KEY_BLOCK_ENTRY_META_OVERHEAD;
/// The maximum bytes that should go into a single key block /// The maximum bytes that should go into a single key block
...@@ -153,6 +156,8 @@ pub enum EntryValue<'l> { ...@@ -153,6 +156,8 @@ pub enum EntryValue<'l> {
/// The uncompressed size of the block data. `0` means the block is stored uncompressed /// The uncompressed size of the block data. `0` means the block is stored uncompressed
/// (and thus the size is the `len` of the block) /// (and thus the size is the `len` of the block)
uncompressed_size: u32, uncompressed_size: u32,
/// CRC32 checksum of the on-disk block data (after compression).
checksum: u32,
block: &'l [u8], block: &'l [u8],
}, },
/// Large-sized value. They are stored in a blob file. /// Large-sized value. They are stored in a blob file.
...@@ -210,6 +215,7 @@ fn write_raw_block_to_file( ...@@ -210,6 +215,7 @@ fn write_raw_block_to_file(
file: &mut BufWriter<File>, file: &mut BufWriter<File>,
block_offsets: &mut Vec<u32>, block_offsets: &mut Vec<u32>,
uncompressed_size: u32, uncompressed_size: u32,
checksum: u32,
block: &[u8], block: &[u8],
) -> Result<u16> { ) -> Result<u16> {
let block_index: u16 = block_offsets let block_index: u16 = block_offsets
...@@ -217,7 +223,7 @@ fn write_raw_block_to_file( ...@@ -217,7 +223,7 @@ fn write_raw_block_to_file(
.try_into() .try_into()
.expect("Block index overflow"); .expect("Block index overflow");
let len: u32 = (block.len() + 4).try_into().unwrap(); let len: u32 = (block.len() + BLOCK_HEADER_SIZE).try_into().unwrap();
let offset = block_offsets let offset = block_offsets
.last() .last()
.copied() .copied()
...@@ -228,6 +234,8 @@ fn write_raw_block_to_file( ...@@ -228,6 +234,8 @@ fn write_raw_block_to_file(
file.write_u32::<BE>(uncompressed_size) file.write_u32::<BE>(uncompressed_size)
.context("Failed to write uncompressed size")?; .context("Failed to write uncompressed size")?;
file.write_u32::<BE>(checksum)
.context("Failed to write checksum")?;
file.write_all(block) file.write_all(block)
.context("Failed to write block data")?; .context("Failed to write block data")?;
Ok(block_index) Ok(block_index)
...@@ -253,7 +261,16 @@ fn write_block_to_file( ...@@ -253,7 +261,16 @@ fn write_block_to_file(
(0, block) (0, block)
}; };
let result = write_raw_block_to_file(file, block_offsets, uncompressed_size, data_to_write); // Checksum is computed on the on-disk data (after compression).
let checksum = checksum_block(data_to_write);
let result = write_raw_block_to_file(
file,
block_offsets,
uncompressed_size,
checksum,
data_to_write,
);
compress_buffer.clear(); compress_buffer.clear();
result result
} }
...@@ -500,6 +517,7 @@ impl<E: Entry> StreamingSstWriter<E> { ...@@ -500,6 +517,7 @@ impl<E: Entry> StreamingSstWriter<E> {
} }
EntryValue::MediumRaw { EntryValue::MediumRaw {
uncompressed_size, uncompressed_size,
checksum,
block, block,
} => { } => {
// Note: tracks compressed block size (not uncompressed) unlike EntryValue::Medium. // Note: tracks compressed block size (not uncompressed) unlike EntryValue::Medium.
...@@ -509,6 +527,7 @@ impl<E: Entry> StreamingSstWriter<E> { ...@@ -509,6 +527,7 @@ impl<E: Entry> StreamingSstWriter<E> {
self.file.as_mut().unwrap(), self.file.as_mut().unwrap(),
&mut self.block_offsets, &mut self.block_offsets,
uncompressed_size, uncompressed_size,
checksum,
block, block,
) )
.context("Failed to write compressed value block")?; .context("Failed to write compressed value block")?;
...@@ -761,33 +780,30 @@ impl<E: Entry> StreamingSstWriter<E> { ...@@ -761,33 +780,30 @@ impl<E: Entry> StreamingSstWriter<E> {
let mut file = self.file.take().unwrap(); let mut file = self.file.take().unwrap();
// Write index block directly to file (index blocks are never compressed). // Write index block (never compressed). Buffer into a Vec first so we can
// compute the checksum, then write via the standard block helper.
let index_entry_count: u16 = (self.key_block_boundaries.len() - 1) let index_entry_count: u16 = (self.key_block_boundaries.len() - 1)
.try_into() .try_into()
.expect("Index entries count overflow"); .expect("Index entries count overflow");
let index_block_size: u32 = (INDEX_BLOCK_HEADER_SIZE let index_block_size: usize =
+ index_entry_count as usize * INDEX_BLOCK_ENTRY_SIZE) INDEX_BLOCK_HEADER_SIZE + index_entry_count as usize * INDEX_BLOCK_ENTRY_SIZE;
.try_into() let mut index_buf = Vec::with_capacity(index_block_size);
.unwrap();
// Register block offset (uncompressed_size = 0 since we store raw).
{ {
let block_len = index_block_size + 4; // +4 for the uncompressed_size header let first_block = self.key_block_boundaries[0].1;
let offset = self let mut index_block = IndexBlockBuilder::new(&mut index_buf, first_block);
.block_offsets for &(hash, block) in &self.key_block_boundaries[1..] {
.last() index_block.put(hash, block);
.copied() }
.unwrap_or_default()
.checked_add(block_len)
.expect("Block offset overflow");
self.block_offsets.push(offset);
}
file.write_u32::<BE>(0)
.context("Failed to write index block header")?;
let first_block = self.key_block_boundaries[0].1;
let mut index_block = IndexBlockBuilder::new(&mut file, first_block);
for &(hash, block) in &self.key_block_boundaries[1..] {
index_block.put(hash, block);
} }
let index_checksum = checksum_block(&index_buf);
write_raw_block_to_file(
&mut file,
&mut self.block_offsets,
0,
index_checksum,
&index_buf,
)
.context("Failed to write index block")?;
// Write block offset table // Write block offset table
for offset in &self.block_offsets { for offset in &self.block_offsets {
...@@ -1155,6 +1171,7 @@ mod tests { ...@@ -1155,6 +1171,7 @@ mod tests {
TestValueKind::MediumRaw(v) => EntryValue::MediumRaw { TestValueKind::MediumRaw(v) => EntryValue::MediumRaw {
// uncompressed_size = 0 means the block is stored as-is (no compression). // uncompressed_size = 0 means the block is stored as-is (no compression).
uncompressed_size: 0, uncompressed_size: 0,
checksum: checksum_block(v),
block: v, block: v,
}, },
TestValueKind::Blob(id) => EntryValue::Large { blob: *id }, TestValueKind::Blob(id) => EntryValue::Large { blob: *id },
...@@ -1614,4 +1631,70 @@ mod tests { ...@@ -1614,4 +1631,70 @@ mod tests {
assert_lookup(&sst, &entries[0], &kc, &vc)?; assert_lookup(&sst, &entries[0], &kc, &vc)?;
Ok(()) Ok(())
} }
/// Flip a single byte in an SST file at the given position.
fn corrupt_sst_byte(dir: &Path, seq: u32, pos: u64) {
use std::io::{Seek, SeekFrom, Write as _};
let sst_path = dir.join(format!("{seq:08}.sst"));
let file_bytes = std::fs::read(&sst_path).unwrap();
let original = file_bytes[pos as usize];
let mut file = std::fs::OpenOptions::new()
.write(true)
.open(&sst_path)
.unwrap();
file.seek(SeekFrom::Start(pos)).unwrap();
file.write_all(&[original ^ 0xFF]).unwrap();
file.sync_all().unwrap();
}
/// Assert that looking up the first entry in a corrupted SST returns a corruption error.
fn assert_corruption_detected(
dir: &Path,
seq: u32,
meta: &StaticSortedFileBuilderMeta<'_>,
entries: &[TestEntry],
) {
let sst = open_sst(dir, seq, meta).unwrap();
let kc = make_cache();
let vc = make_cache();
match sst.lookup(entries[0].hash, &entries[0].key, &kc, &vc) {
Err(err) => {
let msg = format!("{err}");
assert!(
msg.contains("corruption"),
"Expected corruption error, got: {msg}"
);
}
Ok(_) => panic!("Expected checksum error, but lookup succeeded"),
}
}
#[test]
fn checksum_detects_corrupted_compressed_block() {
let dir = tempfile::tempdir().unwrap();
// Medium value is large enough to get its own value block, which will be compressed
let value = vec![0xCD; 8192];
let entries = vec![TestEntry::medium(b"mkey", &value)];
let meta = write_sst(dir.path(), 1, &entries, MetaEntryFlags::default()).unwrap();
// Corrupt the stored checksum of the first block (bytes 4..8).
// This guarantees a mismatch regardless of whether LZ4 decompression succeeds.
corrupt_sst_byte(dir.path(), 1, 4);
assert_corruption_detected(dir.path(), 1, &meta, &entries);
}
#[test]
fn checksum_detects_corrupted_uncompressed_block() {
let dir = tempfile::tempdir().unwrap();
// Single inline entry - the key block will be small and likely stored uncompressed
let entries = vec![TestEntry::inline(b"key1", b"val1")];
let meta = write_sst(dir.path(), 1, &entries, MetaEntryFlags::default()).unwrap();
// Corrupt a byte in the first block's data (after the 8-byte header)
corrupt_sst_byte(dir.path(), 1, BLOCK_HEADER_SIZE as u64 + 1);
assert_corruption_detected(dir.path(), 1, &meta, &entries);
}
} }
...@@ -18,7 +18,7 @@ use crate::{ ...@@ -18,7 +18,7 @@ use crate::{
FamilyConfig, ValueBuffer, FamilyConfig, ValueBuffer,
collector::Collector, collector::Collector,
collector_entry::CollectorEntry, collector_entry::CollectorEntry,
compression::compress_into_buffer, compression::{checksum_block, compress_into_buffer},
constants::{MAX_MEDIUM_VALUE_SIZE, THREAD_LOCAL_SIZE_SHIFT}, constants::{MAX_MEDIUM_VALUE_SIZE, THREAD_LOCAL_SIZE_SHIFT},
key::StoreKey, key::StoreKey,
meta_file::MetaEntryFlags, meta_file::MetaEntryFlags,
...@@ -433,11 +433,15 @@ impl<K: StoreKey + Send + Sync, S: ParallelScheduler, const FAMILIES: usize> ...@@ -433,11 +433,15 @@ impl<K: StoreKey + Send + Sync, S: ParallelScheduler, const FAMILIES: usize>
#[tracing::instrument(level = "trace", skip(self, value), fields(value_len = value.len()))] #[tracing::instrument(level = "trace", skip(self, value), fields(value_len = value.len()))]
fn create_blob(&self, value: &[u8]) -> Result<(u32, File)> { fn create_blob(&self, value: &[u8]) -> Result<(u32, File)> {
let seq = self.current_sequence_number.fetch_add(1, Ordering::SeqCst) + 1; let seq = self.current_sequence_number.fetch_add(1, Ordering::SeqCst) + 1;
let mut buffer = Vec::new(); let mut compressed = Vec::new();
buffer.write_u32::<BE>(value.len() as u32)?; compress_into_buffer(value, &mut compressed)
compress_into_buffer(value, &mut buffer)
.context("Compression of value for blob file failed")?; .context("Compression of value for blob file failed")?;
let mut buffer = Vec::with_capacity(8 + compressed.len());
buffer.write_u32::<BE>(value.len() as u32)?;
buffer.write_u32::<BE>(checksum_block(&compressed))?;
buffer.extend_from_slice(&compressed);
let file = self.db_path.join(format!("{seq:08}.blob")); let file = self.db_path.join(format!("{seq:08}.blob"));
let mut file = File::create(&file).context("Unable to create blob file")?; let mut file = File::create(&file).context("Unable to create blob file")?;
file.write_all(&buffer) file.write_all(&buffer)
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment