Unverified Commit 4bb4e8f1 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

turbo-bitset: make block storage pluggable (#100779)

### What?

`turbo_bitset::Bitset` can now use different storage for its blocks. A
new public `BlockStorage` trait defines what that storage must support.

- `Bitset` is now `Bitset<S: BlockStorage = Vec<usize>>`, so every
existing user keeps `Vec<usize>` storage.
- The trait is implemented for `Vec<usize>`, `SmallVec<[usize; N]>` and
`ArrayVec<usize, N>`. `turbo-bitset` now depends on `smallvec` and
`arrayvec` unconditionally.
- All `Bitset` methods and operators, plus `Encode`, `Decode` and
`ShrinkToFit`, work with any storage. Set operations require both sides
to use the same storage. The `turbo-tasks` integrations remain on the
default `Bitset`.
- The encoding doesn't depend on the storage, so bytes written with one
storage decode into any other.
- With `ArrayVec`, operations that would need more than its capacity of
`N` blocks panic. Decoding a set that doesn't fit returns a
`DecodeError`.
- `shrink_to_fit` only touches the storage when that frees capacity. An
`ArrayVec` or an inline `SmallVec` is left unchanged.

### Why?

This lets small sets use inline storage where that helps, without making
every `Bitset` user change its storage type.

### How?

- **Bounded storage:** when an operation would exceed an `ArrayVec`'s
capacity, empty blocks at either end of the storage are dropped first.
This fixes a case where an insert at the back panicked although the bits
fit, because earlier growth at the front had filled the capacity with
zero blocks.
- **`intersection`:** builds its result with `FromIterator`, so a `Vec`
keeps its exact-size allocation.
- **Inlining:** only the small methods of the non-generic `Vec` storage
are marked `#[inline]`. The generic `SmallVec` and `ArrayVec`
implementations and `Iter::next` have no annotation; removing those had
no effect on the benchmark binary (see below).
- **Tests:** the shared semantic tests run for all three storages. New
tests cover `ArrayVec` overflow and decode errors, `SmallVec`
inline/spill behaviour, `shrink_to_fit`, zero-capacity storage, and
encoding equality across storages. The benchmarks gain cases for
`SmallVec` and `ArrayVec`.

### Verification

- `cargo fmt --check -p turbo-bitset -p turbo-tasks -p turbopack-core`
- `cargo clippy -q -p turbo-bitset -p turbo-tasks -p turbopack-core -p
next-api --all-targets -- -D warnings`
- `RUSTDOCFLAGS="-D warnings" cargo doc -q --no-deps -p turbo-bitset -p
turbopack-core`
- `cargo check -q -p next-api`
- `cargo test -q -p turbo-bitset`
- `cargo test -q -p turbo-tasks --lib`
- `cargo test -q -p turbopack-core --lib`
- `cargo test -q -p turbopack-tests --test snapshot` (no snapshot
changes)
- `cargo bench -q -p turbo-bitset --bench mod -- --test
'bitset/storage_'` (216 cases)
- `cargo run -q --release -p turbo-bitset --example sizes` gives output
identical to before this change.
- Not run: a 32-bit cross-check, because the `i686-unknown-linux-gnu`
standard library is not installed locally. The test and benchmark
changes avoid assuming a 64-bit `usize`.

### Performance caveats

I compared release benchmark binaries for the previous `Vec`-only
implementation and this change. Each side was built in its own target
directory, I checked that the binary hashes differ, and the runs
alternated old/new/new/old.

- Before the final storage changes, the full comparison (336 cases)
averaged **0.992×** of the old timings.
- Those final changes switched `intersection` to `collect` and added
`#[inline]`. After them, a focused rerun of 84 previously slower cases
averaged **1.040×**:

| Group | New / old (geomean) |
| --- | ---: |
| `intersection` | 0.907 |
| `union` | 0.974 |
| `contains` | 1.049 |
| `insert` | 1.057 |
| `iter` | 1.112 |
| `from_iter` | 1.165 |

- **`iter` and `contains`:** some inputs remain slower even though their
normalized hot-loop instructions are the same in both binaries. The
cause is unresolved.
- **Empty `from_iter`:** about 3 ns becomes about 5 ns because of an
extra out-of-line call.

### `#[inline]` annotations

At `17be1d5e`, which annotated `Iter::next` and every `BlockStorage`
method, I compared builds with and without all 43 of those annotations:
full `mod` bench, 768 cases, runs alternating with/without/without/with,
separate target dirs, release profile (thin LTO, one codegen unit).

- Without any of them, the timings were **1.091×** overall (1.086× and
1.095× in the two pairs). The same binary run twice drifted 1.024× and
1.033×, but per group the same-binary drift reached about 30%, so this
is a shared-VM microbenchmark trend, not a proven whole-build effect.
- Removing only the annotation on `Iter::next`, or only the 28 on the
generic `SmallVec`/`ArrayVec` implementations, gives a bench binary
**byte-identical** to the annotated one.
- Removing the 14 on the `Vec` implementation reproduces the whole
difference. Without them, `Vec`'s `with_capacity` and `resize` wrappers
remain separate functions in the binary, even with thin LTO.
- So only the `Vec` annotations are kept. The bench binary of this PR's
tip is byte-identical to the one built at `17be1d5e`.

<!-- NEXT_JS_LLM -->


<!-- fleet b56269ed-2145-4f30-b0fe-dc9762520521 -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarVercel Fleet <fleet@vercel.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: 's avatarLuke Sandberg <210140+lukesandberg@users.noreply.github.com>
parent b35fb85d
......@@ -9113,11 +9113,13 @@ dependencies = [
name = "turbo-bitset"
version = "0.1.0"
dependencies = [
"arrayvec 0.7.6",
"bincode",
"codspeed-criterion-compat",
"rand 0.10.1",
"rustc-hash 2.1.1",
"shrink-to-fit",
"smallvec",
"turbo-bincode",
"turbo-tasks-hash",
]
......
......@@ -255,6 +255,7 @@ allsorts = { version = "0.14.0", default-features = false, features = [
"flate2_rust",
] }
anyhow = "1.0.100"
arrayvec = "0.7.6"
async-trait = "0.1.64"
base64 = "0.22.1"
bincode = { version = "2.0.1", features = ["serde"] }
......
......@@ -9,8 +9,10 @@ edition = "2024"
workspace = true
[dependencies]
arrayvec = { workspace = true }
bincode = { workspace = true }
shrink-to-fit = { workspace = true }
smallvec = { workspace = true }
[dev-dependencies]
criterion = { workspace = true }
......
......@@ -10,6 +10,10 @@
//! The `hash` group hashes the sets with `FxHasher` through `std::hash::Hash`, and `hash_encode`
//! hashes a `Bitset` like a task input is hashed for the persistent cache: encoded with
//! `turbo_bincode`'s hash encoder into an `Xxh3Hash64Hasher`.
//!
//! The `storage_*` groups compare the block storages on a few operations, with the ids
//! `<storage>/<input>`: `Vec` (the default), `SmallVec4` (`SmallVec<[usize; 4]>`) and
//! `ArrayVec` (an `ArrayVec<usize, N>` with enough blocks for the largest universe).
#[path = "common/inputs.rs"]
mod inputs;
......@@ -20,17 +24,22 @@ use std::{
hint::black_box,
};
use arrayvec::ArrayVec;
use bincode::Encode;
use criterion::{BatchSize, BenchmarkId, Criterion, criterion_group, criterion_main};
use criterion::{
BatchSize, BenchmarkGroup, BenchmarkId, Criterion, criterion_group, criterion_main,
measurement::WallTime,
};
use rand::seq::SliceRandom;
use rustc_hash::FxBuildHasher;
use smallvec::SmallVec;
use turbo_bincode::{
TurboBincodeBuffer, new_hash_encoder, turbo_bincode_decode, turbo_bincode_encode,
};
use turbo_bitset::Bitset;
use turbo_bitset::{Bitset, BlockStorage};
use turbo_tasks_hash::Xxh3Hash64Hasher;
use self::inputs::{Input, inputs, rng};
use self::inputs::{Input, UNIVERSES, inputs, rng};
fn bitset(bits: &[u32]) -> Bitset {
bits.iter().copied().collect()
......@@ -141,7 +150,7 @@ fn bench_insert(c: &mut Criterion, name: &str, inputs: &[Input], order: fn(&[u32
inputs,
|i| order(&i.bits),
|bits| {
let mut set = Bitset::new();
let mut set: Bitset = Bitset::new();
for &bit in bits {
black_box(set.insert(bit));
}
......@@ -150,6 +159,95 @@ fn bench_insert(c: &mut Criterion, name: &str, inputs: &[Input], order: fn(&[u32
);
}
/// The `ArrayVec` capacity of the storage benchmarks, enough for the largest universe.
const ARRAY_BLOCKS: usize =
(UNIVERSES[UNIVERSES.len() - 1] as usize).div_ceil(usize::BITS as usize);
/// Inserting the bits of `input` one by one in a fixed random order, with the storage `S`.
fn storage_insert<S: BlockStorage>(
group: &mut BenchmarkGroup<'_, WallTime>,
storage: &str,
input: &Input,
) {
let bits = shuffled(&input.bits);
group.bench_function(BenchmarkId::new(storage, &input.name), |b| {
b.iter(|| {
let mut set = Bitset::<S>::new();
for &bit in black_box(&bits) {
black_box(set.insert(bit));
}
black_box(set);
})
});
}
/// `contains` for the probes of `input`, with the storage `S`.
fn storage_contains<S: BlockStorage>(
group: &mut BenchmarkGroup<'_, WallTime>,
storage: &str,
input: &Input,
) {
let set = input.bits.iter().copied().collect::<Bitset<S>>();
group.bench_function(BenchmarkId::new(storage, &input.name), |b| {
b.iter(|| {
for &bit in black_box(&input.probes) {
black_box(black_box(&set).contains(bit));
}
})
});
}
/// `a |= b` like the `union` group, with the storage `S`.
fn storage_union<S: BlockStorage>(
group: &mut BenchmarkGroup<'_, WallTime>,
storage: &str,
input: &Input,
) {
let a = input.bits.iter().copied().collect::<Bitset<S>>();
let b = input.other_bits.iter().copied().collect::<Bitset<S>>();
group.bench_function(BenchmarkId::new(storage, &input.name), |bencher| {
bencher.iter_batched_ref(
|| a.clone(),
|a| {
*a |= black_box(&b);
black_box(&*a);
},
BatchSize::SmallInput,
)
});
}
/// Runs a `storage_*` function for every compared storage.
macro_rules! for_each_storage {
($bench:ident, $group:expr, $input:expr) => {
$bench::<Vec<usize>>($group, "Vec", $input);
$bench::<SmallVec<[usize; 4]>>($group, "SmallVec4", $input);
$bench::<ArrayVec<usize, ARRAY_BLOCKS>>($group, "ArrayVec", $input);
};
}
/// Compares the block storages on inserting, `contains` and `|=`.
fn bench_storages(c: &mut Criterion, inputs: &[Input]) {
for input in inputs {
assert!(input.universe as usize <= ARRAY_BLOCKS * usize::BITS as usize);
}
let mut group = c.benchmark_group("bitset/storage_insert");
for input in inputs {
for_each_storage!(storage_insert, &mut group, input);
}
group.finish();
let mut group = c.benchmark_group("bitset/storage_contains");
for input in inputs {
for_each_storage!(storage_contains, &mut group, input);
}
group.finish();
let mut group = c.benchmark_group("bitset/storage_union");
for input in inputs {
for_each_storage!(storage_union, &mut group, input);
}
group.finish();
}
pub fn bench(c: &mut Criterion) {
let inputs = inputs();
validate(&inputs);
......@@ -359,6 +457,8 @@ pub fn bench(c: &mut Criterion) {
black_box(hash_encode(black_box(set)));
},
);
bench_storages(c, &inputs);
}
/// The bits in a fixed pseudo-random order.
......
......@@ -27,6 +27,10 @@
//!
//! With fixed-width integers (used to hash values instead of storing them) the bitmap form is
//! always used. The decoder accepts both forms with any config.
//!
//! The encoding doesn't depend on the [`BlockStorage`], so a set can be decoded into a different
//! storage than it was encoded from. Decoding a set that needs more blocks than the storage can
//! hold returns an error.
use bincode::{
Decode, Encode,
......@@ -40,7 +44,7 @@ use bincode::{
impl_borrow_decode,
};
use crate::{BLOCK_BITS, Bitset};
use crate::{BLOCK_BITS, Bitset, BlockStorage};
/// Number of bytes per storage block.
const BLOCK_BYTES: usize = BLOCK_BITS / 8;
......@@ -163,7 +167,7 @@ fn write_list_items<E: Encoder>(
Ok(())
}
impl Bitset {
impl<S: BlockStorage> Bitset<S> {
/// The trimmed byte range of the set, or `None` if it is empty.
fn byte_range(&self) -> Option<ByteRange<'_>> {
let (first_block, blocks) = self.trimmed();
......@@ -179,7 +183,7 @@ impl Bitset {
}
}
impl Encode for Bitset {
impl<S: BlockStorage> Encode for Bitset<S> {
fn encode<E: Encoder>(&self, encoder: &mut E) -> Result<(), EncodeError> {
let Some(range) = self.byte_range() else {
return 0i64.encode(encoder);
......@@ -203,7 +207,7 @@ impl Encode for Bitset {
}
}
impl<Context> Decode<Context> for Bitset {
impl<Context, S: BlockStorage> Decode<Context> for Bitset<S> {
fn decode<D: Decoder<Context = Context>>(decoder: &mut D) -> Result<Self, DecodeError> {
let len = i64::decode(decoder)?;
match len {
......@@ -214,39 +218,51 @@ impl<Context> Decode<Context> for Bitset {
}
}
impl_borrow_decode!(Bitset);
impl_borrow_decode!(Bitset<S>, S: BlockStorage);
/// Allocates the zeroed storage for the bytes `byte_offset..byte_offset + len`, claiming its size
/// from the decoder's limit first. Returns the index of the first block and the storage.
fn allocate<D: Decoder>(
fn allocate<D: Decoder, S: BlockStorage>(
decoder: &mut D,
byte_offset: u64,
len: u64,
) -> Result<(usize, Vec<usize>), DecodeError> {
) -> Result<(usize, S), DecodeError> {
if len > MAX_BYTES || byte_offset + len > MAX_BYTES {
return Err(DecodeError::Other("Bitset extends beyond the u32 range"));
}
let block_bytes = BLOCK_BYTES as u64;
let first_block = byte_offset / block_bytes;
let block_count = ((byte_offset + len).div_ceil(block_bytes) - first_block) as usize;
if block_count > S::max_len() {
return Err(DecodeError::Other("Bitset doesn't fit into its storage"));
}
decoder.claim_bytes_read(block_count * BLOCK_BYTES)?;
Ok((first_block as usize, vec![0; block_count]))
let mut blocks = S::with_capacity(block_count);
blocks.resize(block_count);
Ok((first_block as usize, blocks))
}
fn decode_bitmap<D: Decoder>(decoder: &mut D, len: u64) -> Result<Bitset, DecodeError> {
fn decode_bitmap<D: Decoder, S: BlockStorage>(
decoder: &mut D,
len: u64,
) -> Result<Bitset<S>, DecodeError> {
let byte_offset = u32::decode(decoder)? as u64;
let (first_block, mut blocks) = allocate(decoder, byte_offset, len)?;
let (first_block, mut blocks) = allocate::<_, S>(decoder, byte_offset, len)?;
// Read the payload straight into the storage's bytes.
let start = (byte_offset - first_block as u64 * BLOCK_BYTES as u64) as usize;
// SAFETY: The slice covers exactly the memory of `blocks` and borrows it mutably for its
let block_slice = blocks.as_mut_slice();
// SAFETY: The slice covers exactly the memory of `block_slice` and borrows it mutably for its
// lifetime. `u8` has no alignment requirements, and every byte pattern is a valid `usize`.
let bytes = unsafe {
std::slice::from_raw_parts_mut(blocks.as_mut_ptr().cast::<u8>(), size_of_val(&*blocks))
std::slice::from_raw_parts_mut(
block_slice.as_mut_ptr().cast::<u8>(),
size_of_val(&*block_slice),
)
};
decoder
.reader()
.read(&mut bytes[start..start + len as usize])?;
le_blocks_to_native(&mut blocks, cfg!(target_endian = "big"));
le_blocks_to_native(blocks.as_mut_slice(), cfg!(target_endian = "big"));
Ok(Bitset {
offset: first_block,
blocks,
......@@ -264,12 +280,12 @@ fn le_blocks_to_native(blocks: &mut [usize], big_endian_target: bool) {
}
/// Sets bits in a preallocated storage, keeping the bits of the current block in a register.
struct ListBuilder {
struct ListBuilder<S> {
/// The first bit of the storage.
base: u64,
/// The end of the bits that may be set (exclusive), relative to `base`.
end: u64,
blocks: Vec<usize>,
blocks: S,
/// The index of the current block and its bits, not yet stored in `blocks`.
index: usize,
word: usize,
......@@ -277,7 +293,7 @@ struct ListBuilder {
last: u64,
}
impl ListBuilder {
impl<S: BlockStorage> ListBuilder<S> {
/// Sets the bit `last + difference`, which must be within the storage.
fn push(&mut self, difference: u64) -> Result<(), DecodeError> {
let bit = self.last + difference;
......@@ -286,7 +302,7 @@ impl ListBuilder {
}
let index = (bit / BLOCK_BITS as u64) as usize;
if index != self.index {
self.blocks[self.index] |= self.word;
self.blocks.as_mut_slice()[self.index] |= self.word;
self.index = index;
self.word = 0;
}
......@@ -295,8 +311,8 @@ impl ListBuilder {
Ok(())
}
fn finish(mut self) -> Vec<usize> {
self.blocks[self.index] |= self.word;
fn finish(mut self) -> S {
self.blocks.as_mut_slice()[self.index] |= self.word;
self.blocks
}
}
......@@ -310,7 +326,10 @@ fn decode_difference<D: Decoder>(decoder: &mut D) -> Result<u64, DecodeError> {
Ok(difference as u64)
}
fn decode_list<D: Decoder>(decoder: &mut D, count: u64) -> Result<Bitset, DecodeError> {
fn decode_list<D: Decoder, S: BlockStorage>(
decoder: &mut D,
count: u64,
) -> Result<Bitset<S>, DecodeError> {
if count > 1 << 32 {
return Err(DecodeError::Other(
"Bitset list has more items than the u32 range",
......@@ -324,7 +343,7 @@ fn decode_list<D: Decoder>(decoder: &mut D, count: u64) -> Result<Bitset, Decode
}
let first = u32::decode(decoder)? as u64;
let byte_offset = first / 8;
let (first_block, blocks) = allocate(decoder, byte_offset, len)?;
let (first_block, blocks) = allocate::<_, S>(decoder, byte_offset, len)?;
let base = (first_block * BLOCK_BITS) as u64;
let mut builder = ListBuilder {
base,
......
This diff is collapsed.
//! The storage of the blocks of a [`Bitset`](crate::Bitset).
use arrayvec::ArrayVec;
use smallvec::SmallVec;
/// A growable list of `usize` blocks that a [`Bitset`](crate::Bitset) stores its bits in.
///
/// Implemented for:
///
/// - `Vec<usize>` (the default): grows without a limit, and [`shrink_to_fit`] releases all spare
/// capacity.
/// - `SmallVec<[usize; N]>`: stores up to `N` blocks inline and moves to the heap beyond that, like
/// a `Vec`. [`shrink_to_fit`] moves the blocks back inline when they fit.
/// - `ArrayVec<usize, N>`: always stores the blocks inline, so it holds at most `N` blocks. Growing
/// beyond that panics, and [`shrink_to_fit`] can't release anything.
///
/// The methods follow the `Vec` methods of the same name. Every method that adds blocks panics if
/// the result would exceed [`max_len`] blocks. Collecting more blocks than that from an iterator
/// (`FromIterator`) panics the same way.
///
/// An `ArrayVec<usize, N>` always has a capacity of exactly `N` blocks, so
/// [`with_capacity`](BlockStorage::with_capacity) accepts any request up to `N` and panics above
/// it.
///
/// [`shrink_to_fit`]: BlockStorage::shrink_to_fit
/// [`max_len`]: BlockStorage::max_len
pub trait BlockStorage: Clone + FromIterator<usize> {
/// Creates an empty storage without allocating.
fn new() -> Self;
/// Creates an empty storage with capacity for at least `capacity` blocks.
///
/// The capacity is exact if the storage needs to allocate for it. A storage with a fixed
/// capacity accepts any request up to [`BlockStorage::max_len`] and panics above it.
fn with_capacity(capacity: usize) -> Self;
/// The maximum number of blocks that the storage can hold.
fn max_len() -> usize;
/// The number of blocks that the storage can hold without reallocating.
fn capacity(&self) -> usize;
/// The capacity that a storage of `len` blocks has after [`BlockStorage::shrink_to_fit`]. A
/// set only moves its blocks to shrink the storage if this is below the current capacity.
fn shrunk_capacity(len: usize) -> usize;
/// The stored blocks.
fn as_slice(&self) -> &[usize];
/// The stored blocks, mutably.
fn as_mut_slice(&mut self) -> &mut [usize];
/// Resizes the storage to `len` blocks, appending zero blocks if it grows.
fn resize(&mut self, len: usize);
/// Appends `blocks`.
fn extend_from_slice(&mut self, blocks: &[usize]);
/// Reserves capacity for exactly `additional` more blocks, if the storage allocates.
fn reserve_exact(&mut self, additional: usize);
/// Keeps the first `len` blocks, dropping the rest.
fn truncate(&mut self, len: usize);
/// Removes the first `len` blocks, moving the others to the front.
fn remove_prefix(&mut self, len: usize);
/// Removes all blocks, keeping the capacity.
fn clear(&mut self);
/// Releases as much capacity as possible.
fn shrink_to_fit(&mut self);
}
// Only the `Vec` implementation is `#[inline]`: it isn't generic, so the attribute makes its small
// wrappers available for inlining in other crates; without it, some of them stay out of line even
// with the workspace's thin LTO. The `SmallVec` and `ArrayVec` implementations are generic and
// therefore instantiated in the calling crate.
impl BlockStorage for Vec<usize> {
#[inline]
fn new() -> Self {
Vec::new()
}
#[inline]
fn with_capacity(capacity: usize) -> Self {
Vec::with_capacity(capacity)
}
#[inline]
fn max_len() -> usize {
usize::MAX
}
#[inline]
fn capacity(&self) -> usize {
self.capacity()
}
#[inline]
fn shrunk_capacity(len: usize) -> usize {
len
}
#[inline]
fn as_slice(&self) -> &[usize] {
self
}
#[inline]
fn as_mut_slice(&mut self) -> &mut [usize] {
self
}
#[inline]
fn resize(&mut self, len: usize) {
self.resize(len, 0);
}
#[inline]
fn extend_from_slice(&mut self, blocks: &[usize]) {
self.extend_from_slice(blocks);
}
#[inline]
fn reserve_exact(&mut self, additional: usize) {
self.reserve_exact(additional);
}
#[inline]
fn truncate(&mut self, len: usize) {
self.truncate(len);
}
#[inline]
fn remove_prefix(&mut self, len: usize) {
self.drain(..len);
}
#[inline]
fn clear(&mut self) {
self.clear();
}
#[inline]
fn shrink_to_fit(&mut self) {
self.shrink_to_fit();
}
}
impl<const N: usize> BlockStorage for SmallVec<[usize; N]> {
fn new() -> Self {
SmallVec::new()
}
fn with_capacity(capacity: usize) -> Self {
SmallVec::with_capacity(capacity)
}
fn max_len() -> usize {
usize::MAX
}
fn capacity(&self) -> usize {
self.capacity()
}
fn shrunk_capacity(len: usize) -> usize {
len.max(N)
}
fn as_slice(&self) -> &[usize] {
self
}
fn as_mut_slice(&mut self) -> &mut [usize] {
self
}
fn resize(&mut self, len: usize) {
self.resize(len, 0);
}
fn extend_from_slice(&mut self, blocks: &[usize]) {
self.extend_from_slice(blocks);
}
fn reserve_exact(&mut self, additional: usize) {
self.reserve_exact(additional);
}
fn truncate(&mut self, len: usize) {
self.truncate(len);
}
fn remove_prefix(&mut self, len: usize) {
self.drain(..len);
}
fn clear(&mut self) {
self.clear();
}
fn shrink_to_fit(&mut self) {
self.shrink_to_fit();
}
}
/// Panics because an `ArrayVec` with capacity `N` can't hold `len` blocks.
#[cold]
#[track_caller]
fn array_overflow<const N: usize>(len: usize) -> ! {
panic!("a Bitset storage of ArrayVec<usize, {N}> can't hold {len} blocks")
}
impl<const N: usize> BlockStorage for ArrayVec<usize, N> {
fn new() -> Self {
ArrayVec::new()
}
#[track_caller]
fn with_capacity(capacity: usize) -> Self {
if capacity > N {
array_overflow::<N>(capacity);
}
ArrayVec::new()
}
fn max_len() -> usize {
N
}
fn capacity(&self) -> usize {
N
}
fn shrunk_capacity(_len: usize) -> usize {
N
}
fn as_slice(&self) -> &[usize] {
self
}
fn as_mut_slice(&mut self) -> &mut [usize] {
self
}
#[track_caller]
fn resize(&mut self, len: usize) {
if len > N {
array_overflow::<N>(len);
}
if len <= self.len() {
self.truncate(len);
} else {
for _ in self.len()..len {
self.push(0);
}
}
}
#[track_caller]
fn extend_from_slice(&mut self, blocks: &[usize]) {
if self.try_extend_from_slice(blocks).is_err() {
array_overflow::<N>(self.len() + blocks.len());
}
}
#[track_caller]
fn reserve_exact(&mut self, additional: usize) {
if self.len() + additional > N {
array_overflow::<N>(self.len() + additional);
}
}
fn truncate(&mut self, len: usize) {
self.truncate(len);
}
fn remove_prefix(&mut self, len: usize) {
self.drain(..len);
}
fn clear(&mut self) {
self.clear();
}
fn shrink_to_fit(&mut self) {}
}
This diff is collapsed.
......@@ -285,7 +285,7 @@ fn front_and_back_growth() {
#[test]
fn front_growth_reuses_capacity() {
let block = usize::BITS;
let mut set = Bitset::new();
let mut set: Bitset = Bitset::new();
let mut expected = BTreeSet::new();
for bit in [100 * block + 3, 101 * block] {
set.insert(bit);
......@@ -529,7 +529,7 @@ fn union_with_into_empty_is_exact() {
#[test]
fn union_with_empty_source_keeps_storage() {
let mut removed = Bitset::new();
let mut removed: Bitset = Bitset::new();
removed.insert(5_000);
removed.remove(5_000);
let sources = [Bitset::new(), Bitset::with_len(500), removed];
......@@ -754,7 +754,7 @@ fn front_growth_beyond_the_capacity_keeps_all_bits() {
50 * block - 1,
),
] {
let mut set = Bitset::new();
let mut set: Bitset = Bitset::new();
let mut expected = BTreeSet::new();
for &bit in existing {
set.insert(bit);
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment