turbo-persistence: read compaction inputs with pread instead of mmap (#100336)
## Summary Compaction iterated its input SST files through an mmap with `MADV_SEQUENTIAL`. This does not fit the block layout pattern (which tends to be ValueBlock, ValueBlock, KeyBlock....) because iteration starts from key blocks, so the readahead isn't well targeted. OS level reclaim can recover this memory but it isn't eager and we _know_ after reading a block that we don't need it any more. `StaticSortedFileIter`, which only compaction uses, now always reads blocks with `pread` into owned buffers, independent of the overall database's `AccessMode`, so an open input file holds only the blocks it currently uses. This also enhances SharedBytes with a mechanism to be constructed from an `uninit` buffer (useful for decompression and `pread` targets). This unifies a few codepaths and allows us to avoid some small buffer copies. ### Measurements Full compaction on Linux (podman VM, kernel 7.2, XFS on virtio, page cache dropped before each run, single threaded), 2 rounds: | | time | max RSS | peak mapped file pages | |---|---|---|---| | 1.1 GB zstd database, values 16 B – 8 KiB, mmap (before) | 2.10 / 2.33 s | 315 MB | 301 – 318 MB | | same, `pread` (this PR) | 2.07 / 2.04 s | 10 MB | 5 MB | | 230 MB database of 16Mi tiny entries, mmap (before) | 2.73 / 2.64 s | 245 MB | 201 MB | | same, `pread` (this PR) | 2.35 / 2.37 s | 57 MB | 6 MB | The cgroup's `memory.peak` is the same in both, since it is dominated by page cache, which is charged either way and reclaimable: with a 512 MB cgroup limit both variants complete without OOM kills. What `pread` removes is the input files from the process' RSS. ### Why not mmap with `MADV_RANDOM` and explicit `WILLNEED`/`DONTNEED` That would let the iterator prefetch the blocks it is about to read and drop those behind it, but at best it reaches the memory profile `pread` already has, and adds costs: - `MADV_DONTNEED` on a file mapping only unmaps the pages, they stay in page cache like with `pread`. - With `MADV_RANDOM` there is no readahead, and `WILLNEED` only starts the I/O, it doesn't map pages, so every 4 KiB page still takes a (minor) fault. `pread` copies ~10 KB per call without page table work. - Unmapping requires a TLB shootdown on every CPU running the process, and compaction runs merges on many threads in the same address space as turbo-tasks. This cost grows with concurrency. - `WILLNEED` and `DONTNEED` take kernel level locks, which can lead to contention in addition to general syscall overhead. <!-- NEXT_JS_LLM --> --------- Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by:
Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Showing
This diff is collapsed.
Please
register
or
sign in
to comment