trace-server: shrink reader buffers, compact eagerly

The accumulation buffer only compacted on reaching capacity, and
extend_from_slice doubles, so with a 64 MB read chunk it ratcheted upward
instead of staying near one chunk. On a 62 MB trace it held essentially the
whole file: 687 MB live for a 62 MB input.

To fix, we read 4 MB instead of 64 MB, and drop the consumed prefix as
soon as we finish each parse. We are essentially left with one partial
row. This is safe to change, because the format's re-used row vec
borrows directly from the buffer within a read call and is dropped
before returning.

At our artificial (but reproducible) STOP_AT=2000, that reduces live heap
by ~600 MB at ~7M spans.

I alternated through 4/8/16/64 MB buffers, and found it didn't affect
read time because of the underlying IO, so a large buffer buys us
nothing but memory accumulation.
parent b0e57898
...@@ -24,6 +24,14 @@ use crate::{ ...@@ -24,6 +24,14 @@ use crate::{
const MIN_INITIAL_REPORT_SIZE: u64 = 100 * 1024 * 1024; const MIN_INITIAL_REPORT_SIZE: u64 = 100 * 1024 * 1024;
/// Bytes read from the file per iteration.
///
/// This buys read-latency hiding, and one read in flight is enough for that —
/// there is no throughput reason to keep tens of megabytes resident. It used to
/// be 64 MB, which cost that much in a permanently-live scratch buffer plus a
/// comparable amount in the accumulation buffer it fed.
const READ_CHUNK_SIZE: usize = 4 * 1024 * 1024;
trait TraceFormat { trait TraceFormat {
type Reused: Default; type Reused: Default;
/// Create the initial reused buffer. Override to pre-allocate capacity. /// Create the initial reused buffer. Override to pre-allocate capacity.
...@@ -203,10 +211,13 @@ impl TraceReader { ...@@ -203,10 +211,13 @@ impl TraceReader {
} }
}; };
let mut buffer = Vec::new(); // `buffer` accumulates bytes until they form whole rows; `index` is how
// far the format has consumed. Two chunks of headroom means the common
// case (a partial row left over) never reallocates.
let mut buffer: Vec<u8> = Vec::with_capacity(READ_CHUNK_SIZE * 2);
let mut index = 0; let mut index = 0;
let mut chunk = vec![0; 64 * 1024 * 1024]; let mut chunk = vec![0; READ_CHUNK_SIZE];
loop { loop {
match file.read(&mut chunk) { match file.read(&mut chunk) {
Ok(bytes_read) => { Ok(bytes_read) => {
...@@ -220,12 +231,6 @@ impl TraceReader { ...@@ -220,12 +231,6 @@ impl TraceReader {
return value; return value;
} }
} else { } else {
// If we have partially consumed some data, and we are at buffer capacity,
// remove the consumed data to make more space.
if index > 0 && buffer.len() + bytes_read > buffer.capacity() {
buffer.splice(..index, std::iter::empty());
index = 0;
}
buffer.extend_from_slice(&chunk[..bytes_read]); buffer.extend_from_slice(&chunk[..bytes_read]);
if format.is_none() && buffer.len() >= 8 { if format.is_none() && buffer.len() >= 8 {
let erased_format = if buffer.starts_with(b"TRACEv0") { let erased_format = if buffer.starts_with(b"TRACEv0") {
...@@ -259,6 +264,20 @@ impl TraceReader { ...@@ -259,6 +264,20 @@ impl TraceReader {
return true; return true;
} }
} }
// Drop the consumed prefix now, rather than letting
// `buffer` grow to capacity first and compacting
// then. What is left is normally one partial row, so
// the move is cheap, and it keeps the buffer at
// roughly one chunk instead of ratcheting upward.
//
// Safe to do here because the format's reused row
// vec borrows `buffer` only within a `read` call —
// it is cleared on entry and on drop, and the rows
// are drained before it returns.
if index > 0 {
buffer.drain(..index);
index = 0;
}
if self.store.want_to_read() { if self.store.want_to_read() {
thread::yield_now(); thread::yield_now();
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment