Unverified Commit e08d99ac authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Turbopack: omit Exit+Enter trace row pairs with (almost) no gap (#99780)

## Summary

When a span is exited and then entered again on the same thread almost
immediately, the writer drops both rows. Async spans do this on every
poll. It cuts the number of Enter/Exit rows almost in half.

A pair is only omitted when all of these hold:

- same span, same thread, and the `Exit` is still the last row in the
thread-local buffer (not sent to the writer thread yet)
- nothing was (de)allocated between the Exit and the Enter (otherwise
the Enter would need to report it for whatever ran in between)
- the gap is **≤ 1µs**, or **≤ 0.1%** of the time between the last
`Enter` row written on this thread (of any span; omitted `Enter` rows
don't count) and the Exit. For nested spans the last `Enter` can be a
nested span's, which only shrinks the allowance. Memory samples are
still taken when the `Enter` row is omitted.

Allocations reported with the dropped Exit aren't lost. The baseline is
rolled back, so the next row of the thread reports them, while the span
is on top of the stack again.

The viewer then shows the span as entered during the gap. Its self time
can be slightly too large, by at most 1µs or 0.1% per gap. This is
documented in `contributing/turbopack/tracing.md`.

Implementation:

- `TraceWriter`'s `WriteGuard` can mark exactly what a callback writes
(`mark(marker, |guard| ...)`) and remove that row again with
`remove_last_row(marker)`, as long as it still ends the thread local
buffer. Any other write, or sending the buffer, invalidates it. `Exit`
rows are marked with the span id.
- `RawTraceLayer` keeps per-thread state (last Enter timestamp, last
exit, allocation baseline). It is reset when the thread id changes. The
allowed gap is only computed in `on_enter`, after the cheaper checks
pass. Allocations made by this bookkeeping are excluded from the
counters.
- The clock and the allocation counters can be injected in tests.

Stack (on top of #99765):

1. #99777: `no-memory` flag
2. #99778: Merge allocation counters into Enter/Exit rows
3. #99779: Delta-encode allocation counters, skip unchanged ones
4. #99780: **This PR**: omit Exit+Enter pairs with (almost) no gap
5. #99781: Bump the trace header to `TRACEv1`

## Measurements

`next build` of `bench/heavy-npm-deps`, release native binding built
from source:

| | trace size | Enter rows |
| --- | --- | --- |
| PR 3 | 31.6 MiB | 1,138,850 |
| same-timestamp pairs only | 27.2 MiB | 875,258 |
| this PR (≤ 1µs or ≤ 0.1%) | 22.9 MiB | 626,956 |

A dev build of the binding has almost no same-µs pairs (it's slower), so
measure this with a release build.

## Verification

- Unit tests for each condition: same/different span, thread and
timestamp; allocations in between; an intervening row; a sent buffer;
thread slot reuse; the 1µs fast path; exactly at and just over the 0.1%
bound; chained omissions; allocation totals preserved
- `TraceWriter` tests for marking and removing rows
- Captured a trace with a locally built release binding and loaded it in
the trace server

<!-- NEXT_JS_LLM -->


<!-- fleet 4a52842b-f1a2-43cb-9779-3655caada6a0 -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 0e324e12
...@@ -79,5 +79,9 @@ cargo run --bin turbo-trace-size --release -- /path/to/your/trace-turbopack.bin ...@@ -79,5 +79,9 @@ cargo run --bin turbo-trace-size --release -- /path/to/your/trace-turbopack.bin
cargo run --bin turbo-trace-size --release -- /path/to/your/trace-turbopack.bin --top 100 cargo run --bin turbo-trace-size --release -- /path/to/your/trace-turbopack.bin --top 100
``` ```
### Omitted Exit and Enter rows
To keep trace files small, an `Exit` of a span that is followed by an `Enter` of the same span on the same thread shortly after is omitted together with that `Enter`. That's common for async spans, which are exited and entered again on every poll. This only happens when nothing was (de)allocated in between and the gap is at most 1µs or 0.1% of the time since the last `Enter` row written on that thread (omitted `Enter` rows don't count; for nested spans, it can be later than the span's own `Enter`, which only makes the allowed gap smaller). The viewer shows the span as entered during these gaps, so its self time can be slightly too large.
[turbo-trace-viewer]: https://turbo-trace-viewer.vercel.app/ [turbo-trace-viewer]: https://turbo-trace-viewer.vercel.app/
[youtube-tutorial]: https://www.youtube.com/watch?v=PGO2szAye7A [youtube-tutorial]: https://www.youtube.com/watch?v=PGO2szAye7A
...@@ -2,11 +2,11 @@ use postcard::ser_flavors::Flavor; ...@@ -2,11 +2,11 @@ use postcard::ser_flavors::Flavor;
use crate::trace_writer::WriteGuard; use crate::trace_writer::WriteGuard;
pub struct WriteGuardFlavor<'l> { pub struct WriteGuardFlavor<'a, 'l> {
pub guard: WriteGuard<'l>, pub guard: &'a mut WriteGuard<'l>,
} }
impl Flavor for WriteGuardFlavor<'_> { impl Flavor for WriteGuardFlavor<'_, '_> {
type Output = (); type Output = ();
fn try_push(&mut self, data: u8) -> postcard::Result<()> { fn try_push(&mut self, data: u8) -> postcard::Result<()> {
......
...@@ -15,12 +15,22 @@ const WRITE_BUFFER_SIZE: usize = 100 * 1024 * 1024; ...@@ -15,12 +15,22 @@ const WRITE_BUFFER_SIZE: usize = 100 * 1024 * 1024;
struct TraceInfoBuffer { struct TraceInfoBuffer {
buffer: Vec<u8>, buffer: Vec<u8>,
/// The marker of the last marked row in the buffer, see [`WriteGuard::mark`].
last_row_marker: Option<u64>,
/// The offset where the last marked row starts in `buffer`.
last_row_start: usize,
/// The offset where the last marked row ends in `buffer`. It's only still the last row in
/// the buffer when this is the length of the buffer.
last_row_end: usize,
} }
impl TraceInfoBuffer { impl TraceInfoBuffer {
fn new(capacity: usize) -> Self { fn new(capacity: usize) -> Self {
Self { Self {
buffer: Vec::with_capacity(capacity), buffer: Vec::with_capacity(capacity),
last_row_marker: None,
last_row_start: 0,
last_row_end: 0,
} }
} }
...@@ -34,6 +44,9 @@ impl TraceInfoBuffer { ...@@ -34,6 +44,9 @@ impl TraceInfoBuffer {
fn clear(&mut self) { fn clear(&mut self) {
self.buffer.clear(); self.buffer.clear();
self.last_row_marker = None;
self.last_row_start = 0;
self.last_row_end = 0;
} }
} }
...@@ -70,7 +83,13 @@ impl TraceWriter { ...@@ -70,7 +83,13 @@ impl TraceWriter {
) { ) {
for state in thread_locals.iter() { for state in thread_locals.iter() {
let mut buffer = state.lock(); let mut buffer = state.lock();
if let Some(buffer) = buffer.take() { // Empty buffers (e.g. after the only row was removed again) are left in place.
// An empty buffer would be interpreted as exit signal.
if buffer
.as_ref()
.is_some_and(|buffer| !buffer.buffer.is_empty())
&& let Some(buffer) = buffer.take()
{
stolen_buffers.push(buffer); stolen_buffers.push(buffer);
} }
} }
...@@ -228,9 +247,8 @@ impl<'l> WriteGuard<'l> { ...@@ -228,9 +247,8 @@ impl<'l> WriteGuard<'l> {
trace_writer: &'l TraceWriter, trace_writer: &'l TraceWriter,
) -> Self { ) -> Self {
// Safety: The buffer must not be None, so we initialize it here // Safety: The buffer must not be None, so we initialize it here
if buffer.is_none() { buffer
*buffer = Some(trace_writer.get_empty_buffer(THREAD_LOCAL_INITIAL_BUFFER_SIZE)); .get_or_insert_with(|| trace_writer.get_empty_buffer(THREAD_LOCAL_INITIAL_BUFFER_SIZE));
};
Self { Self {
buffer, buffer,
trace_writer, trace_writer,
...@@ -249,6 +267,31 @@ impl<'l> WriteGuard<'l> { ...@@ -249,6 +267,31 @@ impl<'l> WriteGuard<'l> {
pub fn extend(&mut self, data: &[u8]) { pub fn extend(&mut self, data: &[u8]) {
self.buffer().extend(data); self.buffer().extend(data);
} }
/// Marks exactly what `write` writes with this guard as a row with `marker`. As long as
/// nothing else is written on this thread and the row wasn't sent to the writer thread in
/// between, it can be removed again with [`WriteGuard::remove_last_row`].
pub fn mark(&mut self, marker: u64, write: impl FnOnce(&mut Self)) {
let start = self.buffer().buffer.len();
write(self);
let buffer = self.buffer();
buffer.last_row_marker = Some(marker);
buffer.last_row_start = start;
buffer.last_row_end = buffer.buffer.len();
}
/// Removes the last row written on this thread, if it was marked with `marker` by
/// [`WriteGuard::mark`] and is still at the end of the thread local buffer. Returns whether
/// it was removed.
pub fn remove_last_row(&mut self, marker: u64) -> bool {
let buffer = self.buffer();
if buffer.last_row_marker != Some(marker) || buffer.last_row_end != buffer.buffer.len() {
return false;
}
buffer.last_row_marker = None;
buffer.buffer.truncate(buffer.last_row_start);
true
}
} }
impl Drop for WriteGuard<'_> { impl Drop for WriteGuard<'_> {
...@@ -261,3 +304,119 @@ impl Drop for WriteGuard<'_> { ...@@ -261,3 +304,119 @@ impl Drop for WriteGuard<'_> {
} }
} }
} }
#[cfg(test)]
mod tests {
use std::{
io::{self, Write},
sync::{Arc, Mutex},
};
use crate::trace_writer::{THREAD_LOCAL_INITIAL_BUFFER_SIZE, TraceWriter};
#[derive(Clone, Default)]
struct SharedBuffer(Arc<Mutex<Vec<u8>>>);
impl Write for SharedBuffer {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.0.lock().unwrap().extend_from_slice(buf);
Ok(buf.len())
}
fn flush(&mut self) -> io::Result<()> {
Ok(())
}
}
fn with_writer(f: impl FnOnce(&TraceWriter)) -> Vec<u8> {
let buffer = SharedBuffer::default();
let (writer, guard) = TraceWriter::new(buffer.clone());
f(&writer);
drop(writer);
drop(guard);
let data = Arc::try_unwrap(buffer.0).unwrap().into_inner().unwrap();
data.strip_prefix(b"TRACEv0").unwrap().to_vec()
}
#[test]
fn removes_marked_last_row() {
let data = with_writer(|writer| {
writer.start_write().extend(b"aa");
writer.start_write().mark(1, |guard| guard.extend(b"bbb"));
let mut guard = writer.start_write();
assert!(guard.remove_last_row(1));
guard.extend(b"c");
});
assert_eq!(data, b"aac");
}
/// The mark covers exactly what is written in the callback, not what this guard wrote
/// before or after it.
#[test]
fn marks_only_the_callback() {
let data = with_writer(|writer| {
let mut guard = writer.start_write();
guard.extend(b"aa");
guard.mark(1, |guard| guard.extend(b"bbb"));
drop(guard);
let mut guard = writer.start_write();
assert!(guard.remove_last_row(1));
guard.extend(b"c");
});
assert_eq!(data, b"aac");
let data = with_writer(|writer| {
let mut guard = writer.start_write();
guard.mark(1, |guard| guard.extend(b"a"));
guard.extend(b"b");
drop(guard);
assert!(!writer.start_write().remove_last_row(1));
});
assert_eq!(data, b"ab");
}
#[test]
fn requires_a_matching_marker() {
let data = with_writer(|writer| {
writer.start_write().mark(1, |guard| guard.extend(b"a"));
assert!(!writer.start_write().remove_last_row(2));
// The row is still marked
assert!(writer.start_write().remove_last_row(1));
// Already removed
assert!(!writer.start_write().remove_last_row(1));
});
assert_eq!(data, b"");
}
#[test]
fn marker_is_invalidated_by_the_next_write() {
let data = with_writer(|writer| {
writer.start_write().mark(1, |guard| guard.extend(b"a"));
writer.start_write().extend(b"b");
assert!(!writer.start_write().remove_last_row(1));
});
assert_eq!(data, b"ab");
}
#[test]
fn marker_is_dropped_when_buffer_is_sent() {
// Just below the threshold where the buffer is sent to the writer thread
let fill = THREAD_LOCAL_INITIAL_BUFFER_SIZE * 2 / 3 - 10;
let data = with_writer(|writer| {
writer.start_write().extend(&vec![0; fill]);
// This exceeds the threshold, so the buffer is sent when the guard is dropped
writer.start_write().mark(1, |guard| guard.extend(&[1; 20]));
assert!(!writer.start_write().remove_last_row(1));
});
assert_eq!(data.len(), fill + 20);
}
#[test]
fn removing_the_only_row_leaves_an_empty_buffer() {
let data = with_writer(|writer| {
writer.start_write().mark(1, |guard| guard.extend(b"a"));
assert!(writer.start_write().remove_last_row(1));
});
assert_eq!(data, b"");
}
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment