Unverified Commit 0e324e12 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Turbopack: delta-encode trace allocation counters and skip unchanged ones (#99779)

## Summary

The allocation counters on `Enter`/`Exit` rows are now the **delta**
since the previous report of the same thread, not cumulative per-thread
totals. When nothing was (de)allocated, the delta is omitted (`None`).

Cumulative counters are large numbers, so their varints take many bytes.
Deltas are mostly small or zero: in a typical trace, half of all reports
had an unchanged counter.

- The writer keeps the last reported counters per thread. The first
report of a thread only sets the baseline (`None`). That state also
stores the thread id, because `thread_local` slots are reused after a
thread exits, so a new thread never diffs against an old thread's
counters.
- The trace server attributes the delta directly, with no per-thread
cumulative state.

Stack (on top of #99765):

1. #99777: `no-memory` flag
2. #99778: Merge allocation counters into Enter/Exit rows
3. #99779: **This PR**: delta-encode allocation counters, skip unchanged
ones
4. #99780: Omit Exit+Enter pairs with (almost) no gap
5. #99781: Bump the trace header to `TRACEv1`

## Measurements

`next build` of `bench/heavy-npm-deps`, release native binding built
from source:

| | trace size |
| --- | --- |
| original | 68.2 MiB |
| PR 2 | 58.8 MiB |
| this PR | 31.6 MiB |

## Verification

- Unit tests in `raw_trace.rs` for deltas, omitted zero deltas, the
per-thread baseline and a reused thread slot
- Captured a trace with a locally built binding and loaded it in the
trace server

<!-- NEXT_JS_LLM -->


<!-- fleet 4a52842b-f1a2-43cb-9779-3655caada6a0 -->
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 1012b220
use std::{ use std::{
borrow::Cow, borrow::Cow,
collections::hash_map::Entry,
mem::transmute, mem::transmute,
ops::{Deref, DerefMut}, ops::{Deref, DerefMut},
sync::Arc, sync::Arc,
...@@ -67,7 +66,6 @@ pub struct TurbopackFormat { ...@@ -67,7 +66,6 @@ pub struct TurbopackFormat {
queued_rows: FxHashMap<u64, Vec<InternalRow>>, queued_rows: FxHashMap<u64, Vec<InternalRow>>,
outdated_spans: FxHashSet<SpanIndex>, outdated_spans: FxHashSet<SpanIndex>,
thread_stacks: FxHashMap<u64, Vec<u64>>, thread_stacks: FxHashMap<u64, Vec<u64>>,
thread_allocation_counters: FxHashMap<u64, Allocations>,
self_time_started: FxHashMap<(u64, u64), Timestamp>, self_time_started: FxHashMap<(u64, u64), Timestamp>,
interner: RcStrInterning, interner: RcStrInterning,
} }
...@@ -86,7 +84,6 @@ impl TurbopackFormat { ...@@ -86,7 +84,6 @@ impl TurbopackFormat {
queued_rows: FxHashMap::with_capacity_and_hasher(1_024, Default::default()), queued_rows: FxHashMap::with_capacity_and_hasher(1_024, Default::default()),
outdated_spans: FxHashSet::with_capacity_and_hasher(8_192, Default::default()), outdated_spans: FxHashSet::with_capacity_and_hasher(8_192, Default::default()),
thread_stacks: FxHashMap::with_capacity_and_hasher(64, Default::default()), thread_stacks: FxHashMap::with_capacity_and_hasher(64, Default::default()),
thread_allocation_counters: FxHashMap::with_capacity_and_hasher(64, Default::default()),
self_time_started: FxHashMap::with_capacity_and_hasher(256, Default::default()), self_time_started: FxHashMap::with_capacity_and_hasher(256, Default::default()),
interner: RcStrInterning::new(), interner: RcStrInterning::new(),
} }
...@@ -161,7 +158,7 @@ impl TurbopackFormat { ...@@ -161,7 +158,7 @@ impl TurbopackFormat {
} => { } => {
// Allocations up to this point belong to the span that was running before. // Allocations up to this point belong to the span that was running before.
if let Some(allocations) = allocations { if let Some(allocations) = allocations {
self.process_allocation_counters(store, thread_id, allocations); self.add_allocations(store, thread_id, allocations);
} }
let ts = Timestamp::from_micros(ts); let ts = Timestamp::from_micros(ts);
let stack = self.thread_stacks.entry(thread_id).or_default(); let stack = self.thread_stacks.entry(thread_id).or_default();
...@@ -195,7 +192,7 @@ impl TurbopackFormat { ...@@ -195,7 +192,7 @@ impl TurbopackFormat {
} => { } => {
// Allocations up to this point belong to the span that is exited. // Allocations up to this point belong to the span that is exited.
if let Some(allocations) = allocations { if let Some(allocations) = allocations {
self.process_allocation_counters(store, thread_id, allocations); self.add_allocations(store, thread_id, allocations);
} }
let ts = Timestamp::from_micros(ts); let ts = Timestamp::from_micros(ts);
let stack = self.thread_stacks.entry(thread_id).or_default(); let stack = self.thread_stacks.entry(thread_id).or_default();
...@@ -269,33 +266,6 @@ impl TurbopackFormat { ...@@ -269,33 +266,6 @@ impl TurbopackFormat {
} }
} }
/// Processes the cumulative allocation counters of a thread. The difference to the previous
/// counters of the thread is attributed to the span on top of the thread's stack.
fn process_allocation_counters(
&mut self,
store: &mut StoreWriteGuard,
thread_id: u64,
counters: Allocations,
) {
let diff = match self.thread_allocation_counters.entry(thread_id) {
Entry::Occupied(mut entry) => {
let previous = std::mem::replace(entry.get_mut(), counters);
Allocations {
allocations: counters.allocations - previous.allocations,
allocation_count: counters.allocation_count - previous.allocation_count,
deallocations: counters.deallocations - previous.deallocations,
deallocation_count: counters.deallocation_count - previous.deallocation_count,
}
}
Entry::Vacant(entry) => {
// The first counters of a thread are only the baseline.
entry.insert(counters);
return;
}
};
self.add_allocations(store, thread_id, diff);
}
/// Attributes (de)allocations of a thread to the span on top of the thread's stack. /// Attributes (de)allocations of a thread to the span on top of the thread's stack.
fn add_allocations( fn add_allocations(
&mut self, &mut self,
...@@ -605,15 +575,15 @@ mod tests { ...@@ -605,15 +575,15 @@ mod tests {
}; };
let rows = [ let rows = [
start(1, None, "outer"), start(1, None, "outer"),
// Baseline counters // Nothing allocated before
enter(1, 1, allocations(1000, 1000)), enter(1, 1, None),
start(2, Some(1), "inner"), start(2, Some(1), "inner"),
// 100 bytes allocated in "outer" before "inner" is entered // 100 bytes allocated in "outer" before "inner" is entered
enter(2, 2, allocations(1100, 1000)), enter(2, 2, allocations(100, 0)),
// 50 bytes allocated and 20 bytes deallocated in "inner" // 50 bytes allocated and 20 bytes deallocated in "inner"
exit(3, 2, allocations(1150, 1020)), exit(3, 2, allocations(50, 20)),
// 7 bytes allocated in "outer" after "inner" is exited // 7 bytes allocated in "outer" after "inner" is exited
exit(4, 1, allocations(1157, 1020)), exit(4, 1, allocations(7, 0)),
TraceRow::End { ts: 5, id: 2 }, TraceRow::End { ts: 5, id: 2 },
TraceRow::End { ts: 5, id: 1 }, TraceRow::End { ts: 5, id: 1 },
]; ];
......
...@@ -8,6 +8,7 @@ use std::{ ...@@ -8,6 +8,7 @@ use std::{
time::Instant, time::Instant,
}; };
use thread_local::ThreadLocal;
use tracing::{ use tracing::{
Subscriber, Subscriber,
field::{Visit, display}, field::{Visit, display},
...@@ -73,6 +74,10 @@ pub struct RawTraceLayer<S: Subscriber + for<'a> LookupSpan<'a>> { ...@@ -73,6 +74,10 @@ pub struct RawTraceLayer<S: Subscriber + for<'a> LookupSpan<'a>> {
memory: bool, memory: bool,
/// Reads the allocation counters of the current thread. Only replaced in tests. /// Reads the allocation counters of the current thread. Only replaced in tests.
allocation_counters: fn() -> Allocations, allocation_counters: fn() -> Allocations,
/// The allocation counters of each thread at the time they were last reported, together
/// with the id of the thread. Slots of [`ThreadLocal`] are reused by new threads after a
/// thread exited, so the counters are only used when the thread id matches.
reported_allocations: ThreadLocal<Cell<(u64, Option<Allocations>)>>,
_phantom: PhantomData<fn(S)>, _phantom: PhantomData<fn(S)>,
} }
...@@ -100,6 +105,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> { ...@@ -100,6 +105,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
next_id: AtomicU64::new(1), next_id: AtomicU64::new(1),
memory, memory,
allocation_counters: thread_allocation_counters, allocation_counters: thread_allocation_counters,
reported_allocations: ThreadLocal::new(),
_phantom: PhantomData, _phantom: PhantomData,
} }
} }
...@@ -153,9 +159,29 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> { ...@@ -153,9 +159,29 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
} }
} }
/// Returns the allocation counters to attach to an Enter/Exit row, if memory is tracked. /// Returns the allocations to attach to an Enter/Exit row: what the current thread
fn allocations(&self) -> Option<Allocations> { /// (de)allocated since its previous report. Returns `None` when memory isn't tracked, on
self.memory.then(self.allocation_counters) /// the first report of a thread (which only establishes the baseline), and when nothing
/// was (de)allocated since the previous report.
fn allocations(&self, thread_id: u64) -> Option<Allocations> {
if !self.memory {
return None;
}
let current = (self.allocation_counters)();
let reported = self.reported_allocations.get_or_default();
let (reported_thread_id, previous) = reported.replace((thread_id, Some(current)));
let previous = previous.filter(|_| reported_thread_id == thread_id)?;
let delta = Allocations {
allocations: current.allocations.saturating_sub(previous.allocations),
allocation_count: current
.allocation_count
.saturating_sub(previous.allocation_count),
deallocations: current.deallocations.saturating_sub(previous.deallocations),
deallocation_count: current
.deallocation_count
.saturating_sub(previous.deallocation_count),
};
(delta != Allocations::default()).then_some(delta)
} }
} }
...@@ -204,7 +230,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> { ...@@ -204,7 +230,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> {
if self.memory { if self.memory {
self.maybe_report_memory_sample(ts); self.maybe_report_memory_sample(ts);
} }
let allocations = self.allocations(); let allocations = self.allocations(thread_id);
self.write(TraceRow::Enter { self.write(TraceRow::Enter {
ts, ts,
id: get_id(ctx, id), id: get_id(ctx, id),
...@@ -216,7 +242,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> { ...@@ -216,7 +242,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> {
fn on_exit(&self, id: &span::Id, ctx: tracing_subscriber::layer::Context<'_, S>) { fn on_exit(&self, id: &span::Id, ctx: tracing_subscriber::layer::Context<'_, S>) {
let ts = self.start.elapsed().as_micros() as u64; let ts = self.start.elapsed().as_micros() as u64;
let thread_id = thread::current().id().as_u64().into(); let thread_id = thread::current().id().as_u64().into();
let allocations = self.allocations(); let allocations = self.allocations(thread_id);
self.write(TraceRow::Exit { self.write(TraceRow::Exit {
ts, ts,
id: get_id(ctx, id), id: get_id(ctx, id),
...@@ -435,26 +461,60 @@ pub(crate) mod tests { ...@@ -435,26 +461,60 @@ pub(crate) mod tests {
} }
#[test] #[test]
fn attaches_allocation_counters_to_enter_and_exit() { fn attaches_allocation_deltas_to_enter_and_exit() {
let mut at_enter = Allocations::default();
let mut at_exit = Allocations::default();
let data = capture(RawTraceLayerOptions::default(), || { let data = capture(RawTraceLayerOptions::default(), || {
fake_allocate(10, 1); fake_allocate(10, 1);
let span = tracing::info_span!("test span"); let span = tracing::info_span!("first");
at_enter = fake_allocation_counters();
let guard = span.enter(); let guard = span.enter();
fake_allocate(100, 2); fake_allocate(100, 2);
fake_deallocate(5, 1); fake_deallocate(5, 1);
at_exit = fake_allocation_counters(); drop(guard);
fake_deallocate(1, 1);
let span = tracing::info_span!("second");
let guard = span.enter();
drop(guard);
fake_allocate(3, 1);
let guard = span.enter();
drop(guard); drop(guard);
}); });
let rows = decode(&data); let rows = decode(&data);
assert_eq!( assert_eq!(
enter_exit_allocations(&rows), enter_exit_allocations(&rows),
vec![("enter", Some(at_enter)), ("exit", Some(at_exit))] vec![
// The first report of the thread is the baseline
("enter", None),
(
"exit",
Some(Allocations {
allocations: 200,
allocation_count: 2,
deallocations: 5,
deallocation_count: 1,
})
),
(
"enter",
Some(Allocations {
allocations: 0,
allocation_count: 0,
deallocations: 1,
deallocation_count: 1,
})
),
// Nothing (de)allocated in between
("exit", None),
(
"enter",
Some(Allocations {
allocations: 3,
allocation_count: 1,
deallocations: 0,
deallocation_count: 0,
})
),
("exit", None),
]
); );
assert_eq!(at_exit.allocations - at_enter.allocations, 200);
assert_eq!(at_exit.deallocation_count - at_enter.deallocation_count, 1);
} }
#[test] #[test]
...@@ -473,4 +533,32 @@ pub(crate) mod tests { ...@@ -473,4 +533,32 @@ pub(crate) mod tests {
); );
assert_eq!(rows.len(), 4); assert_eq!(rows.len(), 4);
} }
#[test]
fn resets_allocation_baseline_when_a_thread_slot_is_reused() {
let data = capture(RawTraceLayerOptions::default(), || {
let span = tracing::info_span!("span");
let dispatch = tracing::dispatcher::get_default(|d| d.clone());
// Sequential threads reuse the same thread local slot
for i in 1..=2 {
std::thread::scope(|scope| {
scope.spawn(|| {
fake_allocate(1000 * i, 1);
tracing::dispatcher::with_default(&dispatch, || drop(span.enter()));
});
});
}
});
// Each thread reports its own baseline (`None`) first instead of the difference to the
// counters of the other thread.
assert_eq!(
enter_exit_allocations(&decode(&data)),
vec![
("enter", None),
("exit", None),
("enter", None),
("exit", None)
]
);
}
} }
...@@ -41,8 +41,8 @@ pub enum TraceRow<'a> { ...@@ -41,8 +41,8 @@ pub enum TraceRow<'a> {
id: u64, id: u64,
/// The thread id of the thread that entered the span. /// The thread id of the thread that entered the span.
thread_id: u64, thread_id: u64,
/// The allocation counters of the thread right before the span was entered, if memory /// What the thread (de)allocated since its previous Enter/Exit row, if anything and if
/// is tracked. See [`Allocations`]. /// memory is tracked. See [`Allocations`].
allocations: Option<Allocations>, allocations: Option<Allocations>,
}, },
/// A span has been exited. This means it is not spending CPU time anymore. /// A span has been exited. This means it is not spending CPU time anymore.
...@@ -53,8 +53,8 @@ pub enum TraceRow<'a> { ...@@ -53,8 +53,8 @@ pub enum TraceRow<'a> {
id: u64, id: u64,
/// The thread id of the thread that exits the span. /// The thread id of the thread that exits the span.
thread_id: u64, thread_id: u64,
/// The allocation counters of the thread right before the span was exited, if memory /// What the thread (de)allocated since its previous Enter/Exit row, if anything and if
/// is tracked. See [`Allocations`]. /// memory is tracked. See [`Allocations`].
allocations: Option<Allocations>, allocations: Option<Allocations>,
}, },
/// A event has happened for some span. /// A event has happened for some span.
...@@ -90,14 +90,15 @@ pub enum TraceRow<'a> { ...@@ -90,14 +90,15 @@ pub enum TraceRow<'a> {
}, },
} }
/// The per-thread allocation counters, as attached to [`TraceRow::Enter`] and /// The (de)allocations of a thread, as attached to [`TraceRow::Enter`] and
/// [`TraceRow::Exit`]. /// [`TraceRow::Exit`].
/// ///
/// The counters are cumulative for the thread (allocations made by the tracing /// These are the (de)allocations of the thread since its previous `Enter` or
/// itself are excluded). The difference to the previous counters of the same /// `Exit` row (allocations made by the tracing itself are excluded). A row
/// thread is what was (de)allocated in between. Readers attribute it to the span /// without allocations means that nothing was (de)allocated, or that memory
/// on top of the thread's span stack before the row is applied: for `Enter`, the /// isn't tracked. Readers attribute them to the span on top of the thread's
/// span that was running before (if any), for `Exit`, the span that is exited. /// span stack before the row is applied: for `Enter`, the span that was running
/// before (if any), for `Exit`, the span that is exited.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] #[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct Allocations { pub struct Allocations {
/// Allocated bytes /// Allocated bytes
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment