Unverified Commit 0e324e12 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Turbopack: delta-encode trace allocation counters and skip unchanged ones (#99779)

## Summary

The allocation counters on `Enter`/`Exit` rows are now the **delta**
since the previous report of the same thread, not cumulative per-thread
totals. When nothing was (de)allocated, the delta is omitted (`None`).

Cumulative counters are large numbers, so their varints take many bytes.
Deltas are mostly small or zero: in a typical trace, half of all reports
had an unchanged counter.

- The writer keeps the last reported counters per thread. The first
report of a thread only sets the baseline (`None`). That state also
stores the thread id, because `thread_local` slots are reused after a
thread exits, so a new thread never diffs against an old thread's
counters.
- The trace server attributes the delta directly, with no per-thread
cumulative state.

Stack (on top of #99765):

1. #99777: `no-memory` flag
2. #99778: Merge allocation counters into Enter/Exit rows
3. #99779: **This PR**: delta-encode allocation counters, skip unchanged
ones
4. #99780: Omit Exit+Enter pairs with (almost) no gap
5. #99781: Bump the trace header to `TRACEv1`

## Measurements

`next build` of `bench/heavy-npm-deps`, release native binding built
from source:

| | trace size |
| --- | --- |
| original | 68.2 MiB |
| PR 2 | 58.8 MiB |
| this PR | 31.6 MiB |

## Verification

- Unit tests in `raw_trace.rs` for deltas, omitted zero deltas, the
per-thread baseline and a reused thread slot
- Captured a trace with a locally built binding and loaded it in the
trace server

<!-- NEXT_JS_LLM -->


<!-- fleet 4a52842b-f1a2-43cb-9779-3655caada6a0 -->
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 1012b220
use std::{
borrow::Cow,
collections::hash_map::Entry,
mem::transmute,
ops::{Deref, DerefMut},
sync::Arc,
......@@ -67,7 +66,6 @@ pub struct TurbopackFormat {
queued_rows: FxHashMap<u64, Vec<InternalRow>>,
outdated_spans: FxHashSet<SpanIndex>,
thread_stacks: FxHashMap<u64, Vec<u64>>,
thread_allocation_counters: FxHashMap<u64, Allocations>,
self_time_started: FxHashMap<(u64, u64), Timestamp>,
interner: RcStrInterning,
}
......@@ -86,7 +84,6 @@ impl TurbopackFormat {
queued_rows: FxHashMap::with_capacity_and_hasher(1_024, Default::default()),
outdated_spans: FxHashSet::with_capacity_and_hasher(8_192, Default::default()),
thread_stacks: FxHashMap::with_capacity_and_hasher(64, Default::default()),
thread_allocation_counters: FxHashMap::with_capacity_and_hasher(64, Default::default()),
self_time_started: FxHashMap::with_capacity_and_hasher(256, Default::default()),
interner: RcStrInterning::new(),
}
......@@ -161,7 +158,7 @@ impl TurbopackFormat {
} => {
// Allocations up to this point belong to the span that was running before.
if let Some(allocations) = allocations {
self.process_allocation_counters(store, thread_id, allocations);
self.add_allocations(store, thread_id, allocations);
}
let ts = Timestamp::from_micros(ts);
let stack = self.thread_stacks.entry(thread_id).or_default();
......@@ -195,7 +192,7 @@ impl TurbopackFormat {
} => {
// Allocations up to this point belong to the span that is exited.
if let Some(allocations) = allocations {
self.process_allocation_counters(store, thread_id, allocations);
self.add_allocations(store, thread_id, allocations);
}
let ts = Timestamp::from_micros(ts);
let stack = self.thread_stacks.entry(thread_id).or_default();
......@@ -269,33 +266,6 @@ impl TurbopackFormat {
}
}
/// Processes the cumulative allocation counters of a thread. The difference to the previous
/// counters of the thread is attributed to the span on top of the thread's stack.
fn process_allocation_counters(
&mut self,
store: &mut StoreWriteGuard,
thread_id: u64,
counters: Allocations,
) {
let diff = match self.thread_allocation_counters.entry(thread_id) {
Entry::Occupied(mut entry) => {
let previous = std::mem::replace(entry.get_mut(), counters);
Allocations {
allocations: counters.allocations - previous.allocations,
allocation_count: counters.allocation_count - previous.allocation_count,
deallocations: counters.deallocations - previous.deallocations,
deallocation_count: counters.deallocation_count - previous.deallocation_count,
}
}
Entry::Vacant(entry) => {
// The first counters of a thread are only the baseline.
entry.insert(counters);
return;
}
};
self.add_allocations(store, thread_id, diff);
}
/// Attributes (de)allocations of a thread to the span on top of the thread's stack.
fn add_allocations(
&mut self,
......@@ -605,15 +575,15 @@ mod tests {
};
let rows = [
start(1, None, "outer"),
// Baseline counters
enter(1, 1, allocations(1000, 1000)),
// Nothing allocated before
enter(1, 1, None),
start(2, Some(1), "inner"),
// 100 bytes allocated in "outer" before "inner" is entered
enter(2, 2, allocations(1100, 1000)),
enter(2, 2, allocations(100, 0)),
// 50 bytes allocated and 20 bytes deallocated in "inner"
exit(3, 2, allocations(1150, 1020)),
exit(3, 2, allocations(50, 20)),
// 7 bytes allocated in "outer" after "inner" is exited
exit(4, 1, allocations(1157, 1020)),
exit(4, 1, allocations(7, 0)),
TraceRow::End { ts: 5, id: 2 },
TraceRow::End { ts: 5, id: 1 },
];
......
......@@ -8,6 +8,7 @@ use std::{
time::Instant,
};
use thread_local::ThreadLocal;
use tracing::{
Subscriber,
field::{Visit, display},
......@@ -73,6 +74,10 @@ pub struct RawTraceLayer<S: Subscriber + for<'a> LookupSpan<'a>> {
memory: bool,
/// Reads the allocation counters of the current thread. Only replaced in tests.
allocation_counters: fn() -> Allocations,
/// The allocation counters of each thread at the time they were last reported, together
/// with the id of the thread. Slots of [`ThreadLocal`] are reused by new threads after a
/// thread exited, so the counters are only used when the thread id matches.
reported_allocations: ThreadLocal<Cell<(u64, Option<Allocations>)>>,
_phantom: PhantomData<fn(S)>,
}
......@@ -100,6 +105,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
next_id: AtomicU64::new(1),
memory,
allocation_counters: thread_allocation_counters,
reported_allocations: ThreadLocal::new(),
_phantom: PhantomData,
}
}
......@@ -153,9 +159,29 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
}
}
/// Returns the allocation counters to attach to an Enter/Exit row, if memory is tracked.
fn allocations(&self) -> Option<Allocations> {
self.memory.then(self.allocation_counters)
/// Returns the allocations to attach to an Enter/Exit row: what the current thread
/// (de)allocated since its previous report. Returns `None` when memory isn't tracked, on
/// the first report of a thread (which only establishes the baseline), and when nothing
/// was (de)allocated since the previous report.
fn allocations(&self, thread_id: u64) -> Option<Allocations> {
if !self.memory {
return None;
}
let current = (self.allocation_counters)();
let reported = self.reported_allocations.get_or_default();
let (reported_thread_id, previous) = reported.replace((thread_id, Some(current)));
let previous = previous.filter(|_| reported_thread_id == thread_id)?;
let delta = Allocations {
allocations: current.allocations.saturating_sub(previous.allocations),
allocation_count: current
.allocation_count
.saturating_sub(previous.allocation_count),
deallocations: current.deallocations.saturating_sub(previous.deallocations),
deallocation_count: current
.deallocation_count
.saturating_sub(previous.deallocation_count),
};
(delta != Allocations::default()).then_some(delta)
}
}
......@@ -204,7 +230,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> {
if self.memory {
self.maybe_report_memory_sample(ts);
}
let allocations = self.allocations();
let allocations = self.allocations(thread_id);
self.write(TraceRow::Enter {
ts,
id: get_id(ctx, id),
......@@ -216,7 +242,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> {
fn on_exit(&self, id: &span::Id, ctx: tracing_subscriber::layer::Context<'_, S>) {
let ts = self.start.elapsed().as_micros() as u64;
let thread_id = thread::current().id().as_u64().into();
let allocations = self.allocations();
let allocations = self.allocations(thread_id);
self.write(TraceRow::Exit {
ts,
id: get_id(ctx, id),
......@@ -435,26 +461,60 @@ pub(crate) mod tests {
}
#[test]
fn attaches_allocation_counters_to_enter_and_exit() {
let mut at_enter = Allocations::default();
let mut at_exit = Allocations::default();
fn attaches_allocation_deltas_to_enter_and_exit() {
let data = capture(RawTraceLayerOptions::default(), || {
fake_allocate(10, 1);
let span = tracing::info_span!("test span");
at_enter = fake_allocation_counters();
let span = tracing::info_span!("first");
let guard = span.enter();
fake_allocate(100, 2);
fake_deallocate(5, 1);
at_exit = fake_allocation_counters();
drop(guard);
fake_deallocate(1, 1);
let span = tracing::info_span!("second");
let guard = span.enter();
drop(guard);
fake_allocate(3, 1);
let guard = span.enter();
drop(guard);
});
let rows = decode(&data);
assert_eq!(
enter_exit_allocations(&rows),
vec![("enter", Some(at_enter)), ("exit", Some(at_exit))]
vec![
// The first report of the thread is the baseline
("enter", None),
(
"exit",
Some(Allocations {
allocations: 200,
allocation_count: 2,
deallocations: 5,
deallocation_count: 1,
})
),
(
"enter",
Some(Allocations {
allocations: 0,
allocation_count: 0,
deallocations: 1,
deallocation_count: 1,
})
),
// Nothing (de)allocated in between
("exit", None),
(
"enter",
Some(Allocations {
allocations: 3,
allocation_count: 1,
deallocations: 0,
deallocation_count: 0,
})
),
("exit", None),
]
);
assert_eq!(at_exit.allocations - at_enter.allocations, 200);
assert_eq!(at_exit.deallocation_count - at_enter.deallocation_count, 1);
}
#[test]
......@@ -473,4 +533,32 @@ pub(crate) mod tests {
);
assert_eq!(rows.len(), 4);
}
#[test]
fn resets_allocation_baseline_when_a_thread_slot_is_reused() {
let data = capture(RawTraceLayerOptions::default(), || {
let span = tracing::info_span!("span");
let dispatch = tracing::dispatcher::get_default(|d| d.clone());
// Sequential threads reuse the same thread local slot
for i in 1..=2 {
std::thread::scope(|scope| {
scope.spawn(|| {
fake_allocate(1000 * i, 1);
tracing::dispatcher::with_default(&dispatch, || drop(span.enter()));
});
});
}
});
// Each thread reports its own baseline (`None`) first instead of the difference to the
// counters of the other thread.
assert_eq!(
enter_exit_allocations(&decode(&data)),
vec![
("enter", None),
("exit", None),
("enter", None),
("exit", None)
]
);
}
}
......@@ -41,8 +41,8 @@ pub enum TraceRow<'a> {
id: u64,
/// The thread id of the thread that entered the span.
thread_id: u64,
/// The allocation counters of the thread right before the span was entered, if memory
/// is tracked. See [`Allocations`].
/// What the thread (de)allocated since its previous Enter/Exit row, if anything and if
/// memory is tracked. See [`Allocations`].
allocations: Option<Allocations>,
},
/// A span has been exited. This means it is not spending CPU time anymore.
......@@ -53,8 +53,8 @@ pub enum TraceRow<'a> {
id: u64,
/// The thread id of the thread that exits the span.
thread_id: u64,
/// The allocation counters of the thread right before the span was exited, if memory
/// is tracked. See [`Allocations`].
/// What the thread (de)allocated since its previous Enter/Exit row, if anything and if
/// memory is tracked. See [`Allocations`].
allocations: Option<Allocations>,
},
/// A event has happened for some span.
......@@ -90,14 +90,15 @@ pub enum TraceRow<'a> {
},
}
/// The per-thread allocation counters, as attached to [`TraceRow::Enter`] and
/// The (de)allocations of a thread, as attached to [`TraceRow::Enter`] and
/// [`TraceRow::Exit`].
///
/// The counters are cumulative for the thread (allocations made by the tracing
/// itself are excluded). The difference to the previous counters of the same
/// thread is what was (de)allocated in between. Readers attribute it to the span
/// on top of the thread's span stack before the row is applied: for `Enter`, the
/// span that was running before (if any), for `Exit`, the span that is exited.
/// These are the (de)allocations of the thread since its previous `Enter` or
/// `Exit` row (allocations made by the tracing itself are excluded). A row
/// without allocations means that nothing was (de)allocated, or that memory
/// isn't tracked. Readers attribute them to the span on top of the thread's
/// span stack before the row is applied: for `Enter`, the span that was running
/// before (if any), for `Exit`, the span that is exited.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct Allocations {
/// Allocated bytes
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment