Unverified Commit 1012b220 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Turbopack: merge allocation counters into trace Enter/Exit rows (#99778)

## Summary

Removes the separate `AllocationCounters` (and the unused `Allocation`)
trace rows. Their data is attached to the `Enter`/`Exit` rows as
`allocations: Option<Allocations>` instead.

Before, the writer emitted one `AllocationCounters` row before every
Enter and Exit. Each carried its own tag, timestamp and thread id. Those
rows were **66%** of a typical trace. Since the counters are always
paired with an Enter/Exit, the duplicated tag, timestamp and thread id
can go.

- `TraceRow::Enter`/`Exit` get an `allocations` field (`None` when
memory isn't tracked). The new `Allocations` struct is shared by writer
and readers.
- Trace server: the counters on Enter/Exit go through the same
diff-and-attribute logic as before. The span on top of the thread's
stack before the row is applied gets the allocations: for Enter, the
previously running span; for Exit, the exited span.
- `turbo-trace-size`: the `AllocationCounters`/`Allocation` row kinds
are removed. Enter/Exit now have an `allocations` component.

This is an incompatible format change. The `TRACEv1` header bump comes
last in the stack (PR 5), and older traces are then rejected with an
error. The stack is meant to land together.

Stack (on top of #99765):

1. #99777: `no-memory` flag
2. #99778: **This PR**: merge allocation counters into Enter/Exit rows
3. #99779: Delta-encode allocation counters, skip unchanged ones
4. #99780: Omit Exit+Enter pairs with (almost) no gap
5. #99781: Bump the trace header to `TRACEv1`

## Measurements

`next build` of `bench/heavy-npm-deps`, release native binding built
from source:

| | trace size | rows |
| --- | --- | --- |
| before (PR 1) | 68.2 MiB | 4.65 M |
| this PR | 58.8 MiB | 2.37 M |

## Verification

- New trace server test checks allocation attribution for Enter/Exit
rows through `TurbopackFormat`
- Captured a trace with a locally built binding and loaded it in the
trace server

<!-- NEXT_JS_LLM -->


<!-- fleet 4a52842b-f1a2-43cb-9779-3655caada6a0 -->
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 4bee1b4c
...@@ -70,7 +70,7 @@ And there are different value modes: ...@@ -70,7 +70,7 @@ And there are different value modes:
## Trace file size ## Trace file size
To see what takes up the space in a trace file, use `turbo-trace-size`. It breaks the file down by row type (`Start`, `Enter`, `Exit`, `AllocationCounters`, ...), by span name, by attribute key and shows how many bytes are spent on repeated strings. Raw, gzip and zstd compressed files are supported. All sizes refer to the decompressed trace stream. To see what takes up the space in a trace file, use `turbo-trace-size`. It breaks the file down by row type (`Start`, `Enter`, `Exit`, `Record`, ...), by span name, by attribute key and shows how many bytes are spent on repeated strings. Raw, gzip and zstd compressed files are supported. All sizes refer to the decompressed trace stream.
```sh ```sh
cargo run --bin turbo-trace-size --release -- /path/to/your/trace-turbopack.bin cargo run --bin turbo-trace-size --release -- /path/to/your/trace-turbopack.bin
......
...@@ -26,6 +26,7 @@ fn reads_active_workers_and_exposes_them_in_span_queries() { ...@@ -26,6 +26,7 @@ fn reads_active_workers_and_exposes_them_in_span_queries() {
ts: 10, ts: 10,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::MemorySample { TraceRow::MemorySample {
ts: 20, ts: 20,
...@@ -37,6 +38,7 @@ fn reads_active_workers_and_exposes_them_in_span_queries() { ...@@ -37,6 +38,7 @@ fn reads_active_workers_and_exposes_them_in_span_queries() {
ts: 30, ts: 30,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 30, id: 1 }, TraceRow::End { ts: 30, id: 1 },
]; ];
...@@ -141,11 +143,13 @@ fn blocking_duration_events_keep_ranges_without_inflating_work() { ...@@ -141,11 +143,13 @@ fn blocking_duration_events_keep_ranges_without_inflating_work() {
ts: 0, ts: 0,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::Exit { TraceRow::Exit {
ts: 10, ts: 10,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
duration_event(100, 90, "fetch request", Some(true)), duration_event(100, 90, "fetch request", Some(true)),
duration_event(100, 90, "fetch attempt", Some(true)), duration_event(100, 90, "fetch attempt", Some(true)),
...@@ -153,11 +157,13 @@ fn blocking_duration_events_keep_ranges_without_inflating_work() { ...@@ -153,11 +157,13 @@ fn blocking_duration_events_keep_ranges_without_inflating_work() {
ts: 100, ts: 100,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::Exit { TraceRow::Exit {
ts: 110, ts: 110,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 110, id: 1 }, TraceRow::End { ts: 110, id: 1 },
], ],
...@@ -201,12 +207,14 @@ fn blocking_events_do_not_change_counted_work_correction() { ...@@ -201,12 +207,14 @@ fn blocking_events_do_not_change_counted_work_correction() {
ts: 0, ts: 0,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
duration_event(100, 100, "waiting", Some(true)), duration_event(100, 100, "waiting", Some(true)),
TraceRow::Exit { TraceRow::Exit {
ts: 100, ts: 100,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 100, id: 1 }, TraceRow::End { ts: 100, id: 1 },
], ],
...@@ -235,6 +243,7 @@ fn non_blocking_and_legacy_duration_events_still_count() { ...@@ -235,6 +243,7 @@ fn non_blocking_and_legacy_duration_events_still_count() {
ts: 0, ts: 0,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
duration_event(80, 50, "external work", Some(false)), duration_event(80, 50, "external work", Some(false)),
duration_event(80, 50, "legacy external work", None), duration_event(80, 50, "legacy external work", None),
...@@ -242,6 +251,7 @@ fn non_blocking_and_legacy_duration_events_still_count() { ...@@ -242,6 +251,7 @@ fn non_blocking_and_legacy_duration_events_still_count() {
ts: 100, ts: 100,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 100, id: 1 }, TraceRow::End { ts: 100, id: 1 },
], ],
...@@ -279,23 +289,27 @@ fn blocking_entered_spans_ignore_own_work_but_count_children() { ...@@ -279,23 +289,27 @@ fn blocking_entered_spans_ignore_own_work_but_count_children() {
ts: 0, ts: 0,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
start(2, Some(1), 10, "child work", None), start(2, Some(1), 10, "child work", None),
TraceRow::Enter { TraceRow::Enter {
ts: 10, ts: 10,
id: 2, id: 2,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::Exit { TraceRow::Exit {
ts: 20, ts: 20,
id: 2, id: 2,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 20, id: 2 }, TraceRow::End { ts: 20, id: 2 },
TraceRow::Exit { TraceRow::Exit {
ts: 30, ts: 30,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 30, id: 1 }, TraceRow::End { ts: 30, id: 1 },
], ],
...@@ -344,11 +358,13 @@ fn blocking_elapsed_ranges_invalidate_cached_ends() { ...@@ -344,11 +358,13 @@ fn blocking_elapsed_ranges_invalidate_cached_ends() {
ts: 0, ts: 0,
id: 2, id: 2,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::Exit { TraceRow::Exit {
ts: 10, ts: 10,
id: 2, id: 2,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
], ],
); );
...@@ -367,11 +383,13 @@ fn blocking_elapsed_ranges_invalidate_cached_ends() { ...@@ -367,11 +383,13 @@ fn blocking_elapsed_ranges_invalidate_cached_ends() {
ts: 20, ts: 20,
id: 2, id: 2,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::Exit { TraceRow::Exit {
ts: 30, ts: 30,
id: 2, id: 2,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 30, id: 2 }, TraceRow::End { ts: 30, id: 2 },
], ],
......
...@@ -442,11 +442,13 @@ mod tests { ...@@ -442,11 +442,13 @@ mod tests {
ts: id, ts: id,
id, id,
thread_id: 1, thread_id: 1,
allocations: None,
}); });
push(TraceRow::Exit { push(TraceRow::Exit {
ts: id + 1, ts: id + 1,
id, id,
thread_id: 1, thread_id: 1,
allocations: None,
}); });
if id != 1 { if id != 1 {
push(TraceRow::End { ts: id + 1, id }); push(TraceRow::End { ts: id + 1, id });
......
...@@ -20,7 +20,7 @@ use std::{ ...@@ -20,7 +20,7 @@ use std::{
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use flate2::bufread::MultiGzDecoder; use flate2::bufread::MultiGzDecoder;
use rustc_hash::{FxHashMap, FxHashSet}; use rustc_hash::{FxHashMap, FxHashSet};
use turbopack_trace_utils::tracing::{TraceRow, TraceValue}; use turbopack_trace_utils::tracing::{Allocations, TraceRow, TraceValue};
/// The magic bytes at the start of every (uncompressed) trace file. /// The magic bytes at the start of every (uncompressed) trace file.
pub const TRACE_HEADER: &[u8] = b"TRACEv0"; pub const TRACE_HEADER: &[u8] = b"TRACEv0";
...@@ -36,21 +36,17 @@ pub enum RowKind { ...@@ -36,21 +36,17 @@ pub enum RowKind {
Exit, Exit,
Event, Event,
Record, Record,
Allocation,
AllocationCounters,
MemorySample, MemorySample,
} }
impl RowKind { impl RowKind {
pub const ALL: [RowKind; 9] = [ pub const ALL: [RowKind; 7] = [
RowKind::Start, RowKind::Start,
RowKind::End, RowKind::End,
RowKind::Enter, RowKind::Enter,
RowKind::Exit, RowKind::Exit,
RowKind::Event, RowKind::Event,
RowKind::Record, RowKind::Record,
RowKind::Allocation,
RowKind::AllocationCounters,
RowKind::MemorySample, RowKind::MemorySample,
]; ];
...@@ -62,8 +58,6 @@ impl RowKind { ...@@ -62,8 +58,6 @@ impl RowKind {
TraceRow::Exit { .. } => RowKind::Exit, TraceRow::Exit { .. } => RowKind::Exit,
TraceRow::Event { .. } => RowKind::Event, TraceRow::Event { .. } => RowKind::Event,
TraceRow::Record { .. } => RowKind::Record, TraceRow::Record { .. } => RowKind::Record,
TraceRow::Allocation { .. } => RowKind::Allocation,
TraceRow::AllocationCounters { .. } => RowKind::AllocationCounters,
TraceRow::MemorySample { .. } => RowKind::MemorySample, TraceRow::MemorySample { .. } => RowKind::MemorySample,
} }
} }
...@@ -76,8 +70,6 @@ impl RowKind { ...@@ -76,8 +70,6 @@ impl RowKind {
RowKind::Exit => "Exit", RowKind::Exit => "Exit",
RowKind::Event => "Event", RowKind::Event => "Event",
RowKind::Record => "Record", RowKind::Record => "Record",
RowKind::Allocation => "Allocation",
RowKind::AllocationCounters => "AllocationCounters",
RowKind::MemorySample => "MemorySample", RowKind::MemorySample => "MemorySample",
} }
} }
...@@ -202,6 +194,10 @@ pub struct Components { ...@@ -202,6 +194,10 @@ pub struct Components {
pub start_values: u64, pub start_values: u64,
pub event_values: u64, pub event_values: u64,
pub record_values: u64, pub record_values: u64,
/// The `allocations` field of `Enter` rows (including the `None` tag).
pub enter_allocations: u64,
/// The `allocations` field of `Exit` rows (including the `None` tag).
pub exit_allocations: u64,
} }
/// Collects size statistics of a trace file. /// Collects size statistics of a trace file.
...@@ -344,9 +340,21 @@ impl TraceSizeAnalyzer { ...@@ -344,9 +340,21 @@ impl TraceSizeAnalyzer {
self.strings[StringKind::SpanName as usize].add(name); self.strings[StringKind::SpanName as usize].add(name);
self.strings[StringKind::SpanTarget as usize].add(target); self.strings[StringKind::SpanTarget as usize].add(target);
} }
TraceRow::End { id, .. } | TraceRow::Enter { id, .. } | TraceRow::Exit { id, .. } => { TraceRow::End { id, .. } => {
self.add_span_row(*id, bytes); self.add_span_row(*id, bytes);
} }
TraceRow::Enter {
id, allocations, ..
} => {
self.add_span_row(*id, bytes);
self.components.enter_allocations += allocations_size(allocations);
}
TraceRow::Exit {
id, allocations, ..
} => {
self.add_span_row(*id, bytes);
self.components.exit_allocations += allocations_size(allocations);
}
TraceRow::Record { id, values } => { TraceRow::Record { id, values } => {
self.add_span_row(*id, bytes); self.add_span_row(*id, bytes);
self.components.record_values += self.add_values(values); self.components.record_values += self.add_values(values);
...@@ -361,9 +369,7 @@ impl TraceSizeAnalyzer { ...@@ -361,9 +369,7 @@ impl TraceSizeAnalyzer {
} }
self.components.event_values += self.add_values(values); self.components.event_values += self.add_values(values);
} }
TraceRow::Allocation { .. } TraceRow::MemorySample { .. } => {}
| TraceRow::AllocationCounters { .. }
| TraceRow::MemorySample { .. } => {}
} }
} }
...@@ -547,6 +553,8 @@ impl TraceSizeAnalyzer { ...@@ -547,6 +553,8 @@ impl TraceSizeAnalyzer {
let start = self.row_kind(RowKind::Start).bytes; let start = self.row_kind(RowKind::Start).bytes;
let event = self.row_kind(RowKind::Event).bytes; let event = self.row_kind(RowKind::Event).bytes;
let record = self.row_kind(RowKind::Record).bytes; let record = self.row_kind(RowKind::Record).bytes;
let enter = self.row_kind(RowKind::Enter).bytes;
let exit = self.row_kind(RowKind::Exit).bytes;
let rows = [ let rows = [
("Start", "name", c.start_name), ("Start", "name", c.start_name),
("Start", "target", c.start_target), ("Start", "target", c.start_target),
...@@ -556,6 +564,18 @@ impl TraceSizeAnalyzer { ...@@ -556,6 +564,18 @@ impl TraceSizeAnalyzer {
"fixed (tag, ts, id, parent)", "fixed (tag, ts, id, parent)",
start - c.start_name - c.start_target - c.start_values, start - c.start_name - c.start_target - c.start_values,
), ),
("Enter", "allocations", c.enter_allocations),
(
"Enter",
"fixed (tag, ts, id, thread)",
enter - c.enter_allocations,
),
("Exit", "allocations", c.exit_allocations),
(
"Exit",
"fixed (tag, ts, id, thread)",
exit - c.exit_allocations,
),
("Event", "values", c.event_values), ("Event", "values", c.event_values),
("Event", "fixed (tag, ts, parent)", event - c.event_values), ("Event", "fixed (tag, ts, parent)", event - c.event_values),
("Record", "values", c.record_values), ("Record", "values", c.record_values),
...@@ -754,6 +774,19 @@ fn str_size(s: &str) -> u64 { ...@@ -754,6 +774,19 @@ fn str_size(s: &str) -> u64 {
varint_size(s.len() as u64) + s.len() as u64 varint_size(s.len() as u64) + s.len() as u64
} }
/// Size of the postcard encoded `allocations` field of `Enter`/`Exit` rows.
fn allocations_size(allocations: &Option<Allocations>) -> u64 {
1 + match allocations {
Some(a) => {
varint_size(a.allocations)
+ varint_size(a.allocation_count)
+ varint_size(a.deallocations)
+ varint_size(a.deallocation_count)
}
None => 0,
}
}
/// Size of a postcard encoded [`TraceValue`] including its variant tag. /// Size of a postcard encoded [`TraceValue`] including its variant tag.
fn value_size(value: &TraceValue<'_>) -> u64 { fn value_size(value: &TraceValue<'_>) -> u64 {
1 + match value { 1 + match value {
...@@ -854,11 +887,11 @@ mod tests { ...@@ -854,11 +887,11 @@ mod tests {
use std::borrow::Cow; use std::borrow::Cow;
use flate2::{Compression as GzLevel, write::GzEncoder}; use flate2::{Compression as GzLevel, write::GzEncoder};
use turbopack_trace_utils::tracing::{TraceRow, TraceValue}; use turbopack_trace_utils::tracing::{Allocations, TraceRow, TraceValue};
use crate::{ use crate::{
Compression, NO_SPAN, RowKind, StringKind, TRACE_HEADER, TraceSizeAnalyzer, UNKNOWN_SPAN, Compression, NO_SPAN, RowKind, StringKind, TRACE_HEADER, TraceSizeAnalyzer, UNKNOWN_SPAN,
value_size, allocations_size, value_size,
}; };
fn sample_rows() -> Vec<TraceRow<'static>> { fn sample_rows() -> Vec<TraceRow<'static>> {
...@@ -878,6 +911,12 @@ mod tests { ...@@ -878,6 +911,12 @@ mod tests {
ts: 2, ts: 2,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: Some(Allocations {
allocations: 1 << 40,
allocation_count: 3,
deallocations: 0,
deallocation_count: 0,
}),
}, },
TraceRow::Start { TraceRow::Start {
ts: 300, ts: 300,
...@@ -913,25 +952,22 @@ mod tests { ...@@ -913,25 +952,22 @@ mod tests {
ts: 600, ts: 600,
id: 1, id: 1,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::End { ts: 700, id: 1 }, TraceRow::End { ts: 700, id: 1 },
// This span is never started // This span is never started
TraceRow::End { ts: 800, id: 99 }, TraceRow::End { ts: 800, id: 99 },
TraceRow::Allocation { TraceRow::MemorySample {
ts: 900, ts: 900,
thread_id: 1, memory: 1,
allocations: 1 << 40, memory_pressure: 0,
allocation_count: 3, active_worker_threads: 1,
deallocations: 0,
deallocation_count: 0,
}, },
TraceRow::AllocationCounters { TraceRow::MemorySample {
ts: 900, ts: 950,
thread_id: 1, memory: 1 << 20,
allocations: 1, memory_pressure: 100,
allocation_count: 1, active_worker_threads: 2,
deallocations: 1,
deallocation_count: 1,
}, },
TraceRow::MemorySample { TraceRow::MemorySample {
ts: 1000, ts: 1000,
...@@ -944,6 +980,7 @@ mod tests { ...@@ -944,6 +980,7 @@ mod tests {
ts: 1200, ts: 1200,
id: 3, id: 3,
thread_id: 2, thread_id: 2,
allocations: Some(Allocations::default()),
}, },
TraceRow::Start { TraceRow::Start {
ts: 1100, ts: 1100,
...@@ -1011,6 +1048,22 @@ mod tests { ...@@ -1011,6 +1048,22 @@ mod tests {
"{value:?}" "{value:?}"
); );
} }
for allocations in [
None,
Some(Allocations::default()),
Some(Allocations {
allocations: u64::MAX,
allocation_count: 1 << 20,
deallocations: 127,
deallocation_count: 128,
}),
] {
assert_eq!(
allocations_size(&allocations),
postcard::to_stdvec(&allocations).unwrap().len() as u64,
"{allocations:?}"
);
}
let rows = sample_rows(); let rows = sample_rows();
let mut analyzer = TraceSizeAnalyzer::new(); let mut analyzer = TraceSizeAnalyzer::new();
...@@ -1029,6 +1082,19 @@ mod tests { ...@@ -1029,6 +1082,19 @@ mod tests {
// with a 1 byte length prefix // with a 1 byte length prefix
assert_eq!(analyzer.components.start_name, 12 + 5); assert_eq!(analyzer.components.start_name, 12 + 5);
assert_eq!(analyzer.components.start_target, 10 + 5); assert_eq!(analyzer.components.start_target, 10 + 5);
let (enter, exit) = rows.iter().fold((0, 0), |(enter, exit), row| match row {
TraceRow::Enter { allocations, .. } => (
enter + postcard::to_stdvec(allocations).unwrap().len() as u64,
exit,
),
TraceRow::Exit { allocations, .. } => (
enter,
exit + postcard::to_stdvec(allocations).unwrap().len() as u64,
),
_ => (enter, exit),
});
assert_eq!(analyzer.components.enter_allocations, enter);
assert_eq!(analyzer.components.exit_allocations, exit);
} }
#[test] #[test]
...@@ -1173,6 +1239,7 @@ mod tests { ...@@ -1173,6 +1239,7 @@ mod tests {
ts: 4, ts: 4,
id: 5, id: 5,
thread_id: 1, thread_id: 1,
allocations: None,
}, },
TraceRow::Event { TraceRow::Event {
ts: 5, ts: 5,
......
...@@ -20,7 +20,7 @@ use crate::{ ...@@ -20,7 +20,7 @@ use crate::{
flavor::WriteGuardFlavor, flavor::WriteGuardFlavor,
tokio_workers::active_worker_threads, tokio_workers::active_worker_threads,
trace_writer::TraceWriter, trace_writer::TraceWriter,
tracing::{TraceRow, TraceValue}, tracing::{Allocations, TraceRow, TraceValue},
}; };
/// 10ms in microseconds /// 10ms in microseconds
...@@ -35,8 +35,8 @@ thread_local! { ...@@ -35,8 +35,8 @@ thread_local! {
/// Options for [`RawTraceLayer`]. /// Options for [`RawTraceLayer`].
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub struct RawTraceLayerOptions { pub struct RawTraceLayerOptions {
/// Track memory: write the per-thread allocation counters on every span enter and exit /// Track memory: attach the per-thread allocation counters to every span enter and exit
/// (`AllocationCounters` rows) and periodic process memory samples (`MemorySample` rows). /// and write periodic process memory samples (`MemorySample` rows).
/// These make up a large part of the trace size, so disabling them significantly reduces /// These make up a large part of the trace size, so disabling them significantly reduces
/// trace file size, at the cost of losing all memory information. Defaults to `true`. /// trace file size, at the cost of losing all memory information. Defaults to `true`.
pub memory: bool, pub memory: bool,
...@@ -71,9 +71,21 @@ pub struct RawTraceLayer<S: Subscriber + for<'a> LookupSpan<'a>> { ...@@ -71,9 +71,21 @@ pub struct RawTraceLayer<S: Subscriber + for<'a> LookupSpan<'a>> {
start: Instant, start: Instant,
next_id: AtomicU64, next_id: AtomicU64,
memory: bool, memory: bool,
/// Reads the allocation counters of the current thread. Only replaced in tests.
allocation_counters: fn() -> Allocations,
_phantom: PhantomData<fn(S)>, _phantom: PhantomData<fn(S)>,
} }
fn thread_allocation_counters() -> Allocations {
let counters = TurboMalloc::allocation_counters();
Allocations {
allocations: counters.allocations as u64,
allocation_count: counters.allocation_count as u64,
deallocations: counters.deallocations as u64,
deallocation_count: counters.deallocation_count as u64,
}
}
impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> { impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
/// Creates a layer with the default [`RawTraceLayerOptions`]. /// Creates a layer with the default [`RawTraceLayerOptions`].
pub fn new(trace_writer: TraceWriter) -> Self { pub fn new(trace_writer: TraceWriter) -> Self {
...@@ -87,6 +99,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> { ...@@ -87,6 +99,7 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
start: Instant::now(), start: Instant::now(),
next_id: AtomicU64::new(1), next_id: AtomicU64::new(1),
memory, memory,
allocation_counters: thread_allocation_counters,
_phantom: PhantomData, _phantom: PhantomData,
} }
} }
...@@ -140,16 +153,9 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> { ...@@ -140,16 +153,9 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
} }
} }
fn report_allocations(&self, ts: u64, thread_id: u64) { /// Returns the allocation counters to attach to an Enter/Exit row, if memory is tracked.
let allocation_counters = turbo_tasks_malloc::TurboMalloc::allocation_counters(); fn allocations(&self) -> Option<Allocations> {
self.write(TraceRow::AllocationCounters { self.memory.then(self.allocation_counters)
ts,
thread_id,
allocations: allocation_counters.allocations as u64,
deallocations: allocation_counters.deallocations as u64,
allocation_count: allocation_counters.allocation_count as u64,
deallocation_count: allocation_counters.deallocation_count as u64,
});
} }
} }
...@@ -197,25 +203,25 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> { ...@@ -197,25 +203,25 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> Layer<S> for RawTraceLayer<S> {
let thread_id = thread::current().id().as_u64().into(); let thread_id = thread::current().id().as_u64().into();
if self.memory { if self.memory {
self.maybe_report_memory_sample(ts); self.maybe_report_memory_sample(ts);
self.report_allocations(ts, thread_id);
} }
let allocations = self.allocations();
self.write(TraceRow::Enter { self.write(TraceRow::Enter {
ts, ts,
id: get_id(ctx, id), id: get_id(ctx, id),
thread_id, thread_id,
allocations,
}); });
} }
fn on_exit(&self, id: &span::Id, ctx: tracing_subscriber::layer::Context<'_, S>) { fn on_exit(&self, id: &span::Id, ctx: tracing_subscriber::layer::Context<'_, S>) {
let ts = self.start.elapsed().as_micros() as u64; let ts = self.start.elapsed().as_micros() as u64;
let thread_id = thread::current().id().as_u64().into(); let thread_id = thread::current().id().as_u64().into();
if self.memory { let allocations = self.allocations();
self.report_allocations(ts, thread_id);
}
self.write(TraceRow::Exit { self.write(TraceRow::Exit {
ts, ts,
id: get_id(ctx, id), id: get_id(ctx, id),
thread_id, thread_id,
allocations,
}); });
} }
...@@ -314,6 +320,7 @@ impl Visit for ValuesVisitor { ...@@ -314,6 +320,7 @@ impl Visit for ValuesVisitor {
#[cfg(test)] #[cfg(test)]
pub(crate) mod tests { pub(crate) mod tests {
use std::{ use std::{
cell::Cell,
io::{self, Write}, io::{self, Write},
sync::{Arc, Mutex}, sync::{Arc, Mutex},
}; };
...@@ -323,9 +330,45 @@ pub(crate) mod tests { ...@@ -323,9 +330,45 @@ pub(crate) mod tests {
use crate::{ use crate::{
raw_trace::{RawTraceLayer, RawTraceLayerOptions}, raw_trace::{RawTraceLayer, RawTraceLayerOptions},
trace_writer::TraceWriter, trace_writer::TraceWriter,
tracing::TraceRow, tracing::{Allocations, TraceRow},
}; };
thread_local! {
/// Fake allocation counters of the current thread, controlled by the test.
static FAKE_ALLOCATIONS: Cell<Allocations> = const {
Cell::new(Allocations {
allocations: 0,
allocation_count: 0,
deallocations: 0,
deallocation_count: 0,
})
};
}
fn fake_allocation_counters() -> Allocations {
FAKE_ALLOCATIONS.with(|c| c.get())
}
/// Simulates `count` allocations of `bytes` bytes each on the current thread.
pub(crate) fn fake_allocate(bytes: u64, count: u64) {
FAKE_ALLOCATIONS.with(|c| {
let mut counters = c.get();
counters.allocations += bytes * count;
counters.allocation_count += count;
c.set(counters);
});
}
/// Simulates `count` deallocations of `bytes` bytes each on the current thread.
pub(crate) fn fake_deallocate(bytes: u64, count: u64) {
FAKE_ALLOCATIONS.with(|c| {
let mut counters = c.get();
counters.deallocations += bytes * count;
counters.deallocation_count += count;
c.set(counters);
});
}
#[derive(Clone, Default)] #[derive(Clone, Default)]
struct SharedBuffer(Arc<Mutex<Vec<u8>>>); struct SharedBuffer(Arc<Mutex<Vec<u8>>>);
...@@ -341,12 +384,14 @@ pub(crate) mod tests { ...@@ -341,12 +384,14 @@ pub(crate) mod tests {
} }
/// Runs `f` with a [`RawTraceLayer`] installed as the thread's default subscriber and /// Runs `f` with a [`RawTraceLayer`] installed as the thread's default subscriber and
/// returns the complete trace file content. /// returns the complete trace file content. Allocation counters are read from the fake
/// counters controlled by [`fake_allocate`] and [`fake_deallocate`].
pub(crate) fn capture(options: RawTraceLayerOptions, f: impl FnOnce()) -> Vec<u8> { pub(crate) fn capture(options: RawTraceLayerOptions, f: impl FnOnce()) -> Vec<u8> {
let buffer = SharedBuffer::default(); let buffer = SharedBuffer::default();
let (trace_writer, guard) = TraceWriter::new(buffer.clone()); let (trace_writer, guard) = TraceWriter::new(buffer.clone());
let subscriber = let mut layer = RawTraceLayer::with_options(trace_writer, options);
Registry::default().with(RawTraceLayer::with_options(trace_writer, options)); layer.allocation_counters = fake_allocation_counters;
let subscriber = Registry::default().with(layer);
tracing::subscriber::with_default(subscriber, f); tracing::subscriber::with_default(subscriber, f);
// Flushes all buffers and waits for the writer thread to finish. // Flushes all buffers and waits for the writer thread to finish.
drop(guard); drop(guard);
...@@ -376,16 +421,40 @@ pub(crate) mod tests { ...@@ -376,16 +421,40 @@ pub(crate) mod tests {
rows.iter().filter(|row| predicate(row)).count() rows.iter().filter(|row| predicate(row)).count()
} }
/// Returns the allocation counters attached to each Enter/Exit row, in order.
pub(crate) fn enter_exit_allocations(
rows: &[TraceRow<'_>],
) -> Vec<(&'static str, Option<Allocations>)> {
rows.iter()
.filter_map(|row| match row {
TraceRow::Enter { allocations, .. } => Some(("enter", *allocations)),
TraceRow::Exit { allocations, .. } => Some(("exit", *allocations)),
_ => None,
})
.collect()
}
#[test] #[test]
fn writes_allocation_counters_by_default() { fn attaches_allocation_counters_to_enter_and_exit() {
let data = capture(RawTraceLayerOptions::default(), enter_exit_span); let mut at_enter = Allocations::default();
let mut at_exit = Allocations::default();
let data = capture(RawTraceLayerOptions::default(), || {
fake_allocate(10, 1);
let span = tracing::info_span!("test span");
at_enter = fake_allocation_counters();
let guard = span.enter();
fake_allocate(100, 2);
fake_deallocate(5, 1);
at_exit = fake_allocation_counters();
drop(guard);
});
let rows = decode(&data); let rows = decode(&data);
assert_eq!(count(&rows, |r| matches!(r, TraceRow::Enter { .. })), 1);
assert_eq!(count(&rows, |r| matches!(r, TraceRow::Exit { .. })), 1);
assert_eq!( assert_eq!(
count(&rows, |r| matches!(r, TraceRow::AllocationCounters { .. })), enter_exit_allocations(&rows),
2 vec![("enter", Some(at_enter)), ("exit", Some(at_exit))]
); );
assert_eq!(at_exit.allocations - at_enter.allocations, 200);
assert_eq!(at_exit.deallocation_count - at_enter.deallocation_count, 1);
} }
#[test] #[test]
...@@ -393,17 +462,15 @@ pub(crate) mod tests { ...@@ -393,17 +462,15 @@ pub(crate) mod tests {
let data = capture(RawTraceLayerOptions { memory: false }, enter_exit_span); let data = capture(RawTraceLayerOptions { memory: false }, enter_exit_span);
let rows = decode(&data); let rows = decode(&data);
assert_eq!(count(&rows, |r| matches!(r, TraceRow::Start { .. })), 1); assert_eq!(count(&rows, |r| matches!(r, TraceRow::Start { .. })), 1);
assert_eq!(count(&rows, |r| matches!(r, TraceRow::Enter { .. })), 1); assert_eq!(
assert_eq!(count(&rows, |r| matches!(r, TraceRow::Exit { .. })), 1); enter_exit_allocations(&rows),
vec![("enter", None), ("exit", None)]
);
assert_eq!(count(&rows, |r| matches!(r, TraceRow::End { .. })), 1); assert_eq!(count(&rows, |r| matches!(r, TraceRow::End { .. })), 1);
assert_eq!( assert_eq!(
count(&rows, |r| matches!( count(&rows, |r| matches!(r, TraceRow::MemorySample { .. })),
r,
TraceRow::AllocationCounters { .. }
| TraceRow::Allocation { .. }
| TraceRow::MemorySample { .. }
)),
0 0
); );
assert_eq!(rows.len(), 4);
} }
} }
...@@ -41,6 +41,9 @@ pub enum TraceRow<'a> { ...@@ -41,6 +41,9 @@ pub enum TraceRow<'a> {
id: u64, id: u64,
/// The thread id of the thread that entered the span. /// The thread id of the thread that entered the span.
thread_id: u64, thread_id: u64,
/// The allocation counters of the thread right before the span was entered, if memory
/// is tracked. See [`Allocations`].
allocations: Option<Allocations>,
}, },
/// A span has been exited. This means it is not spending CPU time anymore. /// A span has been exited. This means it is not spending CPU time anymore.
Exit { Exit {
...@@ -50,6 +53,9 @@ pub enum TraceRow<'a> { ...@@ -50,6 +53,9 @@ pub enum TraceRow<'a> {
id: u64, id: u64,
/// The thread id of the thread that exits the span. /// The thread id of the thread that exits the span.
thread_id: u64, thread_id: u64,
/// The allocation counters of the thread right before the span was exited, if memory
/// is tracked. See [`Allocations`].
allocations: Option<Allocations>,
}, },
/// A event has happened for some span. /// A event has happened for some span.
Event { Event {
...@@ -69,37 +75,6 @@ pub enum TraceRow<'a> { ...@@ -69,37 +75,6 @@ pub enum TraceRow<'a> {
#[serde(borrow)] #[serde(borrow)]
values: Vec<(Cow<'a, str>, TraceValue<'a>)>, values: Vec<(Cow<'a, str>, TraceValue<'a>)>,
}, },
/// Data about (de)allocations that happened
Allocation {
/// Timestamp
ts: u64,
/// The thread id of the thread where allocations happened.
thread_id: u64,
/// Allocations
allocations: u64,
/// Allocation count
allocation_count: u64,
/// Deallocations
deallocations: u64,
/// Deallocation count
deallocation_count: u64,
},
/// Data about (de)allocations per thread counters. Actual allocations can
/// be computed from the difference.
AllocationCounters {
/// Timestamp
ts: u64,
/// The thread id of the thread where allocations happened.
thread_id: u64,
/// Allocations
allocations: u64,
/// Allocation count
allocation_count: u64,
/// Deallocations
deallocations: u64,
/// Deallocation count
deallocation_count: u64,
},
/// A snapshot of process memory and non-idle Tokio scheduler workers. /// A snapshot of process memory and non-idle Tokio scheduler workers.
MemorySample { MemorySample {
/// Timestamp /// Timestamp
...@@ -115,6 +90,26 @@ pub enum TraceRow<'a> { ...@@ -115,6 +90,26 @@ pub enum TraceRow<'a> {
}, },
} }
/// The per-thread allocation counters, as attached to [`TraceRow::Enter`] and
/// [`TraceRow::Exit`].
///
/// The counters are cumulative for the thread (allocations made by the tracing
/// itself are excluded). The difference to the previous counters of the same
/// thread is what was (de)allocated in between. Readers attribute it to the span
/// on top of the thread's span stack before the row is applied: for `Enter`, the
/// span that was running before (if any), for `Exit`, the span that is exited.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct Allocations {
/// Allocated bytes
pub allocations: u64,
/// Number of allocations
pub allocation_count: u64,
/// Deallocated bytes
pub deallocations: u64,
/// Number of deallocations
pub deallocation_count: u64,
}
#[derive(Debug, Serialize, Deserialize)] #[derive(Debug, Serialize, Deserialize)]
pub enum TraceValue<'a> { pub enum TraceValue<'a> {
String(#[serde(borrow)] Cow<'a, str>), String(#[serde(borrow)] Cow<'a, str>),
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment