Unverified Commit 207b08b6 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Report process memory footprint (RSS) in TurboMalloc and trace samples (#99527)

Adopts #93362. Closes #93362.

### What?
Adds the process memory footprint (RSS / "Memory" column on macOS) to
TurboMalloc's trace samples, mirroring how OS memory pressure was wired
up in #93333.
### Why?
The trace viewer already plots `memory` (TurboMalloc's accounted bytes)
and, since #93333, OS-level `memory_pressure`. Neither captures the
actual resident size of the process. Tracking the OS-reported footprint
alongside the allocator's view exposes overhead from things the
allocator doesn't account for (e.g. mimalloc reserves, mmap'd files,
executable pages, native dependencies) and makes leaks/regressions in
real RSS visible in traces.
### How?
- New `TurboMalloc::memory_footprint() -> Option<usize>` (bytes),
implemented in
`turbopack/crates/turbo-tasks-malloc/src/memory_footprint.rs` with one
`mod platform` per OS:
- **Linux**: parses the resident-pages field of `/proc/self/statm` and
multiplies by `sysconf(_SC_PAGESIZE)`.
- **macOS**: `task_info(mach_task_self(), TASK_VM_INFO,
…).phys_footprint` — the same value Apple's Activity Monitor labels
"Memory". Guards against older kernels that return a shorter struct by
checking the returned `count` covers `phys_footprint`.
- **Windows**: `GetProcessMemoryInfo(GetCurrentProcess(),
…).WorkingSetSize`. New `windows-sys` features
`Win32_System_ProcessStatus` and `Win32_System_Threading`.
  - Other targets (incl. wasm) return `None`.
- `TraceRow::MemorySample` gains a `memory_footprint: u64` field.
`RawTraceLayer::maybe_report_memory_sample` captures it via
`TurboMalloc::memory_footprint().map(|v| v as u64).unwrap_or(0)` at the
same sampling point as `memory` and `memory_pressure`.
- Trace-server `Store::MemorySample` becomes `(Timestamp, u64, u8, u64)`
carrying `(ts, memory, memory_pressure, memory_footprint)`. A new
`memory_footprint_samples_for_range` mirrors `memory_samples_for_range`
and `memory_pressure_samples_for_range`, sharing the existing
`memory_samples_slice` helper so the three result vectors stay
index-aligned and use the same `MAX_MEMORY_SAMPLES`-cap max-per-chunk
downsampling.
- `ServerToClientMessage::QueryResult` exposes a parallel
`memory_footprint_samples: Vec<u64>` next to `memory_samples` and
`memory_pressure_samples`.
This is a wire-format break for the postcard-encoded raw trace (added a
serde field to `TraceRow::MemorySample`); old trace files cannot be read
by the new trace-server. This matches the precedent set by #93333.
### Testing
- `cargo test -p turbo-tasks-malloc` — 10 pass, including a new
`parses_typical_statm_content` / `returns_none_on_malformed_statm`
parser pair and a runtime
`memory_footprint_is_reported_on_supported_platforms` check that asserts
`Some(_)` and a positive, plausibly bounded byte count on
Linux/macOS/Windows.
- `cargo build`, `cargo clippy --all-targets`, and `cargo fmt --check`
clean for `turbo-tasks-malloc`, `turbopack-trace-utils`, and
`turbopack-trace-server`.
<!-- NEXT_JS_LLM_PR -->

<!-- fleet fdf95ce6-ffc0-474d-8a27-daecba57a97d -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
parent 82794ab4
......@@ -135,10 +135,12 @@ pub struct TraceSpanInfo {
/// fields instead.
///
/// Each entry is `[ts_offset_from_span_start_in_ticks, bytes, pressure,
/// active_worker_threads]`: `bytes` is TurboMalloc memory usage,
/// footprint, active_worker_threads]`: `bytes` is TurboMalloc memory usage,
/// `pressure` is the memory-pressure byte (0 = no pressure, higher = more
/// pressure), and `active_worker_threads` counts non-parked Tokio scheduler
/// workers. `100 ticks = 1 µs`. Capped and downsampled by the store.
/// pressure), `footprint` is the process memory footprint (RSS) in bytes
/// (0 = not reported by the platform), and `active_worker_threads` counts
/// non-parked Tokio scheduler workers. `100 ticks = 1 µs`. Capped and
/// downsampled by the store.
pub memory_samples: Vec<Vec<i64>>,
/// Summary of `memorySamples`; absent when the span's range holds none.
/// Unlike the allocation counters these are absolute live-heap readings, so
......@@ -163,6 +165,9 @@ pub struct TraceMemorySummary {
pub peak: i64,
/// Highest memory-pressure byte in the range (0 = no pressure).
pub max_pressure: u8,
/// Largest process memory footprint (RSS) in bytes in the range (0 = not
/// reported by the platform).
pub max_footprint: i64,
}
/// The result of a `query_trace_spans` call.
......@@ -215,8 +220,14 @@ fn convert_span(s: turbopack_trace_server::SpanInfo) -> TraceSpanInfo {
memory_samples: s
.memory_samples
.into_iter()
.map(|(ts, mem, pressure, workers)| {
vec![ts, mem as i64, pressure as i64, workers as i64]
.map(|(ts, mem, pressure, footprint, workers)| {
vec![
ts,
mem as i64,
pressure as i64,
footprint as i64,
workers as i64,
]
})
.collect(),
memory_summary: s.memory_summary.map(|m| TraceMemorySummary {
......@@ -226,6 +237,7 @@ fn convert_span(s: turbopack_trace_server::SpanInfo) -> TraceSpanInfo {
min: m.min as i64,
peak: m.peak as i64,
max_pressure: m.max_pressure,
max_footprint: m.max_footprint as i64,
}),
children: s.children.into_iter().map(convert_span).collect(),
}
......
......@@ -791,6 +791,11 @@ export interface TraceMemorySummary {
peak: number
/** Highest memory-pressure byte in the range (0 = no pressure). */
maxPressure: number
/**
* Largest process memory footprint (RSS) in bytes in the range (0 = not
* reported by the platform).
*/
maxFootprint: number
}
/** Options for `query_trace_spans`. */
......@@ -954,10 +959,12 @@ export interface TraceSpanInfo {
* fields instead.
*
* Each entry is `[ts_offset_from_span_start_in_ticks, bytes, pressure,
* active_worker_threads]`: `bytes` is TurboMalloc memory usage,
* footprint, active_worker_threads]`: `bytes` is TurboMalloc memory usage,
* `pressure` is the memory-pressure byte (0 = no pressure, higher = more
* pressure), and `active_worker_threads` counts non-parked Tokio scheduler
* workers. `100 ticks = 1 µs`. Capped and downsampled by the store.
* pressure), `footprint` is the process memory footprint (RSS) in bytes
* (0 = not reported by the platform), and `active_worker_threads` counts
* non-parked Tokio scheduler workers. `100 ticks = 1 µs`. Capped and
* downsampled by the store.
*/
memorySamples: Array<Array<number>>
/**
......
......@@ -86,12 +86,18 @@ function summarizeMemorySamples(span: TraceSpanInfo): string | null {
if (!summary) return null
const delta = summary.end - summary.start
const deltaSign = delta >= 0 ? '+' : '-'
const workers = span.memorySamples.map((s) => s[3])
const workers = span.memorySamples.map((s) => s[4])
// Footprint is 0 when the platform does not report it.
const footprintSummary =
summary.maxFootprint > 0
? `, maxFootprint=${formatBytes(summary.maxFootprint)}`
: ''
return (
`samples=${summary.count}, peak=${formatBytes(summary.peak)}, min=${formatBytes(summary.min)}, ` +
`start=${formatBytes(summary.start)}, end=${formatBytes(summary.end)}, ` +
`Δ=${deltaSign}${formatBytes(Math.abs(delta))}, maxPressure=${summary.maxPressure}, ` +
`activeWorkerThreads=${Math.min(...workers)}–${Math.max(...workers)}`
`Δ=${deltaSign}${formatBytes(Math.abs(delta))}, maxPressure=${summary.maxPressure}` +
footprintSummary +
`, activeWorkerThreads=${Math.min(...workers)}–${Math.max(...workers)}`
)
}
......@@ -151,7 +157,7 @@ function renderSpanMarkdown(
const memSummary = summarizeMemorySamples(span)
if (memSummary) {
md += `\n**Process samples (TurboMalloc live bytes, memory pressure, active Tokio workers):** ${memSummary}\n`
md += `\n**Process samples (TurboMalloc live bytes, memory pressure, memory footprint, active Tokio workers):** ${memSummary}\n`
}
if (span.children.length > 0) {
......@@ -218,7 +224,7 @@ export async function startTurboTraceServerCli(
'',
'Allocations: `allocations` / `deallocations` / `allocationCount` and `persistentAllocations`, each with a `self*` counterpart excluding children. Comparing a total to its `self` shows whether a span allocates directly or only through descendants.',
'',
'`persistentAllocations` ranks allocators; it is NOT retained memory. It is allocated-minus-freed per TurboMalloc counters, which never see turbo-tasks cell or cache drops, so a total far above real peak RSS is expected rather than a leak. For absolute memory use `memorySummary` (count/start/end/min/peak/maxPressure, precomputed from `memorySamples`).',
'`persistentAllocations` ranks allocators; it is NOT retained memory. It is allocated-minus-freed per TurboMalloc counters, which never see turbo-tasks cell or cache drops, so a total far above real peak RSS is expected rather than a leak. For absolute memory use `memorySummary` (count/start/end/min/peak/maxPressure/maxFootprint, precomputed from `memorySamples`; maxFootprint is the process RSS peak, 0 = not reported).',
'',
"But live heap is process-wide: a span's samples are just the global series sliced to its time range, so concurrent spans report identical memory however much each allocated. Rank concurrent work by the allocation fields, never by memory.",
'',
......@@ -226,7 +232,7 @@ export async function startTurboTraceServerCli(
'',
'For aggregated groups every allocation field is a group total, while `cpuDuration`, `correctedDuration` and `memorySamples` describe the example span only. `firstSpanId` is first in execution order; use `heaviestSpanId` to reach the member holding the most bytes.',
'',
'Set `outputType: "json"` for full precision and the raw `memorySamples` entries `[tsOffsetTicks, bytes, pressure, active_worker_threads]` (active_worker_threads excludes parked and blocking-pool threads); markdown is a human summary.',
'Set `outputType: "json"` for full precision and the raw `memorySamples` entries `[tsOffsetTicks, bytes, pressure, footprint, active_worker_threads]` (footprint is the process RSS in bytes, 0 = not reported; active_worker_threads excludes parked and blocking-pool threads); markdown is a human summary.',
].join('\n'),
inputSchema: {
parent: z
......
......@@ -38,12 +38,17 @@ mimalloc = { version = "0.1.48", features = [
"extended",
"local_dynamic_tls",
], optional = true }
libc = "0.2"
[target.'cfg(target_os = "macos")'.dependencies]
libc = "0.2"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.60", features = ["Win32_System_SystemInformation"] }
windows-sys = { version = "0.60", features = [
"Win32_System_ProcessStatus",
"Win32_System_SystemInformation",
"Win32_System_Threading",
] }
[features]
custom_allocator = ["dep:mimalloc", "dep:libmimalloc-sys"]
......
mod counter;
mod memory_footprint;
mod memory_pressure;
use std::{
......@@ -173,6 +174,19 @@ impl TurboMalloc {
pub fn memory_pressure() -> Option<u8> {
memory_pressure::memory_pressure()
}
/// Returns the resident memory size of the current process in bytes, or
/// `None` when the current platform does not expose a memory footprint
/// signal or a query for it failed.
///
/// - On Linux this is the resident set size from `/proc/self/statm`.
/// - On macOS this is `rusage_info_v0::ri_phys_footprint` from `proc_pid_rusage` (what Apple
/// calls "memory footprint" / Activity Monitor's "Memory" column).
/// - On Windows this is `PROCESS_MEMORY_COUNTERS::WorkingSetSize`.
/// - On other platforms this returns `None`.
pub fn memory_footprint() -> Option<usize> {
memory_footprint::memory_footprint()
}
}
/// Get the allocator for this platform that we should wrap with TurboMalloc.
......
//! Per-OS memory footprint detection.
//!
//! All implementations return the resident memory size of the current
//! process in bytes. Platforms that do not expose a memory footprint signal
//! (or for which the query fails) return `None`.
/// See [`super::TurboMalloc::memory_footprint`].
pub fn memory_footprint() -> Option<usize> {
platform::memory_footprint()
}
#[cfg(all(target_os = "linux", not(target_family = "wasm"), not(miri)))]
mod platform {
use std::sync::LazyLock;
static PAGE_SIZE: LazyLock<Option<usize>> = LazyLock::new(|| {
// Safety: `sysconf` is thread-safe and never accesses memory through
// its argument.
let value = unsafe { libc::sysconf(libc::_SC_PAGESIZE) };
if value <= 0 {
return None;
}
usize::try_from(value).ok()
});
/// Reads `/proc/self/statm` and returns the resident set size in bytes
/// (the second whole number, multiplied by the system page size).
pub fn memory_footprint() -> Option<usize> {
let content = std::fs::read_to_string("/proc/self/statm").ok()?;
let resident_pages = parse_resident_pages(&content)?;
let page_size = (*PAGE_SIZE)?;
resident_pages.checked_mul(page_size)
}
fn parse_resident_pages(content: &str) -> Option<usize> {
// Expected format: "<size> <resident> <shared> <text> 0 <data> 0"
// where each value is in pages.
let mut iter = content.split_ascii_whitespace();
let _size = iter.next()?;
let resident = iter.next()?;
resident.parse().ok()
}
#[cfg(test)]
mod tests {
use super::parse_resident_pages;
#[test]
fn parses_typical_statm_content() {
assert_eq!(
parse_resident_pages("12345 6789 4321 8 0 1234 0\n"),
Some(6789)
);
}
#[test]
fn returns_none_on_malformed_statm() {
assert_eq!(parse_resident_pages(""), None);
assert_eq!(parse_resident_pages("12345"), None);
assert_eq!(parse_resident_pages("12345 garbage"), None);
}
}
}
#[cfg(target_os = "macos")]
mod platform {
/// Reads `rusage_info_v0::ri_phys_footprint` for the current process via
/// `proc_pid_rusage`. This is what Apple calls the "memory footprint" of
/// a process and is the value reported in Activity Monitor's "Memory"
/// column.
pub fn memory_footprint() -> Option<usize> {
let mut info = std::mem::MaybeUninit::<libc::rusage_info_v0>::zeroed();
// Safety: `proc_pid_rusage` writes a `rusage_info_v0` into the buffer
// when called with `RUSAGE_INFO_V0`. `getpid` is always safe and
// returns the current process id.
let rc = unsafe {
libc::proc_pid_rusage(
libc::getpid(),
libc::RUSAGE_INFO_V0,
info.as_mut_ptr() as *mut libc::rusage_info_t,
)
};
if rc != 0 {
return None;
}
// Safety: `proc_pid_rusage` returned success, so the buffer is
// initialized.
let info = unsafe { info.assume_init() };
usize::try_from(info.ri_phys_footprint).ok()
}
}
#[cfg(windows)]
mod platform {
use std::mem::size_of;
use windows_sys::Win32::System::{
ProcessStatus::{GetProcessMemoryInfo, PROCESS_MEMORY_COUNTERS},
Threading::GetCurrentProcess,
};
/// Reads `PROCESS_MEMORY_COUNTERS::WorkingSetSize`, which is the amount
/// of memory currently resident for the calling process in bytes.
pub fn memory_footprint() -> Option<usize> {
// Safety: `PROCESS_MEMORY_COUNTERS` is a plain C struct consisting
// only of integer fields, so the all-zero bit pattern is a valid value.
let mut counters: PROCESS_MEMORY_COUNTERS = unsafe { std::mem::zeroed() };
// Safety: `counters` is a properly sized and zero-initialized
// `PROCESS_MEMORY_COUNTERS`. `GetCurrentProcess` returns a
// pseudo-handle that does not need closing.
let ok = unsafe {
GetProcessMemoryInfo(
GetCurrentProcess(),
&mut counters,
size_of::<PROCESS_MEMORY_COUNTERS>() as u32,
)
};
if ok == 0 {
return None;
}
Some(counters.WorkingSetSize)
}
}
#[cfg(not(any(
all(target_os = "linux", not(target_family = "wasm"), not(miri)),
target_os = "macos",
windows,
)))]
mod platform {
pub fn memory_footprint() -> Option<usize> {
None
}
}
#[cfg(test)]
mod tests {
use super::memory_footprint;
#[test]
fn memory_footprint_is_reported_on_supported_platforms() {
let value = memory_footprint();
// On all supported platforms the value must be reported.
#[cfg(any(
all(target_os = "linux", not(target_family = "wasm"), not(miri)),
target_os = "macos",
windows,
))]
let value = value.expect("memory_footprint() should return Some on this platform");
// On unsupported platforms we expect None and have nothing further to assert.
#[cfg(not(any(
all(target_os = "linux", not(target_family = "wasm"), not(miri)),
target_os = "macos",
windows,
)))]
let Some(value) = value else {
return;
};
assert!(
value > 0,
"memory_footprint() returned {value}, expected a positive byte count"
);
assert!(
value < isize::MAX as usize,
"memory_footprint() returned {value}, which is implausibly large"
);
}
}
......@@ -225,15 +225,18 @@ pub struct SpanInfo {
/// memory over a span that dominates its window.
///
/// Each tuple is `(ts_offset_from_span_start_in_ticks, bytes, pressure,
/// active_worker_threads)`. `bytes` is TurboMalloc memory usage;
/// `pressure` is the memory-pressure byte (0 = no pressure, higher =
/// more pressure), and `active_worker_threads` counts non-parked Tokio
/// scheduler workers. `100 ticks = 1 µs`. The offset is within the span.
/// footprint, active_worker_threads)`. `bytes` is TurboMalloc memory
/// usage; `pressure` is the memory-pressure byte (0 = no pressure, higher
/// = more pressure); `footprint` is the process memory footprint (RSS) in
/// bytes (0 = not reported by the platform); and `active_worker_threads`
/// counts non-parked Tokio scheduler workers. `100 ticks = 1 µs`. The
/// offset is within the span.
///
/// The store caps the series at `MAX_MEMORY_SAMPLES`; when more samples
/// exist, groups are merged by picking the group's max-memory sample,
/// retaining its timestamp, pressure and worker count.
pub memory_samples: Vec<(i64, u64, u8, u64)>,
/// exist in the range, consecutive groups are merged: the timestamp and
/// value come from the group's max-memory sample, while pressure,
/// footprint and worker count are each the max over the group.
pub memory_samples: Vec<(i64, u64, u8, u64, u64)>,
/// Summary of `memory_samples`. `None` when the span's range holds none.
pub memory_summary: Option<MemorySummary>,
/// Descendants of this span, populated only when `QueryOptions::depth` is
......@@ -259,12 +262,15 @@ pub struct MemorySummary {
pub peak: u64,
/// Highest memory-pressure byte seen in the range (0 = no pressure).
pub max_pressure: u8,
/// Largest process memory footprint (RSS) in bytes seen in the range
/// (0 = not reported by the platform).
pub max_footprint: u64,
}
impl MemorySummary {
/// Summarize a sample series, or `None` if it is empty.
fn from_samples(samples: &[(i64, u64, u8, u64)]) -> Option<Self> {
let (_, first_bytes, first_pressure, _) = *samples.first()?;
fn from_samples(samples: &[(i64, u64, u8, u64, u64)]) -> Option<Self> {
let (_, first_bytes, first_pressure, first_footprint, _) = *samples.first()?;
let mut summary = MemorySummary {
count: samples.len(),
start: first_bytes,
......@@ -272,11 +278,13 @@ impl MemorySummary {
min: first_bytes,
peak: first_bytes,
max_pressure: first_pressure,
max_footprint: first_footprint,
};
for &(_, bytes, pressure, _) in &samples[1..] {
for &(_, bytes, pressure, footprint, _) in &samples[1..] {
summary.min = summary.min.min(bytes);
summary.peak = summary.peak.max(bytes);
summary.max_pressure = summary.max_pressure.max(pressure);
summary.max_footprint = summary.max_footprint.max(footprint);
}
Some(summary)
}
......@@ -497,12 +505,14 @@ fn sort_spans(items: &mut [Located<SpanRef<'_>>], sort: SortMode) {
}
/// Memory samples recorded while `span` was live, offset from its start.
fn memory_samples_for(store: &store::Store, span: &SpanRef<'_>) -> Vec<(i64, u64, u8, u64)> {
fn memory_samples_for(store: &store::Store, span: &SpanRef<'_>) -> Vec<(i64, u64, u8, u64, u64)> {
let span_start = *span.start() as i64;
store
.memory_samples_for_range_with_ts(span.start(), span.end())
.into_iter()
.map(|(ts, mem, pressure, workers)| ((*ts as i64) - span_start, mem, pressure, workers))
.map(|(ts, mem, pressure, footprint, workers)| {
((*ts as i64) - span_start, mem, pressure, footprint, workers)
})
.collect()
}
......@@ -1258,7 +1268,11 @@ mod tests {
#[test]
fn memory_summary_reports_peak_not_last() {
let samples = [(0i64, 100u64, 0u8, 2u64), (1, 900, 3, 1), (2, 200, 1, 3)];
let samples = [
(0i64, 100u64, 0u8, 5000u64, 2u64),
(1, 900, 3, 4000, 1),
(2, 200, 1, 7000, 3),
];
let summary = MemorySummary::from_samples(&samples).expect("samples present");
assert_eq!(summary.count, 3);
assert_eq!(summary.start, 100);
......@@ -1266,6 +1280,8 @@ mod tests {
assert_eq!(summary.min, 100);
assert_eq!(summary.peak, 900);
assert_eq!(summary.max_pressure, 3);
// Footprint is maxed independently of the memory peak.
assert_eq!(summary.max_footprint, 7000);
assert!(MemorySummary::from_samples(&[]).is_none());
}
......
......@@ -32,6 +32,7 @@ fn reads_active_workers_and_exposes_them_in_span_queries() {
ts: 20,
memory: 1234,
memory_pressure: 7,
memory_footprint: 5555,
active_worker_threads: 2,
},
TraceRow::Exit {
......@@ -58,7 +59,8 @@ fn reads_active_workers_and_exposes_them_in_span_queries() {
.memory_samples_for_range_with_ts(Timestamp::from_micros(10), Timestamp::from_micros(30));
assert_eq!(samples[0].1, 1234);
assert_eq!(samples[0].2, 7);
assert_eq!(samples[0].3, 2);
assert_eq!(samples[0].3, 5555);
assert_eq!(samples[0].4, 2);
for aggregated in [false, true] {
let result = query_spans(
......@@ -75,7 +77,10 @@ fn reads_active_workers_and_exposes_them_in_span_queries() {
},
);
assert_eq!(result.spans.len(), 1);
assert_eq!(result.spans[0].memory_samples, vec![(1000, 1234, 7, 2)]);
assert_eq!(
result.spans[0].memory_samples,
vec![(1000, 1234, 7, 5555, 2)]
);
}
}
......
......@@ -252,10 +252,17 @@ impl TurbopackFormat {
ts,
memory,
memory_pressure,
memory_footprint,
active_worker_threads,
} => {
let ts = Timestamp::from_micros(ts);
store.add_memory_sample(ts, memory, memory_pressure, active_worker_threads);
store.add_memory_sample(
ts,
memory,
memory_pressure,
memory_footprint,
active_worker_threads,
);
}
}
}
......
......@@ -19,6 +19,7 @@ use crate::{
#[derive(Serialize, Debug)]
#[serde(tag = "type")]
#[serde(rename_all = "kebab-case")]
#[allow(clippy::large_enum_variant)] // Short-lived message, serialized right away
pub enum ServerToClientMessage {
ViewLine {
#[serde(flatten)]
......@@ -45,6 +46,7 @@ pub enum ServerToClientMessage {
path: Vec<String>,
memory_samples: Vec<u64>,
memory_pressure_samples: Vec<u8>,
memory_footprint_samples: Vec<u64>,
active_worker_threads_samples: Vec<u64>,
concurrency_samples: Vec<f64>,
},
......@@ -299,6 +301,8 @@ fn handle_connection(
store.memory_samples_for_range(span.start(), span.end());
let memory_pressure_samples = store
.memory_pressure_samples_for_range(span.start(), span.end());
let memory_footprint_samples = store
.memory_footprint_samples_for_range(span.start(), span.end());
let active_worker_threads_samples = store
.active_worker_threads_samples_for_range(
span.start(),
......@@ -321,6 +325,7 @@ fn handle_connection(
path,
memory_samples,
memory_pressure_samples,
memory_footprint_samples,
active_worker_threads_samples,
concurrency_samples,
}
......@@ -340,6 +345,7 @@ fn handle_connection(
path: Vec::new(),
memory_samples: Vec::new(),
memory_pressure_samples: Vec::new(),
memory_footprint_samples: Vec::new(),
active_worker_threads_samples: Vec::new(),
concurrency_samples: Vec::new(),
}
......@@ -409,12 +415,14 @@ mod tests {
path: Vec::new(),
memory_samples: vec![1024],
memory_pressure_samples: vec![2],
memory_footprint_samples: vec![4096],
active_worker_threads_samples: vec![3],
concurrency_samples: vec![1.25],
};
let json = serde_json::to_value(message).unwrap();
assert_eq!(json["type"], "query-result");
assert_eq!(json["memorySamples"], serde_json::json!([1024]));
assert_eq!(json["memoryFootprintSamples"], serde_json::json!([4096]));
assert_eq!(json["activeWorkerThreadsSamples"], serde_json::json!([3]));
assert_eq!(json["concurrencySamples"], serde_json::json!([1.25]));
}
......
......@@ -961,18 +961,21 @@ mod tests {
ts: 900,
memory: 1,
memory_pressure: 0,
memory_footprint: 1,
active_worker_threads: 1,
},
TraceRow::MemorySample {
ts: 950,
memory: 1 << 20,
memory_pressure: 100,
memory_footprint: 1 << 21,
active_worker_threads: 2,
},
TraceRow::MemorySample {
ts: 1000,
memory: 1 << 30,
memory_pressure: 200,
memory_footprint: 1 << 31,
active_worker_threads: 3,
},
// Rows of other threads can be written before the `Start` of their span
......
......@@ -27,8 +27,58 @@ use crate::{
/// 10ms in microseconds
const MEMORY_SAMPLE_INTERVAL_US: u64 = 10_000;
/// 1000ms in microseconds. Reading the process memory footprint is a syscall
/// (or a `/proc` read on Linux), so it is refreshed less often than the other
/// memory sample values; samples in between repeat the last reading.
const MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US: u64 = 1_000_000;
static GLOBAL_LAST_MEMORY_SAMPLE: AtomicU64 = AtomicU64::new(0);
static MEMORY_FOOTPRINT_THROTTLE: FootprintThrottle = FootprintThrottle::new();
/// Caches the process memory footprint and refreshes it at most once per
/// [`MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US`].
struct FootprintThrottle {
/// Timestamp of the last claimed read, or [`FootprintThrottle::NEVER`].
last_read_ts: AtomicU64,
/// The most recently read value (`0` until the first read completes).
value: AtomicU64,
}
impl FootprintThrottle {
const NEVER: u64 = u64::MAX;
const fn new() -> Self {
Self {
last_read_ts: AtomicU64::new(Self::NEVER),
value: AtomicU64::new(0),
}
}
/// Returns the footprint for a sample at `ts`. Calls `read` only when no
/// read happened yet or the last one is at least
/// [`MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US`] old, and only if this caller
/// wins the claim on `last_read_ts`. All other callers (including
/// concurrent ones while a read is in progress) get the cached value.
fn get(&self, ts: u64, read: impl FnOnce() -> u64) -> u64 {
let last = self.last_read_ts.load(Ordering::Relaxed);
if last != Self::NEVER && ts.saturating_sub(last) < MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US {
return self.value.load(Ordering::Relaxed);
}
if self
.last_read_ts
.compare_exchange(last, ts, Ordering::Relaxed, Ordering::Relaxed)
.is_err()
{
// Another caller claimed this read.
return self.value.load(Ordering::Relaxed);
}
let value = read();
self.value.store(value, Ordering::Relaxed);
value
}
}
thread_local! {
static THREAD_LOCAL_LAST_MEMORY_SAMPLE: Cell<u64> = const { Cell::new(0) };
}
......@@ -223,10 +273,16 @@ impl<S: Subscriber + for<'a> LookupSpan<'a>> RawTraceLayer<S> {
THREAD_LOCAL_LAST_MEMORY_SAMPLE.with(|tl| tl.set(ts));
let memory = TurboMalloc::memory_usage() as u64;
let memory_pressure = TurboMalloc::memory_pressure().unwrap_or(0);
let memory_footprint = MEMORY_FOOTPRINT_THROTTLE.get(ts, || {
TurboMalloc::memory_footprint()
.map(|v| v as u64)
.unwrap_or(0)
});
self.write(TraceRow::MemorySample {
ts,
memory,
memory_pressure,
memory_footprint,
active_worker_threads: active_worker_threads() as u64,
});
}
......@@ -477,13 +533,21 @@ pub(crate) mod tests {
use std::{
cell::Cell,
io::{self, Write},
sync::{Arc, Mutex},
sync::{
Arc, Barrier, Mutex,
atomic::{AtomicUsize, Ordering},
},
thread,
time::Duration,
};
use tracing_subscriber::{Registry, layer::SubscriberExt};
use crate::{
raw_trace::{RawTraceLayer, RawTraceLayerOptions},
raw_trace::{
FootprintThrottle, MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US, RawTraceLayer,
RawTraceLayerOptions,
},
trace_writer::TraceWriter,
tracing::{Allocations, TraceRow},
};
......@@ -1068,4 +1132,87 @@ pub(crate) mod tests {
);
assert_eq!(elided_total, total);
}
#[test]
fn footprint_is_read_at_most_once_per_interval() {
let throttle = FootprintThrottle::new();
let reads = Cell::new(0);
let read = |value: u64| {
reads.set(reads.get() + 1);
value
};
// The very first sample reads, even at ts 0.
assert_eq!(throttle.get(0, || read(100)), 100);
assert_eq!(reads.get(), 1);
// Within the interval the cached value is reused.
assert_eq!(throttle.get(10_000, || read(200)), 100);
assert_eq!(
throttle.get(MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US - 1, || read(300)),
100
);
assert_eq!(reads.get(), 1);
// Once the interval has elapsed it reads again.
assert_eq!(
throttle.get(MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US, || read(400)),
400
);
assert_eq!(reads.get(), 2);
assert_eq!(
throttle.get(MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US + 10_000, || read(500)),
400
);
assert_eq!(reads.get(), 2);
}
#[test]
fn concurrent_caller_during_a_read_gets_cached_value() {
let throttle = FootprintThrottle::new();
assert_eq!(throttle.get(0, || 100), 100);
// While the claimed read for the next interval is in progress, another
// caller (simulated by a nested call) must not read and gets the
// previous value.
let ts = MEMORY_FOOTPRINT_SAMPLE_INTERVAL_US;
let value = throttle.get(ts, || {
let nested = throttle.get(ts + 10_000, || panic!("must not read concurrently"));
assert_eq!(nested, 100);
200
});
assert_eq!(value, 200);
assert_eq!(throttle.get(ts + 20_000, || panic!("cached")), 200);
}
#[test]
fn simultaneous_first_calls_read_once() {
const THREADS: usize = 8;
let throttle = FootprintThrottle::new();
let reads = AtomicUsize::new(0);
let barrier = Barrier::new(THREADS);
let values: Vec<u64> = thread::scope(|scope| {
let handles: Vec<_> = (0..THREADS)
.map(|_| {
scope.spawn(|| {
barrier.wait();
throttle.get(0, || {
reads.fetch_add(1, Ordering::Relaxed);
// Keep the read in progress while the others race.
thread::sleep(Duration::from_millis(20));
100
})
})
})
.collect();
handles.into_iter().map(|h| h.join().unwrap()).collect()
});
assert_eq!(reads.load(Ordering::Relaxed), 1);
// The winner gets the fresh value; losers get the (initially 0) cache.
assert!(values.iter().all(|&v| v == 0 || v == 100), "{values:?}");
assert!(values.contains(&100), "{values:?}");
assert_eq!(throttle.get(10_000, || panic!("cached")), 100);
}
}
......@@ -85,6 +85,12 @@ pub enum TraceRow<'a> {
/// `TurboMalloc::memory_pressure()`). `0` is used when the current
/// platform does not report a pressure value.
memory_pressure: u8,
/// Process memory footprint in bytes (from
/// `TurboMalloc::memory_footprint()`). `0` is used when the current
/// platform does not report a value. The footprint is refreshed at
/// most about once per second; samples in between repeat the last
/// reading.
memory_footprint: u64,
/// Number of non-parked Tokio scheduler worker threads in this process.
active_worker_threads: u64,
},
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment