turbo-trace-server: speed up loading compressed traces (#99775)
### What? Makes loading compressed (`.zst` / `.gz`) traces in `turbo-trace-server` much faster. On a large trace, compressed loads now beat raw loads from before this change. Raw loads get faster too. | 12M-span synthetic trace (warm cache, median) | before | after | |---|---|---| | raw | ~17s | ~13.5s | | `.zst` | ~26s | ~14s | | `.gz` | ~26s | ~12s | Smaller synthetic traces (2–3M spans) improve as well: - raw goes from 3.5–4.9s to 2.2–3.3s; - compressed goes from 2.2–3.7s to 2.1–2.9s. ### Why? Opening compressed traces was very slow. There were two causes. 1. **Per-batch work was quadratic.** - After every batch of rows, `Store::invalidate_outdated_spans` cleared the cached corrected self time of **every** self-time event of each touched span, plus its ancestors and the root. - For a span with millions of events, that is a full scan per batch. 2. **Compressed input arrived in tiny batches.** - zstd/gzip decoders return about one input buffer of output per `read`, roughly 450 KB. - Raw files arrived in 64 MB reads. - So compressed traces ran that full scan about 100× more often. On the trace above, invalidation alone took ~12.7s for `.zst` versus ~1.5s for raw. ### How? **Constant-time cache invalidation.** - Each span now has a generation counter for the cached corrected self times of its self-time events. - Invalidating a span just bumps the counter. A cached per-event value only counts if it was computed for the span's current generation. - The value and its generation are stored as two atomics in the event, replacing the `OnceLock`, so `SpanEvent` stays 32 bytes. `Span` grows by 8 bytes. - The counter only changes under the store's write lock. Concurrent readers can at worst both compute and store the same value. - Invalidation semantics are unchanged: the touched span, its ancestors and the root. The new store tests pass against both the old and the new implementation. **Decompression on a background thread** (`reader/threaded.rs`). - zstd and gzip streams are decoded on a dedicated thread. It fills batches and sends them to the reader over a bounded channel, so decompression overlaps parsing. - Buffers are recycled. - Errors are sent after the data that came before them, so an error the decoder reports only once is never lost. gzip checksum errors are an example. - After EOF or an error, the thread pauses until the reader asks for more data. This keeps the existing "wait for the file to grow, then retry" behavior. - The reader sees a panic on the decoder thread as an error, not as EOF. - Dropping the decoder closes the channels and joins the thread, both when the thread is paused and when it is blocked on a full channel. - For progress reporting, the reported position is the compressed-file offset the decoder thread has consumed. It never decreases and can run ahead of parsing by the queued batches. - STOP_AT polling now reads the file size from a cloned file handle instead of seeking to the end. **1 MB batches for all formats.** Once per-batch invalidation is cheap, small batches are faster: 1 MB batches parse about 1.3–1.6× faster than 64 MB ones, for raw and compressed input alike. ### Testing `cargo test -p turbopack-trace-server` passes, including new tests for: - **Store:** - cached values are reused until a span is invalidated, then recomputed; - invalidation reaches ancestors and the root; - concurrent corrected-self-time reads match serial results. - **Threaded decoder:** - gzip and zstd streams round-trip exactly; - errors arrive after the preceding data (one-shot errors and a real corrupt gzip CRC); - truncated streams deliver their readable prefix; - `Interrupted` reads are retried, and reading resumes after EOF; - a worker panic surfaces as an error; - dropping the decoder shuts the thread down while it is paused or backpressured; - the reported position is monotonic. - **Reader:** - raw, zstd and gzip versions of a trace load identically; - raw live tailing picks up appended data; - truncated compressed files load their prefix. Checked manually against the previous binary on synthetic traces: - span counts are identical for every format; - truncated and corrupt-checksum files load the same number of spans; - STOP_AT still works. Known limitations, unchanged by this PR: - live tailing of compressed traces doesn't pick up appended data; - gzip checksum errors are still treated like EOF without a warning. <!-- NEXT_JS_LLM --> <!-- fleet 7727f609-8d7f-49aa-8f67-2aefbafe3f2e --> Co-authored-by:vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
Showing
This diff is collapsed.
This diff is collapsed.
Please
register
or
sign in
to comment