Turbopack: delta-encode trace timestamps (#99809)
## Summary
Delta-encodes all timestamps in Turbopack trace files. Absolute
microsecond timestamps take 4–5 bytes as varints. The difference to the
previous row of the same thread usually fits in 1–2 bytes.
- **Format:** every row timestamp (`Start`, `End`, `Enter`, `Exit`,
`Event`, `MemorySample`) is now the difference to the previous timestamp
in the same thread local buffer. The serialized `ts` fields are
`DeltaEncodedTimestamp`, an `i64` newtype that postcard zigzag-encodes,
so a clock that goes backwards still works. `TraceRow` is generic over
its timestamp type: `TraceRow<'a>` is the serialized form, and decoding
returns `TraceRow<'a, u64>` with absolute timestamps, so the types show
whether a row still needs decoding.
- **New row:** `TimestampBase { ts }` carries an absolute timestamp. The
writer puts one before the first row with a timestamp in every buffer.
Buffers are flushed independently and interleave in the file, so each
one can be decoded on its own.
- **Large jumps:** a new base is also written when a difference doesn't
fit in an `i64`.
- **Writer:** the encoder state lives in the buffer and is reset when a
buffer is sent, stolen by the writer thread or reused. Removing the last
row (#99780) also removes its base and restores the encoder state.
- **Readers:** the trace server, `turbo-trace-size` and the tests share
`TimestampDecoder`, which keeps its state across incremental reads. A
timestamp before any base is an error. `turbo-trace-size` reports the
error with its offset.
The header is bumped to `TRACEv1` in #99781, at the top of this
format-changing stack.
Stack (on top of #99765):
1. #99777: `no-memory` flag
2. #99778: Merge allocation counters into Enter/Exit rows
3. #99779: Delta-encode allocation counters, skip unchanged ones
4. #99780: Omit Exit+Enter pairs with (almost) no gap
5. #99808: zstd flags
6. #99809: **This PR**: delta-encoded timestamps
7. #99781: Bump the trace header to `TRACEv1`
8. #99810: `upload-trace` accepts compressed traces
## Measurements
`next build` of `bench/heavy-npm-deps`, release bindings built from the
previous PR and from this one. Medians of 3 rounds that alternate
between the two builds:
| mode | before | after | change |
| --- | --- | --- | --- |
| `1` | 23.94 MB | 21.03 MB | −12.2% |
| `1,zstd` | 6.15 MB | 4.72 MB | **−23.2%** |
| `turbo-tasks` | 155.97 MB | 145.28 MB | −6.9% |
| `turbo-tasks,zstd` | 28.14 MB | 22.13 MB | **−21.4%** |
- **Row sizes:** Enter, Exit and End rows get about 2.2 bytes smaller
each.
- **Overhead:** `TimestampBase` rows add 137 B (35 rows) to the `1`
trace and 896 B (214 rows) to the `turbo-tasks` trace.
## Verification
- **Tests:**
- encoder and decoder round trips, including large jumps and incremental
reads;
- buffers that are sent, stolen or reused, and a buffer that starts with
a `Record` (its base comes before the first row with a timestamp);
- removing the first or last row of a buffer;
- exact timestamps across 2 threads and many buffers.
- **Captures:** the release-build captures are decoded fully by
`turbo-trace-size` and load completely in the trace server, raw and
zstd. Span counts per name match the previous PR.
<!-- NEXT_JS_LLM -->
<!-- fleet 4a52842b-f1a2-43cb-9779-3655caada6a0 -->
---------
Co-authored-by:
vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by:
Tobias Koppers <1365881+sokra@users.noreply.github.com>
Showing
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Please
register
or
sign in
to comment