Unverified Commit 1c981d80 authored by vercel-fleet-prod[bot]'s avatar vercel-fleet-prod[bot] Committed by GitHub

Preserve tooling trace timing and live parents

Co-authored-by: 's avatarWill Binns-Smith <755844+wbinnssmith@users.noreply.github.com>
parent 5ae588fc
...@@ -53,11 +53,13 @@ Early reporting records are retained until registration completes, with original ...@@ -53,11 +53,13 @@ Early reporting records are retained until registration completes, with original
Direct parent handles preserve reporting context after end. Numeric-ID lookup uses live/queued spans and the pinned default startup parent. If an already-ended non-default numeric parent is no longer available, the reporting lane falls back to the known tooling root (or a root if none is available); `next.tooling.parent_id` retains the original numeric relation. Local parentage is unchanged. Reporting IDs are W3C IDs, not the legacy numeric/session IDs; `next.tooling.id` permits correlation. Direct parent handles preserve reporting context after end. Numeric-ID lookup uses live/queued spans and the pinned default startup parent. If an already-ended non-default numeric parent is no longer available, the reporting lane falls back to the known tooling root (or a root if none is available); `next.tooling.parent_id` retains the original numeric relation. Local parentage is unchanged. Reporting IDs are W3C IDs, not the legacy numeric/session IDs; `next.tooling.id` permits correlation.
Only completed operations are written locally and exported by the application SDK. Some existing Webpack invalidation operations can remain unfinished even at shutdown: their children may reference a parent still open in the SDK and absent from both completed exports and the local file. Flushing does not invent an end time for those parents; fixing their underlying lifecycle is separate work.
Failures before successful compilation/registration still flush local traces once the output directory is known, but cannot report through an unavailable application provider. They do not load stale instrumentation output in the finalizer or when the current build has no instrumentation source entry, even with `cleanDistDir: false`. **Immediate process exits bypass finalizers:** in particular, the default `experimental.prerenderEarlyExit` policy exits the prerender worker and coordinator on a render error; use `prerenderEarlyExit: false` when evaluating controlled-failure flushing. Changing that existing exit policy is outside this spike. Hook failures, early replay overflow, missing hooks, abrupt termination and application sampling are explicit remote-reporting boundaries, not reasons to drop local diagnostics on the normal finalization path. Failures before successful compilation/registration still flush local traces once the output directory is known, but cannot report through an unavailable application provider. They do not load stale instrumentation output in the finalizer or when the current build has no instrumentation source entry, even with `cleanDistDir: false`. **Immediate process exits bypass finalizers:** in particular, the default `experimental.prerenderEarlyExit` policy exits the prerender worker and coordinator on a render error; use `prerenderEarlyExit: false` when evaluating controlled-failure flushing. Changing that existing exit policy is outside this spike. Hook failures, early replay overflow, missing hooks, abrupt termination and application sampling are explicit remote-reporting boundaries, not reasons to drop local diagnostics on the normal finalization path.
## Verification ## Verification
The development suites `hmr-trace-timing`, `enabled-features-trace` and `rage-restart-trace` establish the legacy file contract first; the same assertions run with the gate enabled on Webpack and Turbopack. `test/e2e/app-dir/otel-tooling` exercises a custom HTTP exporter, custom dist directory, original timing/parentage, compiler-worker export, provider identity, AlwaysOff independence, registration count and flush ordering. Focused replay tests cover storage bounds, completed/live parents, worker ordering/redelivery and hook rejection/timeout. The existing writer/parser tests remain responsible for low-level buffered I/O and rotation edges. The development suites `hmr-trace-timing`, `enabled-features-trace` and `rage-restart-trace` establish the legacy file contract first; the same assertions run with the gate enabled on Webpack and Turbopack. `test/e2e/app-dir/otel-tooling` exercises a custom HTTP exporter, custom dist directory, original timing/parentage, compiler-worker export, provider identity, AlwaysOff independence, registration count and flush ordering. Parentage checks require either a completed export or matching live-span identity observed by the application's public processor; missing completed parents still fail. Focused replay tests cover storage bounds, completed/live parents, worker ordering/redelivery and hook rejection/timeout. The existing writer/parser tests remain responsible for low-level buffered I/O and rotation edges.
This flag is for dev/build tooling, not an additional `next start` reporting mode. Production request tracing continues to use the existing instrumentation lifecycle; do not carry the prototype flag into a production deployment. Generalizing Request Insights and tooling recorders is separate work. This flag is for dev/build tooling, not an additional `next start` reporting mode. Production request tracing continues to use the existing instrumentation lifecycle; do not carry the prototype flag into a production deployment. Generalizing Request Insights and tooling recorders is separate work.
......
...@@ -147,6 +147,26 @@ describe('Trace', () => { ...@@ -147,6 +147,26 @@ describe('Trace', () => {
]) ])
}) })
it('records manualTraceChild spans with their own start time', () => {
const root = trace('root-span')
// A span that started one minute ago and ran for 1.5s, as happens when
// the event is only recorded after the work completes (e.g. Turbopack
// compilation events forwarded from Rust).
const startMs = Date.now() - 60_000
const stopMs = startMs + 1500
const msToNs = (ms: number) => BigInt(ms) * BigInt(1_000_000)
root.manualTraceChild('manual-child', msToNs(startMs), msToNs(stopMs))
root.stop()
const events = getTraceEvents()
const child = events.find((e) => e.name === 'manual-child')
expect(child).toBeDefined()
// startTime must be the span's actual start, not the time the event was
// recorded (which is when manualTraceChild is called).
expect(child!.startTime).toEqual(startMs)
expect(child!.duration).toEqual(1_500_000)
})
// Builds flush repeatedly -- after each compilation, and again at the // Builds flush repeatedly -- after each compilation, and again at the
// end -- so every flush must leave the file usable. // end -- so every flush must leave the file usable.
it('writes buffered spans on each flush', async () => { it('writes buffered spans on each flush', async () => {
......
...@@ -61,6 +61,8 @@ export class Span { ...@@ -61,6 +61,8 @@ export class Span {
name: string name: string
parentId?: SpanId parentId?: SpanId
startTime?: bigint startTime?: bigint
// Wall-clock start time in milliseconds since epoch, matching `startTime`.
// Defaults to the construction time.
wallStartTimeMs?: number wallStartTimeMs?: number
attrs?: Attributes attrs?: Attributes
reportingParent?: ReportingRecord reportingParent?: ReportingRecord
...@@ -152,6 +154,12 @@ export class Span { ...@@ -152,6 +154,12 @@ export class Span {
attrs, attrs,
reportingParent: this.reporting, reportingParent: this.reporting,
startTime: startTime ? startTime + correction : process.hrtime.bigint(), startTime: startTime ? startTime + correction : process.hrtime.bigint(),
// The span is constructed when the event is recorded, which may be long
// after the work started (e.g. compilation events forwarded from
// Turbopack), so derive the wall-clock start from the given epoch time.
wallStartTimeMs: startTime
? Number(startTime / NUM_OF_MILLISEC_IN_NANOSEC)
: undefined,
}) })
span.stop(stopTime ? stopTime + correction : process.hrtime.bigint()) span.stop(stopTime ? stopTime + correction : process.hrtime.bigint())
} }
......
...@@ -11,6 +11,11 @@ import { readFileSync } from 'node:fs' ...@@ -11,6 +11,11 @@ import { readFileSync } from 'node:fs'
let registrations = 0 let registrations = 0
let provider: TracerProvider let provider: TracerProvider
let registeredProvider: ReturnType<typeof trace.getTracerProvider> let registeredProvider: ReturnType<typeof trace.getTracerProvider>
// Observe live SDK parents without inventing an end for unfinished tooling work.
const unfinished = new Map<
string,
{ name: string; spanId: string; traceId: string; toolingId: unknown }
>()
async function send(body: unknown) { async function send(body: unknown) {
const response = await fetch(process.env.TOOLING_ENDPOINT!, { const response = await fetch(process.env.TOOLING_ENDPOINT!, {
...@@ -51,6 +56,22 @@ export async function register() { ...@@ -51,6 +56,22 @@ export async function register() {
? new AlwaysOffSampler() ? new AlwaysOffSampler()
: new AlwaysOnSampler(), : new AlwaysOnSampler(),
spanProcessors: [ spanProcessors: [
{
onStart(span) {
const spanId = span.spanContext().spanId
unfinished.set(spanId, {
name: span.name,
spanId,
traceId: span.spanContext().traceId,
toolingId: span.attributes['next.tooling.id'],
})
},
onEnd(span) {
unfinished.delete(span.spanContext().spanId)
},
async forceFlush() {},
async shutdown() {},
},
new BatchSpanProcessor({ new BatchSpanProcessor({
exporter, exporter,
maxQueueSize: 20000, maxQueueSize: 20000,
...@@ -83,6 +104,7 @@ export async function flushTraces() { ...@@ -83,6 +104,7 @@ export async function flushTraces() {
pid: process.pid, pid: process.pid,
registrations, registrations,
identityStable: trace.getTracerProvider() === registeredProvider, identityStable: trace.getTracerProvider() === registeredProvider,
unfinished: [...unfinished.values()],
localHasRoot: events.some( localHasRoot: events.some(
(event) => (event) =>
event.name === event.name ===
......
...@@ -26,6 +26,12 @@ type Message = { ...@@ -26,6 +26,12 @@ type Message = {
time?: number time?: number
localHasRoot?: boolean localHasRoot?: boolean
spans?: ExportedSpan[] spans?: ExportedSpan[]
unfinished?: {
name: string
spanId: string
traceId: string
toolingId: unknown
}[]
} }
const milliseconds = ([seconds, ns]: [number, number]) => const milliseconds = ([seconds, ns]: [number, number]) =>
seconds * 1000 + ns / 1e6 seconds * 1000 + ns / 1e6
...@@ -157,8 +163,22 @@ describe('experimental tooling OTel', () => { ...@@ -157,8 +163,22 @@ describe('experimental tooling OTel', () => {
const orphans = tooling.filter( const orphans = tooling.filter(
(span) => span.parentSpanId && !spansById.has(span.parentSpanId) (span) => span.parentSpanId && !spansById.has(span.parentSpanId)
) )
const unfinishedById = new Map(
flushes[0].unfinished?.map((span) => [span.spanId, span])
)
// A framework operation that never ends is absent from both the local
// file and completed SDK exports. Require proof that its real parent
// remains live, rather than fabricating an end or ignoring lost exports.
const missingParents = orphans.filter((span) => {
const parent = unfinishedById.get(span.parentSpanId!)
return (
!parent ||
parent.traceId !== span.traceId ||
parent.toolingId !== span.attributes['next.tooling.parent_id']
)
})
expect( expect(
orphans.slice(0, 5).map((span) => { missingParents.slice(0, 5).map((span) => {
const parent = allSpansById.get(span.parentSpanId!) const parent = allSpansById.get(span.parentSpanId!)
return { return {
name: span.name, name: span.name,
...@@ -166,9 +186,20 @@ describe('experimental tooling OTel', () => { ...@@ -166,9 +186,20 @@ describe('experimental tooling OTel', () => {
parentSpanId: span.parentSpanId, parentSpanId: span.parentSpanId,
parentName: parent?.name, parentName: parent?.name,
parentAttributeCount: Object.keys(parent?.attributes ?? {}).length, parentAttributeCount: Object.keys(parent?.attributes ?? {}).length,
parentToolingId: span.attributes['next.tooling.parent_id'],
localParentName: local.find(
(event) => event.id === span.attributes['next.tooling.parent_id']
)?.name,
unfinishedParent: flushes[0].unfinished?.find(
(unfinishedSpan) => unfinishedSpan.spanId === span.parentSpanId
),
} }
}) })
).toEqual([]) ).toEqual([])
for (const span of orphans) {
const parent = unfinishedById.get(span.parentSpanId!)!
expect(local.some((event) => event.id === parent.toolingId)).toBe(false)
}
for (const span of tooling) { for (const span of tooling) {
expect(span.traceId).toBe(root.traceId) expect(span.traceId).toBe(root.traceId)
} }
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment