Unverified Commit 2e559938 authored by Tobias Koppers's avatar Tobias Koppers Committed by GitHub

Keep requests through middleware in a single OpenTelemetry trace (#100136)

### What?

Requests that pass through middleware / `proxy.ts` now produce **one
OpenTelemetry trace**, instead of two or three.

Before, with OTEL configured in `instrumentation.ts`:

| Request | No outer span | With an outer span (APM, custom server) |
|---|---|---|
| Middleware continues to render (`next()` / rewrite) | 3 traces (2
before 16.3) | 2 traces (1 before 16.3) |
| Middleware redirect / direct response | 2 traces (1 before 16.3) | 2
traces (1 before 16.3) |

After: 1 trace in every case.

### Why?

Two independent causes:

1. **Long-standing (13.5 → canary), #91282.** When middleware lets a
request continue, the router server runs `BaseServer.handleRequest`
twice: a middleware-invoke pass, then a render pass. Each pass extracts
its context from the incoming request headers. Without an incoming
`traceparent`, each pass started its own root trace.
2. **16.3 regression from #95306.** In-process Node.js middleware ran
under `ROOT_CONTEXT` to make `Middleware.execute` a sibling of
`handleRequest`. That also detached it from any outer span, so the
middleware span always started an extra trace.

Customers upgrading 16.2 → 16.3 see this as "doubled traces".

### How?

When middleware lets a request continue, the render is always **nested
inside the middleware span**, in every host:

```
next start / next dev:
 handleRequest GET (bubble, root)
  └─ middleware GET
      └─ GET /page

separately-invoked middleware (Vercel, adapters, direct entrypoints), no parent:
 middleware GET (root)
  └─ GET /page

host span / incoming traceparent P:
 P
  └─ middleware GET            (next start: P → handleRequest GET → middleware GET)
      └─ GET /page
```

**Middleware → render.** The middleware adapter forwards the
`Middleware.execute` span's own W3C context (`traceparent` /
`tracestate`, via the configured propagator) as request-header overrides
on `next()` and rewrite responses. Every host already applies these
overrides to the downstream request, including the `next start` router
server, so no new host protocol is needed.
- Override lists replace the whole downstream header set. So:
- **No user overrides:** the adapter lists every request header the
middleware received, plus the trace headers.
- **User overrides present:** they are kept, and the trace headers are
added.
- **User-set trace headers:** a `traceparent` / `tracestate` the
middleware explicitly set (different from the one it received) wins. A
value copied unchanged from the request, as with `new
Headers(request.headers)`, is replaced so the render still nests.
- Redirects and direct responses are untouched.

**Request → middleware (self-hosted).** Once the middleware-invoke
pass's `handleRequest` span is active, its W3C context is written onto
the request headers. The middleware adapter (Node.js and the Edge
sandbox) extracts it, so the middleware span is a child of that pass.
The middleware-pass span keeps its existing name (`GET`); the route
stays on the render-pass span. If there is no middleware span to forward
(for example OTEL is registered only for Node.js while middleware runs
on Edge), the render falls back to this context and still stays in one
trace.

**#95306 regression.** The `ROOT_CONTEXT` detachment is removed, so an
outer span, an incoming `traceparent`, or the middleware-pass span now
parents Node.js middleware normally.

All of this only runs when OpenTelemetry is active and a valid span
context exists. Without OTEL, no headers are added.

One-time initialization traces stay separate by design: instrumentation
registration, route-module prewarming, and the once-per-process lazy
`RouteModule.prepare` of direct entrypoints.

### Tests

- **New deploy-capable suite
`test/e2e/app-dir/otel-middleware-single-trace`.** It runs Node.js
`proxy.ts` and Edge `middleware.ts` with `@vercel/otel`. Middleware
reports its active trace and span IDs; the render reports its active
trace ID and the `traceparent` it received. The test asserts one trace
and that the render's parent is the middleware span, so no collector is
needed. It runs in dev, start and deploy mode, with no deploy gate. It
covers:
- continue to a page and to a route handler, plus rewrite, redirect and
a direct response;
- a request the middleware doesn't match, and an incoming `traceparent`;
- user header overrides, including a deletion (the render still nests),
and a user-set `traceparent`;
  - no internal `x-middleware-*` headers reaching the client.
- **Existing collector suite `test/e2e/opentelemetry/instrumentation`.**
The middleware tests now assert **real** trace IDs and the exact shape:
the render span's parent is the middleware span, the middleware span's
parent is the middleware-pass `handleRequest` (self-hosted) or the
external / host parent. They also check every span exported for the
request.
- **Setups:** default and direct-entrypoint hosting; Edge and Node.js
middleware; root context and incoming `traceparent`.
  - **Requests:** page, route handler, rewrite and redirect.
  - **New variant:** direct entrypoints with a host parent span.
- **Unit tests** for the override helper
(`forward-trace-context.test.ts`), including unchanged copied trace
headers.

### Verification

- The new suite's nesting checks and the collector shape checks fail
before this change (render parented to `handleRequest` or the external
span) and pass after it.
- New suite passes in dev (Turbopack), start (Turbopack, webpack), and
the same with `__NEXT_CACHE_COMPONENTS=true`.
- Full `test/e2e/opentelemetry/instrumentation/` passes in start mode
(Turbopack); the middleware tests also pass in dev (Turbopack) and start
(webpack).
- Passed on the previous revision (not re-run for the nesting change):
`test/e2e/opentelemetry/client-trace-metadata`,
`test/e2e/app-dir/otel-parent-span-propagation`,
`test/e2e/middleware-request-header-overrides`, and the W3C trace-header
fetch-cache test in `test/e2e/app-dir/app-static`.
- Deploy: an earlier revision's trace-ID assertions passed on Vercel
(default and Cache Components). The new parent-span assertion runs on
Vercel for the first time in this revision's CI.

Fixes #91282

<!-- NEXT_JS_LLM -->


<!-- fleet ce0ecb23-b14c-4679-92a9-e676130f618e -->

---------
Co-authored-by: 's avatarvercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: 's avatarTobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: 's avatarLuke Sandberg <210140+lukesandberg@users.noreply.github.com>
parent a5fe506d
...@@ -908,8 +908,26 @@ export default abstract class Server< ...@@ -908,8 +908,26 @@ export default abstract class Server<
'http.target': req.url, 'http.target': req.url,
}, },
}, },
async (span) => async (span) => {
this.handleRequestImpl(req, res, parsedUrl).finally(() => { if (getRequestMeta(req, 'middlewareInvoke')) {
// When middleware lets the request continue, the router server
// calls `handleRequest` twice: once to run middleware (this pass)
// and once to render. Publish this span's W3C trace context on
// the request, so the middleware span (including the Edge
// sandbox, which extracts it from the request headers) is a
// child of this span instead of starting a new trace. The
// middleware forwards its own span's context to the render pass
// via request-header overrides, which replace this value; it
// only stays when there is no middleware span to forward (e.g.
// OpenTelemetry isn't registered in the Edge runtime), keeping
// the render in this trace.
const traceContext = tracer.getTraceContextHeaders()
if (traceContext) {
Object.assign(req.headers, traceContext)
}
}
return this.handleRequestImpl(req, res, parsedUrl).finally(() => {
if (!span) return if (!span) return
const isRSCRequest = getRequestMeta(req, 'isRSCRequest') ?? false const isRSCRequest = getRequestMeta(req, 'isRSCRequest') ?? false
...@@ -968,6 +986,7 @@ export default abstract class Server< ...@@ -968,6 +986,7 @@ export default abstract class Server<
span.updateName(isRSCRequest ? `RSC ${method}` : `${method}`) span.updateName(isRSCRequest ? `RSC ${method}` : `${method}`)
} }
}) })
}
) )
}) })
......
...@@ -178,6 +178,14 @@ interface NextTracer { ...@@ -178,6 +178,14 @@ interface NextTracer {
*/ */
getTracePropagationData(): ClientTraceDataEntry[] getTracePropagationData(): ClientTraceDataEntry[]
/**
* Returns the W3C trace context (`traceparent` / `tracestate`) of the
* currently active context, as produced by the configured propagator.
* Returns `undefined` when OpenTelemetry is not enabled, no span context is
* active, or the propagator doesn't produce a `traceparent`.
*/
getTraceContextHeaders(): TraceContextHeaders | undefined
/** /**
* Executes a function with the given span set as the active span in the context. * Executes a function with the given span set as the active span in the context.
* This allows child spans created within the function to automatically parent to this span. * This allows child spans created within the function to automatically parent to this span.
...@@ -211,6 +219,11 @@ export interface ClientTraceDataEntry { ...@@ -211,6 +219,11 @@ export interface ClientTraceDataEntry {
value: string value: string
} }
export interface TraceContextHeaders {
traceparent: string
tracestate?: string
}
const clientTraceDataSetter: TextMapSetter<ClientTraceDataEntry[]> = { const clientTraceDataSetter: TextMapSetter<ClientTraceDataEntry[]> = {
set(carrier, key, value) { set(carrier, key, value) {
carrier.push({ carrier.push({
...@@ -260,6 +273,25 @@ class NextTracerImpl implements NextTracer { ...@@ -260,6 +273,25 @@ class NextTracerImpl implements NextTracer {
return entries return entries
} }
public getTraceContextHeaders(): TraceContextHeaders | undefined {
const activeContext = context.active()
if (
!trace.getSpanContext(activeContext) ||
!this.isOpenTelemetryEnabled()
) {
return undefined
}
const carrier: Record<string, string> = {}
propagation.inject(activeContext, carrier)
const { traceparent, tracestate } = carrier
if (!traceparent) {
return undefined
}
return tracestate ? { traceparent, tracestate } : { traceparent }
}
public getActiveScopeSpan(): Span | undefined { public getActiveScopeSpan(): Span | undefined {
const localSpanRecorder = getLocalSpanRecorder() const localSpanRecorder = getLocalSpanRecorder()
const activeLocalSpan = localSpanRecorder?.getActiveLocalSpan() const activeLocalSpan = localSpanRecorder?.getActiveLocalSpan()
...@@ -277,19 +309,6 @@ class NextTracerImpl implements NextTracer { ...@@ -277,19 +309,6 @@ class NextTracerImpl implements NextTracer {
return activeLocalSpan return activeLocalSpan
} }
/**
* Run `fn` with the active span cleared, so spans created inside become new
* roots (or parent to incoming propagated context). Used for in-process
* Node.js middleware so its span is a sibling of the request span, matching
* edge middleware which runs in a detached sandbox.
*/
public runWithDetachedContext<T>(fn: () => T): T {
if (!NEXT_OTEL_PERFORMANCE_PREFIX && !this.isOpenTelemetryEnabled()) {
return fn()
}
return context.with(ROOT_CONTEXT, fn)
}
public withPropagatedContext<T, C>( public withPropagatedContext<T, C>(
carrier: C, carrier: C,
fn: () => T, fn: () => T,
......
...@@ -1866,24 +1866,21 @@ export default class NextNodeServer extends BaseServer< ...@@ -1866,24 +1866,21 @@ export default class NextNodeServer extends BaseServer<
try { try {
// Node.js middleware runs in-process, inside the active // Node.js middleware runs in-process, inside the active
// `handleRequest` span. Detach that span so the middleware span // `handleRequest` span, so the middleware span (and the trace context
// becomes a sibling root (or parents to an incoming traceparent), // it forwards to the render) stays in the request's trace.
// matching edge middleware which runs in a detached sandbox. result = await adapterFn({
result = await getTracer().runWithDetachedContext(() => handler:
adapterFn({ middlewareModule.proxy ||
handler: middlewareModule.middleware ||
middlewareModule.proxy || middlewareModule,
middlewareModule.middleware || request: {
middlewareModule, ...requestData,
request: { body: hasRequestBody
...requestData, ? requestData.body.cloneBodyStream()
body: hasRequestBody : undefined,
? requestData.body.cloneBodyStream() },
: undefined, page: 'middleware',
}, })
page: 'middleware',
})
)
} finally { } finally {
if (hasRequestBody) { if (hasRequestBody) {
await requestData.body.finalize() await requestData.body.finalize()
......
...@@ -30,10 +30,11 @@ import { workAsyncStorage } from '../app-render/work-async-storage.external' ...@@ -30,10 +30,11 @@ import { workAsyncStorage } from '../app-render/work-async-storage.external'
import { InvariantError } from '../../shared/lib/invariant-error' import { InvariantError } from '../../shared/lib/invariant-error'
import type { ResolvedCacheLifeProfiles } from '../config-shared' import type { ResolvedCacheLifeProfiles } from '../config-shared'
import { NEXT_ROUTER_PREFETCH_HEADER } from '../../client/components/app-router-headers' import { NEXT_ROUTER_PREFETCH_HEADER } from '../../client/components/app-router-headers'
import { getTracer } from '../lib/trace/tracer' import { getTracer, type TraceContextHeaders } from '../lib/trace/tracer'
import type { TextMapGetter } from 'next/dist/compiled/@opentelemetry/api' import type { TextMapGetter } from 'next/dist/compiled/@opentelemetry/api'
import { MiddlewareSpan } from '../lib/trace/constants' import { MiddlewareSpan } from '../lib/trace/constants'
import { CloseController } from './web-on-close' import { CloseController } from './web-on-close'
import { forwardTraceContextToRequest } from './forward-trace-context'
import { getEdgePreviewProps } from './get-edge-preview-props' import { getEdgePreviewProps } from './get-edge-preview-props'
import { getBuiltinRequestContext } from '../after/builtin-request-context' import { getBuiltinRequestContext } from '../after/builtin-request-context'
import { getImplicitTags } from '../lib/implicit-tags' import { getImplicitTags } from '../lib/implicit-tags'
...@@ -266,6 +267,11 @@ export async function adapter( ...@@ -266,6 +267,11 @@ export async function adapter(
}) })
let response let response
let cookiesFromResponse let cookiesFromResponse
// The middleware span's trace context to forward to the downstream request
// (see `forwardTraceContextToRequest`), and the request headers as the
// middleware received them.
let forwardedTraceContext: TraceContextHeaders | undefined
let forwardedRequestHeaders: Headers | undefined
response = await propagator(request, () => { response = await propagator(request, () => {
// we only care to make async storage available for middleware // we only care to make async storage available for middleware
...@@ -293,6 +299,16 @@ export async function adapter( ...@@ -293,6 +299,16 @@ export async function adapter(
}, },
}, },
async () => { async () => {
// Forward the middleware span's own context, so the render is always
// nested inside the middleware span, whatever the middleware span is
// parented to (an incoming `traceparent`, a host span, next-server's
// `handleRequest` span, or nothing).
forwardedTraceContext = getTracer().getTraceContextHeaders()
if (forwardedTraceContext) {
// Snapshot before the handler can mutate `request.headers`.
forwardedRequestHeaders = new Headers(request.headers)
}
try { try {
const onUpdateCookies = (cookies: Array<string>) => { const onUpdateCookies = (cookies: Array<string>) => {
cookiesFromResponse = cookies cookiesFromResponse = cookies
...@@ -520,6 +536,18 @@ export async function adapter( ...@@ -520,6 +536,18 @@ export async function adapter(
const finalResponse = response ? response : NextResponse.next() const finalResponse = response ? response : NextResponse.next()
if (forwardedRequestHeaders) {
// Nest the downstream request (render) inside the middleware span. Every
// host applies these request-header overrides to the render, including
// next-server's router server, so this works whether or not the host
// invokes the middleware separately from the render.
forwardTraceContextToRequest(
finalResponse.headers,
forwardedRequestHeaders,
forwardedTraceContext
)
}
// Flight headers are not overridable / removable so they are applied at the end. // Flight headers are not overridable / removable so they are applied at the end.
const middlewareOverrideHeaders = finalResponse.headers.get( const middlewareOverrideHeaders = finalResponse.headers.get(
'x-middleware-override-headers' 'x-middleware-override-headers'
......
import { forwardTraceContextToRequest } from './forward-trace-context'
import { NextResponse } from './spec-extension/response'
const TRACEPARENT = '00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01'
const TRACESTATE = 'vendor=value'
function requestOverrides(headers: Headers) {
const list = headers.get('x-middleware-override-headers')
if (list === null) return undefined
return Object.fromEntries(
list
.split(',')
.filter(Boolean)
.map((key) => [key, headers.get(`x-middleware-request-${key}`)])
)
}
describe('forwardTraceContextToRequest', () => {
const requestHeaders = () =>
new Headers({ accept: 'text/html', cookie: 'a=b', 'x-custom': 'custom' })
it('lists every request header plus the trace context without user overrides', () => {
const response = NextResponse.next()
forwardTraceContextToRequest(response.headers, requestHeaders(), {
traceparent: TRACEPARENT,
tracestate: TRACESTATE,
})
expect(requestOverrides(response.headers)).toEqual({
accept: 'text/html',
cookie: 'a=b',
'x-custom': 'custom',
traceparent: TRACEPARENT,
tracestate: TRACESTATE,
})
})
it('replaces a traceparent the middleware received when there are no user overrides', () => {
const headers = requestHeaders()
headers.set(
'traceparent',
'00-11111111111111111111111111111111-2222222222222222-01'
)
const response = NextResponse.next()
forwardTraceContextToRequest(response.headers, headers, {
traceparent: TRACEPARENT,
})
const overrides = requestOverrides(response.headers)!
expect(overrides.traceparent).toBe(TRACEPARENT)
expect(
response.headers
.get('x-middleware-override-headers')!
.split(',')
.filter((key) => key === 'traceparent')
).toHaveLength(1)
})
it('works for rewrites', () => {
const response = NextResponse.rewrite(new URL('https://example.com/other'))
forwardTraceContextToRequest(response.headers, requestHeaders(), {
traceparent: TRACEPARENT,
})
expect(requestOverrides(response.headers)?.traceparent).toBe(TRACEPARENT)
})
it('only appends the trace context to existing user overrides', () => {
const userHeaders = new Headers({ 'x-user': 'user' })
const response = NextResponse.next({ request: { headers: userHeaders } })
forwardTraceContextToRequest(response.headers, requestHeaders(), {
traceparent: TRACEPARENT,
tracestate: TRACESTATE,
})
expect(requestOverrides(response.headers)).toEqual({
'x-user': 'user',
traceparent: TRACEPARENT,
tracestate: TRACESTATE,
})
})
it('keeps a traceparent explicitly set by the user', () => {
const userTraceparent =
'00-33333333333333333333333333333333-4444444444444444-01'
const response = NextResponse.next({
request: { headers: new Headers({ traceparent: userTraceparent }) },
})
forwardTraceContextToRequest(response.headers, requestHeaders(), {
traceparent: TRACEPARENT,
tracestate: TRACESTATE,
})
expect(requestOverrides(response.headers)).toEqual({
traceparent: userTraceparent,
tracestate: TRACESTATE,
})
})
it('replaces trace context headers copied unchanged into user overrides', () => {
const receivedTraceparent =
'00-11111111111111111111111111111111-2222222222222222-01'
const received = requestHeaders()
received.set('traceparent', receivedTraceparent)
received.set('tracestate', 'received=1')
// The common `new Headers(request.headers)` pattern copies the received
// trace context into the overrides without choosing it.
const userHeaders = new Headers(received)
userHeaders.set('x-user', 'user')
const response = NextResponse.next({ request: { headers: userHeaders } })
forwardTraceContextToRequest(response.headers, received, {
traceparent: TRACEPARENT,
tracestate: TRACESTATE,
})
const overrides = requestOverrides(response.headers)!
expect(overrides.traceparent).toBe(TRACEPARENT)
expect(overrides.tracestate).toBe(TRACESTATE)
expect(overrides['x-user']).toBe('user')
expect(
response.headers
.get('x-middleware-override-headers')!
.split(',')
.filter((key) => key === 'traceparent' || key === 'tracestate')
).toEqual(['traceparent', 'tracestate'])
})
it('treats an empty user override list like no overrides', () => {
const response = NextResponse.next({ request: { headers: new Headers() } })
expect(response.headers.get('x-middleware-override-headers')).toBe('')
forwardTraceContextToRequest(response.headers, requestHeaders(), {
traceparent: TRACEPARENT,
})
expect(requestOverrides(response.headers)).toEqual({
accept: 'text/html',
cookie: 'a=b',
'x-custom': 'custom',
traceparent: TRACEPARENT,
})
})
it('does nothing without a trace context', () => {
const response = NextResponse.next()
forwardTraceContextToRequest(response.headers, requestHeaders(), undefined)
expect(requestOverrides(response.headers)).toBeUndefined()
expect([...response.headers.keys()]).toEqual(['x-middleware-next'])
})
it.each([
['a redirect', () => NextResponse.redirect('https://example.com/')],
['a direct response', () => new NextResponse('hello')],
])('does nothing for %s', (_, createResponse) => {
const response = createResponse()
const before = [...response.headers.entries()]
forwardTraceContextToRequest(response.headers, requestHeaders(), {
traceparent: TRACEPARENT,
})
expect([...response.headers.entries()]).toEqual(before)
})
})
import type { TraceContextHeaders } from '../lib/trace/tracer'
const OVERRIDE_HEADERS = 'x-middleware-override-headers'
const REQUEST_HEADER_PREFIX = 'x-middleware-request-'
/**
* Forwards the middleware's W3C trace context to the downstream request by
* adding `traceparent` / `tracestate` request-header overrides to a middleware
* response that continues the request (`NextResponse.next()` or a rewrite).
*
* Every host applies these overrides to the downstream request: hosts that
* invoke middleware separately from the render (deployments, adapters, direct
* entrypoints) as well as next-server's router server. Forwarding the
* middleware span's context therefore nests the render inside the middleware
* span instead of starting a new trace.
*
* Request-header overrides replace the whole downstream request header set,
* so:
* - Without user overrides (or with an empty list, which hosts ignore), every
* request header the middleware received is listed (unchanged) alongside
* the trace context.
* - With user overrides, they are kept and only the trace context headers are
* added. A trace context header the user explicitly set (to a value other
* than the one the middleware received) wins. One copied unchanged from the
* request (e.g. via `new Headers(request.headers)`) is replaced, as it is no
* explicit choice.
*/
export function forwardTraceContextToRequest(
responseHeaders: Headers,
requestHeaders: Headers,
traceContext: TraceContextHeaders | undefined
): void {
if (!traceContext) {
return
}
// Redirects and direct responses have no downstream request.
if (
!responseHeaders.has('x-middleware-next') &&
!responseHeaders.has('x-middleware-rewrite')
) {
return
}
const traceHeaders: Array<[string, string]> = [
['traceparent', traceContext.traceparent],
]
if (traceContext.tracestate) {
traceHeaders.push(['tracestate', traceContext.tracestate])
}
const userOverrides = responseHeaders.get(OVERRIDE_HEADERS)
// An empty override list is ignored by hosts (no headers are replaced), so
// treat it like no overrides rather than turning it into "only the trace
// context headers".
if (!userOverrides) {
const keys: string[] = []
for (const [key, value] of requestHeaders) {
responseHeaders.set(REQUEST_HEADER_PREFIX + key, value)
keys.push(key)
}
for (const [key, value] of traceHeaders) {
responseHeaders.set(REQUEST_HEADER_PREFIX + key, value)
if (!keys.includes(key)) {
keys.push(key)
}
}
responseHeaders.set(OVERRIDE_HEADERS, keys.join(','))
return
}
const keys = userOverrides
.split(',')
.map((key) => key.trim())
.filter(Boolean)
for (const [key, value] of traceHeaders) {
if (keys.includes(key)) {
if (
responseHeaders.get(REQUEST_HEADER_PREFIX + key) !==
requestHeaders.get(key)
) {
continue
}
} else {
keys.push(key)
}
responseHeaders.set(REQUEST_HEADER_PREFIX + key, value)
}
responseHeaders.set(OVERRIDE_HEADERS, keys.join(','))
}
import { trace } from '@opentelemetry/api'
import { headers } from 'next/headers'
import { Suspense } from 'react'
async function RequestInfo() {
const requestHeaders = Object.fromEntries((await headers()).entries())
const traceId = trace.getActiveSpan()?.spanContext().traceId ?? 'none'
return (
<>
<p id="trace-id">{traceId}</p>
<pre id="headers">{JSON.stringify(requestHeaders)}</pre>
</>
)
}
export default function Page() {
return (
<Suspense>
<RequestInfo />
</Suspense>
)
}
import type { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
export default function Home() {
return <p>otel-middleware-single-trace</p>
}
import { trace } from '@opentelemetry/api'
import { headers } from 'next/headers'
import { Suspense } from 'react'
async function TraceInfo() {
// Reading request headers also makes this run at request time, so the
// reported trace belongs to the request.
const traceparent = (await headers()).get('traceparent') ?? 'none'
const traceId = trace.getActiveSpan()?.spanContext().traceId ?? 'none'
return (
<>
<p id="trace-id">{traceId}</p>
{/* The trace context the render was started from (its parent span). */}
<p id="traceparent">{traceparent}</p>
</>
)
}
export default function Page() {
return (
<Suspense>
<TraceInfo />
</Suspense>
)
}
import { trace } from '@opentelemetry/api'
import { connection } from 'next/server'
export async function GET(request: Request) {
// Run at request time so the reported trace belongs to the request.
await connection()
const traceId = trace.getActiveSpan()?.spanContext().traceId ?? 'none'
// The trace context the render was started from (its parent span).
const traceparent = request.headers.get('traceparent') ?? 'none'
return Response.json({ traceId, traceparent })
}
import { trace } from '@opentelemetry/api'
import { connection } from 'next/server'
import { Suspense } from 'react'
async function TraceId() {
// Run at request time so the reported trace belongs to the request.
await connection()
const traceId = trace.getActiveSpan()?.spanContext().traceId ?? 'none'
return <p id="trace-id">{traceId}</p>
}
// Not matched by the middleware: a baseline for requests without middleware.
export default function Page() {
return (
<Suspense>
<TraceId />
</Suspense>
)
}
import { registerOTel } from '@vercel/otel'
export function register() {
// No exporter is needed: the app itself reports the active trace IDs, so the
// suite also works when deployed (where no local collector is reachable).
registerOTel({ serviceName: 'otel-middleware-single-trace' })
}
import { trace } from '@opentelemetry/api'
import { NextResponse, type NextRequest } from 'next/server'
export const USER_TRACE_ID = '0af7651916cd43dd8448eb211c80319c'
export const USER_SPAN_ID = 'b7ad6b7169203331'
// Shared by `proxy.ts` (Node.js runtime) and `middleware.ts` (Edge runtime).
export function handler(request: NextRequest) {
const spanContext = trace.getActiveSpan()?.spanContext()
const traceId = spanContext?.traceId ?? 'none'
const spanId = spanContext?.spanId ?? 'none'
const { pathname } = request.nextUrl
let response: NextResponse
if (pathname === '/rewrite') {
response = NextResponse.rewrite(new URL('/page', request.url))
} else if (pathname === '/redirect') {
response = NextResponse.redirect(new URL('/page', request.url))
} else if (pathname === '/direct') {
response = new NextResponse('direct response from middleware')
} else if (pathname === '/user-overrides') {
const headers = new Headers(request.headers)
headers.set('x-user-added', 'added-by-middleware')
headers.delete('x-delete-me')
response = NextResponse.rewrite(new URL('/headers-echo', request.url), {
request: { headers },
})
} else if (pathname === '/user-traceparent') {
const headers = new Headers(request.headers)
headers.set('traceparent', `00-${USER_TRACE_ID}-${USER_SPAN_ID}-01`)
response = NextResponse.rewrite(new URL('/headers-echo', request.url), {
request: { headers },
})
} else {
response = NextResponse.next()
}
response.headers.set('x-mw-trace-id', traceId)
response.headers.set('x-mw-span-id', spanId)
return response
}
import type { NextConfig } from 'next'
const nextConfig: NextConfig = {}
export default nextConfig
import { isNextDeploy, nextTestSetup } from 'e2e-utils'
import cheerio from 'cheerio'
const USER_TRACE_ID = '0af7651916cd43dd8448eb211c80319c'
const INCOMING_TRACE_ID = '4bf92f3577b34da6a3ce929d0e0e4736'
const INCOMING_TRACEPARENT = `00-${INCOMING_TRACE_ID}-00f067aa0ba902b7-01`
const TRACE_ID = /^(?!0{32}$)[0-9a-f]{32}$/
const SPAN_ID = /^(?!0{16}$)[0-9a-f]{16}$/
const MATCHER = [
'/page',
'/route-handler',
'/rewrite',
'/redirect',
'/direct',
'/user-overrides',
'/user-traceparent',
]
// The middleware entry file is generated per variant: `config` must be a
// static literal in the entry file itself, and the shared implementation is
// imported relative to the app root, where the entry file is written.
function middlewareEntry(exportName: 'proxy' | 'middleware') {
return [
`import type { NextRequest } from 'next/server'`,
`import { handler } from './middleware-impl'`,
``,
`export function ${exportName}(request: NextRequest) {`,
` return handler(request)`,
`}`,
``,
`export const config = { matcher: ${JSON.stringify(MATCHER)} }`,
``,
].join('\n')
}
// The app reports its active trace IDs itself (no local collector), so these
// assertions also hold when deployed. Every request that passes through
// middleware must stay in a single trace: the trace ID seen by the middleware
// has to be the one seen by the page / route handler that renders afterwards.
// The render must also be nested inside the middleware span: the trace context
// it was started from (the `traceparent` it received) is the middleware span.
describe.each([
// `proxy.ts` always runs in the Node.js runtime.
{ name: 'proxy (nodejs runtime)', file: 'proxy.ts', exportName: 'proxy' },
// `middleware.ts` defaults to the edge runtime.
{
name: 'middleware (edge runtime)',
file: 'middleware.ts',
exportName: 'middleware',
},
] as const)('otel-middleware-single-trace - $name', ({ file, exportName }) => {
const { next } = nextTestSetup({
files: __dirname,
overrideFiles: {
[file]: middlewareEntry(exportName),
},
dependencies: {
'@vercel/otel': '2.1.3',
'@opentelemetry/api': '1.9.1',
'@opentelemetry/api-logs': '0.223.0',
'@opentelemetry/instrumentation': '0.223.0',
'@opentelemetry/resources': '2.12.0',
'@opentelemetry/sdk-logs': '0.223.0',
'@opentelemetry/sdk-metrics': '2.12.0',
'@opentelemetry/sdk-trace-base': '2.12.0',
},
})
function getMiddlewareTraceId(res: Response) {
const traceId = res.headers.get('x-mw-trace-id')
expect(traceId).toMatch(TRACE_ID)
return traceId
}
// Asserts the render is in the middleware's trace and a direct child of the
// middleware span.
function expectNestedInMiddleware(
res: Response,
render: { traceId: string; traceparent: string }
) {
const traceId = getMiddlewareTraceId(res)
const spanId = res.headers.get('x-mw-span-id')
expect(spanId).toMatch(SPAN_ID)
expect(render.traceId).toBe(traceId)
const [, parentTraceId, parentSpanId] = render.traceparent.split('-')
expect({ parentTraceId, parentSpanId }).toEqual({
parentTraceId: traceId,
parentSpanId: spanId,
})
}
async function getPageTrace(res: Response) {
const $ = cheerio.load(await res.text())
return {
traceId: $('#trace-id').text(),
traceparent: $('#traceparent').text(),
}
}
function expectNoInternalHeaders(res: Response) {
for (const key of res.headers.keys()) {
expect(key).not.toMatch(/^x-middleware-(request-|override-headers)/)
}
}
it('nests a page rendered after NextResponse.next() in the middleware span', async () => {
const res = await next.fetch('/page')
expect(res.status).toBe(200)
expectNestedInMiddleware(res, await getPageTrace(res))
expectNoInternalHeaders(res)
})
it('nests a route handler executed after NextResponse.next() in the middleware span', async () => {
const res = await next.fetch('/route-handler')
expect(res.status).toBe(200)
expectNestedInMiddleware(res, await res.json())
expectNoInternalHeaders(res)
})
it('nests a rewritten page in the middleware span', async () => {
const res = await next.fetch('/rewrite')
expect(res.status).toBe(200)
expectNestedInMiddleware(res, await getPageTrace(res))
expectNoInternalHeaders(res)
})
it('traces a middleware redirect', async () => {
const res = await next.fetch('/redirect', { redirect: 'manual' })
expect(res.status).toBe(307)
getMiddlewareTraceId(res)
expectNoInternalHeaders(res)
})
it('traces a direct middleware response', async () => {
const res = await next.fetch('/direct')
expect(res.status).toBe(200)
expect(await res.text()).toBe('direct response from middleware')
getMiddlewareTraceId(res)
expectNoInternalHeaders(res)
})
it('traces a request the middleware does not match', async () => {
const res = await next.fetch('/unmatched')
expect(res.status).toBe(200)
expect(res.headers.get('x-mw-trace-id')).toBeNull()
const $ = cheerio.load(await res.text())
expect($('#trace-id').text()).toMatch(TRACE_ID)
expectNoInternalHeaders(res)
})
it.each(['/page', '/route-handler', '/rewrite'])(
'nests %s in the middleware span with an incoming traceparent',
async (pathname) => {
const res = await next.fetch(pathname, {
headers: { traceparent: INCOMING_TRACEPARENT },
})
expect(res.status).toBe(200)
const render =
pathname === '/route-handler'
? await res.json()
: await getPageTrace(res)
expectNestedInMiddleware(res, render)
// Self-hosted Next.js continues the caller's trace. When deployed, the
// platform decides whether a client-supplied traceparent is trusted, so
// only the single-trace and nesting guarantees are asserted there.
if (!isNextDeploy) {
expect(render.traceId).toBe(INCOMING_TRACE_ID)
}
expectNoInternalHeaders(res)
}
)
it('still applies request header overrides set by middleware', async () => {
const res = await next.fetch('/user-overrides', {
headers: { 'x-delete-me': 'please', 'x-keep-me': 'kept' },
})
expect(res.status).toBe(200)
const $ = cheerio.load(await res.text())
const headers = JSON.parse($('#headers').text())
expect(headers['x-user-added']).toBe('added-by-middleware')
expect(headers['x-keep-me']).toBe('kept')
expect(headers['x-delete-me']).toBeUndefined()
expectNestedInMiddleware(res, {
traceId: $('#trace-id').text(),
traceparent: headers['traceparent'],
})
expectNoInternalHeaders(res)
})
it('respects a traceparent explicitly set by middleware', async () => {
const res = await next.fetch('/user-traceparent')
expect(res.status).toBe(200)
const $ = cheerio.load(await res.text())
expect($('#trace-id').text()).toBe(USER_TRACE_ID)
expectNoInternalHeaders(res)
})
})
export async function GET() {
return new Response(JSON.stringify({ behind: 'middleware' }))
}
export const dynamic = 'force-dynamic'
...@@ -3,6 +3,12 @@ import { SavedSpan } from './constants' ...@@ -3,6 +3,12 @@ import { SavedSpan } from './constants'
export interface Collector { export interface Collector {
getSpans: () => SavedSpan[] getSpans: () => SavedSpan[]
/**
* Like `getSpans`, but also includes spans marked `next.bubble` (e.g.
* next-server's middleware-invoke `handleRequest` span), which are needed to
* check the full span hierarchy of a request.
*/
getAllSpans: () => SavedSpan[]
reset: () => void reset: () => void
shutdown: () => Promise<void> shutdown: () => Promise<void>
} }
...@@ -13,6 +19,7 @@ export async function connectCollector({ ...@@ -13,6 +19,7 @@ export async function connectCollector({
port: number port: number
}): Promise<Collector> { }): Promise<Collector> {
const spans: SavedSpan[] = [] const spans: SavedSpan[] = []
const allSpans: SavedSpan[] = []
const server = new HttpServer(async (req, res) => { const server = new HttpServer(async (req, res) => {
if (req.method !== 'POST') { if (req.method !== 'POST') {
...@@ -33,6 +40,7 @@ export async function connectCollector({ ...@@ -33,6 +40,7 @@ export async function connectCollector({
}) })
const newSpans = JSON.parse(body.toString('utf-8')) as SavedSpan[] const newSpans = JSON.parse(body.toString('utf-8')) as SavedSpan[]
allSpans.push(...newSpans)
const filteredSpans = newSpans.filter((span) => { const filteredSpans = newSpans.filter((span) => {
if (span.attributes?.['next.bubble'] === true) { if (span.attributes?.['next.bubble'] === true) {
return false return false
...@@ -61,8 +69,12 @@ export async function connectCollector({ ...@@ -61,8 +69,12 @@ export async function connectCollector({
getSpans() { getSpans() {
return spans return spans
}, },
getAllSpans() {
return allSpans
},
reset() { reset() {
spans.length = 0 spans.length = 0
allSpans.length = 0
}, },
shutdown() { shutdown() {
return new Promise<void>((resolve, reject) => return new Promise<void>((resolve, reject) =>
......
...@@ -63,6 +63,10 @@ async function main() { ...@@ -63,6 +63,10 @@ async function main() {
'app/app/[param]/rsc-fetch/error/page.js', 'app/app/[param]/rsc-fetch/error/page.js',
], ],
[/^\/behind-middleware$/, 'app/behind-middleware/page.js'], [/^\/behind-middleware$/, 'app/behind-middleware/page.js'],
[
/^\/behind-middleware\/route-handler$/,
'app/behind-middleware/route-handler/route.js',
],
// --- // ---
[/^\/api\/pages\/param\/basic$/, 'pages/api/pages/[param]/basic.js'], [/^\/api\/pages\/param\/basic$/, 'pages/api/pages/[param]/basic.js'],
[/^\/api\/pages\/param\/error$/, 'pages/api/pages/[param]/error.js'], [/^\/api\/pages\/param\/error$/, 'pages/api/pages/[param]/error.js'],
......
...@@ -10,5 +10,15 @@ export async function middleware( ...@@ -10,5 +10,15 @@ export async function middleware(
request: NextRequest, request: NextRequest,
event?: NextFetchEvent event?: NextFetchEvent
): Promise<Response> { ): Promise<Response> {
const { pathname } = request.nextUrl
if (pathname === '/behind-middleware/rewrite') {
// Keep the query string so tests can correlate the rewritten render.
const url = request.nextUrl.clone()
url.pathname = '/behind-middleware'
return NextResponse.rewrite(url)
}
if (pathname === '/behind-middleware/redirect') {
return NextResponse.redirect(new URL('/behind-middleware', request.url))
}
return NextResponse.next() return NextResponse.next()
} }
...@@ -9,5 +9,15 @@ export async function middleware( ...@@ -9,5 +9,15 @@ export async function middleware(
request: NextRequest, request: NextRequest,
event?: NextFetchEvent event?: NextFetchEvent
): Promise<Response> { ): Promise<Response> {
const { pathname } = request.nextUrl
if (pathname === '/behind-middleware/rewrite') {
// Keep the query string so tests can correlate the rewritten render.
const url = request.nextUrl.clone()
url.pathname = '/behind-middleware'
return NextResponse.rewrite(url)
}
if (pathname === '/behind-middleware/redirect') {
return NextResponse.redirect(new URL('/behind-middleware', request.url))
}
return NextResponse.next() return NextResponse.next()
} }
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment