Unverified Commit 0840d521 authored by Luca Forstner's avatar Luca Forstner Committed by GitHub

Add `clientTraceMetadata` experimental option to propagate tracing data to the client (#64256)

### What?

This PR adds an experimental option `clientTraceMetadata` that will use
the existing OpenTelemetry functionality to propagate conventional
OpenTelemetry trace information to the client.

The propagation metadata is propagated to the client via meta tags,
having a `name` and a `content` attribute containing the value of the
tracing value:

```html
<html>
    <head>
        <meta name="baggage" content="key1=val1,key2=val2">
        <meta name="traceparent" content="00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01">
        <meta name="custom" content="foobar">
    </head>
</html>
```

The implementation adheres to OpenTelemetry as much as possible,
treating the meta tags as if they were tracing headers on outgoing
requests. The `clientTraceMetadata` will contain the keys of the
metadata that're going to injected for tracing purpose.

### Why?

Telemetry providers usually want to provide visibility across the entire
stack, meaning it is useful for users to be able to associate, for
example, web vitals on the client, with a span tree on the server. In
order to be able to correlate tracing events from the front- and
backend, it is necessary to share something like a trace ID or similar,
that the telemetry providers can pick up and stitch back together to
create a trace.

### How?

The tracer was extended with a method `getTracePropagationData()` that
returns the propagation data on the currently active OpenTelemetry
context.
We are using `makeGetServerInsertedHTML()` to inject the meta tags into
the HTML head for dynamic requests.
The meta tags are generated through using the newly added
`getTracePropagationData()` method on the tracer.

It is important to mention that **the trace information should only be
propagated for the initial loading of the page, including hard
navigations**. Any subsequent operations should not propagate trace data
from the server to the client, as the client generally is the root of
the trace. The exception is initial pageloads, since while the request
starts on the client, no JS has had the opportunity to run yet, meaning
there is no trace propagation on the client before the server hasn't
responded.

Situations that we do not want tracing information to be propagated from
the server to the client:
- _Prefetch requests._ Prefetches generally start on the client and are
already instrumented.
- _Any sort of static precomputation, including PPR._ If we include
trace information in static pages, it means that all clients that will
land on the static page will be part of the "precomputation" trace. This
would lead to gigantic traces with a ton of unrelated data that is not
useful. The special case is dev mode where it is likely fine to
propagate trace information, even for static content, since it is
usually not actually static in dev mode.
- _Clientside (soft) navigations._ Navigations start on the client and
are usually already instrumented.

### Alternatives considered

An implementation that purely lives in user-land could have been
implemented with `useServerInsertedHTML()`, however, that implementation
would be cumbersome for users to set up, since the implementation of
tracing would have to happen in a) the instrumentation hook, b) in a
client-component that is used in a top-level layout.

### Related issues/discussions

- https://github.com/vercel/next.js/issues/47660
- https://github.com/vercel/next.js/discussions/62353 (Could be used as
an alternative to the server-timing header)
- https://github.com/getsentry/sentry-javascript/issues/9571

---------
Co-authored-by: 's avatarJiachi Liu <inbox@huozhi.im>
parent 6c3577d7
...@@ -514,6 +514,7 @@ pub struct ExperimentalConfig { ...@@ -514,6 +514,7 @@ pub struct ExperimentalConfig {
gzip_size: Option<bool>, gzip_size: Option<bool>,
instrumentation_hook: Option<bool>, instrumentation_hook: Option<bool>,
client_trace_metadata: Option<Vec<String>>,
large_page_data_bytes: Option<f64>, large_page_data_bytes: Option<f64>,
logging: Option<serde_json::Value>, logging: Option<serde_json::Value>,
memory_based_workers_count: Option<bool>, memory_based_workers_count: Option<bool>,
......
...@@ -2057,6 +2057,7 @@ export default async function getBaseWebpackConfig( ...@@ -2057,6 +2057,7 @@ export default async function getBaseWebpackConfig(
emotion: config.compiler?.emotion, emotion: config.compiler?.emotion,
modularizeImports: config.modularizeImports, modularizeImports: config.modularizeImports,
imageLoaderFile: config.images.loaderFile, imageLoaderFile: config.images.loaderFile,
clientTraceMetadata: config.experimental.clientTraceMetadata,
}) })
const cache: any = { const cache: any = {
......
...@@ -420,6 +420,7 @@ export async function exportAppImpl( ...@@ -420,6 +420,7 @@ export async function exportAppImpl(
deploymentId: nextConfig.deploymentId, deploymentId: nextConfig.deploymentId,
experimental: { experimental: {
isAppPPREnabled: checkIsAppPPREnabled(nextConfig.experimental.ppr), isAppPPREnabled: checkIsAppPPREnabled(nextConfig.experimental.ppr),
clientTraceMetadata: nextConfig.experimental.clientTraceMetadata,
swrDelta: nextConfig.experimental.swrDelta, swrDelta: nextConfig.experimental.swrDelta,
}, },
} }
......
...@@ -76,7 +76,10 @@ import { appendMutableCookies } from '../web/spec-extension/adapters/request-coo ...@@ -76,7 +76,10 @@ import { appendMutableCookies } from '../web/spec-extension/adapters/request-coo
import { createServerInsertedHTML } from './server-inserted-html' import { createServerInsertedHTML } from './server-inserted-html'
import { getRequiredScripts } from './required-scripts' import { getRequiredScripts } from './required-scripts'
import { addPathPrefix } from '../../shared/lib/router/utils/add-path-prefix' import { addPathPrefix } from '../../shared/lib/router/utils/add-path-prefix'
import { makeGetServerInsertedHTML } from './make-get-server-inserted-html' import {
getTracedMetadata,
makeGetServerInsertedHTML,
} from './make-get-server-inserted-html'
import { walkTreeWithFlightRouterState } from './walk-tree-with-flight-router-state' import { walkTreeWithFlightRouterState } from './walk-tree-with-flight-router-state'
import { createComponentTree } from './create-component-tree' import { createComponentTree } from './create-component-tree'
import { getAssetQueryString } from './get-asset-query-string' import { getAssetQueryString } from './get-asset-query-string'
...@@ -912,6 +915,11 @@ async function renderToHTMLOrFlightImpl( ...@@ -912,6 +915,11 @@ async function renderToHTMLOrFlightImpl(
tree, tree,
formState, formState,
}: RenderToStreamOptions): Promise<RenderToStreamResult> => { }: RenderToStreamOptions): Promise<RenderToStreamResult> => {
const tracingMetadata = getTracedMetadata(
getTracer().getTracePropagationData(),
renderOpts.experimental.clientTraceMetadata
)
const polyfills: JSX.IntrinsicElements['script'][] = const polyfills: JSX.IntrinsicElements['script'][] =
buildManifest.polyfillFiles buildManifest.polyfillFiles
.filter( .filter(
...@@ -995,6 +1003,7 @@ async function renderToHTMLOrFlightImpl( ...@@ -995,6 +1003,7 @@ async function renderToHTMLOrFlightImpl(
renderServerInsertedHTML, renderServerInsertedHTML,
serverCapturedErrors: allCapturedErrors, serverCapturedErrors: allCapturedErrors,
basePath: renderOpts.basePath, basePath: renderOpts.basePath,
tracingMetadata: tracingMetadata,
}) })
const renderer = createStaticRenderer({ const renderer = createStaticRenderer({
...@@ -1319,6 +1328,7 @@ async function renderToHTMLOrFlightImpl( ...@@ -1319,6 +1328,7 @@ async function renderToHTMLOrFlightImpl(
renderServerInsertedHTML, renderServerInsertedHTML,
serverCapturedErrors: [], serverCapturedErrors: [],
basePath: renderOpts.basePath, basePath: renderOpts.basePath,
tracingMetadata: tracingMetadata,
}), }),
serverInsertedHTMLToHead: true, serverInsertedHTMLToHead: true,
validateRootLayout, validateRootLayout,
......
...@@ -9,15 +9,26 @@ import { renderToReadableStream } from 'react-dom/server.edge' ...@@ -9,15 +9,26 @@ import { renderToReadableStream } from 'react-dom/server.edge'
import { streamToString } from '../stream-utils/node-web-streams-helper' import { streamToString } from '../stream-utils/node-web-streams-helper'
import { RedirectStatusCode } from '../../client/components/redirect-status-code' import { RedirectStatusCode } from '../../client/components/redirect-status-code'
import { addPathPrefix } from '../../shared/lib/router/utils/add-path-prefix' import { addPathPrefix } from '../../shared/lib/router/utils/add-path-prefix'
import type { ClientTraceDataEntry } from '../lib/trace/tracer'
export function getTracedMetadata(
traceData: ClientTraceDataEntry[],
clientTraceMetadata: string[] | undefined
): ClientTraceDataEntry[] | undefined {
if (!clientTraceMetadata) return undefined
return traceData.filter(({ key }) => clientTraceMetadata.includes(key))
}
export function makeGetServerInsertedHTML({ export function makeGetServerInsertedHTML({
polyfills, polyfills,
renderServerInsertedHTML, renderServerInsertedHTML,
serverCapturedErrors, serverCapturedErrors,
tracingMetadata,
basePath, basePath,
}: { }: {
polyfills: JSX.IntrinsicElements['script'][] polyfills: JSX.IntrinsicElements['script'][]
renderServerInsertedHTML: () => React.ReactNode renderServerInsertedHTML: () => React.ReactNode
tracingMetadata: ClientTraceDataEntry[] | undefined
serverCapturedErrors: Error[] serverCapturedErrors: Error[]
basePath: string basePath: string
}) { }) {
...@@ -82,6 +93,17 @@ export function makeGetServerInsertedHTML({ ...@@ -82,6 +93,17 @@ export function makeGetServerInsertedHTML({
}) })
} }
{serverInsertedHTML} {serverInsertedHTML}
{tracingMetadata
? tracingMetadata.map(({ key, value }) => {
return (
<meta
key={`next-trace-data-${key}:${value}`}
name={key}
content={value}
/>
)
})
: null}
{errorMetaTags} {errorMetaTags}
</>, </>,
{ {
......
...@@ -170,6 +170,7 @@ export interface RenderOptsPartial { ...@@ -170,6 +170,7 @@ export interface RenderOptsPartial {
*/ */
isRoutePPREnabled?: boolean isRoutePPREnabled?: boolean
swrDelta: SwrDelta | undefined swrDelta: SwrDelta | undefined
clientTraceMetadata: string[] | undefined
} }
postponed?: string postponed?: string
/** /**
......
...@@ -556,6 +556,7 @@ export default abstract class Server< ...@@ -556,6 +556,7 @@ export default abstract class Server<
experimental: { experimental: {
isAppPPREnabled, isAppPPREnabled,
swrDelta: this.nextConfig.experimental.swrDelta, swrDelta: this.nextConfig.experimental.swrDelta,
clientTraceMetadata: this.nextConfig.experimental.clientTraceMetadata,
}, },
} }
......
...@@ -390,6 +390,7 @@ export const configSchema: zod.ZodType<NextConfig> = z.lazy(() => ...@@ -390,6 +390,7 @@ export const configSchema: zod.ZodType<NextConfig> = z.lazy(() =>
optimizePackageImports: z.array(z.string()).optional(), optimizePackageImports: z.array(z.string()).optional(),
optimizeServerReact: z.boolean().optional(), optimizeServerReact: z.boolean().optional(),
instrumentationHook: z.boolean().optional(), instrumentationHook: z.boolean().optional(),
clientTraceMetadata: z.array(z.string()).optional(),
turbotrace: z turbotrace: z
.object({ .object({
logLevel: z logLevel: z
......
...@@ -365,6 +365,11 @@ export interface ExperimentalConfig { ...@@ -365,6 +365,11 @@ export interface ExperimentalConfig {
*/ */
instrumentationHook?: boolean instrumentationHook?: boolean
/**
* The array of the meta tags to the client injected by tracing propagation data.
*/
clientTraceMetadata?: string[]
/** /**
* Using this feature will enable the `react@experimental` for the `app` directory. * Using this feature will enable the `react@experimental` for the `app` directory.
*/ */
...@@ -919,6 +924,7 @@ export const defaultConfig: NextConfig = { ...@@ -919,6 +924,7 @@ export const defaultConfig: NextConfig = {
turbotrace: undefined, turbotrace: undefined,
typedRoutes: false, typedRoutes: false,
instrumentationHook: false, instrumentationHook: false,
clientTraceMetadata: undefined,
parallelServerCompiles: false, parallelServerCompiles: false,
parallelServerBuildTraces: false, parallelServerBuildTraces: false,
ppr: ppr:
......
import type { FetchEventResult } from '../../web/types' import type { FetchEventResult } from '../../web/types'
import type { TextMapSetter } from '@opentelemetry/api'
import type { SpanTypes } from './constants' import type { SpanTypes } from './constants'
import { LogSpanAllowList, NextVanillaSpanAllowlist } from './constants' import { LogSpanAllowList, NextVanillaSpanAllowlist } from './constants'
...@@ -149,6 +150,12 @@ interface NextTracer { ...@@ -149,6 +150,12 @@ interface NextTracer {
* Returns undefined otherwise. * Returns undefined otherwise.
*/ */
getActiveScopeSpan(): Span | undefined getActiveScopeSpan(): Span | undefined
/**
* Returns trace propagation data for the currently active context. The format is equal to data provided
* through the OpenTelemetry propagator API.
*/
getTracePropagationData(): ClientTraceDataEntry[]
} }
type NextAttributeNames = type NextAttributeNames =
...@@ -171,6 +178,20 @@ const rootSpanIdKey = api.createContextKey('next.rootSpanId') ...@@ -171,6 +178,20 @@ const rootSpanIdKey = api.createContextKey('next.rootSpanId')
let lastSpanId = 0 let lastSpanId = 0
const getSpanId = () => lastSpanId++ const getSpanId = () => lastSpanId++
export interface ClientTraceDataEntry {
key: string
value: string
}
const clientTraceDataSetter: TextMapSetter<ClientTraceDataEntry[]> = {
set(carrier, key, value) {
carrier.push({
key,
value,
})
},
}
class NextTracerImpl implements NextTracer { class NextTracerImpl implements NextTracer {
/** /**
* Returns an instance to the trace with configured name. * Returns an instance to the trace with configured name.
...@@ -185,6 +206,13 @@ class NextTracerImpl implements NextTracer { ...@@ -185,6 +206,13 @@ class NextTracerImpl implements NextTracer {
return context return context
} }
public getTracePropagationData(): ClientTraceDataEntry[] {
const activeContext = context.active()
const entries: ClientTraceDataEntry[] = []
propagation.inject(activeContext, entries, clientTraceDataSetter)
return entries
}
public getActiveScopeSpan(): Span | undefined { public getActiveScopeSpan(): Span | undefined {
return trace.getSpan(context?.active()) return trace.getSpan(context?.active())
} }
......
# files generated by next.js
node_modules/
export const dynamic = 'force-dynamic'
export default function DynamicPage() {
return <h1 id="dynamic-page-header">Dynamic Page</h1>
}
export default function Root({ children }: { children: React.ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
export default function StaticPage() {
return <h1 id="static-page-2-header">Static Page 2</h1>
}
import Link from 'next/link'
export default function StaticPage() {
return (
<>
<h1 id="static-page-header">Static Page</h1>
<Link href="/dynamic-page" id="go-to-dynamic-page">
Go to dynamic page
</Link>
<Link href="/static-page-2" id="go-to-static-page">
Go to static page
</Link>
</>
)
}
import { nextTestSetup } from 'e2e-utils'
describe('clientTraceMetadata', () => {
const { next, isNextDev } = nextTestSetup({
files: __dirname,
dependencies: require('./package.json').dependencies,
})
it('should inject propagation data for a dynamically server-side-rendered page', async () => {
const $ = await next.render$('/dynamic-page')
const headHtml = $.html('head')
expect(headHtml).toContain(
'<meta name="my-test-key-1" content="my-test-value-1">'
)
expect(headHtml).toContain(
'<meta name="my-test-key-2" content="my-test-value-2">'
)
expect(headHtml).toMatch(
/<meta name="my-parent-span-id" content="[a-f0-9]{16}">/
)
expect(headHtml).not.toContain('non-metadata-key-3')
})
it('hard loading a dynamic page twice should yield different dynamic trace data', async () => {
const browser1 = await next.browser('/dynamic-page')
const firstLoadSpanIdContent = await browser1
.elementByCss('meta[name="my-parent-span-id"]')
.getAttribute('content')
const browser2 = await next.browser('/dynamic-page')
const secondLoadSpanIdContent = await browser2
.elementByCss('meta[name="my-parent-span-id"]')
.getAttribute('content')
expect(firstLoadSpanIdContent).toMatch(/[a-f0-9]{16}/)
expect(secondLoadSpanIdContent).toMatch(/[a-f0-9]{16}/)
expect(firstLoadSpanIdContent).not.toBe(secondLoadSpanIdContent)
})
if (isNextDev) {
describe('next dev only', () => {
it('should inject propagation data for a statically server-side-rendered page', async () => {
const $ = await next.render$('/static-page')
const headHtml = $.html('head')
expect(headHtml).toContain(
'<meta name="my-test-key-1" content="my-test-value-1">'
)
expect(headHtml).toContain(
'<meta name="my-test-key-2" content="my-test-value-2">'
)
expect(headHtml).toMatch(
/<meta name="my-parent-span-id" content="[a-f0-9]{16}">/
)
expect(headHtml).not.toContain('non-metadata-key-3')
})
it('soft navigating to a dynamic page should not transform previous propagation data', async () => {
const browser = await next.browser('/static-page')
const initialSpanIdTagContent = await browser
.elementByCss('meta[name="my-parent-span-id"]')
.getAttribute('content')
// We are in dev mode so the static page should contain propagation data
expect(initialSpanIdTagContent).toMatch(/[a-f0-9]{16}/)
await browser.elementByCss('#go-to-dynamic-page').click()
await browser.elementByCss('#dynamic-page-header')
const updatedSpanIdTagContent = await browser
.elementByCss('meta[name="my-parent-span-id"]')
.getAttribute('content')
expect(initialSpanIdTagContent).toBe(updatedSpanIdTagContent)
})
it('soft navigating to a static page should not transform previous propagation data', async () => {
const browser = await next.browser('/static-page')
const initialSpanIdTagContent = await browser
.elementByCss('meta[name="my-parent-span-id"]')
.getAttribute('content')
// We are in dev mode so the static page should contain propagation data
expect(initialSpanIdTagContent).toMatch(/[a-f0-9]{16}/)
await browser.elementByCss('#go-to-static-page').click()
await browser.elementByCss('#static-page-2-header')
const updatedSpanIdTagContent = await browser
.elementByCss('meta[name="my-parent-span-id"]')
.getAttribute('content')
expect(initialSpanIdTagContent).toBe(updatedSpanIdTagContent)
})
})
} else {
describe('next start only', () => {
it('should inject propagation data for a statically server-side-rendered page', async () => {
const $ = await next.render$('/static-page')
const headHtml = $.html('head')
expect(headHtml).not.toContain(
'<meta name="my-test-key-1" content="my-test-value-1">'
)
expect($.html('head')).not.toContain(
'<meta name="my-test-key-2" content="my-test-value-2">'
)
expect($.html('head')).not.toMatch(
/<meta name="my-parent-span-id" content="[a-f0-9]{16}">/
)
})
it('soft navigating to a dynamic page should not transform previous propagation data', async () => {
const browser = await next.browser('/static-page')
await browser.elementByCss('#static-page-header')
const initialSpanIdTag = await browser.eval(
'document.querySelector(\'meta[name="my-parent-span-id"]\')'
)
// We are in prod mode so we are not expecting propagation data to be present for a static page
expect(initialSpanIdTag).toBeNull()
await browser.elementByCss('#go-to-dynamic-page').click()
await browser.elementByCss('#dynamic-page-header')
const updatedSpanIdTag = await browser.eval(
'document.querySelector(\'meta[name="my-parent-span-id"]\')'
)
// After the navigation to the dynamic page, there should still be no meta tag with propagation data
expect(updatedSpanIdTag).toBeNull()
})
it('soft navigating to a static page should not transform previous propagation data', async () => {
const browser = await next.browser('/static-page')
await browser.elementByCss('#static-page-header')
const initialSpanIdTag = await browser.eval(
'document.querySelector(\'meta[name="my-parent-span-id"]\')'
)
// We are in prod mode so we are not expecting propagation data to be present for a static page
expect(initialSpanIdTag).toBeNull()
await browser.elementByCss('#go-to-static-page').click()
await browser.elementByCss('#static-page-2-header')
const updatedSpanIdTag = await browser.eval(
'document.querySelector(\'meta[name="my-parent-span-id"]\')'
)
// After the navigation to the dynamic page, there should still be no meta tag with propagation data
expect(updatedSpanIdTag).toBeNull()
})
})
}
})
import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node'
import { trace } from '@opentelemetry/api'
export async function register() {
if (process.env.NEXT_RUNTIME === 'nodejs') {
const provider = new NodeTracerProvider()
provider.register({
propagator: {
inject(context, carrier, setter) {
setter.set(carrier, 'my-test-key-1', 'my-test-value-1')
setter.set(carrier, 'my-test-key-2', 'my-test-value-2')
// This non-metadata-key-3 is not going to be injected into the page
setter.set(carrier, 'non-metadata-key-3', 'non-metadata-key-3')
setter.set(
carrier,
'my-parent-span-id',
trace.getSpanContext(context).spanId
)
},
extract(context) {
// This is a noop because we don't extract in this test
return context
},
fields() {
return [
'my-parent-span-id',
'my-test-key-1',
'my-test-key-2',
'non-metadata-key-3',
]
},
},
})
}
}
module.exports = {
experimental: {
instrumentationHook: true,
clientTraceMetadata: [
'my-parent-span-id',
'my-test-key-1',
'my-test-key-2',
],
},
}
{
"dependencies": {
"@opentelemetry/api": "^1.8.0",
"@opentelemetry/sdk-trace-node": "^1.21.0"
}
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment