Unverified Commit a831acd9 authored by dan's avatar dan Committed by GitHub

Build the inline Flight script from React's strings instead of re-encoding them (#100142)

React's Flight server on Node writes two kinds of chunks:

- Short rows are UTF-8-encoded into a 4 KB buffer and written as bytes.
- Strings longer than about 1365 characters are [written as JS
strings](https://github.com/facebook/react/blob/b618bbb4/packages/react-server/src/ReactServerStreamConfigNode.js#L61-L72),
leaving the encoding to the consumer.

A row is one line, `id:<json>`, and React [builds it as a single
string](https://github.com/facebook/react/blob/b618bbb4/packages/react-server/src/ReactFlightServer.js#L4793-L4795).
So a row with 1365+ characters of JSON is written as a string too. Pages
with a lot of markup and few Suspense boundaries are mostly such rows.

## Before

`renderToNodeFlightStream` piped React into `new PassThrough()`, a
byte-mode stream. Node encodes every string written to a byte-mode
stream, so each of those strings became a Buffer on the way in. Both
consumers then decoded it: `createFromNodeStream` for SSR, and the
inline-data writer, which did this for each chunk to build the
`__next_f` script:

```js
const decodedString = chunk.toString('utf-8')
htmlInlinedData = htmlEscapeJsonString(JSON.stringify([1, decodedString]))
output.push(Buffer.from(`<script>self.__next_f.push(${htmlInlinedData})</script>`))
```

## After

The stream is object mode, so the strings reach the consumers as
strings. The inline writer escapes them straight into the script. SSR
reads them directly, which React's reader
[supports](https://github.com/facebook/react/blob/b618bbb4/packages/react-client/src/ReactFlightClient.js#L5500-L5508).
Prerender capture, byte counting, Web stream conversion and the HTTP
response still need bytes and encode for themselves.

## Numbers

`sandbox-bench`: 16 fresh VMs per experiment, each running both arms
twice, interleaved. Cells are the 95% range across VMs; bold is p <
0.01.

Each commit against the one before it. `/streaming/heavy` has 240
boundaries, each with a 3 KB string prop and a 1.5 KB text node.
`/streaming/heavy-unicode` (added in this PR) is the same with one
non-Latin-1 character in each string, which makes V8 store them at two
bytes per character.

| Commit | `heavy` median | `heavy-unicode` median | `heavy-unicode`
throughput, 8 clients |
| --- | --- | --- | --- |
| Switch the stream to object mode | no change | −4% to 0% | no change |
| Escape inline strings without decoding | −7% to 0% | **−15% to −11%**
| **+17% to +26%** |
| Encode fragments into one Buffer | −15% to 0% | **−20% to −6%** |
**+8% to +38%** |
| All three | **−16% to −6%** | **−28% to −19%** | **+21% to +42%** |

The inline writer can only skip the decode once it receives strings,
which is what the first commit provides. On its own that commit also
removes the SSR decode and adds the `toWellFormed()` scan, and those
roughly cancel. The next two commits remove the decode and re-encode in
the inline writer.

The whole PR by route. The app routes were measured twice, on new VMs.

| Route | Median latency, 1 client | Throughput, 8 clients |
| --- | --- | --- |
| `/streaming/heavy` | **−16% to −6%** | **+19% to +38%** |
| `/streaming/heavy-unicode` | **−28% to −19%** | **+21% to +42%** |
| `/blog` | **−14% to −7%** | **+11% to +24%** |
| `/blog`, repeated | **−15% to −5%** | **+8% to +23%** |
| `/docs` | **−15% to −9%** | **+8% to +44%** |
| `/docs`, repeated | −14% to −2% | +1% to +20% |
| `/dashboard` | −11% to −1% | **+6% to +22%** |
| `/dashboard`, repeated | no change | **+9% to +32%** |

TTFB moves with the median. `/streaming/chunkstorm` (1 KB strings) and
`/streaming/bulk` (384-byte rows) have no strings over the threshold and
did not move in any commit. `heavy-unicode` checks whether
`toWellFormed()`, which scans two-byte strings, has a measurable cost.
The two-byte route improves more than the one-byte route, because
encoding and decoding two-byte strings is more expensive to begin with,
so whatever the scan costs is small compared to what the other two
commits remove. In CPU profiles, `utf8Write` (the encode) is down 48%,
`Buffer#toString` (the writer's decode) down 94%, `TextDecoder` (the SSR
decode) about 65%. Memory is not recorded by this harness.

<details>
<summary>All cells</summary>

Change with 95% range across the 16 VMs of each experiment. Bold is p <
0.01; "no change" means the range includes zero.

| Route | Median latency, 1 client | TTFB, 1 client | Median latency, 8
clients | Throughput, 8 clients |
| --- | --- | --- | --- | --- |
| `/streaming/heavy` | **−11.0% (−16.3 to −5.7)** | **−15.4% (−19.7 to
−11.1)** | **−17.0% (−25.7 to −8.3)** | **+28.4% (+18.8 to +38.0)** |
| `/streaming/heavy-unicode` | **−23.3% (−27.7 to −18.9)** | **−27.8%
(−32.4 to −23.2)** | **−20.6% (−27.5 to −13.7)** | **+31.4% (+20.6 to
+42.2)** |
| `/blog` | **−10.3% (−13.5 to −7.1)** | **−12.7% (−15.8 to −9.6)** |
**−13.3% (−18.5 to −8.1)** | **+17.8% (+11.2 to +24.4)** |
| `/blog`, repeated | **−10.0% (−14.8 to −5.2)** | **−11.6% (−17.0 to
−6.2)** | −10.7% (−19.2 to −2.2) | **+15.5% (+7.7 to +23.3)** |
| `/docs` | **−11.9% (−15.2 to −8.6)** | **−13.9% (−17.2 to −10.6)** |
no change (−25.2 to +0.4) | **+26.0% (+8.2 to +43.8)** |
| `/docs`, repeated | −8.2% (−14.4 to −2.0) | **−10.3% (−16.7 to −3.9)**
| no change (−14.4 to +7.2) | +10.6% (+1.3 to +19.9) |
| `/dashboard` | −5.5% (−10.5 to −0.5) | −6.9% (−13.5 to −0.3) | −9.3%
(−16.5 to −2.1) | **+13.9% (+6.1 to +21.7)** |
| `/dashboard`, repeated | no change (−8.4 to +2.8) | no change (−12.0
to +3.0) | **−12.2% (−19.7 to −4.7)** | **+20.3% (+8.8 to +31.8)** |
| `/streaming/chunkstorm` | no change (−7.4 to +1.4) | no change (−8.2
to +0.8) | no change (−13.1 to +7.5) | no change (−2.8 to +20.8) |
| `/streaming/bulk` | no change (−6.2 to +2.2) | no change (−6.3 to
+2.3) | no change (−8.0 to +10.2) | no change (−5.9 to +13.5) |

</details>

## Output is unchanged

- **HTML bytes.** Identical between arms on every bench route. A unit
test checks the inline script bytes for mixed string and byte input,
including UTF-8 split across chunks and `</script` split across strings.
- **Lone surrogates.** A JS string can contain half of a surrogate pair.
UTF-8 cannot represent it, so encoding replaces it with U+FFFD. Before,
everything was encoded, so SSR, the inline script and the RSC response
all showed U+FFFD. Now SSR would see the original character while the
client still sees U+FFFD, a hydration mismatch. So the stream runs each
string through `toWellFormed()`, which replaces lone surrogates with
U+FFFD the same way. The e2e test checks all three agree and that
hydration logs no mismatch.
- **`res.write`.** Custom servers still receive Buffers; the response
pipe converts strings before writing. An e2e with a custom server checks
this in dev and production, with and without cache components.

<details>
<summary>Verification</summary>

- Unit: `test/unit/inline-flight-data.test.ts`. Exact inline script
bytes for mixed string and byte input at three sizes; UTF-8 split across
adjacent byte chunks; byte views with a nonzero offset; `</script` split
across string chunks.
- E2E `test/e2e/app-dir/flight-native-chunks`: `test-start-turbo`,
`test-dev-turbo`, `test-start-webpack`.
- E2E `test/e2e/app-dir/custom-server-response-chunks`: same three, plus
start and dev with the cache-components env. Rebuilding without the
response-boundary conversion makes its RSC case fail with `string` and
`uint8array` chunks, so the test does see Next's writes. Compression is
off in the fixture; with it on, the server only sees gzip's output.
- `test/e2e/app-dir/rsc-basic` under `test-start-turbo`.
- `pnpm --filter=next types`, prettier, eslint.

Bench cells: `pr100142-stress` (streaming routes; base is the
bench-route commit, cand is head), `pr100142-realistic` and
`pr100142-realistic-confirm` (`/blog,/dashboard,/docs`, same arms),
`pr100142-buffer` (last commit against its parent, streaming routes).

</details>
parent f58c038c
...@@ -7,14 +7,24 @@ function sleep(ms) { ...@@ -7,14 +7,24 @@ function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms)) return new Promise((resolve) => setTimeout(resolve, ms))
} }
function createPayload(title, payloadBytes) { function createPayload(title, payloadBytes, payloadMarker) {
const prefix = `${title}:` const prefix = `${title}:${payloadMarker}`
if (prefix.length >= payloadBytes) return prefix if (prefix.length >= payloadBytes) return prefix
return `${prefix}${'x'.repeat(payloadBytes - prefix.length)}` return `${prefix}${'x'.repeat(payloadBytes - prefix.length)}`
} }
function createClientPayload({ title, id, payloadBytes, fragmentCount }) { function createClientPayload({
const payload = createPayload(`${title}-client-${id}`, payloadBytes) title,
id,
payloadBytes,
payloadMarker,
fragmentCount,
}) {
const payload = createPayload(
`${title}-client-${id}`,
payloadBytes,
payloadMarker
)
const safeFragmentCount = Math.max(1, fragmentCount) const safeFragmentCount = Math.max(1, fragmentCount)
const fragmentSize = Math.max( const fragmentSize = Math.max(
16, 16,
...@@ -39,6 +49,7 @@ async function StreamedChunk({ ...@@ -39,6 +49,7 @@ async function StreamedChunk({
id, id,
delayMs, delayMs,
payload, payload,
payloadMarker,
clientPayloadBytes, clientPayloadBytes,
clientPayloadFragments, clientPayloadFragments,
}) { }) {
...@@ -48,6 +59,7 @@ async function StreamedChunk({ ...@@ -48,6 +59,7 @@ async function StreamedChunk({
title, title,
id, id,
payloadBytes: clientPayloadBytes, payloadBytes: clientPayloadBytes,
payloadMarker,
fragmentCount: clientPayloadFragments, fragmentCount: clientPayloadFragments,
}) })
...@@ -64,11 +76,15 @@ export function StreamingStressPage({ ...@@ -64,11 +76,15 @@ export function StreamingStressPage({
title, title,
boundaryCount, boundaryCount,
payloadBytes, payloadBytes,
// Prepended to every payload. A single character outside Latin-1 makes V8
// store the whole string in two-byte form, which is the common shape of
// real text (an em dash or emoji in otherwise ASCII content).
payloadMarker = '',
clientPayloadBytes = Math.max(128, Math.floor(payloadBytes / 2)), clientPayloadBytes = Math.max(128, Math.floor(payloadBytes / 2)),
clientPayloadFragments = 4, clientPayloadFragments = 4,
maxDelayMs, maxDelayMs,
}) { }) {
const payload = createPayload(title, payloadBytes) const payload = createPayload(title, payloadBytes, payloadMarker)
const boundaries = Array.from({ length: boundaryCount }, (_, index) => index) const boundaries = Array.from({ length: boundaryCount }, (_, index) => index)
return ( return (
...@@ -94,6 +110,7 @@ export function StreamingStressPage({ ...@@ -94,6 +110,7 @@ export function StreamingStressPage({
id={id} id={id}
delayMs={delayMs} delayMs={delayMs}
payload={payload} payload={payload}
payloadMarker={payloadMarker}
clientPayloadBytes={clientPayloadBytes} clientPayloadBytes={clientPayloadBytes}
clientPayloadFragments={clientPayloadFragments} clientPayloadFragments={clientPayloadFragments}
/> />
......
import React from 'react'
import { StreamingStressPage } from '../_shared/stress-page'
export const dynamic = 'force-dynamic'
// Same shape as /streaming/heavy with two-byte payload strings.
export default function Page() {
return (
<StreamingStressPage
title="stream-heavy-unicode"
boundaryCount={240}
payloadBytes={1536}
payloadMarker="中"
clientPayloadBytes={3072}
clientPayloadFragments={6}
maxDelayMs={8}
/>
)
}
import type { Readable } from 'node:stream' import type { Readable } from 'node:stream'
import { InvariantError } from '../../shared/lib/invariant-error' import { InvariantError } from '../../shared/lib/invariant-error'
import {
encodeTextChunk,
nodeToWebByteStream,
} from '../stream-utils/node-web-streams-helper'
export type AnyStream = ReadableStream<Uint8Array> | Readable export type AnyStream = ReadableStream<Uint8Array> | Readable
...@@ -55,9 +59,7 @@ export class ReactServerResult { ...@@ -55,9 +59,7 @@ export class ReactServerResult {
__non_webpack_require__('node:stream') as typeof import('node:stream') __non_webpack_require__('node:stream') as typeof import('node:stream')
).Readable ).Readable
} }
const webStream = Readable.toWeb( const webStream = nodeToWebByteStream(this._stream)
this._stream
) as ReadableStream<Uint8Array>
const tee = webStream.tee() const tee = webStream.tee()
this._stream = Readable.fromWeb( this._stream = Readable.fromWeb(
tee[0] as import('stream/web').ReadableStream tee[0] as import('stream/web').ReadableStream
...@@ -86,35 +88,36 @@ export class ReactServerResult { ...@@ -86,35 +88,36 @@ export class ReactServerResult {
} }
type ReplayableStreamSubscriber = { type ReplayableStreamSubscriber = {
onChunk: (chunk: Uint8Array) => void onChunk: (chunk: string | Uint8Array) => void
onEnd: () => void onEnd: () => void
onError: (err: Error) => void onError: (err: Error) => void
} }
/** /**
* Buffers all chunks from a Node.js Readable stream and allows creating new * Buffers byte or string chunks from a Node.js Readable stream and allows creating new
* Readable streams that replay the buffered chunks plus any subsequent chunks * Readable streams that replay the buffered chunks plus any subsequent chunks
* from the source. Multiple replay streams can be created independently. * from the source. Multiple replay streams can be created independently.
*/ */
export class ReplayableNodeStream { export class ReplayableNodeStream {
private _chunks: Array<Uint8Array> | null private _chunks: Array<string | Uint8Array> | null
private _done: boolean private _done: boolean
private _error: Error | null private _error: Error | null
private _subscribers: Set<ReplayableStreamSubscriber> private _subscribers: Set<ReplayableStreamSubscriber>
private _objectMode: boolean
constructor(stream: Readable) { constructor(stream: Readable) {
this._chunks = [] this._chunks = []
this._done = false this._done = false
this._error = null this._error = null
this._subscribers = new Set() this._subscribers = new Set()
this._objectMode = stream.readableObjectMode
stream.on('data', (chunk: Buffer | Uint8Array) => { stream.on('data', (chunk: string | Uint8Array) => {
const buf = chunk instanceof Uint8Array ? chunk : new Uint8Array(chunk)
if (this._chunks !== null) { if (this._chunks !== null) {
this._chunks.push(buf) this._chunks.push(chunk)
} }
for (const sub of this._subscribers) { for (const sub of this._subscribers) {
sub.onChunk(buf) sub.onChunk(chunk)
} }
}) })
...@@ -180,6 +183,9 @@ export class ReplayableNodeStream { ...@@ -180,6 +183,9 @@ export class ReplayableNodeStream {
const sourceError = this._error const sourceError = this._error
const stream = new ReadableCtor({ const stream = new ReadableCtor({
// Replay the source representation, including the original boundaries
// that the Node Flight reader uses to frame text rows.
objectMode: this._objectMode,
read() { read() {
if (!bufferDrained) { if (!bufferDrained) {
bufferDrained = true bufferDrained = true
...@@ -269,7 +275,7 @@ export async function createReactServerPrerenderResultFromRender( ...@@ -269,7 +275,7 @@ export async function createReactServerPrerenderResultFromRender(
} }
} else { } else {
for await (const chunk of underlying) { for await (const chunk of underlying) {
chunks.push(chunk instanceof Uint8Array ? chunk : new Uint8Array(chunk)) chunks.push(encodeTextChunk(chunk))
} }
} }
......
...@@ -74,6 +74,7 @@ import { ...@@ -74,6 +74,7 @@ import {
createNodeInlinedDataStream, createNodeInlinedDataStream,
} from './stream-ops' } from './stream-ops'
import type { AnyStream } from './stream-ops' import type { AnyStream } from './stream-ops'
import { encodeTextChunk } from '../stream-utils/node-web-streams-helper'
import { createRenderInBrowserAbortSignal } from './render-in-browser' import { createRenderInBrowserAbortSignal } from './render-in-browser'
import { getInstantTestBootstrapScriptContent } from './instant-test-bootstrap' import { getInstantTestBootstrapScriptContent } from './instant-test-bootstrap'
import { createTurbopackBootstrapScriptStream } from './turbopack-bootstrap-script' import { createTurbopackBootstrapScriptStream } from './turbopack-bootstrap-script'
...@@ -6326,7 +6327,11 @@ async function accumulateStreamChunksInto( ...@@ -6326,7 +6327,11 @@ async function accumulateStreamChunksInto(
try { try {
for await (const value of nodeStream) { for await (const value of nodeStream) {
if (cancelled) break if (cancelled) break
collectStageChunk(accumulator, stageController.currentStage, value) collectStageChunk(
accumulator,
stageController.currentStage,
encodeTextChunk(value)
)
} }
} catch (err) { } catch (err) {
if (!cancelled) { if (!cancelled) {
...@@ -6408,7 +6413,7 @@ async function countStageBytesUntilAbortNode( ...@@ -6408,7 +6413,7 @@ async function countStageBytesUntilAbortNode(
increaseChunkByteLengths( increaseChunkByteLengths(
byteLengths, byteLengths,
stageController.currentStage, stageController.currentStage,
(value as Uint8Array).byteLength typeof value === 'string' ? Buffer.byteLength(value) : value.byteLength
) )
} }
} catch (err) { } catch (err) {
...@@ -11004,7 +11009,7 @@ async function iterateStreamingPrerenderChunks( ...@@ -11004,7 +11009,7 @@ async function iterateStreamingPrerenderChunks(
try { try {
for await (const value of nodeStream) { for await (const value of nodeStream) {
if (cancelled) break if (cancelled) break
onChunk(value) onChunk(encodeTextChunk(value))
} }
} catch (err) { } catch (err) {
if (!cancelled) { if (!cancelled) {
......
import type { AppPageModule } from '../route-modules/app-page/module' import type { AppPageModule } from '../route-modules/app-page/module'
import { renderToNodeFlightStream } from './stream-ops' import { renderToNodeFlightStream } from './stream-ops'
import { getClientReferenceManifest } from './manifests-singleton' import { getClientReferenceManifest } from './manifests-singleton'
import { encodeTextChunk } from '../stream-utils/node-web-streams-helper'
const filterStackFrame = const filterStackFrame =
process.env.NODE_ENV !== 'production' process.env.NODE_ENV !== 'production'
...@@ -30,7 +31,7 @@ export async function serializeValidationErrorsToFlight( ...@@ -30,7 +31,7 @@ export async function serializeValidationErrorsToFlight(
const chunks: Uint8Array[] = [] const chunks: Uint8Array[] = []
for await (const chunk of stream) { for await (const chunk of stream) {
chunks.push(chunk) chunks.push(encodeTextChunk(chunk))
} }
return chunks return chunks
} }
...@@ -50,6 +50,7 @@ import { ...@@ -50,6 +50,7 @@ import {
} from './stream-utils' } from './stream-utils'
import type { DebugChannelPair } from '../debug-channel-server' import type { DebugChannelPair } from '../debug-channel-server'
import type { FlightComponentMod } from '../stream-ops' import type { FlightComponentMod } from '../stream-ops'
import { encodeTextChunk } from '../../stream-utils/node-web-streams-helper'
// eslint-disable-next-line import/no-extraneous-dependencies // eslint-disable-next-line import/no-extraneous-dependencies
import { createFromNodeStream } from 'react-server-dom-webpack/client' import { createFromNodeStream } from 'react-server-dom-webpack/client'
...@@ -385,7 +386,12 @@ async function collectSegmentDataForStage( ...@@ -385,7 +386,12 @@ async function collectSegmentDataForStage(
if (controller.currentStage === RenderStage.Before) { if (controller.currentStage === RenderStage.Before) {
throw new InvariantError('Unexpected chunk emitted in Before stage') throw new InvariantError('Unexpected chunk emitted in Before stage')
} }
writeChunk(stageEntry, controller.currentStage, targetStage, chunk) writeChunk(
stageEntry,
controller.currentStage,
targetStage,
encodeTextChunk(chunk)
)
} }
})(), })(),
// accumulate Debug chunks // accumulate Debug chunks
...@@ -623,9 +629,10 @@ export async function createCombinedPayloadStream( ...@@ -623,9 +629,10 @@ export async function createCombinedPayloadStream(
// Accumulate Flight chunks // Accumulate Flight chunks
(async () => { (async () => {
for await (const chunk of stream) { for await (const chunk of stream) {
allChunks.push(chunk) const bytes = encodeTextChunk(chunk)
allChunks.push(bytes)
if (isRenderable) { if (isRenderable) {
renderableChunks.push(chunk) renderableChunks.push(bytes)
} }
} }
})(), })(),
......
...@@ -27,6 +27,7 @@ import { ...@@ -27,6 +27,7 @@ import {
createDocumentClosingStream as webCreateDocumentClosingStream, createDocumentClosingStream as webCreateDocumentClosingStream,
createRuntimePrefetchTransformStream, createRuntimePrefetchTransformStream,
CLOSE_TAG, CLOSE_TAG,
nodeToWebByteStream,
} from '../stream-utils/node-web-streams-helper' } from '../stream-utils/node-web-streams-helper'
import { indexOfUint8Array } from '../stream-utils/uint8array-helpers' import { indexOfUint8Array } from '../stream-utils/uint8array-helpers'
import { ENCODED_TAGS } from '../stream-utils/encoded-tags' import { ENCODED_TAGS } from '../stream-utils/encoded-tags'
...@@ -112,9 +113,7 @@ function nodeReadableToWebReadableStream( ...@@ -112,9 +113,7 @@ function nodeReadableToWebReadableStream(
if (stream instanceof ReadableStream) { if (stream instanceof ReadableStream) {
return stream return stream
} }
// Readable.toWeb returns stream/web ReadableStream which is structurally return nodeToWebByteStream(stream)
// identical to the global ReadableStream<Uint8Array>.
return Readable.toWeb(stream) as unknown as ReadableStream<Uint8Array>
} }
function webToReadable( function webToReadable(
...@@ -557,7 +556,15 @@ export function renderToNodeFlightStream( ...@@ -557,7 +556,15 @@ export function renderToNodeFlightStream(
// the request signal, which can outlive it. // the request signal, which can outlive it.
const { signal, ...renderOptions } = opts ?? {} const { signal, ...renderOptions } = opts ?? {}
const pt = new PassThrough() // React writes long rows as strings and its Node reader parses them as
// written. A byte-mode stream would encode them, so this one is object mode.
const pt = new Transform({
objectMode: true,
transform(chunk: string | Uint8Array, _encoding, callback) {
// TODO: We can remove this if React starts escaping lone surrogates.
callback(null, typeof chunk === 'string' ? chunk.toWellFormed() : chunk)
},
})
const pipeable = ComponentMod.renderToPipeableStream!( const pipeable = ComponentMod.renderToPipeableStream!(
payload, payload,
clientModules, clientModules,
...@@ -976,12 +983,79 @@ const INLINE_FLIGHT_PAYLOAD_BOOTSTRAP = 0 ...@@ -976,12 +983,79 @@ const INLINE_FLIGHT_PAYLOAD_BOOTSTRAP = 0
const INLINE_FLIGHT_PAYLOAD_DATA = 1 const INLINE_FLIGHT_PAYLOAD_DATA = 1
const INLINE_FLIGHT_PAYLOAD_FORM_STATE = 2 const INLINE_FLIGHT_PAYLOAD_FORM_STATE = 2
const INLINE_FLIGHT_PAYLOAD_BINARY = 3 const INLINE_FLIGHT_PAYLOAD_BINARY = 3
const INLINE_FLIGHT_DATA_SCRIPT_END = Buffer.from('"])</script>')
// Reads every chunk currently buffered so one script covers all of them.
// Returns escaped JSON fragments for a batch of text, the raw bytes when any
// byte run is not valid UTF-8, or null when nothing is buffered.
function readInlineFlightData(stream: Readable): string[] | Buffer | null {
const first: string | Uint8Array | null = stream.read()
if (first === null || !stream.readableObjectMode) {
return first as Buffer | null
}
const chunks = [first]
while (stream.readableLength > 0) {
chunks.push(stream.read())
}
// JSON and HTML escaping act per code unit, so escaping each well-formed
// fragment on its own equals escaping their concatenation.
const fragments: string[] = []
for (let i = 0; i < chunks.length; i++) {
const chunk = chunks[i]
let text: string
if (typeof chunk === 'string') {
text = chunk
} else {
const start = i
while (i + 1 < chunks.length && typeof chunks[i + 1] !== 'string') {
i++
}
const bytes =
start === i
? Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength)
: Buffer.concat(chunks.slice(start, i + 1) as Uint8Array[])
// A multibyte character may span adjacent byte chunks, so validate the
// run as a whole.
if (!isUtf8(bytes)) {
return Buffer.concat(
chunks.map((part) =>
typeof part === 'string' ? Buffer.from(part) : part
)
)
}
text = bytes.toString('utf8')
}
fragments.push(htmlEscapeJsonString(JSON.stringify(text).slice(1, -1)))
}
return fragments
}
function encodeInlineFlightData(fragments: string[], start: Buffer): Buffer {
let byteLength = start.length + INLINE_FLIGHT_DATA_SCRIPT_END.length
for (const fragment of fragments) {
byteLength += Buffer.byteLength(fragment)
}
const buffer = Buffer.allocUnsafe(byteLength)
start.copy(buffer)
let offset = start.length
for (const fragment of fragments) {
offset += buffer.write(fragment, offset, buffer.length - offset, 'utf8')
}
INLINE_FLIGHT_DATA_SCRIPT_END.copy(buffer, offset)
return buffer
}
async function pullFlightData( async function pullFlightData(
dataStream: Readable, dataStream: Readable,
output: PassThrough, output: PassThrough,
startScriptTag: string startScriptTag: string
): Promise<void> { ): Promise<void> {
let scriptStart: Buffer | undefined
let binaryScriptStart: Buffer | undefined
function waitForReadableOrEnd(): Promise<void> { function waitForReadableOrEnd(): Promise<void> {
if (dataStream.readableLength > 0 || dataStream.readableEnded) { if (dataStream.readableLength > 0 || dataStream.readableEnded) {
return Promise.resolve() return Promise.resolve()
...@@ -1008,29 +1082,34 @@ async function pullFlightData( ...@@ -1008,29 +1082,34 @@ async function pullFlightData(
try { try {
while (true) { while (true) {
const chunk: Buffer | null = dataStream.read() const chunk = readInlineFlightData(dataStream)
if (chunk !== null) { if (chunk !== null) {
let htmlInlinedData: string let fragments: string[]
if (isUtf8(chunk)) { if (Array.isArray(chunk)) {
const decodedString = chunk.toString('utf-8') fragments = chunk
htmlInlinedData = htmlEscapeJsonString( } else if (isUtf8(chunk)) {
JSON.stringify([INLINE_FLIGHT_PAYLOAD_DATA, decodedString]) fragments = [
) htmlEscapeJsonString(
JSON.stringify(chunk.toString('utf8')).slice(1, -1)
),
]
} else { } else {
const base64 = Buffer.from( const base64 = Buffer.from(
chunk.buffer, chunk.buffer,
chunk.byteOffset, chunk.byteOffset,
chunk.byteLength chunk.byteLength
).toString('base64') ).toString('base64')
htmlInlinedData = htmlEscapeJsonString( binaryScriptStart ??= Buffer.from(
JSON.stringify([INLINE_FLIGHT_PAYLOAD_BINARY, base64]) `${startScriptTag}self.__next_f.push([${INLINE_FLIGHT_PAYLOAD_BINARY},"`
) )
// Base64 contains no JSON or HTML special characters.
output.push(encodeInlineFlightData([base64], binaryScriptStart))
continue
} }
output.push( scriptStart ??= Buffer.from(
Buffer.from( `${startScriptTag}self.__next_f.push([${INLINE_FLIGHT_PAYLOAD_DATA},"`
`${startScriptTag}self.__next_f.push(${htmlInlinedData})</script>`
)
) )
output.push(encodeInlineFlightData(fragments, scriptStart))
continue continue
} }
......
...@@ -200,7 +200,15 @@ export async function pipeNodeReadableToNodeResponse( ...@@ -200,7 +200,15 @@ export async function pipeNodeReadableToNodeResponse(
finished.resolve() finished.resolve()
}) })
readable.on('data', (chunk: Buffer) => { readable.on('data', (chunk: string | Uint8Array) => {
// Object-mode readables carry strings and raw views. Custom servers
// wrap `res.write` and may assume `Buffer.isBuffer(chunk)`.
if (typeof chunk === 'string') {
chunk = Buffer.from(chunk)
} else if (!Buffer.isBuffer(chunk)) {
chunk = Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength)
}
if (!started) { if (!started) {
started = true started = true
......
...@@ -10,6 +10,7 @@ import { ...@@ -10,6 +10,7 @@ import {
streamFromBuffer, streamFromBuffer,
streamFromString, streamFromString,
streamToString, streamToString,
nodeToWebByteStream,
} from './stream-utils/node-web-streams-helper' } from './stream-utils/node-web-streams-helper'
import { import {
isAbortError, isAbortError,
...@@ -270,24 +271,7 @@ export default class RenderResult< ...@@ -270,24 +271,7 @@ export default class RenderResult<
} }
if (isNodeReadable(this.response)) { if (isNodeReadable(this.response)) {
if (process.env.NEXT_RUNTIME === 'edge') { return nodeToWebByteStream(this.response)
throw new InvariantError(
'Node.js Readable cannot be converted to a web stream in the edge runtime'
)
} else {
let Readable: typeof import('node:stream').Readable
if (process.env.TURBOPACK) {
Readable = (require('node:stream') as typeof import('node:stream'))
.Readable
} else {
Readable = (
__non_webpack_require__(
'node:stream'
) as typeof import('node:stream')
).Readable
}
return Readable.toWeb(this.response) as ReadableStream<Uint8Array>
}
} }
return this.response return this.response
...@@ -313,24 +297,7 @@ export default class RenderResult< ...@@ -313,24 +297,7 @@ export default class RenderResult<
} else if (Buffer.isBuffer(this.response)) { } else if (Buffer.isBuffer(this.response)) {
return [streamFromBuffer(this.response)] return [streamFromBuffer(this.response)]
} else if (isNodeReadable(this.response)) { } else if (isNodeReadable(this.response)) {
if (process.env.NEXT_RUNTIME === 'edge') { return [nodeToWebByteStream(this.response)]
throw new InvariantError(
'Node.js Readable cannot be converted to a web stream in the edge runtime'
)
} else {
let Readable: typeof import('node:stream').Readable
if (process.env.TURBOPACK) {
Readable = (require('node:stream') as typeof import('node:stream'))
.Readable
} else {
Readable = (
__non_webpack_require__(
'node:stream'
) as typeof import('node:stream')
).Readable
}
return [Readable.toWeb(this.response) as ReadableStream<Uint8Array>]
}
} else { } else {
return [this.response] return [this.response]
} }
......
...@@ -14,6 +14,7 @@ import { ...@@ -14,6 +14,7 @@ import {
removeFromUint8Array, removeFromUint8Array,
} from './uint8array-helpers' } from './uint8array-helpers'
import { MISSING_ROOT_TAGS_ERROR } from '../../shared/lib/errors/constants' import { MISSING_ROOT_TAGS_ERROR } from '../../shared/lib/errors/constants'
import { InvariantError } from '../../shared/lib/invariant-error'
import { import {
RSC_HEADER, RSC_HEADER,
NEXT_ROUTER_PREFETCH_HEADER, NEXT_ROUTER_PREFETCH_HEADER,
...@@ -34,6 +35,54 @@ function voidCatch() { ...@@ -34,6 +35,54 @@ function voidCatch() {
// when handling streaming data // when handling streaming data
const encoder = new TextEncoder() const encoder = new TextEncoder()
/** Encode only when a consumer requires bytes; retain binary views as-is. */
export function encodeTextChunk(chunk: string | Uint8Array): Uint8Array {
return typeof chunk === 'string' ? encoder.encode(chunk) : chunk
}
/** Adapt a Node stream to the byte contract used by Web response consumers. */
export function nodeToWebByteStream(
stream: import('node:stream').Readable
): ReadableStream<Uint8Array> {
if (process.env.NEXT_RUNTIME === 'edge') {
throw new InvariantError(
'Node.js Readable cannot be converted to a web stream in the edge runtime'
)
} else {
let Readable: typeof import('node:stream').Readable
if (process.env.TURBOPACK) {
Readable = (require('node:stream') as typeof import('node:stream'))
.Readable
} else if (
process.env.__NEXT_BUNDLER === 'Webpack' ||
process.env.__NEXT_BUNDLER === 'Rspack'
) {
Readable = (
__non_webpack_require__('node:stream') as typeof import('node:stream')
).Readable
} else {
Readable = (require('node:stream') as typeof import('node:stream'))
.Readable
}
const webStream = Readable.toWeb(stream) as ReadableStream<
string | Uint8Array
>
// Readable.toWeb preserves object-mode strings; it does not encode them.
// Already-byte streams need no additional transform.
if (!stream.readableObjectMode && stream.readableEncoding === null) {
return webStream as ReadableStream<Uint8Array>
}
return webStream.pipeThrough(
new TransformStream<string | Uint8Array, Uint8Array>({
transform(chunk, controller) {
controller.enqueue(encodeTextChunk(chunk))
},
})
)
}
}
export function chainStreams<T>( export function chainStreams<T>(
...streams: ReadableStream<T>[] ...streams: ReadableStream<T>[]
): ReadableStream<T> { ): ReadableStream<T> {
......
'use client'
export function Client({ text }: { text: string }) {
return <p id="text">{text}</p>
}
import type { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
import { connection } from 'next/server'
import { Suspense } from 'react'
import { text } from '../data'
import { Client } from './client'
export default function Page() {
return (
<Suspense fallback={<p>Loading</p>}>
<Content />
</Suspense>
)
}
async function Content() {
await connection()
return <Client text={text} />
}
import { nextTestSetup } from 'e2e-utils'
import { text } from './data'
type RecordedRequest = {
url: string
rsc: string | undefined
status: number
chunks: string[]
}
// A custom server (server.js) is not supported in deploy mode.
// @force-gate !deploy
describe('custom-server-response-chunks', () => {
const { next, isNextDev } = nextTestSetup({
files: __dirname,
startCommand: 'node server.js',
serverReadyPattern: /Next mode: (production|development)/,
dependencies: {
'get-port': '5.1.1',
},
})
async function recordedRequests(): Promise<RecordedRequest[]> {
const res = await next.fetch('/__requests')
return res.json()
}
it('writes no string chunks to the HTML response', async () => {
const html = await next.render('/')
expect(html).toContain(text)
const requests = await recordedRequests()
const request = requests.find((r) => r.url === '/' && r.status === 200)
expect(request.chunks.length).toBeGreaterThan(0)
expect(request.chunks).not.toContain('string')
})
it('writes no string chunks to the RSC response', async () => {
const res = await next.fetch('/', { headers: { RSC: '1' } })
expect(res.headers.get('content-type')).toBe('text/x-component')
expect(await res.text()).toContain(text)
const requests = await recordedRequests()
const request = requests.find((r) => r.rsc === '1' && r.status === 200)
expect(request.chunks.length).toBeGreaterThan(0)
expect(request.chunks).not.toContain('string')
if (!isNextDev) {
// Production pipes the Flight stream to the response directly. The dev
// server goes through a Web stream, which hands out plain Uint8Arrays.
expect(new Set(request.chunks)).toEqual(new Set(['buffer']))
}
})
})
// Long enough that React writes it to the Flight stream as a raw string
// rather than encoding it into its byte buffer.
export const text = 'flight-text-'.repeat(400)
import type { NextConfig } from 'next'
const nextConfig: NextConfig = {
// With compression on, the custom server would only see gzip's output.
compress: false,
}
export default nextConfig
const http = require('http')
const { parse } = require('url')
const next = require('next')
const getPort = require('get-port')
// Every chunk passed to `res.write` and `res.end`, per request.
const requests = []
function chunkType(chunk) {
if (Buffer.isBuffer(chunk)) return 'buffer'
if (chunk instanceof Uint8Array) return 'uint8array'
return typeof chunk
}
async function main() {
const dev = process.env.NEXT_TEST_MODE === 'dev'
process.env.NODE_ENV = dev ? 'development' : 'production'
const port = await getPort()
const app = next({ dev, port })
const handle = app.getRequestHandler()
await app.prepare()
const server = http.createServer(async (req, res) => {
const parsedUrl = parse(req.url, true)
if (parsedUrl.pathname === '/__requests') {
res.setHeader('content-type', 'application/json')
res.end(JSON.stringify(requests))
return
}
const entry = { url: req.url, rsc: req.headers.rsc, chunks: [] }
requests.push(entry)
for (const method of ['write', 'end']) {
const original = res[method]
res[method] = function (chunk, ...rest) {
if (chunk != null) {
entry.chunks.push(chunkType(chunk))
}
return original.call(this, chunk, ...rest)
}
}
res.once('finish', () => {
entry.status = res.statusCode
})
try {
await handle(req, res, parsedUrl)
} catch (err) {
console.error('Error occurred handling', req.url, err)
res.statusCode = 500
res.end('internal server error')
}
})
server.once('error', (err) => {
console.error(err)
process.exit(1)
})
server.listen(port, () => {
console.log(`- Local: http://localhost:${port}`)
console.log(`- Next mode: ${process.env.NODE_ENV}`)
})
}
main().catch((err) => {
console.error(err)
process.exit(1)
})
'use client'
import { use, useState } from 'react'
export function Client({
text,
bytes,
later,
}: {
text: string
bytes: Uint8Array
later: Promise<string>
}) {
const resolved = use(later)
const [clicks, setClicks] = useState(0)
return (
<main>
<pre id="text">{text}</pre>
<span id="last-code-unit">{text.charCodeAt(text.length - 1)}</span>
<span id="bytes">{Array.from(bytes).join(',')}</span>
<span id="later-code-unit">
{resolved.charCodeAt(resolved.length - 1)}
</span>
<button onClick={() => setClicks(clicks + 1)}>{clicks}</button>
</main>
)
}
import type { ReactNode } from 'react'
export default function Root({ children }: { children: ReactNode }) {
return (
<html>
<body>{children}</body>
</html>
)
}
import { connection } from 'next/server'
import { Suspense } from 'react'
import { text } from '../data'
import { Client } from './client'
export default function Page() {
return (
<Suspense fallback={<p>Loading</p>}>
<Content />
</Suspense>
)
}
async function Content() {
await connection()
const storage = new Uint8Array([17, 0, 255, 128, 60, 195, 169, 18])
return (
<Client
text={text}
bytes={storage.subarray(1, -1)}
later={Promise.resolve(text)}
/>
)
}
// Long enough that React outlines it as a Flight text row and writes it as a
// raw string. The trailing lone surrogate cannot be encoded as UTF-8.
export const text =
'é中文😀</script><script>globalThis.__flightInjected=1</script>&\u2028\u2029"\\'.repeat(
200
) + '\ud800'
// Every consumer sees the lone surrogate as U+FFFD, which is what UTF-8
// encoding produces for it on the RSC response and in the inline script.
export const expectedText = text.slice(0, -1) + '\ufffd'
export const expectedBytes = '0,255,128,60,195,169'
import { nextTestSetup } from 'e2e-utils'
import { retry } from 'next-test-utils'
import { expectedText, expectedBytes } from './data'
describe('Flight native chunks', () => {
const { next } = nextTestSetup({ files: __dirname })
it('preserves Flight text, binary slices and promise values during SSR', async () => {
const $ = await next.render$('/')
expect($('#text').text()).toBe(expectedText)
expect($('#bytes').text()).toBe(expectedBytes)
expect($('#last-code-unit').text()).toBe('65533')
expect($('#later-code-unit').text()).toBe('65533')
})
it('sends the same text on the RSC response', async () => {
const res = await next.fetch('/', { headers: { RSC: '1' } })
expect(res.headers.get('content-type')).toBe('text/x-component')
expect(await res.text()).toContain(expectedText)
})
it('hydrates the same values without executing embedded script text', async () => {
const browser = await next.browser('/')
expect(await browser.elementByCss('#text').text()).toBe(expectedText)
expect(await browser.elementByCss('#bytes').text()).toBe(expectedBytes)
expect(await browser.elementByCss('#last-code-unit').text()).toBe('65533')
expect(await browser.elementByCss('#later-code-unit').text()).toBe('65533')
expect(await browser.eval('window.__flightInjected || null')).toBeNull()
await browser.elementByCss('button').click()
await retry(async () => {
expect(await browser.elementByCss('button').text()).toBe('1')
})
// A hydration mismatch between the SSR text and the inline Flight data
// would be reported here.
const logs = await browser.log()
expect(logs.filter((log) => log.source === 'error')).toEqual([])
})
})
import type { NextConfig } from 'next'
const nextConfig: NextConfig = {}
export default nextConfig
import { once } from 'node:events'
import { PassThrough, Readable } from 'node:stream'
import { runInNewContext } from 'node:vm'
import { createNodeInlinedDataStream } from 'next/dist/server/app-render/stream-ops.node'
async function collect(stream: Readable): Promise<Buffer> {
const chunks: Buffer[] = []
for await (const chunk of stream) {
expect(Buffer.isBuffer(chunk)).toBe(true)
chunks.push(chunk)
}
return Buffer.concat(chunks)
}
function decodeScripts(html: Buffer): Array<[number, string?]> {
const self = { __next_f: [] as Array<[number, string?]> }
const scripts = html.toString().matchAll(/<script[^>]*>([\s\S]*?)<\/script>/g)
for (const script of scripts) {
runInNewContext(script[1], { self })
}
return self.__next_f
}
describe('Node inline Flight data', () => {
// E2E tests exercise real Flight rendering and hydration. These inputs force
// chunk boundaries that a real server cannot reliably produce on demand.
it.each([
{
name: 'UTF-8 split between adjacent byte chunks',
chunks: [Buffer.from([0xf0, 0x9f]), Buffer.from([0x98, 0x80]), 'tail'],
},
{
name: 'an invalid UTF-8 view with a nonzero byte offset',
chunks: [
'head',
new Uint8Array([17, 0, 255, 128, 18]).subarray(1, 4),
'tail',
],
},
{
name: 'BOMs in both text and byte chunks',
chunks: ['\ufefftext', Buffer.from('\ufeffbytes')],
},
{
name: 'script delimiters and escapes split between text chunks',
chunks: ['<', '/script', '>&', '\\', 'u2028', '\u2028\u2029', '"', '\n'],
},
])('preserves the bytes of $name', async ({ chunks }) => {
const source = new PassThrough({ objectMode: true })
for (const chunk of chunks) source.write(chunk)
source.end()
const html = await collect(
createNodeInlinedDataStream(source, undefined, null) as Readable
)
const received = Buffer.concat(
decodeScripts(html)
.filter(([tag]) => tag === 1 || tag === 3)
.map(([tag, value]) =>
Buffer.from(value!, tag === 1 ? 'utf8' : 'base64')
)
)
const expected = Buffer.concat(
chunks.map((chunk) =>
typeof chunk === 'string' ? Buffer.from(chunk) : chunk
)
)
expect(received).toEqual(expected)
})
it.each([0, 8192, 128 * 1024])(
'preserves the complete script bytes for a batch with a %i-character fragment',
async (length) => {
const source = new PassThrough({ objectMode: true })
const text = 'a'.repeat(length)
for (const chunk of [
'<head>&\n"',
text,
Buffer.from([0xc3]),
Buffer.from([0xa9]),
'\\tail😀',
'',
]) {
source.write(chunk)
}
source.end()
const html = await collect(
createNodeInlinedDataStream(source, 'a"é', '<&>') as Readable
)
const start = '<script nonce="a&quot;é">'
expect(html).toEqual(
Buffer.from(
`${start}(self.__next_f=self.__next_f||[]).push([0]);self.__next_f.push([2,"\\u003c\\u0026\\u003e"])</script>` +
`${start}self.__next_f.push([1,"\\u003chead\\u003e\\u0026\\n\\"${text}é\\\\tail😀"])</script>`
)
)
}
)
it('keeps UTF-8 split across separate availability batches as binary', async () => {
const source = new PassThrough({ objectMode: true })
const html = createNodeInlinedDataStream(
source,
undefined,
null
) as Readable
const output: Buffer[] = []
html.on('data', (chunk) => output.push(chunk))
await once(html, 'data')
for (const bytes of [
Buffer.from([0xf0, 0x9f]),
Buffer.from([0x98, 0x80]),
]) {
const data = once(html, 'data')
source.write(bytes)
await data
}
const end = once(html, 'end')
source.end()
await end
expect(decodeScripts(Buffer.concat(output))).toEqual([
[0],
[3, '8J8='],
[3, 'mIA='],
])
})
})
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment