Unverified Commit 529ddc3c authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

[test] Unflake two `cache-components-dev-streaming` assertions (#97246)

The streaming assertion in `should stream suspense boundaries while
filling caches in the background` polled a bare `<p>` through `retry()`
with its 3000ms default. Because `retry` gives up as soon as `waited +
interval > duration`, the effective budget is about 2.6s, and the
fixture's cache fill alone takes 2000ms, so the assertion had roughly
600ms of headroom. That budget also had to absorb one browser round trip
per attempt, because a selector that matches the fallback as well as the
content cannot wait for anything: `waitForSelector` returns the fallback
immediately, so the waiting had to happen in the test process at a 500ms
granularity.

This change gives the two paragraphs the ids `#cached-fallback` and
`#cached`, in line with every other route in this fixture, which is what
lets the wait move into the browser. Playwright now resolves the moment
the content is revealed, in a single round trip, and #95466 had already
moved the rest of the suite to that pattern. The two assertions that
check what the shell itself delivers still need the original "whichever
element arrives first" probe, and they now express it as the selector
list `#cached, #cached-fallback` instead of relying on `p` to match
both.

The budget for the reveal is 10s rather than the 5s that `elementByCss`
narrows the harness default to, because the wait has to cover more than
the fill. In the linked failure the content bytes had arrived one second
after the shell committed, and the reveal was still at least 2.4s away,
since the browser was busy evaluating the dev bundle. A shorter fill
would not help with that, as it does not shorten the part of the wait
that CI actually spends.

The convergence assertion in `serves a short-expire cache warm on reload
and converges to a fresh value` ran out of the same budget for a
different reason: every attempt performed a full dev page reload on top
of the 1.5s regeneration, and measured runs needed 1.8s to 3.4s. It now
reads the value over HTTP with `next.render$`, the way #97187 does,
which observes the same server-side cache state without downloading and
evaluating the dev bundle. Under the same contention those reads
converge in 1.1s to 1.9s, and they stay there when the load is
quadrupled, because only the 1.5s regeneration gates the loop. The
default budget therefore covers it, and the test drops from 15.5s to
3.6s.

Both fixes were verified by raising the fill to 3.2s to emulate the
CI-side delay: the previous assertions then fail deterministically with
the CI signature, and the new ones pass.

[Flakiness
metrics](https://app.datadoghq.com/ci/test/runs?query=test_level%3Atest%20%40git.repository.id%3A%22github.com%2Fvercel%2Fnext.js%22%20%40test.name%3A%22cache-components-dev-streaming%20should%20stream%20suspense%20boundaries%20while%20filling%20caches%20in%20the%20background%22%20%40test.type%3A%22nextjs%22%20%40test.status%3A%22fail%22&agg_m=count&agg_m_source=base&agg_t=count&citest_explorer_sort=timestamp%2Casc&cols=%40test.status%2Ctimestamp%2C%40test.suite%2C%40test.name%2C%40duration%2C%40test.service%2C%40git.branch&currentTab=overview&eventStack=&fromUser=true&index=citest&start=1783948256248&end=1786540256248&paused=false)
parent 3a20e2ab
...@@ -4,12 +4,12 @@ import { setTimeout } from 'timers/promises' ...@@ -4,12 +4,12 @@ import { setTimeout } from 'timers/promises'
async function Cached() { async function Cached() {
'use cache' 'use cache'
await setTimeout(2000) await setTimeout(2000)
return <p>{new Date().toISOString()}</p> return <p id="cached">{new Date().toISOString()}</p>
} }
export default async function Page() { export default async function Page() {
return ( return (
<Suspense fallback={<p>Loading...</p>}> <Suspense fallback={<p id="cached-fallback">Loading...</p>}>
<Cached /> <Cached />
</Suspense> </Suspense>
) )
......
...@@ -14,24 +14,39 @@ describe('cache-components-dev-streaming', () => { ...@@ -14,24 +14,39 @@ describe('cache-components-dev-streaming', () => {
}) })
// The loading boundary should be streamed immediately, without waiting for // The loading boundary should be streamed immediately, without waiting for
// the cache to be filled. Read it at commit so we don't wait for "load" // the cache to be filled. The selector matches the fallback and the content
// (which only fires once the cache has filled). // alike, so it resolves with whichever one the shell delivers, and only the
expect(await browser.elementByCss('p', { waitUntil: false }).text()).toBe( // fallback is an acceptable answer here. Read it at commit (`waitUntil:
'Loading...' // false`) so we don't wait for "load" (which only fires once the cache has
) // filled).
expect(
await browser
.elementByCss('#cached, #cached-fallback', { waitUntil: false })
.text()
).toBe('Loading...')
// Eventually, the cache content should be streamed in. // Eventually, the cache content should be streamed in. The boundary reveals
await retry(async () => { // `#cached` atomically with its final content, so waiting for the element
expect( // is enough. The wait must cover more than the 2s fill. The content bytes
await browser.elementByCss('p', { waitUntil: false }).text() // arrive while the browser still evaluates the dev bundle, so on a loaded
).toBeDateString() // machine the reveal lands seconds after the response completed. The
}) // explicit 10s is the `waitForElementByCss` default that `elementByCss`
// narrows to 5s.
expect(
await browser
.elementByCss('#cached', { waitUntil: false, timeout: 10000 })
.text()
).toBeDateString()
// Now that the cache is filled, it should be served immediately with the // Now that the cache is filled, it should be served immediately with the
// shell on the next page load, without going through the loading boundary. // shell on the next page load, without going through the loading boundary.
// The combined selector resolves with whichever one the shell delivers, so
// a fallback appearing first would fail this assertion.
await browser.refresh({ waitUntil: 'commit' }) await browser.refresh({ waitUntil: 'commit' })
expect( expect(
await browser.elementByCss('p', { waitUntil: false }).text() await browser
.elementByCss('#cached, #cached-fallback', { waitUntil: false })
.text()
).toBeDateString() ).toBeDateString()
}) })
...@@ -258,15 +273,21 @@ describe('cache-components-dev-streaming', () => { ...@@ -258,15 +273,21 @@ describe('cache-components-dev-streaming', () => {
await browser.elementByCss('#expire-zero', { waitUntil: false }).text() await browser.elementByCss('#expire-zero', { waitUntil: false }).text()
).toBe(coldValue) ).toBe(coldValue)
// That warm reload re-warmed a fresh entry in the background, so a later // That warm read re-warmed a fresh entry in the background, so a later read
// reload converges to the new value. Read after "load" here (a plain // converges to the new value. More than one read is needed, because the
// refresh) since we want the settled value, not the streaming inspection // regeneration that the previous one started has not been written yet. The
// above. // reads go over HTTP instead of through the browser: the value is
// server-rendered, so a plain request observes the same cache state, while
// a dev page reload would spend seconds downloading and evaluating the dev
// bundle and keep the retry from fitting in its budget.
await retry(async () => { await retry(async () => {
await browser.refresh() const $ = await next.render$('/use-cache-expire-zero/reload')
expect(await browser.elementById('expire-zero').text()).not.toBe( const value = $('#expire-zero')
coldValue
) // Without this the assertion below would accept the empty string that a
// missing element yields.
expect(value.length).toBe(1)
expect(value.text()).not.toBe(coldValue)
}) })
}) })
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment