Unverified Commit 3a20e2ab authored by Hendrik Liebau's avatar Hendrik Liebau Committed by GitHub

[test] Unflake `use-cache-default-profile-expire-zero` tests (#97250)

The first assertion in `use-cache-default-profile-expire-zero` retried a
full browser reload until the `expire: 0` cache produced a fresh value.
In dev that value is served warm across reads, so more than one read is
needed, and on a CI runner one of those reloads costs more than the
default 3000ms budget can grant a second attempt for. The dev failure
reports `waited 3162ms` after a single attempt, in a test that took
15.6s to get there.

Neither test asserts anything about the browser: both only read the
rendered value and check that the nested-cache error stayed out of the
CLI output. Reading the value with `next.render$` brings one attempt
down to between 30 and 174ms, so the default budget holds, and it drops
two browser launches. The helper rejects a missing element, because the
empty string that cheerio yields for one would otherwise satisfy the
assertion that the value changed.

The test predates #96354, which replaced the try counting in `retry`
with a clock. Before that change `retry` granted seven attempts
regardless of how long each one took, so a 3s reload was affordable;
after it, any attempt over 2500ms consumes the entire budget on its own.
#97187 makes the same change for two assertions that were failing this
way.

[Flakiness metrics][flakiness-metrics]

[flakiness-metrics]:

https://app.datadoghq.com/ci/test/runs?query=test_level%3Atest%20%40git.repository.id%3A%22github.com%2Fvercel%2Fnext.js%22%20%40test.name%3A%22use-cache-default-profile-expire-zero%20treats%20a%20short%20default%20cacheLife%20profile%20as%20a%20dynamic%20hole%2C%20not%20a%20nested-cache%20error%22%20%40test.type%3A%22nextjs%22%20%40test.status%3A%22fail%22&agg_m=count&agg_m_source=base&agg_t=count&fromUser=false&index=citest&start=1785948830138&end=1786553630138&paused=false
parent 04e9933b
...@@ -8,9 +8,26 @@ describe('use-cache-default-profile-expire-zero', () => { ...@@ -8,9 +8,26 @@ describe('use-cache-default-profile-expire-zero', () => {
files: __dirname, files: __dirname,
}) })
// Reads the rendered cache value over HTTP instead of through the browser. A
// dev page reload costs seconds on a CI runner, almost all of it downloading
// and evaluating the dev bundle, which keeps a retried read from fitting in
// its budget. The value is server-rendered, so a plain request observes the
// same cache state for a fraction of the cost.
async function readCachedValue(pathname: string): Promise<string> {
const $ = await next.render$(pathname)
const value = $('#value')
// Without this the caller asserting that the value changed would accept the
// empty string that a missing element yields.
if (value.length === 0) {
throw new Error(`Found no cached value in the HTML of ${pathname}.`)
}
return value.text()
}
it('treats a short default cacheLife profile as a dynamic hole, not a nested-cache error', async () => { it('treats a short default cacheLife profile as a dynamic hole, not a nested-cache error', async () => {
const browser = await next.browser('/') const initialValue = await readCachedValue('/')
const initialValue = await browser.elementById('value').text()
expect(initialValue).toMatch(isoDateRegExp) expect(initialValue).toMatch(isoDateRegExp)
// The short `expire` comes from the app's configured default profile, and // The short `expire` comes from the app's configured default profile, and
...@@ -22,11 +39,10 @@ describe('use-cache-default-profile-expire-zero', () => { ...@@ -22,11 +39,10 @@ describe('use-cache-default-profile-expire-zero', () => {
) )
// An `expire: 0` cache is a dynamic hole. In production it regenerates on // An `expire: 0` cache is a dynamic hole. In production it regenerates on
// every request; in dev it is served warm across reloads and re-warmed in // every request; in dev it is served warm across reads and re-warmed in the
// the background. Either way, reloads converge to a fresh value. // background. Either way, reads converge to a fresh value.
await retry(async () => { await retry(async () => {
await browser.refresh() const value = await readCachedValue('/')
const value = await browser.elementById('value').text()
expect(value).toMatch(isoDateRegExp) expect(value).toMatch(isoDateRegExp)
expect(value).not.toEqual(initialValue) expect(value).not.toEqual(initialValue)
}) })
...@@ -38,8 +54,7 @@ describe('use-cache-default-profile-expire-zero', () => { ...@@ -38,8 +54,7 @@ describe('use-cache-default-profile-expire-zero', () => {
// default profile is itself dynamic, the outer cache is omitted from // default profile is itself dynamic, the outer cache is omitted from
// prerenders by default, so there is no silent degradation to warn about // prerenders by default, so there is no silent degradation to warn about
// and the nested cache renders as a dynamic hole instead. // and the nested cache renders as a dynamic hole instead.
const browser = await next.browser('/nested') expect(await readCachedValue('/nested')).toMatch(isoDateRegExp)
expect(await browser.elementById('value').text()).toMatch(isoDateRegExp)
expect(next.cliOutput).not.toContain( expect(next.cliOutput).not.toContain(
'nested-use-cache-no-explicit-cachelife' 'nested-use-cache-no-explicit-cachelife'
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment