PagePool: don't block reused-tab callers on standby refill (CS-11139)

[habdelra]

Summary

• Decouples PagePool#getPage from the dedup'd #ensureStandbyPool promise so a slow refill no longer stalls reused-tab callers across the server.
• Moves the standby-pool wait inside #selectEntryForAffinity's last-resort branch, returning per-caller tabStartupMs so the diagnostic captures only the wait actually paid.
• Adds a regression test suite for the refill-doesn't-block-reuse contract.

Closes CS-11139.

What was wrong

getPage did await this.#ensureStandbyPool() synchronously before #selectEntryForAffinity. Concurrent callers shared the same #ensuringStandbys promise, so when standby creation hung — for example, when #evictLRUAffinity waited on page.close() of a Glimmer-tracking-loop render — every caller on that server paid the full hang, even ones who would have landed on a warm reused tab and never touched a standby.

The on-call fingerprint: a 2026-05-13 staging incident where a single hung render on prerender container 5f5f63e1622841ed9e239f176203a512 produced a fleet-wide 90-minute degradation, with bursts of slow tabStartupMs events that included rows with tabReused: true (130 s of "tab startup" for callers that ended up on a warm tab) and 6 indexing jobs aborted by the 150 s upstream timeout across multiple realms — all routed to the same wedged container.

What changed

packages/realm-server/prerender/page-pool.ts

• getPage (around line 942): the synchronous await this.#ensureStandbyPool() is replaced with a fire-and-forget kick (void this.#ensureStandbyPool().catch(log)). Standby refill still runs in the background to keep the pool warm for the next caller.
• #selectEntryForAffinity now reports tabStartupMs on its return shape. The internal awaitStandbyPool helper accumulates only the time this specific caller spent inside #ensureStandbyPool. The fast paths (warm idle tab / orphan claim / dormant-tab commandeer / cross-affinity steal) all return tabStartupMs: 0.
• A new last-resort branch runs at the bottom of #selectEntryForAffinity: if every fast path missed, we now await the refill once and retry #tryClaimOrphanContext / #commandeerDormantTab before throwing 'No standby page available for prerender'. The throw is now genuinely the "refill couldn't produce a tab" case, which used to be unreachable in practice because of the upstream synchronous await.
• tabQueueMs accounting in getPage is adjusted to subtract any standby-refill wait that #selectEntryForAffinity performed, so the wait categories don't double-count.

.claude/skills/indexing-diagnostics/SKILL.md — updates the inline tabStartupMs comment to reflect the per-caller semantics; a tabReused: true row should now have tabStartupMs ≈ 0, and a non-trivial value on a reused-tab row is the CS-11139 dedup-amplification signature.

packages/realm-server/tests/page-pool-standby-refill-test.ts — three new tests pinning the contract:

1. Reused-tab caller returns quickly with tabStartupMs === 0 while the standby refill is hung in the background.
2. Caller that genuinely needs a fresh standby still waits and reports the actual wait in tabStartupMs.
3. Concurrent reused-tab callers on different affinities both return quickly despite a stalled refill — the dedup'd promise no longer leaks waits across callers.

Test plan

• pnpm exec qunit … tests/page-pool-standby-refill-test.ts — 3/3 pass
• pnpm exec qunit … tests/page-pool-expansion-test.ts tests/page-pool-priority-test.ts tests/page-pool-standby-refill-test.ts — 28/28 pass
• pnpm exec qunit … tests/prerender-deadlock-test.ts tests/prerender-cancellation-test.ts — 20/20 pass
• pnpm lint:js clean on touched files (existing lint:types failures in ../base/*.gts from unbuilt sibling packages are pre-existing and unrelated)
• Soak in staging: tabStartupMs p99 falls for tabReused: true rows; ≥60s bucket empties for reused-tab rows
• Follow-up: CS-11140 for the underlying eviction-can't-free-stuck-pages issue this exposes

🤖 Generated with Claude Code

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 46f8177ee3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull request overview

This PR updates the realm-server prerender PagePool so that slow/hung standby-tab refills no longer stall callers that can reuse an already-warm tab, and refines the tabStartupMs diagnostic to reflect only the standby-refill wait actually paid by an individual caller.

Changes:

• Make getPage() kick off standby refills asynchronously (no longer awaiting the deduped refill promise up front).
• Move standby-refill waiting into #selectEntryForAffinity()’s true last-resort path and return per-caller tabStartupMs for accurate attribution.
• Add a regression test suite covering “refill doesn’t block reuse” scenarios and register it in the test index.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
packages/realm-server/prerender/page-pool.ts	Decouples standby refill from the critical path for warm-tab reuse; introduces per-caller tabStartupMs and last-resort refill retry logic.
packages/realm-server/tests/page-pool-standby-refill-test.ts	Adds regression tests ensuring reused-tab callers are not blocked by stalled refills and that true standby-needing callers report the wait.
packages/realm-server/tests/index.ts	Registers the new test module in the realm-server QUnit suite.
.claude/skills/indexing-diagnostics/SKILL.md	Updates operational documentation to reflect the new per-caller tabStartupMs semantics.

Comments suppressed due to low confidence (1)

packages/realm-server/tests/page-pool-standby-refill-test.ts:222

• The elapsed < 200 assertion is timing-sensitive and may be flaky under slow CI or a busy event loop. Consider making this deterministic (assert the promise resolves before unblocking, or use a much more forgiving threshold) so the test doesn’t fail spuriously.

    assert.true(
      elapsed < 200,
      `both reused-tab callers returned in ${elapsed}ms despite blocked standby refill`,
    );

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

Host Test Results

    1 files  ±    0      1 suites  ±0   3h 21m 36s ⏱️ + 1h 33m 23s
2 661 tests +    2  2 646 ✅ +    2  15 💤 ± 0  0 ❌ ±0 
5 278 runs  +2 600  5 249 ✅ +2 586  29 💤 +14  0 ❌ ±0 

Results for commit a24f09b. ± Comparison against earlier commit a099f21.

Realm Server Test Results

    1 files  ±0      1 suites  ±0   14m 30s ⏱️ +41s
1 368 tests +2  1 368 ✅ +3  0 💤 ±0  0 ❌  - 1 
1 447 runs  +2  1 447 ✅ +3  0 💤 ±0  0 ❌  - 1 

Results for commit a24f09b. ± Comparison against earlier commit a099f21.

[habdelra]

[Claude Code] CI failure on prerendering-test.ts > queues same-realm request when tab is transitioning (and the related does not reuse across different realms) traced to a microtask-ordering regression my fix introduced. The new awaitStandbyPool helper in #selectEntryForAffinity always yielded a microtask via await this.#ensureStandbyPool(), even when the refill was a guaranteed no-op (disableStandbyRefill: true with an active tab). That extra hop let a sibling same-affinity caller queue on the busy tab one microtask earlier, inflating its TabQueue's pendingCount past the cross-affinity scan's > 1 filter — and the brand-new-affinity caller threw 'No standby page available for prerender' despite a valid stealable candidate existing.

Fixed in commit e7a467b: awaitStandbyPool now skips the await entirely when #currentStandbyCount >= #desiredStandbyCount (no refill work to do). The helper is synchronous in the no-op case, restoring the relative microtask ordering the cross-affinity scan was designed around. Local sweep of page-pool-* + prerender-deadlock + prerender-cancellation tests: 48/48 pass; the CI re-run should clear both failures.

[habdelra]

[Claude Code] Previous CI fix wasn't enough — the fast-path check was inside an async helper, and await asyncHelper() still yields one microtask on the resolved Promise (the very thing the fast-path was trying to avoid). The cross-realm caller paid that extra hop and lost the microtask race against the same-realm caller, same failure mode as before.

Inlined the check at both call sites in a099f21 — if (currentStandbyCount < desiredStandbyCount) { await … } directly, no helper. The no-op path is now strictly synchronous, restoring the pre-CS-11139 relative microtask ordering. Verified locally: 48/48 page-pool + prerender tests pass.

[habdelra]

[Claude Code] Status update on commit a099f21:

• Tests 79 (does not reuse across different realms) and 100 (queues same-realm request when tab is transitioning) — the original regression my fix targeted — now both PASS. The inline microtask check is correct.
• A different test failed: 46 file prerender returns extracted metadata with "deps include base file-api module" expected true, got false. This is in file-extract dep tracking, unrelated to my changes (which only touch #selectEntryForAffinity standby-refill ordering). Reads like a flake; the workflow auto-retries on push so we'll see what the next run shows.

[habdelra]

[Claude Code] Found the actual cause of test 46 (file prerender returns extracted metadata / "deps include base file-api module"). It's a real regression from CS-11139, not a flake.

#commandeerDormantTab does both standby-adoption AND cross-affinity-steal. Pre-CS-11139, getPage's upfront await #ensureStandbyPool() always produced a fresh standby BEFORE commandeerDormantTab ran in step 2 of #selectEntryForAffinity — so the cross-affinity branch never fired there in practice. After CS-11139 removed the upfront await, the fire-and-forget refill can lag behind, and step 2's commandeer falls through to cross-affinity steal. The stolen tab's runtime has file-api cached from previous tests in other realms, so the host's per-prerender deps tracker doesn't re-track it.

Fixed in a24f09b: add a standbyOnly: true opt on #commandeerDormantTab. All eager step-2 / fallback paths pass it (they should never preemptively steal across realms — only adopt a fresh standby). Cross-affinity steal is reserved for the explicit crossAffinityEntries scan in the entryList===0 last-resort branch, which already awaits the refill so a fresh standby is preferred over a busy cross-realm tab.

48/48 page-pool + prerender unit tests pass locally.