browser-probe: console/errors capture returns 0 on host CLI build (page console + uncaught errors not recorded)

[chubes4]

Summary

wordpress.browser-probe with capture=console,errors,screenshot reliably produces a screenshot, but console.jsonl and errors.jsonl come back empty and summary.json.summary reports consoleMessages: 0, errors: 0 regardless of what the page actually logs or throws.

Repro

Recipe (Playground backend, WP 6.9) that:

1. seeds a published page containing a <script> which both console.log(...)s a marker and triggers an uncaught TypeError (calling an undefined function during the page's load-time JS), then
2. runs:

{
  "command": "wordpress.browser-probe",
  "args": [
    "url=/?pagename=<seeded-page>",
    "wait-for=load",
    "duration=2s",
    "capture=console,errors,screenshot"
  ]
}

Observed summary.json:

{
  "schema": "wp-codebox/browser-probe/v1",
  "capture": ["console", "errors", "screenshot"],
  "summary": { "consoleMessages": 0, "errors": 0, "screenshot": true }
}

files/browser/console.jsonl and files/browser/errors.jsonl are both empty even though:

• the page logs a marker repeatedly on a setInterval (so messages fire well within the 2s post-load window), and
• a deliberately broken variant throws an uncaught page error at load.

Both the success and the broken variant report identical errors: 0 / consoleMessages: 0, so the probe cannot currently distinguish a clean render from one with uncaught page errors.

Impact

Page-error / console capture is the load-bearing signal for browser smokes that assert client-side behaviour (e.g. verifying that one widget's broken callback does not abort an implicit auto-render loop for sibling widgets). With these channels inert, browser-probe can only prove the page loaded + screenshotted, not assert on console/errors.

Likely area

Probably a listener-attachment ordering issue: the Playwright page.on('console') / page.on('pageerror') handlers may be attached after navigation completes, missing load-time events, and/or not being flushed into the artifact files. The screenshot path works, so the runtime + artifact plumbing is otherwise healthy.

Environment

• Host wp-codebox CLI (global install), Playground backend, PHP 8.3 / WordPress 6.9.
• WP plugin side is at v0.5.0; the wordpress.browser-actions command (richer expect/assert steps from the v0.5.0 cookbook) is also not present in this CLI build's wp-codebox commands list — possibly the same packaging lag.

Workaround in the meantime

Consumers can gate the full assertion on summary.consoleMessages > 0 and fall back to a load+screenshot-only smoke when capture is inert, to avoid false greens.

[chubes4]

Update: this is a stale-build artifact, not a source bug

After reading current source (packages/runtime-playground/src/browser-command-runners.ts + browser-probe.ts on main), the console/pageerror capture path is correct:

• Listeners are attached before navigation (page.on('console', ...) / page.on('pageerror', ...) at ~L122/L128, then navigateBrowserProbe(...) at ~L147).
• main also already ships the rich assert=no-console-errors|no-page-errors|no-errors API in browser-probe.ts (~L204, L268-281).

The consoleMessages: 0 / errors: 0 I originally observed came from the host CLI running a stale dist/: /usr/bin/wp-codebox symlinks to a workspace primary checkout whose packages/cli/dist/ predates the console-capture + assertion work (the built dist/browser-probe.js didn't even contain no-page-errors). The primary was checked out at a v0.5.x merge, not a current tag, and dist/ was built earlier still.

So there is no source fix needed — the resolution is to rebuild / update the deployed CLI build. Suggest this can be closed once that's confirmed on the affected host, unless maintainers want to track "shipped CLI dist can silently lag src" as a separate packaging concern.