Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions docs/issues/yobrowser-cdp-graceful-degradation/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Plan

## Source Review

- `YoBrowserPresenter.updateSessionBrowserBounds()` marks a session invisible
when the renderer reports `visible=false` or zero-size bounds.
- `YoBrowserPresenter.getBrowserStatus()` already returns enough state for an
agent-facing recovery hint: initialized, page, navigation flags, visible, and
loading.
- `YoBrowserToolHandler.callTool()` currently checks `getBrowserPage()` before
`cdp_send` and throws a generic initialization error when no page is available.
- `AgentToolManager` currently wraps YoBrowser handler success as `{ content }`;
thrown errors are caught later in the agent runtime and become errored tool
results with text like `Error: ...`.
- `ToolPresenter` can preserve agent tool failures through `rawData.isError` and
`createAgentToolErrorResult`, which is a better fit for recoverable,
structured YoBrowser failures than an untyped exception string.

## Design

- Add a small YoBrowser recoverable error contract for browser availability
failures. The contract should include:
- `code: "yobrowser_unavailable"`
- `recoverable: true`
- `sessionId`
- attempted `method`
- sanitized `browserStatus` from `getBrowserStatus(sessionId)` when available
- concise `suggestedNextActions`
- Detect unavailable-browser states before CDP execution in
`YoBrowserToolHandler.callTool("cdp_send", ...)`:
- missing conversation/session id remains a validation error
- missing or destroyed page maps to the recoverable YoBrowser error
- a known not-ready browser/CDP error that means the browser cannot accept CDP
commands maps to the same recoverable YoBrowser error
- ordinary CDP protocol errors remain ordinary tool errors
- Propagate the recoverable YoBrowser error as an errored agent tool result with
structured content instead of only throwing a generic exception. Prefer the
existing `AgentToolCallResult`/`rawData.isError` path so the runtime marks the
block as an error while preserving the model-readable JSON content.
- Keep the agent-visible payload compact. Do not include stack traces, Electron
internals, full DOM content, screenshots, or local paths.
- Update the YoBrowser tool system prompt only if needed to make the recovery
path explicit. If changed, keep it brief and tool-oriented:
`If cdp_send reports yobrowser_unavailable, inspect get_browser_status and use
load_url to reopen the browser when you have a URL.`

## Event Flow

1. User closes or hides the YoBrowser panel while an agent task is running.
2. Renderer bounds update reaches `YoBrowserPresenter.updateSessionBrowserBounds`
with `visible=false` or an unusable size.
3. YoBrowser session state becomes not visible or no longer CDP-ready.
4. The agent later calls `cdp_send`.
5. `YoBrowserToolHandler` detects the unavailable browser state and builds the
recoverable YoBrowser error payload.
6. Agent tool routing returns that payload as an errored tool result.
7. The agent runtime records the tool call as failed but injects the structured
error content into the next model context.
8. The model can call `get_browser_status`, call `load_url` with an available
URL, ask the user to reopen the panel, or continue without browser
verification.

## Compatibility

- No storage migration is required.
- No tool name, IPC route, or renderer event contract changes are required for
the first increment.
- Existing successful YoBrowser automation remains source-compatible.
- Existing generic failure logs can stay, but the agent-visible error should no
longer depend on raw exception text for the browser-unavailable case.

## Test Strategy

- Update `test/main/presenter/browser/YoBrowserToolHandler.test.ts` to verify
that `cdp_send` on a missing browser returns or raises the recoverable
YoBrowser error contract expected by the chosen propagation path.
- Add or update agent tool manager / tool presenter coverage to verify
recoverable YoBrowser errors become `rawData.isError === true` with structured
model-visible content.
- Add or update agent runtime dispatch coverage to verify the tool block remains
errored and the response text contains the stable `yobrowser_unavailable`
signal.
- Keep existing tests for successful `cdp_send` and `load_url` behavior passing.

## Risks

- If the recoverable error is returned as normal content without `isError`, the
UI and runtime may mark the tool as successful. The implementation should use
the existing errored tool-result path.
- If the error payload is too verbose, it may waste context or obscure the
recovery instruction. Keep only state needed for model recovery.
- If all CDP exceptions are treated as browser unavailable, real page/script/CDP
protocol mistakes could become misleading recovery prompts. Limit mapping to
missing page, destroyed page, detached/closed state, and known not-ready
failures.
114 changes: 114 additions & 0 deletions docs/issues/yobrowser-cdp-graceful-degradation/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# YoBrowser CDP Graceful Degradation

## Problem

GitHub issue #1734 reports that a running agent task can lose browser control when
the user closes the right-side YoBrowser panel mid-session. The browser view is
detached or hidden, but the agent still attempts later `cdp_send` calls for DOM
inspection, scripted interaction, or screenshot verification. Today those calls
surface as generic initialization failures or blocked CDP failures, which gives
the model too little context to decide whether it should reopen the browser,
inspect status, skip browser-dependent verification, or ask the user for help.

## User Story

As a user running a browser-assisted agent task, I need CDP failures caused by an
unavailable YoBrowser session to be reported as meaningful, recoverable tool
errors so the agent can adapt its next step instead of stalling the task.

As an agent, when `cdp_send` cannot execute because the session browser is
closed, detached, hidden, destroyed, or otherwise not ready, I need a compact
error payload that explains the browser state and names the safe recovery tools
available in the same context.

## Acceptance Criteria

- `cdp_send` failures caused by an unavailable session browser are delivered to
the agent as tool errors, not as silent hangs or terminal application crashes.
- The tool error is meaningful to both the model and logs. It includes a stable
error code, the attempted CDP method, the conversation/session id, the current
YoBrowser status when available, whether the failure is recoverable, and a
short recovery hint.
- The tool error explicitly tells the agent that it may call
`get_browser_status` to inspect state and `load_url` to recreate or reopen the
session browser when it still has a target URL. If there is no target URL, the
hint allows the agent to ask the user to reopen the panel or continue without
browser verification.
- The agent runtime preserves the failure as an errored tool result so follow-up
model context can see that `cdp_send` failed, while still allowing the model to
choose a recovery strategy.
- Existing successful `cdp_send`, `load_url`, and `get_browser_status` behavior
remains unchanged.
- Non-browser-availability CDP errors, malformed arguments, missing
conversation ids, permission denials, and user cancellation keep their existing
error semantics unless they can be safely wrapped with the same recoverable
browser-unavailable code.
- The implementation avoids leaking Electron stack traces, internal object
dumps, filesystem paths, or private page content in the agent-visible error.
- Unit coverage verifies the unavailable-browser case, the still-successful CDP
case, and runtime propagation of the structured recoverable error into the
tool result.

## Non-goals

- Do not automatically reattach or reopen the YoBrowser panel in this first
increment.
- Do not add a new renderer-main browser state synchronization channel unless
implementation proves the existing status APIs are insufficient.
- Do not change the public names of `cdp_send`, `load_url`, or
`get_browser_status`.
- Do not retry CDP commands automatically. The model should decide whether to
retry, reopen, skip, or ask the user based on the tool error and conversation
context.
- Do not introduce UI copy or renderer layout changes for this issue.

## Constraints

- The fix should follow the existing Presenter and agent tool routing patterns:
YoBrowser-specific readiness detection belongs near
`YoBrowserPresenter`/`YoBrowserToolHandler`, while tool-result propagation
belongs in the agent tool path.
- Tool outputs are part of the model context, so the error payload must be small,
deterministic, and easy to parse even when prefixed by the runtime's standard
error formatting.
- `get_browser_status` already exposes the primary session state
(`initialized`, `visible`, `loading`, and page information), so the first
implementation should prefer reusing that state over adding broader event
synchronization.

## Proposed Agent-Visible Error Shape

The exact TypeScript representation can be refined during implementation, but
the agent-visible content should be equivalent to:

```json
{
"ok": false,
"error": {
"code": "yobrowser_unavailable",
"message": "YoBrowser is not available for this session, so the CDP command was not run.",
"recoverable": true,
"sessionId": "<conversation id>",
"method": "Page.captureScreenshot",
"browserStatus": {
"initialized": false,
"visible": false,
"loading": false,
"page": null
},
"suggestedNextActions": [
"Call get_browser_status to inspect the current browser state.",
"Call load_url with the target URL to recreate or reopen the session browser.",
"If no URL is available, ask the user to reopen the browser panel or continue without browser verification."
]
}
}
```

## Business Value

This turns a brittle browser-control failure into an agent-readable recovery
signal. The immediate user impact is fewer stalled browser-assisted tasks after
the panel is closed, while the implementation stays smaller and safer than
automatic recovery because it does not mutate browser visibility on behalf of
the model.
18 changes: 18 additions & 0 deletions docs/issues/yobrowser-cdp-graceful-degradation/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Tasks

- [x] Review GitHub issue #1734 and confirm the requested graceful-degradation
direction.
- [x] Inspect the current YoBrowser CDP call path and agent tool error
propagation.
- [x] Write SDD spec, plan, and task breakdown before code changes.
- [x] Define the YoBrowser recoverable error contract in the smallest suitable
module.
- [x] Map unavailable-browser `cdp_send` failures to the recoverable
`yobrowser_unavailable` error.
- [x] Propagate the recoverable error as an errored agent tool result with
structured model-visible content.
- [x] Add focused unit tests for YoBrowser handler behavior and agent runtime
propagation.
- [x] Run `pnpm run format`.
- [x] Run `pnpm run i18n`.
- [x] Run `pnpm run lint`.
57 changes: 57 additions & 0 deletions src/main/presenter/browser/YoBrowserErrors.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import type { YoBrowserStatus } from '@shared/types/browser'

export const YO_BROWSER_UNAVAILABLE_ERROR_CODE = 'yobrowser_unavailable'

export interface YoBrowserUnavailableErrorPayload {
ok: false
error: {
code: typeof YO_BROWSER_UNAVAILABLE_ERROR_CODE
message: string
recoverable: true
sessionId: string
method: string
browserStatus: YoBrowserStatus | null
suggestedNextActions: string[]
}
}

export class YoBrowserUnavailableError extends Error {
readonly payload: YoBrowserUnavailableErrorPayload
readonly originalError?: unknown

constructor(payload: YoBrowserUnavailableErrorPayload, originalError?: unknown) {
super(payload.error.message)
this.name = 'YoBrowserUnavailableError'
this.payload = payload
this.originalError = originalError
}
}

export const isYoBrowserUnavailableError = (error: unknown): error is YoBrowserUnavailableError =>
error instanceof YoBrowserUnavailableError ||
(error instanceof Error &&
error.name === 'YoBrowserUnavailableError' &&
typeof (error as { payload?: unknown }).payload === 'object' &&
(error as { payload?: YoBrowserUnavailableErrorPayload }).payload?.error?.code ===
YO_BROWSER_UNAVAILABLE_ERROR_CODE)

export const buildYoBrowserUnavailablePayload = (
sessionId: string,
method: string,
browserStatus: YoBrowserStatus | null
): YoBrowserUnavailableErrorPayload => ({
ok: false,
error: {
code: YO_BROWSER_UNAVAILABLE_ERROR_CODE,
message: 'YoBrowser is not available for this session, so the CDP command was not run.',
recoverable: true,
sessionId,
method,
browserStatus,
suggestedNextActions: [
'Call get_browser_status to inspect the current browser state.',
'Call load_url with the target URL to recreate or reopen the session browser.',
'If no URL is available, ask the user to reopen the browser panel or continue without browser verification.'
]
}
})
53 changes: 49 additions & 4 deletions src/main/presenter/browser/YoBrowserToolHandler.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
import logger from '@shared/logger'
import { getYoBrowserToolDefinitions } from './YoBrowserToolDefinitions'
import type { YoBrowserPresenter } from './YoBrowserPresenter'
import { BrowserPageStatus, type YoBrowserStatus } from '@shared/types/browser'
import {
YoBrowserUnavailableError,
buildYoBrowserUnavailablePayload,
isYoBrowserUnavailableError
} from './YoBrowserErrors'

export class YoBrowserToolHandler {
private readonly presenter: YoBrowserPresenter
Expand Down Expand Up @@ -42,9 +48,15 @@ export class YoBrowserToolHandler {
throw new Error('CDP method is required')
}

const page = await this.presenter.getBrowserPage(sessionId)
if (!page) {
throw new Error(`Session browser for ${sessionId} is not initialized`)
const status = await this.presenter.getBrowserStatus(sessionId)
const page = status.page
if (
!status.initialized ||
!status.visible ||
!page ||
page.status === BrowserPageStatus.Closed
) {
throw await this.createUnavailableError(sessionId, method, status)
}

try {
Expand All @@ -61,6 +73,7 @@ export class YoBrowserToolHandler {
url: page.url,
status: page.status
})
throw await this.createUnavailableError(sessionId, method, status, error)
}
throw error
}
Expand All @@ -69,11 +82,43 @@ export class YoBrowserToolHandler {
throw new Error(`Unknown YoBrowser tool: ${toolName}`)
}
} catch (error) {
logger.error('[YoBrowserToolHandler] Tool execution failed', { toolName, error })
if (isYoBrowserUnavailableError(error)) {
logger.warn('[YoBrowserToolHandler] Tool execution failed:browser-unavailable', {
toolName,
error: error.payload.error
})
} else {
logger.error('[YoBrowserToolHandler] Tool execution failed', { toolName, error })
}
throw error
}
}

private async createUnavailableError(
sessionId: string,
method: string,
knownStatus?: YoBrowserStatus,
originalError?: unknown
): Promise<YoBrowserUnavailableError> {
if (knownStatus) {
return new YoBrowserUnavailableError(
buildYoBrowserUnavailablePayload(sessionId, method, knownStatus),
originalError
)
}

return this.presenter
.getBrowserStatus(sessionId)
.catch(() => null)
.then(
(status) =>
new YoBrowserUnavailableError(
buildYoBrowserUnavailablePayload(sessionId, method, status),
originalError
)
)
}

private normalizeCdpParams(value: unknown): Record<string, unknown> {
if (typeof value === 'object' && value !== null && !Array.isArray(value)) {
return value as Record<string, unknown>
Expand Down
Loading