From accec90e81089159aefc97f092972029abbfdaea Mon Sep 17 00:00:00 2001 From: zhangmo8 Date: Thu, 28 May 2026 13:59:41 +0800 Subject: [PATCH] feat(agent): show tool result images & support send img to remote channel --- docs/issues/remote-tool-result-images/plan.md | 43 +++++ docs/issues/remote-tool-result-images/spec.md | 36 ++++ .../issues/remote-tool-result-images/tasks.md | 9 + .../agentRuntimePresenter/dispatch.ts | 1 + .../imageGenerationBlocks.ts | 37 ++-- .../presenter/agentRuntimePresenter/index.ts | 1 + .../services/remoteConversationRunner.ts | 161 ++++++++++++++++-- .../agentRuntimePresenter/dispatch.test.ts | 81 ++++++--- .../remoteConversationRunner.test.ts | 67 ++++++++ 9 files changed, 388 insertions(+), 48 deletions(-) create mode 100644 docs/issues/remote-tool-result-images/plan.md create mode 100644 docs/issues/remote-tool-result-images/spec.md create mode 100644 docs/issues/remote-tool-result-images/tasks.md diff --git a/docs/issues/remote-tool-result-images/plan.md b/docs/issues/remote-tool-result-images/plan.md new file mode 100644 index 000000000..22cd3a212 --- /dev/null +++ b/docs/issues/remote-tool-result-images/plan.md @@ -0,0 +1,43 @@ +# Plan + +## Current behavior + +Tool execution can attach visual previews to `tool_call.imagePreviews`. The desktop renderer shows those previews only inside the expanded tool-call details, not as normal assistant image messages. `prepareToolImagePreviewPresentation()` currently promotes only successful built-in `image_generate` previews into assistant `image` blocks. Other tool result images, including screenshots, remain embedded in the tool-call metadata. + +Remote snapshots historically persisted only assistant `image` blocks. The first fix added a fallback that also persists `tool_call.imagePreviews`, but the broader issue is conversation-level visibility: the assistant transcript itself should contain the image result. + +## Implementation approach + +1. Generalize `prepareToolImagePreviewPresentation()` so successful, non-error tool result previews with usable `data` are promoted into assistant `image` blocks for any tool source. +2. Keep the existing special-case behavior for built-in `image_generate`: its previews are promoted and removed from the tool-call detail panel. +3. For other tools, promote usable previews while preserving metadata-only/unusable previews on `tool_call.imagePreviews` so the detail panel can still show what is available. +4. Add stable image block metadata linking promoted images back to the tool call and preview source/title. +5. Keep the remote snapshot fallback for legacy conversations where previews are already stored only in `tool_call.imagePreviews`. +6. Update tests to cover screenshot/tool-output promotion in the normal runtime path and the remote fallback path. + +## Affected interfaces + +- `AssistantMessageBlock` remains unchanged; promoted images use existing `type: 'image'` and `image_data` fields. +- `AssistantMessageExtra` gains optional metadata keys through its existing index signature, such as `toolCallId`, `toolImagePreviewId`, `toolImagePreviewSource`, and `toolImagePreviewTitle`. +- `RemoteConversationSnapshot.generatedImages` remains unchanged. + +## Data flow + +1. Tool execution returns `imagePreviews`. +2. Runtime normalizes the tool result and calls `prepareToolImagePreviewPresentation()`. +3. Usable previews become assistant `image` blocks inserted after the tool-call block. +4. Desktop conversation renders those images as normal assistant images. +5. Remote snapshot persists those image blocks into `generatedImages`; legacy unpromoted previews are also persisted as fallback. + +## Compatibility + +- Existing generated-image behavior remains compatible: built-in `image_generate` still hides promoted previews from the tool detail panel. +- Saved conversations with only `tool_call.imagePreviews` continue remote delivery via the fallback persistence path. +- Error tool results are not promoted into normal image blocks. + +## Test strategy + +- Update `agentRuntimePresenter/dispatch` tests to assert generic successful tool image previews are promoted into assistant image blocks. +- Keep tests for built-in `image_generate`, MCP same-name tool, and error results aligned with the new promotion rules. +- Keep `RemoteConversationRunner` tests covering fallback persistence from `tool_call.imagePreviews`. +- Run focused tests, typecheck, format, i18n, and lint. diff --git a/docs/issues/remote-tool-result-images/spec.md b/docs/issues/remote-tool-result-images/spec.md new file mode 100644 index 000000000..e709ce3a4 --- /dev/null +++ b/docs/issues/remote-tool-result-images/spec.md @@ -0,0 +1,36 @@ +# Tool result images in conversation and remote delivery + +## User need + +Users expect visual tool results to appear as first-class images in the normal chat transcript and in remote-control channels. Today a tool such as `Page.captureScreenshot` can complete successfully and store the screenshot in `tool_call.imagePreviews`, but the assistant may only continue with text or no final content. The image remains hidden behind the tool-call details and remote channels may not receive it unless the result is converted separately. + +## Goal + +Promote suitable function/tool-call image results into assistant `image` blocks so they are visible in the desktop conversation without depending on the model to restate them. Remote delivery should then reuse the same image blocks and, as a compatibility fallback, still handle unpromoted `tool_call.imagePreviews`. + +## Acceptance criteria + +- Successful `tool_call` results with resolvable `imagePreviews` create assistant `image` blocks adjacent to the tool call. +- `Page.captureScreenshot`, MCP image outputs, file-read image previews, and other non-error tool result images can become visible conversation images. +- The tool-call detail panel may still show preview metadata only when an image cannot be promoted or when the tool result is an error. +- The model context can continue safely without requiring the assistant to output the image itself. +- Remote snapshots deliver promoted image blocks through the existing `generatedImages` path and can still deliver legacy/unpromoted tool result previews. +- Raw base64 is not leaked into normal text messages. + +## Constraints + +- Preserve existing image-generation promotion behavior and compatibility for saved conversations. +- Keep channel-specific remote code unchanged where possible. +- Avoid promoting error tool results as normal assistant images. +- Skip previews without usable image data. + +## Non-goals + +- Changing remote channel APIs or settings. +- Adding live streaming of images before tool completion. +- Sending images from tools that only expose remote HTTP URLs without cached/data payloads. +- Reworking renderer image components. + +## Open questions + +None. diff --git a/docs/issues/remote-tool-result-images/tasks.md b/docs/issues/remote-tool-result-images/tasks.md new file mode 100644 index 000000000..df978f69b --- /dev/null +++ b/docs/issues/remote-tool-result-images/tasks.md @@ -0,0 +1,9 @@ +# Tasks + +- [x] Inspect remote snapshot and channel image delivery flow. +- [x] Document the initial remote fallback issue and implementation plan. +- [x] Persist completed tool-call image previews as remote image assets. +- [x] Re-scope the SDD artifacts to include conversation-level image visibility. +- [x] Promote successful generic tool result image previews into assistant image blocks. +- [x] Update focused tests for generic promotion and remote fallback delivery. +- [x] Run formatter, i18n check/generation, lint, typecheck, and relevant tests. diff --git a/src/main/presenter/agentRuntimePresenter/dispatch.ts b/src/main/presenter/agentRuntimePresenter/dispatch.ts index 52febd5f6..6a4ab6398 100644 --- a/src/main/presenter/agentRuntimePresenter/dispatch.ts +++ b/src/main/presenter/agentRuntimePresenter/dispatch.ts @@ -608,6 +608,7 @@ function applyFinalizedToolResults(params: { } const imagePresentation = prepareToolImagePreviewPresentation({ + toolCallId: stagedResult.toolCallId, toolName: stagedResult.toolName, toolSource: stagedResult.toolSource, serverName: stagedResult.serverName, diff --git a/src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts b/src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts index 8c4a5e42d..5e034a942 100644 --- a/src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts +++ b/src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts @@ -6,6 +6,7 @@ import { } from '@shared/agentImageGenerationTool' export function prepareToolImagePreviewPresentation(params: { + toolCallId?: string toolName: string toolSource?: 'mcp' | 'agent' serverName?: string @@ -15,18 +16,12 @@ export function prepareToolImagePreviewPresentation(params: { toolBlockImagePreviews?: ToolCallImagePreview[] promotedBlocks: AssistantMessageBlock[] } { - const { toolName, toolSource, serverName, isError, imagePreviews } = params + const { toolCallId, toolName, toolSource, serverName, isError, imagePreviews } = params if (!imagePreviews) { return { promotedBlocks: [] } } - if ( - toolName !== IMAGE_GENERATE_TOOL_NAME || - toolSource !== 'agent' || - serverName !== IMAGE_GENERATION_TOOL_SERVER_NAME || - isError || - imagePreviews.length === 0 - ) { + if (isError || imagePreviews.length === 0) { return { toolBlockImagePreviews: imagePreviews, promotedBlocks: [] @@ -35,9 +30,12 @@ export function prepareToolImagePreviewPresentation(params: { const timestamp = Date.now() const promotedBlocks: AssistantMessageBlock[] = [] + const remainingToolBlockImagePreviews: ToolCallImagePreview[] = [] + for (const preview of imagePreviews) { const { data, mimeType } = preview if (!data || !mimeType) { + remainingToolBlockImagePreviews.push(preview) continue } @@ -49,7 +47,14 @@ export function prepareToolImagePreviewPresentation(params: { image_data: { data, mimeType - } + }, + extra: { + ...(toolCallId ? { toolCallId } : {}), + toolName, + ...(preview.id ? { toolImagePreviewId: preview.id } : {}), + toolImagePreviewSource: preview.source, + ...(preview.title ? { toolImagePreviewTitle: preview.title } : {}) + } as AssistantMessageBlock['extra'] }) } @@ -60,8 +65,20 @@ export function prepareToolImagePreviewPresentation(params: { } } + if ( + toolName === IMAGE_GENERATE_TOOL_NAME && + toolSource === 'agent' && + serverName === IMAGE_GENERATION_TOOL_SERVER_NAME + ) { + return { + toolBlockImagePreviews: remainingToolBlockImagePreviews, + promotedBlocks + } + } + return { - toolBlockImagePreviews: [], + toolBlockImagePreviews: + remainingToolBlockImagePreviews.length > 0 ? remainingToolBlockImagePreviews : [], promotedBlocks } } diff --git a/src/main/presenter/agentRuntimePresenter/index.ts b/src/main/presenter/agentRuntimePresenter/index.ts index b11ffabaf..3d1731106 100644 --- a/src/main/presenter/agentRuntimePresenter/index.ts +++ b/src/main/presenter/agentRuntimePresenter/index.ts @@ -1265,6 +1265,7 @@ export class AgentRuntimePresenter implements IAgentImplementation { return { resumed: false } } const imagePresentation = prepareToolImagePreviewPresentation({ + toolCallId: toolCall.id, toolName: toolCall.name || '', toolSource: execution.toolSource, serverName: execution.serverName, diff --git a/src/main/presenter/remoteControlPresenter/services/remoteConversationRunner.ts b/src/main/presenter/remoteControlPresenter/services/remoteConversationRunner.ts index 1665ee03e..6e0bcd645 100644 --- a/src/main/presenter/remoteControlPresenter/services/remoteConversationRunner.ts +++ b/src/main/presenter/remoteControlPresenter/services/remoteConversationRunner.ts @@ -61,6 +61,7 @@ const MIME_EXTENSION: Record = { 'image/gif': '.gif', 'image/bmp': '.bmp', 'image/avif': '.avif', + 'image/svg+xml': '.svg', 'application/pdf': '.pdf', 'text/plain': '.txt' } @@ -72,7 +73,8 @@ const IMAGE_MIME_BY_EXTENSION: Record = { '.webp': 'image/webp', '.gif': 'image/gif', '.bmp': 'image/bmp', - '.avif': 'image/avif' + '.avif': 'image/avif', + '.svg': 'image/svg+xml' } const IMAGE_DATA_URL_PATTERN = /^data:(image\/[a-zA-Z0-9.+-]+);base64,(.*)$/s @@ -196,12 +198,140 @@ const resolveGeneratedImageContent = async ( } } + if (normalizedSource.startsWith('http://') || normalizedSource.startsWith('https://')) { + throw new Error('Remote image URLs must be cached before remote delivery.') + } + return { data: decodeBase64Image(stripDataUrlPrefix(normalizedSource), 'base64'), mimeType: normalizeImageMimeType(fallbackMimeType) ?? 'image/png' } } +type RemoteImageAssetCandidate = { + key: string + data: string + mimeType: string + filenameBase: string + logLabel: string +} + +const createRemoteImageAssetCandidate = (params: { + messageId: string + blockIndex: number + data: string | undefined | null + mimeType: string | undefined | null + filenameBase: string + keySuffix: string + logLabel: string +}): RemoteImageAssetCandidate | null => { + const data = params.data?.trim() + if (!data) { + return null + } + + return { + key: `${params.messageId}:${params.blockIndex}:${params.keySuffix}`, + data, + mimeType: normalizeImageMimeType(params.mimeType) ?? 'image/png', + filenameBase: sanitizePathSegment(params.filenameBase, 'generated'), + logLabel: params.logLabel + } +} + +const buildPromotedPreviewKey = ( + toolCallId: string | undefined, + previewId: string | undefined +): string | null => { + const normalizedPreviewId = previewId?.trim() + if (!normalizedPreviewId) { + return null + } + const normalizedToolCallId = toolCallId?.trim() + return normalizedToolCallId + ? `${normalizedToolCallId}:${normalizedPreviewId}` + : normalizedPreviewId +} + +const collectRemoteImageAssetCandidates = ( + messageId: string, + blocks: AssistantMessageBlock[] +): RemoteImageAssetCandidate[] => { + const candidates: RemoteImageAssetCandidate[] = [] + const promotedPreviewKeys = new Set( + blocks + .filter((candidateBlock) => candidateBlock.type === 'image') + .map((candidateBlock) => + buildPromotedPreviewKey( + typeof candidateBlock.extra?.toolCallId === 'string' + ? candidateBlock.extra.toolCallId + : undefined, + typeof candidateBlock.extra?.toolImagePreviewId === 'string' + ? candidateBlock.extra.toolImagePreviewId + : undefined + ) + ) + .filter((key): key is string => typeof key === 'string' && key.length > 0) + ) + + for (const [blockIndex, block] of blocks.entries()) { + if (block.type === 'image' && block.status !== 'pending') { + const candidate = createRemoteImageAssetCandidate({ + messageId, + blockIndex, + data: block.image_data?.data, + mimeType: block.image_data?.mimeType, + filenameBase: + typeof block.extra?.toolImagePreviewSource === 'string' + ? [ + sanitizePathSegment(block.extra.toolImagePreviewSource, 'tool-result-image'), + blockIndex + 1 + ].join('-') + : `generated-${blockIndex + 1}`, + keySuffix: 'image', + logLabel: 'generated image' + }) + if (candidate) { + candidates.push(candidate) + } + continue + } + + if (block.type !== 'tool_call' || (block.status !== 'success' && block.status !== 'error')) { + continue + } + + const previews = block.tool_call?.imagePreviews ?? [] + for (const [previewIndex, preview] of previews.entries()) { + const previewKey = buildPromotedPreviewKey(block.tool_call?.id, preview.id) + if (previewKey && promotedPreviewKeys.has(previewKey)) { + continue + } + + const filenameSource = preview.source?.trim() || 'tool-result-image' + const filenameBase = [ + sanitizePathSegment(filenameSource, 'tool-result-image'), + blockIndex + 1, + previewIndex + 1 + ].join('-') + const candidate = createRemoteImageAssetCandidate({ + messageId, + blockIndex, + data: preview.data, + mimeType: preview.mimeType, + filenameBase, + keySuffix: `toolResultImage:${previewIndex}`, + logLabel: 'tool result image' + }) + if (candidate) { + candidates.push(candidate) + } + } + } + + return candidates +} + const hasAttachmentDownloadSource = (attachment: RemoteInputAttachment): boolean => Boolean( !attachment.failedDownload && @@ -1038,11 +1168,9 @@ export class RemoteConversationRunner { messageId: string, blocks: AssistantMessageBlock[] ): Promise { - const imageBlocks = blocks - .map((block, index) => ({ block, index })) - .filter(({ block }) => block.type === 'image' && block.status !== 'pending') + const imageCandidates = collectRemoteImageAssetCandidates(messageId, blocks) - if (imageBlocks.length === 0) { + if (imageCandidates.length === 0) { return [] } @@ -1077,19 +1205,15 @@ export class RemoteConversationRunner { } const assets: RemoteGeneratedImageAsset[] = [] - for (const { block, index } of imageBlocks) { - const data = block.image_data?.data?.trim() - if (!data) { - continue - } - + for (const [candidateIndex, candidate] of imageCandidates.entries()) { try { - const imageContent = await resolveGeneratedImageContent( - data, - block.image_data?.mimeType?.trim() || 'image/png' + const imageContent = await resolveGeneratedImageContent(candidate.data, candidate.mimeType) + const filenameBase = sanitizePathSegment( + candidate.filenameBase, + `generated-${candidateIndex + 1}` ) const extension = MIME_EXTENSION[imageContent.mimeType.toLowerCase()] || '.img' - const filename = `generated-${index + 1}${extension}` + const filename = `${filenameBase}${extension}` const filePath = path.join(assetDir, filename) try { @@ -1099,17 +1223,18 @@ export class RemoteConversationRunner { } assets.push({ - key: `${messageId}:${index}:image`, + key: candidate.key, path: filePath, mimeType: imageContent.mimeType, filename, sourceMessageId: messageId }) } catch (error) { - console.warn('[RemoteConversationRunner] Failed to persist generated image:', { + console.warn('[RemoteConversationRunner] Failed to persist remote image asset:', { endpointKey, messageId, - index, + key: candidate.key, + type: candidate.logLabel, error }) } diff --git a/test/main/presenter/agentRuntimePresenter/dispatch.test.ts b/test/main/presenter/agentRuntimePresenter/dispatch.test.ts index d86339c3d..7dba6f192 100644 --- a/test/main/presenter/agentRuntimePresenter/dispatch.test.ts +++ b/test/main/presenter/agentRuntimePresenter/dispatch.test.ts @@ -1487,9 +1487,8 @@ describe('dispatch', () => { expect(io.messageStore.updateAssistantContent).toHaveBeenCalled() }) - it('stores image previews from structured tool output', async () => { + it('promotes image previews from structured tool output into assistant image blocks', async () => { const tools = [makeTool('tool_image')] - const cacheImage = vi.fn(async () => 'imgcache://cached.png') const toolPresenter = { getAllToolDefinitions: vi.fn().mockResolvedValue([]), callTool: vi.fn(async (request) => ({ @@ -1497,7 +1496,20 @@ describe('dispatch', () => { rawData: { toolCallId: request.id, content: [{ type: 'image', data: 'AAAA', mimeType: 'image/png' }], - isError: false + isError: false, + imagePreviews: [ + { + id: 'mcp_image-1', + data: 'imgcache://cached.png', + mimeType: 'image/png', + source: 'mcp_image' + }, + { + id: 'metadata-only', + mimeType: 'image/png', + source: 'mcp_image' + } + ] } })), buildToolSystemPrompt: vi.fn().mockReturnValue('') @@ -1523,20 +1535,33 @@ describe('dispatch', () => { 'full_access', new ToolOutputGuard(), 32000, - 1024, - { cacheImage } + 1024 ) - expect(cacheImage).toHaveBeenCalledWith('data:image/png;base64,AAAA') expect(state.blocks[0].tool_call?.imagePreviews).toEqual([ { - id: 'mcp_image-1', - data: 'imgcache://cached.png', + id: 'metadata-only', mimeType: 'image/png', source: 'mcp_image' } ]) - expect(state.blocks).toHaveLength(1) + expect(state.blocks).toHaveLength(2) + expect(state.blocks[1]).toEqual( + expect.objectContaining({ + type: 'image', + status: 'success', + image_data: { + data: 'imgcache://cached.png', + mimeType: 'image/png' + }, + extra: expect.objectContaining({ + toolCallId: 'tc1', + toolName: 'tool_image', + toolImagePreviewId: 'mcp_image-1', + toolImagePreviewSource: 'mcp_image' + }) + }) + ) }) it('promotes image_generate previews into assistant image blocks', async () => { @@ -1601,12 +1626,19 @@ describe('dispatch', () => { image_data: { data: 'imgcache://generated.png', mimeType: 'image/png' - } + }, + extra: expect.objectContaining({ + toolCallId: 'tc1', + toolName: IMAGE_GENERATE_TOOL_NAME, + toolImagePreviewId: 'generated-image-1', + toolImagePreviewSource: 'tool_output', + toolImagePreviewTitle: 'Generated image 1' + }) }) ) }) - it('does not promote same-name MCP image_generate previews', async () => { + it('promotes same-name MCP image_generate previews into assistant image blocks', async () => { const tools = [makeTool(IMAGE_GENERATE_TOOL_NAME)] const toolPresenter = { getAllToolDefinitions: vi.fn().mockResolvedValue([]), @@ -1652,21 +1684,30 @@ describe('dispatch', () => { 1024 ) - expect(state.blocks).toHaveLength(1) + expect(state.blocks).toHaveLength(2) expect(state.blocks[0]).toEqual( expect.objectContaining({ type: 'tool_call', status: 'success' }) ) - expect(state.blocks[0].tool_call?.imagePreviews).toEqual([ - { - id: 'mcp-generated-image-1', - data: 'imgcache://mcp-generated.png', - mimeType: 'image/png', - source: 'tool_output' - } - ]) + expect(state.blocks[0].tool_call?.imagePreviews).toBeUndefined() + expect(state.blocks[1]).toEqual( + expect.objectContaining({ + type: 'image', + status: 'success', + image_data: { + data: 'imgcache://mcp-generated.png', + mimeType: 'image/png' + }, + extra: expect.objectContaining({ + toolCallId: 'tc1', + toolName: IMAGE_GENERATE_TOOL_NAME, + toolImagePreviewId: 'mcp-generated-image-1', + toolImagePreviewSource: 'tool_output' + }) + }) + ) }) it('does not promote image_generate previews when the tool result is an error', async () => { diff --git a/test/main/presenter/remoteControlPresenter/remoteConversationRunner.test.ts b/test/main/presenter/remoteControlPresenter/remoteConversationRunner.test.ts index 3f5547a05..245a04eee 100644 --- a/test/main/presenter/remoteControlPresenter/remoteConversationRunner.test.ts +++ b/test/main/presenter/remoteControlPresenter/remoteConversationRunner.test.ts @@ -578,7 +578,10 @@ describe('RemoteConversationRunner', () => { const cachedImage = Buffer.from('cached generated image') const dataUrlImage = Buffer.from('data url generated image') const rawBase64Image = Buffer.from('raw base64 generated image') + const screenshotImage = Buffer.from('cached screenshot image') + const toolOutputImage = Buffer.from('tool output image') await fs.writeFile(path.join(cacheDir, 'generated.png'), cachedImage) + await fs.writeFile(path.join(cacheDir, 'screenshot.png'), screenshotImage) vi.mocked(app.getPath).mockImplementation((name: string) => name === 'userData' ? userData : '/mock/path' ) @@ -615,6 +618,58 @@ describe('RemoteConversationRunner', () => { data: rawBase64Image.toString('base64'), mimeType: 'image/webp' } + }, + { + type: 'tool_call', + content: '', + status: 'success', + timestamp: 4, + tool_call: { + id: 'tool-screenshot', + name: 'cdp_send', + params: JSON.stringify({ method: 'Page.captureScreenshot' }), + response: JSON.stringify({ data: 'omitted from text' }), + imagePreviews: [ + { + id: 'screenshot-1', + data: 'imgcache://screenshot.png', + mimeType: 'image/png', + title: 'Page.captureScreenshot', + source: 'screenshot' + }, + { + id: 'tool-output-1', + data: `data:image/png;base64,${toolOutputImage.toString('base64')}`, + mimeType: 'image/png', + source: 'tool_output' + }, + { + id: 'metadata-only', + mimeType: 'image/png', + source: 'tool_output' + } + ] + }, + extra: { + toolCallArgsComplete: true + } + }, + { + type: 'image', + content: '', + status: 'success', + timestamp: 5, + image_data: { + data: 'imgcache://screenshot.png', + mimeType: 'image/png' + }, + extra: { + toolCallId: 'tool-screenshot', + toolName: 'cdp_send', + toolImagePreviewId: 'screenshot-1', + toolImagePreviewSource: 'screenshot', + toolImagePreviewTitle: 'Page.captureScreenshot' + } } ]) } @@ -669,11 +724,23 @@ describe('RemoteConversationRunner', () => { key: 'assistant-images:2:image', mimeType: 'image/webp', filename: 'generated-3.webp' + }), + expect.objectContaining({ + key: 'assistant-images:3:toolResultImage:1', + mimeType: 'image/png', + filename: 'tool_output-4-2.png' + }), + expect.objectContaining({ + key: 'assistant-images:4:image', + mimeType: 'image/png', + filename: 'screenshot-5.png' }) ]) await expect(fs.readFile(snapshot.generatedImages![0].path)).resolves.toEqual(cachedImage) await expect(fs.readFile(snapshot.generatedImages![1].path)).resolves.toEqual(dataUrlImage) await expect(fs.readFile(snapshot.generatedImages![2].path)).resolves.toEqual(rawBase64Image) + await expect(fs.readFile(snapshot.generatedImages![3].path)).resolves.toEqual(toolOutputImage) + await expect(fs.readFile(snapshot.generatedImages![4].path)).resolves.toEqual(screenshotImage) expect(snapshot.finalText).toBe('') })