feat(agent): show tool result images & support send img to remote cha…

[zhangmo8]

…nnel

Summary by CodeRabbit

• Documentation

  • Added specification, implementation plan, and task checklist for improving tool result image visibility in conversations.

• New Features

  • Tool-generated images are now promoted as assistant image blocks in conversations for improved visibility.
  • Enhanced remote image handling to collect and persist images from multiple tool sources.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR implements promotion of tool-call image previews into assistant image blocks for improved conversation visibility and remote delivery. It introduces stable metadata linking promoted images back to originating tool calls, refactors remote asset persistence to collect and deduplicate tool-derived image candidates, and updates test coverage for both promotion and persistence paths.

Changes

Planning and Specification

Layer / File(s)	Summary
Documentation and specification docs/issues/remote-tool-result-images/plan.md, docs/issues/remote-tool-result-images/spec.md, docs/issues/remote-tool-result-images/tasks.md	Introduces specification, implementation plan, and task checklist for the tool image promotion effort, covering desired behavior, constraints, compatibility fallbacks, and validation steps.

Image Promotion and Remote Persistence Implementation

Layer / File(s)	Summary
Image preview promotion logic src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts	prepareToolImagePreviewPresentation now destructures toolCallId from params, narrows early-return gating to only skip promotion for missing/error/empty previews (removing tool-identity checks), accumulates non-promotable previews separately, constructs promoted blocks with tool-call metadata in extra fields, and branches return logic on tool identity to preserve image_generate special handling.
Image promotion call site wiring src/main/presenter/agentRuntimePresenter/dispatch.ts, src/main/presenter/agentRuntimePresenter/index.ts	Both applyFinalizedToolResults and respondToolInteraction now pass toolCallId into prepareToolImagePreviewPresentation, providing the stable metadata needed for promoted block links back to the originating tool call.
Image promotion test coverage test/main/presenter/agentRuntimePresenter/dispatch.test.ts	Updated dispatch test suite to verify structured tool-output image previews are promoted into separate assistant blocks while metadata-only previews remain on tool_call, and to validate promoted block metadata including toolCallId, preview id/source, and preview title for image_generate tools.
Remote image asset candidate collection src/main/presenter/remoteControlPresenter/services/remoteConversationRunner.ts	Extended MIME-to-extension and extension-to-MIME maps to support SVG and AVIF formats, added HTTP(S) validation to reject uncached remote image sources, and introduced internal types and helpers to collect deduplicated image candidates from both assistant image blocks and tool-call preview entries, including deterministic keys for deduplicating promoted previews.
Remote persistence pipeline refactoring src/main/presenter/remoteControlPresenter/services/remoteConversationRunner.ts	persistGeneratedImages now collects remote image asset candidates upfront (exiting early if none exist) instead of iterating raw image blocks, then loops over candidates to resolve content, derive sanitized filenames/extensions from candidate MIME data, persist assets with existence checks, and record persisted metadata using candidate-computed keys and labels for deduplication and logging.
Remote persistence test coverage test/main/presenter/remoteControlPresenter/remoteConversationRunner.test.ts	Extended test to include screenshot.png in generated-images cache, expanded mocked assistant message to include tool_call with imagePreviews (screenshot, tool output, metadata-only) plus a follow-up promoted image block with tool-related extra metadata, and added snapshot expectations for two new generated image entries with file-content verification for the newly persisted assets.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• ThinkInAIXYZ/deepchat#1558: The main PR's updates to tool-call image preview promotion/persistence (e.g., wiring toolCallId into prepareToolImagePreviewPresentation and using tool_call.imagePreviews-derived image candidates in remoteConversationRunner) are directly related to this PR's introduction of the imagePreviews pipeline via extractToolCallImagePreviews and the associated tool_call.imagePreviews types/handling.
• ThinkInAIXYZ/deepchat#1610: Both PRs modify remoteConversationRunner.ts's generated-image delivery/persistence pipeline (this PR changes how tool/image candidates are collected and persisted; the referenced PR changes how generated image content/MIME and asset paths are resolved and written), so they're directly related at the code level.

Poem

🐰 Hops past tool blocks with glee,
Snapshots dance in imagery!
Previews promoted, now they shine—
Conversation blocks align! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main feature additions: showing tool result images and supporting image delivery to remote channels, which align with the documented changes across dispatch, imageGenerationBlocks, remoteConversationRunner, and related tests.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch snapshot-img

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/issues/remote-tool-result-images/plan.md`:
- Line 11: The plan text incorrectly implies that `data` alone suffices for
promoting tool result previews; update the sentence in the
`prepareToolImagePreviewPresentation()` description so it requires both `data`
and `mimeType` (matching the runtime contract enforced in
imageGenerationBlocks.ts around the logic in
`prepareToolImagePreviewPresentation`, lines ~31-38), e.g. state that
successful, non-error tool results with usable `data` AND a valid `mimeType` are
promoted into assistant `image` blocks.

In `@docs/issues/remote-tool-result-images/spec.md`:
- Line 13: Update the wording that currently states "resolvable/usable" for
successful `tool_call` results with `imagePreviews` to explicitly require that
each promotable preview contains both payload `data` and a `mimeType`; change
the sentence describing creation of assistant `image` blocks adjacent to the
`tool_call` so it requires `imagePreviews` entries to include `data` and
`mimeType` (apply the same explicit requirement to the other occurrence
referencing `imagePreviews` on the same spec page).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 60865bef-cbab-4478-8e84-76f00b32bade

📥 Commits

Reviewing files that changed from the base of the PR and between 25f9c54 and accec90.

📒 Files selected for processing (9)

• docs/issues/remote-tool-result-images/plan.md
• docs/issues/remote-tool-result-images/spec.md
• docs/issues/remote-tool-result-images/tasks.md
• src/main/presenter/agentRuntimePresenter/dispatch.ts
• src/main/presenter/agentRuntimePresenter/imageGenerationBlocks.ts
• src/main/presenter/agentRuntimePresenter/index.ts
• src/main/presenter/remoteControlPresenter/services/remoteConversationRunner.ts
• test/main/presenter/agentRuntimePresenter/dispatch.test.ts
• test/main/presenter/remoteControlPresenter/remoteConversationRunner.test.ts