Skip to content

Add multimodal support for CodeAgent#875

Draft
codegen-sh[bot] wants to merge 2 commits intodevelopfrom
gen/2ee96ac8-1dd6-41a4-a10c-6b69d0898240
Draft

Add multimodal support for CodeAgent#875
codegen-sh[bot] wants to merge 2 commits intodevelopfrom
gen/2ee96ac8-1dd6-41a4-a10c-6b69d0898240

Conversation

@codegen-sh
Copy link
Contributor

@codegen-sh codegen-sh bot commented Mar 17, 2025

Description

This PR adds multimodal support to the CodeAgent, allowing it to process images in Slack messages. This is the second part of the implementation, building on PR #11444 in the cloud repository which adds the ability to capture image information from Slack messages.

Changes

  1. Updated the LLM class to support multimodal content:

    • Added a _process_messages_for_multimodal method that converts image URLs in the format [Image: filename](URL) to the appropriate multimodal format for Claude 3 models
    • Modified the _generate method to process messages for multimodal content when using Claude 3 models
  2. Updated the CodeAgent class to use multimodal models by default:

    • Added a multimodal parameter (default: True) to the constructor
    • When multimodal is enabled, ensures a Claude 3 model is used even if a non-multimodal model was specified

How to Test

  1. Send a message with an image in a Slack thread
  2. Mention the CodeAgent bot
  3. Verify that the bot can see and process the image in its response

Related PRs

This PR builds on PR #11444 in the cloud repository, which adds the ability to capture image information from Slack messages.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


codegen-bot seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant