Skip to content

Add TodoListMiddleware for code review planning#5813

Merged
suhaibmujahid merged 2 commits into
mozilla:masterfrom
suhaibmujahid:agent-to-do
May 14, 2026
Merged

Add TodoListMiddleware for code review planning#5813
suhaibmujahid merged 2 commits into
mozilla:masterfrom
suhaibmujahid:agent-to-do

Conversation

@suhaibmujahid

Copy link
Copy Markdown
Member

Resolves #5812

Integrate TodoListMiddleware into the code review agent to enable tracking of investigation tasks during reviews.

@suhaibmujahid suhaibmujahid requested a review from Copilot March 17, 2026 21:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@JiwaniZakir JiwaniZakir left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The import from langchain.agents.middleware import TodoListMiddleware is worth scrutinizing — langchain.agents.middleware does not appear to be a documented or stable LangChain module, and there's no pinned version or changelog reference to confirm this API exists. If this is a custom internal class or a very recent addition, it should either be vendored locally or accompanied by a minimum version constraint in the project's dependencies to prevent silent import failures.

The CODE_REVIEW_TODO_TOOL_DESCRIPTION in prompts.py mentions expand_context and find_function_definition by name as examples of "tool-based verification," which is a useful constraint — but the CODE_REVIEW_TODO_PROMPT doesn't echo this same framing. Since the prompt is what the model actually sees at runtime, it would be more effective to include those concrete tool examples there too, rather than only in the tool description that the model may see in a different context window position.

There are no tests covering the new middleware integration — at minimum, a unit test asserting that the agent is constructed with the TodoListMiddleware instance (checking middleware kwarg contents) would catch regressions if the create_agent signature changes. Given that middleware appears to be a new parameter path, this seems like a reasonable place to guard.

Integrate TodoListMiddleware into the code review agent to enable tracking of investigation tasks during reviews.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comment thread services/reviewhelper-api/app/config.py
@suhaibmujahid suhaibmujahid marked this pull request as ready for review May 7, 2026 21:41
@suhaibmujahid suhaibmujahid requested a review from marco-c May 7, 2026 21:41
@suhaibmujahid suhaibmujahid enabled auto-merge (rebase) May 7, 2026 21:42
Comment thread bugbug/tools/code_review/prompts.py

@marco-c marco-c left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you run validation? What is the impact?

@suhaibmujahid

Copy link
Copy Markdown
Member Author

Did you run validation? What is the impact?

Yes, I see improved numbers. Though the experiment itself in not perfect.

Where able to complete with fewer errors. Generated 3 time more comments, which yielded higher recall. With the capabilities that we are planning to add, the to-do will be valuable to improve things further.

image

@suhaibmujahid suhaibmujahid merged commit 59bc09d into mozilla:master May 14, 2026
6 checks passed
@suhaibmujahid suhaibmujahid deleted the agent-to-do branch May 14, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Give the agent a TODO list tool that it uses to plan its review and then follow that plan step by step

4 participants