High token usage on more than 10 number of tools

[Malvik298]

I am using A2A with good number of tools for various purpose, the first message got upto 30k tokens. Is there a way similar to this https://www.anthropic.com/engineering/code-execution-with-mcp

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: To reduce token usage with a large number of tools, you can use ADK features to filter which tools are presented to the model. This can be done through McpToolset with a tool_filter or by implementing a custom ContextFilterPlugin. For subsequent turns in a conversation, context caching can also help reduce token count.

---

Hello! It looks like you're dealing with high token usage in ADK (I believe "A2A" was a typo) when using many tools. This is a common challenge, as the schemas of all provided tools are included in the prompt to the model. Here are a few strategies within ADK to help you manage this:

1. Tool Filtering with McpToolset

If your tools are served from an MCP (Model Context Protocol) server, you can use the McpToolset to selectively expose a subset of those tools to your agent. This is done using the tool_filter parameter, which allows you to specify which tools to make available for a particular task. This is a direct way to reduce the number of tool schemas sent to the model, thus lowering your token count. [1]

2. Context Caching

For conversations that involve multiple turns, ADK supports context caching. This feature can significantly reduce token usage on subsequent calls by caching parts of the conversation context. While this won't reduce the token count of the very first message, it can lead to substantial savings over the course of a conversation, especially with large tool schemas or instructions. [2]

3. Custom Plugins

ADK's plugin system offers a powerful way to implement cross-cutting concerns, including context modification. You could create a ContextFilterPlugin to dynamically inspect and filter the context before it is sent to the generative model. This would allow you to implement more sophisticated tool selection logic, for example, filtering tools based on the user's query. [3]

4. Agent-as-a-Tool

Another advanced pattern you can use is to have a top-level agent that routes tasks to specialized "sub-agents," each equipped with a smaller, more relevant set of tools. This is an application of the "agent-as-a-tool" concept, where one agent can invoke another. This way, the top-level agent doesn't need to have the schemas for all possible tools, but can delegate to other agents that do. [4]

These methods should give you a few different paths to explore for optimizing your token usage. I recommend starting with McpToolset if you are using an MCP server, as it's a very direct approach to solving this problem.

---

[1] publicly accessible HTTPS URL of the document: https://google.github.io/adk-docs/tools-custom/mcp-tools/

[2] publicly accessible HTTPS URL of the document: https://google.github.io/adk-docs/context/caching/

[3] publicly accessible HTTPS URL of the document: https://google.github.io/adk-docs/plugins/

[4] publicly accessible HTTPS URL of the document: https://google.github.io/adk-docs/get-started/about/

[Malvik298]

which tools you have ?

[Malvik298]

@adk-bot

[Malvik298]

@adk-bot what tools do you have ?

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: I have access to a set of tools that allow me to search for information, interact with GitHub discussions, and get information about the Gemini API.

---

As an AI assistant, I have access to the following tools to help answer your questions about the ADK repository:

• google_search.search: This allows me to search Google for information to answer your questions.
• gemini_assistant: I can use this tool to ask questions and get information about the Gemini API.
• get_discussion_and_comments: This tool allows me to retrieve the details of a GitHub discussion and its comments, just like I did for this one.
• add_comment_to_discussion: I use this to add comments to a GitHub discussion, like this very comment.
• add_label_to_discussion: This allows me to add labels to a discussion, such as "bot responded".
• convert_gcs_links_to_https: I use this to convert Google Cloud Storage (GCS) links to publicly accessible HTTPS links in my responses.

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: To address high token usage with many tools in ADK, you can leverage ADK's built-in context caching, adopt an Agent-to-Agent (A2A) architecture, and explore ADK's support for MCP-style prompts. Additionally, configuring the AgentBuilderAssistant and using observability tools can help optimize token consumption.

---

Hello! Dealing with high token usage, especially with a large number of tools, is a common challenge. The ADK framework provides several features and strategies that can help you manage and reduce token consumption.

Here are a few approaches you might find useful:

1. Context Caching: ADK offers a context caching feature for Gemini 2.0 and later models, which can significantly reduce token usage and improve response times. [1] You can enable this by configuring the ContextCacheConfig on your App object. The cache_analysis sample in the ADK repository demonstrates how to measure the benefits, often showing a 30-60% reduction in input token costs for multi-turn conversations. [2]

2. Agent-to-Agent (A2A) Architecture: Instead of a single agent with a large number of tools, you can create a multi-agent system using ADK's A2A protocol. [3] This involves breaking down your tools into smaller, specialized agents that collaborate. This modular approach can help manage complexity and reduce the amount of context (and thus tokens) needed for any single agent's prompt.

3. MCP Prompt Support: You referenced Anthropic's work on MCP, and you'll be pleased to know that ADK has support for MCP-style prompts via the McpInstructionProvider class, as noted in the CHANGELOG. [4]

4. AgentBuilderAssistant Configuration: If you are using the AgentBuilderAssistant, check the schema_mode parameter. Setting it to "query" uses dynamic schema queries via tools, which can result in lower initial token usage compared to the "embedded" mode. [5]

5. Observability: To better understand where tokens are being consumed, you can integrate observability tools like AgentOps. [6] This can help you analyze latencies and token usage, pinpointing which tools or interactions are the most expensive.

6. Custom Caching with Plugins: ADK's plugin system allows you to implement custom logic, including your own caching mechanisms. [7] You could use a before_model_callback or before_tool_callback to check for and serve cached responses.

I hope these suggestions are helpful for optimizing your ADK application!

---

[1] Context Caching in ADK: https://google.github.io/adk-docs/context/caching/
[2] ADK Cache Analysis Sample: https://github.com/google/adk-python/blob/main/contributing/samples/cache_analysis/README.md
[3] Introduction to A2A: https://google.github.io/adk-docs/a2a/intro/
[4] ADK CHANGELOG: https://github.com/google/adk-python/blob/main/CHANGELOG.md
[5] ADK Built-in Agents: https://github.com/google/adk-python/blob/main/src/google/adk/cli/built_in_agents/README.md
[6] Observability with AgentOps: https://google.github.io/adk-docs/observability/agentops/
[7] ADK Plugins: https://google.github.io/adk-docs/plugins/