Skip to content

Support MCP observability for Envoy AI Gateway#13791

Open
wu-sheng wants to merge 1 commit intomasterfrom
feature/mcp-observability
Open

Support MCP observability for Envoy AI Gateway#13791
wu-sheng wants to merge 1 commit intomasterfrom
feature/mcp-observability

Conversation

@wu-sheng
Copy link
Copy Markdown
Member

@wu-sheng wu-sheng commented Apr 4, 2026

Support MCP (Model Context Protocol) observability for Envoy AI Gateway

  • If this is non-trivial feature, paste the links/URLs to the design doc.
  • Update the documentation to include this new feature.
  • Tests(including UT, IT, E2E) are added to verify the new feature.
  • If it's UI related, attach the screenshots below.

Changes

MAL rules (new files):

  • gateway-mcp-service.yaml — 13 MCP service-level metrics (request CPM/latency/percentile, method CPM, error CPM, initialization latency, capabilities, per-backend breakdown)
  • gateway-mcp-instance.yaml — 13 MCP instance-level metrics

LAL rules (modified envoy-ai-gateway.yaml):

  • Split into two rules: envoy-ai-gateway-llm-access-log and envoy-ai-gateway-mcp-access-log
  • LLM logs: persist error responses (>= 400) and upstream failures only
  • MCP logs: persist error responses (>= 400) only
  • Both rules tag ai_route_type (llm or mcp) for searchable filtering

Dashboard (modified service + instance JSON):

  • Added MCP tab with 9 widgets (service) / 6 widgets (instance): request CPM, latency avg/percentile, error CPM, method CPM, initialization latency, backend breakdown

E2E test (modified):

  • Added mcp-server service (tzolov/mcp-everything-server:v3 — MCP reference server with StreamableHttp)
  • Added MCP request steps (initialize + tools/list + tools/call)
  • Added MCP metric verification cases
  • Log query uses ai_route_type=llm tag filter

Config:

  • Added ai_route_type to searchableLogsTags in application.yml

  • Fixed aigw healthcheck binary path (/app instead of aigw)

  • If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.

  • Update the CHANGES log.

@wu-sheng wu-sheng added backend OAP backend related. enhancement Enhancement on performance or codes labels Apr 4, 2026
@wu-sheng wu-sheng added this to the 10.5.0 milestone Apr 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend OAP backend related. enhancement Enhancement on performance or codes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant