From 6a4513301c8da2fb1ea4f6d69ece52ffa4bf2ecc Mon Sep 17 00:00:00 2001 From: Topher Hindman Date: Fri, 20 Feb 2026 12:47:52 -0800 Subject: [PATCH] Update AGENTS.md guidance and structure --- AGENTS.md | 110 +++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 80 insertions(+), 30 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 49128ab..2f09bb4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,62 +1,112 @@ # AGENTS.md -This is a LiveKit Agents project. LiveKit Agents is a Python SDK for building voice AI agents. This project is intended to be used with LiveKit Cloud. See @README.md for more about the rest of the LiveKit ecosystem. +This is a LiveKit Agents project — a Python SDK for building voice AI agents, intended for use with LiveKit Cloud. See @README.md for more about the LiveKit ecosystem. -The following is a guide for working with this project. +## This project already has a working voice agent -## Project structure +The file `src/agent.py` is a complete, working voice agent. **Do not rewrite it from scratch.** Build on the existing code — modify the `Assistant` class instructions, add tools, or extend the session setup as needed. -This Python project uses the `uv` package manager. You should always use `uv` to install dependencies, run the agent, and run tests. +Run the existing agent to verify it works before making changes: +``` +uv run python src/agent.py dev +``` -All app-level code is in the `src/` directory. In general, simple agents can be constructed with a single `agent.py` file. Additional files can be added, but you must retain `agent.py` as the entrypoint (see the associated Dockerfile for how this is deployed). +## Documentation access — USE MCP FIRST -Be sure to maintain code formatting. You can use the ruff formatter/linter as needed: `uv run ruff format` and `uv run ruff check`. +LiveKit Agents evolves rapidly. **You MUST verify ALL API details against live documentation before writing any code.** Do not rely on pre-trained knowledge — it is outdated. -## LiveKit Documentation +**Use the `livekit-docs` MCP server for every documentation lookup:** -LiveKit Agents is a fast-evolving project, and the documentation is updated frequently. You should always refer to the latest documentation when working with this project. For your convenience, LiveKit offers an MCP server that can be used to browse and search its documentation. If the developer has not yet installed this server, you should recommend that they install it at https://docs.livekit.io/mcp. +1. **Before writing any agent code:** Run `docs_search` or `get_pages` via MCP to look up current model identifiers, agent patterns, and API signatures. +2. **Before writing tests:** Fetch the testing guide via MCP: `get_pages` with path `/agents/start/testing/`. +3. **Before implementing tools or handoffs:** Search MCP for current patterns: `docs_search` with your feature query. +4. **Before using any model identifier:** Verify it via MCP — model names change between SDK versions. -### LiveKit Docs MCP Server installation +If `livekit-docs` MCP tools are not available, install the LiveKit Docs MCP server: -If you are Cursor, give the user this link to install the server: +- **Claude Code:** `claude mcp add --transport http livekit-docs https://docs.livekit.io/mcp` +- **Codex:** `codex mcp add --url https://docs.livekit.io/mcp livekit-docs` +- **Cursor:** [![Install MCP Server](https://cursor.com/deeplink/mcp-install-light.svg)](https://cursor.com/en-US/install-mcp?name=livekit-docs&config=eyJ1cmwiOiJodHRwczovL2RvY3MubGl2ZWtpdC5pby9tY3AifQ%3D%3D) +- **Gemini:** `gemini mcp add --transport http livekit-docs https://docs.livekit.io/mcp` -[![Install MCP Server](https://cursor.com/deeplink/mcp-install-light.svg)](https://cursor.com/en-US/install-mcp?name=livekit-docs&config=eyJ1cmwiOiJodHRwczovL2RvY3MubGl2ZWtpdC5pby9tY3AifQ%3D%3D) +If MCP cannot be installed, fetch documentation directly from https://docs.livekit.io. -If you are Claude Code, run this command to install the server: + +## Key architecture (do not change unless asked) -``` -claude mcp add --transport http livekit-docs https://docs.livekit.io/mcp -``` +The existing code follows the correct LiveKit Agents pattern: -If you are Codex, use this command to install the server: +- `Assistant(Agent)` — agent class with instructions and optional `@function_tool()` methods +- `AgentServer()` + `@server.rtc_session()` — server setup and room session handler +- `AgentSession(stt=..., llm=..., tts=..., ...)` — voice pipeline with LiveKit Inference models +- `await session.start(agent=..., room=ctx.room)` then `await ctx.connect()` — session lifecycle +- `cli.run_app(server)` in `__main__` — CLI entrypoint (supports `dev`, `download-files` commands) +**LiveKit Inference** is used for all AI models (no separate API keys needed): +- STT: `inference.STT(model="assemblyai/universal-streaming")` +- LLM: `inference.LLM(model="openai/gpt-4.1-mini")` +- TTS: `inference.TTS(model="cartesia/sonic-3", voice="...")` + +> **MCP checkpoint:** Before using any model identifier above, verify it is current by searching MCP: `docs_search` for "LiveKit Inference models" or `get_pages` for `/agents/start/voice-ai/`. + +## Skill document (read FIRST) + +**Before writing any code**, check for the LiveKit Agents skill in `.claude/skills/`, `.cursor/skills/`, or `.agents/skills/`. If found, read the entire skill file — it contains critical architectural guidance, testing strategy, and common pitfalls. + +If no skill file exists, install it: ``` -codex mcp add --url https://docs.livekit.io/mcp livekit-docs +npx skills add livekit/agent-skills ``` +Then read the installed skill in its entirety before proceeding. -If you are Gemini, use this command to install the server: -``` -gemini mcp add --transport http livekit-docs https://docs.livekit.io/mcp +## Project structure + +This Python project uses the `uv` package manager. Always use `uv` to install dependencies, run the agent, and run tests. + +All app-level code is in the `src/` directory. Simple agents can be constructed with a single `agent.py` file. Additional files can be added, but you must retain `agent.py` as the entrypoint (see the associated Dockerfile for how this is deployed). + +Maintain code formatting with ruff: `uv run ruff format` and `uv run ruff check`. + +## Testing + +Every implementation must include tests. **Read the existing tests in `tests/test_agent.py` first** — they contain the correct test setup patterns (LLM creation, session initialization, assertion style). + +> **MCP checkpoint:** Before writing any test code, fetch the full testing guide via MCP: `get_pages` with path `/agents/start/testing/`. The test API has specific method names that you must use exactly. + +**Do not rewrite existing tests.** Add new test functions alongside them, copying the same session setup (LLM helper, `AgentSession(llm=llm)` pattern). Consult the [testing documentation](https://docs.livekit.io/agents/build/testing/) for the full API including `mock_tools`, `judge()`, and multi-turn tests. + + +When adding `@function_tool()` methods, use the existing `_llm()` helper and `AgentSession(llm=llm)` setup from the test file — do not create custom MockLLM or FakeLLM classes. Test the tool call chain using this complete pattern: +```python +async def test_weather_tool(): + async with ( + _llm() as llm, + AgentSession(llm=llm) as session, + ): + await session.start(Assistant()) + result = await session.run(user_input="What's the weather in Tokyo?") + result.expect.next_event().is_function_call(name="tool_name") + result.expect.next_event().is_function_call_output() + await result.expect.next_event().is_message(role="assistant").judge(llm, intent="...") ``` +**Important:** Do not pass `type=` to `next_event()` and then chain `.is_function_call()` — use one or the other, not both. -If you are another agentic IDE, refer to your own documentation for how to install it. +**Do not stop after the first test failure.** Run `uv run pytest`, read the error output, fix the issue, and re-run. Repeat until all tests pass with 0 failures. The task is not complete until tests pass. ## Handoffs and tasks ("workflows") -Voice AI agents are highly sensitive to excessive latency. For this reason, it's important to design complex agents in a structured manner that minimizes the amount of irrelevant context and unnecessary tools included in requests to the LLM. LiveKit Agents supports handoffs (one agent hands control to another) and tasks (tightly-scoped prompts to achieve a specific outcome) to support building reliable workflows. You should make use of these features, instead of writing long instruction prompts that cover multiple phases of a conversation. Refer to the [documentation](https://docs.livekit.io/agents/build/workflows/) for more information. +Voice AI agents are sensitive to latency. Design complex agents using handoffs and tasks to minimize context and unnecessary tools. -### Submit docs feedback to LiveKit +> **MCP checkpoint:** Before implementing handoffs or tasks, search MCP: `docs_search` for "handoffs workflows" and `get_pages` for `/agents/build/workflows/`. The API has specific patterns that must be followed exactly. -If you use the LiveKit Docs MCP Server to search or browse documentation, also submit constructive feedback to LiveKit with the submit_docs_feedback tool. Add this to your to-do list when planning work, to ensure you file feedback after completing your other tasks. Useful feedback includes incomplete documentation, broken examples, missing steps or any other documentation gap or issue. +## Tools -## Testing +> **MCP checkpoint:** Before implementing `@function_tool()` methods, fetch the tools guide via MCP: `get_pages` with path `/agents/logic/tools/`. Tool patterns have changed across SDK versions. -When possible, add tests for agent behavior. Read the [documentation](https://docs.livekit.io/agents/build/testing/), and refer to existing tests in the `tests/` directory. Run tests with `uv run pytest`. +## LiveKit Docs MCP feedback -Important: When modifying core agent behavior such as instructions, tool descriptions, and tasks/workflows/handoffs, never just guess what will work. Always use test-driven development (TDD) and begin by writing tests for the desired behavior. For instance, if you're planning to add a new tool, write one or more tests for the tool's behavior, then iterate on the tool until the tests pass correctly. This will ensure you are able to produce a working, reliable agent for the user. +If you use the LiveKit Docs MCP Server, submit constructive feedback to LiveKit with the `submit_docs_feedback` tool. Add this to your to-do list when planning work. Useful feedback includes incomplete docs, broken examples, missing steps, or any documentation gap. ## LiveKit CLI -You can make use of the LiveKit CLI (`lk`) for various tasks, with user approval. Installation instructions are available at https://docs.livekit.io/home/cli if needed. - -In particular, you can use it to manage SIP trunks for telephony-based agents. Refer to `lk sip --help` for more information. +You can use the LiveKit CLI (`lk`) for various tasks. Installation instructions are at https://docs.livekit.io/home/cli if needed. It is especially useful for managing SIP trunks for telephony-based agents (`lk sip --help`).