From d2193d42267bfdd30aace6f33c2757c7441c3b28 Mon Sep 17 00:00:00 2001 From: James Grugett Date: Thu, 26 Mar 2026 11:28:59 -0700 Subject: [PATCH 1/3] Refactor knowledge.md into AGENTS.md + docs/ directory Move detailed documentation from knowledge.md into dedicated files under docs/, keeping AGENTS.md as a lightweight table of contents. Co-Authored-By: Claude Opus 4.6 --- AGENTS.md | 38 +++++++++ docs/agents-and-tools.md | 24 ++++++ docs/development.md | 44 +++++++++++ docs/environment-variables.md | 28 +++++++ docs/error-handling.md | 5 ++ docs/git-guidelines.md | 5 ++ docs/testing.md | 11 +++ knowledge.md | 141 ---------------------------------- 8 files changed, 155 insertions(+), 141 deletions(-) create mode 100644 AGENTS.md create mode 100644 docs/agents-and-tools.md create mode 100644 docs/development.md create mode 100644 docs/environment-variables.md create mode 100644 docs/error-handling.md create mode 100644 docs/git-guidelines.md create mode 100644 docs/testing.md delete mode 100644 knowledge.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..090d7e5263 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,38 @@ +# Codebuff + +Codebuff is a tool for editing codebases via natural-language instructions to Buffy (an expert AI programming assistant). + +## Goals + +- Make expert engineers faster (power-user focus). +- Reduce time/effort for common programming tasks. +- Improve via iteration/feedback (learn/adapt from usage). + +## Key Technologies + +- TypeScript monorepo (Bun workspaces) +- Bun runtime + package manager +- Next.js (web app + API routes) +- Multiple LLM providers (Anthropic/OpenAI/Gemini/etc.) + +## Repo Map + +- `cli/` — TUI client (OpenTUI + React) and local UX +- `sdk/` — JS/TS SDK used by the CLI and external users +- `web/` — Next.js app + API routes (the "web API") +- `packages/agent-runtime/` — agent runtime + tool handling (server-side) +- `common/` — shared types, tools, schemas, utilities +- `agents/` — main agents shipped with codebuff +- `.agents/` — local agent templates (prompt + programmatic agents) + +## Docs + +- [`docs/architecture.md`](docs/architecture.md) — Package dependency graph, per-package details, key architectural patterns +- [`docs/request-flow.md`](docs/request-flow.md) — Full request lifecycle from CLI through server and back +- [`docs/error-schema.md`](docs/error-schema.md) — Server error response formats and client-side handling +- [`docs/error-handling.md`](docs/error-handling.md) — ErrorOr pattern for return values +- [`docs/development.md`](docs/development.md) — Dev setup, worktrees, logs, package management, DB migrations +- [`docs/testing.md`](docs/testing.md) — Testing conventions, DI over mocking, tmux CLI testing +- [`docs/environment-variables.md`](docs/environment-variables.md) — Env var rules, DI helpers, loading order +- [`docs/agents-and-tools.md`](docs/agents-and-tools.md) — Agent system, shell shims, tool definitions, referral system +- [`docs/git-guidelines.md`](docs/git-guidelines.md) — Git safety rules diff --git a/docs/agents-and-tools.md b/docs/agents-and-tools.md new file mode 100644 index 0000000000..9441c7ab05 --- /dev/null +++ b/docs/agents-and-tools.md @@ -0,0 +1,24 @@ +# Agents and Tools + +## Agents + +- Prompt/programmatic agents live in `.agents/` (programmatic agents use `handleSteps` generators). +- Generator functions execute in a sandbox; agent templates define tool access and subagents. + +### Shell Shims + +Direct commands without `codebuff` prefix: + +```bash +codebuff shims install codebuff/base-lite@1.0.0 +eval "$(codebuff shims env)" +base-lite "fix this bug" +``` + +## Tools + +- Tool definitions live in `common/src/tools` and are executed via the SDK helpers + agent-runtime. + +## Referral System + +Referral codes are applied via the CLI (web onboarding only instructs the user); see `web/src/app/api/referrals/helpers.ts`. diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000000..b9d41ef486 --- /dev/null +++ b/docs/development.md @@ -0,0 +1,44 @@ +# Development + +## Getting Started + +Start the web server first: + +```bash +bun up +``` + +Then start the CLI separately: + +```bash +bun start-cli +``` + +Other service commands: + +```bash +bun ps # check running services +bun down # stop services +``` + +## Worktrees + +To run multiple stacks on different ports, create `.env.development.local`: + +```bash +PORT=3001 +NEXT_PUBLIC_WEB_PORT=3001 +NEXT_PUBLIC_CODEBUFF_APP_URL=http://localhost:3001 +``` + +## Logs + +Logs are in `debug/console/` (`db.log`, `studio.log`, `sdk.log`, `web.log`). + +## Package Management + +- Use `bun install`, `bun run ...` (avoid `npm`). + +## Database Migrations + +Edit schema using Drizzle's TS DSL (don't hand-write migration SQL), then run the internal DB scripts to generate/apply migrations. diff --git a/docs/environment-variables.md b/docs/environment-variables.md new file mode 100644 index 0000000000..6514dba0f1 --- /dev/null +++ b/docs/environment-variables.md @@ -0,0 +1,28 @@ +# Environment Variables + +## Quick Rules + +- Public client env: `NEXT_PUBLIC_*` only, validated in `common/src/env-schema.ts` (used via `@codebuff/common/env`). +- Server secrets: validated in `packages/internal/src/env-schema.ts` (used via `@codebuff/internal/env`). +- Runtime/OS env: pass typed snapshots instead of reading `process.env` throughout the codebase. + +## Env DI Helpers + +- Base contracts: `common/src/types/contracts/env.ts` (`BaseEnv`, `BaseCiEnv`, `ClientEnv`, `CiEnv`) +- Helpers: `common/src/env-process.ts`, `common/src/env-ci.ts` +- Test helpers: `common/src/testing-env-process.ts`, `common/src/testing-env-ci.ts` +- CLI: `cli/src/utils/env.ts` (`getCliEnv`) +- CLI test helpers: `cli/src/testing/env.ts` (`createTestCliEnv`) +- SDK: `sdk/src/env.ts` (`getSdkEnv`) +- SDK test helpers: `sdk/src/testing/env.ts` (`createTestSdkEnv`) + +## Loading Order + +Bun loads (highest precedence last): + +- `.env.local` (Infisical-synced secrets, gitignored) +- `.env.development.local` (worktree overrides like ports, gitignored) + +## Releases + +Release scripts read `CODEBUFF_GITHUB_TOKEN`. diff --git a/docs/error-handling.md b/docs/error-handling.md new file mode 100644 index 0000000000..9c03872082 --- /dev/null +++ b/docs/error-handling.md @@ -0,0 +1,5 @@ +# Error Handling + +Prefer `ErrorOr` return values (`success(...)`/`failure(...)` in `common/src/util/error.ts`) over throwing. + +See [Error Schema](./error-schema.md) for the full server error response format and client-side handling. diff --git a/docs/git-guidelines.md b/docs/git-guidelines.md new file mode 100644 index 0000000000..df7c6d3f1f --- /dev/null +++ b/docs/git-guidelines.md @@ -0,0 +1,5 @@ +# Git Safety Rules + +- Never force-push `main` unless explicitly requested. +- To exclude files from a commit: stage only what you want (`git add `). Never use `git restore`/`git checkout HEAD -- ` to "uncommit" changes. +- Run interactive git commands in tmux (anything that opens an editor or prompts). diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 0000000000..dcc8ee4e72 --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,11 @@ +# Testing + +- Prefer dependency injection over module mocking; define contracts in `common/src/types/contracts/`. +- Use `spyOn()` only for globals / legacy seams. +- Avoid `mock.module()` for functions; use `@codebuff/common/testing/mock-modules.ts` helpers for constants only. + +CLI hook testing note: React 19 + Bun + RTL `renderHook()` is unreliable; prefer integration tests via components for hook behavior. + +## CLI tmux Testing + +For testing CLI behavior via tmux, use the helper scripts in `scripts/tmux/`. These handle bracketed paste mode and session logging automatically. Session data is saved to `debug/tmux-sessions/` in YAML format and can be viewed with `bun scripts/tmux/tmux-viewer/index.tsx`. See `scripts/tmux/README.md` for details. diff --git a/knowledge.md b/knowledge.md deleted file mode 100644 index e36f194f92..0000000000 --- a/knowledge.md +++ /dev/null @@ -1,141 +0,0 @@ -# Codebuff - -Codebuff is a tool for editing codebases via natural-language instructions to Buffy (an expert AI programming assistant). - -## Goals - -- Make expert engineers faster (power-user focus). -- Reduce time/effort for common programming tasks. -- Improve via iteration/feedback (learn/adapt from usage). - -## Key Technologies - -- TypeScript monorepo (Bun workspaces) -- Bun runtime + package manager -- Next.js (web app + API routes) -- Multiple LLM providers (Anthropic/OpenAI/Gemini/etc.) - -## Repo Map - -- `cli/`: TUI client (OpenTUI + React) and local UX -- `sdk/`: JS/TS SDK used by the CLI and external users -- `web/`: Next.js app + API routes (the “web API”) -- `packages/agent-runtime/`: agent runtime + tool handling (server-side) -- `common/`: shared types, tools, schemas, utilities -- `agents/`: main agents shipped with codebuff -- `.agents/`: local agent templates (prompt + programmatic agents) - -## Request Flow - -1. CLI/SDK sends user input + context to the Codebuff web API. -2. Agent runtime streams events/chunks back through SDK callbacks. -3. Tools execute locally (file edits, terminal commands, search) to satisfy tool calls. - -## Development - -Start the web server first: - -```bash -bun up -``` - -Then start the CLI separately: - -```bash -bun start-cli -``` - -Other service commands: - -```bash -bun ps # check running services -bun down # stop services -``` - -Worktrees (run multiple stacks on different ports): create `.env.development.local`: - -```bash -PORT=3001 -NEXT_PUBLIC_WEB_PORT=3001 -NEXT_PUBLIC_CODEBUFF_APP_URL=http://localhost:3001 -``` - -Logs: `debug/console/` (`db.log`, `studio.log`, `sdk.log`, `web.log`). - -Package management: - -- Use `bun install`, `bun run ...` (avoid `npm`). - -## Agents And Tools - -Agents: - -- Prompt/programmatic agents live in `.agents/` (programmatic agents use `handleSteps` generators). -- Generator functions execute in a sandbox; agent templates define tool access and subagents. - -Shell shims (direct commands without `codebuff` prefix): - -```bash -codebuff shims install codebuff/base-lite@1.0.0 -eval "$(codebuff shims env)" -base-lite "fix this bug" -``` - -Tools: - -- Tool definitions live in `common/src/tools` and are executed via the SDK helpers + agent-runtime. - -## Git Safety Rules - -- Never force-push `main` unless explicitly requested. -- To exclude files from a commit: stage only what you want (`git add `). Never use `git restore`/`git checkout HEAD -- ` to “uncommit” changes. -- Run interactive git commands in tmux (anything that opens an editor or prompts). - -## Error Handling - -Prefer `ErrorOr` return values (`success(...)`/`failure(...)` in `common/src/util/error.ts`) over throwing. - -## Testing - -- Prefer dependency injection over module mocking; define contracts in `common/src/types/contracts/`. -- Use `spyOn()` only for globals / legacy seams. -- Avoid `mock.module()` for functions; use `@codebuff/common/testing/mock-modules.ts` helpers for constants only. - -CLI hook testing note: React 19 + Bun + RTL `renderHook()` is unreliable; prefer integration tests via components for hook behavior. - -### CLI tmux Testing - -For testing CLI behavior via tmux, use the helper scripts in `scripts/tmux/`. These handle bracketed paste mode and session logging automatically. Session data is saved to `debug/tmux-sessions/` in YAML format and can be viewed with `bun scripts/tmux/tmux-viewer/index.tsx`. See `scripts/tmux/README.md` for details. - -## Environment Variables - -Quick rules: - -- Public client env: `NEXT_PUBLIC_*` only, validated in `common/src/env-schema.ts` (used via `@codebuff/common/env`). -- Server secrets: validated in `packages/internal/src/env-schema.ts` (used via `@codebuff/internal/env`). -- Runtime/OS env: pass typed snapshots instead of reading `process.env` throughout the codebase. - -Env DI helpers: - -- Base contracts: `common/src/types/contracts/env.ts` (`BaseEnv`, `BaseCiEnv`, `ClientEnv`, `CiEnv`) -- Helpers: `common/src/env-process.ts`, `common/src/env-ci.ts` -- Test helpers: `common/src/testing-env-process.ts`, `common/src/testing-env-ci.ts` -- CLI: `cli/src/utils/env.ts` (`getCliEnv`) -- CLI test helpers: `cli/src/testing/env.ts` (`createTestCliEnv`) -- SDK: `sdk/src/env.ts` (`getSdkEnv`) -- SDK test helpers: `sdk/src/testing/env.ts` (`createTestSdkEnv`) - -Bun loads (highest precedence last): - -- `.env.local` (Infisical-synced secrets, gitignored) -- `.env.development.local` (worktree overrides like ports, gitignored) - -Releases: release scripts read `CODEBUFF_GITHUB_TOKEN`. - -## Database Migrations - -Edit schema using Drizzle’s TS DSL (don’t hand-write migration SQL), then run the internal DB scripts to generate/apply migrations. - -## Referral System - -Referral codes are applied via the CLI (web onboarding only instructs the user); see `web/src/app/api/referrals/helpers.ts`. From c50cec7d115a2a6b957e05545e5b2724ecdcace2 Mon Sep 17 00:00:00 2001 From: James Grugett Date: Thu, 26 Mar 2026 11:30:08 -0700 Subject: [PATCH 2/3] Fold thin docs back into AGENTS.md, restore request flow summary error-handling.md (2 lines) and git-guidelines.md (3 bullets) were too thin for their own files. Moved them into a Conventions section in AGENTS.md alongside the referral system note. Restored the brief request flow summary that was in the original knowledge.md. Co-Authored-By: Claude Opus 4.6 --- AGENTS.md | 22 +++++++++++++++++----- docs/agents-and-tools.md | 3 --- docs/error-handling.md | 5 ----- docs/git-guidelines.md | 5 ----- 4 files changed, 17 insertions(+), 18 deletions(-) delete mode 100644 docs/error-handling.md delete mode 100644 docs/git-guidelines.md diff --git a/AGENTS.md b/AGENTS.md index 090d7e5263..f6ff940ef9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,14 +25,26 @@ Codebuff is a tool for editing codebases via natural-language instructions to Bu - `agents/` — main agents shipped with codebuff - `.agents/` — local agent templates (prompt + programmatic agents) +## Request Flow + +1. CLI/SDK sends user input + context to the Codebuff web API. +2. Agent runtime streams events/chunks back through SDK callbacks. +3. Tools execute locally (file edits, terminal commands, search) to satisfy tool calls. + +## Conventions + +- Prefer `ErrorOr` return values (`success(...)`/`failure(...)` in `common/src/util/error.ts`) over throwing. +- Never force-push `main` unless explicitly requested. +- To exclude files from a commit: stage only what you want (`git add `). Never use `git restore`/`git checkout HEAD -- ` to "uncommit" changes. +- Run interactive git commands in tmux (anything that opens an editor or prompts). +- Referral codes are applied via the CLI (web onboarding only instructs the user); see `web/src/app/api/referrals/helpers.ts`. + ## Docs -- [`docs/architecture.md`](docs/architecture.md) — Package dependency graph, per-package details, key architectural patterns +- [`docs/architecture.md`](docs/architecture.md) — Package dependency graph, per-package details, architectural patterns - [`docs/request-flow.md`](docs/request-flow.md) — Full request lifecycle from CLI through server and back - [`docs/error-schema.md`](docs/error-schema.md) — Server error response formats and client-side handling -- [`docs/error-handling.md`](docs/error-handling.md) — ErrorOr pattern for return values - [`docs/development.md`](docs/development.md) — Dev setup, worktrees, logs, package management, DB migrations -- [`docs/testing.md`](docs/testing.md) — Testing conventions, DI over mocking, tmux CLI testing +- [`docs/testing.md`](docs/testing.md) — DI over mocking, tmux CLI testing - [`docs/environment-variables.md`](docs/environment-variables.md) — Env var rules, DI helpers, loading order -- [`docs/agents-and-tools.md`](docs/agents-and-tools.md) — Agent system, shell shims, tool definitions, referral system -- [`docs/git-guidelines.md`](docs/git-guidelines.md) — Git safety rules +- [`docs/agents-and-tools.md`](docs/agents-and-tools.md) — Agent system, shell shims, tool definitions diff --git a/docs/agents-and-tools.md b/docs/agents-and-tools.md index 9441c7ab05..4ea7475896 100644 --- a/docs/agents-and-tools.md +++ b/docs/agents-and-tools.md @@ -19,6 +19,3 @@ base-lite "fix this bug" - Tool definitions live in `common/src/tools` and are executed via the SDK helpers + agent-runtime. -## Referral System - -Referral codes are applied via the CLI (web onboarding only instructs the user); see `web/src/app/api/referrals/helpers.ts`. diff --git a/docs/error-handling.md b/docs/error-handling.md deleted file mode 100644 index 9c03872082..0000000000 --- a/docs/error-handling.md +++ /dev/null @@ -1,5 +0,0 @@ -# Error Handling - -Prefer `ErrorOr` return values (`success(...)`/`failure(...)` in `common/src/util/error.ts`) over throwing. - -See [Error Schema](./error-schema.md) for the full server error response format and client-side handling. diff --git a/docs/git-guidelines.md b/docs/git-guidelines.md deleted file mode 100644 index df7c6d3f1f..0000000000 --- a/docs/git-guidelines.md +++ /dev/null @@ -1,5 +0,0 @@ -# Git Safety Rules - -- Never force-push `main` unless explicitly requested. -- To exclude files from a commit: stage only what you want (`git add `). Never use `git restore`/`git checkout HEAD -- ` to "uncommit" changes. -- Run interactive git commands in tmux (anything that opens an editor or prompts). From 952dfa1def35f139ef9c93ecd3cd2e4af9e758d5 Mon Sep 17 00:00:00 2001 From: James Grugett Date: Thu, 26 Mar 2026 11:42:28 -0700 Subject: [PATCH 3/3] Fix transaction test crashing on db/index.ts import MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The test imports db/index.ts which eagerly calls postgres() and drizzle() at the top level. In CI, the dummy DATABASE_URL produces a client without options.parsers/serializers, crashing drizzle's construct(). Mock both postgres and @codebuff/internal/env before importing to avoid the real db initialization — the test only ever spies on db.transaction anyway. Co-Authored-By: Claude Opus 4.6 --- .../src/db/__tests__/transaction.test.ts | 23 ++++++++++++++----- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/packages/internal/src/db/__tests__/transaction.test.ts b/packages/internal/src/db/__tests__/transaction.test.ts index 7f66e034eb..968e85a084 100644 --- a/packages/internal/src/db/__tests__/transaction.test.ts +++ b/packages/internal/src/db/__tests__/transaction.test.ts @@ -3,14 +3,25 @@ import { AnalyticsEvent } from '@codebuff/common/constants/analytics-events' import { createPostgresError } from '@codebuff/common/testing/errors' import { afterEach, beforeEach, describe, expect, it, mock, spyOn } from 'bun:test' -import * as dbModule from '../index' -import { - getRetryableErrorDescription, - isRetryablePostgresError, -} from '../transaction' - import type { Logger } from '@codebuff/common/types/contracts/logger' +// Mock postgres and env before any module that imports db/index.ts is loaded. +// db/index.ts calls postgres(env.DATABASE_URL) and drizzle() at the top level, +// which fails without real env vars / DB. These tests only need db.transaction (spied). +mock.module('postgres', () => ({ + default: () => ({ + options: { parsers: {}, serializers: {} }, + }), +})) +mock.module('@codebuff/internal/env', () => ({ + env: { DATABASE_URL: 'postgres://mock:mock@localhost:5432/mock' }, +})) + +// Now safe to import modules that depend on db/index.ts +const dbModule = await import('../index') +const { getRetryableErrorDescription, isRetryablePostgresError } = + await import('../transaction') + describe('transaction error handling', () => { describe('getRetryableErrorDescription', () => { describe('Class 40 — Transaction Rollback errors', () => {