Skip to content

Competitive feature parity: embeddings, structured output, sequential responses, Docker#42

Merged
jpr5 merged 7 commits intomainfrom
feat/competitive-parity
Mar 20, 2026
Merged

Competitive feature parity: embeddings, structured output, sequential responses, Docker#42
jpr5 merged 7 commits intomainfrom
feat/competitive-parity

Conversation

@jpr5
Copy link
Contributor

@jpr5 jpr5 commented Mar 19, 2026

Summary

Adds four high-value features that fill real gaps relative to VidaiMock, dwmkerr/mock-llm, and piyook/llm-mock:

  • Embeddings endpoint (POST /v1/embeddings) — deterministic embedding generation via SHA-256 hash, fixture matching on inputText, onEmbedding() convenience, automatic fallback to deterministic embeddings when no fixture matches
  • Structured output / JSON moderesponseFormat matching on response_format.type, onJsonOutput() convenience, validation that JSON-mode fixtures contain valid JSON
  • Sequential/stateful responsessequenceIndex on fixtures enables multi-turn sequences (first call → tool call, second call → text), per-fixture match counting in Journal with sibling propagation
  • Docker + Helm — multi-stage Dockerfile (node:22-alpine), GHCR publish workflow (linux/amd64+arm64), basic Helm chart with deployment/service templates

Also includes:

  • Documentation website — 14 static HTML pages in docs/ covering all features with unit test and integration test examples
  • Updated competitive matrix — added VidaiMock, mock-llm, piyook/llm-mock as columns + 7 new feature rows
  • Embeddings drift test — three-way SDK/real/mock shape comparison for OpenAI embeddings
  • 109 new tests (658 total, up from 549)

Test Plan

  • pnpm test — 658 tests passing
  • pnpm run build — clean
  • pnpm run format:check — clean
  • pnpm run lint — clean
  • Full 6-agent code review loop — 2 rounds, zero actionable feedback in Round 2
  • pnpm test:drift — drift tests (requires API keys)
  • docker build -t llmock . — Docker build verification

@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 19, 2026

Open in StackBlitz

npm i https://pkg.pr.new/CopilotKit/llmock/@copilotkit/llmock@42

commit: c1bb46e

@jpr5 jpr5 force-pushed the feat/competitive-parity branch from c1bb46e to c9f1a21 Compare March 19, 2026 19:36
@jpr5 jpr5 changed the base branch from main to fix/drift-remediation March 19, 2026 19:36
@jpr5 jpr5 force-pushed the feat/competitive-parity branch 2 times, most recently from 3a4f226 to 10ea4b2 Compare March 19, 2026 21:24
@jpr5 jpr5 force-pushed the fix/drift-remediation branch 2 times, most recently from aad3d74 to 9597d18 Compare March 19, 2026 22:55
@jpr5 jpr5 force-pushed the feat/competitive-parity branch from 10ea4b2 to 20cffb0 Compare March 19, 2026 23:11
@jpr5 jpr5 force-pushed the fix/drift-remediation branch from 9597d18 to 162cb80 Compare March 19, 2026 23:23
@jpr5 jpr5 force-pushed the feat/competitive-parity branch from 20cffb0 to 272918d Compare March 19, 2026 23:24
@jpr5 jpr5 force-pushed the fix/drift-remediation branch 3 times, most recently from e3f117d to 4b6f190 Compare March 19, 2026 23:34
@jpr5 jpr5 force-pushed the feat/competitive-parity branch from 272918d to e3590d1 Compare March 19, 2026 23:38
jpr5 added a commit that referenced this pull request Mar 19, 2026
- fixture-loader: add isEmbeddingResponse to validateFixtures type check
  and add embedding-specific validation (empty array, non-number elements)
- drift-report-collector: add "OpenAI Embeddings" to PROVIDER_MAP and
  sync the test copy in drift-collector.test.ts
- values.yaml: add nameOverride and fullnameOverride at top
- .dockerignore: remove dead !LICENSE line (no-op since *.md excludes it)
- embeddings.ts/bedrock.ts: thread Logger through defaults, replace
  console.warn with logger.warn, add journal entries on JSON parse
  failure, pass flattenHeaders(req.headers) in all journal.add() calls
- update-competitive-matrix.ts: remove no-op .replace(/\.\.\./g, "...")
@jpr5 jpr5 force-pushed the feat/competitive-parity branch 3 times, most recently from 9ba6a07 to 42aa1da Compare March 20, 2026 00:16
Base automatically changed from fix/drift-remediation to main March 20, 2026 00:32
@jpr5 jpr5 force-pushed the feat/competitive-parity branch from 1c716a1 to 2624669 Compare March 20, 2026 00:34
@jpr5 jpr5 merged commit a165705 into main Mar 20, 2026
9 checks passed
@jpr5 jpr5 deleted the feat/competitive-parity branch March 20, 2026 00:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant