test: provider — temperature, topP, topK model parameter defaults#541
test: provider — temperature, topP, topK model parameter defaults#541anandgupta42 wants to merge 1 commit intomainfrom
Conversation
Add 30 unit tests for ProviderTransform.temperature(), topP(), and topK() which are pure functions that return model-specific sampling defaults. These functions are the sole source of per-model parameter configuration and were previously untested, risking silent regressions when adding or modifying model ID patterns (e.g., kimi-k2 sub-variants, minimax-m2 dot/hyphen variants). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01WZthZmQczd51XXSjhiABNH
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
📝 WalkthroughWalkthroughAdded 123 lines of test coverage for Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/opencode/test/provider/transform.test.ts (1)
2662-2752: Optional: reduce repeated test helper setup.
const m = (id: string) => ({ id } as any)is duplicated across describes; extracting one shared helper (or table-driven inputs) would simplify future additions.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/opencode/test/provider/transform.test.ts` around lines 2662 - 2752, Extract the duplicated test helper `const m = (id: string) => ({ id } as any)` into a single shared helper at the top of the test file (or a common beforeAll scope) and remove the per-describe redeclarations so all suites (e.g., usages in ProviderTransform.temperature, ProviderTransform.topP, ProviderTransform.topK) reuse it; update existing describes to call the shared `m` helper or convert tests to table-driven form if preferred to reduce repetition when adding new model cases.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@packages/opencode/test/provider/transform.test.ts`:
- Around line 2662-2752: Extract the duplicated test helper `const m = (id:
string) => ({ id } as any)` into a single shared helper at the top of the test
file (or a common beforeAll scope) and remove the per-describe redeclarations so
all suites (e.g., usages in ProviderTransform.temperature,
ProviderTransform.topP, ProviderTransform.topK) reuse it; update existing
describes to call the shared `m` helper or convert tests to table-driven form if
preferred to reduce repetition when adding new model cases.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 3335bc85-e70a-4e8b-a50e-5b485f9baa86
📒 Files selected for processing (1)
packages/opencode/test/provider/transform.test.ts
… fixes Consolidates PRs #515, #526, #527, #528, #530, #531, #532, #533, #534, #535, #536, #537, #538, #539, #540, #541, #542, #543 into a single PR. Changes: - 30 files changed, ~3000 lines of new test coverage - Deduplicated redundant tests: - `copilot-compat.test.ts`: removed duplicate `mapOpenAICompatibleFinishReason` tests (already covered in `copilot/finish-reason.test.ts`) - `lazy.test.ts`: removed duplicate error-retry and `reset()` tests - `transform.test.ts`: kept most comprehensive version (#535) over subset PRs (#539, #541) - Bug fixes from PR #528: - `extractEquivalenceErrors`: `null` entries in `validation_errors` crashed with TypeError (`null.message` throws before `??` evaluates). Fixed with optional chaining: `e?.message` - `extractSemanticsErrors`: same fix applied - Updated test from `expect(...).toThrow(TypeError)` to verify the fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
Consolidated into #545 |
… fixes (#545) * test: MCP auth — URL validation, token expiry, and client secret lifecycle Cover security-critical McpAuth functions (getForUrl, isTokenExpired) and McpOAuthProvider.clientInformation() expiry detection that had zero test coverage. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01CqcvvXp5hUVsNU441DFTwb * test: copilot provider — finish reason mapping and tool preparation Add 27 unit tests for three previously untested copilot SDK functions that are critical to the GitHub Copilot provider integration path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: log-buffer, RWLock concurrency, SSE chunk splitting — 13 new tests Cover three untested risk areas: dbt ring buffer overflow (ties to #249 TUI corruption fix), reader-writer lock starvation ordering, and SSE event parsing across chunk boundaries and abort signals. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01153R7Dh9BMKiarndEUraBk * test: SQL tool formatters — check, equivalence, semantics (38 tests) Export and test pure formatting functions across three SQL analysis tools that had zero test coverage. Discovered a real bug: null entries in validation_errors crash extractEquivalenceErrors (TypeError on null.message). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01Lz8zxrbwHXfsC2FbHxXZh9 * test: stats display + MCP OAuth XSS prevention — 26 new tests Add first-ever test coverage for the `altimate-code stats` CLI output formatting and the MCP OAuth callback server's HTML escaping (XSS prevention boundary). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: util — proxy detection and lazy error recovery Add tests for proxied() corporate proxy detection (6 tests) and lazy() error recovery + reset behavior (2 tests) to cover untested code paths that affect package installation and initialization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01EDCRjjHdb1dWvxyAfrLuhw * test: session compaction — observation mask and arg truncation Cover createObservationMask() which generates the replacement text when old tool outputs are pruned during session compaction. Tests verify format correctness, UTF-8 byte counting, arg truncation with surrogate pair safety, unserializable input handling, and fingerprint capping. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01SHDrUNHjUpTwPvcjQcJ4ug * test: bus — publish/subscribe/once/unsubscribe mechanics Zero dedicated tests existed for the core event Bus that powers session updates, permission prompts, file watcher notifications, and SSE delivery. New coverage includes subscriber delivery, unsubscribe correctness, wildcard subscriptions, type isolation, and Bus.once auto-removal. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01GchE7rUZayV1ouLEseVndK * test: lazy utility and credential-store — error retry, reset, sensitive field coverage Cover untested behaviors in lazy() (error non-caching and reset) that power shell detection, plus complete isSensitiveField unit coverage for BigQuery/SSL/SSH fields. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01WoqeutgfwXNcktweCKoLwd * test: provider/transform — temperature, topP, topK, smallOptions, maxOutputTokens Add 35 tests for five previously untested ProviderTransform functions that control model-specific inference parameters for all users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_014NGgCMNXEg4Nn3JCpzDg5w * test: fingerprint + context — fill coverage gaps in core utilities Add tests for Fingerprint.refresh() cache invalidation and dbt-packages tag detection (both untested code paths), plus first-ever unit tests for the Context utility (AsyncLocalStorage wrapper) used by every module. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01N8kgPYhXX7SrYnZKJLiTfC * test: session todo — CRUD lifecycle with database persistence Adds 6 tests for the Todo module (zero prior coverage). Covers insert/get round-trip, position ordering, empty-array clear, replacement semantics, bus event emission, and cross-session isolation. These guard the TUI todo panel against stale or phantom tasks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: finops recommendations + dbt manifest edge cases — 12 new tests Cover untested recommendation logic in warehouse-advisor and credit-analyzer edge cases in dbt manifest parsing that affect real-world dbt projects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01XhZy7vaqdasKH5hQ6H9ee3 * test: provider — sampling parameter functions (temperature, topP, topK) Add 28 tests for ProviderTransform.temperature(), topP(), and topK() which had zero direct test coverage. These pure functions control LLM sampling behavior per model family and wrong values cause degraded output quality. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_011NoVCnMW9Kw6eh92ayU7GB * test: session utilities — isDefaultTitle, fromRow/toRow, createObservationMask Add 17 tests covering two untested modules in the session subsystem: session identity helpers and compaction observation masks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: provider — temperature, topP, topK model parameter defaults Add 30 unit tests for ProviderTransform.temperature(), topP(), and topK() which are pure functions that return model-specific sampling defaults. These functions are the sole source of per-model parameter configuration and were previously untested, risking silent regressions when adding or modifying model ID patterns (e.g., kimi-k2 sub-variants, minimax-m2 dot/hyphen variants). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01WZthZmQczd51XXSjhiABNH * test: agent — .env read protection and analyst write denial Verify security-relevant agent permission defaults: builder agent asks before reading .env files (preventing accidental secret exposure), and analyst agent denies file modification tools (edit/write/todowrite/todoread). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01Wp9YaEvw6jAAL73VVdXFxA * test: docker discovery + copilot provider compatibility Add 20 new tests covering two previously untested modules: 1. Docker container discovery (containerToConfig) — verifies correct ConnectionConfig shape generation from discovered containers 2. Copilot provider finish-reason mapping and response metadata — ensures OpenAI-compatible finish reasons are correctly translated and response timestamps are properly converted Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> https://claude.ai/code/session_01J8xz7ijLjbzEe3mu7ajdWh * test: consolidate 18 test PRs — 434 new tests, deduplicated, with bug fixes Consolidates PRs #515, #526, #527, #528, #530, #531, #532, #533, #534, #535, #536, #537, #538, #539, #540, #541, #542, #543 into a single PR. Changes: - 30 files changed, ~3000 lines of new test coverage - Deduplicated redundant tests: - `copilot-compat.test.ts`: removed duplicate `mapOpenAICompatibleFinishReason` tests (already covered in `copilot/finish-reason.test.ts`) - `lazy.test.ts`: removed duplicate error-retry and `reset()` tests - `transform.test.ts`: kept most comprehensive version (#535) over subset PRs (#539, #541) - Bug fixes from PR #528: - `extractEquivalenceErrors`: `null` entries in `validation_errors` crashed with TypeError (`null.message` throws before `??` evaluates). Fixed with optional chaining: `e?.message` - `extractSemanticsErrors`: same fix applied - Updated test from `expect(...).toThrow(TypeError)` to verify the fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: resolve typecheck errors in test files - `prepare-tools.test.ts`: use template literal type for provider tool `id` - `compaction-mask.test.ts`: use `as unknown as` for branded type casts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove flaky `setTimeout` in todo bus event test `Bus.publish` is synchronous — the event is delivered immediately, no 50ms delay needed. Removes resource contention risk in parallel CI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CodeRabbit review feedback - `formatCheck`: harden validation error formatting against null entries using optional chaining and filter (CodeRabbit + GPT consensus) - `extractEquivalenceErrors`: propagate extracted errors into `formatEquivalence` output to prevent title/output inconsistency - `todo.test.ts`: use `tmpdir({ git: true })` + `await using` for proper test isolation instead of shared project root Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
What does this PR do?
1.
ProviderTransform.temperature(),topP(),topK()—src/provider/transform.ts(30 new tests)These three pure functions are the sole source of model-specific sampling parameter defaults used by the LLM streaming pipeline (
session/llm.tslines 124-128). They map model IDs (via substring matching) to temperature, top-P, and top-K values for Qwen, Claude, Gemini, GLM, MiniMax, and Kimi model families. Despite being critical to output quality, they had zero dedicated unit tests — the existing 2655-linetransform.test.tsonly referencedtemperatureas a mock property, never testing the actual functions.Why it matters: These functions contain fragile substring-matching logic with overlapping patterns. For example:
kimi-k2has sub-variants (thinking,k2.5,k2p5,k2-5) that should return1.0while the basekimi-k2returns0.6minimax-m2has sub-variants (m2.,m25,m21) that returntopK=40while the base returnstopK=20topPfunction checkskimi-k2.5andkimi-k2-5as separate string literalsA regression in any of these branches silently degrades generation quality — users see poor outputs without any error message.
Specific scenarios covered:
0.6) vs thinking/k2.5/k2p5/k2-5 variants (1.0)topK=20) vs m2./m25/m21 variants (topK=40)kimi-k2.5(dot) andkimi-k2-5(hyphen) both return0.95, but plainkimi-k2returnsundefinedundefinedfor all three functionsType of change
Issue for this PR
N/A — proactive test coverage for untested model parameter defaults
How did you verify your code works?
Marker check passed:
bun run script/upstream/analyze.ts --markers --base main --strictChecklist
https://claude.ai/code/session_01WZthZmQczd51XXSjhiABNH
Summary by CodeRabbit