Skip to content

test: provider — temperature, topP, topK model parameter defaults#541

Closed
anandgupta42 wants to merge 1 commit intomainfrom
test/hourly-20260328-1210
Closed

test: provider — temperature, topP, topK model parameter defaults#541
anandgupta42 wants to merge 1 commit intomainfrom
test/hourly-20260328-1210

Conversation

@anandgupta42
Copy link
Copy Markdown
Contributor

@anandgupta42 anandgupta42 commented Mar 28, 2026

What does this PR do?

1. ProviderTransform.temperature(), topP(), topK()src/provider/transform.ts (30 new tests)

These three pure functions are the sole source of model-specific sampling parameter defaults used by the LLM streaming pipeline (session/llm.ts lines 124-128). They map model IDs (via substring matching) to temperature, top-P, and top-K values for Qwen, Claude, Gemini, GLM, MiniMax, and Kimi model families. Despite being critical to output quality, they had zero dedicated unit tests — the existing 2655-line transform.test.ts only referenced temperature as a mock property, never testing the actual functions.

Why it matters: These functions contain fragile substring-matching logic with overlapping patterns. For example:

  • kimi-k2 has sub-variants (thinking, k2.5, k2p5, k2-5) that should return 1.0 while the base kimi-k2 returns 0.6
  • minimax-m2 has sub-variants (m2., m25, m21) that return topK=40 while the base returns topK=20
  • The topP function checks kimi-k2.5 and kimi-k2-5 as separate string literals

A regression in any of these branches silently degrades generation quality — users see poor outputs without any error message.

Specific scenarios covered:

  • Each model family branch for all three functions (qwen, claude, gemini, glm, minimax, kimi)
  • Kimi-k2 sub-branching: base (0.6) vs thinking/k2.5/k2p5/k2-5 variants (1.0)
  • MiniMax-m2 sub-branching: base (topK=20) vs m2./m25/m21 variants (topK=40)
  • TopP asymmetry: kimi-k2.5 (dot) and kimi-k2-5 (hyphen) both return 0.95, but plain kimi-k2 returns undefined
  • Unknown models returning undefined for all three functions

Type of change

  • New feature (non-breaking change which adds functionality)

Issue for this PR

N/A — proactive test coverage for untested model parameter defaults

How did you verify your code works?

bun test test/provider/transform.test.ts    # 140 pass (110 existing + 30 new)

Marker check passed: bun run script/upstream/analyze.ts --markers --base main --strict

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

https://claude.ai/code/session_01WZthZmQczd51XXSjhiABNH

Summary by CodeRabbit

  • Tests
    • Added comprehensive test coverage for parameter transformations across multiple AI model providers.

Add 30 unit tests for ProviderTransform.temperature(), topP(), and topK()
which are pure functions that return model-specific sampling defaults.
These functions are the sole source of per-model parameter configuration
and were previously untested, risking silent regressions when adding or
modifying model ID patterns (e.g., kimi-k2 sub-variants, minimax-m2
dot/hyphen variants).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01WZthZmQczd51XXSjhiABNH
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 28, 2026

📝 Walkthrough

Walkthrough

Added 123 lines of test coverage for ProviderTransform.temperature, ProviderTransform.topP, and ProviderTransform.topK methods. Tests validate numeric outputs across multiple model providers and verify that undefined is returned for unsupported models.

Changes

Cohort / File(s) Summary
ProviderTransform Test Coverage
packages/opencode/test/provider/transform.test.ts
Added comprehensive test cases for temperature, topP, and topK parameter transformation methods across various AI model providers (Qwen, Gemini, MiniMax, GLM, Kimi-K2), with assertions for expected numeric values and undefined returns for unsupported models.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 With whiskers twitching in delight,
New tests shimmer, shining bright,
Each model checked from left to right,
Temperature and topP take flight,
Our code now covered, pure and tight! ✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and concisely summarizes the main change: adding test coverage for three model parameter default functions (temperature, topP, topK).
Description check ✅ Passed The description comprehensively covers what changed and why, includes detailed test scenarios, provides verification steps, and addresses all key template sections despite variations in format.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/hourly-20260328-1210

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/opencode/test/provider/transform.test.ts (1)

2662-2752: Optional: reduce repeated test helper setup.

const m = (id: string) => ({ id } as any) is duplicated across describes; extracting one shared helper (or table-driven inputs) would simplify future additions.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/opencode/test/provider/transform.test.ts` around lines 2662 - 2752,
Extract the duplicated test helper `const m = (id: string) => ({ id } as any)`
into a single shared helper at the top of the test file (or a common beforeAll
scope) and remove the per-describe redeclarations so all suites (e.g., usages in
ProviderTransform.temperature, ProviderTransform.topP, ProviderTransform.topK)
reuse it; update existing describes to call the shared `m` helper or convert
tests to table-driven form if preferred to reduce repetition when adding new
model cases.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/opencode/test/provider/transform.test.ts`:
- Around line 2662-2752: Extract the duplicated test helper `const m = (id:
string) => ({ id } as any)` into a single shared helper at the top of the test
file (or a common beforeAll scope) and remove the per-describe redeclarations so
all suites (e.g., usages in ProviderTransform.temperature,
ProviderTransform.topP, ProviderTransform.topK) reuse it; update existing
describes to call the shared `m` helper or convert tests to table-driven form if
preferred to reduce repetition when adding new model cases.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 3335bc85-e70a-4e8b-a50e-5b485f9baa86

📥 Commits

Reviewing files that changed from the base of the PR and between 9ba2114 and eb24377.

📒 Files selected for processing (1)
  • packages/opencode/test/provider/transform.test.ts

anandgupta42 added a commit that referenced this pull request Mar 28, 2026
… fixes

Consolidates PRs #515, #526, #527, #528, #530, #531, #532, #533, #534,
#535, #536, #537, #538, #539, #540, #541, #542, #543 into a single PR.

Changes:
- 30 files changed, ~3000 lines of new test coverage
- Deduplicated redundant tests:
  - `copilot-compat.test.ts`: removed duplicate `mapOpenAICompatibleFinishReason`
    tests (already covered in `copilot/finish-reason.test.ts`)
  - `lazy.test.ts`: removed duplicate error-retry and `reset()` tests
  - `transform.test.ts`: kept most comprehensive version (#535) over
    subset PRs (#539, #541)
- Bug fixes from PR #528:
  - `extractEquivalenceErrors`: `null` entries in `validation_errors`
    crashed with TypeError (`null.message` throws before `??` evaluates).
    Fixed with optional chaining: `e?.message`
  - `extractSemanticsErrors`: same fix applied
  - Updated test from `expect(...).toThrow(TypeError)` to verify the fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@anandgupta42
Copy link
Copy Markdown
Contributor Author

Consolidated into #545

anandgupta42 added a commit that referenced this pull request Mar 28, 2026
… fixes (#545)

* test: MCP auth — URL validation, token expiry, and client secret lifecycle

Cover security-critical McpAuth functions (getForUrl, isTokenExpired) and
McpOAuthProvider.clientInformation() expiry detection that had zero test coverage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01CqcvvXp5hUVsNU441DFTwb

* test: copilot provider — finish reason mapping and tool preparation

Add 27 unit tests for three previously untested copilot SDK functions
that are critical to the GitHub Copilot provider integration path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: log-buffer, RWLock concurrency, SSE chunk splitting — 13 new tests

Cover three untested risk areas: dbt ring buffer overflow (ties to #249 TUI
corruption fix), reader-writer lock starvation ordering, and SSE event parsing
across chunk boundaries and abort signals.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01153R7Dh9BMKiarndEUraBk

* test: SQL tool formatters — check, equivalence, semantics (38 tests)

Export and test pure formatting functions across three SQL analysis tools
that had zero test coverage. Discovered a real bug: null entries in
validation_errors crash extractEquivalenceErrors (TypeError on null.message).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01Lz8zxrbwHXfsC2FbHxXZh9

* test: stats display + MCP OAuth XSS prevention — 26 new tests

Add first-ever test coverage for the `altimate-code stats` CLI output formatting
and the MCP OAuth callback server's HTML escaping (XSS prevention boundary).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: util — proxy detection and lazy error recovery

Add tests for proxied() corporate proxy detection (6 tests) and
lazy() error recovery + reset behavior (2 tests) to cover untested
code paths that affect package installation and initialization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01EDCRjjHdb1dWvxyAfrLuhw

* test: session compaction — observation mask and arg truncation

Cover createObservationMask() which generates the replacement text when old
tool outputs are pruned during session compaction. Tests verify format
correctness, UTF-8 byte counting, arg truncation with surrogate pair safety,
unserializable input handling, and fingerprint capping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01SHDrUNHjUpTwPvcjQcJ4ug

* test: bus — publish/subscribe/once/unsubscribe mechanics

Zero dedicated tests existed for the core event Bus that powers session updates,
permission prompts, file watcher notifications, and SSE delivery. New coverage
includes subscriber delivery, unsubscribe correctness, wildcard subscriptions,
type isolation, and Bus.once auto-removal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01GchE7rUZayV1ouLEseVndK

* test: lazy utility and credential-store — error retry, reset, sensitive field coverage

Cover untested behaviors in lazy() (error non-caching and reset) that power shell
detection, plus complete isSensitiveField unit coverage for BigQuery/SSL/SSH fields.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01WoqeutgfwXNcktweCKoLwd

* test: provider/transform — temperature, topP, topK, smallOptions, maxOutputTokens

Add 35 tests for five previously untested ProviderTransform functions that
control model-specific inference parameters for all users.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_014NGgCMNXEg4Nn3JCpzDg5w

* test: fingerprint + context — fill coverage gaps in core utilities

Add tests for Fingerprint.refresh() cache invalidation and dbt-packages tag
detection (both untested code paths), plus first-ever unit tests for the
Context utility (AsyncLocalStorage wrapper) used by every module.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01N8kgPYhXX7SrYnZKJLiTfC

* test: session todo — CRUD lifecycle with database persistence

Adds 6 tests for the Todo module (zero prior coverage). Covers insert/get round-trip,
position ordering, empty-array clear, replacement semantics, bus event emission, and
cross-session isolation. These guard the TUI todo panel against stale or phantom tasks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: finops recommendations + dbt manifest edge cases — 12 new tests

Cover untested recommendation logic in warehouse-advisor and credit-analyzer
edge cases in dbt manifest parsing that affect real-world dbt projects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01XhZy7vaqdasKH5hQ6H9ee3

* test: provider — sampling parameter functions (temperature, topP, topK)

Add 28 tests for ProviderTransform.temperature(), topP(), and topK() which
had zero direct test coverage. These pure functions control LLM sampling
behavior per model family and wrong values cause degraded output quality.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_011NoVCnMW9Kw6eh92ayU7GB

* test: session utilities — isDefaultTitle, fromRow/toRow, createObservationMask

Add 17 tests covering two untested modules in the session subsystem:
session identity helpers and compaction observation masks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: provider — temperature, topP, topK model parameter defaults

Add 30 unit tests for ProviderTransform.temperature(), topP(), and topK()
which are pure functions that return model-specific sampling defaults.
These functions are the sole source of per-model parameter configuration
and were previously untested, risking silent regressions when adding or
modifying model ID patterns (e.g., kimi-k2 sub-variants, minimax-m2
dot/hyphen variants).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01WZthZmQczd51XXSjhiABNH

* test: agent — .env read protection and analyst write denial

Verify security-relevant agent permission defaults: builder agent asks before
reading .env files (preventing accidental secret exposure), and analyst agent
denies file modification tools (edit/write/todowrite/todoread).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01Wp9YaEvw6jAAL73VVdXFxA

* test: docker discovery + copilot provider compatibility

Add 20 new tests covering two previously untested modules:

1. Docker container discovery (containerToConfig) — verifies correct
   ConnectionConfig shape generation from discovered containers
2. Copilot provider finish-reason mapping and response metadata —
   ensures OpenAI-compatible finish reasons are correctly translated
   and response timestamps are properly converted

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01J8xz7ijLjbzEe3mu7ajdWh

* test: consolidate 18 test PRs — 434 new tests, deduplicated, with bug fixes

Consolidates PRs #515, #526, #527, #528, #530, #531, #532, #533, #534,
#535, #536, #537, #538, #539, #540, #541, #542, #543 into a single PR.

Changes:
- 30 files changed, ~3000 lines of new test coverage
- Deduplicated redundant tests:
  - `copilot-compat.test.ts`: removed duplicate `mapOpenAICompatibleFinishReason`
    tests (already covered in `copilot/finish-reason.test.ts`)
  - `lazy.test.ts`: removed duplicate error-retry and `reset()` tests
  - `transform.test.ts`: kept most comprehensive version (#535) over
    subset PRs (#539, #541)
- Bug fixes from PR #528:
  - `extractEquivalenceErrors`: `null` entries in `validation_errors`
    crashed with TypeError (`null.message` throws before `??` evaluates).
    Fixed with optional chaining: `e?.message`
  - `extractSemanticsErrors`: same fix applied
  - Updated test from `expect(...).toThrow(TypeError)` to verify the fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve typecheck errors in test files

- `prepare-tools.test.ts`: use template literal type for provider tool `id`
- `compaction-mask.test.ts`: use `as unknown as` for branded type casts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove flaky `setTimeout` in todo bus event test

`Bus.publish` is synchronous — the event is delivered immediately,
no 50ms delay needed. Removes resource contention risk in parallel CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CodeRabbit review feedback

- `formatCheck`: harden validation error formatting against null entries
  using optional chaining and filter (CodeRabbit + GPT consensus)
- `extractEquivalenceErrors`: propagate extracted errors into
  `formatEquivalence` output to prevent title/output inconsistency
- `todo.test.ts`: use `tmpdir({ git: true })` + `await using` for
  proper test isolation instead of shared project root

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants