Skip to content

test: provider — sampling parameter functions (temperature, topP, topK)#539

Closed
anandgupta42 wants to merge 1 commit intomainfrom
test/hourly-20260328-0910
Closed

test: provider — sampling parameter functions (temperature, topP, topK)#539
anandgupta42 wants to merge 1 commit intomainfrom
test/hourly-20260328-0910

Conversation

@anandgupta42
Copy link
Copy Markdown
Contributor

@anandgupta42 anandgupta42 commented Mar 28, 2026

What does this PR do?

Adds 28 new unit tests for three completely untested pure functions in the provider transform module that control LLM sampling behavior.

1. ProviderTransform.temperature()src/provider/transform.ts:292-308 (11 new tests)

This function returns model-specific temperature values that directly affect output quality. Zero tests existed. A refactor or new model addition could silently break temperature for existing model families. New coverage includes:

  • Qwen models → 0.55 (recommended by Qwen docs)
  • Claude models → undefined (API-controlled)
  • Gemini models → 1.0
  • GLM-4.6/4.7 models → 1.0
  • Minimax-M2 models → 1.0
  • Kimi-K2 variant detection (4 separate patterns: thinking, k2., k2p, k2-5) each tested individually so single-branch deletions are caught
  • Base Kimi-K2 → 0.6 (critical boundary vs. variants returning 1.0)
  • Fallthrough to undefined for unknown models

2. ProviderTransform.topP()src/provider/transform.ts:310-317 (7 new tests)

Controls nucleus sampling. Each model family pattern (kimi-k2.5, kimi-k2p5, kimi-k2-5) is tested individually to catch single-pattern regressions. Coverage includes:

  • Qwen → 1, Minimax-M2/Gemini/Kimi-K2.5 variants → 0.95
  • Fallthrough to undefined for non-matching models

3. ProviderTransform.topK()src/provider/transform.ts:319-327 (10 new tests)

Controls top-K sampling with a tricky inner dispatch for Minimax-M2 versioned vs. base models. Each version pattern (m2., m25, m21) is tested separately. Coverage includes:

  • Minimax-M2 versioned (m2.5/m25/m21) → 40 vs. base M2 → 20 (important boundary)
  • Gemini → 64
  • Fallthrough to undefined

Type of change

  • New feature (non-breaking change which adds functionality)

Issue for this PR

N/A — proactive test coverage for untested sampling parameter logic

How did you verify your code works?

bun test test/provider/transform.test.ts    # 139 pass (111 existing + 28 new)

Checklist

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

https://claude.ai/code/session_011NoVCnMW9Kw6eh92ayU7GB

Summary by CodeRabbit

  • Tests
    • Enhanced test coverage for sampling parameter transforms across multiple AI models (including temperature, topP, and topK parameters).

Add 28 tests for ProviderTransform.temperature(), topP(), and topK() which
had zero direct test coverage. These pure functions control LLM sampling
behavior per model family and wrong values cause degraded output quality.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_011NoVCnMW9Kw6eh92ayU7GB
Copy link
Copy Markdown

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 28, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 76324666-6fb2-41a8-a08f-ff937e77a6b2

📥 Commits

Reviewing files that changed from the base of the PR and between 9ba2114 and c55940b.

📒 Files selected for processing (1)
  • packages/opencode/test/provider/transform.test.ts

📝 Walkthrough

Walkthrough

Added 127 lines of new test cases to packages/opencode/test/provider/transform.test.ts covering sampling parameter transforms. Tests verify that ProviderTransform.temperature, ProviderTransform.topP, and ProviderTransform.topK return expected numeric values based on model ID across multiple provider models including Qwen, Gemini, Minimax, and Kimi-K2.

Changes

Cohort / File(s) Summary
Sampling Parameter Transform Tests
packages/opencode/test/provider/transform.test.ts
Added comprehensive test suites for temperature, topP, and topK parameter transforms with assertions across multiple model variants and categories.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested labels

contributor

Poem

🐰 A thousand models tested with care,
Temperature, topP, topK everywhere,
Each assertion's a carrot, each suite a delight,
The sampling parameters now shining bright! 🥕✨

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately and specifically summarizes the main change: adding tests for three sampling parameter functions (temperature, topP, topK) in the provider module.
Description check ✅ Passed The description is comprehensive and follows the template structure with clear Summary and Test Plan sections, though it lacks explicit checklist items in the final output format.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/hourly-20260328-0910

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

anandgupta42 added a commit that referenced this pull request Mar 28, 2026
… fixes

Consolidates PRs #515, #526, #527, #528, #530, #531, #532, #533, #534,
#535, #536, #537, #538, #539, #540, #541, #542, #543 into a single PR.

Changes:
- 30 files changed, ~3000 lines of new test coverage
- Deduplicated redundant tests:
  - `copilot-compat.test.ts`: removed duplicate `mapOpenAICompatibleFinishReason`
    tests (already covered in `copilot/finish-reason.test.ts`)
  - `lazy.test.ts`: removed duplicate error-retry and `reset()` tests
  - `transform.test.ts`: kept most comprehensive version (#535) over
    subset PRs (#539, #541)
- Bug fixes from PR #528:
  - `extractEquivalenceErrors`: `null` entries in `validation_errors`
    crashed with TypeError (`null.message` throws before `??` evaluates).
    Fixed with optional chaining: `e?.message`
  - `extractSemanticsErrors`: same fix applied
  - Updated test from `expect(...).toThrow(TypeError)` to verify the fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@anandgupta42
Copy link
Copy Markdown
Contributor Author

Consolidated into #545

anandgupta42 added a commit that referenced this pull request Mar 28, 2026
… fixes (#545)

* test: MCP auth — URL validation, token expiry, and client secret lifecycle

Cover security-critical McpAuth functions (getForUrl, isTokenExpired) and
McpOAuthProvider.clientInformation() expiry detection that had zero test coverage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01CqcvvXp5hUVsNU441DFTwb

* test: copilot provider — finish reason mapping and tool preparation

Add 27 unit tests for three previously untested copilot SDK functions
that are critical to the GitHub Copilot provider integration path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: log-buffer, RWLock concurrency, SSE chunk splitting — 13 new tests

Cover three untested risk areas: dbt ring buffer overflow (ties to #249 TUI
corruption fix), reader-writer lock starvation ordering, and SSE event parsing
across chunk boundaries and abort signals.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01153R7Dh9BMKiarndEUraBk

* test: SQL tool formatters — check, equivalence, semantics (38 tests)

Export and test pure formatting functions across three SQL analysis tools
that had zero test coverage. Discovered a real bug: null entries in
validation_errors crash extractEquivalenceErrors (TypeError on null.message).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01Lz8zxrbwHXfsC2FbHxXZh9

* test: stats display + MCP OAuth XSS prevention — 26 new tests

Add first-ever test coverage for the `altimate-code stats` CLI output formatting
and the MCP OAuth callback server's HTML escaping (XSS prevention boundary).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: util — proxy detection and lazy error recovery

Add tests for proxied() corporate proxy detection (6 tests) and
lazy() error recovery + reset behavior (2 tests) to cover untested
code paths that affect package installation and initialization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01EDCRjjHdb1dWvxyAfrLuhw

* test: session compaction — observation mask and arg truncation

Cover createObservationMask() which generates the replacement text when old
tool outputs are pruned during session compaction. Tests verify format
correctness, UTF-8 byte counting, arg truncation with surrogate pair safety,
unserializable input handling, and fingerprint capping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01SHDrUNHjUpTwPvcjQcJ4ug

* test: bus — publish/subscribe/once/unsubscribe mechanics

Zero dedicated tests existed for the core event Bus that powers session updates,
permission prompts, file watcher notifications, and SSE delivery. New coverage
includes subscriber delivery, unsubscribe correctness, wildcard subscriptions,
type isolation, and Bus.once auto-removal.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01GchE7rUZayV1ouLEseVndK

* test: lazy utility and credential-store — error retry, reset, sensitive field coverage

Cover untested behaviors in lazy() (error non-caching and reset) that power shell
detection, plus complete isSensitiveField unit coverage for BigQuery/SSL/SSH fields.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01WoqeutgfwXNcktweCKoLwd

* test: provider/transform — temperature, topP, topK, smallOptions, maxOutputTokens

Add 35 tests for five previously untested ProviderTransform functions that
control model-specific inference parameters for all users.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_014NGgCMNXEg4Nn3JCpzDg5w

* test: fingerprint + context — fill coverage gaps in core utilities

Add tests for Fingerprint.refresh() cache invalidation and dbt-packages tag
detection (both untested code paths), plus first-ever unit tests for the
Context utility (AsyncLocalStorage wrapper) used by every module.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01N8kgPYhXX7SrYnZKJLiTfC

* test: session todo — CRUD lifecycle with database persistence

Adds 6 tests for the Todo module (zero prior coverage). Covers insert/get round-trip,
position ordering, empty-array clear, replacement semantics, bus event emission, and
cross-session isolation. These guard the TUI todo panel against stale or phantom tasks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: finops recommendations + dbt manifest edge cases — 12 new tests

Cover untested recommendation logic in warehouse-advisor and credit-analyzer
edge cases in dbt manifest parsing that affect real-world dbt projects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01XhZy7vaqdasKH5hQ6H9ee3

* test: provider — sampling parameter functions (temperature, topP, topK)

Add 28 tests for ProviderTransform.temperature(), topP(), and topK() which
had zero direct test coverage. These pure functions control LLM sampling
behavior per model family and wrong values cause degraded output quality.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_011NoVCnMW9Kw6eh92ayU7GB

* test: session utilities — isDefaultTitle, fromRow/toRow, createObservationMask

Add 17 tests covering two untested modules in the session subsystem:
session identity helpers and compaction observation masks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: provider — temperature, topP, topK model parameter defaults

Add 30 unit tests for ProviderTransform.temperature(), topP(), and topK()
which are pure functions that return model-specific sampling defaults.
These functions are the sole source of per-model parameter configuration
and were previously untested, risking silent regressions when adding or
modifying model ID patterns (e.g., kimi-k2 sub-variants, minimax-m2
dot/hyphen variants).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01WZthZmQczd51XXSjhiABNH

* test: agent — .env read protection and analyst write denial

Verify security-relevant agent permission defaults: builder agent asks before
reading .env files (preventing accidental secret exposure), and analyst agent
denies file modification tools (edit/write/todowrite/todoread).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01Wp9YaEvw6jAAL73VVdXFxA

* test: docker discovery + copilot provider compatibility

Add 20 new tests covering two previously untested modules:

1. Docker container discovery (containerToConfig) — verifies correct
   ConnectionConfig shape generation from discovered containers
2. Copilot provider finish-reason mapping and response metadata —
   ensures OpenAI-compatible finish reasons are correctly translated
   and response timestamps are properly converted

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

https://claude.ai/code/session_01J8xz7ijLjbzEe3mu7ajdWh

* test: consolidate 18 test PRs — 434 new tests, deduplicated, with bug fixes

Consolidates PRs #515, #526, #527, #528, #530, #531, #532, #533, #534,
#535, #536, #537, #538, #539, #540, #541, #542, #543 into a single PR.

Changes:
- 30 files changed, ~3000 lines of new test coverage
- Deduplicated redundant tests:
  - `copilot-compat.test.ts`: removed duplicate `mapOpenAICompatibleFinishReason`
    tests (already covered in `copilot/finish-reason.test.ts`)
  - `lazy.test.ts`: removed duplicate error-retry and `reset()` tests
  - `transform.test.ts`: kept most comprehensive version (#535) over
    subset PRs (#539, #541)
- Bug fixes from PR #528:
  - `extractEquivalenceErrors`: `null` entries in `validation_errors`
    crashed with TypeError (`null.message` throws before `??` evaluates).
    Fixed with optional chaining: `e?.message`
  - `extractSemanticsErrors`: same fix applied
  - Updated test from `expect(...).toThrow(TypeError)` to verify the fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve typecheck errors in test files

- `prepare-tools.test.ts`: use template literal type for provider tool `id`
- `compaction-mask.test.ts`: use `as unknown as` for branded type casts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove flaky `setTimeout` in todo bus event test

`Bus.publish` is synchronous — the event is delivered immediately,
no 50ms delay needed. Removes resource contention risk in parallel CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CodeRabbit review feedback

- `formatCheck`: harden validation error formatting against null entries
  using optional chaining and filter (CodeRabbit + GPT consensus)
- `extractEquivalenceErrors`: propagate extracted errors into
  `formatEquivalence` output to prevent title/output inconsistency
- `todo.test.ts`: use `tmpdir({ git: true })` + `await using` for
  proper test isolation instead of shared project root

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants