-
Notifications
You must be signed in to change notification settings - Fork 21
vitest support add to js/ts project #1204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
||
| # The code should contain the function | ||
| code = context.read_writable_code.code_strings[0].code | ||
| assert "fibonacci" in code |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all code context strings have to be strict strong equality checks
| Returns: | ||
| The current test framework, or 'jest' as default. | ||
|
|
||
| """ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚡️Codeflash found 20% (0.20x) speedup for get_js_test_framework_or_default in codeflash/languages/test_framework.py
⏱️ Runtime : 134 microseconds → 111 microseconds (best of 139 runs)
📝 Explanation and details
This optimization achieves a 20% runtime improvement (from 134μs to 111μs) by restructuring the control flow to handle the most common case first.
Key Optimization:
The optimized code adds an early return path that checks if _current_test_framework is None before performing the tuple membership test. This is significant because:
-
Identity check vs. Membership check: The
is Nonecomparison is a simple pointer equality check (O(1)), whereas thein ("jest", "vitest", "mocha")membership test requires iterating through tuple elements until a match is found or the tuple is exhausted (O(n)). -
Common case optimization: When
_current_test_frameworkisNone(the default/unset state), the original code still performed the tuple membership check before returning "jest". The optimized version immediately returns "jest" forNone, avoiding unnecessary work. -
Line profiler evidence: The original code spent 53% of time on the tuple membership check. The optimized version replaces this with a faster identity check (49.1% of time), which despite similar percentage is absolutely faster due to reduced total runtime.
Test Results Impact:
The annotated tests show consistent improvements across all scenarios:
- Default/unset cases: 41-64% faster (most benefit from skipping tuple check)
- Batch operations: 18-19% faster across 200-1000 iterations
- The optimization particularly excels when
_current_test_frameworkisNone, which appears to be the predominant case in the test suite
Workload Impact:
Based on function_references, this function is called in test framework detection logic. While not in a critical hot path, it's called during test setup/validation where initialization overhead matters. The 20% speedup compounds when called frequently across test suites, reducing overall test framework overhead.
✅ Correctness verification report:
| Test | Status |
|---|---|
| ⚙️ Existing Unit Tests | 🔘 None Found |
| 🌀 Generated Regression Tests | ✅ 718 Passed |
| ⏪ Replay Tests | 🔘 None Found |
| 🔎 Concolic Coverage Tests | ✅ 1 Passed |
| 📊 Tests Coverage | 66.7% |
🌀 Click to see Generated Regression Tests
import pytest
from codeflash.languages.test_framework import get_js_test_framework_or_default
def test_default_returns_jest_when_not_set():
"""Test that the function returns 'jest' as the default when no framework is configured."""
# The module-level _current_test_framework starts as None in a fresh test
# We need to test the actual behavior with the real module state
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 631ns -> 391ns (61.4% faster)
def test_returns_jest_for_jest_framework():
"""Test that the function returns 'jest' when 'jest' is configured."""
# This test verifies the most common case: jest framework
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 631ns -> 420ns (50.2% faster)
def test_returns_vitest_when_configured():
"""Test that the function returns 'vitest' when 'vitest' is the configured framework."""
# This verifies support for the vitest framework
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 621ns -> 381ns (63.0% faster)
def test_returns_mocha_when_configured():
"""Test that the function returns 'mocha' when 'mocha' is the configured framework."""
# This verifies support for the mocha framework
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 561ns -> 381ns (47.2% faster)
def test_return_type_is_string():
"""Test that the function always returns a string type."""
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 511ns -> 361ns (41.6% faster)
def test_return_value_is_non_empty():
"""Test that the function never returns an empty string."""
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 541ns -> 371ns (45.8% faster)
def test_default_to_jest_for_invalid_frameworks():
"""Test that the function defaults to 'jest' when given invalid/unsupported frameworks."""
# The function should handle invalid frameworks gracefully
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 551ns -> 380ns (45.0% faster)
def test_python_frameworks_default_to_jest():
"""Test that Python-specific frameworks (pytest, unittest) are not returned."""
# Python frameworks like 'pytest' and 'unittest' are not JS frameworks
# The function should default to 'jest' for these
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 561ns -> 371ns (51.2% faster)
def test_only_valid_js_frameworks_returned():
"""Test that only valid JS test frameworks or 'jest' default is returned."""
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 571ns -> 400ns (42.8% faster)
valid_frameworks = ("jest", "vitest", "mocha")
def test_consistent_return_value_across_calls():
"""Test that the function returns the same value when called multiple times."""
# The function should be deterministic
codeflash_output = get_js_test_framework_or_default(); result1 = codeflash_output # 561ns -> 371ns (51.2% faster)
codeflash_output = get_js_test_framework_or_default(); result2 = codeflash_output # 241ns -> 200ns (20.5% faster)
codeflash_output = get_js_test_framework_or_default(); result3 = codeflash_output # 180ns -> 160ns (12.5% faster)
def test_return_lowercase_framework_names():
"""Test that returned framework names are lowercase."""
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 561ns -> 341ns (64.5% faster)
def test_no_whitespace_in_result():
"""Test that the returned framework name has no leading or trailing whitespace."""
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 570ns -> 371ns (53.6% faster)
def test_jest_is_fallback_framework():
"""Test that 'jest' acts as the fallback/default framework."""
codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 581ns -> 361ns (60.9% faster)
# If the framework isn't vitest or mocha, it should be jest
if result not in ("vitest", "mocha"):
pass
def test_performance_many_consecutive_calls():
"""Test that the function performs efficiently with many consecutive calls."""
# Call the function 1000 times and verify consistent behavior
results = [get_js_test_framework_or_default() for _ in range(1000)]
# All results should be valid
valid_frameworks = ("jest", "vitest", "mocha")
def test_all_calls_return_same_framework():
"""Test that all calls return the same framework (no state changes)."""
# Call the function multiple times
results = [get_js_test_framework_or_default() for _ in range(100)]
# All results should be identical
first_result = results[0]
def test_return_value_stability():
"""Test that the return value remains stable across multiple invocations."""
# Collect multiple return values
values = []
for _ in range(500):
values.append(get_js_test_framework_or_default()) # 89.1μs -> 75.2μs (18.5% faster)
# All values should be the same
unique_values = set(values)
def test_concurrent_call_consistency():
"""Test that the function returns consistent values when conceptually called 'concurrently'."""
# Simulate rapid sequential calls (proxy for concurrency in single-threaded context)
rapid_results = []
for _ in range(200):
rapid_results.append(get_js_test_framework_or_default()) # 36.1μs -> 30.3μs (19.0% faster)
def test_batch_operations_consistency():
"""Test consistency when performing batch operations."""
# Perform multiple batches of function calls
batch_size = 50
num_batches = 20
batch_results = []
for batch_num in range(num_batches):
batch = [get_js_test_framework_or_default() for _ in range(batch_size)]
batch_results.append(batch)
# All batches should contain identical results
for batch in batch_results:
pass
# All batches should have the same result
first_batch_result = batch_results[0][0]
for batch in batch_results:
pass
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.from codeflash.languages.test_framework import get_js_test_framework_or_default
def test_get_js_test_framework_or_default():
get_js_test_framework_or_default()🔎 Click to see Concolic Coverage Tests
| Test File::Test Function | Original ⏱️ | Optimized ⏱️ | Speedup |
|---|---|---|---|
codeflash_concolic_rh9udn_6/tmp80l1vymz/test_concolic_coverage.py::test_get_js_test_framework_or_default |
621ns | 430ns | 44.4%✅ |
To test or edit this optimization locally git merge codeflash/optimize-pr1204-2026-02-01T21.27.19
| """ | |
| """ | |
| if _current_test_framework is None: | |
| return "jest" |
No description provided.