Skip to content

Conversation

@Saga4
Copy link
Contributor

@Saga4 Saga4 commented Jan 30, 2026

No description provided.

@github-actions github-actions bot added the workflow-modified This PR modifies GitHub Actions workflows label Jan 30, 2026

# The code should contain the function
code = context.read_writable_code.code_strings[0].code
assert "fibonacci" in code
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all code context strings have to be strict strong equality checks

@Saga4 Saga4 marked this pull request as draft January 30, 2026 20:09
@Saga4 Saga4 marked this pull request as ready for review February 1, 2026 20:44
@Saga4 Saga4 merged commit 1b2463e into main Feb 1, 2026
24 of 28 checks passed
@Saga4 Saga4 deleted the add_vitest_support_to_js branch February 1, 2026 20:44
Returns:
The current test framework, or 'jest' as default.

"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚡️Codeflash found 20% (0.20x) speedup for get_js_test_framework_or_default in codeflash/languages/test_framework.py

⏱️ Runtime : 134 microseconds 111 microseconds (best of 139 runs)

📝 Explanation and details

This optimization achieves a 20% runtime improvement (from 134μs to 111μs) by restructuring the control flow to handle the most common case first.

Key Optimization:

The optimized code adds an early return path that checks if _current_test_framework is None before performing the tuple membership test. This is significant because:

  1. Identity check vs. Membership check: The is None comparison is a simple pointer equality check (O(1)), whereas the in ("jest", "vitest", "mocha") membership test requires iterating through tuple elements until a match is found or the tuple is exhausted (O(n)).

  2. Common case optimization: When _current_test_framework is None (the default/unset state), the original code still performed the tuple membership check before returning "jest". The optimized version immediately returns "jest" for None, avoiding unnecessary work.

  3. Line profiler evidence: The original code spent 53% of time on the tuple membership check. The optimized version replaces this with a faster identity check (49.1% of time), which despite similar percentage is absolutely faster due to reduced total runtime.

Test Results Impact:

The annotated tests show consistent improvements across all scenarios:

  • Default/unset cases: 41-64% faster (most benefit from skipping tuple check)
  • Batch operations: 18-19% faster across 200-1000 iterations
  • The optimization particularly excels when _current_test_framework is None, which appears to be the predominant case in the test suite

Workload Impact:

Based on function_references, this function is called in test framework detection logic. While not in a critical hot path, it's called during test setup/validation where initialization overhead matters. The 20% speedup compounds when called frequently across test suites, reducing overall test framework overhead.

Correctness verification report:

Test Status
⚙️ Existing Unit Tests 🔘 None Found
🌀 Generated Regression Tests 718 Passed
⏪ Replay Tests 🔘 None Found
🔎 Concolic Coverage Tests 1 Passed
📊 Tests Coverage 66.7%
🌀 Click to see Generated Regression Tests
import pytest
from codeflash.languages.test_framework import get_js_test_framework_or_default

def test_default_returns_jest_when_not_set():
    """Test that the function returns 'jest' as the default when no framework is configured."""
    # The module-level _current_test_framework starts as None in a fresh test
    # We need to test the actual behavior with the real module state
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 631ns -> 391ns (61.4% faster)

def test_returns_jest_for_jest_framework():
    """Test that the function returns 'jest' when 'jest' is configured."""
    # This test verifies the most common case: jest framework
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 631ns -> 420ns (50.2% faster)

def test_returns_vitest_when_configured():
    """Test that the function returns 'vitest' when 'vitest' is the configured framework."""
    # This verifies support for the vitest framework
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 621ns -> 381ns (63.0% faster)

def test_returns_mocha_when_configured():
    """Test that the function returns 'mocha' when 'mocha' is the configured framework."""
    # This verifies support for the mocha framework
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 561ns -> 381ns (47.2% faster)

def test_return_type_is_string():
    """Test that the function always returns a string type."""
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 511ns -> 361ns (41.6% faster)

def test_return_value_is_non_empty():
    """Test that the function never returns an empty string."""
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 541ns -> 371ns (45.8% faster)

def test_default_to_jest_for_invalid_frameworks():
    """Test that the function defaults to 'jest' when given invalid/unsupported frameworks."""
    # The function should handle invalid frameworks gracefully
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 551ns -> 380ns (45.0% faster)

def test_python_frameworks_default_to_jest():
    """Test that Python-specific frameworks (pytest, unittest) are not returned."""
    # Python frameworks like 'pytest' and 'unittest' are not JS frameworks
    # The function should default to 'jest' for these
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 561ns -> 371ns (51.2% faster)

def test_only_valid_js_frameworks_returned():
    """Test that only valid JS test frameworks or 'jest' default is returned."""
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 571ns -> 400ns (42.8% faster)
    valid_frameworks = ("jest", "vitest", "mocha")

def test_consistent_return_value_across_calls():
    """Test that the function returns the same value when called multiple times."""
    # The function should be deterministic
    codeflash_output = get_js_test_framework_or_default(); result1 = codeflash_output # 561ns -> 371ns (51.2% faster)
    codeflash_output = get_js_test_framework_or_default(); result2 = codeflash_output # 241ns -> 200ns (20.5% faster)
    codeflash_output = get_js_test_framework_or_default(); result3 = codeflash_output # 180ns -> 160ns (12.5% faster)

def test_return_lowercase_framework_names():
    """Test that returned framework names are lowercase."""
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 561ns -> 341ns (64.5% faster)

def test_no_whitespace_in_result():
    """Test that the returned framework name has no leading or trailing whitespace."""
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 570ns -> 371ns (53.6% faster)

def test_jest_is_fallback_framework():
    """Test that 'jest' acts as the fallback/default framework."""
    codeflash_output = get_js_test_framework_or_default(); result = codeflash_output # 581ns -> 361ns (60.9% faster)
    # If the framework isn't vitest or mocha, it should be jest
    if result not in ("vitest", "mocha"):
        pass

def test_performance_many_consecutive_calls():
    """Test that the function performs efficiently with many consecutive calls."""
    # Call the function 1000 times and verify consistent behavior
    results = [get_js_test_framework_or_default() for _ in range(1000)]
    
    # All results should be valid
    valid_frameworks = ("jest", "vitest", "mocha")

def test_all_calls_return_same_framework():
    """Test that all calls return the same framework (no state changes)."""
    # Call the function multiple times
    results = [get_js_test_framework_or_default() for _ in range(100)]
    
    # All results should be identical
    first_result = results[0]

def test_return_value_stability():
    """Test that the return value remains stable across multiple invocations."""
    # Collect multiple return values
    values = []
    for _ in range(500):
        values.append(get_js_test_framework_or_default()) # 89.1μs -> 75.2μs (18.5% faster)
    
    # All values should be the same
    unique_values = set(values)

def test_concurrent_call_consistency():
    """Test that the function returns consistent values when conceptually called 'concurrently'."""
    # Simulate rapid sequential calls (proxy for concurrency in single-threaded context)
    rapid_results = []
    for _ in range(200):
        rapid_results.append(get_js_test_framework_or_default()) # 36.1μs -> 30.3μs (19.0% faster)

def test_batch_operations_consistency():
    """Test consistency when performing batch operations."""
    # Perform multiple batches of function calls
    batch_size = 50
    num_batches = 20
    
    batch_results = []
    for batch_num in range(num_batches):
        batch = [get_js_test_framework_or_default() for _ in range(batch_size)]
        batch_results.append(batch)
    
    # All batches should contain identical results
    for batch in batch_results:
        pass
    
    # All batches should have the same result
    first_batch_result = batch_results[0][0]
    for batch in batch_results:
        pass
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.
from codeflash.languages.test_framework import get_js_test_framework_or_default

def test_get_js_test_framework_or_default():
    get_js_test_framework_or_default()
🔎 Click to see Concolic Coverage Tests
Test File::Test Function Original ⏱️ Optimized ⏱️ Speedup
codeflash_concolic_rh9udn_6/tmp80l1vymz/test_concolic_coverage.py::test_get_js_test_framework_or_default 621ns 430ns 44.4%✅

To test or edit this optimization locally git merge codeflash/optimize-pr1204-2026-02-01T21.27.19

Suggested change
"""
"""
if _current_test_framework is None:
return "jest"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

workflow-modified This PR modifies GitHub Actions workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants