feat(benchmarks): add BAL benchmarks for the optimization strategies introduced by BALs

[jochem-brouwer]

🗒️ Description

This PR adds BAL tests, for more information and background see jochem-brouwer#1

Huge thanks to @fselmo for refactoring and adding tests!

🔗 Related Issues or PRs

N/A.

✅ Checklist

• All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:

  uvx tox -e static

• All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
• All: Considered updating the online docs in the ./docs/ directory.
• All: Set appropriate labels for the changes (only maintainers can apply labels).
• Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
• Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
• Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

Cute Animal Picture

[LouisTsai-Csie]

Hi @jochem-brouwer, based on your BAL benchmark description, I put together a small refactor below. I’ve run it locally and it passes.

@pytest.mark.valid_from("Amsterdam")
def test_tx_dependency(
    benchmark_test: BenchmarkTestFiller,
    pre: Alloc,
    fork: Fork,
) -> None:
    """
    Benchmark BAL with transaction-dependent execution.

    Deploy a contract that reads storage slot 0, computes a
    keccak256 hash chain until gas is nearly exhausted, and writes
    the result back. Each transaction depends on the previous
    one's storage write, preventing parallel execution.
    """
    target_slot = 0

    setup = Op.MSTORE(
        0,
        Op.SLOAD(target_slot, key_warm=False),
        # gas accounting
        old_memory_size=0,
        new_memory_size=32,
    )

    loop_body = Op.MSTORE(
        0,
        Op.SHA3(0, 32, data_size=32),
        # gas accounting
        old_memory_size=32,
        new_memory_size=32,
    )

    cleanup = (
        Op.SSTORE(
            target_slot,
            Op.MLOAD(0),
            # gas accounting
            key_warm=True,
            original_value=1,
            current_value=1,
            new_value=2,
        )
    )

    reserve_gas = cleanup.gas_cost(fork) + 50
    condition = Op.GT(Op.GAS, reserve_gas)

    attack_code = setup + While(body=loop_body, condition=condition) + cleanup

    attack_contract = pre.deploy_contract(
        code=attack_code,
        storage={0: 1},
    )

    benchmark_test(
        tx=Transaction(
            to=attack_contract,
            sender=pre.fund_eoa(),
        ),
        skip_gas_used_validation=True,
    )

Each transaction starts by reading slot 0, continues the hash chain, and then stores the final result back to slot 0. I didn’t change the overall design, but I simplified the per-transaction logic since our benchmark test wrapper already generates multiple transactions while respecting tx gas limit cap.

[fselmo]

@LouisTsai-Csie can you please help review jochem-brouwer#1 when you get a chance 👀? It builds on the TODOs here and adds more test cases. If it's not far off and only needs tweaks, I think we can at least merge it and work on subsequent updates here so that we have a more complete picture PR'd to ethereum/execution-specs, and since this is still in Draft. Wdyt?

[raxhvl]

issue(perf, non-blocking): Loop wastes 5 gas per cycle

The while loop compiles to:

 [setup][JUMPDEST][body][condition][compute jumpdest][JUMPI][cleanup]

and the jumpdest at runtime using an offset:

 JUMPDEST          ← we want to jump back here         
    body                                                                
    condition
    PUSH4 <offset>  ← "how far back is the JUMPDEST?" 
    PC               ← "where am I right now?"  
    SUB              ← PC - offset = JUMPDEST address                 
    JUMPI                                                  

So PC + SUB (= 5 gas) is a workaround ["I don't know where I am in absolute terms, but I know how far back to jump."]

i.e: the root cause is: While is not aware of setup.

Suggestion

Introduce an optional setup code for While generator ( = for loop primitive) so jumpdest can be computed at compile-time JUMPDEST = len(setup):

  setup
  JUMPDEST          
    body
    condition
    PUSH1  len(setup)    
    JUMPI

cc: @marioevz @LouisTsai-Csie

[jochem-brouwer]

This is a good point and it looks like this is part of the definition of While. I think it's possible to generalize this idea and thus to remove the calculate-jumpdest-at-runtime for all while loops. (Should be addressed in a refactor at some point)

[jochem-brouwer]

I have tested against EthJS to verify that the tests fill and added some minor changes. The tests fill, and on a quick inspection it also looks like they execute the expected behavior (loops without OOGs).

I think we can merge this one for BAL-specific benchmarks.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.01%. Comparing base (be3678d) to head (2dfb2fb).
⚠️ Report is 170 commits behind head on forks/amsterdam.

Additional details and impacted files

@@                 Coverage Diff                 @@
##           forks/amsterdam    #2197      +/-   ##
===================================================
- Coverage            86.07%   86.01%   -0.07%     
===================================================
  Files                  599      599              
  Lines                39472    36904    -2568     
  Branches              3780     3771       -9     
===================================================
- Hits                 33977    31744    -2233     
+ Misses                4862     4551     -311     
+ Partials               633      609      -24     

Flag	Coverage Δ
unittests	86.01% <ø> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[LouisTsai-Csie]

Just leave a note here, i hope PR #2103 could help when it is merged

[fselmo]

This lgtm and we should ideally get this in now so going to merge :). @LouisTsai-Csie you mentioned you'd like to refactor here. Can you create an issue to track the refactor? Going to ping you here so you do not forget :)