Skip to content

fix(simulator): OpcodeStats Add/Sub must use union of keys#193

Closed
meyer9 wants to merge 1 commit into
mainfrom
meyer9/fix-opcodestats-add-sub
Closed

fix(simulator): OpcodeStats Add/Sub must use union of keys#193
meyer9 wants to merge 1 commit into
mainfrom
meyer9/fix-opcodestats-add-sub

Conversation

@meyer9
Copy link
Copy Markdown
Collaborator

@meyer9 meyer9 commented Jun 1, 2026

Problem

OpcodeStats.Add and OpcodeStats.Sub (in runner/payload/simulator/simulatorstats/types.go) only iterate the other argument, silently dropping keys present in the receiver o but absent from other:

func (o OpcodeStats) Add(other OpcodeStats) OpcodeStats {
    result := make(OpcodeStats)
    for opcode, count := range other {          // iterates 'other' only
        result[opcode] = o[opcode] + count
    }
    return result
}

Calling OpcodeStats{"A": 1, "B": 2}.Add(OpcodeStats{}) returns {} — the receiver is gone.

Downstream impact in sendTxs

The simulator worker accumulates per-tx config like:

expected := t.payloadParams.Mul(float64(t.numCalls+1) * t.scaleFactor)  // has all base opcodes + precompiles
blockCounts := expected.Sub(actual).Round()                             // first tx: actual is empty
...
t.actualNumConfig = t.actualNumConfig.Add(blockCounts)

For the first tx, actual is empty. The buggy Sub iterates actual (empty) → produces blockCounts with empty Opcodes/Precompiles maps. Then actual.Add(blockCounts) likewise iterates blockCounts (now also empty for these fields) → still empty.

Result: every tx in every block has empty Precompiles in its SimulatorConfig call, so the on-chain Run() skips all precompile work that the YAML config specified.

Observable consequence

Workloads with precompiles in their config — base-mainnet-simulation is the only one in mainnet-config.yml — under-fill the gas budget because per-tx gas is far below what calcNumCalls predicted from the gas-estimation call (which DID include precompiles). Concrete data from production run test-1779411190779930:

Run Payload GasLimit gas/per_block Fill %
-0 base-mainnet-simulation 150 M 49 M 33%
-1 base-mainnet-simulation 200 M 49 M 24%
-2 base-mainnet-simulation 250 M 51 M 20%

Note the flat ~49–51 M ceiling regardless of GasLimit — that's the per-tx cost of the non-precompile portion (storage + accounts + opcodes) × 100 txs, which is what the worker actually executes. Other workloads (storage-create-full-block, etc.) don't declare precompiles in their config, so they're unaffected and fill ~98%.

Fix

Both Add and Sub now iterate the union of keys:

 func (o OpcodeStats) Add(other OpcodeStats) OpcodeStats {
-    result := make(OpcodeStats)
+    result := make(OpcodeStats, len(o)+len(other))
+    for opcode, count := range o {
+        result[opcode] = count
+    }
     for opcode, count := range other {
-        result[opcode] = o[opcode] + count
+        result[opcode] += count
     }
     return result
 }

Sub matches: keys only in other produce negative entries (correct arithmetic semantics).

Tests added

runner/payload/simulator/simulatorstats/types_test.go:

  • TestOpcodeStatsAdd_UnionOfKeys / TestOpcodeStatsSub_UnionOfKeys — direct union-of-keys check
  • TestOpcodeStatsAdd_EmptyOther / TestOpcodeStatsAdd_EmptyReceiver / TestOpcodeStatsSub_EmptyOther — empty-side edge cases
  • TestStatsSubAdd_FirstTxBlockCountsIncludePrecompiles — regression test for the production usage pattern in sendTxs

All pass. go test ./... clean across the repo.

Expected outcome after this PR + #192 land together

base-mainnet-simulation at 150M/200M/250M should fill to ~98% (gas budget becomes the cap, not the missing precompiles). Other workloads unchanged. The 1.05x→2.0x pre-init buffer in #192 covers the slightly higher consumption that comes from actually executing precompile work per tx.

Draft because end-to-end validation needs one CI run.

Before this change, both OpcodeStats.Add and OpcodeStats.Sub iterated only
the 'other' argument:

  for opcode, count := range other {
      result[opcode] = o[opcode] + count
  }

This silently dropped any key present in the receiver 'o' but absent from
'other'. For Add(empty) the result was an empty map, losing 'o' entirely.

Concrete consequence in sendTxs: per-tx blockCounts is computed as
  blockCounts = expected.Sub(actual).Round()
where 'expected' is base x Mul(numCalls+1 x scaleFactor) (carrying all base
opcodes + precompiles) and 'actual' starts empty for the first tx. With the
buggy Sub iterating actual=empty, blockCounts.Opcodes and
blockCounts.Precompiles were ALWAYS empty for every tx in every block.

The downstream effect: workloads with precompile counts in their config
(e.g. base-mainnet-simulation: bls12381MapG2, ecrecover, bls12381G1Add,
bls12381G1MultiExp) had per-tx execution that skipped all precompile work.
Each tx therefore consumed far less gas than the gas estimate predicted,
and blocks under-filled the gas budget. Example: base-mainnet-simulation @
GasLimit=250M (block_time=1s) reported gas/per_block ~51M (20% of limit)
in production runs because per-tx skipped the heaviest workload component.

Fix: both Add and Sub now iterate the union of keys. For Sub, a key present
only in 'other' produces a negative result entry (matches arithmetic
semantics).

Includes unit tests for the union semantics and a regression test for the
first-tx scenario in sendTxs.
@cb-heimdall
Copy link
Copy Markdown
Collaborator

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

@meyer9
Copy link
Copy Markdown
Collaborator Author

meyer9 commented Jun 1, 2026

Superseded by #194 which combines this OpcodeStats fix with the pre-init buffer bump and observed-gas recalibration into one coordinated change.

@meyer9 meyer9 closed this Jun 1, 2026
@meyer9 meyer9 deleted the meyer9/fix-opcodestats-add-sub branch June 1, 2026 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants