execution indexer v2

[pk910]

Execution Data Storage Rework: BlockDB-backed Transaction Details

Moves per-transaction detail data (events, call traces, state changes) out of the database and into the blockdb storage layer (Pebble/S3). The el_tx_events table is removed entirely. Only lightweight indexes remain in the DB, while raw event/trace data lives in size-limited, prunable blockdb objects.

Indexing Modes

Three modes provide granular control over resource usage:

Mode	Config	What's stored
Disabled	enabled: false	No EL indexing
Lightweight	detailsEnabled: false	DB only: transactions, token transfers, accounts, balances
Full	detailsEnabled: true	Lightweight + per-block detail objects in blockdb (events, call traces, state diffs)

Full mode optionally fetches call traces via debug_traceBlockByHash when tracesEnabled: true.

Key Changes

Storage format — Custom versioned binary format for per-block objects with a fixed-size index header for O(1) random access to individual transaction data. Each section (events, call traces, state changes) is independently snappy-compressed. S3 range reads can fetch just the index, then specific sections by offset.

BlockDB extensions — New ExecDataEngine interface on both Pebble and S3 backends supporting write (with size tracking), read, partial/range read, delete, and slot-range pruning.

Trace fetching — Streaming JSON parser for debug_traceBlockByHash responses (can be hundreds of MB). Parses per-tx, converts to binary format, compresses, and frees memory incrementally. Call traces are normalized across EL clients (Geth, Nethermind, Erigon, Besu, Reth).

New DB tables:

• el_transactions_internal — lightweight index of internal calls from call traces (from, to, value, call type)
• el_event_index — lightweight searchable index for events (source address, topic1)

Pruning — Two independent axes: time-based retention (existing, extended to blockdb) and size-based eviction (detailsMaxSize). Oldest blocks evicted first when size limit exceeded.

UI — New transaction detail tabs for internal transactions (call trace tree), events (loaded from blockdb), and state changes. New internal transactions tab on address pages. Graceful degradation when blockdb data is pruned.

Schema Migration

• Drops el_tx_events table
• Adds data_status and data_size columns to el_blocks
• Creates el_transactions_internal and el_event_index tables with indexes

Files Changed

41 files changed, ~5800 insertions, ~530 deletions across blockdb, DB layer, indexer, RPC client, handlers, and templates.

Screenshots