Upgrade to PebbleDB v2 + Add DefaultComparer Config Option #2695

Kbhat1 · 2026-01-12T23:17:45Z

Describe your changes and provide context

PebbleDB v2 Upgrade:

Benchmarks show significant improvements in write/read throughput and latency
Updated imports across sei-db/db_engine/pebbledb/ and sei-db/db_engine/pebbledb/mvcc/
Added required v2 comparer functions (ComparePointSuffixes, CompareRangeSuffixes)
Updated metrics API to use v2 field names

DefaultComparer Option:

UseDefaultComparer config enables Pebble's default lexicographic comparer instead of custom MVCCComparer
Only available programmatically via StateStoreConfig - intentionally NOT exposed in TOML
Useful for modules like EVM where MVCC iteration isn't needed
MVCC key encoding already uses BigEndian for versions, so ordering is compatible

Testing performed to validate your change

Added unit tests
Tested on node to verify state sync + queries worked

github-actions · 2026-01-12T23:18:28Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Jan 16, 2026, 8:02 PM

codecov · 2026-01-12T23:25:39Z

Codecov Report

❌ Patch coverage is 24.07407% with 41 lines in your changes missing coverage. Please review.
✅ Project coverage is 43.74%. Comparing base (91ef5cb) to head (8d16815).

Files with missing lines	Patch %	Lines
sei-db/state_db/ss/test/storage_test_suite.go	0.00%	36 Missing ⚠️
sei-db/db_engine/pebbledb/mvcc/db.go	69.23%	4 Missing ⚠️
sei-db/config/config.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2695      +/-   ##
==========================================
- Coverage   43.76%   43.74%   -0.02%     
==========================================
  Files        1914     1914              
  Lines      159504   159536      +32     
==========================================
- Hits        69801    69785      -16     
- Misses      83284    83329      +45     
- Partials     6419     6422       +3

Flag	Coverage Δ
sei-chain	`45.73% <24.07%> (-0.04%)`	⬇️
sei-cosmos	`38.20% <ø> (-0.01%)`	⬇️
sei-db	`68.72% <ø> (ø)`
sei-tendermint	`47.26% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-db/db_engine/pebbledb/batch.go	`100.00% <ø> (ø)`
sei-db/db_engine/pebbledb/db.go	`89.39% <100.00%> (-0.75%)`	⬇️
sei-db/db_engine/pebbledb/iterator.go	`100.00% <ø> (ø)`
sei-db/db_engine/pebbledb/mvcc/batch.go	`77.90% <ø> (ø)`
sei-db/db_engine/pebbledb/mvcc/comparator.go	`60.00% <ø> (ø)`
sei-db/db_engine/pebbledb/mvcc/iterator.go	`54.20% <ø> (ø)`
sei-db/config/config.go	`0.00% <0.00%> (ø)`
sei-db/db_engine/pebbledb/mvcc/db.go	`55.25% <69.23%> (ø)`
sei-db/state_db/ss/test/storage_test_suite.go	`0.00% <0.00%> (ø)`

... and 20 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blindchaser · 2026-01-13T15:47:07Z

sei-db/db_engine/pebbledb/mvcc/db.go


+	// Select comparer based on config. Note: UseDefaultComparer is NOT backwards compatible
+	// with existing databases created with MVCCComparer - Pebble will refuse to open due to
+	// comparer name mismatch. Only use UseDefaultComparer for NEW databases.


if mismatch happens, is there a way we can show some errors to the user?

or the pebble already has the error inside?

It will fail to open the db @blindchaser , so error will show up

Will paste snippet of the error

Kbhat1 · 2026-01-14T20:28:22Z

@yzang2019 @yzang2019 @jewei1997 please take a look again

sei-db/db_engine/pebbledb/mvcc/db_test.go

sei-db/db_engine/pebbledb/mvcc/comparator.go

Kbhat1 · 2026-01-16T16:11:40Z

sei-db/db_engine/pebbledb/mvcc/db_test.go

+// instead of MVCCComparer. This is useful for new databases that don't need backwards compatibility.
+// Note: Iterator tests are skipped because DefaultComparer doesn't have the Split function
+// configured for MVCC key encoding, so NextPrefix/SeekLT operations won't work correctly.
+func TestStorageTestSuiteDefaultComparer(t *testing.T) {


@masih likely a cleaner way to do this, but we want to eventually remove all iteration from state store. Right now its an artifact of cosmos (iteration) in v2

Kbhat1 · 2026-01-16T16:11:59Z

sei-db/state_db/ss/test/storage_test_suite.go

 }

 func (s *StorageTestSuite) TestDatabaseIteratorEmptyDomain() {
+	if slices.Contains(s.SkipTests, s.T().Name()) {


@masih again, likely a cleaner way to do this, but we want to eventually remove all iteration from state store. Right now its an artifact of cosmos (iteration) in v2

sei-db/db_engine/pebbledb/mvcc/comparator.go

masih · 2026-01-16T16:31:13Z

sei-db/db_engine/pebbledb/mvcc/db.go

 }

 func OpenDB(dataDir string, config config.StateStoreConfig) (*Database, error) {
 	cache := pebble.NewCache(1024 * 1024 * 32)


Unrelated to this PR: we probably should bump this value. It seems small for seid?

sei-db/db_engine/pebbledb/mvcc/db.go

masih · 2026-01-16T16:46:29Z

sei-db/db_engine/pebbledb/mvcc/db_test.go

+		"TestStorageTestSuiteDefaultComparer/TestDatabasePrune",
+		"TestStorageTestSuiteDefaultComparer/TestDatabasePruneKeepRecent",
+		"TestStorageTestSuiteDefaultComparer/TestParallelWriteAndPruning",
+	}


This is unfortunately error-prone, and can silently result in false negative test failures if we rename any of the tests.

My advice would be to avoid this. I see 2 options:

fork the test sets that we want to have going forward for the default comparator. Then when the MVCC comparator retires remove entire tests for it.

break up the test suits into 2, and have one extend the other where the extended suit runs the current superset of tests for MVCC. This solves your concern about code drift and would avoid the kind of config we see here as well as conditional skip tests. Let me know if you have questions on how to extend the suit while avoiding duplicate code

Yea I think for (1) I was concerned about the forks not matching. (2) seems reasonable, let me think on this though if there is a more efficient approach

Support default mvcccomparer

d2039fc

Set default

7ab6584

Kbhat1 requested review from blindchaser and yzang2019 and removed request for yzang2019 January 12, 2026 23:56

Kbhat1 added the non-app-hash-breaking label Jan 12, 2026

yzang2019 approved these changes Jan 12, 2026

View reviewed changes

Kbhat1 requested a review from jewei1997 January 13, 2026 05:03

blindchaser reviewed Jan 13, 2026

View reviewed changes

Migrate to Pebbledb v2

5a50264

Kbhat1 changed the title ~~Support Default Pebble Comparer~~ PebbleDB V2 and Default Comparer Jan 14, 2026

Kbhat1 changed the title ~~PebbleDB V2 and Default Comparer~~ Upgrade to PebbleDB v2 + Add DefaultComparer Config Option Jan 14, 2026

Kbhat1 requested review from blindchaser and yzang2019 January 14, 2026 17:53

Merge branch 'main' into PebbleDefaultMVCC

877b682

Kbhat1 added 2 commits January 14, 2026 18:06

Merge branch 'main' into PebbleDefaultMVCC

d261e7d

Update tests

97b9973

yzang2019 reviewed Jan 14, 2026

View reviewed changes

sei-db/db_engine/pebbledb/mvcc/db_test.go Show resolved Hide resolved

blindchaser approved these changes Jan 15, 2026

View reviewed changes

yzang2019 reviewed Jan 15, 2026

View reviewed changes

sei-db/db_engine/pebbledb/mvcc/comparator.go Outdated Show resolved Hide resolved

Kbhat1 requested a review from masih January 16, 2026 15:13

Kbhat1 commented Jan 16, 2026

View reviewed changes

masih reviewed Jan 16, 2026

View reviewed changes

Kbhat1 added 3 commits January 16, 2026 12:01

Merge branch 'main' into PebbleDefaultMVCC

115790b

compression

9ca5967

FormatMajorVersion set to default pebble.FormatVirtualSSTables

d5937f4

Kbhat1 added 2 commits January 16, 2026 14:57

Default comparator

28a37ea

Remove redundant check

8d16815

Upgrade to PebbleDB v2 + Add DefaultComparer Config Option #2695

Are you sure you want to change the base?

Upgrade to PebbleDB v2 + Add DefaultComparer Config Option #2695

Uh oh!

Conversation

Kbhat1 commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your changes and provide context

Testing performed to validate your change

Uh oh!

github-actions bot commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kbhat1 commented Jan 14, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kbhat1 Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kbhat1 commented Jan 12, 2026 •

edited

Loading

github-actions bot commented Jan 12, 2026 •

edited

Loading

codecov bot commented Jan 12, 2026 •

edited

Loading

Kbhat1 Jan 16, 2026 •

edited

Loading