Skip to content

fix(server): warn and skip non-numeric filenames in consumer offset directories instead of panicking#3135

Open
atharvalade wants to merge 5 commits intoapache:masterfrom
atharvalade:fix/panic-on-non-numeric-consumer-offset-filenames
Open

fix(server): warn and skip non-numeric filenames in consumer offset directories instead of panicking#3135
atharvalade wants to merge 5 commits intoapache:masterfrom
atharvalade:fix/panic-on-non-numeric-consumer-offset-filenames

Conversation

@atharvalade
Copy link
Copy Markdown
Contributor

@atharvalade atharvalade commented Apr 15, 2026

Which issue does this PR close?

Closes #3132

Rationale

An operator accidentally placing a file (e.g. .DS_Store, editor swap file, backup) in the consumer offsets directory crashes the server on startup due to a panic! on non-numeric filenames.

What changed?

Both load_consumer_offsets and load_consumer_group_offsets in core/server/src/streaming/partitions/storage.rs panicked when encountering files with non-numeric names during startup offset loading.

The fix replaces both panic! calls with warn! log messages and continue, so the server gracefully skips unexpected files. Added 9 integration tests in core/integration/tests/storage/consumer_offsets.rs covering valid files, non-numeric files (.DS_Store, backup.bak), empty directories, subdirectories, and nonexistent paths for both functions.

Local Execution

  • Passed
  • Pre-commit hooks ran

AI Usage

  1. Opus 4.6
  2. Minimal AI used
  3. All tests verified locally with cargo test, cargo clippy, cargo fmt --check, and CI scripts
  4. Yes, all code can be explained

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 15, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.69%. Comparing base (611fca0) to head (5ef2977).

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #3135       +/-   ##
=============================================
- Coverage     74.10%   52.69%   -21.41%     
  Complexity      943      943               
=============================================
  Files          1159     1157        -2     
  Lines        102033    90321    -11712     
  Branches      79083    67389    -11694     
=============================================
- Hits          75607    47596    -28011     
- Misses        23765    40157    +16392     
+ Partials       2661     2568       -93     
Components Coverage Δ
Rust Core 46.47% <100.00%> (-28.86%) ⬇️
Java SDK 60.14% <ø> (ø)
C# SDK 69.07% <ø> (-0.31%) ⬇️
Python SDK 81.43% <ø> (ø)
Node SDK 91.53% <ø> (ø)
Go SDK 39.43% <ø> (ø)
Files with missing lines Coverage Δ
core/server/src/streaming/partitions/storage.rs 71.50% <100.00%> (+4.30%) ⬆️

... and 277 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

If you need a review, please ensure CI is green and the PR is rebased on the latest master. Don't hesitate to ping the maintainers - either @core on Discord or by mentioning them directly here on the PR.

Thank you for your contribution!

@github-actions github-actions Bot added the stale Inactive issue or pull request label Apr 23, 2026
@hubcio
Copy link
Copy Markdown
Contributor

hubcio commented Apr 23, 2026

can you make it a integration test instead of unit? code in core/server will be replaced when we introduce clustering, thus test will be lost.

@github-actions github-actions Bot removed the stale Inactive issue or pull request label Apr 24, 2026
@atharvalade atharvalade force-pushed the fix/panic-on-non-numeric-consumer-offset-filenames branch from 4fc2476 to 39db2e4 Compare April 24, 2026 15:30
@atharvalade
Copy link
Copy Markdown
Contributor Author

can you make it a integration test instead of unit? code in core/server will be replaced when we introduce clustering, thus test will be lost.

Good call, moved the tests from core/server unit tests to core/integration/tests/storage/consumer_offsets.rs. They follow the same pattern as tests/state/ (direct server:: imports, no full harness needed). This way they survive the server ti server-ng migration and can be re-run against the new binary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

server: panic on non-numeric filenames in consumer offset directories

2 participants