fuzz: improve CI iteration strategy, add corpus minimization and summary reporting#4497
Draft
joostjager wants to merge 3 commits intolightningdevkit:mainfrom
Draft
fuzz: improve CI iteration strategy, add corpus minimization and summary reporting#4497joostjager wants to merge 3 commits intolightningdevkit:mainfrom
joostjager wants to merge 3 commits intolightningdevkit:mainfrom
Conversation
|
👋 Thanks for assigning @TheBlueMatt as a reviewer! |
e88eb4c to
b2ea4c4
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4497 +/- ##
==========================================
+ Coverage 86.19% 86.21% +0.01%
==========================================
Files 161 160 -1
Lines 107459 107514 +55
Branches 107459 107514 +55
==========================================
+ Hits 92621 92688 +67
+ Misses 12219 12208 -11
+ Partials 2619 2618 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
b7fd027 to
1e31813
Compare
Add handlers for SendTxInitRbf, SendTxAckRbf, SendTxRemoveInput, and
SendTxRemoveOutput in the chanmon_consistency fuzz target. These
variants were reachable but not matched, causing panics on the wildcard
arm ("Unhandled message event").
SendTxInitRbf became reachable after commit 5873660 added splicing
support without updating the fuzz target's message delivery logic.
AI tools were used in preparing this commit.
The sanity check (cargo test on fuzz targets) doesn't use the restored corpus and was blocking the actual fuzz run. Move it to a separate fuzz_sanity job so both run in parallel. AI tools were used in preparing this commit.
Replace the fixed 30s run_time with iteration counts scaled to 8x corpus size (plus a 1000 baseline) with a 10-minute hard cap per target. This ensures the full corpus is replayed with room for mutations, while small targets finish quickly. On main (and on PRs with the fuzz-minimize label), run honggfuzz corpus minimization after each target to prune inputs that don't contribute unique coverage, keeping the cache size manageable. Print a summary table at the end with per-target stats: iterations, corpus sizes before/after fuzzing and minimization, and run times. Other changes: - Use -q (quiet) to suppress per-iteration status output - Set 3s per-input timeout (-t 3) for all targets - Pass FUZZ_MINIMIZE env var from PR label in workflow AI tools were used in preparing this commit.
1e31813 to
63d6802
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fuzzing has become increasingly important with the recent wave of changes: async persist, channel manager refactors, splicing, and zero-fee channels. These are complex state machine changes where the fuzzer is one of our best tools for catching edge cases.
This PR gives the fuzz CI some overdue attention. The main goals are visibility into what the fuzzer is actually doing, and making the iteration budget more meaningful:
fuzz-minimizelabel.Builds on #4496, so the fuzzer actually passes.