Skip to content

fuzz: improve CI iteration strategy, add corpus minimization and summary reporting#4497

Draft
joostjager wants to merge 3 commits intolightningdevkit:mainfrom
joostjager:fuzz-clean-logs
Draft

fuzz: improve CI iteration strategy, add corpus minimization and summary reporting#4497
joostjager wants to merge 3 commits intolightningdevkit:mainfrom
joostjager:fuzz-clean-logs

Conversation

@joostjager
Copy link
Contributor

@joostjager joostjager commented Mar 19, 2026

Fuzzing has become increasingly important with the recent wave of changes: async persist, channel manager refactors, splicing, and zero-fee channels. These are complex state machine changes where the fuzzer is one of our best tools for catching edge cases.

This PR gives the fuzz CI some overdue attention. The main goals are visibility into what the fuzzer is actually doing, and making the iteration budget more meaningful:

  • The old fixed 30-second wall clock limit meant fast targets wasted time while slow targets barely got through their corpus. Iteration counts now scale with corpus size, so every target replays its full corpus with room for new mutations.
  • On main, corpus minimization runs after each target to prune redundant inputs and keep the cache from growing indefinitely. PRs can opt in via the fuzz-minimize label.
  • A summary table is printed at the end of each run, giving visibility into per-target corpus sizes, deltas, and timings:
    ==== Fuzz Summary ====
    Target                                     Iters  Corpus     Fuzzed       Fuzz time    Minimized     Min. time
    ------                                     -----  ------  --------------- ---------  --------------- ---------
    base32_target                              27216    3277   3286 (+9)           0m4s    346 (-2940)        0m4s
    bech32_parse_target                        91160   11270  11285 (+15)          0m4s    959 (-10326)       0m8s
    bolt11_deser_target                        99144   12268  12273 (+5)           0m5s   1288 (-10985)      0m26s
    chanmon_consistency_target                 11904    1363   2531 (+1168)       6m56s   1908 (-623)       36m44s
    chanmon_deser_target                      150112   18639  18675 (+36)          0m8s   1250 (-17425)      0m15s
    ======================
    
  • The fuzz sanity check (cargo test on fuzz targets) is split into its own parallel job since it doesn't depend on the corpus.

Builds on #4496, so the fuzzer actually passes.

@ldk-reviews-bot
Copy link

ldk-reviews-bot commented Mar 19, 2026

👋 Thanks for assigning @TheBlueMatt as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@joostjager joostjager force-pushed the fuzz-clean-logs branch 2 times, most recently from e88eb4c to b2ea4c4 Compare March 19, 2026 15:08
@codecov
Copy link

codecov bot commented Mar 19, 2026

Codecov Report

❌ Patch coverage is 83.66013% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.21%. Comparing base (7d82e14) to head (b7fd027).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
lightning/src/ln/channel.rs 83.14% 14 Missing and 1 partial ⚠️
lightning/src/ln/channelmanager.rs 80.00% 6 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4497      +/-   ##
==========================================
+ Coverage   86.19%   86.21%   +0.01%     
==========================================
  Files         161      160       -1     
  Lines      107459   107514      +55     
  Branches   107459   107514      +55     
==========================================
+ Hits        92621    92688      +67     
+ Misses      12219    12208      -11     
+ Partials     2619     2618       -1     
Flag Coverage Δ
tests 86.21% <83.66%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joostjager joostjager force-pushed the fuzz-clean-logs branch 4 times, most recently from b7fd027 to 1e31813 Compare March 20, 2026 13:06
Add handlers for SendTxInitRbf, SendTxAckRbf, SendTxRemoveInput, and
SendTxRemoveOutput in the chanmon_consistency fuzz target. These
variants were reachable but not matched, causing panics on the wildcard
arm ("Unhandled message event").

SendTxInitRbf became reachable after commit 5873660 added splicing
support without updating the fuzz target's message delivery logic.

AI tools were used in preparing this commit.
The sanity check (cargo test on fuzz targets) doesn't use the
restored corpus and was blocking the actual fuzz run. Move it to
a separate fuzz_sanity job so both run in parallel.

AI tools were used in preparing this commit.
Replace the fixed 30s run_time with iteration counts scaled to 8x
corpus size (plus a 1000 baseline) with a 10-minute hard cap per
target. This ensures the full corpus is replayed with room for
mutations, while small targets finish quickly.

On main (and on PRs with the fuzz-minimize label), run honggfuzz
corpus minimization after each target to prune inputs that don't
contribute unique coverage, keeping the cache size manageable.

Print a summary table at the end with per-target stats: iterations,
corpus sizes before/after fuzzing and minimization, and run times.

Other changes:
- Use -q (quiet) to suppress per-iteration status output
- Set 3s per-input timeout (-t 3) for all targets
- Pass FUZZ_MINIMIZE env var from PR label in workflow

AI tools were used in preparing this commit.
@joostjager joostjager requested a review from TheBlueMatt March 20, 2026 14:42
@joostjager joostjager changed the title Fuzz clean logs fuzz: improve CI iteration strategy, add corpus minimization and summary reporting Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants