Add AdaBoost classifier to linfa-ensemble #427
Merged
+888
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implements SAMME (Stagewise Additive Modeling using a Multiclass Exponential loss function) algorithm for multi-class classification using ensemble learning. ## Features - Sequential boosting with adaptive sample weighting - Multi-class classification support (SAMME algorithm) - Weighted voting for final predictions using model alpha values - Automatic convergence handling and early stopping - Resampling-based approach compatible with any base learner ## Implementation Details - AdaBoost struct with model weights (alpha values) tracking - AdaBoostParams following ParamGuard pattern for validation - Configurable n_estimators and learning_rate hyperparameters - Full trait implementations: Fit, Predict, PredictInplace - Comprehensive error handling with proper error types ## Testing - 12 unit tests covering parameter validation and model training - 6 doc tests for API documentation - Achieves 90-93% accuracy on Iris dataset with decision stumps - Tests for different learning rates and tree depths ## Documentation - Extensive inline documentation with algorithm explanation - Working example (adaboost_iris.rs) with multiple configurations - References to original AdaBoost paper (Freund & Schapire, 1997) - Comparison with scikit-learn implementation ## Performance - Successfully trains on Iris dataset (150 samples, 3 classes) - Supports decision stumps (depth=1) and shallow trees - Model weights properly reflect learner performance 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixes rustdoc warning about redundant explicit links. Changed [AdaBoost](AdaBoost) to [AdaBoost] as recommended by rustdoc linter.
Adds three new tests to improve code coverage: - test_adaboost_early_stopping_on_perfect_fit: Tests early stopping on linearly separable data - test_adaboost_single_class_error: Tests error handling for single-class datasets - test_adaboost_classes_method: Tests that classes are properly identified This should improve patch coverage from 81.69% to ~85%+
Fix import ordering and line wrapping to match rustfmt standards.
Implements all requested changes from PR review: 1. Replace rand:: imports with ndarray_rand::rand:: for consistency 2. Change sample_weights from f32 to f64 for better precision 3. Fix learning_rate cancellation bug in weight update formula - Previously: weight *= ((alpha / learning_rate) as f32).exp() - Now: weight *= alpha.exp() - This ensures learning_rate actually affects sample weight updates 4. Fix classes field to store actual labels (T::Elem) instead of usize - Made AdaBoost struct generic over label type L - Stores original class labels for proper type safety 5. Remove duplicate y_array definition in predict_inplace 6. Add base learner error details to error message for better debugging 7. Add test_adaboost_different_learning_rates to verify learning_rate effects on model weights All tests passing with no warnings or clippy issues. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #427 +/- ##
==========================================
+ Coverage 76.92% 77.00% +0.07%
==========================================
Files 104 106 +2
Lines 7321 7461 +140
==========================================
+ Hits 5632 5745 +113
- Misses 1689 1716 +27 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
follow-up of #423