[SYSTEMDS-3948] Row-wise Sparsity Estimator#2466
Open
ywcb00 wants to merge 6 commits intoapache:mainfrom
Open
Conversation
…ise sparsity estimator works for the matrix multiplication and bind test cases for now
…to the unity tests for sparsity estimation
…ontainer for row wise sparsity vectors to simplify access and allow storing it with chain nodes
…mator with element-wise and single operations
…wise and single operations NOTE: using average case estimation per row
Baunsgaard
reviewed
May 7, 2026
Contributor
Baunsgaard
left a comment
Baunsgaard
left a comment
There was a problem hiding this comment.
Some comments. hope it helps.
| } | ||
|
|
||
|
|
||
| // Row Wise Sparsity Estimator |
Contributor
There was a problem hiding this comment.
You can optionally have all of these classes extend another class where you add these tests. Instead of copying the code on all of them.
Comment on lines
+242
to
+251
| // Row Wise Sparsity Estimator | ||
| // @Test | ||
| // public void testRowWiseEqzero() { | ||
| // runSparsityEstimateTest(new EstimatorRowWise(), m, k, sparsity, eqzero); | ||
| // } | ||
|
|
||
| // @Test | ||
| // public void testRowWiseDiag() { | ||
| // runSparsityEstimateTest(new EstimatorRowWise(), m, m, sparsity, diag); | ||
| // } |
| case PLUS: | ||
| case NEQZERO: | ||
| case EQZERO: | ||
| return new MatrixCharacteristics(nodeLeft.getRows(), nodeLeft.getCols(), |
Contributor
There was a problem hiding this comment.
consider calling the getters above the switch case, to avoid all these repeated calls.
| } | ||
| } | ||
|
|
||
| public static class RSVector { |
Contributor
There was a problem hiding this comment.
Wrapping the double[] primitive is not really good practice. This adds an indication that the JIT compiler have to resolve, and slows you down. I would suggest not adding this to the code. and instead on the individual functions code up the solution without this functional indirection.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi,
this PR adds the row-wise sparsity estimator from:
Lin, Chunxu, Wensheng Luo, Yixiang Fang, Chenhao Ma, Xilin Liu and Yuchi Ma;
On Efficient Large Sparse Matrix Chain Multiplication;
Proceedings of the ACM on Management of Data 2 (2024): 1 - 27.
Note that the row sparsity propagation, as described in the publication, applies to MM chains only. Other operations use fallback methods for sparsity estimation w/ row sparsity vectors, which create a cut in the sparsity estimation DAG.