fix: real benchmarks, Cypher multi-row fix, and honest README#293
Open
aepod wants to merge 6 commits intoruvnet:mainfrom
Open
fix: real benchmarks, Cypher multi-row fix, and honest README#293aepod wants to merge 6 commits intoruvnet:mainfrom
aepod wants to merge 6 commits intoruvnet:mainfrom
Conversation
The execute_match() function previously collapsed all match results into a single ExecutionContext via context.bind(), which overwrote previous bindings. MATCH (n:Person) on 3 Person nodes returned only 1 row. This commit refactors the executor to use a ResultSet pipeline: - type ResultSet = Vec<ExecutionContext> - Each clause transforms ResultSet → ResultSet - execute_match() expands the set (one context per match) - execute_return() projects one row per context - execute_set/delete() apply to all contexts - Cross-product semantics for multiple patterns in one MATCH Also adds comprehensive tests: - test_match_returns_multiple_rows (the Issue ruvnet#269 regression) - test_match_return_properties (verify correct values per row) - test_match_where_filter (WHERE correctly filters multi-row) - test_match_single_result (1 match → 1 row, no regression) - test_match_no_results (0 matches → 0 rows) - test_match_many_nodes (100 nodes → 100 rows, stress test) Co-Authored-By: claude-flow <ruv@ruv.net>
RETURN n.name now produces column "n.name" instead of "?column?". Property expressions (Expression::Property) are formatted as "object.property" for column naming, matching standard Cypher behavior. Co-Authored-By: claude-flow <ruv@ruv.net>
Built from commit b2347ce Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
Built from commit 2adb949 Platforms updated: - linux-x64-gnu - linux-arm64-gnu - darwin-x64 - darwin-arm64 - win32-x64-msvc 🤖 Generated by GitHub Actions
Phase 2 of the ruvector remediation plan. Replaces simulated benchmarks with real measurements: - Python harness: hnswlib (C++) and numpy brute-force on same datasets - Rust test: ruvector-core HNSW with ground-truth recall measurement - Datasets: random-10K and random-100K, 128 dimensions - Metrics: QPS (p50/p95), recall@10 vs ground truth, memory, build time Key findings: - ruvector recall@10 is good: 98.3% (10K), 86.75% (100K) - ruvector QPS is 2.6-2.9x slower than hnswlib - ruvector build time is 2.2-5.9x slower than hnswlib - ruvector uses ~523MB for 100K vectors (10x raw data size) - All numbers are REAL — no hardcoded values, no simulation Co-Authored-By: claude-flow <ruv@ruv.net>
- Add independent benchmark report comparing ruvector-core vs hnswlib (C++) vs numpy brute-force - 10K vectors: 443 QPS / 98.3% recall (vs hnswlib 1153 QPS / 98.95% recall) - 100K vectors: 86 QPS / 86.75% recall (vs hnswlib 250 QPS / 74.27% recall) - Fix README "100% recall" claim — actual recall is 86.75-98.3% depending on scale - Fix "simulated Python baseline" — now compared against real hnswlib competitor - Include raw JSON data and full methodology documentation Co-Authored-By: claude-flow <ruv@ruv.net>
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MATCHqueries now return all matching rows instead of collapsing to 1Cypher Fix (Issue #269)
The
execute_match()function collapsed all match results into a singleHashMap— eachcontext.bind()call overwrote the previous value. Three matches → only the last survived.Fix: Implemented proper
ResultSetpipeline (Vec<ExecutionContext>) threaded through all statement executors. Cross-product expansion for multiple patterns in one MATCH.Key tests:
test_match_returns_multiple_rows— 3 nodes → 3 rows (was returning 1)test_match_return_properties— Alice + Bob both returned correctlytest_match_where_filter— WHERE correctly filters multi-row resultstest_match_many_nodes— 100 nodes → 100 rowsAlso fixed column name generation for property expressions (
n.nameinstead of?column?).Real Benchmark Results
All measurements are real — recall measured against brute-force ground truth, no simulated competitors.
10,000 Vectors (128d, M=32, ef=200)
100,000 Vectors (128d, M=32, ef=200)
Key Findings
README Corrections
Test plan
🤖 Generated with claude-flow