-
Notifications
You must be signed in to change notification settings - Fork 110
BE-317: HashQL: Implement TraversalExtraction pass
#8331
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: bm/be-315-hashql-differentiate-between-graphfilter-sources
Are you sure you want to change the base?
BE-317: HashQL: Implement TraversalExtraction pass
#8331
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
PR SummaryIntroduces graph read filter bodies and a new pass to extract vertex traversals, wiring it into the post-inline pipeline.
Written by Cursor Bugbot for commit 6ffafd7. This will update automatically on new commits. Configure here. |
| let new_local = if let Some(offset) = | ||
| (self.pending_locals_offset..self.pending_locals.len()).find(|&index| { | ||
| self.traversals | ||
| .lookup(self.total_locals.plus(self.pending_locals_offset + index)) | ||
| .is_some_and(|pending| pending.projections == place.projections) | ||
| }) { | ||
| self.total_locals.plus(offset) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Incorrect local ID calculation in deduplication logic
The code incorrectly computes the local ID when checking for existing extractions:
let new_local = if let Some(offset) =
(self.pending_locals_offset..self.pending_locals.len()).find(|&index| {
self.traversals
.lookup(self.total_locals.plus(self.pending_locals_offset + index)) // BUG: double-adds offset
.is_some_and(|pending| pending.projections == place.projections)
}) {
self.total_locals.plus(offset) // Returns: total_locals + indexProblem: Line 179 adds pending_locals_offset twice:
indexalready ranges frompending_locals_offset..pending_locals.len()- Then it adds
pending_locals_offset + indexagain - But line 182 returns
total_locals.plus(offset)whereoffset == index
Impact: The lookup searches for the wrong local ID (e.g., _110 instead of _105), causing deduplication to always fail. This creates duplicate extracted locals for the same projection within a basic block, violating the stated deduplication guarantee.
Fix:
let new_local = if let Some(offset) =
(self.pending_locals_offset..self.pending_locals.len()).find(|&index| {
self.traversals
.lookup(self.total_locals.plus(index)) // Remove duplicate offset
.is_some_and(|pending| pending.projections == place.projections)
}) {
self.total_locals.plus(offset)| let new_local = if let Some(offset) = | |
| (self.pending_locals_offset..self.pending_locals.len()).find(|&index| { | |
| self.traversals | |
| .lookup(self.total_locals.plus(self.pending_locals_offset + index)) | |
| .is_some_and(|pending| pending.projections == place.projections) | |
| }) { | |
| self.total_locals.plus(offset) | |
| let new_local = if let Some(offset) = | |
| (self.pending_locals_offset..self.pending_locals.len()).find(|&index| { | |
| self.traversals | |
| .lookup(self.total_locals.plus(index)) | |
| .is_some_and(|pending| pending.projections == place.projections) | |
| }) { | |
| self.total_locals.plus(offset) | |
Spotted by Graphite Agent
Is this helpful? React 👍 or 👎 to let us know.
Merging this PR will not alter performance
Comparing Footnotes
|
🤖 Augment PR SummarySummary: This PR extends HashQL MIR to support graph read filter bodies and adds a new post-inlining transform to extract traversal paths. Changes:
Technical Notes: Traversal extraction only runs on 🤖 Was this summary useful? React with 👍 or 👎 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| let new_local = if let Some(offset) = | ||
| (self.pending_locals_offset..self.pending_locals.len()).find(|&index| { | ||
| self.traversals | ||
| .lookup(self.total_locals.plus(self.pending_locals_offset + index)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The per-basic-block dedup check seems to miscompute the candidate local: index already ranges over absolute pending_locals indices, but the lookup uses pending_locals_offset + index, which will overshoot after the first block and prevent dedup in later blocks. This can lead to multiple extracted locals (and traversal entries) for the same projection within a single basic block.
🤖 Was this useful? React with 👍 or 👎
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## bm/be-315-hashql-differentiate-between-graphfilter-sources #8331 +/- ##
==============================================================================================
+ Coverage 66.47% 66.65% +0.18%
==============================================================================================
Files 762 764 +2
Lines 67681 68084 +403
Branches 3799 3812 +13
==============================================================================================
+ Hits 44988 45382 +394
- Misses 22158 22164 +6
- Partials 535 538 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1001 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1526 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 51 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 107 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 25 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |

🌟 What is the purpose of this PR?
Add support for graph read filter bodies in MIR, enabling property path extraction for graph traversal.
🔍 What does this change?
GraphReadFilteras a new body source type in the MIR builder guideTraversalExtractionpass to materialize vertex projections in graph read filtersPostInlineto run the traversal extraction pass and collect resultsappendmethod toIdVecfor more efficient vector operationstype_id_uncheckedandDisplayimplementation forPlacePre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
🛡 What tests cover this?