Open
Conversation
![orca-security-eu[bot]](https://avatars.githubusercontent.com/in/276244?s=60&v=4){kind=small}
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
 Passed |
Secrets |  0  0  0  0 |
View in Orca |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## dev/1.33 #1798 +/- ##
============================================
+ Coverage 87.27% 87.32% +0.05%
============================================
Files 269 269
Lines 18777 18797 +20
============================================
+ Hits 16388 16415 +27
+ Misses 2389 2382 -7 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
dirkkul
force-pushed
the
filter_iterator
branch
from
4ae8fd9 to
a4a7917
Compare
Feed shard_cursors from each SearchReply back into the next SearchRequest, enabling per-shard pagination state for filtered iterators. Also updates protobuf definitions to include the shard_cursors field. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request adds support for filtering in iterators and implements shard cursor pagination for more efficient iteration over large collections in Weaviate. The changes include updates to gRPC protocol buffer definitions across multiple API versions (v4216, v5261, v6300) and modifications to the iterator and query execution logic.
Changes:
- Added shard cursor pagination support to enable efficient iteration with position tracking across shards
- Introduced filter parameter to iterator methods for server-side filtering during iteration
- Updated gRPC protocol buffer definitions to support new batch operations (BatchReferences, BatchStream) and filter operators (OPERATOR_CONTAINS_NONE, OPERATOR_NOT)
- Added version check to ensure filter functionality is only used with Weaviate 1.33.0+
Reviewed changes
Copilot reviewed 49 out of 49 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| weaviate/proto/v1//v1/.py(i) | Auto-generated protocol buffer files adding new RPC methods, filter operators, shard_cursors field, and generative AI providers |
| weaviate/collections/iterator.py | Added shard_cursors tracking and filters parameter to both sync and async iterators |
| weaviate/collections/grpc/query.py | Added shard_cursors parameter to query request building |
| weaviate/collections/queries/base_executor.py | Extract shard_cursors from response and include in QueryReturn |
| weaviate/collections/queries/fetch_objects/query/executor.py | Added shard_cursors parameter to fetch_objects method signatures |
| weaviate/collections/collection/sync.py | Added filters parameter to iterator with version check |
| weaviate/collections/collection/async_.py | Added filters parameter to async iterator with version check |
| weaviate/collections/classes/internal.py | Added shard_cursors field to QueryReturn dataclass |
| integration/test_iterator.py | Added test for iterator with filters and fixed existing test |
| integration/test_client.py | Updated generative model name in test |
| .github/workflows/main.yaml | Updated Weaviate 1.33 version reference |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.