Skip to content

MB-69881: Re-architect vector search #68

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
CascadingRadium wants to merge 2 commits into
base: master
Choose a base branch
from
Open

MB-69881: Re-architect vector search #68

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
012edde
fix
CascadingRadium Dec 24, 2025
6ce2ed1
refactor interface
CascadingRadium Dec 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 13 additions & 7 deletions segment_vector.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ package segment
import (
"encoding/json"

index "github.com/blevesearch/bleve_index_api"
"github.com/RoaringBitmap/roaring/v2"
index "github.com/blevesearch/bleve_index_api"
)

type VecPostingsList interface {
Expand Down Expand Up @@ -58,11 +58,18 @@ type VecPostingsIterator interface {
}

type VectorIndex interface {
// @params: Search params for backing vector index (like IVF, HNSW, etc.)
// Search performs a kNN search for the given query vector and returns a postings list.
// - qVector: the query vector
// - k: the number of similar vectors to return
// - params: additional search parameters
Search(qVector []float32, k int64, params json.RawMessage) (VecPostingsList, error)
// @eligibleDocIDs: DocIDs in the segment eligible for the kNN query.
SearchWithFilter(qVector []float32, k int64, eligibleDocIDs []uint64,
params json.RawMessage) (VecPostingsList, error)
// SearchWithFilter performs a kNN search for the given query vector, filtering results based on eligible document IDs.
// - qVector: the query vector
// - k: the number of similar vectors to return
// - eligibleList: list of eligible document IDs to consider
Comment on lines +66 to +69
Copy link

Copilot AI Dec 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation comment states "filtering results based on eligible document IDs" which might be slightly misleading since the new API abstracts away the specific representation of eligible documents. Consider updating to: "filtering results based on eligible documents" to better reflect the abstraction introduced by EligibleDocumentList.

Suggested change
// SearchWithFilter performs a kNN search for the given query vector, filtering results based on eligible document IDs.
// - qVector: the query vector
// - k: the number of similar vectors to return
// - eligibleList: list of eligible document IDs to consider
// SearchWithFilter performs a kNN search for the given query vector, filtering results based on eligible documents.
// - qVector: the query vector
// - k: the number of similar vectors to return
// - eligibleList: list of eligible documents to consider

Copilot uses AI. Check for mistakes.
// - params: additional search parameters
SearchWithFilter(qVector []float32, k int64, eligibleList index.EligibleDocumentList, params json.RawMessage) (VecPostingsList, error)
// Close releases any resources held by the VectorIndex.
Close()
Size() uint64

Expand All @@ -71,8 +78,7 @@ type VectorIndex interface {

type VectorSegment interface {
Segment
InterpretVectorIndex(field string, requiresFiltering bool, except *roaring.Bitmap) (
VectorIndex, error)
InterpretVectorIndex(field string, except *roaring.Bitmap) (VectorIndex, error)
}

type VecPosting interface {
Expand Down
Loading