feat(ivf): Add numpy array support for IVF assemble functions#287
Open
feat(ivf): Add numpy array support for IVF assemble functions#287
Conversation
- Add assemble_from_clustering and assemble_from_file methods that accept numpy arrays directly (py_data parameter) in addition to file paths - Refactor assemble_dynamic_from_clustering to avoid double data copy: - Internal _impl function takes data by const reference - Smart dispatching detects ImmutableMemoryDataset and passes by reference - Add timing instrumentation for assemble operations - Add comprehensive tests for numpy array assembly in both IVF and DynamicIVF - Zero-copy path: numpy views passed directly without data duplication This enables users to build IVF indices from in-memory numpy arrays without intermediate file I/O, improving performance for dynamic workflows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This enables users to build IVF indices from in-memory numpy arrays without intermediate file I/O, improving performance for dynamic workflows.
- Internal _impl function takes data by const reference
- Smart dispatching detects ImmutableMemoryDataset and passes by reference