Skip to content

Refine the vectorize annotations to cover all cases (+ extensive tests/examples)#136

Open
sl-at-ibm wants to merge 6 commits intomainfrom
SL-deeper-vectorize-beyondNvidia-tests
Open

Refine the vectorize annotations to cover all cases (+ extensive tests/examples)#136
sl-at-ibm wants to merge 6 commits intomainfrom
SL-deeper-vectorize-beyondNvidia-tests

Conversation

@sl-at-ibm
Copy link
Copy Markdown
Collaborator

@sl-at-ibm sl-at-ibm commented Apr 13, 2026

This PR keeps track of the coverage for all vectorize-related situations. To do so it adds a bunch of create-X tests. Along the way, a change in the code (besides tests) is done.

{tables,collections} x {None, header, sharedSecret} x {create, get} x {typed, untyped}
    + nonstring 'parameters' where applies
    + superposition of CollectionVector and CollectionVectorize

The matrix is now complete and runs on the post-125-merged codebase... except one case (double attribute, typed collection, test CreateCollection_WithVectorizeSharedSecretDoubleAttribute_Typed) which:

  1. required adding a few constructors to ColumnVectorAttribute (part of this PR)
  2. still fails to pick up dimension an metric properly in the collection being created

Note that most of the 15 added tests require some DB/ops setup (API keys and such). These new tests are all disabled by default anyway.

Examples are tested on Astra/HCD, anything that applies.

Vectorize status:

COLLS   None                Typed   | OK    full test in branch
COLLS   shSecr              Typed   | OK    full test in branch
COLLS   shSecr(2attrs)      Typed   | OK    full test in branch
COLLS   Header(create+get)  Typed   | OK    full test in branch
------------------------------------+
COLLS   None                Untyped | OK    full test in branch
COLLS   shSecr              Untyped | OK    full test in branch
COLLS   Header(create+get)  Untyped | OK    full test in branch
====================================+
TABLE   None                Typed   | OK    full test in branch
TABLE   shSecr              Typed   | OK    full test in branch
TABLE   shSecr+params       Typed   | OK    full test in branch
TABLE   Header(create+get)  Typed   | OK    full test in branch
------------------------------------+
TABLE   None                Untyped | OK    full test in branch
TABLE   shSecr              Untyped | OK    full test in branch
TABLE   shSecr+params       Untyped | OK    full test in branch
TABLE   Header(create+get)  Untyped | OK    full test in branch

Note on header-based auth

The header is passed as a certain field within certain *CommandOption classes. There is an asymmetry here between tables and collections:

                    DatabaseCommandOptions
                    /                    \
DatabaseCollectionCommandOptions        DatabaseTableCommandOptions
                                          |
                                        CreateTableCommandOptions

All three "bottom" classes support the new <className>() { EmbeddingApiKey = embeddingAPIKey } pattern to be passed to the appropriate Get/Create x Table/Collection method: however there are two different ones for tables ... because of "ifNotExists" (which does not make sense when Getting a table, nor does it for a collection).

@sl-at-ibm sl-at-ibm force-pushed the SL-deeper-vectorize-beyondNvidia-tests branch from 52abad6 to b866c02 Compare April 16, 2026 14:09
@sl-at-ibm sl-at-ibm marked this pull request as ready for review April 16, 2026 14:59
@sl-at-ibm sl-at-ibm requested a review from toptobes April 16, 2026 14:59
@sl-at-ibm
Copy link
Copy Markdown
Collaborator Author

@stephenatsembit This is a little bit I think we missed in making 125 final (the coexistence of the ColumnVector and ColumnVectorize on a collection document class, I mean). See PR message.

@sl-at-ibm sl-at-ibm changed the title Showcase and tracking for all vectorize combinations (probably not for merging) Vectorize 'CreateX' matrix ( +necessary changes along the way) Apr 16, 2026
@sl-at-ibm sl-at-ibm changed the title Vectorize 'CreateX' matrix ( +necessary changes along the way) Refine the vectorize annotations to cover all cases (+ extensive tests/examples) Apr 17, 2026
@sl-at-ibm sl-at-ibm force-pushed the SL-deeper-vectorize-beyondNvidia-tests branch from 8527707 to fd9ce57 Compare April 17, 2026 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants