feat(EvoAgentBench): add OpenAI-compatible embedding API for BrowseComp-Plus FAISS search#235
Open
CZH-THU wants to merge 1 commit into
Open
feat(EvoAgentBench): add OpenAI-compatible embedding API for BrowseComp-Plus FAISS search#235CZH-THU wants to merge 1 commit into
CZH-THU wants to merge 1 commit into
Conversation
…ISS search Allow query embeddings to be fetched via OpenAI-compatible APIs (vLLM, ModelScope, etc.) instead of loading models locally, with optional L2 normalization that skips already-normalized vectors. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BrowseComp-Plus previously loaded embedding models locally (HuggingFace + tevatron
DenseModel), which requires significant GPU memory. This change adds optional OpenAI-compatible embedding API support so query vectors can be fetched from external services (vLLM, sglang, ModelScope, etc.) while keeping the FAISS index on CPU.Changes:
faiss_searcher.py: Whenembedding.api_baseis configured, skip local model loading and call/v1/embeddingsvia the OpenAI client. Query text still uses the BrowseComp-Plustask_prefixfor index compatibility. Includes${ENV_VAR}resolution for secrets and_maybe_l2_normalize()to skip re-normalization when the API already returns unit vectors. Usesencoding_format="float"for ModelScope compatibility.start_mcp.py: Passesembedding.*fields frominformation_retrieval.yamlto MCP server CLI args.information_retrieval.yaml: Adds commented example config for the embedding API block.Local model loading remains the default when
embedding.api_baseis not set.Area
Verification
Checklist
.envfiles, dependency folders, or generated output.Notes for Reviewers
faiss_searcher.py), not inbrowsecomp_plus.py— the domain adapter only starts the MCP server viastart_mcp.py.normalizemust match how the FAISS index was built; defaultfalsealigns with the officialqwen3-embedding-8bindex. Whennormalize: true, already-normalized API responses are detected (L2 norm ≈ 1) and left unchanged.Qwen/Qwen3-Embedding-8B).searcher_type: faisssupports these new args;bm25/customsearchers are unaffected.Example config: