Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 6 additions & 32 deletions tutorials/33_Hybrid_Retrieval.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -224,38 +224,14 @@
"embedding_retriever = InMemoryEmbeddingRetriever(document_store)\n",
"bm25_retriever = InMemoryBM25Retriever(document_store)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FC81c8RBrRFf"
},
"source": [
"### 2) Join Retrieval Results\n",
"\n",
"Haystack offers several joining methods in [`DocumentJoiner`](https://docs.haystack.deepset.ai/docs/documentjoiner) to be used for different use cases such as `merge` and `reciprocal_rank_fusion`. In this example, you will use the default `concatenate` mode to join the documents coming from two Retrievers as the [Ranker](https://docs.haystack.deepset.ai/docs/rankers) will be the main component to rank the documents for relevancy."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "GYso6_8BrhY8"
},
"outputs": [],
"source": [
"from haystack.components.joiners import DocumentJoiner\n",
"\n",
"document_joiner = DocumentJoiner()"
]
},
},
{
"cell_type": "markdown",
"metadata": {
"id": "r8_jHzmosbC_"
},
"source": [
"### 3) Rank the Results\n",
"### 2) Rank the Results\n",
"\n",
"Use the [TransformersSimilarityRanker](https://docs.haystack.deepset.ai/docs/transformerssimilarityranker) that scores the relevancy of all retrieved documents for the given search query by using a cross encoder model. In this example, you will use [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) model to rank the retrieved documents but you can replace this model with other cross-encoder models on Hugging Face."
]
Expand All @@ -279,9 +255,9 @@
"id": "Y5jzzvUIstQ4"
},
"source": [
"### 4) Create the Hybrid Retrieval Pipeline\n",
"### 3) Create the Hybrid Retrieval Pipeline\n",
"\n",
"Add all initialized components to your pipeline and connect them."
"Add all initialized components to your pipeline. Both the BM25 and Embedding retrievers are connected to the same ranker input."
]
},
{
Expand All @@ -298,13 +274,11 @@
"hybrid_retrieval.add_component(\"text_embedder\", text_embedder)\n",
"hybrid_retrieval.add_component(\"embedding_retriever\", embedding_retriever)\n",
"hybrid_retrieval.add_component(\"bm25_retriever\", bm25_retriever)\n",
"hybrid_retrieval.add_component(\"document_joiner\", document_joiner)\n",
"hybrid_retrieval.add_component(\"ranker\", ranker)\n",
"\n",
"hybrid_retrieval.connect(\"text_embedder\", \"embedding_retriever\")\n",
"hybrid_retrieval.connect(\"bm25_retriever\", \"document_joiner\")\n",
"hybrid_retrieval.connect(\"embedding_retriever\", \"document_joiner\")\n",
"hybrid_retrieval.connect(\"document_joiner\", \"ranker\")"
"hybrid_retrieval.connect(\"bm25_retriever\", \"ranker\")\n",
"hybrid_retrieval.connect(\"embedding_retriever\", \"ranker\")"
]
},
{
Expand Down