The hybrid search bm25 vector approach combines two distinct retrieval methods: lexical search (BM25) for exact term matching and semantic search (vector) for conceptual understanding. By merging the results of both approaches, hybrid systems can deliver more relevant and robust results than either method could alone. This architecture is a common component in advanced AI systems, including many Retrieval-Augmented Generation (RAG) pipelines.

Key takeaways

  • Complementary Strengths: Hybrid search combines two types of retrieval. BM25 excels at matching exact terms, product codes, and rare keywords. Vector search excels at understanding paraphrases, related concepts, and the user's intent. Together, they cover a wider range of queries.
  • A Multi-Step Process: Hybrid search is a pipeline. First, it queries a keyword index (BM25) and a vector index in parallel. Next, it takes the two separate lists of results and fuses them into a single, reranked list for the user.
  • Fusion is Key: You cannot simply add BM25 and vector scores together because their scales are incompatible. A common and robust fusion method is Reciprocal Rank Fusion (RRF), which combines results based on their rank position, not their raw scores.
  • Evaluation is Non-Negotiable: A hybrid system is not automatically better. Success requires careful tuning and evaluation. You must measure its performance against both a BM25-only and a vector-only baseline to prove its value.
  • When to Use It: Consider hybrid search when your documents contain a mix of specific identifiers (like error codes or SKUs) and descriptive, natural language text.

Hybrid search is an information retrieval technique that combines two different search methods: lexical search and semantic search. It merges the results from both to produce a single, more relevant ranked list. This approach captures both the literal words in a query and its underlying conceptual meaning.

The two components are often called sparse and dense retrieval.

  • Sparse Retrieval (Lexical): This method represents documents as a sparse matrix, counting which words appear where. BM25 (Best Match 25) is the most common sparse retrieval algorithm. It finds documents containing the exact query keywords, but it's smarter than a simple keyword match. It gives more weight to rare words and considers how often a word appears in a document, a core principle of its probabilistic framework (BM25 and Beyond).

  • Dense Retrieval (Semantic): This method uses an embedding model to convert both the query and the documents into numerical vectors. A vector is a list of numbers representing a piece of text's position in a high-dimensional "meaning space." The system then finds documents whose vectors are closest to the query's vector, indicating semantic similarity, a key concept in dense retrieval models (Sparse Meets Dense). This allows it to find a document about "car trouble" even if the query was "automotive problems."

By combining them, hybrid search gets the best of both worlds. BM25 ensures that documents with exact, critical keywords are found, while vector search finds conceptually related documents that might use different wording.

How Does a Hybrid Search System Work?

A hybrid search system with BM25 and vector retrieval operates as a parallel pipeline that retrieves two sets of results and then intelligently merges them. The system doesn't search a single, combined index. Instead, it queries two separate indexes and fuses the outputs before presenting the final ranking to the user.

Here is a step-by-step breakdown of the process.

  1. Query Processing: When a user submits a query, the system sends it down two parallel paths.

    • For BM25: The query is tokenized (broken into words or sub-words), and common stop words might be removed. This prepares it for the lexical index.
    • For Vector Search: The query is fed into an embedding model, which converts the text into a dense vector.
  2. Dual Retrieval: The system queries both indexes simultaneously with the processed queries.

    • The BM25 index returns a ranked list of documents based on its lexical scoring formula. The formula considers term frequency (TF), inverse document frequency (IDF), and document length to score relevance.
    • The vector index searches for the document vectors most similar to the query vector (e.g., using cosine similarity or dot product), returning a separate ranked list.
  3. Result Fusion: The two ranked lists are combined into one. This is the most critical step. Since BM25 scores and vector similarity scores are not on the same scale, they cannot be added directly, a common challenge in hybrid systems (Sparse Meets Dense). Instead, systems typically use a rank-based method like Reciprocal Rank Fusion (RRF), which we'll explore in the next section.

  4. Optional Reranking: For applications requiring the highest possible precision, the fused list of top candidates can be passed to a third model, often a cross-encoder. A cross-encoder jointly examines the query and each document to produce a highly accurate final score, a technique used in information retrieval competitions (NIT Agartala at TREC 2023). This step adds latency but can significantly improve the final ordering.

Combining Results: A Worked Example of Reciprocal Rank Fusion (RRF)

Reciprocal Rank Fusion (RRF) is a simple yet powerful technique to combine result lists from different search systems without needing to compare their incompatible scores. It creates a new score based on a document’s position in each list. The lower the rank (i.e., the better the position), the higher the contribution to the final RRF score.

The formula for the RRF score of a document d is:

Code
RRF_Score(d) = Σ (1 / (k + rank(d)))
  • rank(d) is the position of the document d in a result list (e.g., 1 for the top result, 2 for the second, and so on).
  • k is a tuning constant that moderates the influence of documents that appear high up in the rankings.
  • The Σ (sigma) means we sum this value across all the result lists a document appears in.

Illustrative Example

Let's say a user queries for "troubleshooting database connection". The system retrieves the top 3 results from both BM25 and Vector Search.

Step 1: Get results from both retrievers

Rank BM25 Results Vector Search Results
1 Doc A ("Fix DB connection errors") Doc C ("Guide to database timeouts")
2 Doc B ("Database setup guide") Doc A ("Fix DB connection errors")
3 Doc D ("Server maintenance log") Doc E ("Solving network latency")

Step 2: Calculate RRF scores for each unique document

For this example, we will use a k value of 60. If a document doesn't appear in a list, its contribution from that list is zero.

  • Doc A: Appears at rank 1 in BM25 and rank 2 in Vector.

    • Score = (1 / (60 + 1)) + (1 / (60 + 2))
    • Score = 0.01639 + 0.01613 = 0.03252
  • Doc B: Appears at rank 2 in BM25 only.

    • Score = (1 / (60 + 2)) + 0
    • Score = 0.01613
  • Doc C: Appears at rank 1 in Vector only.

    • Score = 0 + (1 / (60 + 1))
    • Score = 0.01639
  • Doc D: Appears at rank 3 in BM25 only.

    • Score = (1 / (60 + 3)) + 0
    • Score = 0.01587
  • Doc E: Appears at rank 3 in Vector only.

    • Score = 0 + (1 / (60 + 3))
    • Score = 0.01587

Step 3: Create the final fused ranking

Now, we rank all the documents by their final RRF score in descending order.

Rank Document Final RRF Score Why?
1 Doc A 0.03252 Appeared high in both lists.
2 Doc C 0.01639 Was the top semantic match.
3 Doc B 0.01613 Was a strong keyword match.
4 Doc D 0.01587 A weaker keyword match.
5 Doc E 0.01587 A weaker semantic match.

Doc A is the clear winner because it was ranked highly by both systems, demonstrating both lexical and semantic relevance. RRF successfully surfaced it to the top.

When Does BM25 Beat Vector Search (and Vice Versa)?

Neither BM25 nor vector search is universally superior; their strengths are complementary. A hybrid approach is valuable precisely because one method can succeed where the other fails. Understanding when each excels helps you diagnose search problems and decide if hybrid is the right fit.

BM25 (Lexical) Excels When... Vector Search (Semantic) Excels When...
Queries contain specific identifiers. For error codes (502 Bad Gateway), product SKUs (XPS-13-9310), or legal citations, exact matching is essential. Queries are conceptual or paraphrased. A user might search for "foods that give you energy" and expect a document about "high-carbohydrate snacks".
The vocabulary is niche or new. If a new acronym or brand name appears, an embedding model trained on older data may not understand it. BM25 will match it immediately. The query and document use different vocabulary. For example, "car problems" vs. "automotive issues." Vector search bridges this vocabulary gap.
Exact names or rare terms are crucial. Searching for a specific person, "Dr. Evelyn Reed," requires matching those exact tokens. The user's intent is ambiguous. For a short query like "data science," vector search can surface conceptually related documents about statistics, machine learning, and Python libraries.
You need to avoid false positives from "similar" concepts. In a medical context, "hypotension" (low blood pressure) is semantically close but clinically opposite to "hypertension" (high blood pressure). BM25 will not confuse them.

A simple way to decide:

  • Identifier-heavy search? Start with BM25.
  • Purely conceptual search? Start with vector search.
  • Mixed or unknown user queries? Hybrid is likely the strongest approach.

How Should You Evaluate a Hybrid Search System?

Implementing hybrid search is not a guaranteed win. You must rigorously evaluate its performance to ensure the added complexity is justified. A good evaluation framework measures whether relevant documents are being found and ranked correctly.

Follow these steps for a robust evaluation:

  1. Establish a Baseline. Before you build the hybrid system, measure the performance of its components separately. You need to know how BM25-only and vector-only search perform on their own. The hybrid system is only successful if it consistently outperforms both individual baselines.
  2. Curate a Test Set. Create a representative set of queries that reflect what your users will actually search for. For each query, manually identify and label the relevant documents in your collection. This "ground truth" is essential for calculating accuracy metrics.
  3. Measure Retrieval Recall. The first job of a search system is to find the right documents. Use a metric like Recall@K to measure if the relevant documents are present in the top K results returned by each retriever before fusion. If a document isn't in either initial list, it can never make it to the final ranking.
  4. Measure Ranking Quality. Once you have a fused list, you need to measure how well it's ranked.
    • Mean Reciprocal Rank (MRR): Measures how close the first correct answer is to the top of the list. It's good for question-answering tasks.
    • Normalized Discounted Cumulative Gain (nDCG): A more sophisticated metric that rewards systems for placing more relevant documents higher in the rankings. It's ideal for general search where multiple results can be relevant. For a deeper dive into these metrics, see our guide on RAG evaluation metrics.
  5. Perform Failure Analysis. Don't just look at the scores. For queries where the system performs poorly, investigate why. Was the relevant document missed by BM25? By the vector search? Or was it found by both but ranked incorrectly after fusion? This analysis will tell you which part of your system needs tuning.

Common Mistakes and Limits to Know

Building a production-ready hybrid search system involves navigating several common pitfalls and understanding its inherent limitations. Avoiding these mistakes can save significant time and lead to a much more effective system.

Common Mistakes

  • Naively Adding Scores: The most common mistake is to directly add the raw score from BM25 to the raw similarity score from a vector search. These scores are on completely different scales and distributions, a point highlighted in TREC RAG track research (UTokyo-HitU at TREC 2025). Doing this will produce unpredictable rankings where one system almost always dominates the other. Always use a rank-based fusion method like RRF or carefully normalize scores before combining them.
  • Retrieving Too Few Candidates: The fusion step can only work with the documents it is given. If you only retrieve the top 10 results from BM25 and the top 10 from vector search, a relevant document at position 11 in either list will never be seen. Increasing the number of candidates (e.g., to k=100) from each retriever increases the chance that all relevant documents make it to the fusion stage, improving recall at the cost of slightly higher latency.
  • Ignoring Pre-processing: The performance of a hybrid system depends heavily on how you prepare your data. For vector search, your chunking strategy is critical. For BM25, choices about tokenization, stemming (reducing words to their root form), and stop-word lists can have a huge impact on what gets matched. These pre-processing steps must be tuned and tested for your specific dataset.

Limitations

  • Increased Complexity and Cost: A hybrid system is two search systems in one. This means more infrastructure to maintain, two indexes to keep updated, and more complex query logic. This translates to higher operational costs and more potential points of failure.
  • Tuning is Required: There are no universal "best" settings. The optimal balance between BM25 and vector contributions, the number of candidates to retrieve, and the RRF k parameter all depend on your specific documents and user queries, making query-adaptive approaches an area of ongoing research (Query-Adaptive Hybrid Search). You must be prepared to experiment and tune these parameters based on your evaluation results.
  • Latency: Running two queries and a fusion step will almost always be slower than running a single query. While often measured in milliseconds, this added latency can be significant in real-time applications. Adding a cross-encoder for reranking will increase latency even further.

What to learn next

Sources and further reading