Hybrid Search Beats Pure Vector Search in Published Benchmarks

Vector search alone is the weakest retrieval setup that published benchmarks still treat as a baseline. Across 18 BEIR datasets, a BM25 first stage plus a small reranker scored 11% higher than BM25, while most dense retrievers of the period scored lower. Hybrid search for RAG is not a tuning trick. It is what the evidence has pointed to since 2021.

If you need the short answer first, here it is for a production retrieval augmented generation system. Run keyword and vector retrieval in parallel, merge the two ranked lists, and rerank the top 50 to 150 candidates before generation. Use Reciprocal Rank Fusion when you have no labelled queries. Switch to a tuned weighted combination once you have about 40. Test the reranker on your own queries before you trust it.

What hybrid search and a reranker actually do

Three components appear in every result below, so the terms need fixing first. Each one ranks documents differently, and the reason hybrid search works is that they fail on different queries.

BM25 is the standard keyword ranking function. It scores a document by how often your query terms appear in it, discounted for common words and long documents. It needs no training and it cannot understand a synonym.

Vector search, also called dense retrieval, converts the query and each document into an embedding and returns the nearest neighbours. It matches meaning rather than words. It also misses exact strings it has never seen, such as part numbers, error codes and internal product names.

Fusion: how two ranked lists become one

Hybrid search runs both retrievers and merges their output. There are two common merge methods, and the choice between them turns out to matter.

Reciprocal Rank Fusion (RRF) ignores the raw scores and uses only each document's position in each list. A document ranked highly by both retrievers rises to the top. It needs no calibration, which is why most vector databases ship it as the default.

Convex combination normalises both sets of scores to a common range and takes a weighted average. One weight, usually called alpha, decides how much the vector score counts against the keyword score. It needs tuning, and the evidence below says the tuning pays.

Rerankers: a second, slower opinion

A reranker is a model that reads the query and one candidate document together and outputs a relevance score. The usual form is a cross-encoder. Because it reads both texts jointly, it judges relevance far more precisely than an embedding comparison can.

That precision is expensive. A cross-encoder must run once per candidate, per query, so it cannot search a whole corpus. It only reorders a shortlist that a cheaper retriever has already produced. In every architecture in this post, retrieval decides what is possible and reranking decides what reaches the model.

BEIR: BM25 beat most dense retrievers out of domain

BEIR is the benchmark most retrieval claims are measured against. Thakur and colleagues published it at NeurIPS 2021 with 18 public datasets across 9 task types, from biomedical question answering to argument retrieval. They evaluated 10 retrieval systems without training any of them on the target data.

That zero-shot setup is the important part. It is the situation your RAG system is in when you point a general embedding model at your company's documents. The model has never seen your domain, your vocabulary or your question style.

The headline result has aged better than most 2021 machine learning findings. In the authors' words, BM25 remains a strong baseline for zero-shot text retrieval. Several neural retrievers that beat BM25 by 7 to 18 points on the in-domain MS MARCO set lost to it once the domain changed.

Where dense retrieval won, and where it lost

The average hides a split that is more useful than the average itself. Dense retrieval did not lose everywhere. It won on the datasets that look like its training data and lost on the ones that do not.

Figures from BEIR Table 2. Higher is better. The first three rows are cases where the dense model beat BM25. The next four are cases where it lost. The reranker beat BM25 on 16 of 18 datasets and lost on ArguAna and Touché-2020.

Read the TREC-COVID row twice. The dense model scored 0.481 on biomedical literature where plain keyword search scored 0.656. Specialist vocabulary is exactly what a general embedding model has seen least of, and specialist vocabulary is what most internal knowledge bases are made of.

I think this is the single most under-used fact in RAG design. Teams pick an embedding model from a leaderboard built on web text, then deploy it on contracts, clinical notes or support tickets, and never check it against keyword search on their own data.

Index size is part of the bill

BEIR also measured storage, which rarely makes it into the discussion. For the BioASQ corpus of roughly 15 million documents, the authors report that ColBERT needed about 900GB of index against 18GB for BM25. Dense single-vector models sat in between, at 3GB per million documents in their latency test. Adding BM25 to a vector stack is cheap in storage terms, which matters if you are already paying for vector database capacity priced by stored dimensions.

Hybrid fusion beat both retrievers on all 9 datasets tested

BEIR compared retrievers one at a time. The more direct test of hybrid search came in a study by Sebastian Bruch, Siyu Gai and Amir Ingber, first posted in October 2022 and revised in May 2023 with an ACM journal DOI. They fused BM25 with the small embedding model all-MiniLM-L6-v2 and measured the result on MS MARCO and 8 BEIR datasets.

Six of those datasets were out of domain for the embedding model. That makes them the closest match in the study to a RAG system running on private documents.

Per-dataset figures are from Table 2 of the paper, test split, convex combination at alpha 0.8, RRF at the standard constant of 60. The paper reports no cross-dataset average. The last row is our own unweighted mean, so treat it as a summary rather than a published result.

Three things stand out. On these six datasets, BM25 alone averaged higher than vector search alone. The convex combination beat the better single retriever on every dataset in the paper, all 9 including the in-domain three. And RRF did not: it fell below the best single retriever on HotpotQA, on MS MARCO and on Quora.

RRF is the safe default, not the best one

The authors are blunt about it. They found that a convex combination outperforms RRF in both in-domain and out-of-domain settings, and that it needs only a small set of labelled queries to tune its single weight. They also found RRF more sensitive to its own parameters than earlier work suggested, and that RRF tuned on one domain generalised poorly to others.

Elastic's search team reached a compatible conclusion on BEIR in July 2023. Their hybrid retrieval evaluation found RRF lifted average nDCG@10 by 18% over BM25 and 1.4% over their learned sparse model, with no tuning. A calibrated linear combination reached 24% and 6% respectively, and beat RRF with about 40 annotated queries. Elastic sells the search engine, so read it as a vendor measurement on a public benchmark, which is still more than most RAG advice offers.

My position: ship RRF on day one, because it cannot hurt you much. Then build the labelled set and move to a weighted combination. Leaving RRF in place forever is the most common way a team pays for hybrid search and collects half of its benefit.

Rerankers delivered the largest gain, at a latency price

Go back to the BEIR chart. The best performer was not a better embedding model. It was the simplest pipeline in the paper, BM25 retrieving 100 candidates and a 6-layer MiniLM cross-encoder reordering them.

That model averaged 11% above BM25 and beat it on 16 of 18 datasets. The authors' own summary is that the cross-attentional re-ranking model performs the best, and that cross-attention appears important for generalising to new domains.

The latency bill is set by the shortlist length

Those numbers are from 2021 hardware and a 2021 model, so do not copy them into a capacity plan. The shape is what transfers. Reranking cost scales with the number of candidates you score, so the depth of the shortlist is the main dial you control.

BEIR's authors put it directly: dense retrievers ran 20 to 30 times faster than the re-ranking models on the same GPU. In practice that means the reranker belongs at the end of the pipeline, on a list of tens or low hundreds of candidates, never on the corpus.

The trade is usually worth it for RAG, for a reason specific to generation. A chat answer already waits seconds for the language model. A few hundred extra milliseconds that remove a near-miss passage from the context window are cheaper than the wrong answer that passage would cause.

What the RAG-specific results add

BEIR and the fusion study measure ranking quality on question and passage sets. RAG adds a constraint they do not: only the top handful of chunks reaches the model. So the useful RAG metric is whether the right chunk lands in that handful.

Anthropic published exactly that measurement in its September 2024 contextual retrieval write-up. Across codebases, fiction, arXiv papers and science papers, Anthropic measured how often the right chunk missed the top 20. The rate was 5.7% with embeddings alone and 1.9% with contextual embeddings, contextual BM25 and a reranker. The reranker scored the top 150 candidates and passed 20 forward.

Each step stacked on the last. Adding keyword search to contextual embeddings took the failure rate from 3.7% to 2.9%. Reranking took it to 1.9%. That is a vendor measurement on undisclosed queries, so it is directional, but it matches the ordering in the academic results. How it fits into a full retrieval diagnosis is covered in the analysis of why most RAG failures are ranking failures.

The practical point is that none of these gains came from a bigger model or a longer context window. They came from retrieval plumbing that costs very little once built. That matters when the alternative being pitched to you is to pay for long-context calls instead of fixing retrieval.

Where this argument is weakest

The case for hybrid search rests partly on old models and partly on benchmarks that suit keyword search. Both weaknesses are real, and both deserve stating before you change an architecture on the strength of this post.

The dense models in BEIR are from 2021

TAS-B, ANCE and DPR are early dense retrievers. Current embedding models are far stronger on BEIR itself. SFR-Embedding-Mistral, one model the BRIGHT authors highlight, scored 59.0 nDCG@10 on the MTEB retrieval subset, a level no 2021 dense model approached. On generic web-style queries, a modern embedding model alone may beat BM25 outright.

The BEIR paper also flags its own bias. Some datasets were labelled using lexical systems, giving an unfair disadvantage to non-lexical approaches. Some of BM25's apparent strength is an artefact of how the answers were judged. The fusion study, which used a weak 384-dimension embedding model, is open to a similar objection.

What survives both objections is narrower and still decisive. Fusion beat the better single retriever on every dataset in the Bruch study, including the three where the vector model was trained in domain. Hybrid search does not require keyword search to be good. It requires the two methods to make different mistakes, and they do.

Rerankers can make reasoning-heavy retrieval worse

The BRIGHT benchmark, published in 2024 and revised in March 2025, tests 1,384 queries across 12 datasets where finding the right document requires reasoning. A typical task is matching a coding problem to the technique that solves it. Every retriever struggled. BM25 averaged 14.5 nDCG@10 and the best model in its main table, a 7.7 billion parameter Qwen embedding model, averaged 22.5.

Here the small cross-encoder that won BEIR backfired. Reranking BM25's top 10 with a MiniLM cross-encoder dropped its average from 14.3 to 13.1, and to 8.3 when reranking the top 100. A GPT-4 reranker on the same top 10 raised it to 17.4. The lesson is that a reranker is a model with a training distribution of its own, and it can be wrong in a new domain the same way an embedding model can.

So the claim in this post is narrower than its title. Hybrid retrieval is the better default on the evidence available. Reranking is usually the better default, but it is the component you must test on your own queries before you ship it.

A decision tree for hybrid search RAG

Most of the published evidence converts into four questions. Answer them in order and you have an architecture you can defend in a design review, and a test plan to check it.

The tree deliberately has one exit to pure vector search, and it is narrow. If your queries are conversational, your documents are general prose and nothing in them is an identifier, a modern embedding model alone may be enough. Very few production knowledge bases look like that.

Two implementation notes follow from the evidence. First, keep the two retrievers independent so you can measure each one alone, because a fusion score that hides which side found the answer cannot be debugged. Second, record the labelled set as a permanent asset. It is the input to fusion tuning, reranker selection and every model swap after that, and it belongs in the same harness as the rest of your LLM evaluation suite.

If you are designing the wider stack, the retrieval layer sits alongside access control, logging and model routing in the enterprise AI reference architecture. Hybrid retrieval is the part of that stack where published evidence is strongest, which makes it the cheapest decision to get right.

Frequently asked questions

Where to start this week

Pull 50 real questions from your logs or your support queue. For each one, have someone who knows the domain mark the passage that answers it. This takes an afternoon, and every decision in this post depends on it.

Then run three retrieval configurations against that set and record recall at 5 and at 20: your current vector search, BM25 alone, and the two fused with RRF. If BM25 alone beats your vector search on any slice of questions, you have found the reason your RAG answers are worse than your demo. The fix is a configuration change rather than a new model.

Related reading