Combine lexical and vector retrieval

You have watched different methods win on different Queries. Word-based search handles DEP-1042. Vector search connects undo a bad release with the rollback guide. Structured lookup returns the Pro plan record without ranking anything.

I use hybrid retrieval when I want to combine more than one retrieval method. It keeps those strengths instead of forcing every Query through one index.

Fuse ranks, not raw scores

BM25 scores and cosine similarities live on different scales. Adding them directly would make whichever scale happens to be larger dominate the result.

Reciprocal Rank Fusion uses position instead. The complete calculation is short:

from collections.abc import Sequence


def reciprocal_rank_fusion(
    rankings: Sequence[Sequence[str]],
    *,
    rank_constant: int = 60,
) -> dict[str, float]:
    scores: dict[str, float] = {}

    for ranking in rankings:
        for rank, chunk_id in enumerate(ranking, start=1):
            scores[chunk_id] = (
                scores.get(chunk_id, 0.0)
                + 1.0 / (rank_constant + rank)
            )

    return dict(sorted(scores.items(), key=lambda item: item[1], reverse=True))


scores = reciprocal_rank_fusion([
    ["chunk_exact", "chunk_shared"],
    ["chunk_shared", "chunk_semantic"],
])

chunk_shared appears in both lists, so it rises. The formula gives a result more points when it appears near the top of either list.

Reciprocal Rank Fusion is usually shortened to RRF. Retrieve a wider candidate list from word search and another from vector search. Feed the ordered Chunk IDs to RRF, sort by the new score, and then load the corresponding Chunks.

Fusion is only one step in the request. Let us place it in the full retrieval order so filtering and deduplication happen in the right places.

Keep the order of operations visible

I use this sequence:

1. validate the Query and access filters;
2. filter eligible Sources;
3. retrieve word-search and vector-search candidates;
4. fuse their ranked Chunk IDs;
5. remove duplicates and cap repeated Chunks from one Source;
6. select the final candidates for Evidence or reranking.

Structured lookups can join after the Query has been mapped to a known tool. A plan record does not need to compete with prose on a made-up relevance scale.

The order above gives us a working hybrid retriever. The values inside it, such as how many candidates to fetch, still need to come from failed and successful Queries rather than guesswork.

Tune with failures, not instinct

Run the same Evaluation Set against word search, vector search, and hybrid retrieval. Look at which Query categories improve. Also inspect request time and repeated Chunks.

There is no universal top-K. If K is too small, you miss Evidence. If it is too large, generation receives redundant text and costs more. Compare several values and count how often the expected Source appears in the first K results.

When the expected Chunk is present but still too low in the fused list, a reranker may help. That is the problem we will test next.

Rerank measured candidates