Hybrid Search: When Pure Vector Search Isn't Enough

User searches: "error KB-4092 kubernetes pod crash."

Your vector database returns results about Kubernetes pod crashes, container restarts, and OOMKill errors. All semantically relevant. None mention error code KB-4092.

The actual answer is in a support ticket titled "KB-4092: Pod CrashLoopBackOff due to missing ConfigMap." It's in your database. But vector search didn't surface it because "KB-4092" has no semantic meaning — it's an arbitrary code. Cosine similarity between "KB-4092" and "Pod CrashLoopBackOff" is essentially zero.

This is where pure vector search fails. And it fails more often than you'd think — product IDs, error codes, people's names, acronyms, specific versions, exact phrases. All the stuff that keyword search has handled perfectly for 30 years.

Hybrid search combines vector search with keyword search. And it's not optional — it's essential for production RAG.

The Two Worlds of Search

Keyword Search (BM25)

BM25 (Best Matching 25) is the algorithm behind traditional search engines. It's been the standard since the 1990s.

How it works:

Tokenize the query into keywords: "kubernetes", "pod", "crash", "KB-4092"

Look up which documents contain those keywords (inverted index)

Score each document based on:

Term frequency (TF): How often the keyword appears in the document

Inverse document frequency (IDF): How rare the keyword is across all documents

Document length: Shorter documents with the keyword score higher

python

Simplified BM25 scoring

def bm25_score(query_terms, document, corpus):
score = 0
    for term in query_terms:
        tf = document.count(term)  # Term frequency
        df = sum(1 for doc in corpus if term in doc)  # Document frequency
        idf = log((len(corpus) - df + 0.5) / (df + 0.5))  # Inverse doc frequency
        score += idf * (tf * (k1 + 1)) / (tf + k1 * (1 - b + b * len(doc) / avg_doc_len))
    return score

BM25 excels at:

Exact term matching ("KB-4092")

Rare, specific terms (high IDF = high signal)

Acronyms, product names, codes

"Did the document contain this exact thing?"

BM25 fails at:

Synonyms ("car" vs "automobile" — different tokens, zero match)
Paraphrasing ("how to fix" vs "troubleshooting steps")

Conceptual similarity ("machine learning" vs "neural networks")

Vector Search (Semantic)

You know this from the last few episodes. Embed the query, find nearest vectors.

Vector search excels at:

Semantic similarity ("fix" ≈ "troubleshoot" ≈ "resolve")

Paraphrase matching

Conceptual relevance

Cross-language similarity

Vector search fails at:

Exact term matching (error codes, IDs)

Rare tokens that weren't well-represented in training

When the user knows exactly what they want by name

The Obvious Solution

Use both. Combine the results.

How Hybrid Search Works

User Query: "error KB-4092 kubernetes pod crash"

↓
         ┌─────────┴──────────┐
         ↓                    ↓

Vector Search BM25 Search

(semantic) (keyword)

↓                    ↓

Results: Results:

1. Pod troubleshooting 1. KB-4092 support ticket ★
2. CrashLoopBackOff 2. KB-4091 related error
3. OOMKill guide 3. Kubernetes crash docs
4. Container restart 4. Error code reference

↓                    ↓
         └─────────┬──────────┘
                   ↓
            Fusion Algorithm
            (combine + re-rank)
                   ↓
            Final Results:
            1. KB-4092 support ticket ★ (BM25 hit + semantic)
            2. Pod troubleshooting (semantic)
            3. CrashLoopBackOff guide (both)
            4. Error code reference (BM25)

The key question: how do you combine the two result lists?

🔧

Reciprocal Rank Fusion (RRF)

RRF is the most popular fusion algorithm. It's simple, effective, and requires no tuning.

The idea: convert each result's position (rank) in each list into a score, then sum the scores.

RRF_score(doc) = Σ 1 / (k + rank(doc))

for each result list

Where k is a constant (typically 60) that prevents the top result from dominating.

Example

Vector search results: BM25 results:

Rank 1: Doc A Rank 1: Doc C

Rank 2: Doc B Rank 2: Doc A

Rank 3: Doc C Rank 3: Doc D

Rank 4: Doc D Rank 4: Doc B

RRF scores (k=60):

Doc A: 1/(60+1) + 1/(60+2) = 0.0164 + 0.0161 = 0.0325 ← Winner!

Doc B: 1/(60+2) + 1/(60+4) = 0.0161 + 0.0156 = 0.0317

Doc C: 1/(60+3) + 1/(60+1) = 0.0159 + 0.0164 = 0.0323

Doc D: 1/(60+4) + 1/(60+3) = 0.0156 + 0.0159 = 0.0315

Final ranking: A, C, B, D

Doc A appears in both lists at decent positions — it becomes the top result. This is RRF's superpower: documents that appear in multiple lists get boosted, regardless of the scoring scale.

python

def reciprocal_rank_fusion(result_lists, k=60):
scores = {}
    for results in result_lists:
        for rank, doc_id in enumerate(results, 1):
            if doc_id not in scores:
                scores[doc_id] = 0
            scores[doc_id] += 1 / (k + rank)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)
vector_results = ["doc_a", "doc_b", "doc_c", "doc_d"]
bm25_results = ["doc_c", "doc_a", "doc_d", "doc_b"]
final = reciprocal_rank_fusion([vector_results, bm25_results])
print(final)  # [('doc_a', 0.0325), ('doc_c', 0.0323), ...]

Why RRF Works

Scale-independent — BM25 scores (0-20) and cosine similarity (0-1) don't need normalization

No weights to tune — it just works, no hyperparameters beyond k

Robust — documents that appear in only one list still get ranked

📊

Weighted Hybrid Search

Sometimes you know one search type matters more. Weighted combination lets you control the balance:

python

def weighted_hybrid(vector_scores, bm25_scores, alpha=0.7):
"""
    alpha: weight for vector search (0 = all BM25, 1 = all vector)
    """
    # Normalize scores to 0-1
    v_norm = normalize(vector_scores)
    b_norm = normalize(bm25_scores)
combined = {}
    for doc_id in set(list(v_norm.keys()) + list(b_norm.keys())):
        v_score = v_norm.get(doc_id, 0)
        b_score = b_norm.get(doc_id, 0)
        combined[doc_id] = alpha * v_score + (1 - alpha) * b_score
return sorted(combined.items(), key=lambda x: x[1], reverse=True)

Alpha Behavior

The 0.7 sweet spot works for most RAG applications — primarily semantic with keyword as a safety net.

Hybrid Search in Practice

Qdrant

python

from qdrant_client import QdrantClient, models

client = QdrantClient("localhost", port=6333)

Hybrid search with RRF

results = client.query_points(
collection_name="my_docs",
    prefetch=[
        models.Prefetch(
            query=query_vector,       # Vector search
            using="dense",
            limit=20,
        ),
        models.Prefetch(
            query=query_text,         # BM25 keyword search
            using="sparse",           # Sparse vector (BM25-like)
            limit=20,
        ),
    ],
    query=models.FusionQuery(fusion=models.Fusion.RRF),  # Combine with RRF
    limit=10,

)

Weaviate

graphql

{

Get {

Document(
      hybrid: {
        query: "error KB-4092 kubernetes pod crash"
        alpha: 0.7  # 0 = all BM25, 1 = all vector
      }
      limit: 10
    ) {
      title
      content
      _additional { score }
    }

}

}

PostgreSQL (pgvector + tsvector)

sql

-- Combine vector similarity with full-text search

WITH vector_results AS (

SELECT id, content,
           1 - (embedding <=> query_embedding) as vector_score
    FROM documents
    ORDER BY embedding <=> query_embedding
    LIMIT 20

),

text_results AS (

SELECT id, content,
           ts_rank(to_tsvector('english', content),
                   plainto_tsquery('english', 'KB-4092 kubernetes pod crash')) as text_score
    FROM documents
    WHERE to_tsvector('english', content) @@ plainto_tsquery('english', 'KB-4092 kubernetes pod crash')
    LIMIT 20

)

SELECT COALESCE(v.id, t.id) as id,

COALESCE(v.content, t.content) as content,
       0.7 * COALESCE(v.vector_score, 0) + 0.3 * COALESCE(t.text_score, 0) as combined_score

FROM vector_results v

FULL OUTER JOIN text_results t ON v.id = t.id

ORDER BY combined_score DESC

LIMIT 10;

When Pure Vector Fails: Real Examples

Query Vector Search Returns What You Actually Need Fix

These all share a pattern: identifiers, names, codes, and numbers that have no semantic meaning in embedding space.

🔬

Practical Takeaways

Hybrid search is mandatory for production RAG — pure vector misses exact matches

RRF is the safest fusion method — no tuning needed, works out of the box

Alpha = 0.7 (vector-dominant) works for most RAG applications

BM25 handles the long tail — error codes, IDs, names that embeddings can't capture

Your vector DB probably supports it — Qdrant, Weaviate, and pgvector all have hybrid built in

Test with real queries — especially queries containing specific identifiers

🛡️

What's Next?

Episode 21: Reranking — Hybrid search gives you a good candidate list, but the top results aren't necessarily the best. Cross-encoder rerankers score each result against the query with much higher accuracy — turning "good enough" results into "exactly right."

← Previous

Ep 19: Chunking Strategies

Next →

Ep 21: Reranking

Next: Episode 21 — Reranking

Your vector DB returns 10 results. The perfect answer is at position #7. Reranking fixes that.

This is part of a 98-episode series covering AI engineering from tokens to production deployment.