Improve RAG with a Reranker

Two stages beat one

Vector search is fast but approximate. A reranker (a cross-encoder) re-scores the top results by reading the query and each chunk together — more accurate, but slower, so you only run it on the top ~50.

# retrieve 50 cheaply, then rerank to the best 5
candidates = vector_search(query, k=50)
top5 = reranker.rank(query, candidates)[:5]

The payoff

Reranking often gives the biggest single jump in RAG answer quality after good chunking. Hosted options (Cohere, Voyage) and open models (BGE reranker) both work well.