0x50Lesson 6 of 11

Retrieve, filter, and rerank evidence

Cast a wide net, then reorder candidates with a cross-encoder and keep the set diverse.

20 min 6-question quiz 2 code exercises

By the end of this lesson you can

Explain the two-stage retrieve-then-rerank design
Contrast bi-encoders and cross-encoders
Remove near-duplicates with maximal marginal relevance

Good retrieval is usually a funnel:

Retrieve many candidates fast (say 50-100) with BM25, embeddings or both. Goal: high recall - don’t miss the answer.
Rerank them with a slower, smarter model and keep the best few (say 5-20). Goal: high precision.

Why two stages? An embedding model is a bi-encoder: it encodes the query and each chunk separately, so chunk vectors can be computed in advance - fast, but the model never sees the two texts together. A cross-encoder reranker reads the query and a chunk together and outputs a relevance score. It understands far better (“does this passage actually answer the question?”), but must run once per candidate at query time - too slow for millions of chunks, perfect for 50.

Try it

Up to 15 XP

Retriever or reranker?

Which stage is each statement about?

0 of 5 sortedScore 0/0

“Must search millions of chunks in milliseconds”
“Reads the question and the passage together”
“Optimized for recall: don’t lose the answer”
“Scores maybe 50 candidates per question”
“Uses vectors computed at indexing time”

Diversity: don’t send five copies

Corpora are full of near-duplicates: the same policy on three pages, overlapping chunks. Sending five versions of one fact wastes the context window and crowds out other evidence.

Maximal marginal relevance (MMR) picks chunks one at a time, balancing relevance against similarity to what’s already picked: $\text{score} = \lambda \cdot \text{relevance} - (1 - \lambda) \cdot \max \text{similarity to selected}$ . With $\lambda = 1$ it’s pure relevance; lower $\lambda$ favors variety.

two_stage.py

1candidates = {"c1": 0.82, "c2": 0.80, "c3": 0.79}
2cross_scores = {"c1": 0.30, "c2": 0.91, "c3": 0.55}
3reranked = sorted(candidates, key=lambda chunk: cross_scores[chunk], reverse=True)
4print(reranked)

Output

['c2', 'c3', 'c1']

Key takeaways

Retrieve broadly for recall, then rerank a short list for precision.
Cross-encoders read query and passage together - accurate but slow, so use them on few candidates.
Use MMR or deduplication so the context holds diverse evidence.

Lesson quiz

6 questions · pass with 5 correct · up to 50 XP

Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.

Practice: write Python

Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.

Exercise 1

Rerank and cut

+25 XP

The first line is a threshold. Each following line is id first_stage_score reranker_score. Print the candidates ordered by reranker score as id score (2 decimals), keeping only those at or above the threshold, then dropped: IDS (in input order) or dropped: none.

Four candidates

main.py

Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Exercise 2

Pick diverse chunks with MMR

+25 XP

Input: k lambda, then a JSON object of relevance scores, then a JSON object of pairwise similarities keyed "a b" (symmetric; missing pairs are 0).

Repeatedly pick the unselected chunk with the highest $\lambda \cdot \text{relevance} - (1 - \lambda) \cdot \max \text{similarity to the selected chunks}$ (the max is 0 when none are selected; ties go to the alphabetically first id). Print the picks space-separated.

Balanced (lambda 0.5)
Pure relevance (lambda 1)

main.py

Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Questions about this lesson

Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.

Loading posts…