What is precision in retrieval?

Precision is the fraction of retrieved items that are actually relevant — true positives divided by total retrieved. High precision means few irrelevant chunks make it into your context. Low precision wastes context tokens and can distract the model with off-topic passages.

What is recall in retrieval?

Recall is the fraction of all relevant items that you successfully retrieved — true positives divided by total relevant. High recall means you are not missing important context. In RAG, low recall is dangerous because the model simply cannot answer correctly if the supporting passage was never retrieved.

What is the F1 score?

F1 is the harmonic mean of precision and recall, giving a single balanced number between 0 and 1. It rewards systems that are good at both rather than excelling at one. Use it when you care about precision and recall roughly equally; weight differently if one matters more for your use case.

What is mean reciprocal rank (MRR)?

MRR measures how high up the first relevant result appears. It is the reciprocal of the rank of the first relevant item (1 if it is first, 0.5 if second, and so on). It matters when only the top results feed the model, because a relevant chunk buried at rank 10 may never be used.

How many relevant IDs do I need to label?

Even a small labelled set per query is useful, but more is better and more representative. Label every chunk you know is genuinely relevant for the query. Unlabelled relevant chunks count as false negatives and will unfairly depress recall, so be thorough.

Retrieval Precision & Recall Calculator

If you are building RAG, your answer quality is capped by your retrieval quality. This calculator turns two ID lists — what your retriever returned and what is actually relevant — into the four metrics that matter: precision, recall, F1, and mean reciprocal rank.

How it works

You paste two sets of IDs. The tool computes the overlap and derives:

Precision = relevant retrieved ÷ total retrieved (are the results clean?).
Recall = relevant retrieved ÷ total relevant (did you miss anything?).
F1 = harmonic mean of precision and recall (a single balanced score).
MRR = 1 ÷ rank of the first relevant result (does a good hit appear near the top?).

All computation is local — nothing is uploaded.

Tips and interpretation

In RAG, recall is usually the more dangerous failure: if the supporting passage was never retrieved, the model cannot answer correctly no matter how clever the prompt. Low precision wastes context and can distract the model, but at least the right answer is present. Watch MRR when only the top few chunks feed the model — a relevant chunk buried at rank 10 may never be used. Label your ground-truth set thoroughly: any genuinely relevant chunk you forget to list counts as a false negative and unfairly drags down recall.