Relevance Feedback (Rocchio)

You type a query into a search engine. The results are almost right — some hit the mark, others miss completely. So you click "more like this" or mark a few results as helpful and a few as useless. Somehow the next batch is better. What just happened?

Behind that improvement is a beautifully simple geometric idea from 1971: Rocchio's relevance feedback algorithm. Every document and every query lives in a high-dimensional space of words. Relevant documents cluster together in that space. If you point out which results were good, the algorithm can move your query vector toward the centroid of the good ones and away from the bad ones — getting geometrically closer to what you really wanted.

The update rule is exact: given a query $\mathbf{q}$, a set of relevant documents $D_R$ and a set of non-relevant documents $D_{NR}$, the new query is

$$\mathbf{q}' = \alpha\,\mathbf{q} + \beta\,\frac{1}{|D_R|}\sum_{d \in D_R}\mathbf{d} - \gamma\,\frac{1}{|D_{NR}|}\sum_{d \in D_{NR}}\mathbf{d}$$

where $\alpha$, $\beta$, $\gamma$ are weights you choose. The term is solved — there is no open problem here — but the elegance of the geometry and the breadth of its descendants make it worth understanding deeply. Related ideas appear in nearest-neighbor search and in how modern systems like neural network training learn from labeled examples.

The canvas below shows a 2-D word space (think of the axes as two important terms). Each colored dot is a document; the large blue circle is your current query. Click any document to toggle it between relevant (green check), non-relevant (red cross), and unrated (gray). Then click Apply Rocchio to watch the query recenter.

<!-- {{c_intro}} -->
<p class="hint">{{hint_para}}</p>
<canvas id="canvas" width="400" height="280"></canvas>
<div class="status" id="status">{{status_initial}}</div>
<div class="btns">
  <button id="applyBtn" type="button">{{btn_apply}}</button>
  <button id="resetBtn" type="button" class="ghost">{{btn_reset}}</button>
</div>
<div class="legend">
  <span class="leg-item"><span class="dot unrated"></span> {{legend_unrated}}</span>
  <span class="leg-item"><span class="dot relevant"></span> {{legend_relevant}}</span>
  <span class="leg-item"><span class="dot nonrelevant"></span> {{legend_nonrelevant}}</span>
</div>

* { box-sizing: border-box; }
body { font-family: system-ui, sans-serif; color: #222; margin: 0; }
.hint { font-size: .88rem; color: #444; margin: 0 0 .6rem; line-height: 1.45; }
canvas { display: block; border: 1px solid #cdd9e3; border-radius: 10px;
         background: #f7f9fb; cursor: pointer; max-width: 100%; }
.status { font-size: .95rem; font-weight: 600; margin: .5rem 0; min-height: 1.4em; }
.status.moved { color: #0a7d33; }
.status.idle  { color: #555; }
.btns { display: flex; gap: .5rem; flex-wrap: wrap; margin-bottom: .4rem; }
button { font: 600 14px system-ui, sans-serif; padding: .45rem .9rem;
         border: 1px solid #1d3557; background: #1d3557; color: #fff;
         border-radius: 8px; cursor: pointer; }
button.ghost { background: #fff; color: #1d3557; }
.legend { display: flex; gap: 1rem; flex-wrap: wrap; font-size: .82rem; color: #555; }
.leg-item { display: flex; align-items: center; gap: .35rem; }
.dot { width: 12px; height: 12px; border-radius: 50%; display: inline-block; }
.dot.unrated    { background: #8ab0c8; border: 1.5px solid #5a7088; }
.dot.relevant   { background: #3cb371; border: 1.5px solid #2a7a50; }
.dot.nonrelevant{ background: #e63946; border: 1.5px solid #c92f3c; }

// Code not found

Notice that after feedback the query moves toward the green cluster and away from the red ones. Hit Reset to start fresh with a new random layout. Try marking all documents on one side as relevant — the query shoots straight toward them.

Rocchio is solved — the formula above is exact — but its complexity landscape is interesting.

Computational cost. Computing the new query takes $O(|D_R| \cdot V + |D_{NR}| \cdot V)$ time, where $V$ is the vocabulary size. That is simply a sum of sparse vectors. In practice, with TF-IDF representations, both $D_R$ and $D_{NR}$ are tiny (the user rates a handful of results), so the update is nearly instantaneous.

Optimality within the model. Under the vector space model with cosine similarity, Rocchio is the optimal linear update: no linear function of the feedback documents can bring the query closer to the relevant centroid in fewer steps. The proof is a straightforward consequence of the definition of centroid.

Where assumptions break down. The algorithm trusts that relevant documents form a single convex cluster. When they do not — for example, the query "jaguar" is relevant both as a car and as an animal — a single centroid averages them into nowhere useful. This is called query drift, and it is the algorithm's principal failure mode.

Negative feedback is asymmetric. Subtracting non-relevant centroids helps only weakly in practice; the positive term dominates, and $\gamma$ is usually set much smaller than $\beta$. Research has shown that blind feedback (treating the top-$k$ returned documents as pseudo-relevant without any user clicks) works surprisingly well, which is why pseudo-relevance feedback is still widely deployed.

Rocchio sits in a long line of ideas about closing the gap between what a user types and what they mean — a gap that pattern matching algorithms cannot bridge because they have no model of meaning, only of character sequences.

The Rocchio update, or a close cousin of it, appears in a surprising range of systems:

• Classic IR systems: SMART (System for the Mechanical Analysis and Retrieval of Text, Cornell 1960s–80s) used relevance feedback as a core loop; Rocchio's formula was its refinement step.
• Web search: early Google and AltaVista experimented with explicit feedback; implicit feedback (dwell time, clicks) is a massive-scale behavioral version of the same idea.
• Recommender systems: the "more like this" feature of many platforms shifts an implicit user-preference vector exactly as Rocchio shifts a query vector — toward liked items, away from skipped ones.
• Image and multimedia retrieval: representing images as feature vectors and letting users mark relevant examples is called content-based image retrieval (CBIR); Rocchio is frequently the update rule.
• Dense retrieval and LLMs: modern bi-encoder systems (DPR, E5, OpenAI embeddings) store documents and queries as dense vectors. Averaging relevant document embeddings to update a query embedding is Rocchio in neural clothing. Hard-negative mining — deliberately including non-relevant documents during training — is the $\gamma$ term baked into the loss function.

Understanding Rocchio means understanding the geometry that underlies retrieval at every scale, from a 1970s batch system to a 2020s language model.

Rocchio's formula is about fifty years old and still alive — not because no one has tried to replace it, but because the geometric intuition underneath it is simply correct: relevant documents cluster, and moving toward their center improves recall.

The algorithm is solved in the sense that no open problem remains about the formula itself. What remains open is every hard question around it: how to learn good weights $\alpha$, $\beta$, $\gamma$; how to handle multi-topic queries without drift; how to extend the idea to dense neural embeddings. Those questions drive modern research in information retrieval and in the retrieval-augmented systems that power today's AI assistants.

The next time a recommendation surprises you with exactly what you needed, there is a decent chance that something Rocchio-shaped moved a vector in your direction.