Retrieval's bottleneck is moving from finding to selecting

Retrieval systems first find candidate passages, then select which ones reach the answer model. Better embeddings, query decomposition, and iterative search have improved the finding stage, shifting the main bottleneck toward selection. A benchmark that traces required “gold” passages through a production pipeline shows the change: around 18 months ago, roughly 80% of failed cases occurred because search never found the needed passage; now failures are split about evenly between search and later selection. At the passage level, 34% of the gold chunks that search finds are still withheld from the downstream retriever. In the tested setup, 50 chunks are retrieved, up to 100 are reranked, and only the top 15 are passed on.

Returning more passages can recover missing information, but it also consumes tokens, raises costs, and fills context with distracting material that may look relevant. Expanding from five to ten chunks increases the share of questions whose required evidence reaches the answer model by 14%, but precision falls, and extra context can impair accuracy. Agentic retrieval does not eliminate this tension: its decisions still depend on what it can afford to read, while rerankers may overlook passages useful only in combination. Kapa’s deep retriever can add or remove chunks according to each question’s needs; on the benchmark, it outperforms a grep-based agent while returning about one-eighth as much text and achieving six times its precision. The remaining challenge is measuring how to preserve necessary evidence with minimal noise.