Jev handed our LLM the wrong page a tenth as often as our embeddings

Sayora tested a page-selection approach that hands whole pages (with short descriptions) to an LLM receptionist and compared it to their production chunked-embedding retrieval across 168 caller questions from four small-business sites (22-232 pages). The decision model Jev picked the correct page first for 88% of the 128 answerable questions versus 70% for chunked embeddings; Jev handed the LLM a wrong page only 3% of the time versus 33% for embeddings. Jev also recognized when a site lacked an answer far more reliably (saying “not on the website” for 95% of unanswerable items versus 23% for embeddings). The test used Claude Haiku to write 30-word page descriptions and Claude Opus to generate caller-style questions and to judge whether pages answered those questions.

Jev is a non-generative decision model that scores up to ~255 discrete options (pages) plus a “none” choice; that explicit “none” option and whole-page descriptions are central to its safety and precision. Alternative pickers (embeddings on descriptions, BM25, GPT-6 Luna, and hybrids) trailed Jev, and analysis shows embeddings often prefer long, generic guides because cosine similarity favors shared words over page intent. Jev’s lookups complete quickly (about 0.28s) and cheaply, and the approach also surfaces gaps in website content that businesses may want to address.