A lightweight MCP server and CLI that crawls websites for AI agents by using a small decision model to score every link and passage so the agent's LLM only reads material that likely answers its question. It fetches pages (four at a time), segments them into passages, filters boilerplate, and issues one Jev scoring request per page that returns yes/no probabilities; all returned text is copied verbatim with URL, heading path and character offsets. The design emphasizes low cost, speed and safety: decisions cost cents per crawl, caching avoids repeat charges, the scorer cannot write text (reducing hallucination and prompt-injection risk), and integrations exist for MCP clients like Claude Code, Claude Desktop and Cursor. Tools include crawl, scrape and map; the project is Python, Apache-2.0 licensed and configurable via a Typesafe API key. It does not run JavaScript.
Benchmarks show substantial cost and speed advantages: an example crawl of docs.python.org reduced ~80,034 tokens to 1,873 for $0.0069 in 8.3s. On 50 WebWalkerQA tasks it answered 40% and fetched the answer page 54% for $0.025 per task versus 30%/33%/$0.194 when an LLM chose links and 8%/11%/$0.228 for breadth-first. Versus Crawl4AI on 20 tasks it fetched 10 versus 5 answer pages, median crawl time 29s versus 43s, and had fewer site-load failures; cost-per-correct-answer comparisons favor Jevcrawl ($0.06 vs $0.65 and $2.85). Results note small sample sizes and that Crawl4AI drives full browsers while Jevcrawl omits JS.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Attention mechanisms in neural networks are fundamental to the success of large language models and the AI revolution. They enable models to process language at scale, raising questions about whether attention is the key to true intelligence. (quantamagazine.org)
AI will become an invisible, pervasive environment, like air, composed of countless tiny interacting agents forming a superorganism. This ecosystem of AI will transcend individual parts, co-evolving and supporting each other in complex, adaptive ways. (kevinkelly.substack.com)
AlphaNIST is an open-source project aimed at creating a standardized dataset for evaluating and benchmarking AI models. It is designed to facilitate research and development in artificial intelligence by providing a common testing framework. (alpha-nist.github.io)
PostEra is hiring AI researchers to develop agentic systems that automate drug discovery processes using machine learning. The role involves building models to analyze biological data and support decision-making in medicinal chemistry and biology. (jobs.lever.co)
Nick Reese moved AI inference for USA Arms in-house by building three servers with AMD Radeon AI PRO R9700 cards. This shift aims to improve AI processing efficiency and reduce reliance on outsourcing, reflecting a broader trend toward in-house AI infrastructure. (nicholasreese.com)
Parsimony is an open-source benchmark that measures how much code AI models add, penalizing excess code to promote cleaner solutions. It ranks models based on net code units added, encouraging efficient coding practices. (parsimony.lasu.dev)
Today's best Hacker News stories, summarized and screenshotted, one email a day.