JustFit is a local inference and serving system for running the 27B Qwen3.8-Splash language-and-vision model on Apple-silicon Macs with limited unified memory. It targets long contexts and concurrent requests by compressing and sharing KV-cache pages, keeping a speculative draft model resident, and coordinating CPU, GPU, and Neural Engine work. Its components manage compressed-history attention, memory and workspace lifetimes, persistence of model state, and device scheduling. The system exposes an OpenAI-compatible API for chat, images, streaming, structured output, and controlled reasoning. The documented prebuilt installation supports macOS 27 or later on reference M4 Pro, M5 Pro, and M6 systems; the pinned model download is about 17.4 GB.
Tests report 320K aggregate retained-page capacity across two live requests using warm prefixes, while retaining the draft model; this is not a 320K context for one request or a guarantee for cold prefill. In fixed 16K tasks, JustFit v2’s median decode rate is 2.0-2.5 times v1’s, and median prefill is 9-46% faster than upstream Splash 1.3.0, while singleton decode is 2-9% slower. These are whole-system comparisons across different hardware, formats, and profiles, not isolated mechanism benchmarks. Accelerator behavior varies by chip and execution phase: for example, M6 can use the Neural Engine for language work during eligible decode periods, while vision offload is conditional and does not guarantee faster image processing. The project is Apache-licensed and provides release packages, setup instructions, measurement protocols, and a technical preprint.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Richard Sutton argues that brute-force computation alone cannot produce intelligence and that AI needs clever new methods. He says the science of AI is far from finished. (twitter.com)
Anthropic is accused of charging Russian Claude users before identity checks, then cutting off access after verification and retaining biometric data. Critics say account bans can affect linked GitHub projects and Russian citizens living abroad, and accuse the company of failing to answer public questions. (twitter.com)
LLM coding agents boost workplace productivity but can prevent developers from learning through hands-on problem-solving and weaken their technical skills. Developers should preserve their expertise by building projects manually, studying fundamentals, and practicing without AI tools in their own time. (ducktape.blot.im)
A fabricated Newtype interview led The Diplomat to retract its article about a Korean startup’s AI war test in Ukraine. The retraction followed a diplomatic dispute. (thediplomat.com)
Microsoft Research’s AI Frontiers group brought together about 50 researchers to develop agent systems and foundational AI advances. After the team joined Microsoft AI in September, its research-first approach clashed with leadership’s emphasis on scaling. (hunch.net)
Microsoft researcher John Langford has left the company. Microsoft’s AI Frontiers group has been declared finished. (businessexaminer.ca)
Today's best Hacker News stories, summarized and screenshotted, one email a day.