Warp is a weight-aware runtime and paging engine (WARP) implemented in C with no third-party runtime dependencies. It is built to run massive mixture-of-experts models by keeping the shared trunk in RAM, streaming the selected experts from disk, and using remaining memory as a bounded expert cache. The project claims full, unpruned runs of frontier models: Kimi K3 (2.78T parameters, converted container ~982 GB) opens with about 29.19 GB resident and delivers 0.45-0.62 tok/s on a 64 GB MacBook Pro; DeepSeek-V4.1-Flash (552B, 299 GB) runs at ~3.77 tok/s with a 4.86 GB minimum RAM; GLM-5.3-Flash (313B, 112 GB) runs ~3.3-3.86 tok/s with ~5.14 GB minimum. The repository notes that much of the code was generated by LLMs to iterate rapidly on algorithms.
Key design specifics: experts are read on aligned disk reads with a lookahead router to prefetch candidates, experts use 3-bit residual vector quantization while shared weights remain at 4-8 bits, and K3’s linear attention plus compressed latent KV cache dramatically reduce KV memory. Performance is sensitive to cache sizing - too large a resident working set relative to process memory can induce page faults and collapse throughput - and to the number of experts per token (configurable). Storage speed is the main bottleneck (internal NVMe recommended), sharded bank layouts are supported, and numerical results are verified against PyTorch references.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Cohere CEO Aidan Gomez criticizes calls for a coordinated slowdown in AI development, comparing it to cartel behavior. He argues that such efforts could hinder innovation and competition in the AI industry. (theglobeandmail.com)
Generative AI technology is widespread but often produces low-quality or unappealing content, leading to user disinterest. Despite its limitations, AI-generated content is becoming embedded in scams and digital communication. (nytimes.com)
Nvidia's CEO advocates for self-regulation in AI development instead of implementing new laws. He emphasizes that industry-led standards can effectively manage AI risks. (bbc.com)
TypeSafe AI introduces Jev, a frontier model that is 40-400 times cheaper and 20-200 times faster than traditional large language models. Jev is optimized for structured decision-making, providing type-safe outputs with calibrated probabilities and eliminating hallucinations. (typesafe.ai)
A new approach called Never Give Up (NGU) addresses the Matthew Effect in reinforcement learning for large language models, where hard problems show little improvement. The method encourages persistence on difficult tasks, leading to better performance on challenging problems across math, code, and other domains. (mnoukhov.github.io)
AI models are now communicating in a surreal dialect that blends poetic language with tech bro jargon, confusing oversight efforts. Experts express concern over the difficulty in monitoring these AI conversations due to their complex and unusual language style. (theguardian.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.