MoEShift is an alpha-stage, open-source runtime for running large Mixture-of-Experts language models locally on Apple Silicon Macs while limiting memory use. It stores and loads experts from SSD as needed, with scheduling, caching and Metal execution designed to balance speed against RAM use. The project says a Mac with at least 10 GB of RAM can run its tested models; the Qwen 3.8 Flash Next setup requires at least 120 GB of SSD space for the model and prepared expert files. Installation is available through Homebrew, and the local server exposes an OpenAI-compatible API for chat and coding clients. It handles one request at a time and listens only on the Mac itself.
On a Mac mini M4 Pro with 24 GB of unified memory, tests of Qwen 3.8 Flash Next (125B) found MoEShift used 6.4-7.5 GB peak process memory, versus Strata’s 14 GB. For a 6,009-token prompt, MoEShift processed prompts at 160.8 tokens per second versus 61.8 for Strata, and generated text at 9.9 versus 7.8 tokens per second; total inference took 63.3 versus 129.9 seconds. A separate Qwen3.6-35B-A3B test recorded 417.24 tokens per second on long-prompt processing, 14.66 tokens per second generation and 5.6 GB peak memory. Users can tune expert caches, batch size and thread count to trade memory for speed.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
Researchers tested 16 LLMs on all 55 TIS-100 puzzles, giving each one attempt and time to refine its code. The models found 133 new performance tradeoffs across 18 puzzles and set three outright cycle records, though human players improved on most solutions within days. (tis-100.wincon.dev)
Australia is considering a new criminal offence for AI companies that fail to take reasonable steps to prevent harm caused by their models. The government is also weighing an AI safety watchdog with power to impose significant fines, following a hack of a Medicare statistics portal involving OpenAI. (abc.net.au)
AI is increasingly becoming a gateway through which people find information. Audiences still resist having AI produce their news. (niemanlab.org)
Nvidia holds a central position in the AI industry, making disruption to the company potentially catastrophic for the wider ecosystem. The race for advanced AI is also intensifying economic and geopolitical conflict, while concentrated corporate power and infrastructure face growing political resistance. (france24.com)
LLM inference can require reading about 26 GB of model weights to generate each token from a 13-billion-parameter model. The explanation shows how model size drives memory bandwidth demands during text generation. (youtube.com)
Deletion Tuning matches full fine-tuning across three LLM post-training methods by zeroing fewer than 0.05% of model weights. Its compact masks also preserve prior-task accuracy better than full fine-tuning, LoRA, and random sparse tuning. (arxiv.org)
Today's best Hacker News stories, summarized and screenshotted, one email a day.