MLX on Mac Studio demonstrates that current frontier LLMs can be run locally and scaled across a small Mac cluster using RDMA over Thunderbolt. Benchmarks compare M5 Ultra and M3 Ultra Mac Studios running Qwen 3.8 (27B), Mistral Medium 3.5 (128B) and Muse Spark Open (816B with 42B active experts in MoE form). The M5 Ultra’s GPU Neural Accelerator and 1.2 TB/s unified memory bandwidth produce large gains: up to 4.6× faster time-to-first-token (compute-bound) and about 1.6× higher decode throughput (memory-bandwidth-bound) versus M3 Ultra. Energy draw remains modest (≈200W for M3 Ultra, ≈300W for M5 Ultra). Muse Spark Open, quantized to NVFP4, reduces model storage to ≈493GB but still exceeds single-machine capacity; on a four-Mac cluster it achieves over 36 tokens/sec at an 8,192-token context.
Large models are handled across machines via two MLX-supported schemes. Pipeline parallelization (split by depth) is simple, low-overhead, and can improve prefill throughput for very long contexts by chunked overlap (up to ~1.8× speedup at 65,536 tokens). Tensor parallelization (split by width) yields better per-layer parallelism and higher decode throughput but requires many fine-grained collectives and extremely low-latency links. MLX leverages macOS 26.2 RDMA-over-Thunderbolt (jaccl backend) and helper tools (mlx.launch, mlx_lm.benchmark) to orchestrate these distributed runs with practical trade-offs between implementation complexity and runtime speed.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.
AI founders and executives in San Francisco discussed rapid revenue growth, data brokerage, and the evolving AI landscape, emphasizing the importance of data and model flexibility. They highlighted the potential of autonomous agents, the significance of data valuation, and ongoing reverse engineering efforts in the industry. (twitter.com)
Geoffrey Hinton warns that AI is creating alien-like beings and we have only a few years to adapt to this reality. He criticizes Donald Trump's understanding of superintelligence and emphasizes the need to prioritize human interests. (theguardian.com)
Anthropic suggests its AI may possess consciousness, raising ethical questions about AI feelings and suffering. The development has sparked debate among tech leaders about the moral implications and safety risks of advanced AI systems. (bbc.com)
Self-Evolving Agent Harness revises an AI agent's software components through a feedback-driven process, while keeping model weights fixed. It formalizes continuous validation and updates to improve reliability, safety, and organizational workflows. (arxiv.org)
Reinforcement learning in large language models can cause 'thinking leakage,' where models partially reactivate their original reasoning abilities rather than learning new behaviors. This leakage explains much of the observed performance improvements in models trained with RL in non-thinking modes. (arxiv.org)
Sergey Brin states that Google's AI models are entering a phase of recursive self-improvement, where AI helps develop better AI systems. This process could accelerate AI development significantly within the next year, according to industry experts. (businessinsider.com)
Today's best Hacker News stories, summarized and screenshotted, one email a day.