hn.today

Qwen3.8-Flash-Next 180B running in real time on a 16G MacBook Pro

twitter.com11 points2 comments
Screenshot of Qwen3.8-Flash-Next 180B running in real time on a 16G MacBook Pro

A demonstration claims that Qwen3.8-Flash-Next, a 180-billion-parameter model, can run locally and in real time on a MacBook Pro with 16 GB of memory. It reportedly uses about 9 GB of RAM while performing agent-style tasks in OpenCode, without an internet connection. The remaining model weights - about 112 GB - are stored on the laptop’s SSD rather than loaded into memory, making a model far larger than the machine’s RAM usable on an ordinary laptop.

The key technical claim concerns how disk access is scheduled. Standard memory-mapping methods can access weights from a drive, but the model requests them only when needed, leaving the GPU waiting for data and reducing speed. The described method changes when the system knows which weights are coming, allowing them to be fetched earlier and avoiding the usual performance penalty. The SSD remains the bottleneck: each generated token still requires reading roughly 500-700 MB from disk, and the model was not designed for this kind of operation. The demonstration therefore points both to a practical way to run oversized models today and to a larger opportunity: designing future models to anticipate and prefetch weights from storage.

Read on twitter.com2 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.