Deep Seek v4.1 M5 Max at 17 tokens/s

This describes a storage- and latency-focused stack for running huge mixture-of-experts models off SSDs and the measured speedups from a forked inference engine. On an Apple M5 Max with 128 GB RAM streaming a 518 GB DeepSeek V4.1 model, the forked ds4 engine reduces unnecessary reads and splits reads across devices to cut tail latency: prompt processing on a single internal SSD rose from ~16.2 to 28.0 tokens/s (1.73×) and to 43.6 tokens/s with two external drives (2.69×); steady decode improved from ~10.6 to 14.4 and 17.4 tokens/s respectively. Time-to-first-token for a 512-token prompt fell from 31.5 s to 18.3 s on one drive and 11.7 s on three. GLM-5.3 and Kimi K3 benchmarks show similar storage-driven scaling (roughly 1.8× gains).

The performance gains come from reading only the experts actually selected by routing (about 187 of 384 per layer for a 512-token chunk), splitting each required file read across devices proportional to measured device rates, dispatching read slices by expected completion, and staging layer N+1 during GPU compute for layer N. Because each layer waits on the slowest routed expert, the critical metric is per-read latency/tail timing rather than aggregate GB/s; adding drives shortens the longest slice and reduces barriers. The release includes the modified ds4 branch, measurement tools (sampler, live dashboard, trace and ceiling tests), and instructions - reproducing exact numbers requires byte-identical model replicas on multiple devices.