Janas: Run LLMs larger than RAM by streaming experts from SSD

Janas is a small, dependency-free C toolchain whose JANAS-LLM component runs very large mixture-of-experts (MoE) language models on ordinary machines by streaming expert weights from NVMe SSD into an in-memory cache: weights stay on disk, the experts needed for each token are read on demand, and frequently used experts are retained in RAM. It supports CPU-only or optional GPU paths, accepts models converted from Hugging Face, and is optimized around measured hardware characteristics (AVX2/AVX-VNNI, fast NVMe). Benchmarks on an Intel Core Ultra 9 laptop with 32 GB RAM and a 1 TB NVMe (about 5.7 GB/s) show big gains versus llama.cpp: for example, a 48 GB Qwen3-Next-80B model ran on that 32 GB machine with Janas at ~62 tok/s reading and ~37 tok/s writing versus ~16 and ~2.8 tok/s for llama.cpp; across many models Janas reported substantially higher read/write throughput.

The project is experimental and intentionally aggressive: it uses O_DIRECT, fills free memory with an expert cache, and can saturate the disk and push a desktop into swap or, in one case, trigger a GPU driver reset. It writes only modest local cache data and converted models where directed. Tested primarily on Debian 13 and Windows 11 on the single laptop above, Janas includes a benchmarking tool that auto-tunes thread and I/O choices for each machine; users should expect platform-specific variability and report results from janas-bench when trying other hardware.