Running Qwen3.8-Flash-Next NVFP4 at 184 tok/s on a single RTX 6000 Pro

Mesh Applied argues that AI’s practical bottleneck is increasingly how quickly and cheaply a capable model can produce useful predictions, rather than access to the most powerful model available. It frames this as shortening the “joule-to-prediction” chain: reduce the time and waste between energy use and a task’s result. Its model strategy is to target common hardware, then combine post-training, quantization, and specialized kernels to maximize token throughput while preserving useful capabilities. The company says open models already handle many everyday tasks, including drafting, document search, tutoring, triage, and lightweight agents, so fast responses can matter more than frontier-level ability.

Two model releases illustrate that approach. Goethe Qwen3.8-27B NVFP4 is a compressed dense model aimed at document retrieval, personal agents, small coding jobs, and other tasks where speed matters. On one 32GB RTX 5090, it delivers about 255 tokens per second at one request and 686 at four concurrent requests; it can also run with 24GB of VRAM. Qwen3.8-Flash-Next NVFP4 targets agentic coding, which involves navigating and editing code across directories and drawing on the model’s learned knowledge. It is designed for a 96GB card, such as an RTX 6000 Pro, and supports long contexts and concurrent work. On an RTX 6000 Pro MaxQ, it reaches about 184 tokens per second at concurrency one and 334 at four. Mesh Applied reports coding and math performance slightly above Gemini 3.8-Flash and Claude Opus-4.6, with estimated self-hosting costs below one cent per million output tokens.