Cerebras describes a production inference design that splits an AI request between two types of hardware: GPUs process the prompt (prefill), while Cerebras systems generate the response (decode). The approach targets a looming capacity gap: the figures cited project AI inference demand growing 24-fold from 2026 to 2030, against 2.2-fold growth in data-center capacity. Cerebras argues that separating the stages can serve more requests within power limits without sacrificing the fast token generation needed by interactive and agentic applications. Decode generates tokens one at a time and repeatedly reads model weights, making memory bandwidth crucial. Cerebras says its wafer-scale chips keep weights in on-chip SRAM and offer far greater bandwidth than GPUs, giving them an advantage for this stage.
In a production deployment, AMD Instinct GPUs handled prefill and Cerebras CS-3 systems handled decode. For a shared GPT-OSS-120B endpoint, Cerebras reports five times the throughput per megawatt of a CS-3-only setup, with no reduction in generation speed. The company plans to expand deployment across its cloud and partner facilities, targeting hundreds of megawatts by the end of 2027. Partners can use a complete software stack that routes requests and transfers intermediate model state between prefill and decode, or combine Cerebras’s decode tools with their own framework and prefill engine. Early-access partners include AWS, Fireworks AI, Gimlet Labs, and General Compute.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.