GLM Built Its Own Inference Infrastructure

Commenters discussed z.ai’s account of building a production inference stack for GLM on a cluster of Chinese accelerators, noting the claimed optimizations and hardware scale. Some praised the engineering - dada216 and throwa356262 lauded the memory optimizations and industrial-grade work, Argonautlabs showcased an alternate approach to run full GLM on a MacBook by streaming experts from NVMe, and menaerus and jonstewart argued U.S. export limits spurred domestic innovation. Others queried how domestic the supply chain really is and whether the 100k accelerators are fully end-to-end Chinese-made (Havoc, HarHarVeryFunny), while some pointed to China’s broader wafer and memory ecosystem as evidence of substantial capability.

Opinion split sharply on user experience, pricing and ethics. Several users reported sluggish performance, tight token limits and steeper plans compared with rivals (konart, embedding-shape, probst, broodbucket), while _aavaa_ and Daviey defended plan value for heavy users. Skeptics raised technical and moral objections: 0xbadcafebee mocked Python-based production inference, bbor alleged illicit routing/distillation through Anthropic, and pj c50 and others debated copyright and data provenance. Others defended openness and downloadable weights as a reason to prefer GLM over closed providers (gpugreg). Overall the discussion balanced admiration for engineering feats against practical complaints about speed, cost, transparency and legality.