Z.ai says GLM 5.5 built on 1T web pages

Z.ai’s upcoming GLM-5.5 family is described as relying on a large, internally built pretraining pipeline rather than mainly on public datasets. Chief scientist Jie Tang reportedly outlined the system at a Tsinghua University lecture. The planned models come in 32-billion, 122-billion, and 256-billion parameter sizes, and their preparation draws on six distinct data streams designed to convert varied material into training-ready information.

The pipeline reportedly includes one trillion web pages, crawled at roughly 10 billion per day and grouped using large-scale template clustering; two billion scientific PDFs processed with GLM-OCR, which is said to run four times faster through speculative decoding; and 50 million books, with layout correction and filtering for corrupted text. For code, 500 million repositories are transformed into executable tasks that include tests and issue descriptions. The remaining sources are 10 billion videos, intended to supply long-sequence knowledge, and 300 million hours of audio, transcribed through automatic speech recognition into trillions of tokens. The central claim is that this scale and deliberate processing across web, scholarly, literary, programming, video, and audio data form the foundation of GLM-5.5’s training.