End‑to‑End 4‑Bit Training

The piece explains how to train large language models with low-precision arithmetic while limiting the rounding errors that can damage learning. On NVIDIA GB300 GPUs, NVFP4 uses about 4.5 bits per value and can run matrix multiplication up to six times faster than BF16, but its coarser representation makes full 4-bit training too inaccurate. The approach instead applies NVFP4 to the forward pass in computationally expensive Mixture of Experts layers, where quantization has a comparatively small effect on accuracy. Backward passes use MXFP8, with re-quantization and stochastic rounding to reduce bias in gradients. The team also quantizes GPU communications to address the bottleneck exposed when computation became faster.

NVFP4 represents values with four-bit numbers and scales each block of 16 using a higher-precision factor. This preserves a useful range, but the scaling groups change when tensors are transposed for backward computation. Using identical scales across larger 16-by-16 tiles can align forward and backward representations, yet makes quantization vulnerable to outliers that force many small values to zero. These tradeoffs help explain why the recipe assigns different formats to different training stages rather than using four bits everywhere. Inkling 2 Small was trained with this numerical setup from pretraining through reinforcement-learning rollouts, avoiding end-of-training calibration. The reported recipe delivers an 8% pretraining speedup and a 20% reinforcement-learning speedup over full MXFP8 training.