GPU-Accelerated Weight Teleportation

Large-scale reinforcement learning requires rollout samplers to use nearly the same model weights as trainers, but moving full checkpoints can consume substantial time and stall sampling. The proposed alternative is delta weight synchronization: the trainer XORs its old and new weights to create a bitwise change mask, which the sampler applies to its copy. Because RL updates touch very few weights, the mask is sparse and can be compressed losslessly. A disk-based version reduces transfer size but still incurs compression, disk I/O, and full checkpoint reload costs.

The design moves compression, transfer, and patching into a GPU-based pipeline. Trainer GPUs divide ownership of the checkpoint and compress their portions in parallel; rollout GPUs decompress masks and apply them directly in HBM with a Triton XOR kernel, avoiding disk and accommodating low-precision and physically reordered model layouts. An LZ4 baseline is fast but produces a 5.36 GB payload - 6.1 times larger than the CPU Zstandard result - and receiver work takes 6.041 seconds; its GPU buffers can also force eviction of the KV cache, preventing overlap with decoding. The final system combines parallel GPU compression, two-stage compression, heterogeneous CPU/GPU decompression, and Blackwell’s dedicated Decompression Engine. For a 445 GB model with 0.085% of weights changed, the production configuration reports end-to-end synchronization below four seconds and sampler pauses below 0.23 seconds, with the decompression hardware becoming the pause-time bottleneck.