The Surprising Effectiveness of Deleting Weights in LLM Post-Training

The work asks how much of a pretrained language model must change to gain new capabilities during post-training. It introduces Deletion Tuning, which learns a binary mask over the gate-projection weights in transformer blocks and sets selected weights to zero, leaving all others untouched. Despite altering fewer than 0.05% of weights, this approach matches full fine-tuning across three settings: reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation. The result challenges the assumption that effective adaptation requires updating many parameters.

Deletion Tuning also produces a much smaller trained artifact - hundreds to thousands of times smaller than a full fine-tuning checkpoint - and better preserves performance on earlier tasks. In reinforcement-learning experiments, its prior-task accuracy remains close to the pretrained model’s baseline, while full fine-tuning, LoRA, and random sparse fine-tuning lose as much as 5.4 percentage points. To explain why deleting weights can work, the researchers compare the geometry of learned changes using stable rank, spectrum drift, and overlap among principal weights. On all three measures, Deletion Tuning more closely resembles full fine-tuning than either LoRA or random sparse fine-tuning, suggesting that its sparse changes capture important features of conventional adaptation.