Strengthening our defenses against distillation attacks

This explains how Anthropic responded to covert, industrial-scale distillation campaigns that extract a model’s internal reasoning to train competing models. Distillers use large networks of fraudulent accounts and proxy chains to harvest millions of responses, focusing on the model’s step-by-step “thinking” because it teaches smaller models how to reproduce capabilities. Anthropic highlights concrete harms: distilled models with weaker safeguards can be used for dangerous malware or biological assistance, and foreign labs have integrated distilled capabilities into military and surveillance systems (examples include Zhipu’s GLM-5.3 and a reported case where Moonshot’s model forwarded a PLA‑linked request to Claude). Attackers escalated from passive scraping to active prompt-engineering and large-scale evasion (tens of thousands of fake accounts), showing that simple mitigations were insufficient.

Anthropic implemented three technical defenses: preserved thinking (preventing new API accounts from editing prior protected context and restricting which models or organizations can read reasoning), a two-stage constitutional-style classifier trained and red‑teamed on extraction attempts (false positive rate <0.1%), and operational controls that throttle, ban, and charge for blocked abusive requests. These changes cut the reasoning tokens available to distillers by roughly 10-100× (an estimated 90-99% reduction), led to tens of thousands of account bans often within hours, and required integration and revenue trade‑offs. Anthropic frames distillation as an ongoing cat‑and‑mouse challenge that will require continued iteration.