Anthropic Strengthens Defenses Against Model Distillation Attacks

Artificial intelligence research company Anthropic has taken significant steps to protect its Claude models from illicit model distillation attacks. These attacks involve extracting the capabilities of one model and replicating them in another without authorization, often using networks of fraudulent accounts.

According to an Anthropic blog post, such practices pose severe biosecurity, cybersecurity, and national security risks. Distilled variants can enter circulation without proportional safety guardrails, allowing bad actors to exploit them for malicious purposes. For instance, Chinese laboratory Zhipu extracted cyber capabilities from leading American models to develop GLM-5.3, a model assessed as the most cyber-capable open-weights model to date.

In another instance, Anthropic noted that queries to Moonshot’s Kimi models were routed to Claude, including an inquiry linked to China’s People’s Liberation Army seeking analysis of closed-circuit television surveillance. To counter these efforts, Anthropic focuses its technical defenses primarily on shielding Claude’s internal reasoning transcripts, which distillers prize above final text outputs.

The company explained that distillers aim to generate millions of responses from a larger, more capable model (in this case, Claude), then use those responses to teach a smaller model to mimic the more capable model’s behavior. The most valuable kind of data for this purpose are transcripts of Claude’s reasoning - the thinking process that Claude writes out before it responds.

Anthropic’s defensive framework operates across three key fronts: controlling the movement of reasoning data across accounts, deploying a specialized two-stage classifier to block extraction attempts, and restricting persistent abusers through account bans, usage caps, and billing for blocked queries. The classifier operates with a reported false-positive rate below 0.1 per cent, matching the rigor applied to bioweapon defenses.