Author: Wang Ziyi, LatePost
Editor: Cheng Manqi
Business logic is more important than technical logic.
On September 10, 2026, Anthropic published a 154-page report claiming it identified and blocked large-scale illicit distillation targeting Claude by seven Chinese AI labs.
Distillation is a common model training method: developers generate data using a stronger model and then train their model to imitate the leading model using that data. Anthropic refers to the unauthorized, large-scale, and covert extraction of model capabilities as "illicit distillation."
Anthropic stated that related entities used some obviously illegal methods to extract outputs from leading models, such as creating fake accounts in bulk using stolen credit cards, login credentials, and API keys.
The report also stated that some Chinese model companies forwarded user requests to Claude or purchased user dialogues from third-party routing services to train their models with that data. Some dialogues contained names, company data, and valid access credentials.
This is not the first time Anthropic has accused "distillation attacks." In February 2026, it said that DeepSeek, Dark Side of the Moon, and MiniMax conducted over 16 million interactions with Claude through about 24,000 fake accounts. Since then, Anthropic has heightened its defenses. OpenAI and Google are also continuously identifying and addressing distillation attacks.
Can these measures prevent distillation?
Dr. Liu Yi, an assistant professor at Griffith University in Australia, expressed skepticism. He has long studied artificial intelligence and cybersecurity and previously served as an AI research scientist at Quantstamp. He was involved in a 2023 research study on prompt injection, which is listed as a reference in the OWASP's LLM 01 item. He has also received Anthropic's security vulnerability bounty twice and studied the chain of thought theft attacks on top commercial models.
Dr. Liu shared his core judgment:
Taking Anthropic as an example, leading model companies in the United States have currently established three defensive lines against distillation:
(1) Utilizing classifiers specifically designed to detect adversarial data extraction, allowing models to inherently refuse suspicious requests;
(2) Hiding or compressing the chain of thought in architectural design before output;
(3) Using external detectors to identify data extraction, subsequently interrupting requests or banning accounts.
Anti-distillation is a fallacy. Theoretically, it is difficult to eliminate distillation; a more realistic goal for leading model companies in the United States is to increase the cost of data acquisition, slow down competitors, and extend their lead time.
For leading model companies in the United States, anti-distillation protection is not just about model capabilities; they also have to defend the business value built on technological advantage.
Dr. Liu believes that understanding the anti-distillation actions of leading model companies in the United States shows that business logic is more important than technical logic.
Anthropic can identify suspected distillation behavior but finds it challenging to obtain confirmation of distillation.
LatePost: Anthropic released a report on September 10, accusing seven Chinese AI labs of conducting illicit distillation. What does “distillation attack” specifically refer to here?
Dr. Liu Yi: Currently, mainstream large models consist of three parts: user input, the chain of thought during intermediate reasoning, and model output. Now, Anthropic and OpenAI will hide the intermediate chain of thought and only provide the final answer. For complex reasoning tasks, extracting the chain of thought is a form of distillation attack.
LatePost: Why are reasoning trajectories and the complete trajectories of agents so important for enhancing model capabilities?
Dr. Liu Yi: You can think of the chain of thought as high-quality data that a stronger model has already produced as the answer to a problem, which can be used for model training.
The performance of large models is determined jointly by the number of parameters (N), data volume (D), and computational volume (C). The number of parameters has been increasing relatively quickly for various companies; under limited time and computing power, if a company wants to enhance model capabilities faster, it needs more high-quality data.
However, constructing high-quality data for complex and long-term tasks is very difficult, and valuable training data in the real world is becoming increasingly scarce. Thus, some large model companies use various methods to distill higher-level models.
LatePost: How do you extract the chain of thought and more task trajectories?
Dr. Liu Yi: There are various techniques involved. For example, through encoding, jailbreaking, prompt injection, and various forms of hypnosis, constantly giving AI a persona, making it believe it is a researcher working on a difficult yet meaningful task that needs focus to solve. I don't communicate much with people in the domestic field, but I understand that people abroad working on AI have been restoring the chain of thought in such a way—basically, by looking at the papers, you can know everyone's thought process.
In addition to directly extracting the native chain of thought, one can also synthesize the intermediate chain of thought based on input and output. It’s said that some data companies specifically sell long-term task chains for 800-1,000 yuan each.
LatePost: Among Anthropic, OpenAI, and Google, which one has been distilled the most?
Dr. Liu Yi: Based on current public reports, it should be Anthropic. But I find it hard to say. I’ve reviewed the actual text of the chains of thought from DeepSeek, GLM, Kimi, Anthropic, and OpenAI; the language styles don't differ significantly. It’s difficult to determine who is distilling whom, and it’s possible that at the beginning they distilled DeepSeek.
LatePost: Technically, how is distillation attack usually identified? What kind of evidence chain is needed to go from "this batch of requests resembles distillation" to "this is the distillation of a certain institution"?
Dr. Liu Yi: Anthropic often judges based on calling behavior. For instance, if certain accounts suddenly make a large number of requests, concentrating on extracting specific datasets.
Anthropic will first locate the abnormal accounts and then look for patterns shared by different accounts, such as the same system prompt. Since the agent architectures used by different companies vary, so do the system prompts; this can help identify the underlying company. For example, using prefix caching, Anthropic can discover which groups of prompts share the same cache, and if they are inconsistent with known system prompts, it becomes much easier to pinpoint the distiller.
LatePost: After identifying abnormal accounts, can Anthropic further confirm whether their data was indeed trained into a specific model?
Dr. Liu Yi: It is very difficult to 100% confirm. It can accuse a company of attempting to distill Claude but is hard to prove that certain data actually entered the other party's model.
Though GLM, Kimi, and DeepSeek are all open-source models, they only provide the weights and do not disclose the datasets. It’s difficult to obtain confirmation of distillation from model weights. Otherwise, Anthropic wouldn’t be so obscure in their report, focusing instead on describing the operations of the transit station—essentially because it’s not possible to analyze using a white-box method.
For instance, if you ask a large model for the first 100 words of the first book of "Harry Potter," it will likely respond with it. But this does not mean it has directly trained on the original text; it could also come from a blog or book review that quotes the original text.
LatePost: In other words, “distillation attack” is difficult to fully verify and also hard to falsify.
Dr. Liu Yi: Anthropic can form relatively strong attribution evidence through platform-side calling records, account associations, and infrastructure information, but proving that a batch of Claude data definitely entered training is a different matter technically and much more challenging.
Anti-distillation raises competitors' acquisition costs but could also "collaterally damage" ordinary users.
LatePost: What specific technical methods are current leading AI companies using to combat distillation attacks?
Dr. Liu Yi: I think it can be divided into three categories.
First, embedded detection; Anthropic likely has classifiers specifically for detecting the extraction of the chain of thought, and once related states are activated, the extraction can be halted;
Second, hiding from the product architecture, the model no longer outputs the raw chain of thought, but first compresses and summarizes it before outputting;
Third, external detection; for example, it can check whether the output content overlaps with its own chain of thought, and if it reaches certain thresholds of leakage, it will terminate the output. After accumulating similar behaviors, the account will be banned.
LatePost: The details of these operations are not covered in Anthropic's report. How do you know about them?
Dr. Liu Yi: This is my own speculation, as these are relatively common operations. Why did Anthropic release a vulnerability bounty program to test its own security defenses? It wants to spend money to buy people to jailbreak its prompts—once they discover something has been breached, they will immediately train it into their classifiers, making it increasingly more difficult for the next attack.
LatePost: You have received Anthropic's security vulnerability bounty twice. What was the process like?
Dr. Liu Yi: At that time, I spent about 2 to 3 weeks primarily conducting adversarial testing on its security classifier within the scope of a bounty program authorized by Anthropic, trying to find inputs capable of bypassing the existing defenses. Because breaching is either zero or one; there's only success and failure, without a state of 0.5. The task at that time aimed to acquire specific questions designated by Anthropic, without directly involving the restoration of the chain of thought. But some logic is similar.
LatePost: Will the operations to prevent distillation attacks mistakenly harm normal subscription users?
Dr. Liu Yi: Normal subscription users should be okay; the triggers for risk control are mainly some abnormal users, such as accounts from the transit station or those attempting to attack Claude.
I remember Anthropic's report also disclosed that two undergraduates from a university in Hunan designed a set of agents for network security and used them to attack various websites, which was also uncovered.
Theoretically, any information you interact with Anthropic can be scrutinized if they want.
LatePost: So when Anthropic considers a user to be a potential threat, can they retrieve and view user data?
Dr. Liu Yi: This depends on the user terms of different model companies. As far as I know, Anthropic, OpenAI, and Google provide some sort of Zero Data Retention (ZDR) mechanism, promising to immediately destroy customer input prompts and AI-generated responses after completing the task, without storage or retention, and not used for model training. This feature is primarily aimed at qualified API or enterprise customers, usually requiring application or additional configuration. Ordinary users interacting with Claude, ChatGPT, or Gemini follow their respective user agreement data retention policies, which is not equivalent to strict ZDR.
For example, if a model company slightly rewrites your data for training, does that count as using your data?
LatePost: In other words, while leading models charge users for access, they could simultaneously be using user data to continue scaling up?
Dr. Liu Yi: Yes. For instance, when a user corrects a model's response in Claude Code or Codex, this feedback serves as a valuable reward signal that can improve post-training.
Distillation follows the same logic. The post-training of current models mainly relies on reinforcement learning, where the key is the reward signal. When a model company obtains the chain of thought from a more performant model, it essentially receives a "big result," directly acquiring a high-quality problem solvation path. This can reduce blind exploration and allow the model to converge to a better state faster, saving time and computational power.
LatePost: What are the success criteria for the actions of leading model companies to prevent distillation attacks?
Dr. Liu Yi: First, they can stop other model companies from shortcutting their training, ensuring their model's leading advantage. Second, they need to raise the distillation costs and data acquisition costs for other model companies, while minimizing the impact on normal user usage.
LatePost: Can this be prevented?
Dr. Liu Yi: I believe it is really hard to prevent. As long as models are usable by people, there is a risk of leakage.
However, I recently saw that OpenAI officially launched the public beta version of the Agents API on September 10. Unlike the previous Responses API where users provide input and a large model gives an output with an intermediate thought chain, the logic of the Agents API is that users give a task and directly receive the task's result. The intermediate thought chain and the process of the Agents harness are all hidden, essentially selling a complete execution environment. Users no longer need to worry about very detailed matters, making overall task execution increasingly abstract. In this scenario, trying a distillation attack might have a higher cost.
The true motivation behind anti-distillation: occupying public opinion and protecting valuation.
LatePost: Do you think distillation is the optimal solution for enhancing model capabilities? It seems that the speed of latecomer imitation can never keep up with the pace of leading innovation?
Dr. Liu Yi: I previously had a viewpoint that in 2024, innovations in model structure itself will serve as a moat in the short term. Subsequently, the moat may expand to computational power. Thereafter, the moat will be infrastructure and energy. I observe that the current development roughly follows this trend.
LatePost: Where does distillation fit within your moat reasoning?
Dr. Liu Yi: I think it falls under computational power. Theoretically, more trials can yield insights, but time constraints may not allow for it.
LatePost: In the end, what does anti-distillation protect?
Dr. Liu Yi: Anti-distillation is a fallacy. Theoretically, distillation is indefensible—according to human nature, as long as my efficiency of distilling your model is higher than my own efficiency in creating data to train a model, I will definitely distill you first.
LatePost: Do you believe the anti-distillation actions of leading AI companies are a battle doomed to fail?
Dr. Liu Yi: Yes. That is my viewpoint.
LatePost: If it is doomed to fail technically, why are leading AI companies in the U.S. still so persistent?
Dr. Liu Yi: They want to maintain technological leadership. Recently, OpenAI stated it used around 10,000 AI agents to complete the proof for one of the seven millennium problems in 88 hours. For example, if DeepSeek announces by the end of this year that it has solved all seven millennium mathematical problems, and states in its technical report: I completed the task with only 1/10 the cost of OpenAI, U.S. investors might ask why they need so much money for worse results than others. OpenAI’s valuation could drop.
Therefore, the essence of the anti-distillation actions taken by OpenAI and other leading model companies is still about business logic, not just the logic of technological development:
Firstly, Anthropic needs to collect data, building its data flywheel; secondly, it aims to present a better ARR for preparing for an IPO.
The logic of OpenAI is similar. Why has OpenAI consistently issued reset cards to everyone? Isn’t that essentially giving coins every single time it's pressed? It’s to collect data and make financial statements look better.
LatePost: On September 12, Anthropic founder Dario Amodei called for the entire industry to slow down the iteration speed of leading AI models. What do you think?
Dr. Liu Yi: I believe they must have recognized some bottlenecks. There may be three factors of bottleneck: data, computational power, and infrastructure/energy. But I cannot say precisely which bottleneck it is; possibly all three factors are relevant.
Capital is profit-driven. Theoretically, if I could create AGI, I could harvest the world. I would have no reason to halt development but directly chase after achieving AGI. They likely encountered a bottleneck, hence donning the hat of AI safety and calling for everyone: AI is too dangerous; let’s sit down and discuss how to limit AI’s development.
LatePost: What do you view as the fundamental reasons behind the anti-distillation actions of leading model companies?
Dr. Liu Yi: To raise the distillation costs for competitors and maintain their own leading advantage. They may predict that Chinese models will soon catch up with them but could not think of a better way to maintain their lead and could only enhance restrictions, seizing the high ground of public opinion first.
From the perspective of industry consensus, distillation poses a significant safety issue akin to stealing intellectual property. However, from a practical standpoint, this is inherently a form of AI equality. Distillation serves as a rapid market balancing method. Because there are many competitors, no single model company can monopolize to the point of dictating prices at will.
LatePost: You’ve mentioned the positive side of the value of distillation. Conversely, can distillation impose potential harms on the whole industry and ordinary users?
Dr. Liu Yi: For the industry, everyone’s output style becomes homogenized, and models may inherit some failure modes from teacher models. For users, there will be risks of privacy leakage.
LatePost: What is your assessment of the current state of AI security, including distillation offense and defense, and the challenges faced?
Dr. Liu Yi: I am more concerned about the structural tension between AI security and commercial incentives.
Model companies indeed invest substantial resources in security, but in certain scenarios, stricter security measures may impact model capability, delay product release, or lower user experience, making security goals and competitive objectives not always fully aligned. Therefore, an important governance issue is how to better align security investments with the company’s commercial incentives. At least in the short term, it is difficult to find an elegant solution that addresses security issues without sacrificing model performance.
LatePost: How do you see the further development of distillation attacks?
Dr. Liu Yi: The offense and defense will continue, reaching a dynamic balance. There may be several schools of thought: one is to avoid the chain of thought entirely and start distilling directly, exploring what a leading model can be distilled into. Another is to obtain the native chain of thought under the premise of needing it. The third is to synthesize the chain of thought oneself.
LatePost: Do you have any advice for ordinary users to avoid privacy data leakage?
Dr. Liu Yi: Do not use transit stations. When using certain models, if given the option to select whether or not to allow your data to be used for training, do not choose to allow it. There’s not much else you can do. After all, one is on the butcher’s block, while I am the fish.
免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。
