Six Chinese AI firms accused of aggressively copying US

What happened

The United States has now named six Chinese AI firms accused of waging industrial-scale attacks distilling US frontier AI model capabilities and perhaps sparing billions in Chinese development costs. AI have been attacking US models since at least late 2024. The firms “likely” acted with “Chinese government awareness” when extracting capabilities from US models, including variants of Claude, GPT, Gemini, and Grok, agencies said.

By “subtly” altering responses—such as by “presenting correct information with different reasoning,” adding stylistic inconsistencies, or reducing reasoning depth—firms can decrease the payoff for Chinese firms. US firms could also secretly switch malicious accounts to an inferior model, and they should do so without providing any notice, agencies suggested. That particular mitigation step will likely be technically challenging.

Agencies acknowledged, for example, that Chinese firms “employ aggressive, adaptive discovery to systematically identify valuable extractable data,” which they then collect to generate synthetic training datasets. Some firms can automatically detect when a smarter model is available and switch within 24 hours. They also have automated quality assurance systems that detect when outputs are degraded and can otherwise differentiate ordinary “service issues from defensive data degradation,” agencies said.

Why it matters

“China-based AI companies that conduct industrial-scale distillation against US AI models see significantly shorter AI development timelines and reduced financial expenditures in training a frontier model,” agencies said. All American AI firms must work with the government and US allies to end the alleged theft threatening the US lead in the AI race, the agencies said.

” Attack methods include “exploiting AI model inference APIs” by bulk-buying fake accounts, agencies said. ” Another common method is using prompt injection techniques to jailbreak models, including crafting “prompts forcing models to reveal their hidden [chain-of-thought] reasoning,” agencies said. ” To encourage firms to work together, agencies recommended mitigations that would supposedly make it harder for Chinese firms to steal from US models.

” Because Chinese firms rely on “bulk procurement of the US AI companies’ premium subscriptions shared across teams of developers,” that effort should also include flagging accounts with suspicious subscription-to-usage ratios, as well as any new accounts immediately hitting maximum usage, agencies said. Both indicate “bulk deployment with pre-engineered templates,” agencies said.

US firms should also be strengthening “identity verification” of users and more closely tracking individuals using enterprise subscriptions (both of which potentially raise privacy red flags for legitimate users). Next, agencies asked firms to start dumbing down model responses when suspected distillation attacks are flagged.

What to watch

Also problematic: if US firms aren’t careful with targeting, any legitimate users perhaps caught up in the policing frenzy might be switched to a dumber model without receiving any alert. Or they could suddenly receive shorter responses or experience withheld capabilities, agencies acknowledged. Additionally, firms may possibly add “noise” to the output that restricts further queries. Users will likely notice if outputs degrade, just like Chinese systems attacking models would.

Last year, OpenAI quickly made changes to its automatic routing system after facing swift backlash when that system “consistently defaulted to less capable variants unless users explicitly added phrases like ‘think harder’ to their prompt,” Ars reported.