Open-weight AI models are catching up to the frontier.

What happened

6 Sol and Anthropic’s Mythos, a Chinese open-weight model has narrowed the gap with the industry’s leaders. 7 on cyber and bio capabilities, according to a new report from AI safety nonprofit SaferAI. But the divide between frontier capabilities and safety practices is growing. 2 refused none of the offensive cyber or dual-use biology tasks it was given. ” (CyberGym is a benchmark that evaluates cybersecurity capabilities.

Anthropic’s Opus 5, for example, can search for vulnerabilities in uncompiled source code, but not compiled software, per the model’s system card. The reasoning is that this makes it harder to use Opus 5 for offensive purposes. Others include rigorous pre-deployment safety evaluations, publishing risk assessments, and withholding model weights if a system is perceived as too dangerous.

ai didn’t publish a safety framework, pre-deployment testing commitments, or risk assessment for the model. ai whether it conducted internal or third-party frontier safety evaluations before release, but did not receive a response. Chinese leaders have increasingly acknowledged the risks of advanced AI.

At the World AI Conference last month, Chinese President Xi Jinping emphasized the importance of open-weight models, while also stressing the necessity of ensuring AI remains a tool under strict human control.

Why it matters

With open-weight models rapidly approaching the capabilities of the world’s leading AI systems, the debate is moving from whether they can compete to how society manages risks once they are released. “The frontier of capability is not the frontier of risk, and so we do have to take into account the state of the mitigations as well to assess the risk properly,” Henry Papadatos, executive director of SaferAI, told TechCrunch.

ai could apply safety measures to its hosted API, those protections become unenforceable once someone runs the weights on their own hardware, where they can remove or modify any safeguards, fine-tune the models, or change system prompts. Frontier developers like OpenAI and Anthropic tend to rely on safeguards like classifiers, refusal training, and API-level controls to limit dangerous cyber and biological assistance.

Those measures are far from foolproof: jailbreaks routinely bypass protections on deployed models. 1 Pro. According to the report, jailbreaks succeed when attackers combine multiple manipulation techniques — including roleplaying, authority impersonation, fake conversation history, and follow-up prompts — to amplify weak points in a model’s defenses.

But the safeguards in place for closed models don’t work at all on open-weight models, which are designed to run on any infrastructure with any set of safeguards — or lack thereof. “The objective should clearly be that the good capabilities — the safe ones — are accessible to anyone, and then we try to remove the bad ones, even in an open source fashion,” Papadatos said.

One technique Papadatos noted could help is called “pre-training data filtering,” which is when an AI company removes offensive cybersecurity information from their training data and then trains the model on the curated dataset. Some research suggests this can reduce hazardous biological knowledge without harming overall model performance. However, for cybersecurity, data filtering is much less practical.

It’s difficult to train a general model that excels at coding but isn’t also a good hacker. Because coding has become AI’s biggest moneymaker, developers face pressure to keep improving those capabilities even as they search for ways to limit misuse. Because of that, frontier developers have increasingly relied on other mitigations instead. One approach has been to selectively restrict the kinds of cybersecurity assistance models will provide.

What to watch

Graham Webster, who studies Chinese AI policy at the Stanford Cyber Policy Center, told TechCrunch that China has robust regulations governing AI, but those rules have historically focused on politically sensitive content, misinformation, and social stability rather than catastrophic AI risks like offensive cyber capabilities and biological misuse. S.

AI thinkers are, in general, more concerned with this existential catastrophic [idea] than the Chinese community,” Webster said, adding that many Chinese policy researchers believe that if there’s truly going to be a novel frontier risk, American companies will likely encounter it first.