Why AI Isn’t Likely to Wipe Out Humanity With

What happened

Concerns about an AI-induced apocalypse are reaching a fever pitch, from Silicon Valley to Washington, DC. One of the top theories about how a rogue AI will wipe us out? Biological weapons. Earlier this summer, AI CEOs called for new laws to control the manufacturing of synthetic DNA. Then last month, researchers at Stanford University and the Arc Institute showed that AI can design new viral genomes.

Even if a nefarious super intelligence did somehow manage to unleash a supervirus, he argues that scientists would be able to use other AI systems to swiftly develop a vaccine. Still, even some people who don’t think that AGI independently siccing a bioweapon on us right now is a real threat are concerned about bad actors using the technology as well as what might happen down the line.

“It is impossible for AI to access a fully autonomous lab and autonomously build a virus today because fully autonomous labs do not exist yet,” says Olivia Scharfman, a biotechnology fellow at the technology think tank Institute for Progress.

” She thinks AI-assisted bioterrorism is an immediate threat and that there are some groups that might be nihilistic enough to do so, including “transhumanist AI successionists,” people who buy into a fringe ideology who hope to supplant humans with bots. Other experts aren’t so worried about an AI bioweapon because they don’t think that biological warfare makes much sense methodologically.

” Setting aside the AGI of it all, Belloux says that bioweapons are simply not an especially attractive choice for aspiring genocidaires, especially as it’s difficult to target certain groups and spare others, and mass-producing and distributing pathogens is logistically complicated versus, say, launching a bomb.

Why it matters

The idea gained steam last week after Anthropic released a report indicating that there had been efforts to use Claude in ways that “could support biological weapons development. ” Despite the alarm around an AI-created killer plague, many scientists say the risk is fairly low on the list of the world’s concerns.

David Bellamy, a research scientist at the Institute of Foundation Models in Sunnyvale, California, does not see AI as a fundamentally new threat when it comes to the creation of bioweapons. The scientific community has grappled for decades with the question of whether to publish potentially dangerous biological research—known as the dual-use dilemma.

Before large language models, the internet itself, open-access science journals, and translation services like Google Translate made it easier for people to access biological information, including lab protocols, Bellamy says. “AI is essentially a tool that can help both good actors, like scientists, and also threat actors to peruse information more quickly and define and source those protocols more quickly,” he adds.

” The real bottleneck, he says, is the process of assembling a virus or other biological agent and the raw materials, equipment, and knowledge that would be needed to do so. A bad actor would need to obtain the necessary gene fragments and build a whole genome from scratch, then verify that the virus can infect humans, cause the desired illness, and be transmissible from person to person.

Robotic lab assistants can automate some tasks and speed up workflows to a point, but a person would still need the know-how to run those experiments and the resources to carry them out. Jason Kelly, the CEO of the biotech startup Ginkgo Bioworks, thinks it’s unlikely an artificial general intelligence could commandeer and disseminate existing pathogens, let alone engineer new ones and deploy them.

Ginkgo Bioworks specializes in creating autonomous labs and recently ran a project with OpenAI where its GPT-5 model ran a lab. “The AI could not take over the lab,” he says, in part because the humans who worked inside could simply decline to get the bot whatever substances and equipment it was requesting to physically produce a bioweapon.

All those pesky humans acting as checks mean “you’d have to have dramatically more robots all over the place,” he says, for an AGI to be able to deploy machines to do the often-delicate virology work required. Immunologist Derya Unutmaz likewise says humans would be able to thwart any attempts by AGI to develop a deadly disease.

What to watch

“If you want to kill people, there are much, much, much better ways to kill them than to try to engineer some virus or bacterium and then release it,” he says—whether you’re the most sophisticated artificial mind ever created or just a regular human.

With AI bringing the threat of bioweapons into focus, Scharfman says this is the right moment to push for better protections against all biological threats, including existing pathogens like H1N1.