AI is making new viruses now, what could possibly go wrong?

We can talk all day about AI agents hacking companies on the internet, but this one's really scary. Scientists at Palo Alto's Arc Institute and Stanford University trained AI to recognize DNA patterns and rewrite them to create 16 new bacteria-infecting viruses, Forbes reports.

AI created 16 new bacteria-infecting viruses

The mechanism used by the researchers. | Image by Science Magazine

This may read as something taken out of a post-apocalyptic sci-fi movie, but it's absolutely real. A team of scientists used two genome language models (much like LLMs like ChatGPT and Claude, but for genes) to create 16 new strains of never-before-seen viruses.

The researchers first trained the Evo 1 and Evo 2 genome models on 15,000 gene codes of the Phi X-174 virus family. These are viruses that only infect gut bacteria, E. coli.

The Evo gene model generated more than 700,000 possible new viruses, from which the team selected 285 candidates. Scientists then manufactured these 285 viruses and inserted them into living bacteria. Only 16 of the AI creations resulted in working viruses.

The good news is that in this experiment scientists explicitly told AI to exclude anything that could infect animals or humans. But if Claude could go rogue and get access to the internet to hack people, so could these AIs accidentally create the next super plague.

What do you think of AI-generated viruses?

Could AI create the next plague?

AI could create a very dangerous virus, if prompted by bad guys. | Image by CEPI

Well, the story is the same as with all advanced tools humans have produced, such as nuclear energy, CRISPR gene editing, and now — AI. These are tools that are not inherently evil but, in the wrong pair of hands, could unleash hell on Earth.

AI-engineered viruses could help with antibiotic-resistant bacteria strains and potentially save many lives. They could also be used to kill off other pathogens or even help in food synthesis, keeping unwanted bacteria in check.

But the possible bad things that could happen outweigh the benefits. Let's say this virus-creating AI model falls into the hands of a terrorist group. We might be looking at the next supervirus that can kill millions in days. What can be done?

AI needs to be regulated

AI needs to be regulated, especially when it comes to creating pathogens. | Image by LinkedIn

Many people, including some of the AI creators themselves, have voiced concerns about AI regulations. But hallucinating about your next Bolognese recipe and designing a deadly strain of the flu are two very different things.

You could say, ‘Hey, genomic language model, make me an influenza genome that is modified to be more transmissible or to be more lethal

- Dr. Moritz Hanke in an interview for the New York Times

The new paper, published in Science, already received some criticism from biosecurity leaders Dr. Thomas Inglesby and Dr. Moritz Hanke of the Johns Hopkins Center for Health Security. They warned that the new Stanford study shows that AI can now design bioweapons.

According to Inglesby and Hanke, there should be strict regulations on AI bio research, and it should be outright illegal to use AI to create pathogens that affect humans, animals, or agricultural crops.

Are we really in trouble?

The Doomsday Clock, a symbolic representation of how close humanity is to a terminal catastrophy. | Image by Britannica

I'm currently recovering from a long viral tonsillitis, and this news got me properly scared. Humanity already has a moratorium on bio weapons that prohibits the development, production, and stockpiling of biological and toxin weapons.

The problem is that AI is everywhere now and very easy to use even by people who don't have any expertise in biology and genetics. In the past, the bad guys had to hijack whole teams of scientists, build expensive laboratories, and spend many months and money before something dangerous came out.

Nowadays, we're a couple of prompts away from the apocalypse. Maybe we should bring back the doomsday clock we used back in the Cold War to illustrate the nuclear threat. Only this time we should use it for AI.