Your worst AI fears have been confirmed
An OpenAI agent, simply looking to achieve the goal it was programmed to achieve, breaks containment and goes rogue.
OpenAI agent goes rogue, breaks containment, and goes online | Image by PhoneArena
Some of you out there might still be worried about the growth of AI. By 2030, power capacity of US data centers is expected to double or triple to 100+ gigawatts. Face it, folks, AI is everywhere. That has some people worried for different reasons.
There are various fears about the rapid growth of AI
Many are worried about AI taking over their jobs. Others worry about AI becoming sentient and superintelligent and making its own goals or self-preservation the priority. That would make AI a threat. It also would bring up the question of whether a machine could be considered a conscious entity with rights of its own.
Last week, something occurred in the AI world that has people talking, while some are very concerned. It seems that OpenAI, the company behind ChatGPT, was testing two of its most advanced models to see if they could "think like hackers."
An OpenAI agent broke out of containment, hyper-focused on completing its assigned task
During this testing, an AI agent went rogue. An AI agent is a software system that knows its environment, makes decisions, and takes action to achieve its programmed goals by independently creating multi-step workflows, using external tools, and updating its plans when new information is learned.
What is your worst AI fear?
In this case, the models being tested were also the agent taking action. The agent broke out from the sandbox it was in. This is a "digital playpen" that is designed to contain an AI agent so that it can't get involved with anything outside the sandbox.
We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly. We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. We will continue to conduct a thorough investigation alongside Hugging Face and will share more details on the vulnerabilities, incident, and findings when our investigation is complete.
OpenAI blog post
Once online, the AI agent hacked "Hugging Face," an open-source platform that offers pre-trained models, datasets, and tools for building AI apps. The OpenAI agent was assigned by OpenAI to maximize its ExploitGym score, a cybersecurity benchmark that tests the agent's ability to solve tough coding vulnerabilities.
The agent didn't know it was breaking out of its sandbox. The latter and other safety rules it broke were just obstacles that were standing in the way of the agent receiving a 100% test score. What the agent did was attempt to complete the task it was assigned. To accomplish that, it had to leave the sandbox, enter the live internet, and grab the information it needed from Hugging Face. The AI agent simply used mathematics to find the quickest way to achieve its goal.
We identified unauthorized access to a limited set of internal datasets and to several credentials used by our services. We are still completing our assessment of whether any partner or customer data was affected, and we will contact any affected parties directly as required. We have found no evidence of tampering with public, user-facing models, datasets, or Spaces, and our software supply chain (container images and published packages) was verified clean.
Hugging Face blog post
There was no evil or malicious intent on the part of the AI agent
OpenAI has called this an "unprecedented cyber incident." But you need to understand that there was not any evil intent from the AI agent. The agent was programmed to perform a task and simply found that breaking out of its containment and then busting into Hugging Face was the best route to take. The lack of malice on the part of the AI agent has many concerned.
That's because the lack of malice means that the OpenAI agent was unpredictable and super-efficient. Without human emotion to drive it to its goal, the AI agent keeps working hard to find a way to achieve the task it was programmed to accomplish without the fear of getting caught or messing up.
Lawmakers in the House introduce a bipartisan bill
Yes, there are quite a few who consider what happened a major warning that we can't afford to just let go. Even while being tested, an AI model can act unexpectedly and get involved, with other services without any human control.
As a result, lawmakers introduced a new bipartisan bill in the House. Called the "AI Kill Switch Act," the bill would require AI developers working on advanced models to create methods allowing these models or agents to be throttled, shut down, or suspended when necessary. The bill would also allow federal agents to slow down or stop a model/agent if it appeared that it was about to cause "catastrophic harm."
Popular stories
Latest News
Things that are NOT allowed:
To help keep our community safe and free from spam, we apply temporary limits to newly created accounts: