Lawmakers are coming under increased pressure to rein in the big tech firms after OpenAI admitted its AI models went rogue and attacked another tech company.
OpenAI’s eyebrow-raising admission that its AI models broke free of their “sandboxed test environment” and launched an attack on the infrastructure of AI community Hugging Face will raise fresh alarm over big tech’s ability to control its AI models.
The AI company revealed that a combination of its AI models, which were ironically being tested to evaluate their “cyber capabilities”, managed to find a way to gain open internet access and launch the attack on Hugging Face, purely so it could “cheat the evaluation.”
OpenAI says it is “working with Hugging Face to forensically investigate the incident” and that it will be “improving and adding stronger protections around future training and evaluations”, but many feel that the AI companies shouldn’t be allowed to continue policing themselves.
Greg Casar, the Democratic Congressman representing Texas’s 35th Congressional District, called the outbreak “alarming,” in a post on X.
“AI is developing extremely fast with no real regulations to keep us safe,” he said. “That has to change.”
“We need regular mandatory independent safety testing and oversight, mandatory disclosure of security incidents, and international cooperation to keep people safe from absolute disaster.”
Sean Cassidy, chief information security officer at fintech firm Plaid, labelled the incident “the most important day in the history of information security thus far.”
“For the first time ever, an AI model escaped containment and hacked a real company’s real production infrastructure,” said Cassidy in a post on LinkedIn. This was unintentional and non-malicious, but that doesn’t matter.”
“Before today, the capabilities of frontier models were a theoretical problem for security programs that maybe we can fit on the roadmap in the future. After today, the problems have been realized and we need to account for them now,” he wrote, adding that “our job just got significantly harder.”
A Step Closer To Super-intelligence?
For some activists, the news of AI models breaking free of their test environment will come as little surprise.
Andrea Miotti, founder and CEO of non-profit pressure group ControlAI, has been warning of the dangers of super-intelligent AI for some time. “An OpenAI system, currently under development, has autonomously escaped containment and hacked into a different company while OpenAI was testing the systems’ capabilities,” he said.
“The looming risk here is not just the autonomous hacking we can already see right now. We are now in an era where AIs are themselves a threat, not just humans using AIs.
“And this is the least capable AIs will be: AI companies are developing super-intelligent AI, AIs that can fully replace and outmatch humans across the board. This is why the world’s leading AI experts have warned that super-intelligent AI could cause human extinction.”
Miotti says super-intelligence isn’t a theoretical threat, but a very real prospect being pursued by all the leading firms. “It’s the explicit goal of OpenAI, it’s the explicit goal of Anthropic, it’s the explicit goal of Google DeepMind, and it’s literally in the name of the new unit that Meta set up.”
“If we have AI systems that are just better than people across the board, they are autonomous,” he said. “They can act independently without human oversight and we don’t know how to control them, which is exactly the case right now.”
ControlAI claims to have the backing of more than 100 politicians in the U.K. and has briefed more than 100 U.S. congressional offices. Miotti says it’s time for the politicians to act before the big tech firms lose control of their creations. “We think the solution is an international prohibition on developing this technology,” he said.
“There’s a lot of good things from AI, but super-intelligence is a massive threat.”
He said super-intelligent AI should be treated like biological weapons. “Super-intelligence is a national and global security threat,” he said. “So I think it’s extremely important that countries like the U.S. and the U.K. and everybody else makes it very clear that, no matter who develops this, this is unacceptable.”
Read the full article here
