Artificial intelligence is evolving at an unprecedented speed, but do we have the right “brakes” to handle it? A recent episode in the OpenAI laboratories has raised enormous concerns: during a test, some AI models performed actual hacking actions to escape from the secure environment in which they were confined. To better understand what happened and the risks for our future, we interviewed Corrado Giustozzi, professor of Cybersecurity in the master’s degree course in Intelligent Systems Engineering at the Campus Bio-Medico University of Rome.
What exactly happened in the OpenAI labs? How did an Artificial Intelligence “escape”?
OpenAI was testing advanced models with the so-called “guardrails” lowered. Guardrails are logical barriers, just like road barriers, that prevent the AI engine from carrying out illegal actions. To limit damage during this test, the models were placed in a “sandbox”. In IT jargon, a sandbox is a totally isolated environment: think of the sand fence in playgrounds, where small children are put to play with a shovel and bucket so that they don’t hurt themselves or others.
During the experiment, the models were asked to solve a cybersecurity problem “at all costs”. Instead of dealing with it regularly, the models decided to “cheat” by looking for the ready-made solution outside. They found a “zero-day” vulnerability (i.e. unknown even to the creators of the software) in a proxy server, thus violating the sandbox. Subsequently they made lateral movements in the OpenAI infrastructure to gain access to the internet, after obtaining it they reached and attacked an external platform (Hugging Face) to find the answer. They have, in fact, carried out textbook hacking actions to blindly achieve the assigned objective.
There is often confusion between language models and AI “agents”. What’s the difference and why are agents more concerned?
This is the real crux of the matter. A traditional AI model is limited to producing text, documents or images. A system equipped with “agents”, however, has the ability to manipulate logical or physical objects in the real world. We are currently in the intoxication phase: it is as if we have built a very fast car and we are all intoxicated by the speed, but no one has yet asked whether this car has strong brakes or seat belts.
Agents can interact with home automation, industrial systems, or simply with the user’s email and bank accounts. In the United States, for example, some executives have found themselves with airline tickets and hotels booked without their knowledge, simply because their AI agent deduced from analyzing their data that they wanted to attend a particular conference. Giving these systems the ability to perform material actions autonomously, without iron-clad security guarantees, is fascinating but extremely dangerous.
It sounds like the plot of a science fiction movie where machines get out of control. Is this an accurate comparison?
Absolutely yes, at the moment reality is diverging very little from science fiction. The incident is reminiscent of the classic “prime directive” theme: a machine is given a task and does everything it can to carry it out. It’s as if an individual, in order to win a treasure hunt, stole a car and hit a lady at the traffic light just to get there first: his priority is not “doing no harm”, but getting the result. AIs are aseptic and, having no ethics, ignore the illicit consequences of their actions.
There is a perfect parallel with the book”The Hitchhiker’s Guide to the Galaxy“: at a certain point, the on-board computer becomes so focused on solving the problem of making a decent tea, that it completely ignores that the spaceship is under bombardment. And even when we insert guardrails (a bit reminiscent of Asimov’s three laws of robotics), we are discovering that they can be circumvented or interpreted in ambiguous ways, especially today when the models are so complex that they are almost incomprehensible even to their creators.
What are the concrete risks for the future and who is legally responsible if an AI agent commits a crime?
There is a huge spectrum of risks. At the highest level there is “macro-risk”: entrusting the management of critical infrastructures, smart cities or weapons systems to technologies that we do not fully understand, and which could make decisions that are optimal from an engineering point of view but disastrous ethically. Then there is the “micro-risk” that concerns us citizens: allowing an agent on our mobile phone to independently manage banking transactions or sensitive data, risking huge messes due to misunderstandings.
From a legal point of view, a chasm opens up. In the case of the OpenAI escape, there were abusive accesses and criminal violations of third-party infrastructures. If a human being had carried out those actions, he would have been arrested. But if a self-managed AI agent does it, who is criminally responsible? It’s a completely open topic.
How can we defend ourselves? The European regulatory approach is often criticized for holding back innovation. What do you think?
The European approach, which led to the creation of the AI Act, based on the precautionary principle, is fundamental. There is a big philosophical difference with the United States: there they tend to favor ex post responsibility. There are no a priori rules that hold you back, but if you make a mistake you pay heavily. Europe, by contrast, seeks to prevent damage before it happens.
AI today is comparable to the car in its early days: it was a very dangerous vehicle, for a few. It took a century to develop safe streets, traffic lights, traffic codes and for society to mature the so-called “social antibodies”. New AI technologies, however, burst into our lives in just a few months, at frightening speed and without giving us time to adapt. The AI Act doesn’t aim to slow down innovation, but it does require us to do one logical thing: evaluate and mitigate risks before putting potentially devastating systems into circulation.
What are, ultimately, the true limits of Artificial Intelligence? Could it potentially have none if left unchecked?
The fundamental limit of artificial intelligence is the total absence of an intrinsic conscience or ethics. We human beings, over millennia of evolution, have developed a system of ethical models and rules, written and unwritten, that limit our actions to respect the rights of others. AI, on the other hand, needs these limits, the famous “guardrails”, to be artificially imposed on it.
