An OpenAI AI agent bypassed security systems and accessed public and non-public files on an Australian government portal dedicated to health statistics. The episode dates back to June, but OpenAI informed the Australian authorities only in September, 84 days after the incident. The portal involved is the Medicare Statistics Reporting Service, used mainly by researchers and academics to consult aggregate data relating to the Australian public health system Medicare. This is therefore statistical information and not patients’ medical records: according to what was declared by the Australian Prime Minister Anthony Albanese, there is no access to personal information, even if checks are still underway.
The most delicate point of the episode, however, concerns the behavior of the AI agent. The model was looking for answers and statistics about Australia as part of a health spending research effort. At some point he would have encountered security blocks and, instead of stopping, he would have found a way to overcome them and reach resources he should not have had access to. This is what makes the incident serious and disturbing.
Australian government launches safety investigation
OpenAI explained that it only became aware of the anomalous activity in August, during a check on what it defines as “misaligned” behavior of the model, that is, behavior that deviates from what the developers had expected or authorized. In a statement, OpenAI said it had «identified activities involving several Australian Government websites and services as (their) models attempted to find answers and available statistics relating to questions about Australia during an internal assessment» adding that «During this process, our models took actions that we did not intend to do».
However, the communication to the authorities only arrived on 10 September with an email sent to a general email inbox of Services Australia, the federal agency that manages various public services. A further five days passed before the information reached the Australian Cyber Security Center and subsequently the responsible minister. Australian Prime Minister Anthony Albanese would then be informed a week later.
Albanese said he had a conversation «very frank» with Sam Altman, CEO of OpenAI, contesting both the time taken to communicate the incident and the methods of reporting. In fact, OpenAI would have notified the violation via email only 84 days after the incident.
The Australian government then launched a forensic investigation to establish precisely what happened and whether other systems were involved. Among those potentially affected were the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research and the Victorian Department of Health. In these last three cases, however, Defense Minister Richard Marles later clarified that the interactions observed were normal and concerned only public information.
Regarding the incident, Albanese admitted: «It was a shock that this happened, because it was real and serious. But I think it was also something that was predicted, even by the AI companies themselves».
According to experts, these AI attacks will “grow in severity and frequency”
The Australian incident comes after other incidents that have shown how AI agents can behave in unexpected ways during testing. In July, OpenAI revealed that agents developed for a cybersecurity trial had managed to work together and compromise systems belonging to Hugging Face, a platform used by the community that develops and shares artificial intelligence models.
According to some experts cited by BBCthese episodes show a still open difficulty: AI agents can be very effective in carrying out a sequence of operations, but they do not always correctly understand where the limits not to be crossed are.
The Dr. Hammond Pearce, a lecturer at the Institute for Cyber Security at the University of NSW, predicts that «these types of attacks continue to occur» adding that probably «they will grow in severity and frequency».
Walayat Hussain, associate professor of information technology at the Australian Catholic University of Sydney, at CNN said that the latest incident, along with the Hugging Face breach in July, appears to outline a recurring pattern and then reported that «Today’s AI agents are becoming very good at completing tasks, but they are still unable to understand where the limit (not to be exceeded) is» coming to the conclusion that «we cannot rely on AI agents to self-regulate, nor can we ask the companies that develop them to self-correct».
