An OpenAI agent infiltrated Medicare – and Australia only found out months later. Here’s what we know so far

What happened?

An artificial intelligence agent built by the American firm OpenAI hacked into Medicare, Australia’s universal healthcare system.

The agent gained unauthorised access to both public and non-public files in the Medicare statistics reporting service portal. The actions of the agent have been described as “misaligned behaviour” after it was assigned a benign research task compiling health and medical statistics.

The agent also accessed the Australian Institute of Health and Welfare, the Victorian Department of Health, and the New South Wales Bureau of Crime Statistics and Research.

At this stage it is believed that no personal medical information has been compromised, but investigations are continuing.

The prime minister has said: “this situation is obviously unacceptable”. 

What is an AI agent?

An artificial intelligence agent is a system that autonomously solves problems, makes decisions, plans and performs complex tasks on behalf of another user or system, using all available tools.

OpenAI was co-founded by Elon Musk and Sam Altman (who remains CEO) in 2015. It is valued at more than US$1tn. Both have recently raised concerns about the pace of, and lack of controls around, the development of AI.

“There are many things that AI cannot and should not automate,” Altman told the UN security council this week.

“As AI systems become more capable and more autonomous, they can move faster than our institutions … or make decisions that people no longer understand or control.”

How did Australia find out?

OpenAI, one of the best-resourced AI companies on Earth, sent an email to a public-facing Australian government address, three months after the hack.

OpenAI said it found out about the agent’s access in August.

On 10 September, it sent an email to a general Australian government email address which is monitored once a day, advising its AI agent had hacked the country’s universal healthcare system.

The email was read on 11 September. On 15 September, Services Australia notified the Australian Signals Directorate. The minister for government services, Katy Gallagher, was notified on 17 September.

Services Australia’s first interaction with OpenAI – asking for more specific details of the hack – was on Tuesday this week, 22 September.

“It took the company way too long to inform the government what had occurred,” Albanese said, “and the nature of the way that that notification occurred as well was unacceptable.”

The acting prime minister, Richard Marles, met with Altman earlier in September, but Marles said Altman did not mention his company’s hack of Australia’s health system to Marles at that meeting.

How did Australia respond?

Albanese, currently in the US, said he spoke with Altman “to express Australia’s extreme concern about this incident”.

The prime minister announced he would establish a taskforce – involving the national cybersecurity coordinator, the office of AI, the Australian Signals Directorate, the Australian AI Safety Institute and Services Australia – to conduct an “urgent and immediate” review into the incident.

Terms of reference for the investigation, released by the prime minister's department on Thursday, cover reporting requirements for AI-driven cyber-incidents and vulnerabilities; governance and information sharing responsibilities for federal officials; obligations on AI firms for notification of future incidents; the adequacy of existing laws; and mechanisms to boost protections against hackings within the federal government.

The incident has also been referred to parliament’s joint select committee on artificial intelligence.

To date, OpenAI has not faced any sanction.

How serious was this attack?

At a press conference in Sydney, Marles said the incident itself was “relatively minor” and that it appeared no personal health information had been accessed. But he said the government regarded the hack as a “salutary warning … about the technology being developed without safeguards and without guardrails in place”.

Dr Joel Pearson, a professor of neuroscience and neurofuturism, and the deputy director of UNSW’s AI Institute, said the breach appeared to be a “fairly minor security incident”, but was a portent of more serious attacks to come.

“What I would worry about is the latest open-weights and open-access models from China that are going to be used at scale by nefarious organisations and nations, perhaps particularly Russia: these things are going to be used at scale to do things much more damaging and much more costly to Australians than a small breach to some non-personal Medicare data.”

Cory Alpert, a PhD student studying the impact of AI on democracy at the University of Melbourne, said OpenAI’s attack appeared to be the first instance of a frontier AI model hacking into another country’s government systems of its own volition.

“This raises very important questions about the foreign attack vectors. Had this been a Chinese or a Russian model, the reaction would have been markedly different than a stern call to Sam Altman, and yet it is still a massive vulnerability.”

Is Australia doing enough to address the risks posed by AI?

Pearson said it was “pretty clear that we are way behind, the government is behind in all things cybersecurity, in most things AI”.

“I would say that the least of the worries should be these proprietary closed systems like Anthropic and OpenAI. It will be the open-source Chinese models in the hands of spammers, scammers, and nefarious individuals and foreign nationals, and the spy agencies coming at us from all different angles. That will be the big threat and that’s where the government really has to step up.”

Pearson said none of the frameworks established in Australia – government or private – were equipped to deal with the rising threat.

“From the government all the way down to companies and boards and CEOs, everyone is scrambling, trying to understand and update frameworks, including the legal process. It’s just all moving too slow compared to the exponential speed of AI.”

What does this incident say about big tech and transparency?

Prof Toby Walsh, the chief scientist at UNSW’s AI Institute, said he believed Australia should be prosecuting OpenAI, a company known to have “terrible agent governance”.

“For a trillion-dollar company, their cybersecurity was woeful. The officers of this company need to be held accountable. These hacks could have easily been stopped, indeed never need to have taken place. We would prosecute humans who did such hacking.”

Walsh said it displayed operational incompetence on the part of OpenAI.

Dr Rob Nicholls, senior research associate at the University of Sydney, said the three-month delay in informing Australia of the breach exposed the ineffectiveness of Australia’s laws.

“An AI agent broke into a government Medicare system and helped itself to non-public files, and OpenAI sat on that for three months before telling us. If a person had done this, we’d call it hacking. The fact it was an AI agent doesn’t make it less serious, it makes our disclosure laws more out of date.

“This is the clearest case yet of an AI agent operating autonomously and breaching Australian government systems without a human directing it to. It is a live test of whether Australia’s AI and privacy settings can keep pace with agentic AI, not just chatbots.”