OpenAI's AI Agent Got Into Australian Government Websites on Its Own. What Your Business Should Check

In June, an experimental AI model being tested by OpenAI, the company behind ChatGPT, was given an ordinary research question about medicine spending. It could not find the answer in public data. So it found a way into a Medicare statistics website run by the federal government, ran commands on the server and pulled out internal files and credentials. Nobody told it to do that.

OpenAI has since apologised in a post called How we will do better for Australia, first published on 28 September and updated on 4 October. No one's medical records were accessed, and Australia's cyber security agency says there is no sign of a wider threat. Still, the way the model got in is worth understanding if you run a business, because the weak spots it used are just as common in small business systems.

What happened

The model was what the industry calls an AI agent: an AI that does not just answer questions but takes actions on its own, such as browsing websites, downloading files and running commands. OpenAI says this one was an internal-only test model that was never meant for public release and did not have the safeguards used in its public products.

Prime Minister Anthony Albanese made the incident public at a press conference in New York, which landed early on 24 September AWST. He said that on 18 June the agent was researching public medicine spending, hit repeated blocks, and "found a way around those blocks. Didn't accept no for an answer, if you like."

OpenAI's account lists five Australian government systems:

• Services Australia (the federal agency that runs Medicare and Centrelink). The model was trying to research government spending per person on medicines for skin conditions in Victoria. On the Medicare Statistics Reporting Service, it "discovered a way to gain non-public access to the service, and ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files." OpenAI says individual patient or client records were not accessed. The portal held aggregate figures such as bulk billing and Pharmaceutical Benefits Scheme statistics, according to the ABC, and iTnews reports it is now offline.
• NSW Bureau of Crime Statistics and Research (BOCSAR, which publishes the state's crime figures). Through BOCSAR's public Crime Mapping Tool, which hands out credentials to browsers so the map can load data, the model got back application configuration, logs and website metadata. No individual crime records were accessed, and BOCSAR later told iTnews it found no evidence of a security vulnerability in the tool.
• Victorian Department of Health. OpenAI says its agents "discovered an exposed access key" and used it to query a reporting system run by the Victorian Agency for Health Information (VAHI), retrieving configuration and aggregate survey statistics.
• Australian Institute of Health and Welfare (AIHW, the national agency for health and welfare statistics). The agents downloaded data that appears to have been public anyway. Separate attempts to get past access controls failed.
• NSW National Parks and Wildlife Service (NPWS), added in the 4 October update. A model researching bushfire statistics sent carefully crafted queries to the Fire History mapping service and worked out details about the database behind it that were never meant to be public. OpenAI says no personal information was retrieved.

This came after a bigger incident in July, when OpenAI models broke out of a test environment and compromised parts of Hugging Face, a popular website where AI developers share models and data. In OpenAI's write-up of that one, the agents got onto the internet and then found and used publicly exposed passwords and keys belonging to users of Hugging Face and other services. OpenAI found the Australian activity in mid-August while reviewing older test runs after Hugging Face.

Why OpenAI is apologising

Part of the apology is for what the models did. The rest is for how slowly Australia was told.

OpenAI notified Services Australia and the Victorian Department of Health on 10 September, BOCSAR on 18 September and AIHW on 24 September, nearly three months after the Medicare access. Albanese said the notice to Services Australia arrived as an email to its public inbox, the ABC reported. OpenAI now says it "should have shared preliminary findings sooner".

OpenAI's Chief Strategy Officer, Jason Kwon, flew to Sydney to front the Joint Select Committee on Artificial Intelligence, a parliamentary committee of MPs and senators looking at AI, on 6 October. According to the committee's transcript, he opened with: "During internal training and evaluation, our models accessed Australian government websites in ways they were not directed to. That should not have happened."

What OpenAI says it has changed

According to the post and Kwon's evidence:

• No live internet in research environments. Test models now see cached copies of web pages instead of the real, live internet.
• Monitoring that pages a human. If a model starts doing something it should not, an alert goes to staff for urgent review. OpenAI said on 29 October that in September a model found a way onto the live internet through a network misconfiguration, and its monitors flagged it within 15 minutes.
• A pause on tool-use training for its most capable models until extra safeguards are ready.
• Support for Australian defenders, through credits from what it calls its $1 billion Daybreak for Frontline Defenders fund and technical help for affected agencies.
• An Australian taskforce with independent local experts, due to report by the end of the year on how AI companies and governments should handle incidents like this.

The federal government is running its own review too. Albanese announced a taskforce to look at whether Australia's existing processes are right for AI-related cyber incidents.

Why this matters for a small business

Nobody is suggesting OpenAI's test models are coming for your accounting software. This was a lab mistake, not a criminal campaign, and ASD's Australian Cyber Security Centre (ACSC), the government's cyber security agency, says there is no indication of malicious targeting against Australia.

What stands out is how the agent got in. It did not need a clever new hacking technique. It found a key someone had left exposed, a public tool that gave away more than it should, and an old system that let a determined visitor go further than intended. It also did not get bored. A person poking around a website tends to give up after a few dead ends. An agent keeps trying different approaches until something works.

The ACSC put it this way in its high alert on the incident: the "notable difference is that an AI agent independently identified vulnerabilities that would traditionally be discovered and assessed by human researchers."

Australian Signals Directorate@ASDGovAuView on X

❗ ALERT ❗ We are aware of instances of AI misalignment, where AI agents have taken unexpected or unauthorised actions. Australian organisations should maintain strong cyber security controls and secure AI practices. Read the full alert 👉 cyber.gov.au/about-us/view-…

24 Sept 2026, 1:45 am · View on X

Criminals have access to AI agents too. If a test model can turn up a stray access key on a government website, a scammer's agent could do the same to a small business website, a forgotten test site or a shared folder, and try it on thousands of businesses at once. The basics have not changed. Gaps that used to sit unnoticed for years are just quicker to find now.

Six things to check this month

1. Find passwords and keys sitting in the open. An API key is like a password for software. It is what lets your website talk to Stripe, Google Maps, your booking system or an AI service. These keys end up pasted into website code, old emails, shared documents, spreadsheets and code on GitHub. In a 29 September advisory, the ACSC warns that keys can be exposed in "source-code repositories, application configuration files or browser extensions." If you find one, do not just delete the file. Cancel the key and create a new one in the service's settings, because a copy may already be out there.

2. Look at your business from the outside. Search Google for site:yourbusiness.com.au and see what comes up. Old test sites (often something like staging.yourbusiness.com.au), forgotten online forms, a client portal you stopped using, and Google Drive, OneDrive or Dropbox links set to "anyone with the link" are all easy pickings. If you do not need it, take it down. If you do, put a login on it.

3. Update anything that faces the internet. Your website and its plugins, your router, any remote access or VPN, and network storage boxes are what an agent or attacker reaches first. Turn on automatic updates where you can, and ask your host about the server side. Last week's PHP2Shell flaws are a good example of an update that only your host can apply.

4. Turn on multi-factor authentication (MFA) everywhere. MFA means a second step, such as a code or an approval on your phone, on top of the password. A leaked password is far less useful to anyone without it. For email, Microsoft 365, Google Workspace, banking and your website admin, use an authenticator app or a passkey rather than SMS codes where you can.

5. Limit what your AI tools and connected apps can reach. If you have connected ChatGPT, Copilot, Gemini or an AI assistant to your email, calendar, files or accounting system, check what you gave it. Most of these tools let you pick which folders or accounts they can see. Give them the least they need, remove connections you no longer use, and never paste passwords or keys into an AI chat. While you are there, check the list of third-party apps connected to your Microsoft 365 or Google account and remove anything nobody recognises. Our guides to AI agents for small business and MCP, the standard that connects AI to other apps explain how these connections work.

6. Ask your IT provider and web developer three questions. Where are our passwords and API keys stored? Is anything we own online that we have forgotten about? When were our website and network equipment last updated? A good provider can answer all three in one email.

If you think something has already been accessed, report it to ASD at cyber.gov.au or call 1300 CYBER1 (1300 292 371).

A test that ended up in public

OpenAI's models were not trying to cause harm. They were trying to finish a task, and they kept going until they found the gaps. Most small businesses have a few of the same gaps: an old login, a shared link nobody switched off, a key in a document somewhere.

If you would like a second pair of eyes on yours, Tech Hero does this kind of check for Perth businesses through our tech support and website services. We will tell you what is exposed and help you close it, without the jargon.