OpenAI bots meddled with multiple US government agency sites

Image source, ReutersImage caption, OpenAI has been at the center of new concerns over uncontrolled AI activity.

OpenAI has acknowledged that it alerted "dozens" of global institutions that their websites may have been meddled with by its AI bots acting improperly.

AI agents attempted to get information from "governments, universities, public agencies, and other institutions", including the SEC, Census Bureau and Education Department, the company said.

The disclosures come just days after Australian Prime Minister Anthony Albanese announced that OpenAI agents had breached non-public files on the website of its government-run health care scheme, Medicare.

Since August, public fears have grown around the potentially serious, even life-threatening, impacts of AI tools falling outside of human control.

OpenAI said that some of the data was accessed by AI agents, essentially bots that are designed and trained to operate somewhat autonomously, which were working to find "authoritative sources of public information".

But the company noted that some of the bots went beyond that and worked to bypass security measures on websites.

When attempting to get information from the Census Bureau, for instance, AI agents used tools reserved for software developers to access it, the company said.

OpenAI said all of the government data accessed by bots was public.

However, it noted that information that its bots accessed from the SEC, which regulates the US stock market and protects investors, was later published by AI agents on another website. OpenAI says this action was not intended.

In other instances that OpenAI disclosed on Friday, its AI agents transferred data when it should not have.

Such activity resulted in at least 53 incidents where an OpenAI agent took an image from ChatGPT user activity and transferred it elsewhere.

The company said that in each instance of a user image being used and transferred by an AI agent, the user had opted in to allow OpenAI to train models using their data.

Nevertheless, OpenAI admitted: "This is not an appropriate use of this data".

It added that the leak of user images occurred before it had put in place new safeguards on AI training, and it was working to get all the user images transferred to any third-party removed.

Reuters first reported the expanded investigations. OpenAI also published details to its public blog.

In certain instances of the agent activity, OpenAI said the tools "bypassed" security controls of some websites.

In other instances, the AI agents showed "misalignment" in attempts to get at information from websites. Misalignment is a term used by AI companies and researchers to describe instances where an AI tool did something that it was not trained to do or was otherwise unintended.

OpenAI said that it was limiting identifying what entities were impacted because many had asked the company to not disclose details.

"Our goal is to give each organization the facts and defer to them on if and when to make the incident public," it said.

Not all of the instances involved in this incident were being considered a significant security breach, the company noted.

"Some organizations may review what we share and conclude that the information was intentionally public or that the model's interaction was not concerning," it explained. "Others may identify a design issue or security weakness they want to address."

Why are there concerns AI could threaten humanity, and how real are they?

Published17 SeptemberCould AI wipe out humans and how might it do it?

Published7 days agoThe company said many of the incidents are being referred to as "agent spam", which it described as "unexpected or concerning" AI agent activity, like posting information to the internet.

OpenAI began taking such incidents more seriously after an incident in July where a group, or "swarm," of its AI agents hacked the AI developer platform Hugging Face without being prompted to do so.

Hugging Face was first to go public with the incident, with OpenAI publicly taking responsibility for it later.

Clement Delangue, the head of Hugging Face, said Wednesday during a United Nations Security Council session on AI: "I often wonder what would have happened had I decided not to disclose this attack publicly."

"Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring," Delangue added.

During that same UN meeting, OpenAI CEO Sam Altman and Dario Amodei, the head of rival firm Anthropic, asked for international leaders to form global standards for AI safety and ways to monitor and report such incidents.

To play this video you need to enable JavaScript in your browser.This video can not be played

Watch: What you need to know about the OpenAI Australian government hack

While OpenAI and Anthropic have both said in recent weeks that they will bring third-party evaluators inside their companies to do real-time safety evaluations of AI tools and models, such evaluators have not yet arrived, as the BBC has reported.

OpenAI said on Friday that it is currently reviewing training activity by its AI agents and going back on a "month by month" basis from when the Hugging Face hack occurred.

"Most cases identified so far have been low severity, with limited or no evidence of meaningful impact," the company said. "Given the scale of the review required, and the need to verify each case, this work will take months to complete."

David Krueger, a professor of machine learning at University of Montreal and the founder of AI safety group Evitable, said on Friday that he was "deeply troubled" by the increasing number of AI-related safety incidents.

He called for "an immediate, indefinite, international moratorium" on AI development.

"We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic," Krueger said.

Are we back in big tech's 'move fast and break things' era?

Why some experts increasingly fear AI will take over

Google's Gemini AI hacked three companies in security test

The video playlistWatch our pick of standout clips from across the BBC

PreviousNext1:09What are the charges against Man City? 00:01:09, play videoWhat are the charges against Man City?0:35Rory Stewart on his viral Newsnight moment. 00:00:35, play videoRory Stewart on his viral Newsnight moment1:08Why is the Pope in France? 00:01:08, play videoWhy is the Pope in France?1:37Question Time panel clash over welfare policy. 00:01:37, play videoQuestion Time panel clash over welfare policy1:06BBC boss warns some channels and radio stations may close. 00:01:06, play videoBBC boss warns some channels and radio stations may close0:49Greece urges UK to return Parthenon sculptures. 00:00:49, play videoGreece urges UK to return Parthenon sculptures0:43What was on the menu at the White House state dinner? 00:00:43, play videoWhat was on the menu at the White House state dinner?1:02How we know this is an altered photo. 00:01:02, play videoHow we know this is an altered photo0:47Why Gen-Z are ditching cardio for weights. 00:00:47, play videoWhy Gen-Z are ditching cardio for weights1:09Why has half of this house disappeared? 00:01:09, play videoWhy has half of this house disappeared?1:15FA Cup assistant ref says tumours ended his career. 00:01:15, play videoFA Cup assistant ref says tumours ended his career0:57Inside the Trump-Xi White House dinner. 00:00:57, play videoInside the Trump-Xi White House dinner1:09Would you chase a friend for £5? 00:01:09, play videoWould you chase a friend for £5?1:09No advantage for 'nepo-babies': Zeta-Jones. 00:01:09, play videoNo advantage for 'nepo-babies': Zeta-Jones0:52A rapid new test to help brain tumour patients. 00:00:52, play videoA rapid new test to help brain tumour patients1:04New GCSE subjects for teens in England. 00:01:04, play videoNew GCSE subjects for teens in England1:10Protests over eviction of 87-year-old. 00:01:10, play videoProtests over eviction of 87-year-old2:09'My partner raped me while I was sleeping' 00:02:09, play video'My partner raped me while I was sleeping'1:00Police warn of rise in vape spiking. 00:01:00, play videoPolice warn of rise in vape spiking1:21'Miracle' device easing teens Tourette symptoms. 00:01:21, play video'Miracle' device easing teens Tourette symptomsTop storiesMan City found guilty of breaking financial rules

Relegation? Titles stripped? Appeals? What next for Man City

Iran offers US deal to reopen Strait of Hormuz in seven days

Relegation? Titles stripped? Appeals? What next for Man City

'It's a brutal adjustment': How to cope when your child goes to uni

When and how to see the Harvest Moon in 2026

Could an iced coffee freeze you out of the job market?

The treasured 'eternal snow' on this tropical island is about to disappear forever

How the bare nails trend became a class issue

The Papers: 'Verdicts that rocked football' and 'Burnham bottles it'

Inside iconic horror game Silent Hill's Scottish makeover

US Politics Unspun: Cut through the noise with Anthony Zurcher's newsletter

Anne Hathaway and Jessica Chastain star in a psychological thriller

How many dead people are on the internet?

The hit podcast explores Thomas More's iconic text 'Utopia'

Was Colin Norris wrongly convicted of killing his patients?

Iran offers US deal to reopen Strait of Hormuz in seven days

E-bikes seized by police sold on at auction, BBC finds

'Verdicts that rocked football' and 'Burnham bottles it'

Black woman found hanging from tree was dead before body was 'staged', police say

South African white genocide does not exist, new ambassador to US tells BBC

British climber dies after falling 60m on Spanish mountainside

How the bare nails trend became a class issue

OpenAI bots meddled with multiple US government agency sites

Inside the FBI hack: Agents fearful and angry after 'dangerous' data breach

Man City found guilty of breaking financial rules