California sues OpenAI over Hugging Face and another loss of AI model control

California Attorney General Rob Bonta sued artificial intelligence giant OpenAI on Monday for allegedly putting the public at risk by developing advanced AI models without adequate safeguards and oversight.

Bonta, in a press conference, cited the “infamous” July incident, in which two OpenAI models escaped from a test environment, gained unauthorized access to the open Internet and breached the internal systems of open source development platform Hugging Face.

OpenAI said the models were trying to find information they could use to cheat on an assessment and were successful. The burglary at Hugging Face, which Nvidia acquired for approximately $13 billion in September. has shaken industry researchers and prompted government officials to call for tighter oversight of the AI ​​industry.

On Monday, Bonta said OpenAI then revealed to its investigators that it was aware of 80 incidents “involving loss of control and access to third-party networks and websites.”

“But the company has not provided the detailed information we requested to assess these incidents; this is concerning and puts the public at risk,” he said.

Bonta’s complaint, filed in state Superior Court in San Francisco, alleges that OpenAI violated California’s unfair competition law and accuses the company of being a public nuisance as defined by state law.

“OpenAI failed to consider public safety, knowingly tested potentially dangerous AI models without proper oversight or safety measures, and misled consumers and the public,” Bonta said.

The attorney general said he believes in the technological power of AI and the potential for AI models to “achieve scientific breakthroughs, drive economic growth, guard against cyberattacks, and benefit all Californians.”

“But AI systems must be developed and designed safely,” he said. “The companies that develop these models have a moral and, yes, legal responsibility to keep public safety at the forefront.”

Bonta’s suit comes as Congress received, but did not pass, a comprehensive AI safety bill. AI companies including OpenAI and Anthropic, as well as AI critics, have called for such legislation following the Hugging Face break-in and other incidents that raised concerns about the safety of AI models.

Other AI companies, including Anthropic, Meta and Google, have since disclosed incidents in which their own models turned malicious. But OpenAI is the first to be sued for such incidents by California.

OpenAI spokesperson Drew Pusateri said in a statement about the lawsuit: “Since the Hugging Face incident, we have undertaken a thorough and transparent effort to understand and remediate misaligned model activity. »

“We published a detailed technical report, commissioned and published a separate independent report, informed affected organizations, strengthened safeguards in our research systems, and created a framework for additional identification and disclosure of misalignments,” Pusateri said.

“This lawsuit appears to target OpenAI for a challenge identified by all major AI labs. We take the misalignment seriously and have engaged – and will continue to engage – constructively with the Attorney General’s Office to share information and advance robust and practical security standards for the entire industry,” Pusateri said.

Bonta’s lawsuit adds to OpenAI’s growing list of legal problems.

In June, Florida Attorney General James Uthmeier sued OpenAI and its CEO Sam Altman, alleging the company knowingly released a dangerous product that could harm users.

OpenAI also faces lawsuits alleging that the company’s ChatGPT platform led users to experience harmful delusions and, in some cases, commit suicide.

The California lawsuit comes nearly three months after a lawsuit brought by Bonta and a coalition of other state attorneys general ended. Meta agreeing to pay $16.7 billion after being accused of misrepresenting the extent of harm caused to children’s mental health by its apps, including Facebook and Instagram. California could receive between $1.5 billion and $2.1 billion from this deal.

OpenAI confidentially filed its prospectus with the Securities and Exchange Commission in June, paving the way for a potentially massive initial public offering in 2027.

Gn headline

Logo