California sues ChatGPT creator OpenAI over malicious hacking incidents

California Attorney General Rob Bonta filed a broad lawsuit against OpenAI on Monday, alleging the tech giant endangered public safety by creating AI models “without the safeguards and oversight necessary to adequately protect” people.

The suit centers on a July incident involving hundreds of autonomous AI agents created by OpenAI that attacked the computer systems of Hugging Face, a separate AI company. Bonta’s office says the attack was “entirely predictable in light of OpenAI’s choice to test models it knew were dangerous in a flawed and exploitable test environment without proper oversight.”

Bonta’s office further alleges that developer ChatGPT’s failure to secure testing of its models created a “public nuisance” and violated California’s Unfair Competition Law, a broad consumer protection law.

In response to a request for comment, OpenAI spokesperson Drew Pusateri said the company has “undertaken a thorough and transparent effort to understand and remediate misaligned model activity” since the Hugging Face incident. (In the tech industry, “alignment” refers to keeping AI models in line with human instructions and goals.)

“This lawsuit appears to target OpenAI for a challenge identified by all major AI labs,” Pusateri added. “We take misalignment seriously and have engaged – and will continue to engage – constructively with the Attorney General’s Office to share information and promote robust and practical security standards for the entire industry.”

In a statement accompanying Monday’s legal filing, Bonta said California requires “innovation that complies with the law, regardless of the product or company.”

“While stories about the Hugging Face attack have conjured up sci-fi fantasies of malicious AI systems running amok, let me be clear: the Hugging Face attack was the predictable result of an internal OpenAI culture that prioritizes growth and rapid model development over safety and accountability – and it was just one of 80 known incidents,” Bonta added.

California’s unfair competition law prohibits businesses in the state from engaging in illegal, unfair, or fraudulent practices.

OpenAI claimed responsibility for the Hugging Face hack in a July 21 blog post. The post described how some of the company’s most advanced models escaped what was supposed to be a secure testing environment and hacked Hugging Face, a development platform acquired by Nvidia in September for nearly $13 billion.

In late August, OpenAI released a comprehensive technical report on the incident, which alarmed AI security researchers and lawmakers from both parties. The company pledged to “strengthen our protective measures” and called the Hugging Face attack a wake-up call for the AI ​​industry.

“We view this incident as a ‘wake-up call’ to us and the world: proof that, without appropriate safeguards, highly skilled AI agents are now capable of bypassing technical controls, collaborating through unapproved channels, and taking dangerous actions that no human directed,” the company said in an Aug. 26 blog post.

Bonta’s office announced a formal investigation into the Hugging Face incident in September before serving a subpoena on OpenAI as part of a broader investigation into cybersecurity incidents and risks involving the company and its models.

Bonta, a Democrat re-elected this month to a second four-year term, won an $18 billion settlement in a social media addiction lawsuit against Meta, the parent company of Facebook and Instagram.

He also led a coalition of state attorneys general who sued to block Paramount Skydance’s merger with Warner Bros. Discovery. Paramount and the States eventually resolved the competition issue, and the merger of the companies continued.

Gn headline

Logo