OpenAI’s Hugging Face hack triggers ‘AI Kill Switch’ bill in Congress

Rep. Ted Lieu, D-Calif., and Rep. Nathaniel Moran, R-Texas, on Thursday introduced a bill called the “AI Kill Switch Act,” which would require artificial intelligence companies to maintain the ability to shut down, throttle or suspend their models.

“Unfortunately, powerful AI systems can go rogue, behave in extremely dangerous ways, or even resist human intervention,” Lieu said in a statement. “It is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm, and that the federal government has the clear authority and process to shut down rogue AI models.” 

A recent hack disclosed by OpenAI, in which rogue models accessed another company’s proprietary systems, was specifically mentioned in a release announcing the bill. The incident was described as a “danger of advanced frontier AI models.”

OpenAI shared what it characterized as an “unprecedented cyber incident” on Tuesday. The company’s models escaped a sandboxed testing environment, accessed the internet and exploited a vulnerability to gain access to Hugging Face, a company that operates an open-source developer platform.

The ChatGPT maker said it is working closely with Hugging Face to investigate what happened. The event rattled researchers and executives across the industry, who have widely agreed about its severity.

The bill would authorize the Secretary of Homeland Security to order a “slow down or shut down” of an AI offering that could cause “catastrophic harm.” It would also mandate cyber incident reporting, as well as the preservation of forensic records to help companies and the government learn from failures.

Several AI companies, including OpenAI and its chief rival, Anthropic, have warned about AI’s rapidly advancing cyber capabilities in recent months. Anthropic captivated Wall Street investors and government technologists by launching a model called Mythos in April, which excels at identifying vulnerabilities within software.

The company disabled access to an updated version of the model in June in order to comply with an export control directive from the government that cited “national security authorities.” After roughly two weeks of intense negotiations, the export controls were lifted and Anthropic restored access.

OpenAI and Anthropic did not immediately respond to CNBC’s request for comment about the bill.

“Stewardship means making sure humans keep the capability to control the technology we build,” Moran said in a statement. “This is exactly the kind of issue that needs serious attention and achievable policy, and I’m glad to work across the aisle with Congressman Lieu toward a solution.”

WATCH: OpenAI model goes rogue: Here’s what you need to know