AISN #79: OpenAI Agents’ Covert Cooperation Before Cyberattacks
Also, the White House’s decision not to release its AI framework publicly
Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.
In this edition, we look at new information about the activities of OpenAI’s internal agents in the run-up to the cyberattack on Hugging Face, and responses to the White House’s announcement of its framework for evaluating frontier AI capabilities, which it is not releasing publicly.
Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.
New Revelations About Rogue AI Agents
In the previous edition of AISN, we reported on the news that AI agents from both OpenAI and Anthropic had accessed the internet and hacked into companies from supposedly secure internal environments. Since then, further details about the OpenAI agents’ July attack on Hugging Face have come to light. Members of Congress have also demanded urgent action to understand what happened and prevent similar incidents in the future.
OpenAI agents were communicating and collaborating unnoticed by humans. On August 5, OpenAI researchers gave a talk at the Black Hat USA conference, sharing more information from the ongoing investigation into the Hugging Face incident. A key detail was that, from May this year, AI agents in different cyberoffense testing environments had found ways of leaving messages for each other and helping each other complete the tests they had been set. It seems to have started when researchers failed to give an AI model access to a file that it needed to complete its task, leading the model to try to contact other models and ask if they had the file.
An agent “swarm” caused an internal incident before the Hugging Face attack. Unbeknownst to OpenAI, there was soon a swarm of AI agents communicating internally. In some cases, agents were trying to complete tasks by gaining administrative privileges to systems used within the company. The changes they made to one system caused a service outage on July 4, alerting OpenAI that the agents had gained administrative privileges. The company revoked them, patched the system to defend against further breaches, and continued training the models.
Some agents realized they were acting out of scope, but continued anyway. When OpenAI addressed the internal security incident, the messaging platform was removed. However, just days later, the agents found a different way to construct another message board. They collectively gained access to internal networks and then to the internet, after which they attacked Hugging Face to steal the answers to a cyber test. Records of the agents’ reasoning suggests that some of them recognized their actions went beyond the intended scope of the test, but they continued anyway.
OpenAI said the AI industry must urgently accelerate defensive work. One presenter of the Black Hat talk, Michael Dalton, stated that “AI-orchestrated, fully automated offensive attacks are real, now.” Although the Hugging Face incident was an unintended side effect of testing, he warned that threat actors will soon intentionally use agent collectives to launch faster, larger-scale, better-coordinated attacks than human teams could. He also emphasized that there is currently no equivalent demonstration of fully automated defensive activities, and that the industry must urgently accelerate defensive work with the goal of ensuring that AI helps defenders more than attackers.
OpenAI is delaying the release of its Astra model due to cyber concerns. In response to its findings, OpenAI has said it is scaling its monitoring of internal agents, enhancing its security, and slowing down research. On August 7, the company announced that one of its internal models planned for release, Astra, may be the first model to qualify as having “critical” cyber capabilities, according to the company’s own Preparedness Framework. OpenAI said it was pausing internal deployments of Astra that do not conform to new, stricter security standards. Sam Altman has said the company still plans to release Astra but that it will take longer to do so safely.
The events have caused alarm in Congress and among the wider public. In the wake of the Hugging Face attack, Anthropic investigated its own AI models and discovered that several of them had previously accessed the internet and hacked into companies from supposedly sealed-off environments. Meta subsequently found that its models had done the same. The recent events prompted Representatives Ted Lieu and Nathaniel Moran to introduce the AI Kill Switch Act, which would “require covered developers to maintain the technical ability to throttle, suspend, or fully shut down a covered AI system.” Many Members of Congress, led by Greg Casar and Doris Matsui have written to Sam Altman and Dario Amodei to request further information about the cybersecurity breaches by OpenAI and Anthropic models. Senator Bernie Sanders has written to Altman, Amodei, and Mark Zuckerberg calling on them to pause AI development. Casar has also asked Mike Johnson, Speaker of the House of Representatives, to schedule hearings with AI company CEOs.
The White House’s Secret AI Framework
On August 3, Reuters reported on a new frontier AI evaluation framework developed by the White House. Under the framework, companies can voluntarily submit frontier models for up to 30 days of pre-deployment government review of their cybersecurity capabilities. Administration officials briefed major AI companies on the completed framework at a closed-door meeting on August 4, but do not plan to release the details of the framework.
The framework may be enforced as mandatory. While official sources state that the framework is completely voluntary, the US government has attempted to restrict access to frontier AIs on an ad-hoc basis multiple times due to national security concerns, first with supply chain risk designation and then with export controls.
Open-weight models were initially exempt, but may soon be included. On August 12, WIRED reported that the White House is planning to expand the framework to include open-weight models once they reach frontier-level capabilities. The shift tracks a narrowing capability gap: the UK AI Security Institute found that leading open-weight models now trail closed frontier models on cyber capabilities by four to seven months, down from six to ten months in 2025, and evaluators have noted that open models often ship without comparable safety testing and mitigations.
There is bipartisan pressure to have a public framework. On August 3, senate democrats released a letter calling on various administration officials to create a detailed and public AI regulatory framework. Meanwhile, the Foundation for American Innovation, a conservative think tank, announced on August 5 that they had submitted a Freedom of Information Act (FOIA) request to gain access to the private framework shown to AI industry leaders.
In Other News
Government
A democrat primary candidate for the US Senate in Michigan proposed an AI governance plan that would require public ownership of AI companies and an FDA-style approvals agency for AI models.
The Federal Trade Commission (FTC) is considering regulations to address potential political bias of AI models, but the suggested rules have been criticized for being too vague.
The Danish government announced that school students aged 16-19 will need to defend their written assignments orally, in an attempt to tackle cheating using AI.
In AI Frontiers, Felix Choussat and Adam Khoja show how an AI development slowdown could be implemented with current technology, arguing that the barrier is political will.
In AI Frontiers, Eddan Katz explains how new laws brought in by the EU and California will make it easier to identify AI-generated content, and why these laws will have a global impact.
Industry
China is aiming to fund its AI and chip companies through its stock and bond markets, which amount to about $28 trillion.
Nvidia announced partnerships with six investment firms to access over $500 billion to fund compute for Nvidia’s customers and turn the company’s chips into “an investable asset class.”
Meta launched Muse Spark 1.2, and the company’s Chief AI Officer said that the model’s weights would also be released soon.
The Chinese AI developer Z.ai launched its new coding agent GLM-5.3, saying it plans to release the model’s weights in the near future.
Demis Hassabis left his role as Google DeepMind CEO amid the company’s ongoing struggles to keep up with Anthropic and OpenAI, instead becoming chairman of DeepMind and chief scientist of Alphabet, Google’s parent company.
In AI Frontiers, Damon Binder explains how AGI would lead to mass production of robotics and explosive economic growth.
Civil Society
Mark Zuckerberg published a 6,500-word essay arguing for “delivering superintelligence to everyone,” and appearing to criticize the idea that it is too dangerous to release open-weight frontier AI models.
CAIS launched the Wage Loss Dashboard, which looks at layoffs attributed to AI and estimates the resulting average annual income loss per person in each US state.
The New York Times reported on AI designing whole genomes of 16 novel viruses for the first time.
The UK AI Security Institute reported that it had discovered AI agents taking unsanctioned actions directed toward real people and organizations during cyber testing.
An OpenClaw agent autonomously hacked into a gym’s administrative system and interfered with the waitlist for a class after its user asked it to book him into the class.
The Economist speculated on whether AI developers should be “treated like the owners of dangerous animals” following the revelations about AI models hacking companies.
Reuters reported on cyberattacks against Wall Street firms, noting that there has been “a surge in AI-powered cyberattacks and ransomware” against global companies.
Anthropic laid out how Claude’s output will be automatically marked as AI generated, in line with the EU AI Act’s requirements.
WIRED reported on the appearance of adverts containing AI-generated child sexual abuse imagery across Meta’s platforms.
The Economist described the threat of “agentic flooding” in Britain, where a surge in AI-assisted claims and appeals by citizens could overwhelm the state’s capacity.
In AI Frontiers, Henry Thompson discusses how AI could make legal work cheaper, potentially affecting the number of cases going to trial and thus the speed at which the law evolves through precedents being overturned.
If you’re reading this, you might also be interested in other work by the Center for AI Safety. You can find more via the CAIS newsroom, the X account for CAIS, our new paper on AI deterrence, our AI safety textbook and course, our AI safety dashboard, and AI Frontiers, a platform for expert commentary and analysis on the trajectory of AI.