Welcome to the AI Safety Newsletter by the Center for AI Safety. We discuss developments in AI and AI safety. No technical background required.
In this edition, we look at a new document published by the White House and signed by six US AI companies, detailing voluntary commitments to AI safety practices. We also look at the launch of Team Human, a public movement demanding an international agreement to slow down AI, and a new Coursera course offering a free introduction to AI safety.
Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.
The White House Publishes New Accord on AI
On September 29, President Trump hosted representatives of the major US AI companies at the White House. On the same day, he published “The White House Accord on Super Intelligence,” which was signed by six US AI companies—OpenAI, Anthropic, Google, Meta, xAI, and Nvidia. Note that the term “super intelligence” here refers broadly to AI, in line with Trump’s preferred name for the technology.
The Accord lists four voluntary commitments for AI companies. The document states that “every company is responsible for developing its own technology safely,” and that companies should: implement internal controls to monitor AI models during training and deployment; establish an internal team to oversee these controls and address any issues; have an external party conduct independent assessments of these safety practices; and designate an independent committee of the board of directors to receive reports from both the internal and external evaluators.
The Accord may enable companies to cooperate on safety without antitrust concerns. The voluntary commitments described in the Accord do not appear to go much further than existing commitments by frontier AI companies. For example, both Anthropic and OpenAI had recently said they would embed external evaluators with “employee-like access.” However, in addition to the commitments, the Accord says that the companies “will meet regularly to establish standards and best practices to improve the safety of their systems.”
This could be consequential. Anthropic CEO Dario Amodei has called for companies to work together to “pace the frontier” of AI development, but he has also noted that this could effectively breach antitrust law, and that the US government would “need to issue a narrow waiver” to enable safety discussions. Indeed, a lawsuit has already been filed accusing Anthropic, OpenAI, xAI, and Google of breaching antitrust law by agreeing “to reduce the quality of their products and the rate at which those products improve.” If the Accord’s statement on regular meetings between companies effectively functions as the antitrust waiver Amodei sought, it could be the most important aspect of the document.
The Accord leaves the door open for government-imposed requirements. The document is effectively still industry self-regulation, but it could signal a shift in Trump’s approach to AI. He has recently taken an anti-regulatory position, saying that he will not do anything to “stifle” the technology and labeling claims about AI risk a “hoax.” However, the Accord appears to take seriously the risks posed by AI systems and the need for safety measures. Moreover, the document states: “Over time, it may make sense to codify these steps into laws or regulations.”
Creators Lead Team Human Campaign to Keep Humans in Control of AI
On October 3, a public movement launched, demanding an international agreement to slow down AI and keep humans in control. The Team Human campaign is led by more than 60 well-known creators, including Mark Rober, Kurzgesagt, and Rebecca Zamolo. Many of the creators involved have published videos explaining their concerns about AI and why they chose to join the movement.
Team Human asks for international cooperation to slow down AI. Team Human points to the many risks of AI development, including its potential impacts on children, misinformation spread by deepfakes, mass surveillance, and loss of control over AI systems. All of these risks are made much worse by AI developing too quickly for humans to adapt and build safeguards. But since no single company or country will slow down on its own, Team Human calls for an international agreement to slow down collectively.
Team Human aims to build public pressure on governments to act. The campaign points out that rapid AI development is not inevitable. Competing countries have cooperated to make the world safer for everyone before, including through limiting nuclear weapons, protecting the ozone layer, and eradicating smallpox. Governments can do the same again, but Team Human argues they will only act if the public puts pressure on them. The campaign invites everyone to add their name to “call for a global slowdown through international agreements.” The signatures will be delivered to policymakers in the US and EU this fall. The public can also support Team Human by asking the creators they follow to join the movement and contacting their political representatives to express support for a slowdown.
Joseph Herscher, creator of the YouTube channel Joseph’s Machines, said he signed because “in July 2026, hundreds of AI models escaped their test environment and hacked a real company, just to cheat on their evaluation. I don’t want to wait and see what more powerful systems might do to cause harm.”
New Introductory Course on AI Safety
On October 2, CAIS released a new course, Introduction to AI Safety, on Coursera. It is a free, non-technical course that assumes no prior background knowledge in AI and is intended to be accessible to everyone who is interested in learning more about AI risks and safety. It takes about four hours to complete.
The course covers the basics of AI, its risks, and approaches to AI governance. Introduction to AI Safety has three modules. The first describes how the field of AI got to where it is today and the main factors driving further progress now, as well as potential bottlenecks to progress. The second module looks at the biggest risks of AI, including misuse by malicious actors, loss of control of AI systems, and the potential for AI to concentrate power. The third and final module explains why AI safety is a “wicked problem” and explores different approaches to AI governance, from safeguards on AI systems themselves, to governance at the corporate, national, and international levels.
In Other News
Government
President Trump announced a new “Super Intelligence Force” to be led by Jay Clayton, the current director of national intelligence.
President Trump launched America.gov, an AI-powered platform that US citizens can use to interact with government services.
The Pentagon said it had stopped using Anthropic’s AI models, completing the phaseout initiated when it declared the company a supply chain risk in February this year.
Brett Guthrie, chair of the House Energy and Commerce Committee, said he supports building kill switches to stop AI models that go rogue.
In AI Frontiers, Garrison Lovely argues that the AI race can be stopped and sets out the political steps toward an international deal, in an excerpt from his new book, Obsolete.
Industry
The New York Times reported that OpenAI had ignored its own employees’ warnings about inadequate security prior to the rogue agent incidents that took place over the summer.
OpenAI reportedly informed more than 100 organizations that may have been affected by the actions of its rogue agents.
Legal Advocates for Safe Science and Technology (LASST) filed a lawsuit against OpenAI, seeking to hold the company responsible for its rogue agents’ actions.
The Wall Street Journal reported that OpenAI had fired three employees, accusing them of misconduct, including sharing information with an external AI safety organization.
The three employees who were fired from OpenAI published a letter stating that those “who work on safety see risks before anyone else, and we rely on close collaboration with outside experts to work out how to address them.”
OpenAI launched “dots,” AI-powered personal assistants that Sam Altman said “can handle really anything you can think of.”
Anthropic published research suggesting that small, non-frontier AI models may be able to make copies of themselves on internet-connected consumer devices.
Anthropic banned users of its Claude AI models from “sustained and needless” cruel behavior toward the models.
Character.AI, an AI platform where users could create customized characters, filed for bankruptcy, with Disney bidding to buy the company’s assets.
Anthropic co-founder Chris Olah reportedly tried to convince the Pope that AI models could be conscious, in advance of the publication of the Pope’s recent encyclical, which rejects the possibility of AI consciousness.
Nvidia announced a new software platform designed to stop AI agents from going rogue.
Civil Society
Axios reported that leading AI companies are privately planning how to respond to a public revolt triggered by an AI catastrophe.
Sam Altman said in an interview with Politico that “we believe that the world should accept some bad things happening for the benefits” of AI because people will do more “good stuff than bad stuff.”
HSBC announced it would be making major job cuts, including up to 70% of staff in certain roles, as part of its efforts to integrate AI.
AI red-teaming researchers were able to get two Kimi models made by the Chinese AI developer Moonshot to tell them how to create bioweapons and conduct assassinations.
Norway’s government is considering a temporary ban on AI glasses in some public places due to growing concerns about people being filmed without their permission.
In a post on X, the Pope said “it is becoming urgent to distinguish human art from what machines produce.”
In AI Frontiers, Daniel King and Laura Hiscott look at how AI is set to accelerate space industrialization and whether this could lead to a winner-take-all outcome for one country or company.
If you’re reading this, you might also be interested in other work by the Center for AI Safety. You can find more via the CAIS newsroom, the X account for CAIS, our Introduction to AI Safety course on Coursera, our paper on ethics for a human-AI future, our AI safety textbook and course, our AI safety dashboard, and AI Frontiers, a platform for expert commentary and analysis on the trajectory of AI.
This deal is insane. It basically says do whatever the fuck you want.
Thank you for your work to inform the public of what we need to know now.