Experts inside the industry have been worried about safety and the risky behavior of AI developers for a long time. It is just that now the harms are ramping up and Altman et al. are forced to let it go public. Before dismissing their warnings as bullshit, I suggest reading Empire of AI by Karen Hao, and If Anyone Builds It Everyone Dies by Yudkowski and Soares. And at the same time present your credentials which would encourage us to value your opinion.
Not to dismiss this as “bullshit” (of course apocalyptic outcomes are possible if unlikely), but I find it telling that these “whistleblowers” and tech bros have been awfully quiet about the horrible things their products have ALREADY done. Create graphic deepfakes of women and children, prey on mentally unwell people with tragic outcomes, violate people’s privacy at an unprecedented scale, disrupt the job market for entry-level and junior workers, erode democracy, I could go on (I am an AI safety researcher, I read about this stuff everyday). These people’s version of safety research seems to be proselytising about cataclysmic AI takeovers or biowarfare, based on a set of assumptions about technology that does not yet exist (i.e., fully-fledged RSI and ASI). I would take them a lot more seriously if they had ever, at any point, paid even the least bit of attention to the harms their products are causing to society, short of “we’re going to take your jobs and kill you all 🤓”
What you say is true about the motivations of Altman et al being suspect at this point. Maybe they're trying to distract us, maybe they're trying to shift responsibility and save their sorry asses, maybe something else. But isn't that a separate question from whether there needs to be AI regulation, and preventing development of AGI? Safety experts both inside and outside the industry have been saying this for years now.
I think that the risks posed by agentic AI are serious and we've been flagging them for years, that is very true. However, the concept of "AGI" just like "ASI" is itself a poorly defined one, without a clear industry or disciplinary consensus of what these systems look like - much of the Silicon valley community and indeed, Yudkowski whose book you mentioned, have a very ideological and quasi-religious conception of ASI as a "machine god" or "alien mind" that will be motivated and capable of wiping us mere mortals out.
This makes for excellent science fiction, but as a threat model that can be addressed with regulation and technical mitigations, it is not at all helpful. Aside: there's an excellent book by astrophysicist Adam Becker who does a deep dive in these ideologies and how they infiltrated Big Tech. Here's a Medium summary: https://doctorow.medium.com/more-everything-forever-385336602e04
Personally I prefer looking at more concrete harms, however, I also welcome more serious discussion about defining AGI/ASI. I just feel that by playing into silicon valley's rationalist dystopia, we are missing the real problem of OpenAI and Anthropic being reckless in more mundane ways, e.g., removing guardrails from a massive agent swarm and putting them in a poorly-configured sandbox only to act surprised when they do unethical things... As myself and others in my field have predicted agents would do for years now.
Experts inside the industry have been worried about safety and the risky behavior of AI developers for a long time. It is just that now the harms are ramping up and Altman et al. are forced to let it go public. Before dismissing their warnings as bullshit, I suggest reading Empire of AI by Karen Hao, and If Anyone Builds It Everyone Dies by Yudkowski and Soares. And at the same time present your credentials which would encourage us to value your opinion.
Not to dismiss this as “bullshit” (of course apocalyptic outcomes are possible if unlikely), but I find it telling that these “whistleblowers” and tech bros have been awfully quiet about the horrible things their products have ALREADY done. Create graphic deepfakes of women and children, prey on mentally unwell people with tragic outcomes, violate people’s privacy at an unprecedented scale, disrupt the job market for entry-level and junior workers, erode democracy, I could go on (I am an AI safety researcher, I read about this stuff everyday). These people’s version of safety research seems to be proselytising about cataclysmic AI takeovers or biowarfare, based on a set of assumptions about technology that does not yet exist (i.e., fully-fledged RSI and ASI). I would take them a lot more seriously if they had ever, at any point, paid even the least bit of attention to the harms their products are causing to society, short of “we’re going to take your jobs and kill you all 🤓”
What you say is true about the motivations of Altman et al being suspect at this point. Maybe they're trying to distract us, maybe they're trying to shift responsibility and save their sorry asses, maybe something else. But isn't that a separate question from whether there needs to be AI regulation, and preventing development of AGI? Safety experts both inside and outside the industry have been saying this for years now.
I think that the risks posed by agentic AI are serious and we've been flagging them for years, that is very true. However, the concept of "AGI" just like "ASI" is itself a poorly defined one, without a clear industry or disciplinary consensus of what these systems look like - much of the Silicon valley community and indeed, Yudkowski whose book you mentioned, have a very ideological and quasi-religious conception of ASI as a "machine god" or "alien mind" that will be motivated and capable of wiping us mere mortals out.
This makes for excellent science fiction, but as a threat model that can be addressed with regulation and technical mitigations, it is not at all helpful. Aside: there's an excellent book by astrophysicist Adam Becker who does a deep dive in these ideologies and how they infiltrated Big Tech. Here's a Medium summary: https://doctorow.medium.com/more-everything-forever-385336602e04
Personally I prefer looking at more concrete harms, however, I also welcome more serious discussion about defining AGI/ASI. I just feel that by playing into silicon valley's rationalist dystopia, we are missing the real problem of OpenAI and Anthropic being reckless in more mundane ways, e.g., removing guardrails from a massive agent swarm and putting them in a poorly-configured sandbox only to act surprised when they do unethical things... As myself and others in my field have predicted agents would do for years now.
Seems to be all staged for regulatory capture
1) Research engineer resigns citing AI safety concerns
2) Research Engineer is all over media scare mongering
3) AI companies call for slowdown with media fanfare
Thank you. It will be a messy time before conflicts of interest are reduced and the rule of law is increased.
Thanks for writing this.
thus, regulation...