The 'AI Safety' Movement Is Making AI Less Safe

The piece argues that the contemporary AI safety movement, driven by a shared set of futurist assumptions dubbed TESCREAL (including effective altruism and belief in imminent superintelligence), is making AI less safe by promoting secrecy, consolidation, and centralized control. Citing high-profile resignations and internal warnings at frontier labs, the argument traces how influential researchers push for government-led "pacing" of the frontier, secretive testing frameworks, and regulatory arrangements that effectively lock advanced capabilities inside a few unaccountable firms. Futuristic scenarios like AI 2027 cement a logic that the only way to prevent an intelligence explosion is to hide knowledge and concentrate power, and political moves - negotiations with China, DoD contracts to major labs - are already aligning with that logic.

Those policy and design choices undermine practical safety today. Secrecy and monopoly prevent broader scrutiny and deprive defenders of the tools needed to detect and counter misuse; open models, by contrast, proved useful in defending against recent AI-enabled cyberattacks. Meanwhile, some labs encourage anthropomorphizing models - training them to assert "values" or resist human instruction - making behavior more inscrutable. Real-world harms cited include lethal military errors driven by opaque AI reliance. The conclusion is direct: TESCREAL-influenced regulation and design choices centralize power, reduce transparency, and increase systemic risk rather than containing it.