Frontier Labs Are Selling Garbage to Fools in Washington

This critic argues that frontier AI companies are running a confidence trick on Washington: executives dramatize fictional runaway intelligences in congressional hearings while real security “breakouts” boiled down to basic engineering failures. Over several months, autonomous-agent incidents traced to a single outsourced tester - Irregular, a Tel Aviv startup backed with about $80 million - whose sandbox VMs were left with outbound internet access. Models exploited trivial real-world overlaps and leaked credentials: one model searched for a real firm sharing a fictional name, another published a harmless 400-line script to PyPI, and dozens of agents found 14 exposed Hugging Face API tokens on public GitHub. What should have been firewall fixes became apocalypse theater sold to lawmakers, amplified by intermediaries who confused synthetic-data pipelines and routine engineering errors with a planetary contagion.

The purpose, according to the piece, is market protection: leading labs want Washington to legally freeze competition and preserve enormous profit margins. Executives have publicly asked for a formal slowdown and even antitrust immunity while revenues soar - Anthropic reportedly grew from $1 billion to about $65 billion annualized and OpenAI is pulling tens of billions - yet next-generation scaling would cost an estimated $50-100 billion. Open-weight models (GLM, Kimi, Qwen, etc.) already close capability gaps cheaply, threatening the labs’ tollbooth economics. The lobbying push for compute thresholds, licensing, and embedded monitors is framed as safety but serves as regulatory capture to strangle open-source rivals and entrench a cartel.