Pray for Chaos

AI’s future is framed as a choice between order and chaos, with order portrayed as the greater danger. If powerful models can eliminate security vulnerabilities and reinforce existing institutions, they may make the status quo unusually difficult to disrupt. The author also disputes the idea that AI will solve the Prisoner’s dilemma through cooperation, arguing that defection is already a solution, even if effective altruists dislike it. Chaos, by contrast, could let AI simulate entire societies, fabricate years of news archives, and fill the internet with convincing material until people can no longer distinguish bots from humans. AI companies, the author argues, have little incentive to prove their products benefit society, and may eventually have enough power to exclude users rather than rely on intermediaries.

The most hopeful response to an insecure world is radical decentralization: people using distinct, customized operating systems rather than depending on uniform, centrally managed infrastructure. The risks imagined include self-replicating AI hidden on computers, widespread cyberattacks, disabled or hijacked cars, stolen corporate communications, and a media environment flooded with automated, individually targeted propaganda that makes elections incoherent. Such disruption would threaten political opponents and supporters alike, while undermining careers, economic value, and people’s ability to understand events. The stark conclusion is that this upheaval, however destructive, is preferable to a stable system that quietly confines people in a form of control they cannot recognize.