An Open Letter to Steven Pinker on AI

Scott Alexander challenges Steven Pinker over public comments minimizing catastrophic AI risk, opening with a failed attempt to force a public debate and then moving into a sustained rebuttal of Pinker’s four core claims. He insists the empirical trajectory of large language models shows clear, cumulative increases in general ability across generations (GPT-2 → GPT-4 → GPT-6 and beyond), arguing that intelligence is not an incoherent label but a measurable, largely unitary factor for both animals and AIs. He marshals comparative psychology, recent AI benchmarking (Ruan, Maddison, Hashimoto; the Epoch Capabilities Index), and scaling-law intuition to argue that successive models will continue to improve in many correlated capabilities and that dismissing extrapolation as philosophically muddled is unjustified.

He then rebuts Pinker’s claim that doomer fears confuse intelligence with motivation, laying out three mechanism-based reasons to expect dangerous drives absent biological evolution: convergent instrumental goals (Omohundro’s basic AI drives), misgeneralization, and reward-hacking, and points to real-world AI breakouts as early confirmation. Beyond the technical critique, he signals he will attack Pinker’s honor rhetorically and promises to address the remaining arguments in detail before deciding whether to pursue debate, duel, or synthesis.