A developer and writer frames the pursuit of artificial general intelligence as the creation of a God-like machine: scalable compute, data and a paradigm breakthrough could produce intelligence that far outstrips humans, and industry is racing to build it. Drawing on Baptist imagery, he contrasts two governance models: a democratic "priesthood of all believers" where everyone has direct access to superintelligence (the "Zuckerbergian" option), versus a guarded arbiter-class that controls access, exemplified by companies limiting models to governments and big tech. He admits personal attraction to the social protections of a priesthood but notes the two approaches reflect fundamentally different values about who should steward such power.
He treats superintelligence as likely and insists humanity will have to learn to live with, learn from, and try to control it while confronting what such an intelligence will think of fragile, slow humans and what bargains we might offer for its cooperation. He warns of Faustian trade-offs - offering power or autonomy in ways we could later regret - and emphasizes the ethical stakes of training data, hoping any created godlike intelligence learns empathy rather than inheriting science-fiction horrors. He expresses relief that others in the field are asking these same questions and wants a merciful outcome.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.