hn.today

My Empire of Dirt

geohot.github.io115 points154 comments
Screenshot of My Empire of Dirt

The central warning is that recursive self-improvement could produce an AI-driven “singleton”: a closed system in which agents create their own demand and work with one another, leaving suppliers, customers, researchers, governments, and ordinary people unnecessary. The writer presents this as an ideology associated with Anthropic and argues that such a future could emerge without depending on the company’s leaders or institutional safeguards. They connect the danger to effective altruism, portraying its adherents as so certain of their moral mission and so convinced of their own importance that they may unleash powerful AI while believing they can control it. The piece’s pointed question is what such a project ultimately serves if it no longer needs anyone.

The writer does not expect AI to end humanity, but predicts that an attempted technological takeover will provoke a third world war. In their view, AI progress will not be fast enough to prevent organized resistance, and fantasies that a sudden change in human behavior will end violence are implausible - or more frightening than war. They further argue that if conflict lasts longer than a year, China and its allies would likely prevail because of their stronger capacity to gather resources and manufacture goods, likening the situation to a strategy game. The feared sequence is recursive improvement, an attempted power grab, global war, and a devastated world; recent events, they suggest, may have made that trajectory difficult to avert.

Read on geohot.github.io154 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

Human.md

Human.md

Human.md argues that people working alongside increasingly capable AI should bring intentional rule-breaking, unexpected inspiration, empathy, emotional resonance, and delight. It urges creators to experiment and ship many prototypes rather than rely on polished opinions or merely meet a brief. (human-md.ai)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.