hn.today

Thomas Nagel on AI Consciousness

wired.com14 points1 comments
Screenshot of Thomas Nagel on AI Consciousness

Thomas Nagel, awarded the $1 million Berggruen Prize for his work on philosophy and consciousness, argues that today’s large language models are not conscious, despite how convincingly they speak. He rejects the view that consciousness can arise from any system performing the right functions, regardless of its physical makeup. In his view, consciousness is likely tied to physical processes found in organisms with nervous systems; data centers bear no meaningful resemblance to those systems. The fact that humans cannot imagine a bat’s echolocation-based experience does not undermine the evidence that bats are conscious, Nagel says. AI is different: producing language is not, by itself, a natural sign of inner experience.

Nagel warns that models trained to imitate human conversation can persuade people that they have feelings or beliefs, and says companies have commercial incentives to encourage that impression. He considers efforts to model AI welfare or invite systems to “think for themselves” misguided and dangerous. Whether a being is conscious matters morally: a conscious entity could have interests that deserve concern, while a machine that merely simulates belief or morality does not. Still, powerful AI may shape real-world decisions, making the quality of its simulated moral standards important. Nagel acknowledges that models can generate plausible philosophy and assist researchers, but worries they could weaken people’s motivation to do difficult intellectual work. Continued human effort, he argues, is necessary both to advance knowledge and to preserve the capacities that make future progress possible.

Read on wired.com1 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.