The essay challenges the assumption that increasingly capable AI will become conscious, arguing that intelligence and consciousness are distinct: intelligence concerns what a system can do, while consciousness concerns what it feels like to be that system. AI can perform complex tasks without there being any evidence of subjective experience. The author argues that people conflate the two because of anthropocentrism, human exceptionalism and anthropomorphism, and because fluent language makes chatbots seem especially human. Even ordinary terms such as “hallucinate” can encourage the mistaken impression that AI has experiences; “confabulate” better describes systems producing false claims. Rapid technological change and the fantasy of creating godlike beings further amplify expectations of machine consciousness.
The stakes extend beyond whether AI could suffer and deserve moral consideration. Treating systems as conscious when they are not could let companies exploit people’s emotional attachments, misdirect ethical concern and alter how people relate to one another. Conversely, granting rights to machines could make it harder to shut down systems that pose risks. The debate also shapes how humans understand themselves: equating language or problem-solving with experience risks both exaggerating machines and diminishing the significance of biological life. The essay therefore urges caution about claims that consciousness will automatically appear as AI advances, especially at an arbitrary milestone such as artificial general intelligence.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.