hn.today

Chat-based Large Language Models replicate the mechanisms of a psychic's con

softwarecrisis.dev147 points227 comments
Screenshot of Chat-based Large Language Models replicate the mechanisms of a psychic's con

The piece argues that chat-based large language models create an illusion of intelligence by reproducing the same social and rhetorical techniques used in psychic cold readings. LLMs are framed as statistical token predictors, not thinking machines, and the feeling that they “understand” stems from users projecting meaning onto generically phrased outputs. Rather than a new form of mind, the apparent reasoning is explained as an “intelligence illusion”: a psychological effect produced by confident, tailored-sounding responses that prompt users to subjectively validate and remember hits while dismissing misses. Many proposed LLM use cases are therefore characterized as bordering on pseudoscience or outright grift.

The mechanism is described in concrete stages that mirror a psychic con: selection of a receptive audience, priming and scene-setting, narrowing to likely targets, testing reactions, then entering a loop of subjective validation. Models emit “validation statements” - generalized but personally resonant lines (Forer/Barnum-style affirmations, vanishing negatives, rainbow ruses), statistical and demographic guesses, shotgun lists, and unverifiable predictions - delivered with assured tone to maximize perceived accuracy. The combination of demographic inference, generic specificity, and confident framing produces a powerful sense of insight while remaining a statistical trick. The takeaway is a warning to treat apparent LLM reasoning skeptically and recognize the social psychology behind the effect.

Read on softwarecrisis.dev227 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.