A single internal neuron in a language model can strongly steer its responses toward a particular persona or behavior. Researchers tested this by adding a fixed amount to one neuron’s activation at every generated token, then asking Qwen3-8B a range of questions. Among its 442,368 neurons, they found switches for speaking German, acting like a pirate, presenting as a child, and adding celebratory toasts to answers. For example, pushing neuron 7,601 in layer 9 made the model respond as a pirate in all 30 tested cases; another neuron produced German replies in all 30. Similar effects included a cat persona and a frightened voice in other models.
To distinguish genuine switches from random disruption, each candidate had to outperform 30 neurons from the same layer, tested with equally strong pushes, and retain fluency on questions beyond those used to find it. Random neurons changed behavior by just 1.4 percentage points on average, while reported switches had to shift it by at least 40 points. Effects depended on push strength: a narrow “sweet spot” could produce fluent, consistent behavior, while stronger pushes often made outputs incoherent. The researchers applied this mapping approach to Qwen3-8B, two Qwen3.5 models, and OLMo-3-7B. The findings show that some neurons act like surprisingly precise behavioral controls, but do not imply the model genuinely feels the emotions or embodies the identities it expresses.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.