Neural Networks Are Already Clean

Commenters discussed a recent claim by Nick Cammarata that modern neural networks are far more interpretable than previously thought and that many neurons and circuits are "legible." Supporters like sebmellen and mariano54 praised the work and its pedagogical tools, while x312 distilled the claim as a rebuttal to polysemanticity: what looked like mixed neurons were methodological artifacts and many neurons are essentially monosemantic. Proponents were upbeat about mechanistic interpretability becoming tractable, and some (wilmichael, baq) celebrated the clarity and interactive presentation.

Many commenters pushed back. bluejay2387 and yurimo called the conclusions overreaching and pointed to prior work (e.g., multimodal gap, superposition, SAEs) suggesting more nuance; NaiveEmpiricist warned emergent hubs in the small unexplained fraction could still matter greatly. antics and others raised alignment implications, arguing that "clean" representations might make dangerous synthesis easier rather than safer. bob1029 emphasized architecture and redundancy as key to readability, tuvix argued for open weights to enable verification, and juanjmanfredi noted skeptics including Yudkowsky. Aesthesia and smokel critiqued the tone as hyped, so opinion mainly divides between optimism about mechanistic progress and caution that methodological limits, emergence, and real-world risks remain unresolved.