Chomsky and the Two Cultures of Statistical Learning

The discussion responds to Noam Chomsky’s criticism of statistical language learning: that it can imitate linguistic patterns without explaining them, and that matching data is not genuine scientific understanding. The author agrees that engineering performance alone does not settle scientific questions, and that enormous models can be hard to interpret. But successful predictions and failures, and how learning changes with more data, can still reveal useful properties. Science needs both theories and the accumulation of facts; gathering and modeling observations is not an inferior or novel activity. Nor does a model have to reproduce the exact process people use to speak in order to help explain or interpret language.

The argument distinguishes mathematical models, probabilistic models, and models trained from data: these categories overlap but are not identical. Statistical or probabilistic approaches are not inherently shallow; the ideal gas law, for example, summarizes uncertainty about countless molecules while offering both predictions and insight. Simple rules can also be inaccurate: “I before E except after C” performs worse on the cited corpus than the general “I before E” rule, while a more accurate spelling model would need to capture deeper features such as sounds and word origins. The author argues that language likewise requires models broad enough to combine words, syntax, meaning, context, and discourse. Chomsky’s objections to older word-prediction models do not establish that probabilistic learning is impossible; theoretical results on formal languages do not directly rule out learning natural language. Human beings plainly have an innate capacity to learn language, but its mechanisms - and whether it is language-specific - remain unknown.