I Deleted 168,000 Lines of Comments. My AI Agent Didn't Notice

An engineer removed about 168,000 lines of comments - roughly a third of the repository - and ran the same AI agent (Claude Code with Opus 5.5 at high effort) against two real tasks on both the commented and comment-free codebases. Each branch was compiled and verified identical aside from comments, and the agent performed three runs per branch per task. Changes were validated by linting, type checks, existing tests and, for the larger task, a hidden post-run test; a separate blind AI judge scored each result against a predefined checklist. All six runs for each task passed the checks. On the small task (tile all open windows) average scores were 18.3 with comments versus 19.0 without, and token usage dropped from 2.18M to 1.71M (about 22% fewer tokens). On the bigger task (keep a window on top) average scores were 21.7 with comments versus 22.0 without, and token usage was essentially unchanged (~9.06M vs 9.12M).

The clear finding is that comments did not meaningfully affect the agent’s ability to understand or implement features: they neither improved nor degraded result quality, so comments should be kept for human readers rather than for agent performance. The experiment also demonstrates agent nondeterminism - single runs can be misleading - so multiple trials are necessary. Results are limited to one model, two tasks and three runs per branch, enough to exclude large effects but not small ones. The author notes that removing documentation files had an observable effect, to be examined next.