LLMs Play Zachtronics' TIS-100

TIS-100 BENCH tests whether large language models can solve and optimize programs in Zachtronics’ programming puzzle game, where tiny processors exchange values and solutions are judged by cycles, nodes, and instructions. Sixteen models attempted all 47 main-game puzzles and eight community tournament puzzles, with difficulty-based time limits and access to the manual, test inputs, and diagnostic feedback - but not community solutions or exact records. A result counted only if it improved the community’s tradeoff frontier and passed 100,000 tests on an independent simulator. The models produced 133 new frontier points across 18 puzzles; 18 points beat existing records outright, while the rest offered new combinations of speed, code size, and processor count.

Opus 5.5 and GPT-6 Astra led, each solving all 55 puzzles and finding improvements on 11. Opus produced more new points, while Astra scored slightly closer to community records; the difference was not statistically clear. Optimization proved distinct from simply solving puzzles, and the strongest results clustered on hard puzzles, with none on easy ones. Models were consistently better at matching records for processor count than for instruction count or speed, making cycle optimization their main weakness. The benchmark also shows that more spending or more tool use does not reliably yield better results. A detailed example describes Opus cutting a tournament puzzle’s cycle count by 29% through a two-speed search, at the cost of extra nodes and instructions.