SlopCodeBench 10/2026 – Opus 5.5, Astra wins overall, xhigh considered harmful

A benchmarking update for SlopCodeBench reports frontier coding models evaluated across 36 problems and 196 checkpoints per setting, finding strict pass rates clustered tightly between 12.2% and 16.3% and core pass rates between about 65.8% and 74.0%. GPT-6 Astra achieved the highest strict score (32/196, 16.33%) while Opus 5.5 Medium led core success (145/196, 73.98%). The campaign compares Medium and XHigh effort tiers and their costs: XHigh ran 2.11-4.22× more expensive than Medium in paired runs but did not reliably produce more strict passes (Opus: 28→25 strict passes, 4.22× cost; GPT-6 SOL: 30→26, 2.11× cost; Astra: 29→32, 2.91× cost). Only one run per setting was executed, so the analysis emphasizes that higher compute/effort was not a consistent way to buy more strict checkpoint success in these measurements.

Final-program behavior and code hygiene remain serious problems: most final checkpoints still fail many tests (failed test-case counts per campaign ran from ~1,435 to ~2,831) and only zero to three problems ended with strictly passing final checkpoints. The benchmark intentionally counts leftover scratch and generated files as slop; a scanner flags slop patterns and repeated code, producing SLOC-weighted coverage metrics (Opus XHigh lowest flagged share at 77.75%, Opus Medium highest at 94.25% before filtering). The results show frontier models produce brittle, messy revisions and expose a clear cost-versus-quality tradeoff rather than a simple payoff for higher effort.