We Tested Coding Agent Harnesses. Thinking Harder Beat All of Them

A benchmark of four coding-agent workflows tested whether planning skills, specialized reviewers, and orchestration improve software engineering when the underlying model stays the same. Using Opus 5.5 in Claude Code, researchers compared vanilla Claude Code, the Superpowers skills plugin, Compound Engineering, and Claude Code’s built-in /goal and /code-review tools on 100 real tasks from the sudo L7 benchmark. Its expert-written rubrics assess more than whether code works, including verification, practical judgment, persistence, and communication. The central finding was that raising the model’s thinking effort from medium to xhigh produced the largest gain: 11.3 points, versus 2.4 for Superpowers and 3.3 for /goal plus /code-review. Compound Engineering, with 37 skills and extensive agent reviews, cost eight times as much as vanilla Claude Code without a statistically clear improvement.

More thinking particularly improved verification, persistence, and collaboration with the user. In one example, an xhigh-effort agent traced a requested change through a second code path and prevented a fix that would still crash; it scored 100 rather than 14, at a cost of 41 cents rather than 12. Elaborate review can also miss the underlying product need: in another task, 17 agents failed to notice that sending a Zoom passcode in a registration confirmation would give it to unpaid users. The findings favor vanilla Claude Code as a baseline, lightweight add-ons when their modest gains justify the cost, and higher effort when quality matters more than expense. Because the results cover one model and task distribution, teams should benchmark workflows on their own work before adopting costly harnesses.