GVS5H tests whether several fresh instances of the same language model can improve hard coding performance by coordinating through a shared filesystem. The approach requires no training: instances divide a problem, exchange plans and notes, and update a shared solution ledger. Across 11 open- and closed-weight models on the 100 latest hard LiveCodeBench problems, orchestration produced gains of up to 25.6 percentage points on pinned backends, but did not help every model. The authors attribute improvements mainly to problem decomposition and persistent context.
The headline result is that locally served Qwen3.8-27B rises from 66.8% to 92.4% pass@1 with orchestration, approaching Claude Fable 5’s 90.4% score; this offers a route to similar performance using self-hostable weights. Orchestrated Qwen3.8 Flash Next scores 93.0% at 9% of Fable 5’s cost, offering a lower-cost alternative through a hosted model. The repository includes code, run data, plots, and instructions for reproducing evaluations with an API key and the uv tool. Its claims are specific to this benchmark and setup, and the results indicate that adding coordinated passes is not a reliable improvement for all models.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.