RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

RRSI (Regularized Recursive Self-Improvement) is a method and codebase for iteratively improving an LLM agent’s harness - the prompts, control flow, tools, memory and other surrounding infrastructure - while avoiding overfitting to a fixed training split. Instead of constraining what edits are allowed, RRSI regularizes how the search moves through an open edit space. On the proposal side it uses an annealed edit budget, conditions proposals on full edit history so refuted hypotheses aren’t reissued, and redirects stalled runs toward untested components. On the selection side a critic filters candidates for suite-specific leakage, a noise-adjusted floor and within-band rules prevent spurious gains, a cost rule requires added inference to be justified by measured improvement, and stagnant components are pruned. Every candidate lives in its own git worktree and the edit history records per-edit hypotheses, scores, costs and verdicts for auditability.

One loop drives three domain instances - coding (Terminal-Bench 2.1), a document-work agent (Harvey LAB), and an engineering-design agent (EngDesign) - with the proposer, analyst, critic and frozen policy typically using Claude Opus 4.8 (and experiments with Gemini 3.5 Flash). Each round drafts two candidates, screens them, evaluates both on the evolve set, and fast-forwards the incumbent branch to the winner. Measured against the unevolved harness H_0, RRSI produced consistent gains: Terminal-Bench +6.0 (74.2→80.2), SWE-bench OOD +1.8, Harvey LAB evolve +1.1 and held-out +2.3, JobBench +4.7, GDPval +3.5, APEX +3.7, EngDesign +4.9 and Frontier-Eng OOD +4.3. The repository includes code, evaluation tooling, and instance-specific setup and benchmarks.