DeepMind Paper: Dream-RSI: Recursive Self-Improvement Through Evolving Worlds

Dream-RSI describes a framework for scalable recursive self-improvement in autonomous agents by separating exploration orchestration from the underlying coding agent. It confronts the trade-off that fixed exploration strategies fail as search spaces grow, while online policy optimization must navigate vast meta-search spaces with delayed, expensive long-horizon feedback. The key idea is to treat accumulated discovery history as a replay simulator built from realized discovery trees and to perform "dreaming" - off-policy evaluation and refinement - inside that simulator. A lightweight orchestration layer makes exploration explicit and programmable, yielding immediate, low-cost feedback for evaluating and improving exploration policies without repetitive online rollouts.

Refined exploration policies are then redeployed online to generate new discoveries, which expand the simulator pool in a self-reinforcing loop. Empirical work across algorithm engineering, mathematical optimization, and GPU kernel engineering demonstrates competitive or improved discovery quality while substantially reducing discovery cost. By enabling cheap simulated dreaming and iterative redeployment, the framework accelerates policy tuning, reduces the burden of long-horizon evaluations, and produces more adaptive exploration strategies as search spaces evolve.