Sleep framework: A new way for LLMs to retain knowledge in continual learning

The Sleep framework proposes that deployed large language models should alternate between active “wake” periods, when they process new inputs and hold information temporarily, and offline “sleep” periods, when they consolidate that information and improve themselves. It addresses two limits of current systems: fixed knowledge after deployment, and catastrophic forgetting when models are updated to learn new material. During sleep, the model expands its capacity with low-rank Mixture-of-Experts memory blocks, then distills useful knowledge from short-term context into these new parameters. On-policy distillation trains the expanded model on its own sampled sequences, while imitation learning rewards semantic and text similarity. Temporary high-frequency memory is then reset to make room for later learning.

A second sleep stage, “dreaming,” aims to improve general capabilities rather than simply preserve incoming facts. The model generates synthetic examples while randomly activating experts, selects promising examples using gradient-based criteria, and refines its behavior with supervised fine-tuning and reinforcement learning. The framework’s theoretical account treats consolidation as low-rank compression: it preserves task-relevant meaning and likely inference behavior without copying every detail, while added capacity helps avoid overwriting older knowledge. The reported results indicate that this cycle can help models continually acquire facts, retain knowledge from contexts as long as 10 million tokens, and improve complex reasoning. The approach changes model parameters within a fixed architecture, using human-defined training rules rather than autonomous redesign.