Context Language Models

Context Language Models (CLMs) are language models that natively manage their own context by treating the context as a mutable file that the model can read and update without external constraints. Giving models direct control over a context file lets them learn what to retain, summarize, or discard and naturally extends to multi-agent systems by maintaining per-agent context files. Moving context-management into model behavior enables both in-context and parametric learning of context strategies and allows models to be steered via natural-language instructions that can be optimized through skill-tuning loops.

Zero-shot CLMs constructed from existing models outperform prior context-management strategies across diverse tasks: +11.4% accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, +5% with 59% fewer FLOPs on a 12-hour EdgeBench benchmark, and a 65% larger improvement on a 24-hour multi-repository agent-swarm task at equal compute. Evolving instruction prompts through a skill-optimization loop improves held-out accuracy by up to 35.9 points while reducing compute. An online reinforcement-learning approach boosts Qwen3.5-9B performance on BrowseComp-Plus by 47.6% using 12% fewer FLOPs. A co-designed Suffix Cache Reuse serving strategy further reduces server-side compute by 35% relative to standard SGLang at matched performance.