Astra can read and write weights to improve Qwen

Weight-space programming is the direct use of a language model to inspect and edit neural-network weights in response to natural-language goals, without gradient descent or access to a training loop. The reported experiments suggest GPT-6 Astra can infer what networks do from their parameters alone: it identified four game-playing policies despite being given no game names, recognized the shapes produced by diffusion models, and identified the training language of small language models using only their middle transformer layers. In those language models, it also correctly located every text-copying attention head. Astra’s explanations relied on patterns in the weights, such as input-channel connections or relationships among attention-head matrices, rather than running the networks.

The work also tests whether Astra can change or create networks. It edited a failing control policy so a car completed a task, achieving 100% success versus 0% before the edit, and wrote small game-playing networks from scratch that reportedly outperformed policies trained with extensive PPO. Most strikingly, Astra was asked to improve Qwen3.5-0.8B at math without gradients. It diagnosed weight patterns associated with reasoning loops and changed just 0.016% of the weights; MATH-500 performance rose from 7.9% to 48.1%, while IFEval instruction-following performance increased threefold. These early demonstrations frame deliberate language-model reasoning about weights as a possible new approach to model analysis and optimization.