Beating benchmarks by weight-space optimizing decision models with DSPy

dspy-wsp is a prototype DSPy optimizer that lets a powerful “writer” model inspect and directly edit a smaller model’s weights, without gradients or conventional training. Its strongest reported results are on decision models: across five open models ranging from 0.6B to 9B parameters, GPT-6-Astra made edits using up to 25 forward passes, at a cost of roughly $5-10 per run. On 400 held-out Typed Decisions cases, every model improved. WeightSpace alone added 1.9 to 16.1 accuracy points; combined with ReAnchor, which refits decision thresholds, gains ranged from 6.5 to 20 points. For example, the 600M d1-omni rose from 50.9% accuracy to 66.9% with WeightSpace and 69.1% with both optimizers; the 2B d1 model reached 77.9%, up from 60.6%.

Each optimization run gives the writer model successive turns to explain an intended change, inspect or edit weights with sandboxed Python, or test the program on practice examples. A fixed forward-pass budget limits exploration, after which the best edit becomes a patch. The tool accepts DSPy modules and metrics, and the edited model runs locally through MLX on Apple silicon or PyTorch on Linux; patches or complete checkpoints can be saved. The prototype also supports vLLM for text-generating models, though those runs can be slow and costly. The approach is promising but early: testing has covered few environments and model families, and results on text-generation tasks may be less compelling.