Weight-space programming lets a language model directly write and revise a neural network’s weights. To test whether this is a new kind of training, Claude Fable 5.2 was asked to build a fixed CNN for MNIST from scratch, setting all 5,994 weights by hand. It could use training loss, accuracy, and a confusion matrix over as many as 50 attempts, but could not fit parameters, search, use PyTorch, or see the test set. Across three runs, test accuracy reached 88.3-91.3%, with training loss still improving at the limit. The strongest result was close to what Adam achieved after 50 gradient steps.
The surprising finding is that Claude’s method resembles pre-deep-learning computer vision more than modern neural-network training. It began with formula-based edge filters, hand-drawn digit templates, and log-odds scores akin to naive Bayes. It then added detectors for loops and corners, tested features separately by repurposing the confusion matrix, and gradually built a classifier from measured feature responses. This mirrors the older “fixed feature extractor plus trainable classifier” approach, where performance depended on human-designed features. The experiment suggests that weight-space programming can reconstruct such a pipeline through repeated analysis and hand-crafted changes, rather than learning representations through gradient descent. The same historical pattern is also identified in the Go-playing network built in an earlier weight-writing experiment.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.