Learning Without Backpopagation

The piece explains using evolutionary, gradient-free methods to train neural controllers for embodied agents instead of backpropagation. Populations of simulated “creatures” are ranked by a fitness function - initially total food eaten averaged across four worlds, then extended with movement penalties and a base metabolic cost (α = 0.0016, β = 0.002 per step) - so selection must balance exploration and energy use. Each creature’s brain is a small neural net with nine inputs (bearing and distance to nearest food and creature represented as sine/cos, plus relative mass), two hidden layers of 16 tanh units, 466 parameters, and two outputs (turn, speed). New generations are produced by cloning top performers, mutating weights with Gaussian noise (σ-scale 0.05), and injecting random genomes (12 of 128), while weight-averaging (breeding) is avoided because it often washes out distinct lineages. The write-up shows that population selection approximates a gradient via the log-derivative identity, so evolution is a noisy estimator of ∇θE[F(θ+σε)].

Results emphasize that reward shaping and evolutionary mechanics determine behavior more than model size. Adding movement costs without a base metabolism suppresses exploratory dashing; with the right balance creatures adopt pause-and-dash tactics and moderate speeds. Introducing predation (larger agents can eat smaller ones, and eaten agents are excluded from the next generation) yields pursuit and evasion strategies and shifts incentives - evolving larger sensing increases food intake but shortens lives. Overall, evolution produces effective, interpretable behaviors but requires careful fitness design, trades diversity for rapid convergence, and can be brittle when combining disparate solutions.