hn.today

A visual explanation of reinforcement learning

aatishb.com166 points9 comments
Screenshot of A visual explanation of reinforcement learning

A visual demonstration uses a simple digital creature, the “spark,” to show how reinforcement learning can produce behavior. The spark has two light-sensing neurons, two flipper-controlling neurons, and three adjustable parameters: the strengths of straight and crossed neural connections, plus a drive that sets its movement without light. The reader can adjust these values to make the spark seek a light, while a score measures how much light it encounters. This slider exercise illustrates learning as trial and error: change the brain’s wiring, observe the result, and use feedback to guide the next change.

The spark then learns on its own through repeated random adjustments. After each trial, its wiring shifts toward the adjustment if its reward - average brightness along its path - was better than usual, and away from it if the reward was worse. Over roughly 2,000 trials, it progresses from spinning or meandering to reliably following the light. Many sparks starting with different wiring eventually converge on successful behavior, though their learning paths include detours and setbacks. Changing the reward changes what they learn: avoiding light can produce fleeing or freezing, while reward for fast approach produces charging. The demonstration connects this simplified process to biological learning, including songbirds refining songs through dopamine-linked feedback. Real brains are vastly more complex, but the central idea is that reward and punishment help learning navigate a large space of possible neural configurations.

Read on aatishb.com9 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

ChatGPT Wallet

ChatGPT Wallet

ChatGPT Wallet lets ChatGPT agents make purchases using tokenized card details, with users reviewing each purchase and approving it through a security check. Launching first for US ChatGPT Pro subscribers, it is built with major payment networks and preserves eligible card rewards and protections. (openai.com)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.