hn.today

Spending 10 months building the most human-like chess bot

mbuffett.com3 points1 comments
Screenshot of Spending 10 months building the most human-like chess bot

A developer spent ten months building Chessbook-1, a chess bot engineered to behave like a human opponent at any Lichess rating and time control. It was trained on games through July 2026 and evaluated on two September 2026 Lichess datasets: 30,000 random rated positions (all time controls) and 28,000 rated blitz positions (4,000 per rating band). Each model received the position, player ratings and any supported clock/history features, then produced a probability distribution over legal moves; performance was measured by top-1 accuracy and the probability placed on the actual human move. Chessbook-1 reads clocks and move history, Maia-3 reads history but not clocks, and Allie 2.0 reads both but is much larger.

Chessbook-1 predicts human moves strongly, matches Maia-3 overall and beats it where Maia’s training is out-of-distribution (bullet and time scrambles), and effectively ties Allie 2.0 on blitz (pooled top-1: Allie 57.5%, Maia 57.3%, Chessbook-1 57.3%; probability on human ~27-28%). Crucially, Chessbook-1 is the only model calibrated to play at the intended Elo across bands: its average centipawn loss and blunder-rate differences versus humans stay near zero (within about ±1 cp and ±0.6 percentage points), while Maia and Allie over-perform at low Elo and underperform at high Elo. Chessbook-1 also predicts human think time and deliberately errs in time scrambles like humans.

Read on mbuffett.com1 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

Evals for 3D CAD

Evals for 3D CAD

MUSE is a new benchmark for evaluating text-to-CAD models on complex assembly tasks, focusing on functionality, manufacturability, and assemblability. Experiments reveal current models struggle with producing engineering-ready designs, highlighting the need for improved AI tools in industrial product design. (emergentmind.com)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.