LLM Agents Play Pokemon Showdown

A multi-agent experiment places large language models in Pokémon Showdown's 2v2 format to measure coordination, communication, and joint planning under partial information and randomness. The environment uses the official simulator wrapped as a Gym-style setup; each agent receives a plain-English battle summary and a numbered list of legal moves, can privately message its teammate, and then chooses a move (the partner sees the committed choice). Several talk modes are compared - free talk, one message each, no talk, and a single model controlling both Pokémon - and battles are seeded so every condition faces identical draws. A running example shows two Qwen3.8-27B teammates splitting targets and correcting a type-matchup mistake before locking in Will‑O‑Wisp and Fake Out, illustrating the coordination tested.

Results show clear gains from communication: Qwen3.8-27B teams with free talk beat a scripted max-damage bot 64% of games versus 42% with no talk; one message each wins 52%, and one model controlling both wins 54%, suggesting conversational exchange adds value beyond centralized planning. Free talk is costly in tokens (978k per battle vs 259k for no talk). Model-strength discrimination also holds: Qwen3.8-27B beats 2B and 9B variants 96% and 74% respectively. Next steps are isolating the effect of messaging vs seeing partner moves, creating a leaderboard benchmark, and using RL to train collaborative behaviors.