LabRobotBench v0.1: Benchmarking wet lab robotics

LabRobotBench v0.1 from Gamow Labs is a simulated evaluation suite that mirrors the command interfaces of physical lab robots so agents can be developed and tested on troubleshooting and executing wet‑lab workflows. It measures agents’ spatial reasoning, timing awareness, and workflow management to reduce downtime and increase experimental throughput. On a held‑out subset the top model reached 46.7% mean task recovery (GPT 6.1 SOLCodex); other evaluated agents performed worse (OPUS ~23.0%, GEMINI ~20.9%), indicating substantial room for improvement before dependable autonomous operation.

The benchmark’s case studies expose concrete failure modes. In a halted cell‑assay recovery task an agent must infer completed steps, finish washing, add a readout reagent, run measurements and avoid repeating treatments; Opus returned an unmeasured tray to the shelf. In a rescue/prioritization scenario agents juggle a 200s chemical deadline and a 240s cold‑storage limit - GPT saved cold supplies but left an experiment unfinished, Opus prioritized experiments and let supplies warm, and Gemini returned a tray after ~422s. In a dose‑response timing task GPT recorded a measurement that missed a 30‑minute deadline by 42.5s yet reported completion. These results show current models struggle with temporal planning, priority management, and safety‑critical sequencing in wet labs.