Canto: A speech model built for the real world

Canto is a real-time speech recognition model built to handle everyday dictation recorded on laptops, earbuds, and headsets with background noise and interruptions. Wispr evaluated Canto on a 10-hour set of Wispr Flow dictations from over 2,300 opt-in users and found it achieved the lowest word error rate (WER) versus models from Google, OpenAI, AssemblyAI, and Deepgram. A separate 3-hour challenge set (traffic, music, competing speech, wind, low volume, whispers, and very short utterances) placed Canto second overall behind a large, non-real-time multimodal model (Gemini 3.1 Pro) but first among practical low-latency models; Gemini led on noisy audio, while Canto tied for best on low-volume and short dictations. On public English benchmarks, Canto tied for best on LibriSpeech and remained competitive on FLEURS and Common Voice.

Canto’s training pipeline starts from a model pretrained on millions of hours of speech and text, then applies supervised fine-tuning followed by reinforcement learning with Group Relative Policy Optimization (GRPO), which compares multiple sampled transcripts (rollouts) per audio and updates toward higher-reward outputs. Infrastructure to generate and score rollouts at scale enables targeted improvements - contextual recognition, personalization, diarization, and handling failure modes. User corrections (opt-in) are converted into cleaner training signals by detecting likely ASR edits via forced-alignment confidence and edit shape, grafting only the probable correction into the reference. Experiments show RL makes the model more responsive to supplied vocabulary dictionaries but can also increase erroneous adoption of phonetically similar distractors, motivating training strategies to balance context use and discrimination.