Oído implements open-vocabulary English speech recognition entirely on a $5 ESP32-S3 microcontroller (240 MHz dual-core Xtensa LX7, 512 KB SRAM, 8 MB PSRAM, 16 MB flash) without any neural accelerator. It packs a 13M-parameter Conformer-CTC model quantized to int8 weights with per-frame dynamic int8 activations and int8 attention, executing matrix and attention kernels on the chip’s SIMD instructions and using a memory-aware schedule tuned to flash and PSRAM bandwidth. An int4 variant fits in 8.3 MB, a 1.3M-parameter on-chip language model is supported, and a transducer head is available; a two-core schedule produced incorrect silicon results and is disabled. A streaming mode yields partial transcripts while the speaker talks.
Measured on LibriSpeech, on-device arithmetic reaches 3.70%/8.23% WER (test-clean/test-other), within 0.12 percentage points of full-precision performance; with the on-chip LM WERs fall to 3.33%/7.15%, and the transducer head gives 2.99%/6.67%. The system produces 2.3-2.8× fewer errors than the vendor’s recognizer on the same chip and outperforms Whisper tiny and Moonshine tiny in many noise and reverberation conditions. Firmware running on one core processes audio at 1.97× real time (a 4 s utterance transcribes ≈8.7 s after end); streaming final-text latency ranges 4.1-11.1 s for short clips. Code and models are released.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.