Opposable is an Android app that runs a 2-billion-parameter multimodal agent on the phone to read the screen via the accessibility service and tap or type in other apps, with cloud agents (Claude, ChatGPT, Codex) optional. The on-device model is GUI-Owl 1.5 (a Qwen3-VL fine-tune) packaged for llama.cpp, quantized to 4 bits: about 1.05 GB for the language model and 445 MB for the vision tower. The vision encoder is a 24-layer ViT (≈0.4B params) at 416×960 input exported to ONNX and compiled for NPUs; on a Galaxy S25 NPU the encoder runs fp16 in ~236 ms (vs ~3.9 s on an M2 CPU). On a Mac M2 with CPU optimizations (cropping status/nav bars, dropping flat-background image tokens, KV caching and grounded early exit) a full step takes ~7.3 s (3.6 s encode, 2.5 s prefill, 1.1 s decode) and scores about 27-28/36 on a graded 36-screen test.
On unseen end-to-end tasks the on-device agent is weaker than cloud models: whole-task success was 4/8 and 3/8 across two runs, but when given subgoals with on-screen verification it succeeded 6/8 in both runs. Limitations include NPU support currently only compiled for Snapdragon (others fall back to GPU/CPU), first-time NPU graph preparation takes ~72 s (0.39 s thereafter), and some inputs still require CPU encoding. The app is free; the screen-model download is ~1.5 GB and shows which encoder your phone picked and per-screen timings.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.