Linux-use – agent-first GUI control for Linux

linux-use is an open-source command-line tool for controlling Linux desktop applications through their accessibility interfaces. Instead of interpreting screenshots or clicking pixel coordinates, an agent can inspect the AT-SPI2 accessibility tree, identify widgets by references, and invoke their own actions. The tool provides commands to list accessible applications, read interactive elements, act on widgets, and watch for interface changes. Its design deliberately excludes language-model code: linux-use supplies sensing and control, while an external agent manages the decision-making loop. It also guards against unreliable results by rejecting stale or unverifiable element references, reporting when reads are truncated, and streaming accessibility events so agents can wait for changes rather than repeatedly poll.

The project has verified basic reading, action, typing, and watching in six GTK applications, and has tested Qt 5 menu actions. Chromium and Edge accessibility requires a browser flag; Electron, Java, and Flutter support have not been tested. It was manually tested on only one setup - Ubuntu 22.04.3 with GNOME 42.9 on X11 - and has no automated test suite, so it is explicitly an early-stage tool. Accessibility paths work on Wayland, but synthetic clicking and key input do not. Written in Machin and distributed as a small native binary under the MIT license, it follows an agent-oriented CLI design with JSON output, meaningful exit codes, typed errors, and diagnostic guidance.