CATT (Consumer Agent Testing Tool) is a surrogate test host that runs scripted or model-driven agents against MCP servers, MCP apps, websites and HTTP APIs to verify service-side outcomes, collect sealed evidence and compare results across builds. It emulates Muse and Instinct behaviors via a shared connector testing contract (routing, memory, payments, browser/HTTP tasks and scheduling), while recording the documented, unofficial, assumed or unknown basis for any Muse-specific behavior. Supported test lanes include connector-run (tools, account/approval rules, outcomes), connector extensions (routes, memory, custom API clients, browser actions, wallet cards, background jobs and simulated users), scenario scaffolding, gadget registration checks, MCP protocol conformance, embedded MCP app surface runs in Chromium, website accessibility snapshots, and API trials with approval-bound calls. Evidence can be reported, exported, replayed and diffed; playback and diffs require sealed runs.
Installation and operation require Python 3.12+, uv, and Node.js 20+, with Playwright/Chromium for browser lanes; CI exercises Linux (3.12) and macOS (3.14) and three quickstarts demonstrate loopback and hosted sandbox use. Some features depend on macOS Seatbelt sandboxing and are unsupported or blocked on Linux; live payment rails, Vault and Files remain outside the shared contract and a passing test does not certify compatibility with Muse or any hosted assistant. Examples, readiness checks, test adapters and development extensions are included to scaffold connector readiness, API mappings and local testing workflows.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.