hn.today

Show HN: CATT 0.2.1: Test agents/services against a Muse/Instinct-alike, in CI

github.com7 points2 comments
Screenshot of Show HN: CATT 0.2.1: Test agents/services against a Muse/Instinct-alike, in CI

CATT (Consumer Agent Testing Tool) is a surrogate test host that runs scripted or model-driven agents against MCP servers, MCP apps, websites and HTTP APIs to verify service-side outcomes, collect sealed evidence and compare results across builds. It emulates Muse and Instinct behaviors via a shared connector testing contract (routing, memory, payments, browser/HTTP tasks and scheduling), while recording the documented, unofficial, assumed or unknown basis for any Muse-specific behavior. Supported test lanes include connector-run (tools, account/approval rules, outcomes), connector extensions (routes, memory, custom API clients, browser actions, wallet cards, background jobs and simulated users), scenario scaffolding, gadget registration checks, MCP protocol conformance, embedded MCP app surface runs in Chromium, website accessibility snapshots, and API trials with approval-bound calls. Evidence can be reported, exported, replayed and diffed; playback and diffs require sealed runs.

Installation and operation require Python 3.12+, uv, and Node.js 20+, with Playwright/Chromium for browser lanes; CI exercises Linux (3.12) and macOS (3.14) and three quickstarts demonstrate loopback and hosted sandbox use. Some features depend on macOS Seatbelt sandboxing and are unsupported or blocked on Linux; live payment rails, Vault and Files remain outside the shared contract and a passing test does not certify compatibility with Muse or any hosted assistant. Examples, readiness checks, test adapters and development extensions are included to scaffold connector readiness, API mappings and local testing workflows.

Read on github.com2 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Programming

We have traits at home

We have traits at home

Zig uses two main approaches for polymorphism: runtime interfaces with dynamic dispatch and compile-time duck typing for static polymorphism. These methods offer different tradeoffs in performance and safety, with Zig emphasizing simplicity and compile-time checks over complex abstractions. (axelmagn.com)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.