AI Circuit Breaker is a drop-in reverse proxy for OpenAI-compatible agent traffic that prevents runaway agents from draining API budgets by detecting repeated tool calls and enforcing hard spend limits. Traffic is routed through a volatile-memory inspection engine that fingerprints prompts (SHA-256) and uses a sliding-window/deque to spot repeating requests; when a loop or cap breach is detected the proxy blocks the request with a structured JSON error (e.g., type: circuit_breaker_loop_detected) instead of forwarding payloads upstream. Integration is simple: change the SDK base_url (examples provided for Python, TypeScript and curl) while keeping existing client code. The gateway operates over HTTP/2 with SSE support, keeps zero payload persistence (no disk writes), and exposes traces and a live simulation for testing.
Detection and enforcement are optimized for low latency and fine-grained accounting: RAM-hash checks are benchmarked under 15 ms and overall overhead is advertised below 35 ms, while hourly and daily monetary ceilings (illustrative examples $2/hour, $10/day) provide backpressure and 429 responses when exceeded. An illustrative failure autopsy shows a SQL-retry loop repeating 120/min being halted on the third repeat, modeled to avoid ~$680 exposure. Built in Python and FastAPI by Roan de Jager, the project publishes docs, a GitHub repo, and a demo gateway for evaluation.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.