hn.today

Show HN: AI SRE Arena, an Open Benchmark for AI SRE Agents on Kubernetes

github.com22 points8 comments
Screenshot of Show HN: AI SRE Arena, an Open Benchmark for AI SRE Agents on Kubernetes

Project Arena is a vendor-neutral benchmark and starter kit for evaluating AI-driven site reliability investigation tools on Kubernetes. It automates a repeatable flow: deploy a disposable cluster (local kind or AWS EKS), install an observability product, inject and verify faults across a selectable suite (21 full scenarios or a 6-scenario smoke suite), capture investigation records, and score final reports with a configurable AI judge. The toolset requires Python 3.10+, kubectl, and Docker/kind for local use; EKS paths add Terraform and the AWS CLI. Deployment supports direct kubectl mutations or a GitOps path via Argo CD, and the suite includes network-policy-aware configurations, multi-architecture image builds, and commands for building, pushing, and running scenarios.

A published benchmark compared Edge Delta’s native AI, Grafana’s native AI, and Anthropic Claude driven through each platform across 21 incidents, with grading by GPT-6-Astra against a rubric. Edge Delta detected/investigated 18/21 (85.7%) and Grafana 12/21 (57.1%). Claude-led investigations scored higher on supported mitigation and implementation readiness (16/21, 76.2%) than the native products (Edge Delta 44.4% supported mitigation; 56.2% readiness). On a controlled 12-incident subset, Edge Delta led root-cause correctness (11/12, 91.7%), Grafana trailed (9/12, 75%), and Claude was 10/12 (83.3%); blast-radius and mitigation metrics showed smaller differences. Scoring categories are detection, root cause, blast radius, supported final mitigation, and implementation readiness; full results, CSV downloads, rubric, and methodology are provided.

Read on github.com8 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Web

Bevy 0.20

Bevy 0.20

Bevy 0.20 introduces faster and more accurate Solari rendering with macOS support, along with numerous UI widget additions and shader language updates. The release also features syntax improvements, event triggers, and quality-of-life enhancements for the Rust-based game engine. (bevy.org)

Agents in the Cloud

Agents in the Cloud

Agents in the Cloud enables developers to run coding agents in isolated cloud environments, freeing them from traditional hardware constraints. It offers a free, open-source platform that supports various models and subscriptions, with easy setup on Mac or servers using providers like Hetzner or DigitalOcean. (agentsinthecloud.com)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.