hn.today

AI agent fixed the bug, but what else did it change?

blog.trailofbits.com43 points6 comments
Screenshot of AI agent fixed the bug, but what else did it change?

Two new plugins, patching-bugs and regression-hunt, let engineers delegate reproducing, fixing, and regression-checking to AI agents while keeping maintainers in control of security assumptions and patch rules. Patching-bugs reproduces reported bugs on live deployments, generates fixes guided by invariant-based repair, and validates them using a three-arm proof (original fails, patched passes, revert fails). Regression-hunt traces diffs to exposed interfaces, reproduces candidate regressions on fresh live contexts, and requires adjudicated, live proof-of-concept evidence before reporting. A review of over 130 merged PRs produced 21 confirmed regressions, and regression-hunt produced 25% fewer false positives than a baseline Codex agent, although stricter evidence requirements left some known regressions classified as borderline.

Design choices emphasize short, opinionated guidance; modular steps; strict input/output schemas that encode claims, revisions, checks, and raw evidence; and live PoC for every claim. Plugins incorporate project-specific THREAT_MODEL.md and PATCHING_GUIDELINES.md so agents follow local definitions of vulnerabilities and patching expectations. Planned work includes formal evaluations of patch correctness and regressions, trace analysis to see what agent steps get skipped, and making plugins more configurable to balance false positives and coverage. The goal is a repeatable, auditable pipeline that scales patching quality as AI accelerates bug discovery.

Read on blog.trailofbits.com6 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Security

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.