HAProxy lead developer Willy Tarreau describes using AI models to find defects in HAProxy and argues that their value depends on pairing model capability with developers’ validation and repair work. A trusted colleague used Anthropic’s Mythos model to uncover complex bugs, but later batches of its reports were difficult to act on: they lacked code excerpts, clear explanations, reproducers, fixes, and even tested commit identifiers. Using Opus to interpret the reports and suggest patches helped the team; after substantial reworking, about one-third of the proposed fixes were merged. In contrast, many unsolicited AI-generated security reports provide no reproducer or patch, leaving maintainers to investigate uncertain claims.
Tarreau criticizes restricted access to powerful models and safeguards that block ordinary defensive work, such as inspecting C code, reproducing bugs, or tracing root causes. He says these barriers fall hardest on open-source maintainers, while people who submit low-quality findings can still gain access and benefit professionally. Models including newer Opus releases and GLM can find real bugs, but refusals and missing context undermine their usefulness. Developers need tools that scan specified code, demonstrate defects with reusable reproducers, propose and test fixes, rule out false positives, and check for related bugs - not tools focused on exploits or intrusion. Safeguards, he argues, should enable this repair workflow rather than treating work on provided source code as an attack.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.