Bug Finding Considered Harmful?

Property-based testing checks broad rules about a system by sampling many possible inputs, complementing approaches such as fuzzing and formal verification. Its purpose is not simply to uncover defects but to help build reliable software. Yet testing researchers work outside the systems they examine: they often understand them less deeply than maintainers and have different incentives. AI tools sharpen this tension by making it possible to generate and validate huge numbers of bug reports. After filing more than 500 reports in a month, the author began to question whether every confirmed defect deserves attention, especially when reports may burden maintainers or delay more useful work.

The central argument is that bug discovery and bug repair are profoundly unequal tasks. A defect can often be demonstrated with a small, convincing example, while fixing it may require substantial effort and difficult choices among possible solutions. Fixing each case locally can also create endless work, whereas thoughtful fixes must account for the system’s broader future. Security flaws and silent failures remain strong reasons for testing after software is built, but ordinary correctness defects should be reported and prioritized selectively rather than treated as obligations to fix all bugs. Testing is most valuable when integrated into development, helping teams build useful, expressive, and performant systems correctly from the outset, instead of only hardening them afterward.