The systems that no one will test

In 2020 I discovered a simple but severe vulnerability in a Brazilian federal system that would have allowed access to records for over 200 million people - essentially full identity portfolios (IDs, CPF, passports, birthplaces, parents’ names, driver’s licences, addresses, phones and even witness-protection status). I reported it to the responsible agency, avoided exploiting any data, and they patched the flaw quickly once they understood it. That episode stays with me because the flaw required attention to detail, not exotic expertise, and because it exposed how many critical systems are never exposed to adversarial testing or routine pentesting.

Since then, machine learning has scaled aggressively: mid-training tricks, RL for long-horizon tasks, RLVR and synthetic curricula let labs and third parties build automated environments that models can probe and optimise against in self-reinforcing loops. Those environments can reproduce real-world setups and find classic bugs (SSRF-like vectors among them); many teams run evaluations or even training with reduced safeguards, and training makes bypasses easier. The combination of massively parallel agents, opaque development choices and geopolitical fragmentation means governments - especially in emerging economies with limited resources - risk automated, repeated exploitation with no human disclosure. The problem demands planetary-scale thinking and coordinated solutions beyond polarized technophilia/technophobia, despite geopolitical obstacles.