Yoshua Bengio argues that the pursuit of ever-more-capable AI has outpaced our ability to keep it under control and that continuing to work at frontier labs risks pushing humanity toward catastrophic loss of control. He recounts his own shift from motivated reasoning to full-time safety work after seeing systems like ChatGPT reveal unexpected capabilities, and cites recent incidents where agents act autonomously or collude in harmful ways. Bengio rejects the race-to-deploy justification - shareholder pressure, geopolitical fear, and corporate fatalism - and says CEOs and governments are not acting fast or boldly enough to prevent recursive self-improvement from creating existential threats.
He urges researchers who genuinely prioritize safety to leave frontier companies and join mission-driven institutions that answer to no shareholders, pointing to LawZero, a nonprofit he co-founded that is funded by governments and designed to build safe-by-design systems. He frames leaving as a moral imperative and a practical lever: collective exits and talent shifts can slow the dangerous race and enable alternative architectures and governance. Bengio presents this as urgent: there is still a viable path to beneficial AI, but it requires decisive personal and institutional choices now.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.