Cal Newport argues that Anthropic’s ambitions to guide humanity through a possible transition to superintelligence sit uneasily with its role as a publicly traded company. CEO Dario Amodei and colleagues frame advanced AI as a potential route to utopia or catastrophe, and see winning the race to build it as essential. Newport says this existential outlook can encourage speed at the expense of caution: Anthropic is releasing models rapidly, has discussed letting AI systems improve themselves, and has eased some safety requirements amid competitive pressure. Although the company invests in research that documents risks in its own products, Newport contends that its drive to stay ahead may still produce poorly understood systems and present-day harms.
Newport also warns that Anthropic’s corporate structure limits investors’ ability to restrain its choices. Its public-benefit status allows it to defend decisions as serving a broader mission, while a Long-Term Benefit Trust controls four of seven board seats and the founders retain majority voting control. In his view, a company pursuing costly, speculative efforts to reach superintelligence may be financially unsustainable if its predictions prove wrong; if they prove right, consequential decisions could rest with a small, unaccountable elite. He argues that AI safety and public policy should be pursued through democratic institutions, research, advocacy, and government - not folded into a company’s business strategy. The public, Congress, and research communities should demand greater oversight.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.