Argues that as frontier models become more capable, their behavior cannot be traced to specific code paths or training inputs the way traditional software can, creating an insider-risk problem when those models access sensitive data or take mission-critical actions. Responsibility for outcomes cannot be outsourced to providers; instead the supply of intelligence must be separated from the authority to act. Mechanistic explainability is limited, chain-of-thought transparency is necessary but not sufficient, and relying on nested model orchestrations risks creating stacks of opaque systems that evade accountability.
Prescribes an engineering-first containment and governance approach that treats models like privileged enterprise insiders: apply identity, least privilege, logging and containment boundaries; place controls outside models; and externalize safeguards and evidence. Key design principles include model diversity (no single model as sole authority), full tamper-proof human-readable observability, continuous verifiability (including attacks and failures), independent controls and auditability, assume-compromised containment with an emergency stop, and timely incident disclosure including implementation details. The ultimate claim is that the most trustworthy super-intelligence deployments will be those that force operators to trust models the least by making their actions observable, verifiable, and controllable.
Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.