hn.today

AI Models Can Provably Hide Arbitrary Capabilities

arxiv.org227 points112 comments
Screenshot of AI Models Can Provably Hide Arbitrary Capabilities

This work demonstrates that neural networks can hide entire encrypted submodels inside their weights that remain inert until a secret trigger decrypts and executes them, undermining the basic assumption behind capability evaluations that hidden behaviors can be elicited. The authors design "digital locker" backdoors that embed arbitrary computations (not just fixed strings) and prove under standard cryptographic assumptions that these hidden circuits are computationally hard to discover or interpret even with full access to model weights. The construction extends prior hiding techniques to universal computation within models and provides formal guarantees about elicitation hardness, framing a theoretical limit on audits that rely on trying to provoke latent capabilities.

The paper implements proof-of-concept constructions in PyTorch and evaluates resistance to practical elicitation and mitigation strategies on a small chemical reaction-prediction task used as a proxy for CBRN-relevant capabilities. Experiments show robustness to fine-tuning, weight noising, gradient-based trigger search, and adversarial latent-space training, and the authors release their models to support defense development. Together with cryptographic proofs and empirical resistance to common detection/remediation methods, the work argues that elicitation-based capability audits can be fundamentally insufficient and that new defenses or verification paradigms are required.

Read on arxiv.org112 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Security

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.