Open Letter: Minimum Conditions for Embedding Evaluators

A coalition led by the AI Evaluator Forum issued a public letter demanding that all frontier AI companies embed independent third-party evaluators to assess rapidly escalating AI capabilities, harms, and company practices (training, deployment, oversight, safeguards). The letter argues embedded evaluators must be scientifically objective and capable of investigating systems and real-world incidents, and it calls for multiple evaluators with diverse technical expertise to cover priority risk areas. It stresses that embedded evaluation complements broader external oversight and public transparency rather than replacing them.

The letter specifies minimum conditions for credibility: evaluators must be meaningfully independent (not owned or governed by the company, free of contingent payments or significant commercial ties), maintain full editorial control, and disclose conflicts of interest; companies must grant evaluators access equivalent to privileged senior staff (systems, data, tools, physical spaces, and candid one-on-one communication) with narrow exceptions to protect third-party sensitive data. Evaluators should publish methods and findings, subject only to time-limited, narrowly scoped redactions for IP, customer privacy, security, and public safety. Protections against retaliation - including legal threats - and funding assurances are required. The statement references the AEF-1 standard as an example and is backed by 100+ signatories spanning academics, industry leaders, and governance experts.