The 'Godfather of AI' backs a new watchdog plan to track OpenAI and Anthropic's AI risks from the inside
Anthropic and OpenAI said they'd welcome evaluators to track AI risks. A group of watchdogs, backed by pioneers like Geoffrey Hinton, has its answer.
TL;DR
- The AI Evaluator Forum has outlined "minimum conditions" for third-party AI risk watchdogs.
- These conditions are aimed at ensuring "scientific objectivity, transparency, independence, and robust protections against interference" for evaluators.
- The letter was signed by dozens of individuals from the AI ecosystem, including Geoffrey Hinton and Stuart Russell.
- This initiative responds to commitments from Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman to welcome third-party evaluators.
- Evaluators should have unfiltered communication with company boards and the ability to release findings publicly.
- They should be shielded from retaliation and granted access to the same resources as company assessors.
- The forum includes groups like METR and the AI Verification and Evaluation Research Institute (AVERI).
- Microsoft CEO Satya Nadella has expressed support for the idea of embedded evaluators.