Anthropic update on independent evaluators
Last month, we committed to stronger independent, third-party assessment of our models and safety practices. We also signed the White House Accord on Super Intelligence, in which companies pledged to partner with independent external auditors or evaluators. Today, we’re announcing that we’ve started to give employee-like access to a team from METR, a nonprofit that evaluates frontier AI models.
TL;DR
- Anthropic is providing METR, a nonprofit, with employee-like access to evaluate its frontier AI models and safety practices.
- This follows a commitment to stronger independent assessment and signing the White House Accord on Super Intelligence.
- Evaluators need expertise, independence, sufficient access, and freedom to publish their findings.
- Anthropic is also engaging with SecureBio and Accenture, developing three tiers of evaluator access.
- The goal is to improve public accountability and close the knowledge gap between AI companies and external evaluators.