OpenAI’s misalignment monitoring system had “a number of novel vulnerabilities,” third-party evaluators found.

From July 21st to August 11th, two employees from METR and Redwood Research conducted a “pilot” to stress test the system, and METR published a high-level overview of their findings, as reported by CNBC. One exploit would have allowed agents to “prevent hundreds of recent actions (including the actions involved in the exploit itself) from being monitored.”