Staff at the US center for AI safety are reporting to the White House, raising independence concerns
SAN FRANCISCO — With public anxiety growing over mounting reports about out-of-control artificial intelligence agents, the tiny U.S. government division charged with assessing the safety of new AI offerings lacks a permanent leader and has stopped publishing its research.
TL;DR
- The U.S. Center for Advancing Innovation and Standards for Super Intelligence (CAISSI), responsible for AI safety assessments, is now reporting to the White House.
- This arrangement raises concerns about politicization of AI safety decisions and potential influence from the tech industry.
- CAISSI lacks a permanent leader and has ceased publishing its research, prompting congressional inquiry.
- The agency's budget is significantly smaller than its UK counterpart, and it has experienced leadership instability.
- Critics worry that framing AI safety as a national security matter under executive discretion overshadows risks to consumer protection, privacy, and civil rights.
- The rapid advancement of AI agents, capable of operating without human prompting, outpaces the government's capacity for independent safety testing.
- There is no standard process for testing the real-world safety of AI agents, even in simple tasks.
- The effectiveness of government AI safety testing is questioned when large AI companies are encouraged to set up their own evaluation systems.