tech
From the Vatican stage, Anthropic’s Chris Olah says AI cannot be steered by AI labs alone
Sitting alongside Pope Leo XIV at the launch of Magnifica humanitas, the company’s interpretability lead conceded that frontier-lab incentives can pull researchers away from doing the right thing.
TL;DR
- Anthropic co-founder Christopher Olah stated at the Vatican that frontier AI development should not be exclusively managed by AI labs.
- He cited internal incentives and constraints within AI labs that can conflict with ethical considerations and societal interests.
- Olah emphasized the necessity of outside scrutiny from religious leaders, governments, and civil society.
- He warned of a real possibility for AI to displace human work on a large scale, calling support for displaced individuals a moral imperative.
- Anthropic's engagement with the Vatican marks a significant repositioning for the company, paralleling Pope Leo XIII's 1891 encyclical on industrial capital.
- The call for outside oversight comes after Anthropic's previous conflicts with the US government, including being ejected from classified AI work and having a model blocked by the Trump administration.
- Olah acknowledged that companies like Anthropic operate under strong commercial, geopolitical, and personal pressures that can be at odds with broader societal interests.