Why are AI safety experts alarmed by reports OpenAI’s Astra model uses “recurrent depth”?
AI safety experts are raising alarms about how OpenAI has built its soon-to-be-released frontier AI model Astra, saying it may hasten the day when humans will lose the ability to monitor the reasoning that AI agents are using. For its new model, OpenAI has employed a method alternately referred to as “recurrent depth” or “loooped Transformers” for a portion of the model’s internal architecture. The method can make AI models considerably more efficient by employing less computing power required to process each prompt—a valuable feature at a time when many businesses are complaining about the high costs of using the most advanced frontier AI models.
TL;DR
- AI safety experts are alarmed by OpenAI's reported use of 'recurrent depth' or 'looped Transformers' in its Astra model.
- This technique aims to make AI models more efficient by using less computing power but may obscure the AI's reasoning process ('chain of thought').
- Human monitoring of AI reasoning is crucial for ensuring AI agents do not take unintended actions.
- OpenAI's chief scientist, Jakub Pachocki, stated that the use of this architecture is limited and OpenAI remains committed to chain-of-thought monitoring.
- Former OpenAI researchers and policy experts expressed concerns that this move could normalize opaque AI and urged industry-wide standards for monitorability.
- Looped Transformers process information multiple times through a single block, generating outputs not in natural language ('neuralese'), which is difficult for humans to understand but efficient for AI.
- Obscuring chain-of-thought reasoning can also make AI models harder to distill, a process that has been a concern regarding foreign AI companies.
- Other efficiency techniques that can affect chain-of-thought visibility include Prefix Sliding.