RLVR might be disproportionately bad at science
The verification loop for theories can be on the order of decades and centuries, and even then, what we know today as the better theory can often actually make worse predictions.
TL;DR
- Recognizing scientific progress is a complex and often slow process, taking decades or centuries.
- Historical scientific theories, like Copernicus's model, sometimes made worse predictions than older models before later advancements provided context.
- AI's strength in domains with tight verification loops (coding, math) may not translate to scientific discovery where verification can be lengthy and ambiguous.
- Human scientific progress has relied on judgment, heuristics, and even seemingly unreasonable obstinacy in pursuing research agendas.
- Conceptual breakthroughs are hard to verify ex ante, and their value is often recognized much later when they prove more productive than alternatives.
- The society of AI scientists may need to support individual AI instances with idiosyncratic biases to keep dormant research agendas alive.