Don’t be fooled—LLMs don’t reason
Ten years after AlphaGo’s match against Go champion Lee Sedol, today’s AI still isn’t tapping into the machinery that made that win possible.
TL;DR
- AlphaGo's win against Lee Sedol was attributed to its reasoning capabilities, not just computational power, enabling creative moves.
- Modern AI, like LLMs, operates mainly on 'system 1' intuition (fast, associative), lacking 'system 2' deliberation (slow, step-by-step reasoning).
- LLMs' 'chain of thought' process is an iterated 'system 1' prediction, not genuine separate reasoning, and can be concocted after the fact.
- Trustworthy AI for critical applications requires explicit, persistent, and inspectable epistemic states, and a clear separation between knowledge and reasoning.
- A new approach to AI reasoning is needed, inspired by AlphaGo's architecture, which maintains and updates a game tree or similar epistemic state to ensure auditable conclusions.
- This advanced AI would function like the scientific method, accumulating certified knowledge through evidence-based belief revision.