“We were looking at it wrong”: The explosive claim that’s upended the AI world
An exclusive interview with Nick Cammarata, the researcher whose 26,000-word viral blog post has AI safety folks celebrating.
TL;DR
- Nick Cammarata, a researcher at Goodfire, has published a lengthy blog post claiming neural networks may be far easier to understand than previously believed.
- This claim has sparked both excitement and skepticism, with some hailing it as a major breakthrough and others deeming it unsubstantiated.
- Cammarata's previous work in mechanistic interpretability involved probing individual neurons, but the phenomenon of superposition suggested complex, tangled structures.
- He now suggests that a different approach, focusing on what concepts neurons might have in common rather than their apparent unrelatedness, reveals an underlying logic.
- While Cammarata believes full understanding of LLMs is possible with enough time and effort, other researchers like Chris Potts, Jacob Hilton, and Neel Nanda express doubt about the significance and conclusiveness of his findings.
- Cammarata aims to provide optimism to the AI safety community, suggesting that understanding and controlling AI is not an impossible task.