tech
These startups are chasing the next big thing in LLMs
Meet the new kids nipping at the heels of the AI giants.
TL;DR
- Transformers, the core technology behind current LLMs, are computationally expensive and have limitations in processing long sequences of data and maintaining large context windows.
- Startups are developing new methods to address these limitations, including sparse attention, power retention, liquid neural networks, diffusion models for text generation, and state space models.
- These innovations aim to create faster, more efficient, and potentially more capable LLMs, challenging the current technological paradigm.
- Companies like Subquadratic, Manifest AI, Liquid AI, Inception, and Pathway are pioneering these new approaches.
- The ultimate goal is to create AI systems that are not only more cost-effective but also capable of more advanced and novel forms of reasoning.