tech
A startup claims it broke through a bottleneck that’s holding back LLMs
Subquadratic has now shared more details about its new model. But some are still skeptical.
TL;DR
- Subquadratic claims its new LLM, SubQ, is faster, cheaper, and more energy-efficient than existing models.
- SubQ can process up to 12 times more text at once, enabling complex data-heavy tasks.
- The model utilizes sparse attention, moving away from the dense attention mechanism in traditional transformers.
- Independent tests by Appen show SubQ to be significantly faster than previous sparse-attention techniques and competitive with top models in coding tasks.
- Subquadratic claims SubQ can handle tasks at a fraction of the cost of current leading LLMs.
- The model boasts a context window up to 12 million tokens, substantially larger than most current top models.
- Despite positive test results, some skepticism remains due to the model's partial reliance on existing weights from another model.