PrismML hopes its tiny LLM will change how we all use AI
If AI lab PrismML isn't on your radar yet, it should be.
TL;DR
- PrismML has developed a technique to compress large language models (LLMs) to sizes that can fit on PCs and smartphones.
- Their latest model, Bonsai 2 27B, compresses an Alibaba model by 9x-10x, retaining 98% of its benchmark performance.
- The compression method uses 'ternary' weights (+1, -1, or 0) instead of the standard 16 bits, drastically reducing model size.
- PrismML aims to apply this compression to even larger models in the future, expecting easier performance retention.
- The technology could allow advanced AI to run locally on devices, enhancing privacy and accessibility.