tech
OpenAI and Broadcom Announce Chip Designed for LLM Inference at Scale
The silicon race is heating up amid the struggle to keep up with demand.
TL;DR
- OpenAI and Broadcom have announced a new chip named Jalapeño, tailored for large language model (LLM) inference.
- The Jalapeño chip is an ASIC designed specifically for LLM inference, incorporating insights from OpenAI's researchers and future model roadmaps.
- Early testing suggests Jalapeño will deliver significantly better performance per watt than current state-of-the-art solutions.
- This collaboration aims to reduce OpenAI's dependence on external chip suppliers like Nvidia and enhance overall efficiency.
- The custom silicon is also a response to the global compute crunch and high demand for data center capacity.
- Both companies expect Jalapeño chips to be deployed in data centers by the end of the year.
- The development and production of the chip took nine months.