tech
OpenAI and Broadcom unveil LLM-optimized inference chip
OpenAI and Broadcom (NASDAQ: AVGO) today unveiled Jalapeño, OpenAI’s first Intelligence Processor: an accelerator architected around OpenAI’s vision for the future of LLM inference, and the first AI accelerator in a multi-generation compute platform the companies are building together to make advanced AI faster, more reliable, and more accessible to more people.
TL;DR
- OpenAI and Broadcom have unveiled Jalapeño, OpenAI's first Intelligence Processor designed for LLM inference.
- Jalapeño is part of a multi-generation compute platform aimed at making advanced AI faster, more reliable, and accessible.
- The chip was designed from scratch based on OpenAI's understanding of LLM fundamentals and future roadmap.
- Early testing suggests Jalapeño will offer substantial performance per watt improvements over current technologies.
- The architecture is optimized to reduce data movement and balance compute, memory, and networking resources.
- This collaboration represents a commitment to scaling the physical infrastructure for AI over the next decade.
- Jalapeño is designed for both current and future LLMs across the industry, suitable for interactive LLM products at scale.
- OpenAI is designing more of its infrastructure stack, from chip architecture to product experience, to optimize for efficiency and access.
- The development cycle for Jalapeño, from design to manufacturing, took nine months, leveraging AI models to accelerate the process.
- The goal is to democratize AI by making advanced models dependable and affordable for a wider range of users.