SpaceXAI
SpaceXAI builds Grok, an AI chatbot with voice chat, image and video generation, real-time search, and advanced reasoning. Try Grok at grok.com.
TL;DR
- The AI training system, Colossus, was built in 122 days.
- It was expanded to 200,000 H100 GPUs in 92 days.
- The system features 170PB/s aggregate memory bandwidth.
- It has 2.8Tb/s per-server network bandwidth.
- The total storage for training data and checkpoints is 0.5EB.
- The system reached full scale with 200K GPUs by February 2025.
- There is a roadmap to scale to 1 million GPUs.
- AI progress is driven by compute power.
- The company highlights its unprecedented scale and speed in building AI infrastructure.