Google's Gemma 4 AI models get 3x speed boost by predicting future tokens
Up to 3x the speed with no loss of quality—is it too good to be true?
TL;DR
- Google's Gemma 4 open models now feature experimental Multi-Token Prediction (MTP) drafters.
- MTP uses speculative decoding to predict future tokens, speeding up AI generation.
- These models are built on the same technology as Google's Gemini AI but are optimized for local execution.
- The new models are licensed under Apache 2.0, offering more permissiveness.
- MTP can make Gemma models up to three times faster, with specific gains noted on Pixel phones and Apple's M4 silicon.
- The core Gemma model verifies all draft tokens, ensuring no quality degradation.
- Faster transformers are available via MLX, VLLM, SGLang, and Ollama frameworks.