Better prompt caching for GPT-6
Higher cache hit rates and new tools to help persistent agents run faster and cost less.
TL;DR
- GPT-6 enhances prompt caching for persistent agents, allowing them to work longer on complex tasks at reduced costs.
- Improved prompt caching system delivers higher hit rates by default and offers discounts for reused context within a 30-minute window.
- New tools include a Prompt Caching Dashboard for monitoring performance and a diagnostics tool to understand cache misses.
- Developers can optimize caching by choosing what to cache, adjusting reasoning effort without breaking cache, preserving cache as tools change, and prewarming the cache.
- Early adopters have reported significant cost reductions (up to 36%) and improved cache hit rates (from 83% to over 90%).
- GitHub Copilot has already seen over a 50% reduction in prompt tokens requiring fresh processing due to these caching improvements.