tech
Our AI Costs Jumped 230 Percent. I’m Not Setting Token Budgets—Yet.
What a surge in daily credit usage taught me about managing operations on the frontier
TL;DR
- Daily credit usage for AI models surged by almost 2.5 times following the rollout of a new model.
- An initial unexpected credit depletion led to a realization of the new spending reality.
- The company uses multiple AI models, making costs harder to predict due to varying pricing structures.
- Implementing strict, intricate allocation schemes is deemed impractical due to the rapid pace of AI model development.
- The company is currently avoiding imposing immediate token budget limits to facilitate ongoing experimentation.