tech
EY says its 'invisible' AI router has helped cut token consumption by up to 60%
The Big Four firm EY has installed a router behind its AI tools to help manage token spend.
TL;DR
- EY deployed an "invisible" router behind its AI tools to manage internal AI spending and direct queries to the most appropriate AI model.
- The router acts as an intermediary, deciding where employee prompts go without changing the user experience.
- This system helps manage rising token costs, a growing concern for companies as AI providers increasingly charge based on usage.
- EY's AI Pulse survey found 82% of senior AI leaders are concerned about token usage.
- The router has helped reduce token consumption by 60% since its implementation in April, combined with training and governance strategies.
- EY has set token budgets for employees based on roles and departments, with an approval process for exceeding allowances.
- 64% of senior leaders surveyed reported their organizations now monitor AI token usage and have budgetary guardrails.
- EY emphasizes focusing AI investment on areas with the deepest impact and measuring AI's value through task speed and employee initiative exploration.