Can tech companies learn to love cheaper AI models?

tech

Can tech companies learn to love cheaper AI models?

If those same AI workloads can be handled by cheaper models without affecting quality, it would mean a massive shift in the economics of AI.

TL;DR

The AI industry's reliance on larger, more powerful models is being questioned due to increasing costs.
Companies are starting to consider smaller and cheaper AI models as a viable alternative.
Brian Armstrong predicts that 80% of AI workloads will shift to 99% cheaper models within 12-18 months.
This shift could dramatically change the economics of AI, impacting major labs like OpenAI and Anthropic.
Initial tests, such as by legal AI tool Harvey, show that cheaper models can perform comparably to larger ones for certain tasks.
The trend is about the size of models (large vs. small), not necessarily proprietary versus open-weight models.
Rising token prices and reduced investor subsidies are increasing cost pressure on users for the first time.
If successful, this transition could temper the demand for inference and question the justification for training frontier models.

Weiter zum Originalartikel

clem 🤗 reposted

brexton

There was an inflection point recently where the tide shifted to model pickers and OSS

Mix of tokenmaxxing/cost fatigue, nemotron coalition, harness step functions, brains/claws, etc

Long live those who embrace open superintelligence like @NotionHQ @FactoryAI @PrimeIntellect

clem 🤗 reposted

Brian Armstrong

Good take

My guess is

- demand for intelligence is near infinite

- but 80% of workloads will be running on 99% cheaper models within 12-18 months

- 20% of workloads will still run on latest gen models where IQ maxing is important (scientific breakthroughs, higher level ochestrator agents?)

- rough analogy might be what % of macbooks or gaming PCs sold have the maxed out specs for CPU/GPU, prices are falling much faster than Moore's law here though

- this leads me to think the limiting factor will be energy and compute, not better models

At Coinbase we're working hard on routing prompts to cheaper models where appropriate, and in some cases have been able to keep costs roughly flat, while token usage continues to grow exponentially.

clem 🤗

Great to see AI maturing and finally adopting what we've been preaching for a while now: multi-model workloads!

It will go even further progressively though with most companies not only using dozens of models but also adapting them, post-training them, optimizing them for their specific use-cases and constraints.

Ultimately there will be as many models as code repositories because models are the new code!

clem 🤗 reposted

Avanika Narayan

100p! in our recent intelligence per watt (ipw) paper, @JonSaadFalcon & i find that 71.3% of real world chat and reasoning queries can be shifted from frontier lms to local lms!

link to ipw paper in comments below 👇 https://t.co/lLgluh8pPL

clem 🤗

Narrative violation: according to @Stanford research, local models can answer 71.3% of real-world chat and reasoning queries accurately, up from 23.2% in 2023. Obviously at a fraction of the cost and energy consumption of frontier APIs.

The obvious conclusion: you don't need a frontier model for most tasks. The future is multi-model: local, open-source, smaller and cheaper for the majority of workloads, frontier APIs when no other choices!

clem 🤗 reposted

Yash Patil

This is exactly right.

People are starting to look for cheaper model alternatives and realizing two things at once: open-source models are already very good, and the ability to train and serve them efficiently at scale can change the economics pretty meaningfully.

Tokens are still being subsidized, demand is ramping quickly, and the compute crunch is likely to persist. That will push companies toward using the right model for each task instead of defaulting to the most expensive one.

We’re still early, but I expect open-weight adoption to accelerate much faster than most people think.

clem 🤗 reposted

Tommy

The most basic way AI could blow up imo. I'm not saying it does but this is the most obvious way I can see it happening

- Per seat subscriptions are massively subsidized. The flat fee was priced way below what heavy usage actually costs

- For real business use you have to move to the API anyway. Data protections, work integrations and compliance officer approval

- On the API you pay metered rates, and businesses are burning credits way faster than the per seat pricing ever led them to expect

- This is everywhere right now. Internally for us, Codex users, Uber torching its entire 2026 AI budget in 4 months, the Microsoft comments. Just go try an API

I shared more on this here: https://t.co/iZrqrCAIRW

- And I don't think most businesses have the money to keep paying increasing API rates without a real change to how they operate (caps needed)

- Because they have a cheap alternative. They can reach open source models through any aggregator (OpenRouter, Venice, Baseten, Together) and still get strong privacy. Venice private data centers, or E2EE/TEE serving GLM 5.1.

More on open source inference provider raises here: https://t.co/7kf56P44yQ

- And the discount is enormous. DeepSeek V4 codes within a hair of Opus on SWE bench at roughly 1/30th the price, and the cheapest open models run closer to 1/100th

- Chinese labs open source frontier grade models. The model is the single biggest cost an inference provider has, and they get it for free

- This idea dies if China goes closed source. That is actually bullish web2 AI labs, because if everyone is closed you pay up for the best intelligence. China goes closed source if they are tired of giving away an asset and they want the revenue and data flow to train new models

- Is this showing up in web2 AI lab revenue yet? No. Revenue is off the charts. Anthropic went from 9B to 47B run rate in five months

- So go forward, what happens?

- I think revenue slowly starts leaking to the open source inference providers (see Venice usage, OpenRouter's $113M raise, Baseten is raising at $11B or triple its valuation in three months, on revenue that went from $200M to $600M annualized in a single quarter)

- It doesnt move overnight, but it caps the labs ability to raise prices, and margins are already deeply negative. OpenAI is reportedly running near negative 122%

- With margins that bad there is no cash flow, so the labs are fully dependent on outside capital to buy GPUs, train models, and keep subsidizing usage (I.e. see Google tapping $80b equity sale, granted 30b for employee RSU taxes. Clearly they think Equity is overvalued or you wouldn't sell it)

- The break comes when that capital stops. Pricing is capped so margins cant improve, and the moment investors lose conviction on payback, the whole flow reverses

- Why would they lose conviction on payback? Back to the start - the inability to improve margins or get businesses to pay more

- This is also limiting, if we start making new drugs with AI or create entirely new businesses, you better believe people will pay up to the max for AI usage

Yann LeCun reposted

clem 🤗

clem 🤗 reposted

Nick Frosst

this model is the opposite of mythos.

Its small, cost effective, apache 2.0, and locally deployable. This is the way LLMs should go.

small, open source, transparent and sovereign

large, expensive, proprietary and hegemonic

clem 🤗

This graph captures what’s broken about AI evals: they structurally favor closed-source APIs that can route, fallback, ensemble, and optimize behind the scenes with no transparency.

No offense, @ArtificialAnlys, but how is comparing one model to two models fair? https://t.co/iuCu5GlWoQ

clem 🤗 reposted

Michael Mignano

For the first time in ~3 years, it feels like the AI table has been flipped over. Yes, the labs and hyperscalers will have the highest chance of resetting it before everyone else, given their vast capital, frontier model, and compute advantages.

But there is now a window for a new ecosystem to emerge. A "rebel alliance," which is what we're calling it at @USV.

We are excited by open weight models, distributed compute, human-aligned agents, routing, open source harnesses, the orchestration layer, and a lot more.

Basically, anything that gives people and enterprises powerful intelligence while maintaining tight incentive alignment, we're into it.

clem 🤗 reposted

Chubby♨️

Crazy: A 3B model is now reaching highly competitive results on verifiable reasoning tasks.

VibeThinker-3B scores 94.3 on AIME26, 80.2 Pass@1 on LiveCodeBench v6, and 96.1% on unseen LeetCode contests.

The gains appear to come primarily from post-training on top of Qwen2.5-Coder: curriculum SFT, multi-domain RL, offline self-distillation, and a final RL-based instruct stage.

The core implication: certain forms of verifiable reasoning may be highly compressible into small dense models.

Frontier-scale models still matter for broad knowledge and general-purpose capability, but compact reasoning models are becoming a serious complementary path.

Love to see it!