The token bill comes due: Inside the industry scramble to manage AI’s runaway costs

tech

The token bill comes due: Inside the industry scramble to manage AI’s runaway costs

The whole conversation shifted from tokenmaxxing and 'go fast' to 'we need guardrails, how do we control this?'

TL;DR

Companies are significantly overspending on AI, with some exhausting multi-year budgets within months.
Increased AI adoption and autonomous agents have drastically raised token consumption, leading to budget crises.
A new market is forming with startups and established vendors offering tools to track, measure, and optimize AI spending.
The Linux Foundation has unveiled the Tokenomics Foundation to establish standards for AI token usage and billing, similar to FinOps for cloud spending.
Productivity gains from heavy AI token usage are being questioned, with concerns about bugs and rewrites, and unclear ROI.
Managing AI costs presents a data problem of unprecedented scale, requiring new tooling and accounting systems.
Existing tech companies are adding AI spend management features, while new dedicated platforms are emerging.
The Tokenomics Foundation aims to create common definitions and frameworks for AI tokenomics, but solutions are needed immediately as usage is projected to multiply.
The best ROI may come from broad, moderate AI adoption rather than pushing heavy users to consume more.
There are parallels being drawn between AI cost management and earlier challenges in telecom and cloud expense management.

Continue reading the original article

clem 🤗 reposted

brexton

There was an inflection point recently where the tide shifted to model pickers and OSS

Mix of tokenmaxxing/cost fatigue, nemotron coalition, harness step functions, brains/claws, etc

Long live those who embrace open superintelligence like @NotionHQ @FactoryAI @PrimeIntellect

clem 🤗 reposted

Tommy

The most basic way AI could blow up imo. I'm not saying it does but this is the most obvious way I can see it happening

- Per seat subscriptions are massively subsidized. The flat fee was priced way below what heavy usage actually costs

- For real business use you have to move to the API anyway. Data protections, work integrations and compliance officer approval

- On the API you pay metered rates, and businesses are burning credits way faster than the per seat pricing ever led them to expect

- This is everywhere right now. Internally for us, Codex users, Uber torching its entire 2026 AI budget in 4 months, the Microsoft comments. Just go try an API

I shared more on this here: https://t.co/iZrqrCAIRW

- And I don't think most businesses have the money to keep paying increasing API rates without a real change to how they operate (caps needed)

- Because they have a cheap alternative. They can reach open source models through any aggregator (OpenRouter, Venice, Baseten, Together) and still get strong privacy. Venice private data centers, or E2EE/TEE serving GLM 5.1.

More on open source inference provider raises here: https://t.co/7kf56P44yQ

- And the discount is enormous. DeepSeek V4 codes within a hair of Opus on SWE bench at roughly 1/30th the price, and the cheapest open models run closer to 1/100th

- Chinese labs open source frontier grade models. The model is the single biggest cost an inference provider has, and they get it for free

- This idea dies if China goes closed source. That is actually bullish web2 AI labs, because if everyone is closed you pay up for the best intelligence. China goes closed source if they are tired of giving away an asset and they want the revenue and data flow to train new models

- Is this showing up in web2 AI lab revenue yet? No. Revenue is off the charts. Anthropic went from 9B to 47B run rate in five months

- So go forward, what happens?

- I think revenue slowly starts leaking to the open source inference providers (see Venice usage, OpenRouter's $113M raise, Baseten is raising at $11B or triple its valuation in three months, on revenue that went from $200M to $600M annualized in a single quarter)

- It doesnt move overnight, but it caps the labs ability to raise prices, and margins are already deeply negative. OpenAI is reportedly running near negative 122%

- With margins that bad there is no cash flow, so the labs are fully dependent on outside capital to buy GPUs, train models, and keep subsidizing usage (I.e. see Google tapping $80b equity sale, granted 30b for employee RSU taxes. Clearly they think Equity is overvalued or you wouldn't sell it)

- The break comes when that capital stops. Pricing is capped so margins cant improve, and the moment investors lose conviction on payback, the whole flow reverses

- Why would they lose conviction on payback? Back to the start - the inability to improve margins or get businesses to pay more

- This is also limiting, if we start making new drugs with AI or create entirely new businesses, you better believe people will pay up to the max for AI usage

Yann LeCun reposted

DavidLinthicum

The Emperor Has No Clothes: Why the AI Infrastructure Buildout Math Doesn't Work

I have to give IBM CEO Arvind Krishna credit. He's saying what many of us in this industry have been thinking but haven't been willing to say out loud. The math just doesn't add up.

Here's what I'm seeing that's deeply troubling. We're in the middle of another mass hallucination. Just like the dot-com bubble, just like blockchain, just like the metaverse — everyone is convinced that building massive data centers will automatically create massive wealth.

But here's the thing about building infrastructure. You actually have to sell what's inside it.

Let's talk numbers. The planned data center buildout over the next 5-10 years is staggering. We're talking about commitments in the hundreds of gigawatts globally. The capital expenditure commitments are in the trillions. Yet when you look at the actual demand signals, not the projections, not the potential, but the actual consumption patterns, there's a massive gap. These AI companies are betting everything on demand that simply doesn't exist at the scale they're planning for.

Let me be direct. AI services are expensive. Enterprise adoption is slow. Consumer AI is still finding its footing. And the compute requirements being promised by the hyperscalers require a level of demand that would represent a fundamental shift in how businesses consume technology. That's a big ask.

I've seen this pattern before. The overbuilding. The belief that if you build it, they will come. The groupthink that turns critical analysis into heresy. The result is always the same. Companies are going to touch the stove. We're going to see massive write-downs. We're going to see pivots, shutdowns, and strategic reviews. We're going to see companies that spent years and billions trying to be the AI infrastructure leader become case studies in how not to read a market.

The IBM CEO is right. The math doesn't work. And unlike 1999, we don't have the excuse of we didn't know. We know exactly what's happening. We just don't want to believe it because the alternative, being a skeptic while everyone else is piling in, feels like career suicide. It's not. The ones who survive the next decade will be the ones who built for reality, not fantasy.

Wake up. The emperor has no clothes.

As reported by Futurism, Krishna laid out striking calculations: a 1 gigawatt data center costs roughly $80 billion today. If one company commits 20-30 gigawatts, that's $1.5 trillion in capital expenditure. The total commitments across the industry for chasing AGI are approximately 100 gigawatts, equaling $8 trillion. To break even, you'd need $800 billion in profit just to cover the interest. That's not investment. That's hoping.

https://t.co/4DAnF5OPfa

clem 🤗 reposted

Grace Isford

A huge moment for open source AI - as token usage skyrockets across orgs, so do concerns over cost & data/vendor lock-in & necessity of a multi-AI strategy

Keep an eye on @Lux_Capital portfolio cos @huggingface @togethercompute @SakanaAILabs & more! 🚀 https://t.co/MXQMaENKmw

clem 🤗 reposted

Brian Armstrong

How to keep AI spend flat while token usage grows exponentially: Not with friction and spend alerts. With better defaults, routing, and caching.

Better Defaults (not Usage Caps) – Engineers can choose any model they want, but defaults matter. We’re experimenting with defaulting to open weight models like GLM 5.2 and Kimi 2.7 through our LLM gateway, while still encouraging engineers to choose the right model for the task. 91% of our employees were never hitting their usage caps, so instead of lowering caps and driving up alerts, we're moving to cheaper defaults. Note that code reviews use a diversity of models, so they can check each other's work.

Better Routing – In our custom harnesses, we preprocess prompts and route to the best model for the job, considering cache hits and model pricing. For instance, you may want a frontier model for planning, but not for execution where they can be overkill. Ultimately, humans shouldn't be choosing models - AI can automate this task.

Better Caching – Cache misses are the easiest way to drive your cost up. All of our requests are cache aware, so we’re reusing a warm cache wherever possible. For example, our cache hit rate went from 5% → 60% in LibreChat once properly implemented.

Keep Context Lean – Start fresh sessions when switching tasks. Scope file context narrowly. Disconnect unused tools. Don't just compact. The goal isn't fewer tokens used, it's fewer tokens wasted.

Better Visibility – Our engineers can use as many tokens as they want, from whatever model they want, but we’ve made usage visible – and the more you spend on AI, the more impact we expect.

The goal isn't to suppress usage. It's to build the infrastructure that makes exponential growth sustainable.

Putting this into practice has cut our AI spend nearly in half, while our token usage continues to grow.

clem 🤗

the future of AI is multi-model (including a majority of open-source ones provided by @huggingface of course!)!

clem 🤗 reposted

Chad Wahlquist

I think 90% of the conversations I am having with executives in the last two weeks they have told me they are reducing spend on tokens and moving to open weight models.

Yann LeCun reposted

Gavin Baker

More data than open-source AI is taking share from OpenAI and Anthropic. Open source has gone from 28% token share to 62% token share @vercel over the last 2 months. Chart from @rauchg

Super impressive given that the sum of OpenAI and Anthropic accelerated in July. So net token/AI infra demand accelerated even more than the acceleration we saw at the frontier. And suspect Grok growing even faster than open-source and we saw some of this in the @tryramp data.

Open-source AI taking share is positive for AI infrastructure demand as it lowers margins at the model layer and an open-source token costs just as much compute to produce as a frontier token. Nothing about open-source AI inference is “free.”

Most likely end state IMO is that closed, frontier tokens are 60-90% of economic value but only 15 to 25% of tokens.