Is your AI token budget ready? Here is what you need to know.

Is your AI token budget ready? Here is what you need to know

If your AI spend has quietly crept past what you budgeted for this year, you're not alone, and it isn't going to improve. Token-based billing is becoming one of the strangest cost lines finance has ever had to plan for. It’s hard to predict and even harder to keep pace with. Here’s what is going on, and what you should be asking before it catches you out.

A few articles back, when I discussed how to build solid FP&A tools using AI, I said one of the questions to ask is about long-term AI token costs. I kept thinking about this, because this breaks with how we have forecast and budgeted in the past, so it deserves our attention.

What is a token anyway?

On one level, you can think about AI tokens the same way you think about tokens at a beer festival or carnival. You exchange currency for tokens at a certain rate, and then go on to “spend” these tokens for the rest of the day. A bottle of water might cost you one token, but a premium craft ale will cost you several tokens more. By the end of the day, you have run out of tokens and topped up twice, and have no idea how much you spent.

An added complexity with AI is that you spend tokens when you send information, and when you receive the reply, and it's the length of each that drives the cost. A vague question invites a long, rambling answer and requires multiple iterations to get to the answer, costing you more overall. Then, if you dictate your questions, or ask for a spoken reply, you can use even more tokens.

Now add in the experimentation needed when you first use a new technology, the various models available with differing token cost structures, and the fact that agentic AI spends tokens every time it runs rather than just once when you build it. It starts to become very clear that budgeting for AI is not the same as planning how many coffee pods you’ll need to buy for the office this year, and this is before you try to attach an ROI to the spend.

This almost unfathomable, hard-to-pin-down cost would chase many away if it were another expense item. However, as I’ve said before, we need to encourage AI use in our organisations to get and stay ahead.

Things have already gone wrong

This catch-22 has already tripped up companies around the world.  Uber used up its annual AI token budget by April this year. And a recent Accenture report, The CIO’s guide to AI tokenomics, mentions a large European bank that went from near zero to €5 million in token spend in a year, with one workload running at €200,000 a week before anyone noticed.

This isn't finance's first rodeo with technology costs that break the model though. We've had to adjust our FP&A activities to get to grips with software-as-a-service licensing, where you easily end up paying for a hundred seats but only twenty people ever log in, and cloud storage, where someone flips a switch and your monthly fee doubles. Tokens are the same pattern again, except moving faster, on a steeper trajectory, and even less visible, because there's often no seat count or storage limit to check against.

Cheaper won't save your budget

Although token costs are falling as AI companies compete for market share, Jevons paradox is at play here. The more token costs fall, the more we use AI, negating the savings. (Don’t believe me? Think about your first broadband internet connection. It probably seemed extravagant. Now think about if you could run your office on a 1.5 Mbps T1 line today. I thought so.) The research from Accenture found that the executives it surveyed expect token prices to fall 19% over the next two years but usage to rise 78%. You don’t need to do the sums to see where this is going.

We are driving this increase in usage as well. We keep telling our teams to hand the boring, repetitive, keyboard-heavy work over to AI, which is good advice. But that also means that instead of filtering a customer list down to one region and uploading just that slice, the obvious move is to upload the whole list and ask AI to do the filtering, because that's exactly the kind of grunt work we told people to stop doing. It's the logical behaviour and expensive behaviour at the same time.

Where's the ROI?

We keep telling our teams to work more efficiently to drive productivity, which implies we are expecting to see a return on our AI investment. But that is proving to be difficult to quantify so far. The experimental stuff we can let go of. We can agree to commit a certain AI budget to playing around and finding out how it can help us. But when AI is operationalised, and spreads across our organisation, surely we should see an ROI? Except here the maths is different: cost is usually the number you know going in, and with tokens it’s the one that keeps moving, often outrunning the return it's supposed to measure.

The ROI question is tricky to answer, and I don't think anyone has entirely cracked it yet. But now that we've got a proper handle on how odd this cost actually is, it's worth setting out what we should be asking, rather than winging it and treating it like any other cost.

Your first question should be when, and how, we actually measure a return. Do we wait a year and try to reverse engineer the value after the fact, or do we agree, before a token gets spent on a new use case, what it should achieve and how we'll know if it has done so. The next question is how we control spend without stifling the experimentation we need to get good at this. If every AI initiative has to justify itself financially from day one, we lose the ability to tinker that got us anywhere useful with any new technology in the past. The third question is more basic: whose job is this? Finance holds the budget, IT holds the infrastructure, and the team actually using the tool holds the context to say whether it's working. Split three ways like that, accountability for AI spend can end up sitting with nobody at all.

In many ways this is the same discipline we’d bring to any material spending decision. Ask the question early, build the measurement in from the start, know who answers for the number when someone asks.

And none of this is a reason to slow down with AI. I remain convinced the businesses that use AI well will outrun the ones that don't. But plan for the spend the way you'd plan for any other cost of doing business, and you'll be the one bringing discipline to this conversation, instead of the one explaining the bill shock.