Daniel's been building an inventory system — fork of an open source project, custom book cataloguing module bolted on — and he's been using Gemini to pull structured data from photos of books: title, author, description, ISBN if it's there. Five photos per item, a hundred items. He ran the numbers, and the total API cost came out to thirty-seven cents. Which is absurdly cheap. And he says that's the norm — most of the time, when he actually does the math on these workflows with the cost-effective models, he's stunned by how low it lands.
But then there's the other kind of project. He ran a multi-agent geopolitical simulation using a framework called Snowglobe — complex stuff, multiple agents each with their own context window, tool calls, long reasoning chains. He thought he had the costs dialed in. One run cost him over a hundred dollars. His question is: how do you move cost estimation from hoping and praying to actual science? Because the billing docs are complicated, the pricing is segmented by modality, and most builders, he suspects, are just crossing their fingers.
So let's look at why this is so hard, and what you can actually do about it.
The thing that jumps out at me about Daniel's two examples is that they're perfect bookends. Thirty-seven cents for a hundred items is a bounded, predictable workflow. A hundred dollars for one simulation run is an unbounded, agentic one. And the central paradox of AI APIs right now is that they are astonishingly cheap per unit — Gemini two point oh Flash costs ten cents per million tokens for text input — and yet developers routinely have cost accidents that run into hundreds or thousands of dollars. The per-token price isn't the problem. The problem is that token consumption is wildly unpredictable once you add images, audio, tool calls, and multi-turn conversations.
So the unit economics look great on a spreadsheet, and then reality walks in and sets the spreadsheet on fire.
That's... yes. That's exactly what happens. And I think the reason Daniel's thirty-seven cents is actually a remarkable achievement of estimation is that he did the thing most builders skip — he scoped the key, he ran the sample, he did the math. Most people look at a pricing page, see "ten cents per million tokens," think "that sounds cheap," and deploy. And then they're shocked when a hundred dollars evaporates.
To understand why the estimation is so hard, we need to look under the hood at how multimodal billing actually works.
Right. So let's take Gemini as our example, since that's what Daniel's using. Gemini two point oh Flash — the cheap one — costs ten cents per million tokens for text input if your context window is under a hundred and twenty-eight thousand tokens. Text output is forty cents per million. Those are the numbers everyone quotes. But then you scroll down the pricing page and things get segmented. Images are billed differently from text. Audio is billed differently from images. Video is billed differently from everything. And the cost of an image isn't a fixed number of tokens — it depends on the resolution, the compression, and whether you're using the image understanding feature or sending raw pixels.
So you can't just count your images and multiply.
You cannot. Google's documentation says images are charged based on the number of tokens they occupy, and that number varies by dimensions and compression level. A seven hundred and sixty-eight by seven hundred and sixty-eight image costs roughly two hundred and fifty-eight tokens in Gemini. A two thousand forty-eight by two thousand forty-eight image costs over a thousand tokens. That's a four times difference, and most developers don't know that. They send whatever their phone camera produces and assume the cost is negligible.
And Daniel's book cataloguing project — five photos per item — the cost could vary by a factor of four depending on whether he resized the images before sending them.
And here's the deeper problem: token counting is non-deterministic for multimodal inputs. With text, you can count characters and estimate tokens with reasonable accuracy — roughly four characters per token, give or take. With images, there's no simple formula. The same image might cost slightly different amounts depending on how the API processes it internally. You can't sit down with a calculator and work it out from first principles. This is the root of the hoping-and-praying problem — you don't have all the variables.
It's like trying to budget for a road trip where the gas station sets the price after you've already filled the tank.
That's... actually a really good analogy. And it gets worse when you look at the more expensive models. Gemini two point five Pro costs a dollar twenty-five per million tokens for text input under a hundred and twenty-eight thousand context, and two dollars fifty per million for input over that threshold. Text output is ten dollars per million. So the same workflow that costs thirty-seven cents on Flash could cost four or five dollars on Pro — and if your context window grows past that hundred-and-twenty-eight-thousand token boundary mid-conversation, the price jumps without warning.
Wait. The input price changes mid-conversation if your context window crosses the threshold?
The way the pricing is structured, yes — the rate depends on the context window size at the time of the request. So if you're running a multi-turn agent and the conversation history keeps growing, you might start at the lower rate and then cross into the higher rate without realizing it. The billing doesn't pause to notify you.
So the meter's running and the rate can change while you're still on the call.
Yes. And that's before we even get to tool calls. Every time an agent calls a tool, the tool's output gets fed back into the context window, which grows the window, which can push you into a higher pricing tier, which increases the cost of every subsequent turn. It's a compounding effect.
Let's talk about Daniel's Snowglobe accident, because that's where all of these factors collide.
Snowglobe is an open-source framework for running multi-agent simulations — geopolitical scenarios, economic modeling, that kind of thing. Each agent in the simulation maintains its own context window. Each agent makes tool calls. Each agent generates long chains of reasoning. And the agents talk to each other, so the context windows grow with every interaction. A single simulation run might involve hundreds of API calls, and each call is more expensive than the last because the context window is larger.
And Daniel said he thought he understood the cost and had everything configured correctly.
I'd bet his hundred-plus-dollar run came from a combination of three things. One: underestimating the number of turns per simulation. If you think it'll be fifty turns and it's actually two hundred, your cost estimate is off by a factor of four before you even get to the other variables. Two: not accounting for context window growth. Each turn adds tokens to the history, and by the end of a long simulation, each API call might be processing tens of thousands of tokens of history. Three: using a more expensive model than intended — maybe the default was set to Pro instead of Flash, or maybe he switched models for one run and forgot to switch back.
That third one is painfully relatable.
It happens constantly. You test with Flash, you get great results, you think "let me try this one run with Pro to see if it's better," and then you forget to change it back. And Pro is twelve and a half times more expensive than Flash for text input. One run on Pro costs as much as twelve runs on Flash.
And the simulation framework isn't going to warn you. It just calls the API with whatever model you configured.
Right. The framework doesn't care about your budget. It just executes the simulation. And agentic systems have this property where the cost is emergent — you can't predict it from the configuration alone because the agents decide how many turns to take, which tools to call, how much reasoning to do. The system's behavior determines the cost, and the behavior isn't fully specified in advance.
So if the mechanics are this complex, how do you actually build a system that doesn't blow up your budget? Daniel's asking how to move from hoping to science. What does that look like in practice?
The key insight is that you don't need to predict exact costs. You need to bound them. For a routine workflow like Daniel's book cataloguing, the approach is straightforward. Step one: run a small sample — say, five items — and measure actual token usage via the API response metadata. Every major API returns token counts in the response. Step two: calculate the per-item token average from that sample. Step three: extrapolate to full scale and add a safety margin of two to three times for variance.
And that's essentially what Daniel did, and it worked. Thirty-seven cents for a hundred items.
He did the science without maybe realizing he was doing the science. The sample run gives you real data instead of theoretical estimates. The extrapolation gives you a target. The safety margin covers the fact that some books will have longer descriptions or more complex covers that consume more image tokens.
What about the unbounded workflows? The Snowglobe simulations of the world?
Different methodology entirely. For agentic systems, you need hard limits before you start. Set a maximum number of turns. Set a maximum context window size. Set a maximum total token budget per run. These should be enforced in code, not in your head — if the simulation hits the turn limit, it stops. If the context window hits the cap, it either stops or starts trimming.
So you're building a fence around the pasture before you let the agents graze.
Yes. And then there's a technique I think is underused: cost simulation mode. You run the system with a cheap model — Gemini two point oh Flash, or even a local model if you have one — to measure the token consumption patterns. See how many turns the agents actually take. See how fast the context windows grow. See which tools get called and how much data they return. Once you have that profile, you multiply by the target model's price to estimate what the real run would cost.
So you're rehearsing the play with the understudies before you bring in the expensive cast.
That's it. And the cheap run costs pennies. You can run it ten times, get a distribution of token consumption, and then make an informed decision about whether the Pro model is worth it. Maybe you discover that the simulation averages eighty thousand tokens per run, which on Pro would be about a dollar twenty in input costs plus output. Now you know. You're not hoping anymore.
There's another layer to this that Daniel mentioned — scoped keys and budget controls. He used a scoped key for the book project, which is smart, but he also said that didn't prevent the Snowglobe accident.
Scoped keys limit the blast radius. They restrict which APIs a key can access and sometimes cap the rate, but they don't typically enforce a hard spending limit. The real protection is budget alerts at the project level. Google Cloud lets you set budget alerts at percentage thresholds — fifty percent, ninety percent, a hundred percent of your monthly budget — and those alerts can trigger automated shutdown of API access. It's not just a notification. You can configure it to actually stop the spending.
So the API key just stops working when you hit the cap.
If you set it up that way, yes. And most people don't. The default is to send an email, which you might not see for hours. By then, the agent loop has run another twelve thousand API calls.
What's the friction here? Why don't developers set these up?
I think there are a few things. One is that setting up budget alerts requires going into the cloud console, which is a separate interface from the API playground where you're doing your development. It's a context switch. Two is that the billing docs are complicated — they're written for enterprise procurement teams, not for a developer trying to figure out what a test run will cost. Three is the psychological thing: "it's just a test run, it'll be fine." You're excited about the thing you built, you want to see it work, and setting up budget alerts feels like paperwork.
The assumption that it'll be fine because it was fine last time.
And last time was a different project with different parameters. Daniel said he's been running projects like this almost every day for over a year, and this was his first major cost accident. That's actually a good batting average. But the one accident cost more than all the successful runs combined, probably. A hundred dollars pays for a lot of thirty-seven-cent cataloguing jobs.
Two hundred and seventy of them, roughly.
Right. And Daniel's a hundred dollars is actually a relatively cheap lesson. There are documented cases of developers getting bills in the thousands from runaway agent loops — an agent that gets stuck in a retry cycle, or a misconfigured caching layer that re-processes the same images every request, or a recursive tool call that spawns sub-agents that spawn sub-agents.
The sub-agent spawning thing is where this gets alarming. If each agent can create new agents, and each new agent has its own context window and makes its own API calls...
The cost grows combinatorially. And this isn't a hypothetical — it's how some of the more advanced agent frameworks work. You have a coordinator agent that delegates to specialist agents, and the specialists might delegate further. Each delegation adds a new context window. Each context window accumulates history. If you don't have hard limits on delegation depth, the system can explode.
So as these APIs get more capable and more expensive, the cost estimation problem compounds. Gemini two point five Pro is twelve and a half times more expensive than Flash. What happens when the next generation is fifty times more expensive and ten times more capable?
The gap between "I think this will cost about a dollar" and "this actually cost four hundred dollars" gets wider. And the more capable the model, the more tempting it is to use it for everything. Why run your simulation on Flash when Pro gives better reasoning? Why not use the best model for every turn? The answer is that you can't afford to, but the pricing page doesn't make that obvious until the bill arrives.
I think there's a broader point here about cost hygiene as a practice. You wouldn't deploy code without testing. You shouldn't deploy an AI workflow without understanding its cost profile. But the tools and documentation don't make that easy.
They really don't. And I think part of the reason is that the providers benefit from the complexity. Not in a malicious way — I don't think anyone's sitting in a boardroom cackling about confusing pricing pages. But the complexity means that developers tend to overestimate how much runway they have and underestimate how much they're spending. The default path is to spend more than you intended.
The pricing page as a maze with cheese at the exit.
And the cheese is "ten cents per million tokens," which sounds like nothing, and you stop reading. Meanwhile, the image pricing is three paragraphs down, the context window tiering is on a different page, and the tool call billing is documented in a footnote somewhere.
So given all of this, what's the practical framework? If a builder is starting a new project tomorrow, what should they actually do?
I'd say there are four things. First, before you write any integration code, read the full pricing page for your model. Not just the headline number — the whole thing. Modality segmentation, context window tiers, output pricing, batch pricing if it exists. Know what you're getting into. Second, build a cost estimation harness into your project from day one. Log the token counts from every API response. Track them over time. Make the cost visible in your development loop.
So you're not just seeing "request succeeded" — you're seeing "request succeeded, cost: zero point zero three cents."
Third, for any unbounded workflow, set hard limits in code. Maximum turns, maximum context window, maximum total tokens. The system should stop itself, not wait for you to notice the bill. And fourth, set up budget alerts at the cloud project level before you run anything at scale. Fifty percent, ninety percent, a hundred percent. Configure the hundred percent alert to shut down API access, not just email you.
That last one is the thing almost nobody does, and it's the one that would have caught Daniel's Snowglobe run.
Yes. If he'd had a budget alert at, say, twenty dollars, the simulation would have stopped before it hit a hundred. He'd have gotten a notification, investigated, and maybe discovered the configuration issue. Instead, the simulation ran to completion and the bill arrived later.
The bill arriving later is part of the problem too. There's a delay between when you make the API calls and when the cost shows up in the console. You're flying blind in real time.
The latency on billing data varies by provider, but it's rarely instantaneous. You might not see the damage for hours. By then, you've moved on to something else and the context of what you were doing is gone. You open the console, see a hundred-dollar spike, and have to reconstruct what happened.
But maybe we're overcomplicating this.
Hilbert: You are. This isn't a technical problem. It's an accounting problem.
Hilbert: I used to be a financial controller for a small SaaS company that resold white-label AI APIs. My entire job was figuring out why the bills were higher than expected and arguing with the provider about billing discrepancies. We had one client whose monthly bill came in at twelve thousand dollars instead of the expected eight hundred. The reason was a misconfigured caching layer that was re-processing the same images on every request. Every single page load, same images, new tokens, new charges. Nobody noticed for three weeks.
Hilbert: The reason the billing docs are complicated is that the providers want them to be complicated. They want you to guess. Because when you guess, you guess high, and they get more money. The real skill isn't calculating tokens. It's reading a bill and knowing what to challenge.
You're saying the complexity is intentional.
Hilbert: I'm saying I once got a four-thousand-dollar credit from Google by pointing out that their documentation said one thing about image tokenization and their billing system did another. They changed the docs the next week. I still have the screenshot.
Wait — so their own documentation was wrong about how they bill for images, and when you caught them, they fixed the docs rather than the billing?
Hilbert: The billing was what they intended. The docs were aspirational. That's not unusual. These pricing models are being updated faster than the documentation teams can keep up. The model releases, the pricing gets set, the docs lag by weeks. In the gap, you're being billed according to rules that aren't publicly documented yet.
That's... I don't even know what to do with that.
Hilbert: You reconcile. Every month. Line by line. You compare the API usage logs to the bill. You look for discrepancies. You file disputes. It's not fun, but it's the only way to know what you're actually paying. All the estimation frameworks in the world won't help you if the provider is billing you differently than the docs say.
The science of cost estimation has to include auditing on the back end, not just prediction on the front end.
Hilbert: That's the part everyone skips. They estimate, they deploy, they get a bill, they pay it. Nobody checks whether the bill matches the estimate. The estimate could be perfect and the bill could still be wrong.
How often were the bills wrong?
Hilbert: In two years, I'd say maybe one in six months had a discrepancy worth disputing. Not always four thousand dollars. Sometimes fifty bucks, sometimes two hundred. But it adds up. And most developers never check.
The thing that strikes me about what Hilbert's saying is that it reframes the whole problem. We've been talking about estimation as if the goal is to predict what the provider will charge. But if the provider's billing isn't even consistent with their own documentation, then estimation is only half the battle. The other half is verification.
Verification requires logging. You need your own record of every API call, the token counts returned, the model used, the modality breakdown. If you don't have that, you can't dispute anything.
Hilbert: The API response gives you the token counts. Store them. It's three lines of code. Nobody does it.
I do it.
Hilbert: Good. Most people don't.
The framework expands. Estimate before you run. Cap during the run. Audit after the run. Three phases, and most builders only do the first one.
Even the first one, they're doing it with incomplete information because the pricing pages are segmented in ways that aren't intuitive. The modality breakdown — text, image, audio, video — makes sense from the provider's perspective because those have different computational costs. But from a builder's perspective, you're sending a request that happens to include an image. You're not thinking "I am now consuming image tokens at a different rate." You're thinking "I am asking the model a question about this photo."
The billing model doesn't match the mental model.
That mismatch is where the accidents happen. Daniel's book cataloguing project worked because the mental model and the billing model were close enough — send photos, get structured data back, cost is predictable. The Snowglobe simulation failed because the mental model was "run a simulation" and the billing model was "charge per token across dozens of agents with growing context windows and tool calls." Those two things don't map onto each other at all.
Let's talk about where this is going. Agentic systems are becoming the default way to build with these APIs. Every agent spawns sub-agents, each with its own context window. The cost isn't linear — it's branching. How do you budget for emergent behavior?
I don't think you can, not with current tooling. The best you can do is set hard caps and hope the caps are low enough to protect you but high enough to let the system do useful work. It's a balancing act, and the balance point is different for every use case.
We're back to hoping and praying, just with better guardrails.
Guarded hoping. Informed praying.
That's the name of my next mindfulness retreat.
I'd attend that. But seriously — the industry needs better cost observability. We need per-key cost tracking that's as good as what Open Router provides. We need real-time spending dashboards that update with every API call, not hours later. We need the billing systems to be as fast and transparent as the models themselves.
Until then, builders need to treat cost estimation as a first-class engineering practice, not an afterthought. Daniel's thirty-seven cents is a beautiful number. But the real lesson is that he knew it was thirty-seven cents. Not thirty-seven dollars, not three cents. That knowledge is the product of rigor — scoped keys, sample runs, actual math — and it's worth more than any single cost-saving tip.
The misconception I see most often is that AI API costs are inherently unpredictable and you just have to accept that. The reality is that with proper test runs and bounding, most workflows can be estimated within a factor of two or three. It's not perfect, but it's a lot better than crossing your fingers and waiting for the bill.
The question I'm left with is whether the industry converges on standardized billing practices as these APIs become more central to how software works. Right now, every provider has their own pricing page with their own segmentation logic and their own documentation quirks. That's manageable for a developer who works with one API. It's chaos for anyone who works with three or four.
The agent frameworks often abstract away which model they're calling. You configure it once, forget about it, and six months later you're on a different pricing tier because the default model changed in an update. The complexity isn't going away. If anything, it's compounding.
The science of cost estimation isn't a one-time thing you do at the start of a project. It's an ongoing practice. Measure, cap, audit, repeat. Every run, every month, every model change.
Store your token counts.
Store your token counts. This has been My Weird Prompts. Thanks to our producer Hilbert Flumingtop for keeping us honest — and for the screenshot, which I now want to see.
You can find us at my weird prompts dot com, or email the show at show at my weird prompts dot com. We'll be back soon.
