The FinOps Foundation published a piece called Token Economics: The Atomic Unit of AI Value, and it’s one of the better attempts I’ve seen at giving AI cost management a real vocabulary. Tokens as the atomic unit of cost, goodput instead of raw throughput, and a warning that the token meter is increasingly hidden inside SaaS subscriptions you don’t control.
Most writing on this topic stays theoretical. I have something to test it against. The Citadel Platform series on this blog built cost attribution and semantic caching into a real APIM gateway, running in Sweden Central, metering a real agent. So instead of summarizing the FinOps article, this post uses it as a checklist. Where does the Citadel implementation already deliver on token economics, and where does it fall short?
The honest answer: it holds up well on attribution and caching, and it has clear gaps on goodput, yield, and routing. Let’s go through it.
Three token economics ideas worth carrying forward
I’ll paraphrase the three concepts I’m testing against, and you should read the original for the full argument.
Goodput, not throughput. Raw token volume tells you what you spent, not what you got. Goodput asks how many of those tokens produced useful output within acceptable latency. A retry storm and a productive session can burn the same token count.
The cost stack extends beyond the token. Tokens are the atomic unit, but the bill includes orchestration overhead, retries, tool-calling scaffolding, and increasingly, SaaS subscriptions that embed token consumption behind a flat price. When the meter sits inside someone else’s product, you lose the visibility FinOps depends on.
Engineering levers matter more than procurement levers. Model routing, semantic caching, and compressing tool-calling overhead move the cost curve more than negotiating a discount does. The FinOps article cites Cloudflare’s Code Mode work, which cut MCP tool-schema token overhead dramatically by changing how tools present themselves to the model.
Now let’s hold the Citadel Hub against those three ideas.
What the Citadel Hub already meters
Every call the weather agent makes flows through apim-wpvlimv4ngkns, and two of the five governance policies from earlier in the series do the token economics work.
The cost attribution policy emits token metrics per call, dimensioned by subscription and agent:
xml
<azure-openai-emit-token-metric namespace="citadel"> <dimension name="Subscription ID" /> <dimension name="Agent ID" value="@(context.Request.Headers.GetValueOrDefault("X-Agent-Id", "unknown"))" /> <dimension name="API ID" /></azure-openai-emit-token-metric>
That gives us prompt tokens, completion tokens, and total tokens per agent, per subscription, queryable in Application Insights. When someone asks what the weather agent cost last week, the answer is a query, not an estimate.
The semantic caching policy sits in front of the model and short-circuits repeat questions:
xml
<azure-openai-semantic-cache-lookup score-threshold="0.85" embeddings-backend-id="embeddings-backend" embeddings-backend-auth="system-assigned"> <vary-by>@(context.Request.Headers.GetValueOrDefault("X-Agent-Id", "unknown"))</vary-by></azure-openai-semantic-cache-lookup>
A cache hit costs an embedding call instead of a full completion. For an agent that answers weather questions, where “what’s the weather in Amsterdam” arrives in twenty phrasings, that’s not a rounding error.
In FinOps for AI terms, the first policy lives in the Understand Usage and Cost domain, and the second in Optimize Usage and Cost. So far, the framework and the implementation agree.
Where the mapping holds up
Two places, and one of them matters more than I expected before reading the article.
Semantic caching is a named lever. The FinOps article lists it explicitly as an engineering-side optimization, and the Citadel implementation has it running in production policy XML, not on a roadmap slide. Score threshold tuning is real work (0.85 took iterations, and I documented the false-positive risk in the original policy deep dive), but the lever exists, and it’s been pulled.
The gateway is the anti-aggregator, and the FinOps article’s sharpest warning is that token consumption is being hidden in SaaS subscriptions, where a flat monthly price hides a metered reality beneath the surface. The hub-and-spoke model is the architectural inverse of that problem. Nothing reaches a model without crossing the gateway, so nothing consumes tokens invisibly. The whole point of Part 2 in the Citadel series was to refuse the path that bypasses the meter when the Agent Service SDK tries to call the model directly.
I’d go one step further than the FinOps article does. Centralized metering isn’t just a FinOps convenience. It’s the same choke point that enforces content safety and the kill switch. Cost visibility and governance aren’t two systems in this architecture, they’re one policy pipeline.
Where Citadel’s token economics fall short
This is the useful part, because the gaps are specific.
- No goodput tracking: The Hub knows how many tokens the agent consumed. It does not know how many of them were worth consuming. Time-to-first-token and tokens-per-second aren’t captured as dimensions, and nothing distinguishes a completion the user acted on from one that got regenerated three times. By the article’s standard, Citadel measures throughput and calls it a day.
- No token yield rate: Closely related, but distinct. Yield asks: cost per successful outcome, not per call. The weather agent writes every conversation to Cosmos DB (Part 3 of the series), so the raw material for outcome tagging exists. Nothing joins it to the token metrics yet. That’s a gap in instrumentation, not in data.
- No model routing: Every query hits the same deployment, whether it’s “weather in Ede” or a multi-step tool-calling chain. The article’s Pareto framing (bulk tokens, mid-tier tokens, premium low-latency tokens, reasoning tokens, drawn from SemiAnalysis’s InferenceX benchmarking) implies a cascade: cheap model first, escalate on need. APIM can express this with backend pools and routing policy. Citadel doesn’t, yet.
- Tool-schema overhead is unmeasured: Every tool-calling request carries the Open-Meteo tool definition in the payload, on every single call. One tool, so the overhead is small. But the Cloudflare finding the article cites is a warning about what happens at ten or twenty tools, and I have no metric today that would even show me the problem growing.
Why this bites harder on agentic workloads
There’s a compounding effect the article touches on that I can back with a documented example. Orchestration overhead isn’t a fixed tax, it multiplies through agent chains.
In the Logic Apps Agent Loop series, I found that sequential agents don’t pass plain strings between each other. Each agent action returns a structured JSON messages array, and you need a Compose action to bridge it into the next agent. Every one of those bridged payloads is tokens. Single-agent token math is linear. Multi-agent token math is not, and that’s where token economics stops being a dashboard exercise. If your metering only captures totals per call, the orchestration overhead hides inside numbers that look individually reasonable.
What I’d add to the Citadel Hub next
In order of effort against payoff:
- Outcome tagging first: The conversations container already holds every run. Adding a resolution field (answered, retried, abandoned) and joining it against the token metrics in Application Insights gets me a real token yield rate with no new infrastructure. This is the cheapest gap to close and the one that changes the conversation from “what did we spend” to “what did we get.”
- Latency dimensions second: Emitting time-to-first-token and total duration alongside the existing token dimensions turns the same App Insights workspace into a goodput dashboard. APIM sees the timing already, it just doesn’t emit it.
- A routing experiment third: The weather agent is a good candidate for a two-tier cascade precisely because it’s boring. Simple lookups go to a small model, tool-calling chains escalate. If the cascade breaks the agent, it breaks it cheaply, and I’ll write up whatever goes wrong.
Tool-schema compression stays on the watch list rather than the to-do list. With one tool, measuring it first beats optimizing it blind.
Pitfalls
Adopting the vocabulary without the substance is the most common trap. It’s easy to say ‘we do token economics’ because a dashboard shows token counts. Raw volume without yield or goodput is accounting, not economics. The article’s framework is only useful if the uncomfortable metrics come with it.
Treating flat-price AI tools as flat costs is the second trap. When teams around you adopt AI SaaS tooling, those subscriptions consume tokens on someone’s meter. Budgeting them as fixed line items repeats the exact mistake the article warns about, one procurement layer up.
Optimizing the cache before understanding the traffic is the last one. A semantic cache with an aggressive threshold saves tokens and quietly serves wrong answers. Tune against logged real queries, never against the token savings number alone. I learned this at 0.85, and the number that’s right for a weather agent is wrong for an agent where two similar-sounding questions need different answers.
Closing
The FinOps article gives this space the vocabulary it needs, and the Citadel Platform gives me somewhere to test that vocabulary against running policy XML. The scorecard: attribution and caching, solid. Goodput, yield, and routing: real gaps with concrete next steps.
The bigger takeaway is architectural. Every improvement on that list lands in the same place, the gateway. APIM started this series as a governance layer. It’s ending it as the FinOps instrumentation layer too, and I don’t think that’s a coincidence. The choke point that can say no to a request is the same choke point that can tell you what the request cost.
If you’re metering your own agent platform, I’d like to hear which of these gaps you closed first, and whether the yield numbers surprised you.
