Finding the Right Memory: Vector, Full-Text, and Hybrid Search in Cosmos DB

Posted on September 3, 2026 by steefjan1970

Post 3 of 6 on Cosmos DB agent memory search, because storing memory well doesn’t guarantee you’ll retrieve the right piece.

Post 2 ended with the schema settled and retrieval still open. This post closes that gap: the practical mechanics of Cosmos DB agent memory search, one container, four query patterns. Run the same question four different ways against that container, and it comes back with four different answers, because “find the right memory” isn’t one query pattern; it’s at least three, and knowing which one to reach for is most of the job.

Vector Indexing for Cosmos DB Agent Memory Search

Cosmos DB supports two vector index types, and the right one depends almost entirely on how many vectors you’re searching, not on anything specific to agents.

quantizedFlat compresses each vector and scans the compressed space exactly. It suits smaller workloads (tens of thousands of vectors) and trades a small amount of accuracy for lower RU cost and faster scans. For a single tenant’s short-term memory, this is often enough on its own.

DiskANN, on the other hand, indexes vectors for approximate nearest-neighbor search and scales to hundreds of thousands or billions of embeddings, with dynamic updates and strong recall even at that size. Post 1 already leaned on DiskANN as part of the case for Cosmos DB as a unified store; this is the mechanism behind that claim.

Sharding the Vector Index for Multitenant Isolation

DiskANN doesn’t have to search across every vector in the container. A vectorIndexShardKey partitions the index itself by a property you choose: session, user, or tenant, so a query only searches candidates within that shard instead of the whole container.

That maps directly onto the partition key work from post 2: set the vectorIndexShardKey to tenantId, or to the same [tenantId, threadId] pair you already use as the partition key, and semantic search for one tenant never touches another tenant’s vectors. A global, unsharded index still works and makes searching everything at once simpler, but it’sonly appropriate for a single-tenant app or a genuinely shared knowledge base where cross-tenant recall is the point rather than a leak.

Full-Text Search: When Precision Beats Semantics

Vector search finds what’s semantically similar. Sometimes semantically similar isn’t what you want — a customer asking about “the refund policy” needs the actual refund policy language, not five conceptually related passages about returns in general.

Full-text search on Cosmos DB handles that case through BM25, a statistical ranking function that scores by term frequency and document length. Cosmos DB applies linguistic processing automatically: tokenization, stemming, case normalization, so “running” still matches “run” or “ran.” It’s the right tool whenever exact terms or phrases carry meaning that a vector embedding would blur.

Hybrid Search: Combining Both with RRF

Most agent memory queries don’t need to choose between semantic and lexical relevance; they need a blend of both. That’s what Reciprocal Rank Fusion (RRF) does: it takes the vector-similarity ranking and the BM25 ranking for the same result set and merges them into one combined rank, instead of forcing a pick between the two.

In practice, this shows up as a single ORDER BY RANK RRF(...) clause, which the next section demonstrates directly.

Four Ways to Ask the Same Question

Take the turn-based schema from post 2 — tenantId, threadId, turnIndex, messages, embedding, content and run the same underlying question against it four ways. (content is a flat, denormalized copy of the turn’s text, added specifically because Cosmos DB doesn’t support wildcard array paths like /messages/*/content in a full-text policy or index the full-text and hybrid queries below point at c.content rather than c.messages for exactly that reason.)

Most recent, by recency:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
ORDER BY c.turnIndex DESC

Semantic, by vector similarity:

			
SELECT TOP 5 c.messages, VectorDistance(c.embedding, @queryVector) AS score
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
ORDER BY VectorDistance(c.embedding, @queryVector)

Hybrid, blending both with RRF:

			
SELECT TOP 5 c.messages, VectorDistance(c.embedding, @queryVector) AS score
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
ORDER BY VectorDistance(c.embedding, @queryVector)

Keyword, by exact phrase:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
  AND FULLTEXTCONTAINS(c.content, @phrase)
ORDER BY c.turnIndex DESC

		

Run all four against a thread where a customer asked about refunds three times, in different words, across twenty turns, and the differences stop being theoretical fast: recency surfaces whichever turn happened most recently, even if it’s off-topic; semantic search pulls in every conceptually related turn, including the ones that used different words entirely; hybrid balances the two; keyword search returns only the turns that used the customer’s actual phrase, and ranks them by recency underneath that filter.

Running These Queries in Data Explorer

The four queries above use parameterized SQL, the same form search.py, from the companion repo behind this series, sends through the Python SDK, which binds @tenantId, @queryVector, and @phrase properly before the query runs. Paste them as-is into the Azure Portal’s Data Explorer query pane instead, and two things break, neither of which is a schema or code bug:

Data Explorer’s query box doesn’t bind named parameters. A query that leaves @tenantId unresolved either matches nothing and returns “No results” silently, or for VectorDistance() inside ORDER BY and FullTextScore() fails to compile outright, because both functions require their arguments to resolve to literal values at query-compile time rather than at execution time.

Swap every @parameter for a literal value and all four run cleanly. Against the seeded sample data (tenantId = "contoso", threadId = "thread-1234", searching for "refund"):

Recency, with literals:

			
SELECT TOP 5 c.messages, c.turnIndex 
  FROM c WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234" 
  ORDER BY c.turnIndex DESC

Semantic, with literals:

			
SELECT TOP 5 c.messages, VectorDistance(c.embedding, [0.8196, 0.6392, -0.2471, 0.1608, -0.8667, 0.4902, -0.2549, 0.2235]) AS score
FROM c
WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234"
ORDER BY VectorDistance(c.embedding, [0.8196, 0.6392, -0.2471, 0.1608, -0.8667, 0.4902, -0.2549, 0.2235])

Hybrid, with literals:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234"
ORDER BY RANK RRF(
    VectorDistance(c.embedding, [0.8196, 0.6392, -0.2471, 0.1608, -0.8667, 0.4902, -0.2549, 0.2235]),
    FullTextScore(c.content, "refund")
)

		

Keyword, with literals:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234"
  AND FULLTEXTCONTAINS(c.content, "refund")
ORDER BY c.turnIndex DESC

		

Pitfalls

Reaching for DiskANN on a small dataset. DiskANN’s approximate search and sharding options solve a scale problem. Below roughly ten thousand vectors, quantizedFlat gets equivalent recall for less operational complexity and lower RU cost. Default to DiskANN because it sounds like the “serious” choice, and you’ve added index-shard decisions to a workload that never needed them.

A global vector index in a multitenant app. Skip the vectorIndexShardKey, and a semantic query searches every candidate in the entire container, tenant boundaries or not. Nothing stops the query from surfacing another tenant’s conceptually similar memory in the result set unless a WHERE clause happens to filter it back out after the fact, and relying on a filter to catch what the index itself should have scoped is the kind of gap that shows up in an audit, not in testing.

Forgetting WHERE filters still apply. Vector and hybrid queries look like they replace normal filtering, but ORDER BY VectorDistance(...) or ORDER BY RANK RRF(...) still runs inside a WHERE-scoped query, same as any other. Leave the WHERE c.tenantId = @tenantId AND c.threadId = @threadId clause off a semantic query, and it searches everything the container holds, not just the thread the agent is currently in.

Next: Coordinating Multiple Agents

That settles Cosmos DB agent memory search for a single agent working alone. Coordinating what several agents know about the same conversation is a different problem, and it’s where change feed, a mechanism post 1 already covered as a callback to the 2023 retail monitoring work, comes back to tie multi-agent state together. That’s post 4.

Sources

Microsoft Learn — Agent memories in Azure Cosmos DB for NoSQL
Microsoft Learn — Vector search in Azure Cosmos DB
Microsoft Learn — Sharded DiskANN (vectorIndexShardKey)
Microsoft Learn — Hybrid search in Azure Cosmos DB
GitHub — steefjan1/cosmos-agent-memory-lab — the runnable sample behind these four queries, and behind posts 2 and 4

The Microsoft AI Stack in 2026 and the Certification Trail That Runs Through It

Posted on September 2, 2026 by steefjan1970

I spent some time observing what’s inside the Microsoft AI stack. Foundry, Agent Framework, Logic Apps, AI Search, Purview, Entra. After a while, I had a picture of how the pieces fit and decided to draw one myself to share.

Then recently Microsoft published AI-500, an expert certification for multi-agent systems. That made me curious whether Microsoft’s view of the platform matches my own. So I plotted the certification trail against my diagram. This post shows the result.

The Microsoft AI stack as I see it

Six layers. Five of them stack vertically. The sixth runs down the side, through all the others.

Models sit at the bottom: GPT-5, Claude, Mistral, Grok, Microsoft’s own MAI and Phi, Llama, DeepSeek, and the open catalog. This is the least differentiated layer. In Foundry, swapping one model for another is a configuration change. It gets the most attention and deserves the least.

Infrastructure comes next. Foundry is the hub, alongside Azure OpenAI, Azure ML, AKS, Container Apps, App Service, and Foundry Local for the edge. This is hosting, serving, and compute. Solid and well understood.

Data and context: the real moat

I split the data layer in two, because it hides the most important part of the platform. Layer 3a holds the sources of truth: Microsoft Graph, SharePoint, Exchange, Fabric and OneLake, Dataverse, and the vector stores in Cosmos DB, Azure SQL, and PostgreSQL.

Layer 3b is context. Work IQ, Fabric IQ, Foundry IQ, and Azure AI Search turn enterprise data into grounding that respects who is asking. Permission-trimmed retrieval must honor Entra ACLs at query time, not filter after ranking. AI Search supports this through document-level access control. Get it wrong and answers leak, or recall collapses.

This is where the hard engineering hours go. Swapping a model is a config change. Graph-grounded context is months of work.

Agents, Copilots, and the layer vendors leave out

The agentic platform sits above the data. Microsoft Agent Framework merged Semantic Kernel and AutoGen. Next to it sit Foundry Agent Service, Copilot Studio, Logic Apps, Azure Functions, Service Bus, and Event Grid. MCP and A2A handle tools and agent-to-agent communication.

The Copilot layer is on top: Microsoft 365, GitHub, Security, Dynamics 365, Power Platform, and Teams. This is distribution. These are the surfaces people already live in. As a result, enterprise AI adoption is easier when data, permissions, infrastructure, and applications already exist in one ecosystem.

Then comes the sixth layer of the Microsoft AI stack, the one vendor diagrams leave out: governance and evidence. Entra ID and Entra Agent ID handle identity. Purview covers labels, DLP, and audit. Content Safety and API Management provide guardrails and the AI gateway. Defender, Sentinel, Azure Monitor, Log Analytics, and Azure Policy deliver traces, evaluations, and control evidence.

I work for a regulated organization. Before anything goes live, Legal and Internal Audit ask four questions. Who approved the agent? What data did it access? Which controls applied? What happened when it made a wrong decision? Distribution makes deployment easier. However, without traceable evidence it does not make the system production ready. That is why this layer runs vertically through my drawing.

The Microsoft AI certification trail

I knew AI-900. I had not followed what replaced it. This week I learned that Microsoft now offers a full set of AI certifications, from fundamentals to an expert exam. The expert exam, AI-500, is in beta. I read the study guide. Its scope says a lot: orchestration patterns, agent-to-agent protocols, observability, guardrails, and cost control. Architecture work, end to end.

What made it click was laying the whole trail side by side. Each step has its own verb.

AI-901, Azure AI Fundamentals. Understand the concepts and services.
AI-103, AI Apps and Agents Developer. Build applications and agentic solutions on Foundry.
AI-200, Azure AI Cloud Developer. Engineer cloud-native AI properly.
AI-300, ML Operations Engineer. Operate models in production.
GH-300 and GH-600, Copilot and Agentic AI Developer. Ship with agents working beside you.
AI-500, Multi-Agent AI Solutions Expert. Orchestrate systems of agents at scale.

Plotting the trail on the Microsoft AI stack

Six exams, six layers of the Microsoft AI stack. I mapped each exam to its layers, in its study guide exercises, and its center of gravity.

The fundamentals exam touches every layer at concept depth. AI-103 lives in models, infrastructure, context, and the agentic platform. AI-200 moves down into infrastructure and data. AI-300 sits on infrastructure and the evidence plane, because operating models in production is mostly monitoring and evaluation. Meanwhile, the GitHub exams live at the top, where developers meet agents in the editor.

AI-500 is the interesting one. It spans context, the agentic platform, and governance. It does not test models at all. In fact, three of its five headline topics belong to the plane most stack diagrams leave out.

That was the moment the two pictures agreed. My diagram says the hard part of the Microsoft AI stack is context and evidence, not model choice. Microsoft’s expert exam tests context and evidence, not model choice. I did not expect a certification roadmap to confirm an architecture opinion, but here we are.

Where this is the wrong answer

Do not read the trail as a ladder you must climb in order. If you already run agents in production, AI-500 reflects your work, and AI-901 will teach you nothing. Platform engineers should look at AI-200 and AI-300 first. Developers should start with the GitHub exams.

Also, do not read my layer mapping as Microsoft’s. It is my reading of the study guides, and beta study guides move.

Finally, do not confuse the certificate with the evidence. Passing AI-500 shows you know what a control looks like. It does not produce the audit trail for your agent. That still takes Purview configured, Entra Agent ID issued, traces flowing to Log Analytics, and someone signing off. The exam is a map of the work. The work is still the work!

Credits

The Azure icons come from the official Azure architecture icon set. Product marks belong to their owners. Microsoft’s announcements are on the Skills Hub blog: Multi-Agent AI Solutions Expert, AI Apps and Agents Developer Associate, and GitHub Agentic AI Developer. The stack diagram builds on a five-layer picture that circulated on LinkedIn; the split data layer and the governance plane are my additions.

Designing a Cosmos DB Agent Memory Schema

Posted on September 1, 2026 by steefjan1970

Post 2 of 6 on Cosmos DB agent memory schema design before you write a line of agent code.

Post 1 made the case for one database instead of three. This post is about the decisions that determine whether that one database actually holds up: partition key and item shape, both of which you choose before an agent ever writes a turn. Get these wrong, and no amount of DiskANN or 99.999% SLA saves you from a hot partition or a rewrite-the-whole-item cost curve later.

Partition Keys: The Foundation of a Cosmos DB Agent Memory Schema

Because Cosmos DB automatically partitions data, the partition key is the single most consequential choice in the schema. It decides how writes distribute, how queries scope, and eventually how much a bad decision costs to unwind. Three strategies cover most agent memory scenarios, and each trades distribution for locality differently.

Partition Keys

GUID as the partition key. Every item lands in its own logical partition. Writes distribute as evenly as possible, so this works well for high-volume, write-heavy logging where you rarely need to reassemble a conversation later think raw telemetry more than chat history. The cost shows up on read: reconstructing a thread means a cross-partition query.
threadId as the partition key. All turns in a conversation share one partition key, so “give me the last 10 turns” or “vector search within this thread” both stay inside a single partition. This is the default for conversational agents and RAG apps, provided threads are numerous and varied enough to avoid concentrating writes on a few hot values.
[tenantId, threadId] as a hierarchical partition key. This is where the series comes full circle: I covered hierarchical partition keys in Azure Cosmos DB’s Latest Performance Features back in 2023, for the same tenant-then-item pattern. Nothing about the mechanism changed for agent workloads: threads still colocate under their tenant, tenant-level queries still avoid scanning every partition, and Cosmos DB still sub-partitions past the 20 GB logical-partition ceiling the same way it always did. Only the data moving through it is new.

There’s no universally correct answer here, so match the strategy to the query pattern that matters most: threadId if “give me this conversation” dominates, GUID if raw write throughput dominates, hierarchical if tenant isolation is a governance requirement and not just a nice-to-have.

Three Ways to Shape the Memory Item

Partition key decides where data lives. Item shape decides what it costs to read and write once it’s there, and that’s the other half of a Cosmos DB agent memory schema.

One document per turn (recommended default). Each item holds a complete exchange: a user prompt, the agent’s reply, and any tool call in between, and carries threadId, turnIndex, and an embedding alongside it. Most single- and multi-agent apps default to this shape, because it balances a small, cheap-to-write item against enough context to be useful on its own. “Latest N turns” is a simple ORDER BY turnIndex query, and ttl can expire old turns individually instead of touching the whole thread.

One document per response. Every user message, agent reply, and tool result gets its own item, all sharing a threadId. This is the most granular option, useful when you need to embed and search every single utterance independently, but it multiplies item count and RU cost on read. It loses the natural “question and answer together” unit that a semantic cache wants.

One document per thread. The whole conversation lives in one item that grows with every append. Reading the full history is a single read, which sounds appealing until a long-running thread turns every new turn into a full-item rewrite. Treat this as an anti-pattern unless the thread is short and you can bound its length by design: a five-turn onboarding flow, maybe; an open-ended assistant conversation, no.

TTL as Memory Lifecycle Management

Short-term memory should disappear on its own, and time-to-live is how Cosmos DB does that without a cleanup job. Set a default ttl on the container, and every item expires after that many seconds unless it overrides the value itself; set the container default to -1 instead, and Cosmos DB turns on ttl without expiring anything, unless an item sets its own positive ttl field. That second mode is the more useful one for agent memory, since it lets long-term memories (ttl: -1 on the item, meaning never expire) sit in the same container as short-term turns (ttl: 3600, gone in an hour) without a separate container or a background job doing the deleting.

A Worked Schema

Here’s the turn-based model from the previous section as an actual container and item.

Create the container with a hierarchical partition key, and turn on ttl at the container level:

			
az cosmosdb sql container create \
  --account-name my-cosmos-account \
  --database-name agentmemory \
  --name turns \
  --resource-group my-rg \
  --partition-key-path "/tenantId" "/threadId" \
  --ttl -1

		

A single turn, ready to insert:

			
{
  "id": "b9c5b6ce-2d9a-4a2b-9d76-0f5f9b2a9a91",
  "tenantId": "contoso",
  "threadId": "thread-1234",
  "turnIndex": 7,
  "messages": [
    { "role": "user", "content": "What's our refund policy for accessories?" },
    { "role": "agent", "content": "Refund policy is 30 days for unopened items." }
  ],
  "embedding": [0.013, -0.092, 0.551],
  "ttl": 3600
}

		

And writing it with the Python SDK:

			
from azure. cosmos import CosmosClient
client = CosmosClient(url, credential)
container = client.get_database_client("agentmemory").get_container_client("turns")
container.upsert_item({
    "id": "b9c5b6ce-2d9a-4a2b-9d76-0f5f9b2a9a91",
    "tenantId": "contoso",
    "threadId": "thread-1234",
    "turnIndex": 7,
    "messages": [
        {"role": "user", "content": "What's our refund policy for accessories?"},
        {"role": "agent", "content": "Refund policy is 30 days for unopened items."},
    ],
    "embedding": [0.013, -0.092, 0.551],
    "ttl": 3600,
})

		

Swap ttl: 3600 for ttl: -1 on any item you want to keep past the container default as summarized long-term memory; for instance, it survives while the rest of the thread ages out on schedule.

Seeing It Live

The rest of this section shows the same schema running against a real Cosmos DB account, not just described on paper.

The provision.py creating the turns container with the [/tenantId, /threadId] hierarchical partition key and container-level ttl.

The same container in Data Explorer’s Scale & Settings pane partition key and ttl exactly as provision.py set them.

A seeded turn in Data Explorer’s Items view tenantId, threadId, turnIndex, messages, and embedding, matching the JSON above field for field.

Pitfalls

Hot partitions from low-cardinality tenant keys. A hierarchical [tenantId, threadId] key only distributes well if tenants themselves are numerous and reasonably balanced in volume. One enterprise customer generating 80% of total traffic under a single tenantId value creates a hot partition no amount of threadId variety underneath it fixes. Check tenant volume distribution before committing to this key, not after.

Unbounded thread-per-item growth. It’s tempting to reach for one-document-per-thread because “just read the whole conversation” feels simpler in application code. In practice, an item that grows by one append per turn racks up RU cost on every single write as the item gets larger, and Cosmos DB item size limits eventually cap how long a thread can run at all. If a thread’s length isn’t predictable and short, don’t model it this way.

Forgetting embeddings need to be top-level fields. If you nest an embedding array inside a messages object, it won’t be indexable for vector search — it has to sit at the top level of the item, alongside threadId and turnIndex, for the container’s vector policy to pick it up. This one is easy to miss because the item still writes successfully; it just never shows up in a vector query, and that failure is silent until someone notices recall is worse than expected.

Next: Finding the Right Memory

The schema in this post gets data into Cosmos DB efficiently. It doesn’t yet get the right memory back out at the moment an agent needs it — that’s the job of vector, full-text, and hybrid search, which is where post 3 picks up. That settles the Cosmos DB agent memory schema; retrieval is the next problem worth solving properly.

Sources

Microsoft Learn — Agent memories in Azure Cosmos DB for NoSQL
Microsoft Learn — Time to live (TTL) in Azure Cosmos DB.
Microsoft Learn — Hierarchical partition keys
Your own archive — Azure Cosmos DB’s Latest Performance Features
GitHub — steefjan1/cosmos-agent-memory-lab — the runnable sample behind this schema, and behind posts 3 and 4

Don’t Build Around Today’s Model. Build for the AI Control Plane

Posted on August 31, 2026 by steefjan1970

Microsoft isn’t just shipping AI models. It is building toward a model-agnostic AI control plane. The bigger story behind the recent Microsoft Foundry updates is not another model launch. It is the architecture direction. Once you see it, the individual announcements fall into place as layers of one stack:

AI Control Plane → Model Router → Agents → RAG/Knowledge → MCP/Tools → Enterprise Systems

The AI control plane takes shape

Two August updates make the direction concrete. First, the model router update expanded the router to 28 regions for global standard and 21 data zone regions. The supported pool now includes Anthropic Claude Opus 4.8 and the GPT-5.6 family, while deprecated models such as DeepSeek-V3.1 and the gpt-5-chat line were pruned. Second, Foundry added DeepSeek-V4-Flash-0731 and NVIDIA Nemotron 3.5 Lightning to the catalog, each available through multiple deployment paths: Direct from Azure, Fireworks on Foundry, or the Hugging Face collection on managed compute.

I covered the router update in more detail on InfoQ, including the caveats around behavioral stability. Here, I want to focus on what the updates mean for your architecture.

Neither update is spectacular on its own. Together, however, they show a platform where models arrive, improve, and retire underneath a stable endpoint. Your application keeps calling the same integration while the pool refreshes. That is control plane behavior, not model shipping.

Agent ≠ Model

Here is the key idea for enterprise architects: an agent is not a model. An agent is an orchestration unit with instructions, knowledge, and tools. The model is a swappable dependency underneath it.

The model router makes that separation operational. It selects a model per request, optimizing for quality, cost, and latency within the regions your governance allows. You can run it in balanced, quality, or cost mode, and you can restrict routing to an approved subset of models. Moreover, every response includes a model field that shows which model handled the request, so the routing decisions leave an auditable trail.

Microsoft frames this as a hill climb: model selection as a continuous, measured loop rather than a one-time decision. In an ecosystem where the frontier moves monthly, a hardcoded model choice goes stale fast. An agent bound to the control plane instead of a specific model can evolve without a rebuild every time the leaderboard changes.

What this means for your architecture

For enterprise architects, the practical guidance follows directly. Treat model selection as configuration, not code. Put governance, observability, and policy at the control plane layer, because that is where they survive model churn. Ground agents in your own knowledge through RAG, and connect them to enterprise systems through MCP and tools. Consequently, each layer can evolve at its own pace. The stack outlives any single model.

Where this is the wrong answer

Model-agnostic routing is not free, and it is not always right. If your workload requires reproducible behavior, for example in regulated decision flows, a pinned model version beats a router that refreshes its pool automatically. Your evaluations were run against a specific model; a silent pool refresh invalidates them until you re-run. There are operational caveats too: Anthropic models still need to be deployed separately before the router can reach them, and routing modes take time to propagate. Finally, a single well-tuned small model per agent remains the simpler option for narrow, high-volume tasks. Routing adds a layer you must monitor. Only accept that cost when model diversity actually pays for itself.

Closing thoughts

The direction is clear even where the details will shift. Models are becoming interchangeable parts. The durable investment is the control plane: routing, governance, evaluation, and the connective tissue to your enterprise systems. So don’t build around today’s model. Build for the control plane, and let the models come and go.

What’s your approach: one model per agent, or model-agnostic agents?

Why Cosmos DB Ends Up as the Agent Memory Database

Posted on August 30, 2026 by steefjan1970

The first post in a series on Cosmos DB agent memory for AI agents, starting nine years before “AI agent” was a category.

In 2017, I built a proof of concept for a customer: a knowledge base on Cosmos DB, using the Graph model and Search, running at roughly 1,000 euros a month. I presented it at CloudBrew. One attendee wasn’t impressed:

“The most uninteresting talk of the day came from Steef-Jan Wiggers, who, in my opinion, delivered an hour-long marketing pitch for CosmosDB. I think it’s expensive for what it currently offers, and many developers could architect something with just as much performance without needing CosmosDB.”

He wasn’t wrong that 1,000 euros a month raises eyebrows as a line item. He was wrong about what the line item paid for: the knowledge base was the product a subscription business planned to sell. Compare the cost to the revenue it enabled, and it’s negligible. Compare it to nothing, and of course it looks “expensive.” I made the same point about Figma’s AWS bill last year: $109 million a year sounds alarming until you check it against $821 million in revenue and a business model that requires sub-100ms real-time collaboration for 13 million users. In short, cost without context is just a number that sounds big.

So here’s the same argument, nine years later, with a different workload. Agent memory, chat turns, tool call results, embeddings, and user preferences are expensive to store the wrong way and reasonably cheap to store the right way, and increasingly “the right way” means one database instead of three. I’ll come back to the actual RU numbers in post 6; for now, this post is about why the architecture argument holds up before cost even enters the picture.

The Same Shape of Problem, Nine Years Apart

Strip away “AI agent” and look at what you’re actually storing: short-lived, high-volume, time-ordered records that need fast writes and selective recall. That’s chat turns and tool outputs today. It’s also, structurally, what I modeled in a Cosmos DB Conf 2023 talk on end-to-end retail process monitoring, messages and batches flowing between an ERP, a WMS, and a PIM system, which I tracked so a retailer could tell where something broke.

Different domain, same shape, though: append-heavy writes, a need to reconstruct “what happened, in order,” and a downstream system (an incident manager then, an LLM now) that needs the right slice of history on demand, not the whole history every time.

In general, agent memory falls into two categories:

Short-term (episodic/working) memory — the last 5–10 turns of a conversation, intermediate tool call results, partial task state. Useful for the current task, disposable afterward (Cosmos DB’s time-to-live feature is a natural fit here; more on that in post 2).
Long-term memory — user preferences, summarized threads, facts the agent should persist and recall across sessions.

Both need somewhere to live, but the default answer for the last few years has been: somewhere different.

Why the Stitched Stack Breaks Down for Cosmos DB Agent Memory

The common pattern from 2022 through 2025 was to give each concern its own database: an in-memory store for caching and session state, a relational database for operational data and conversation logs, a purpose-built vector database for embeddings. A reasonable instinct, in theory: each tool for its own job.

In practice, though, it doesn’t hold up once an agent is the thing reading and writing across all three, on every turn.

Each piece has a real weakness once agents are the workload, not an afterthought:

Pure vector databases tend to offer no strong read/write guarantees, limited ingestion throughput, availability below 99.9%, a single (eventual) consistency level, and thin multitenancy support. Fine for an embeddings side-project. Shaky as the record of what an agent told a customer.
Relational databases fight the fluid, evolving schema of agent state, new fields, new memory types, and nested tool outputs without migrations and, often, downtime.
In-memory caches are fast and don’t persist, which is exactly the opposite of what long-term memory needs.

As a result, three systems also means three consistency models, three availability profiles, and three places a multi-agent system can silently desynchronize. And that complexity tax doesn’t show up in any single service’s bill, which is part of why it’s easy to miss until something breaks in production.

The Unified Case — and How Much of It I’d Already Used

The pitch for Cosmos DB agent memory as a unified layer rests on a small set of properties: single-digit-millisecond latency, a 99.999% availability SLA on the NoSQL API, DiskANN-based vector indexing built into the same store as the operational data, multi-master writes, and five selectable consistency levels from strong to eventual. In plain terms, that’s one system that’s fast enough for the hot path, available enough for production, and flexible enough to hold embeddings next to the record they came from.

In fact, two of the pieces that make this work aren’t new to me, or new to this blog.

Change feed, for example. In the retail monitoring solution, change feed was the mechanism that turned a write into a trigger: a new record landing in Cosmos DB fired a Function, which could raise an incident. That’s the same primitive I’ll use in post 4 to coordinate handoffs between agents in a multi-agent system: one agent’s write becomes another agent’s signal to act, without polling.

Hierarchical partition keys, likewise. I covered these in Azure Cosmos DB’s Latest Performance Features back in 2023: partitioning by tenant, then by item, to keep related data colocated while avoiding the 20 GB logical partition ceiling. The mechanism hasn’t changed; what’s changed is the workload. Post 2 uses the same [tenantId, threadId] pattern to isolate one customer’s agent conversations from another’s.

Even so, I didn’t build either feature for AI agents. Both turned out to be exactly what agent memory needs a decent sign that the underlying database was solid before the AI use case arrived, and nobody retrofitted it to fit.

Where This Series Is Headed

This post is the framing argument for Cosmos DB agent memory. From here, the rest of the series gets specific:

Post 2 — designing the agent memory schema itself: partition key choice, TTL, and the turn-based data model that works best in practice.
Post 3 — vector, full-text, and hybrid search for recalling the right memories, not just any memories.
Post 4 — multi-agent state and coordination, including change feed as the handoff mechanism.
Post 5 — wiring Cosmos DB into Microsoft Foundry Agent Service as bring-your-own thread storage.
Post 6 — the cost conversation, properly this time: RU drivers, semantic caching, and what this actually costs to run at scale.

The 2017 knowledge base cost 1,000 euros a month and paid for itself many times over as a revenue-generating product. Ultimately, the question worth asking about agent memory infrastructure in 2026 isn’t “is this expensive”; it’s the same question it always was: expensive relative to what?

Sources

Microsoft Learn — AI agents in Azure Cosmos DB
Microsoft Learn — Time to live (TTL) in Azure Cosmos DB.
Your own archive — Azure Cosmos DB’s Latest Performance Features
Your own archive — Figma AWS Costs Explained: Beyond the Hype and Panic
Microsoft Learn Shows — Leveraging Azure Cosmos DB for End-to-End Monitoring of Retail Processes

When Agentic Workloads Break the PaaS Assumptions

Posted on August 27, 2026 by steefjan1970

This series started with a map and grew into seven pieces. Five layers came first: compute, where load shape picks the service; messaging and orchestration, where two questions replace four product choices; data patterns, where idempotency and the outbox keep a platform correct; governance and identity, where policy and audit become the compliance posture; and observability and FinOps, where behaviour and cost become visible. Then came the lens: from design to demonstrable operation, the shift from “is it built?” to “can we operate it responsibly?”

Every one of those pieces rests on a shared assumption. The system does what you told it to do. You wrote the workflow, you defined the routes, you set the policies, and the platform executes them. That assumption has held for every integration platform I’ve built. Agentic workloads break it. So this capstone asks what changes when the thing making decisions inside your platform is a model, not your code, and why each layer, plus the readiness lens itself, deserves a second look because of it.

The assumption agentic workloads break

Conventional integration is deterministic. A message arrives, a workflow runs its defined steps, a router sends it where the rules say. You can read the code and know what will happen. You can test every path. When something fails, you trace it to a step you wrote.

Agentic workloads replace part of that determinism with a model that decides at runtime. The agent reads context, picks a tool, interprets the result, and chooses the next action. Moreover, it does so differently depending on inputs you didn’t fully anticipate. That’s the point of it: the flexibility is the feature. But it means you can no longer read the code and know what will happen. So the ground under every layer shifts: behavior is no longer exactly what you specified.

None of this argues against agentic workloads. It argues for revisiting each layer with the shift named explicitly. Let’s do that.

Compute: the loop changes the shape of the work

The compute layer sorted workloads by load shape: steady request traffic to App Service, event-driven bursts to Functions or Container Apps. Agentic workloads add a shape that sorting didn’t account for: the loop.

An agent doesn’t process a request and return. It reasons, calls a tool, waits, observes, and reasons again, sometimes for many cycles, before it finishes. That’s neither a clean request-response nor a discrete event. Instead, it’s a long-running loop of unpredictable duration with external calls in the middle. So the compute question changes. You’re no longer asking “steady or bursty” alone. You’re asking how to host something that runs for seconds or minutes, holds state across tool calls, and scales on a dimension concurrent reasoning loops that CPU-and-memory autoscale captures poorly. Container Apps with event-driven scaling often fit better here than App Service, and the orchestration frequently belongs in a workflow engine rather than raw compute.

Messaging and orchestration: the agent is a non-deterministic router

The messaging layer drew a clean line. Deterministic routing rules sent messages where the logic dictated. An agent orchestrating tool calls is, in effect, a router too, but a non-deterministic one. It decides which tool to call from its reading of the context, not from a rule you wrote.

The reliability consequences are real. Delivery guarantees still matter; an agent that triggers a business action still needs that action to occur exactly once, so Service Bus and the idempotency store in the data layer remain as relevant as ever. What changes is predictability. You can’t fully anticipate which actions the agent will trigger, or in what order. Therefore, the orchestration has to stay correct under sequences you didn’t design for. One practical lesson from building these loops applies directly: agent outputs rarely arrive as the clean structures a deterministic step would emit, so you build explicit bridges between agent actions rather than assuming shape.

Data: state and correctness under non-determinism

The data patterns held a platform correct when systems it didn’t control misbehaved. Agentic workloads make those patterns more necessary, not less, and they add one more.

Idempotency matters more because an agent may retry a tool call or repeat an action as it reasons, so the dedup store carries a heavier load. The outbox matters just as much, because an agent-triggered write still has to propagate reliably. Workflow state matters more too, since the reasoning loop is exactly the kind of long-running, restart-surviving process that needs durable state and a correlation ID. And then the new one: conversation and context state. An agent carries context across turns, and that context has to live somewhere durable and queryable, which explains why a flexible document store keeps showing up as the default for agentic conversation state. The access pattern points at the store. Same principle as the map, applied to a new kind of state.

Governance and identity: where the assumptions break hardest

This layer changes most, and I’d insist any integration architect think it through before shipping an agentic workload.

The governance layer secured a deterministic platform. Identity answered who the caller was; policy constrained what the platform could be. Both still matter. However, agentic workloads open a gap that neither fully closes. Identity secures who the agent is. It does not touch what a poisoned tool result or a manipulated retrieved document makes the agent do. Prompt injection rides in through the data the agent requested inside the reasoning loop, downstream of the perimeter check everyone assumes protects them.

So the governance layer needs additions a deterministic platform never required:

Authorization moves per-action: A validated identity at the edge isn’t enough. Each tool call the agent makes needs its own check: is this specific action allowed for this tenant right now? The perimeter check happens once; the risk recurs on every call inside the loop.
Recovery means compensation, not retry: Agent actions have side effects across systems. A failed sequence three actions deep can’t restart from the top; it needs compensating actions to undo what already happened. That’s saga-style thinking, and you design it; it doesn’t emerge.
Containment has to be possible: When an agent misbehaves, you stop it fast, and at more than one layer. Layered containment, from a single configuration flip-up to a full block, turns “contain the agent” from an incident-call debate into a seconds-long operation.
Evaluation becomes a first-class layer: Operational observability tells you the agent is running. It doesn’t tell you the agent’s outputs are quietly degrading. Under the EU AI Act’s oversight and transparency duties, that stops being optional polish and becomes evidence you’re meeting an obligation.

The readiness lens, asked again

The design-to-operation post posed the question that decides go-live: not “is it built?” but “can we operate it safely, recoverably, auditably, and predictably?” Agentic workloads sharpen every word of that sentence.

Safely now includes per-action authorization and containment, because the threat walks in as data. Recoverably now means compensation and sagas, because retry alone can’t undo side effects. Auditably now covers what the agent accessed, which tool it called, why it acted, and what policy constrained it evidence the EU AI Act increasingly expects. And predictably is precisely the property the agent gave up, which is why the surrounding architecture has to supply it instead. The production baseline, the demonstrable-versus-designed test, the three moments of readiness all of it still applies. Each bar sits higher.

The revised framework

The map closes with five questions. For agentic workloads, they hold, and each gains a harder edge. Before an agentic workload goes near a real system, I’d add these:

Can I host a long-running reasoning loop, not just a request or an event? Can my orchestration stay correct when I can’t predict the action sequence? Does my data layer hold conversation state as well as business state, with idempotency doing heavier duty? Is authorisation per-action, not just per-identity? Can I contain a misbehaving agent in seconds? And can I evidence what the agent did, why, and whether its quality held?

Those aren’t different questions from the series. They’re the same layers, asked again under non-determinism, and then held up against the readiness lens one more time.

The shape of it

Agentic workloads don’t replace the Azure PaaS foundation an integration architect builds on. They stress it. Every layer in this series still includes compute, messaging, data, governance, and observability, but each one now supports a workload that decides for itself at runtime. The compute layer meets the loop. The messaging layer meets a non-deterministic router. The data layer meets conversation state and heavier idempotency. The governance layer meets a threat that walks in through the front door as data. And the readiness lens meets a workload that surrendered predictability, so the architecture has to supply it.

The through-line of the whole series holds here too. The model is the least differentiated part of a production agent. What separates a demo from something you can run against real systems in a regulated industry is the architecture around it: the same layers, asked harder, and proven in operation rather than promised in design. So the foundation was never wasted. It’s exactly what agentic workloads need, applied with the assumptions made explicit.

That’s the series. Start at the Azure PaaS map for the layer-by-layer foundation, take the design-to-operation lens with you as the test, and come back here for what changes when the workload thinks for itself.

Azure Functions Behind API Management: What the Happy Path Diagram Leaves Out

Posted on August 26, 2026 by steefjan1970

Recently, I noticed another Azure architecture infographic on LinkedIn. Four boxes, left to right: clients, API Management, Function App, backend services. Underneath it, a tidy line. Function App handles the code, API Management handles the control. Together they deliver powerful and secure APIs.

(Source: LinkedIn post)

Nothing in that diagram is wrong. However, it’s not the whole story.

I have reviewed this pattern several times, in reference architectures, in project designs, and in my own work. The four boxes are always right and the design behind them is often not. The gap sits in what the arrows imply rather than in what the boxes say. So let us redraw it, and then walk through the six things the four box version quietly leaves out.

What the original gets right

Credit where it belongs. The separation of concerns in that infographic is sound, and plenty of teams still get it backwards.

API Management owns the contract. It publishes the API, applies policy, meters consumption, and gives consumers something stable to build against. Azure Functions owns the work. It runs your business logic, scales with demand, and bills for execution rather than for uptime.

That split matters because the alternative is worse. Teams that skip the gateway end up implementing authentication, throttling, and versioning inside every function, in slightly different ways, maintained by whoever touched it last. I wrote about a related boundary problem in Azure Functions, Logic Apps, and Power Automate: choosing the right tool, and the same principle applies here. The value of a gateway is not the features it lists. The value is the code it lets you delete.

So the boxes are right. Now for the arrows.

Gap one: a gateway is not a firewall

The original diagram lists “Security (OAuth, API Key)” inside the API Management box and leaves it there. That single bullet does a lot of quiet work, because it invites you to treat the gateway as your perimeter.

API Management applies policy. It validates tokens, enforces quotas, transforms payloads, and rejects malformed requests against a schema. It does not run an OWASP rule set, and it is not designed to absorb a volumetric attack aimed at your public hostname.

For anything internet facing, put Azure Front Door Premium or Application Gateway in front, with a WAF policy attached. Then lock the origin so the gateway only accepts traffic that arrived through the edge. Otherwise you have bought a policy engine and called it a perimeter.

This is also the cheapest gap to close, which makes it the most annoying one to find missing during a penetration test.

Gap two: subscription keys meter, tokens authenticate

Here is the conflation that causes the most damage in practice. A subscription key and an access token appear side by side in most diagrams, as if they were two ways of doing the same thing.

They are not. A subscription key identifies a product. It answers the question “which consumer agreement does this call belong to”, which is a billing and quota question. It does not tell you who the caller is, it travels in a header that ends up in scripts and log files, and it is shared across everyone using that product.

Authentication happens in the validate-jwt policy. That is where you check the signature, the audience, the issuer, and the claims that decide whether this particular caller may create an order.

			
<validate-jwt header-name="Authorization" failed-validation-httpcode="401">
  <openid-config url="https://login.microsoftonline.com/{{entra-tenant-id}}/v2.0/.well-known/openid-configuration" />
  <audiences>
    <audience>{{orders-api-audience}}</audience>
  </audiences>
  <required-claims>
    <claim name="roles" match="any">
      <value>Orders.Write</value>
    </claim>
  </required-claims>
</validate-jwt>

		

Once that is in place, the rate limit should follow the same identity. Throttling by IP address punishes everyone behind a corporate NAT and protects you from nobody who has a laptop and patience. Throttling by a claim from the validated token gives you a quota per caller, which is what the consumer actually agreed to.

			
<rate-limit-by-key calls="60" renewal-period="60"
  counter-key="@(context.Request.Headers.GetValueOrDefault("Authorization","").AsJwt()?.Claims.GetValueOrDefault("oid", "anonymous"))" />

Gap three: the arrow nobody draws

Now for the one that matters most.

Every version of this diagram shows one arrow into the Function App. That arrow implies the gateway is the way in. It is not. It is a way in.

An HTTP triggered function sits on a public hostname by default. Anyone holding that hostname and a function key can call it directly, and that call skips the WAF, the token validation, the rate limit, the audit trail, and every policy you carefully wrote. The gateway becomes a convention rather than a control, and conventions do not survive contact with an incident.

Teams close this in three stages, usually in this order.

Layer one, the function key. Store the host key as a secret named value in API Management, backed by Key Vault, and let the backend inject it as x-functions-key. The key never appears in your policy file or your repository. This stops crawlers and casual discovery. It does not stop anyone who has ever seen the key, and keys have a way of ending up in Postman collections, log files, and support tickets.

Layer two, token validation on the function itself. Turn on the built-in authentication, point it at the app registration that represents your API, and an untokened call now fails at the platform before your code runs. Better. Still not closed, because a valid token replayed straight at the function hostname bypasses everything the gateway added on top.

Layer three, take it off the internet. Set publicNetworkAccess to Disabled, put a private endpoint in front of the app, and integrate the gateway into the same virtual network. The hostname stops resolving from outside. Now there is genuinely one route in.

			
resource functionApp 'Microsoft.Web/sites@2023-12-01' = {
  properties: {
    publicNetworkAccess: networkIsolation ? 'Disabled' : 'Enabled'
    virtualNetworkSubnetId: networkIsolation ? functionSubnetId : null
  }
}

		

Layer three is the only one that turns your architecture diagram into a statement about reality. It also has a prerequisite that catches people out, which is the next gap.

Gap four: the tier decides the architecture

Private networking is not a checkbox you add at the end. It is gated by the API Management tier and by the Functions hosting plan, and those two choices constrain everything else in the design.

The v2 tiers changed the arithmetic here. Outbound virtual network integration used to mean Premium, which put private connectivity out of reach for a lot of internal APIs. Standard v2 brought it within reach of an ordinary project budget. On the compute side, the Flex Consumption plan brought virtual network integration to a consumption billing model, which used to mean choosing between cost and connectivity.

The same is true of time. Both the gateway and the function cap how long a request may run, and those caps differ by tier and plan. If your design assumes a two minute synchronous call, you have made a tier decision without realising it.

Pick the tier and the hosting plan before you pick the features, and verify the current limits on Microsoft Learn rather than trusting a diagram. Both sides of this moved several times during 2025 and 2026, and anything I write here has a shelf life.

Gap five: the dashed response arrow assumes synchronous

Look at the original infographic again. The response arrow is dashed and runs straight back to the client. It quietly assumes every call finishes inside the request.

Plenty do not. Bulk imports, report generation, anything that fans out to a slow downstream system. The instinct is to raise the timeout, first on the function, then on the gateway, until the whole chain waits for the slowest possible caller. That is not a fix. It is a queue with worse ergonomics, and it fails under load rather than under test.

The pattern that works is the asynchronous HTTP API. Accept the request, start the work, and answer immediately.

			
var instanceId = await durableClient.ScheduleNewOrchestrationInstanceAsync(
    nameof(BulkImportOrchestrator), orders);
var response = request.CreateResponse(HttpStatusCode.Accepted);
response.Headers.Add("Location", $"{request.PublicBaseUrl()}/bulk/{instanceId}");
response.Headers.Add("Retry-After", "5");

		

One detail deserves attention, because it connects back to gap three. Durable Functions ships a helper called CreateCheckStatusResponse that builds the polling URLs for you. Those URLs point at the function hostname. That leaks your backend to every caller, and it breaks the moment you disable public network access.

So build the status URL from a setting that holds the gateway address instead. Callers should never learn the name of your function app, and they certainly should not be given it in a header.

Gap six: one request, two telemetry stores

Both boxes in the diagram write to Application Insights, usually to two separate resources with two separate sampling configurations. Neither picture shows that, because telemetry is drawn as a property rather than as a system.

The result shows up during your first real incident. You have a gateway trace that ends at the backend call and a function trace that starts somewhere in the middle, and no reliable way to join them. Then somebody discovers that the gateway sampled at one rate and the function at another, so half the pairs do not exist at all.

Two habits fix this. Set a correlation id at the gateway when the caller did not supply one, and propagate it through the function into every log line.

			
<set-variable name="correlationId"
  value="@(context.Request.Headers.GetValueOrDefault("x-correlation-id", context.RequestId.ToString()))" />
<set-header name="x-correlation-id" exists-action="override">
  <value>@((string)context.Variables["correlationId"])</value>
</set-header>

		

Then align the sampling on both sides, and write the query that joins them before you need it rather than during an outage. I made a similar argument about governed telemetry in the Foundry Citadel Platform series, where an SDK routed its calls around the gateway entirely and the only reason we noticed was that the traces did not add up.

Where this is the wrong answer

Everything above assumes you need API Management. Often you do not.

A gateway earns its place when there is a portfolio to govern. Several APIs, consumers outside your own team, versions that have to coexist, quota that differs per consumer, one place where policy and audit live. Under those conditions API Management pays for itself in the coordination it removes.

For a single internal API with one consumer, it is a monthly bill wrapped around a proxy. Built-in authentication with Entra ID on the function, a private endpoint if it is internal, and you are done. Add the gateway when the second consumer actually appears, because that is when versioning and per consumer quota start to matter.

There is also a middle answer that gets skipped. One public API that needs a WAF but has no consumer lifecycle should sit behind Front Door and nothing else. You get the perimeter without buying governance you are not using yet.

A sample you can deploy

I put the whole thing in a repository, because policy fragments in a blog post are easy to agree with and harder to run.

One azd up deploys API Management Standard v2, a Flex Consumption function app on .NET 8 isolated, a storage account with shared key access disabled, and a shared Application Insights resource. The Orders API has a synchronous endpoint, an asynchronous bulk import built on Durable Functions, and a health endpoint the gateway can probe.

The three layers from gap three are deployment switches rather than prose:

Setting	What it turns on
default	Layer one, function key held as a secret named value
`ENTRA_TENANT_ID` and `API_AUDIENCE`	Layer two, `validate-jwt` and quota by `oid` claim
`NETWORK_ISOLATION=true`	Layer three, private endpoint and public access disabled

Deploy it with the default settings, then call the function hostname directly and watch it answer. That single curl makes the argument better than this whole post does.

Closing

The four box diagram is a good summary and a bad specification. Function App handles the code, API Management handles the contract, and neither of them handles the network. That last part is where these designs usually fail, and it is the part no infographic ever draws.

Production Readiness: Closing the Execution Gap

Posted on August 25, 2026 by steefjan1970

Every post in this series so far has covered a layer: compute, messaging and orchestration, data patterns, governance and identity, observability and FinOps. This one isn’t a layer. It’s a question that cuts across all of them: when is the platform actually ready for production?

That question is harder than it looks, because “we built it” and “we can run it” are different claims. I’ve watched more than one integration platform pass every technical checkpoint and still fall short of production-ready, not because the design missed anything, but because nobody had turned that design into something enforceable, operable, and provable. So this post is about the gap between those two states, and how you close it. It’s the through-line under every layer, and it’s the thing that turns a strong architecture into a platform you can responsibly put load on.

The shift: from “is it built?” to “can we operate it?”

Here’s the single most useful reframe I know for this stage. Stop asking “is the platform technically built?” and start asking “can we operate it safely, recoverably, auditably, and predictably?”

Those are not the same question. The first is about whether the components exist and connect. The second is about whether, when something goes wrong at 2 am, someone can see what happened, understand it, recover from it, and prove afterward that they handled it correctly. A platform can pass the first test comfortably and fail the second completely. And the second test is the one that actually determines whether you should go live. So the moment you catch yourself saying “it works,” push on: does it work in a demo, or does it work under a failure you didn’t plan for?

The core move: make the implicit explicit

Most integration platforms at this stage share the same shape. The design is good, and someone has largely written it down. But a lot of what matters lives in documentation, in code that’s still evolving, or in the heads of the people who built it. That works fine while a small team builds the foundation. It stops working the moment the first real production use case lands, because implicit choices become whatever the first team happens to decide.

The fix is a production baseline: an explicit, enforceable statement of what production use demands. It names the mandatory components and patterns, pins down the environment profiles, fixes the security controls and monitoring standards, and settles the recovery agreements and release criteria. Its job is to pull those decisions out of documents and habits and into something the platform enforces, so the first production use case inherits the decisions rather than reinventing them.

Without that baseline, every early integration is free to make its own choices, and you accumulate inconsistency, technical debt, and a future re-platforming you didn’t budget for.

Design versus demonstrable: the recurring gap

The same gap shows up in every layer, and once you see it, you can’t unsee it. On security, the design names Zero Trust, least privilege, and pipeline-driven change, yet permanent broad access rights sit in the environment, quietly contradicting it. Observability design specifies OpenTelemetry and required fields, but the alert rules and dashboards never make it into the infrastructure. And the CI/CD design describes a full release chain with quality gates, while the pipelines really only cover the dev environment.

In each case the design is right and the demonstrable working is missing. That’s the pattern to hunt for when you assess readiness: not “did someone design this?” but “does the platform actually enforce this design somewhere it can prove?” Anything that lives only as intent is a gap, however good the intent.

What operational readiness actually covers

Technical function is necessary but not sufficient. Operational readiness asks a distinct set of questions, and that set decides go-live. In practice, it comes down to whether you can demonstrably answer these:

Can you see what’s happening: monitoring, chain-level tracing, message-level insight? When an incident hits, does someone triage it, and do they have the information they need to do so? For recovery, can the platform handle errors, replay from a known point, and fall back on a real Business Continuity and Disaster Recovery plan rather than an RTO written in a document? Afterward, can you prove what happened through an audit trail, an access log, and change history? On access control, do you separate permanent from elevated rights, and dev from production? For cost, can you attribute it, budget against it, and tier retention? And finally, can you hand it over — have you defined operational ownership, or does every incident route back to the people who built it?

If any of those answers is “only in the design,” please fix it before go-live, not after the first incident teaches you the hard way.

The boundary that decides scale: central versus decentral

The other thing production-readiness has to settle is a boundary, not just a checklist. Most modern integration platforms want value-stream teams to deliver independently within central guardrails. That’s the right ambition. But it only works when you draw the line between central platform ownership and team autonomy deliberately, because that line runs through every layer: API governance, messaging configuration, RBAC, pipeline use, monitoring, error handling, lifecycle, support.

Get the line wrong and you recreate the exact problem the platform set out to solve. The platform team becomes the bottleneck again — the single point through which every change, approval, and incident has to pass. So team autonomy isn’t just a tooling question. It needs explicit ownership agreements, release paths, access models, quality controls, and operational responsibilities. The tooling enables autonomy; the agreements make it safe.

Shared components are where this bites hardest. A shared API gateway, a shared message broker, a shared logging workspace these touch technology, security, governance, operations, cost, and autonomy all at once. They make the platform economical, and they carry the biggest scaling risk. For each one, decide deliberately: why does it stay shared rather than isolated, who owns it, who may change it, how do you monitor it, and how do you attribute its cost? Leave those implicit and the shared component quietly becomes everyone’s dependency and no one’s responsibility.

Readiness isn’t one moment — it’s three

The last reframe worth making: “ready” isn’t a single bar. It’s three different bars at three different moments, and conflating them is how platforms either over-build early or under-prepare for scale.

First go-live. The bar here is the minimum production baseline and demonstrable operational readiness. Not every capability has to be complete, but the ones that are preconditions for running safely in production do. This is where the baseline, the recovery plan, and the release criteria have to be real.

First team onboarding. The bar shifts to whether the federated model actually works in practice. Can one real team deliver independently within the central guardrails, without quality, security, or consistency buckling under the first real use? This is a practice test, and it’s better, even, to let some things get concrete here rather than designing them fully in the abstract.

Scaling to many teams. Now the bar is repeatability. Anything that worked at one team through direct conversation now has to become standardised, documented, and reproducible: lifecycle policy, cost allocation, onboarding, support model, versioning, exception handling. Direct alignment doesn’t scale; product-steering does.

Naming which moment a given concern belongs to is half the battle. It stops you from demanding scale-grade rigour before first go-live, and from discovering at team five that nobody built the repeatable version.

Where this thinking gets over-applied

Consistent with the series, the honesty section. “Production baseline” thinking is right, but it can tip into paralysis.

Not everything has to be complete before first go-live. The three-moments split exists precisely so you don’t. Demanding full lifecycle policy, mature FinOps, and a complete federation model before a single use case runs is how a platform never ships. Match the rigour to the moment.

A baseline that only flags is a baseline that gets ignored. The whole point of the production baseline is that the platform enforces it. A pile of documented-but-unenforced standards manufactures the appearance of readiness without the substance, which is more dangerous than an honest gap, because it invites false confidence.

You can make a decision deliberately without making it central. Drawing the central-versus-decentral line carefully doesn’t mean pulling everything central. Sometimes the deliberate call is “this is team-owned,” and recording that reasoning is the point, not the direction.

The shape of it

For an integration architect, production-readiness isn’t a technical checkpoint; it’s the shift from a platform that’s built to one you can operate responsibly. Make the implicit explicit in a production baseline. Hunt the gap between what’s designed and what’s demonstrable. Answer the operational-readiness questions before go-live, not after. Draw the central-versus-decentral line deliberately, especially for shared components. And treat “ready” as three moments, not one. Do that, and the layers from this series stop being a good architecture on paper and become a platform you can actually run.

This is the lens that ties the series together. The Azure PaaS map has the layer-by-layer foundation; this post is the question you hold every layer up against before you put production load on it.

Azure Functions vs. Logic Apps vs. Power Automate: When to Use What

Posted on August 21, 2026 by steefjan1970

If you work anywhere near the Microsoft ecosystem, you have probably run into all three of these services. You have probably also run into the confusion around them. They all “automate” something. They all show up in architecture conversations. On the surface, their marketing pages sound almost interchangeable.

This post continues the Cloud Perspectives Azure PaaS series. It follows recent entries on Azure Functions as a serverless agents runtime and managed identity in Logic Apps Standard. Those posts went deep on one service. This one steps back and compares all three. The usual shorthand, Functions for code, Logic Apps for integration, Power Automate for business users, is cleaner than reality.

In practice, Functions is a capable integration tool in its own right. Logic Apps’ headline B2B/EDI capability comes bundled with an extra resource and its own bill. Power Automate is not even an Azure product. Picking the wrong tool does not just produce a clunkier solution either. It can mean months of maintenance pain, licensing costs nobody budgeted for, or a workflow that cannot scale. Here is a more honest breakdown of how the three differ, and when each one earns its place in your architecture.

Where each service actually lives

Before comparing capabilities, it helps to see where these tools sit organizationally. That placement drives billing, governance, and who owns the resource day to day.

Azure Functions and Logic Apps are Azure resources. You provision them in the Azure portal, under an Azure subscription, next to your virtual machines and storage accounts. Platform teams building governance models, like the ones described in Azure governance and identity for integration architects, treat them accordingly.

Power Automate is different. It is licensed through Microsoft 365 and the Power Platform admin center. That difference is not a footnote. It determines which admin center you open when something breaks, and which budget line absorbs the cost.

Azure Functions: built for developers who want full control

What it is

Azure Functions is Microsoft’s serverless compute service. You write code in C#, Python, JavaScript, TypeScript, Java, PowerShell, and more. It runs in response to an event. That event might be an HTTP request, a new file landing in Blob Storage, a message hitting a queue, or a timer firing. You never manage the underlying servers. You simply ship functions and let Azure handle the scaling.

It is also a first-class integration tool

Functions gets typecast as “the compute one” while Logic Apps gets credited with “integration.” That framing sells Functions short. Its HTTP triggers and rich set of bindings include Service Bus, Event Grid, Cosmos DB, and Blob Storage. Those let a function sit in the middle of a system-to-system exchange just as naturally as a Logic App can.

For stateful, long-running orchestration across multiple systems, the exact scenario people usually reach for Logic Apps for, Durable Functions provides that same pattern in code. You get full testability and source control with it. For developers building the kind of serverless orchestration covered in Azure Functions as a serverless agents runtime, Functions is a legitimate path for integration work, not a fallback.

When to use it

Reach for Functions when you need custom logic, complex calculations, or heavy data transformation that a visual designer cannot express cleanly. Reach for it when you want total control over code, dependencies, and third-party libraries. It also fits when you are building microservices and APIs that must perform well under load. It also fits when you are doing systems integration and would rather express it in code than in a visual designer.

Typical use cases: processing images uploaded to Blob Storage, powering a custom REST API for a mobile or web app, running scheduled jobs, handling real-time IoT telemetry, and orchestrating multi-step integration workflows through Durable Functions.

Trade-offs

Functions requires real programming skills. Cold starts on the Consumption plan can add latency to infrequent workloads. You also own more of the maintenance and security surface than a managed workflow tool would give you.

On the upside, Functions is usually the cheapest of the three at scale. The Consumption plan includes a substantial monthly free grant, around one million executions and 400,000 GB-seconds. You only pay for what you actually run.

Logic Apps: built for enterprise-grade integration

What it is

Logic Apps is Azure’s platform-as-a-service for orchestrating workflows across systems. Think of it as the enterprise integration layer. It gives you a visual designer, so it sits at a lower code level than Functions. Even so, it targets IT and integration teams rather than casual business users. Logic Apps shines when you need to connect many systems reliably, at scale, with proper DevOps practices wrapped around it. That is the kind of governance discussed in Azure data patterns for integration architects.

When to use it

Reach for Logic Apps when you need to integrate multiple systems, spanning cloud, on-premises, and SaaS, with formal reliability and monitoring requirements. It also fits B2B or EDI-style exchanges over AS2, X12, or EDIFACT. And it fits any scenario where you want a pay-per-execution model that supports high-volume enterprise workflows without managing infrastructure yourself.

Typical use cases: syncing a CRM like Salesforce with an on-premises SQL database, exchanging invoices with trading partners using industry-standard protocols, and orchestrating responses to Azure alerts or resource deployments.

The Integration Account catch

One nuance is easy to gloss over. The B2B/EDI capability is not something a plain Logic App gives you out of the box. It requires provisioning a separate resource, an Integration Account, to store trading partners, agreements, schemas, and certificates. You then link that account to your Logic App.

Integration Accounts carry their own tiered pricing across Free, Basic, Standard, and Premium levels. In other words, “use Logic Apps for B2B/EDI” really means “use Logic Apps plus an Integration Account.” That adds both cost and an extra resource to manage, something a lot of comparisons leave out entirely.

Trade-offs

Logic Apps requires an active Azure subscription and has a steeper learning curve than Power Automate. Unlike Power Automate, it also has no built-in desktop or RPA automation. Billing runs on trigger, action, and connector executions, which can add up faster than an equivalent Functions workload.

Still, you get native Visual Studio and Git integration, strong monitoring, and no hard execution limits. All of that matters at enterprise scale.

Power Automate: built for business users who need speed

What it is

Power Automate is the low-code, no-code member of the trio, built for productivity rather than infrastructure. It lets business users, not developers, automate day-to-day tasks such as approvals, notifications, report generation, and data syncing between Microsoft 365 apps.

It is not actually Azure

Here is a distinction worth making explicit, since the three tools so often get lumped together as “Azure services.” Power Automate is not an Azure service. It belongs to the Microsoft Power Platform, licensed alongside Power Apps, Power BI, and Copilot Studio. That licensing typically runs through Microsoft 365 plans, standalone per-user or per-flow licenses, or a free tier, not an Azure subscription.

The one exception is pay-as-you-go licensing, which lets you bill Power Automate usage against an Azure subscription as an alternative payment mechanism. Even so, that is a billing convenience, not evidence that Power Automate lives in Azure.

Functions and Logic Apps are Azure resources you provision in the Azure portal, under an Azure subscription, alongside your VMs and storage accounts. Power Automate, by contrast, is a Microsoft 365 offering that happens to interoperate with Azure resources through connectors. That is a real architectural distinction, not just a licensing footnote. It affects who owns the resource, where governance sits, and which admin center you troubleshoot in when something breaks.

When to use it

Reach for Power Automate when you want to boost individual or team productivity without writing code. It also fits when you need to automate UI-based tasks on legacy software through desktop flows, essentially RPA. It fits too when your workflow lives mostly inside Microsoft 365, across Outlook, Teams, SharePoint, and similar apps.

Typical use cases: routing document approvals through Teams and Outlook, using desktop flows to pull data out of an old legacy application, and automatically saving email attachments to a SharePoint folder.

Trade-offs

Power Automate is the fastest option to deploy for non-developers, and it integrates tightly with the Power Platform. That said, licensing can get expensive at scale, debugging and version control stay limited compared to code-first tools, and performance throttles kick in on high-volume runs.

A quick rule of thumb, caveats included

If the job needs raw coding power, fine-grained control, or code-first integration, reach for Azure Functions. That includes integration work. Do not rule it out just because Logic Apps carries the “integration” label.
If the job needs visual, governed workflow orchestration across systems, reach for Logic Apps. Budget for an Integration Account on top if EDI or B2B is involved.
If the job needs a fast, no-code fix for a Microsoft 365-centric business process, reach for Power Automate. Account for it as Power Platform or M365 licensing rather than an Azure cost.

The real power comes from combining them

These three are not really competitors. They are layers. A common pattern looks like this: Power Automate handles the front-end business process, say a Teams approval flow. That triggers a Logic App to orchestrate the broader integration across systems. The Logic App, in turn, calls an Azure Function to run the custom logic or heavy computation that neither low-code tool expresses well.

Used this way, each tool does the part it is actually good at. Power Automate handles speed and accessibility. Logic Apps handles governed integration at scale. Azure Functions handles anything that needs real code. The mistake is not choosing one of these. It is assuming you have to choose only one.

Azure Observability and FinOps for Integration Architects

Posted on August 20, 2026 by steefjan1970

In the Azure PaaS map post, observability was folded into the governance layer, with a note that Application Insights and Azure Monitor are non-negotiable. That was true, but it undersold the topic. For an integration platform specifically, observability isn’t a sub-bullet of governance. It’s the layer that decides whether you can actually run the thing in production.

So this post pulls observability out and gives it room. And it brings FinOps along, because the two share a root: you can’t manage what you can’t see. One makes system behaviour visible; the other makes cost visible. Both turn a platform from “it runs” into “we can run it responsibly.” Azure observability and FinOps, treated together, are what separate a platform that works in a demo from one you can operate under real load.

The gap between design and demonstrable operation

Here’s the pattern I see most often on integration platforms. The observability design is excellent. There’s a logging standard, a tracing approach, a set of required fields. Then you look at the actual infrastructure, and none of it is enforced. The alert rules aren’t there. The dashboards aren’t built. The diagnostic settings aren’t wired. The design lives in a document; the platform doesn’t know about it.

That gap matters more than it sounds. A monitoring standard that depends on discipline and review isn’t a platform capability; it’s a hope. The moment a team ships an integration without the dashboards, the standard quietly failed. So the real work in this layer isn’t designing observability. It’s making observability demonstrable: wired into the infrastructure, enforced in the pipeline, and impossible to skip.

Let’s walk what that means in practice.

OpenTelemetry as a platform contract, not a suggestion

Most mature integration platforms land on OpenTelemetry as the instrumentation standard. That’s the right call. W3C Trace Context propagates a trace across services, traces and metrics and logs share a model, and you avoid inventing your own correlation scheme. So far, so good.

The catch is that “we use OpenTelemetry” is a design statement, not an enforced one. For it to be a contract, three things must be true. First, the required fields, resource attributes, trace fields, and domain identifiers have to be defined explicitly, not left to each team’s judgment. Second, that definition has to be validated somewhere automatically, ideally at pull request. Third, the platform components themselves have to emit the standard, so a trace actually runs unbroken from the API gateway through messaging to the backend. Miss any of those, and you have telemetry that mostly correlates, which is worse than none, because it looks trustworthy right up until the incident where it isn’t.

Tracing the chain, not just the components

Azure gives you per-resource monitoring for free. You can see API Management’s metrics, Service Bus’s queue depth, and a Function’s execution count. That’s component monitoring, and it’s necessary but not sufficient. An integration platform’s job is to move a message across those components, so the question that matters is whether you can follow a single message or transaction through the entire chain.

That end-to-end view has to map onto the layers of your integration architecture, because each layer asks a different question. The consumer-facing layer cares about availability, latency, error rates, and throttling per channel. The process layer cares about routing, transformations, retries, and failures in async steps. The system-facing layer cares about dependencies on backends’ response times, timeouts, and contract breaks. Without that layered, chain-aware view, you get plenty of technical detail per Azure resource and almost no ability to reason about the integration as a whole.

Message-level insight and the async recovery problem

Component metrics tell you the platform is busy. They don’t help the person who has to answer “what happened to order 47821?” For that, an operator needs message-level insight: business identifiers, error categories, chain status, the last successful step, and retry state. Structured logging with domain attributes a flow ID, a message ID, a route key, and an error category is what makes that possible. And it has to come with explicit data classification, masking, retention, and access rules, because business identifiers in logs are exactly the kind of data a regulator asks about.

Then there’s recovery, which is where the compute choice comes back to bite. Async, message-driven processing needs a replay story: when something fails partway through, you need to know how far it got and re-drive it from there. A workflow engine often gives you some of this out of the box. Raw compute like Functions doesn’t, so you have to design the replay mechanism yourself, as part of the integration pattern rather than an afterthought.

The pattern that works: treat the message on the bus as a reference, not the full payload. Pair it with the claim-check pattern, in which the bus carries technical and functional metadata: trace ID, flow ID, message ID, route key, error category, retry count, and a pointer to the payload, safely stored in storage. Define checkpoints along the flow. Then, on failure, you can determine where processing succeeded and re-drive from the right point, with idempotency (from the data patterns post) making the re-drive safe. For fully synchronous request-response, re-driving belongs with the caller; the platform’s job there is clear error codes and traceability.

Monitoring as a Definition of Done

The single highest-leverage move in this layer costs almost nothing: make monitoring a Definition of Done for every integration. No integration ships without its dashboard, its alerts, its trace-context propagation, its required log fields, its retention setting. And this is the part that turns it from aspiration into capability: the checklist runs as a quality gate in the pipeline, not as a line in a review someone might skip.

That one change moves observability from “depends on the discipline of whoever built it” to “the platform won’t let you skip it.” It’s the difference between a standard and an enforced standard, and it’s the cheapest high-value thing on this entire list.

FinOps: cost is just another signal you can’t yet see

Everything above is about making system behavior visible. FinOps is the same discipline applied to cost. On an integration platform, it fails in the same way because cost visibility typically ends at the subscription or resource group boundary. That’s too coarse. It can’t tell you what an individual integration costs, or an API, or a queue, or a team’s share of a shared component.

Three FinOps problems come up on every integration platform:

Attribution needs a taxonomy. Without a consistent tagging scheme for value stream, team, environment, integration, API, owner, and cost category, cost remains a lump sum. With one, you can steer on cost per integration product rather than cost per subscription. This is the foundation; nothing else works without it.
Shared components are the hard part. Compute is easy to attribute when each team runs its own. But a shared API Management instance, a shared Service Bus namespace, a shared Log Analytics workspace those get used by everyone and billed centrally, and if you never build a distribution model, nobody owns the cost. The shared components that make the platform economical are exactly the ones whose cost is hardest to place. That’s not a reason to isolate everything; it’s a reason to deliberately decide the split.
Storage and retention are FinOps levers hiding within a compliance requirement. Observability generates data logs, traces, payloads held for replay, and dead-lettered messages. Compliance dictates how long you keep it. But retention length and storage tier are separate decisions. Data you must keep for audit doesn’t have to sit in a hot, queryable tier the whole time. Tie retention to data classification, then move cold data to cheaper tiers. The requirement is “keep it”; the FinOps move is “keep it cheaply.”

The through-line: FinOps on an integration platform isn’t financial reporting after the fact. It’s a design and governance concern, sitting right next to observability, because both are about seeing what the platform is actually doing.

Where this layer gets over-applied

Consistent with the series, the honesty section. Observability and cost control both have a failure mode of doing too much.

Not every signal deserves an alert. An alert that fires on something nobody acts on trains people to ignore alerts. Alert on what changes a decision; leave the rest on a dashboard. Alert fatigue is a real operational risk, not a sign of thoroughness.

Not every message needs full payload logging. Metadata-first is the right default. Payload logging belongs where there’s functional need and explicit consent, with masking and retention — not everywhere, because “log everything” is how sensitive data ends up somewhere it shouldn’t, and how your storage bill quietly triples.

Not every cost needs fine-grained attribution. Building per-message cost tracking for a low-volume internal integration spends more effort than the insight is worth. Match the granularity of attribution to the scale of the spend.

The shape of it

For an integration architect, observability and FinOps answer the same question in two currencies: what is the platform actually doing, and what is it actually costing? Wire OpenTelemetry in as an enforced contract. Trace the chain, not just the components. Give operators message-level insight and a real replay story. Make monitoring a Definition of Done the pipeline enforces. Then apply the same visibility to cost: a tagging taxonomy, a distribution model for shared components, and retention tiered by classification. Get both right, and the platform stops being a black box you hope is behaving and becomes one you can actually operate.

Want the layer this sits inside? The Azure PaaS map puts observability and governance in context against compute, integration, and data, and walks the five-question framework across all of them.