Finding the Right Memory: Vector, Full-Text, and Hybrid Search in Cosmos DB

Posted on September 3, 2026 by steefjan1970

Post 3 of 6 on Cosmos DB agent memory search, because storing memory well doesn’t guarantee you’ll retrieve the right piece.

Post 2 ended with the schema settled and retrieval still open. This post closes that gap: the practical mechanics of Cosmos DB agent memory search, one container, four query patterns. Run the same question four different ways against that container, and it comes back with four different answers, because “find the right memory” isn’t one query pattern; it’s at least three, and knowing which one to reach for is most of the job.

Vector Indexing for Cosmos DB Agent Memory Search

Cosmos DB supports two vector index types, and the right one depends almost entirely on how many vectors you’re searching, not on anything specific to agents.

quantizedFlat compresses each vector and scans the compressed space exactly. It suits smaller workloads (tens of thousands of vectors) and trades a small amount of accuracy for lower RU cost and faster scans. For a single tenant’s short-term memory, this is often enough on its own.

DiskANN, on the other hand, indexes vectors for approximate nearest-neighbor search and scales to hundreds of thousands or billions of embeddings, with dynamic updates and strong recall even at that size. Post 1 already leaned on DiskANN as part of the case for Cosmos DB as a unified store; this is the mechanism behind that claim.

Sharding the Vector Index for Multitenant Isolation

DiskANN doesn’t have to search across every vector in the container. A vectorIndexShardKey partitions the index itself by a property you choose: session, user, or tenant, so a query only searches candidates within that shard instead of the whole container.

That maps directly onto the partition key work from post 2: set the vectorIndexShardKey to tenantId, or to the same [tenantId, threadId] pair you already use as the partition key, and semantic search for one tenant never touches another tenant’s vectors. A global, unsharded index still works and makes searching everything at once simpler, but it’sonly appropriate for a single-tenant app or a genuinely shared knowledge base where cross-tenant recall is the point rather than a leak.

Full-Text Search: When Precision Beats Semantics

Vector search finds what’s semantically similar. Sometimes semantically similar isn’t what you want — a customer asking about “the refund policy” needs the actual refund policy language, not five conceptually related passages about returns in general.

Full-text search on Cosmos DB handles that case through BM25, a statistical ranking function that scores by term frequency and document length. Cosmos DB applies linguistic processing automatically: tokenization, stemming, case normalization, so “running” still matches “run” or “ran.” It’s the right tool whenever exact terms or phrases carry meaning that a vector embedding would blur.

Hybrid Search: Combining Both with RRF

Most agent memory queries don’t need to choose between semantic and lexical relevance; they need a blend of both. That’s what Reciprocal Rank Fusion (RRF) does: it takes the vector-similarity ranking and the BM25 ranking for the same result set and merges them into one combined rank, instead of forcing a pick between the two.

In practice, this shows up as a single ORDER BY RANK RRF(...) clause, which the next section demonstrates directly.

Four Ways to Ask the Same Question

Take the turn-based schema from post 2 — tenantId, threadId, turnIndex, messages, embedding, content and run the same underlying question against it four ways. (content is a flat, denormalized copy of the turn’s text, added specifically because Cosmos DB doesn’t support wildcard array paths like /messages/*/content in a full-text policy or index the full-text and hybrid queries below point at c.content rather than c.messages for exactly that reason.)

Most recent, by recency:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
ORDER BY c.turnIndex DESC

Semantic, by vector similarity:

			
SELECT TOP 5 c.messages, VectorDistance(c.embedding, @queryVector) AS score
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
ORDER BY VectorDistance(c.embedding, @queryVector)

Hybrid, blending both with RRF:

			
SELECT TOP 5 c.messages, VectorDistance(c.embedding, @queryVector) AS score
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
ORDER BY VectorDistance(c.embedding, @queryVector)

Keyword, by exact phrase:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = @tenantId AND c.threadId = @threadId
  AND FULLTEXTCONTAINS(c.content, @phrase)
ORDER BY c.turnIndex DESC

		

Run all four against a thread where a customer asked about refunds three times, in different words, across twenty turns, and the differences stop being theoretical fast: recency surfaces whichever turn happened most recently, even if it’s off-topic; semantic search pulls in every conceptually related turn, including the ones that used different words entirely; hybrid balances the two; keyword search returns only the turns that used the customer’s actual phrase, and ranks them by recency underneath that filter.

Running These Queries in Data Explorer

The four queries above use parameterized SQL, the same form search.py, from the companion repo behind this series, sends through the Python SDK, which binds @tenantId, @queryVector, and @phrase properly before the query runs. Paste them as-is into the Azure Portal’s Data Explorer query pane instead, and two things break, neither of which is a schema or code bug:

Data Explorer’s query box doesn’t bind named parameters. A query that leaves @tenantId unresolved either matches nothing and returns “No results” silently, or for VectorDistance() inside ORDER BY and FullTextScore() fails to compile outright, because both functions require their arguments to resolve to literal values at query-compile time rather than at execution time.

Swap every @parameter for a literal value and all four run cleanly. Against the seeded sample data (tenantId = "contoso", threadId = "thread-1234", searching for "refund"):

Recency, with literals:

			
SELECT TOP 5 c.messages, c.turnIndex 
  FROM c WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234" 
  ORDER BY c.turnIndex DESC

Semantic, with literals:

			
SELECT TOP 5 c.messages, VectorDistance(c.embedding, [0.8196, 0.6392, -0.2471, 0.1608, -0.8667, 0.4902, -0.2549, 0.2235]) AS score
FROM c
WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234"
ORDER BY VectorDistance(c.embedding, [0.8196, 0.6392, -0.2471, 0.1608, -0.8667, 0.4902, -0.2549, 0.2235])

Hybrid, with literals:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234"
ORDER BY RANK RRF(
    VectorDistance(c.embedding, [0.8196, 0.6392, -0.2471, 0.1608, -0.8667, 0.4902, -0.2549, 0.2235]),
    FullTextScore(c.content, "refund")
)

		

Keyword, with literals:

			
SELECT TOP 5 c.messages, c.turnIndex
FROM c
WHERE c.tenantId = "contoso" AND c.threadId = "thread-1234"
  AND FULLTEXTCONTAINS(c.content, "refund")
ORDER BY c.turnIndex DESC

		

Pitfalls

Reaching for DiskANN on a small dataset. DiskANN’s approximate search and sharding options solve a scale problem. Below roughly ten thousand vectors, quantizedFlat gets equivalent recall for less operational complexity and lower RU cost. Default to DiskANN because it sounds like the “serious” choice, and you’ve added index-shard decisions to a workload that never needed them.

A global vector index in a multitenant app. Skip the vectorIndexShardKey, and a semantic query searches every candidate in the entire container, tenant boundaries or not. Nothing stops the query from surfacing another tenant’s conceptually similar memory in the result set unless a WHERE clause happens to filter it back out after the fact, and relying on a filter to catch what the index itself should have scoped is the kind of gap that shows up in an audit, not in testing.

Forgetting WHERE filters still apply. Vector and hybrid queries look like they replace normal filtering, but ORDER BY VectorDistance(...) or ORDER BY RANK RRF(...) still runs inside a WHERE-scoped query, same as any other. Leave the WHERE c.tenantId = @tenantId AND c.threadId = @threadId clause off a semantic query, and it searches everything the container holds, not just the thread the agent is currently in.

Next: Coordinating Multiple Agents

That settles Cosmos DB agent memory search for a single agent working alone. Coordinating what several agents know about the same conversation is a different problem, and it’s where change feed, a mechanism post 1 already covered as a callback to the 2023 retail monitoring work, comes back to tie multi-agent state together. That’s post 4.

Sources

Microsoft Learn — Agent memories in Azure Cosmos DB for NoSQL
Microsoft Learn — Vector search in Azure Cosmos DB
Microsoft Learn — Sharded DiskANN (vectorIndexShardKey)
Microsoft Learn — Hybrid search in Azure Cosmos DB
GitHub — steefjan1/cosmos-agent-memory-lab — the runnable sample behind these four queries, and behind posts 2 and 4

Designing a Cosmos DB Agent Memory Schema

Posted on September 1, 2026 by steefjan1970

Post 2 of 6 on Cosmos DB agent memory schema design before you write a line of agent code.

Post 1 made the case for one database instead of three. This post is about the decisions that determine whether that one database actually holds up: partition key and item shape, both of which you choose before an agent ever writes a turn. Get these wrong, and no amount of DiskANN or 99.999% SLA saves you from a hot partition or a rewrite-the-whole-item cost curve later.

Partition Keys: The Foundation of a Cosmos DB Agent Memory Schema

Because Cosmos DB automatically partitions data, the partition key is the single most consequential choice in the schema. It decides how writes distribute, how queries scope, and eventually how much a bad decision costs to unwind. Three strategies cover most agent memory scenarios, and each trades distribution for locality differently.

Partition Keys

GUID as the partition key. Every item lands in its own logical partition. Writes distribute as evenly as possible, so this works well for high-volume, write-heavy logging where you rarely need to reassemble a conversation later think raw telemetry more than chat history. The cost shows up on read: reconstructing a thread means a cross-partition query.
threadId as the partition key. All turns in a conversation share one partition key, so “give me the last 10 turns” or “vector search within this thread” both stay inside a single partition. This is the default for conversational agents and RAG apps, provided threads are numerous and varied enough to avoid concentrating writes on a few hot values.
[tenantId, threadId] as a hierarchical partition key. This is where the series comes full circle: I covered hierarchical partition keys in Azure Cosmos DB’s Latest Performance Features back in 2023, for the same tenant-then-item pattern. Nothing about the mechanism changed for agent workloads: threads still colocate under their tenant, tenant-level queries still avoid scanning every partition, and Cosmos DB still sub-partitions past the 20 GB logical-partition ceiling the same way it always did. Only the data moving through it is new.

There’s no universally correct answer here, so match the strategy to the query pattern that matters most: threadId if “give me this conversation” dominates, GUID if raw write throughput dominates, hierarchical if tenant isolation is a governance requirement and not just a nice-to-have.

Three Ways to Shape the Memory Item

Partition key decides where data lives. Item shape decides what it costs to read and write once it’s there, and that’s the other half of a Cosmos DB agent memory schema.

One document per turn (recommended default). Each item holds a complete exchange: a user prompt, the agent’s reply, and any tool call in between, and carries threadId, turnIndex, and an embedding alongside it. Most single- and multi-agent apps default to this shape, because it balances a small, cheap-to-write item against enough context to be useful on its own. “Latest N turns” is a simple ORDER BY turnIndex query, and ttl can expire old turns individually instead of touching the whole thread.

One document per response. Every user message, agent reply, and tool result gets its own item, all sharing a threadId. This is the most granular option, useful when you need to embed and search every single utterance independently, but it multiplies item count and RU cost on read. It loses the natural “question and answer together” unit that a semantic cache wants.

One document per thread. The whole conversation lives in one item that grows with every append. Reading the full history is a single read, which sounds appealing until a long-running thread turns every new turn into a full-item rewrite. Treat this as an anti-pattern unless the thread is short and you can bound its length by design: a five-turn onboarding flow, maybe; an open-ended assistant conversation, no.

TTL as Memory Lifecycle Management

Short-term memory should disappear on its own, and time-to-live is how Cosmos DB does that without a cleanup job. Set a default ttl on the container, and every item expires after that many seconds unless it overrides the value itself; set the container default to -1 instead, and Cosmos DB turns on ttl without expiring anything, unless an item sets its own positive ttl field. That second mode is the more useful one for agent memory, since it lets long-term memories (ttl: -1 on the item, meaning never expire) sit in the same container as short-term turns (ttl: 3600, gone in an hour) without a separate container or a background job doing the deleting.

A Worked Schema

Here’s the turn-based model from the previous section as an actual container and item.

Create the container with a hierarchical partition key, and turn on ttl at the container level:

			
az cosmosdb sql container create \
  --account-name my-cosmos-account \
  --database-name agentmemory \
  --name turns \
  --resource-group my-rg \
  --partition-key-path "/tenantId" "/threadId" \
  --ttl -1

		

A single turn, ready to insert:

			
{
  "id": "b9c5b6ce-2d9a-4a2b-9d76-0f5f9b2a9a91",
  "tenantId": "contoso",
  "threadId": "thread-1234",
  "turnIndex": 7,
  "messages": [
    { "role": "user", "content": "What's our refund policy for accessories?" },
    { "role": "agent", "content": "Refund policy is 30 days for unopened items." }
  ],
  "embedding": [0.013, -0.092, 0.551],
  "ttl": 3600
}

		

And writing it with the Python SDK:

			
from azure. cosmos import CosmosClient
client = CosmosClient(url, credential)
container = client.get_database_client("agentmemory").get_container_client("turns")
container.upsert_item({
    "id": "b9c5b6ce-2d9a-4a2b-9d76-0f5f9b2a9a91",
    "tenantId": "contoso",
    "threadId": "thread-1234",
    "turnIndex": 7,
    "messages": [
        {"role": "user", "content": "What's our refund policy for accessories?"},
        {"role": "agent", "content": "Refund policy is 30 days for unopened items."},
    ],
    "embedding": [0.013, -0.092, 0.551],
    "ttl": 3600,
})

		

Swap ttl: 3600 for ttl: -1 on any item you want to keep past the container default as summarized long-term memory; for instance, it survives while the rest of the thread ages out on schedule.

Seeing It Live

The rest of this section shows the same schema running against a real Cosmos DB account, not just described on paper.

The provision.py creating the turns container with the [/tenantId, /threadId] hierarchical partition key and container-level ttl.

The same container in Data Explorer’s Scale & Settings pane partition key and ttl exactly as provision.py set them.

A seeded turn in Data Explorer’s Items view tenantId, threadId, turnIndex, messages, and embedding, matching the JSON above field for field.

Pitfalls

Hot partitions from low-cardinality tenant keys. A hierarchical [tenantId, threadId] key only distributes well if tenants themselves are numerous and reasonably balanced in volume. One enterprise customer generating 80% of total traffic under a single tenantId value creates a hot partition no amount of threadId variety underneath it fixes. Check tenant volume distribution before committing to this key, not after.

Unbounded thread-per-item growth. It’s tempting to reach for one-document-per-thread because “just read the whole conversation” feels simpler in application code. In practice, an item that grows by one append per turn racks up RU cost on every single write as the item gets larger, and Cosmos DB item size limits eventually cap how long a thread can run at all. If a thread’s length isn’t predictable and short, don’t model it this way.

Forgetting embeddings need to be top-level fields. If you nest an embedding array inside a messages object, it won’t be indexable for vector search — it has to sit at the top level of the item, alongside threadId and turnIndex, for the container’s vector policy to pick it up. This one is easy to miss because the item still writes successfully; it just never shows up in a vector query, and that failure is silent until someone notices recall is worse than expected.

Next: Finding the Right Memory

The schema in this post gets data into Cosmos DB efficiently. It doesn’t yet get the right memory back out at the moment an agent needs it — that’s the job of vector, full-text, and hybrid search, which is where post 3 picks up. That settles the Cosmos DB agent memory schema; retrieval is the next problem worth solving properly.

Sources

Microsoft Learn — Agent memories in Azure Cosmos DB for NoSQL
Microsoft Learn — Time to live (TTL) in Azure Cosmos DB.
Microsoft Learn — Hierarchical partition keys
Your own archive — Azure Cosmos DB’s Latest Performance Features
GitHub — steefjan1/cosmos-agent-memory-lab — the runnable sample behind this schema, and behind posts 3 and 4

Don’t Build Around Today’s Model. Build for the AI Control Plane

Posted on August 31, 2026 by steefjan1970

Microsoft isn’t just shipping AI models. It is building toward a model-agnostic AI control plane. The bigger story behind the recent Microsoft Foundry updates is not another model launch. It is the architecture direction. Once you see it, the individual announcements fall into place as layers of one stack:

AI Control Plane → Model Router → Agents → RAG/Knowledge → MCP/Tools → Enterprise Systems

The AI control plane takes shape

Two August updates make the direction concrete. First, the model router update expanded the router to 28 regions for global standard and 21 data zone regions. The supported pool now includes Anthropic Claude Opus 4.8 and the GPT-5.6 family, while deprecated models such as DeepSeek-V3.1 and the gpt-5-chat line were pruned. Second, Foundry added DeepSeek-V4-Flash-0731 and NVIDIA Nemotron 3.5 Lightning to the catalog, each available through multiple deployment paths: Direct from Azure, Fireworks on Foundry, or the Hugging Face collection on managed compute.

I covered the router update in more detail on InfoQ, including the caveats around behavioral stability. Here, I want to focus on what the updates mean for your architecture.

Neither update is spectacular on its own. Together, however, they show a platform where models arrive, improve, and retire underneath a stable endpoint. Your application keeps calling the same integration while the pool refreshes. That is control plane behavior, not model shipping.

Agent ≠ Model

Here is the key idea for enterprise architects: an agent is not a model. An agent is an orchestration unit with instructions, knowledge, and tools. The model is a swappable dependency underneath it.

The model router makes that separation operational. It selects a model per request, optimizing for quality, cost, and latency within the regions your governance allows. You can run it in balanced, quality, or cost mode, and you can restrict routing to an approved subset of models. Moreover, every response includes a model field that shows which model handled the request, so the routing decisions leave an auditable trail.

Microsoft frames this as a hill climb: model selection as a continuous, measured loop rather than a one-time decision. In an ecosystem where the frontier moves monthly, a hardcoded model choice goes stale fast. An agent bound to the control plane instead of a specific model can evolve without a rebuild every time the leaderboard changes.

What this means for your architecture

For enterprise architects, the practical guidance follows directly. Treat model selection as configuration, not code. Put governance, observability, and policy at the control plane layer, because that is where they survive model churn. Ground agents in your own knowledge through RAG, and connect them to enterprise systems through MCP and tools. Consequently, each layer can evolve at its own pace. The stack outlives any single model.

Where this is the wrong answer

Model-agnostic routing is not free, and it is not always right. If your workload requires reproducible behavior, for example in regulated decision flows, a pinned model version beats a router that refreshes its pool automatically. Your evaluations were run against a specific model; a silent pool refresh invalidates them until you re-run. There are operational caveats too: Anthropic models still need to be deployed separately before the router can reach them, and routing modes take time to propagate. Finally, a single well-tuned small model per agent remains the simpler option for narrow, high-volume tasks. Routing adds a layer you must monitor. Only accept that cost when model diversity actually pays for itself.

Closing thoughts

The direction is clear even where the details will shift. Models are becoming interchangeable parts. The durable investment is the control plane: routing, governance, evaluation, and the connective tissue to your enterprise systems. So don’t build around today’s model. Build for the control plane, and let the models come and go.

What’s your approach: one model per agent, or model-agnostic agents?

Why Cosmos DB Ends Up as the Agent Memory Database

Posted on August 30, 2026 by steefjan1970

The first post in a series on Cosmos DB agent memory for AI agents, starting nine years before “AI agent” was a category.

In 2017, I built a proof of concept for a customer: a knowledge base on Cosmos DB, using the Graph model and Search, running at roughly 1,000 euros a month. I presented it at CloudBrew. One attendee wasn’t impressed:

“The most uninteresting talk of the day came from Steef-Jan Wiggers, who, in my opinion, delivered an hour-long marketing pitch for CosmosDB. I think it’s expensive for what it currently offers, and many developers could architect something with just as much performance without needing CosmosDB.”

He wasn’t wrong that 1,000 euros a month raises eyebrows as a line item. He was wrong about what the line item paid for: the knowledge base was the product a subscription business planned to sell. Compare the cost to the revenue it enabled, and it’s negligible. Compare it to nothing, and of course it looks “expensive.” I made the same point about Figma’s AWS bill last year: $109 million a year sounds alarming until you check it against $821 million in revenue and a business model that requires sub-100ms real-time collaboration for 13 million users. In short, cost without context is just a number that sounds big.

So here’s the same argument, nine years later, with a different workload. Agent memory, chat turns, tool call results, embeddings, and user preferences are expensive to store the wrong way and reasonably cheap to store the right way, and increasingly “the right way” means one database instead of three. I’ll come back to the actual RU numbers in post 6; for now, this post is about why the architecture argument holds up before cost even enters the picture.

The Same Shape of Problem, Nine Years Apart

Strip away “AI agent” and look at what you’re actually storing: short-lived, high-volume, time-ordered records that need fast writes and selective recall. That’s chat turns and tool outputs today. It’s also, structurally, what I modeled in a Cosmos DB Conf 2023 talk on end-to-end retail process monitoring, messages and batches flowing between an ERP, a WMS, and a PIM system, which I tracked so a retailer could tell where something broke.

Different domain, same shape, though: append-heavy writes, a need to reconstruct “what happened, in order,” and a downstream system (an incident manager then, an LLM now) that needs the right slice of history on demand, not the whole history every time.

In general, agent memory falls into two categories:

Short-term (episodic/working) memory — the last 5–10 turns of a conversation, intermediate tool call results, partial task state. Useful for the current task, disposable afterward (Cosmos DB’s time-to-live feature is a natural fit here; more on that in post 2).
Long-term memory — user preferences, summarized threads, facts the agent should persist and recall across sessions.

Both need somewhere to live, but the default answer for the last few years has been: somewhere different.

Why the Stitched Stack Breaks Down for Cosmos DB Agent Memory

The common pattern from 2022 through 2025 was to give each concern its own database: an in-memory store for caching and session state, a relational database for operational data and conversation logs, a purpose-built vector database for embeddings. A reasonable instinct, in theory: each tool for its own job.

In practice, though, it doesn’t hold up once an agent is the thing reading and writing across all three, on every turn.

Each piece has a real weakness once agents are the workload, not an afterthought:

Pure vector databases tend to offer no strong read/write guarantees, limited ingestion throughput, availability below 99.9%, a single (eventual) consistency level, and thin multitenancy support. Fine for an embeddings side-project. Shaky as the record of what an agent told a customer.
Relational databases fight the fluid, evolving schema of agent state, new fields, new memory types, and nested tool outputs without migrations and, often, downtime.
In-memory caches are fast and don’t persist, which is exactly the opposite of what long-term memory needs.

As a result, three systems also means three consistency models, three availability profiles, and three places a multi-agent system can silently desynchronize. And that complexity tax doesn’t show up in any single service’s bill, which is part of why it’s easy to miss until something breaks in production.

The Unified Case — and How Much of It I’d Already Used

The pitch for Cosmos DB agent memory as a unified layer rests on a small set of properties: single-digit-millisecond latency, a 99.999% availability SLA on the NoSQL API, DiskANN-based vector indexing built into the same store as the operational data, multi-master writes, and five selectable consistency levels from strong to eventual. In plain terms, that’s one system that’s fast enough for the hot path, available enough for production, and flexible enough to hold embeddings next to the record they came from.

In fact, two of the pieces that make this work aren’t new to me, or new to this blog.

Change feed, for example. In the retail monitoring solution, change feed was the mechanism that turned a write into a trigger: a new record landing in Cosmos DB fired a Function, which could raise an incident. That’s the same primitive I’ll use in post 4 to coordinate handoffs between agents in a multi-agent system: one agent’s write becomes another agent’s signal to act, without polling.

Hierarchical partition keys, likewise. I covered these in Azure Cosmos DB’s Latest Performance Features back in 2023: partitioning by tenant, then by item, to keep related data colocated while avoiding the 20 GB logical partition ceiling. The mechanism hasn’t changed; what’s changed is the workload. Post 2 uses the same [tenantId, threadId] pattern to isolate one customer’s agent conversations from another’s.

Even so, I didn’t build either feature for AI agents. Both turned out to be exactly what agent memory needs a decent sign that the underlying database was solid before the AI use case arrived, and nobody retrofitted it to fit.

Where This Series Is Headed

This post is the framing argument for Cosmos DB agent memory. From here, the rest of the series gets specific:

Post 2 — designing the agent memory schema itself: partition key choice, TTL, and the turn-based data model that works best in practice.
Post 3 — vector, full-text, and hybrid search for recalling the right memories, not just any memories.
Post 4 — multi-agent state and coordination, including change feed as the handoff mechanism.
Post 5 — wiring Cosmos DB into Microsoft Foundry Agent Service as bring-your-own thread storage.
Post 6 — the cost conversation, properly this time: RU drivers, semantic caching, and what this actually costs to run at scale.

The 2017 knowledge base cost 1,000 euros a month and paid for itself many times over as a revenue-generating product. Ultimately, the question worth asking about agent memory infrastructure in 2026 isn’t “is this expensive”; it’s the same question it always was: expensive relative to what?

Sources

Microsoft Learn — AI agents in Azure Cosmos DB
Microsoft Learn — Time to live (TTL) in Azure Cosmos DB.
Your own archive — Azure Cosmos DB’s Latest Performance Features
Your own archive — Figma AWS Costs Explained: Beyond the Hype and Panic
Microsoft Learn Shows — Leveraging Azure Cosmos DB for End-to-End Monitoring of Retail Processes

Agentic AI Design Patterns on Azure

Posted on August 15, 2026 by steefjan1970

Like a lot of people on LinkedIn right now, I have seen no shortage of infographics laying out agentic AI design patterns: tidy boxes and arrows for Tool Use, ReAct, Reflection, Planning, Orchestrator, Sequential Chain, Parallel Fan-out/Fan-in, Hierarchical, and P2P Mesh. These diagrams are a fine way to get the shape of an idea across. However, a diagram cannot tell you whether the thing actually works, or what breaks when you try to stand it up for real. So I spent the last stretch actually building agentic AI on Azure: all nine patterns, for real, on Azure’s PaaS offerings. That meant writing Bicep IaC, running azd up, making actual HTTP calls against the deployed endpoints, and running actual Application Insights queries when things went wrong. In the end I had nine independently deployable reference samples, not nine slides.

Writing the Bicep was the easy part. Getting all nine actually to stand up and respond correctly was where the real lessons were. This is a retrospective on what broke, why, and what I’d tell someone else to do to avoid it.

Lesson 1: model availability is regional, and the error message won’t always tell you clearly

The first failure mode hit me repeatedly, across multiple patterns. azd up would warn that gpt-4.1 “was not found” in the target region, then fail validation outright with InvalidResourceProperties: The specified SKU 'Standard' for model 'gpt-4.1' is not supported in this region.

This is not a bug in the Bicep. It is genuinely true that not every Azure OpenAI model and SKU combination is available in every region, and availability also varies by subscription. centralus and westeurope both rejected gpt-4.1 for me at different points, while swedencentral consistently worked across every pattern in the repo. So if you are scripting or templating Azure OpenAI deployments, do not hardcode a region and assume it will work everywhere. Instead, treat azd env set AZURE_LOCATION swedencentral as a reflexive first move whenever a fresh pattern’s deploy attempt fails.

Lesson 2: Logic Apps Standard is powerful, and standing it up correctly is its own skill

The Sequential Chain pattern (input → extract → draft → validate → output, a fixed pipeline) is a textbook fit for a low-code workflow engine. So I built it on Logic Apps Standard first, and it ended up costing me, by a wide margin, the most debugging time of the whole project. Here is what went wrong, roughly in order:

ServiceProvider-type triggers cannot have a recurrence property. If you copy a recurrence block over from an ApiConnection-style trigger example, you get a WorkflowProcessingFailed error, and the message does not obviously point at the fix.
Connector parameter names are not always what you would guess. Azure literally names the built-in OpenAI connector’s connection parameter openAIEndpoint, not endpoint. I only discovered this from Azure’s own runtime error message, not by guessing at the schema.
Windows versus Linux hosting changes what app settings you need, and not symmetrically. Windows-hosted Premium plans require WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE, but carrying those same settings over to a Linux plan actively breaks content sync.
WorkflowStandard (WS1) is a hard requirement, not a suggestion. Azure explicitly rejects an ElasticPremium (EP1) substitution with “Logic Apps can be deployed only to ‘WorkflowStandard’ Sku or App Service Environments,” even though the two SKUs share the same underlying compute family.
Regional SKU quota is real, and it will not show up until you deploy. SubscriptionIsOverQuotaForSku for “WS1 VMs” hit me in two different regions. I eventually fixed it by decoupling the Logic App’s region from the rest of the pattern’s resources through a separate Bicep parameter, which let it land somewhere with quota headroom.
Deterministic role-assignment names collide across resource recreation. If you name a role assignment with guid(scope, resourceId, roleName) and then delete and recreate the resource whose managed identity receives that role, Azure refuses to update the existing assignment to point at the new principal ID. So you first have to find and delete the stale assignment by its exact resource ID.
Finally, after fixing all of the above, I ran into a Logic App instance that had simply wedged itself: Sequence contains no elements on startup, Kudu unreachable or crawling, and no combination of RBAC or config fixes helping. The only real fix was deleting and recreating the whole site resource.

Durable Functions

After all that, I rebuilt the same pattern on Durable Functions instead. An orchestrator calls three sequential activities, each a plain Azure OpenAI chat completion call, and the built-in RetryOptions does the retry work that Logic Apps’ declarative retry policies would otherwise have done. It deployed in under two minutes and worked on the first real test. The conceptual case for a low-code workflow engine still holds here, since a fixed sequence of stages is exactly what that kind of tool is for. However, if you want to move fast while building agentic AI on Azure, and you already have a working Durable Functions convention elsewhere in your stack, do not underestimate how much extra surface area Logic Apps Standard adds compared with staying in code you already know how to debug.

Lesson 3: RBAC is not one system, and Cosmos DB has its own

The P2P Mesh pattern (three Azure Functions reacting to each other’s events through Event Grid, with Cosmos DB tracking completion state) threw a 403 Forbidden: "Request blocked by Auth... does not have required RBAC permissions to perform action Microsoft.DocumentDB/databaseAccounts/readMetadata". This happened even though the function app’s managed identity already had every role I expected under Microsoft.Authorization/roleAssignments.

Here is the catch: Cosmos DB runs its own, separate, native RBAC system. A normal Azure role assignment against a Cosmos account grants control-plane access, but it does nothing for data-plane operations like reading or writing items. Instead, you need a Microsoft.DocumentDB/databaseAccounts/sqlRoleAssignments resource, scoped specifically to one of Cosmos’s own built-in role definitions. (The “Cosmos DB Built-in Data Contributor” role has a fixed, well-known GUID: 00000000-0000-0000-0000-000000000002.) So if your Bicep only includes the generic role-assignment resource type pointed at a Cosmos account, it silently grants nothing useful.

(Once I fixed that, the very next error was a 404 reading “Owner resource does not exist” on the database and container the app code expected. Provisioning the Cosmos account is not the same as provisioning the database and container inside it. Remember that if your Bicep stops at the account level.)

Lesson 4: pick the “native” endpoint type when Azure offers one

Still on P2P Mesh: wiring Event Grid subscriptions to Azure Functions endpoints through a raw webhook URL (https://<app>.azurewebsites.net/runtime/webhooks/eventgrid?functionName=X&code=<key>) produced a persistent 401 Unauthorized, Webhook endpoint validation failed. It survived two fixes I tried first: swapping the master key for the eventgrid_extension system key, and then directly inspecting the deployed function, which confirmed its eventGridTrigger binding matched what Event Grid expected. Neither fix mattered, because a plain authenticated GET against that same URL came back 400 Bad Request. That proved the key itself was fine, so the actual problem had to be something specific to Event Grid’s own validation handshake against a webhook-typed endpoint.

The actual fix was switching az eventgrid event-subscription create from a raw webhook URL to --endpoint-type azurefunction --endpoint <function-resource-id>. That is the native destination type Event Grid offers for Azure Functions, and it authenticates through the ARM resource ID instead of a function key embedded in a URL. It worked immediately. So the broader lesson: if a platform offers a first-class integration type for a specific target, prefer it over hand-rolling the equivalent with a generic webhook. That way, you inherit the platform’s own handling of edge cases you would not think of yourself.

Lesson 5: “Azure AI Foundry” isn’t one resource type

The Orchestrator pattern, a central agent hosted on Azure AI Foundry Agent Service that delegates to specialist tools, failed with a raw DNS error: Name or service not known, against a hostname the app was trying to reach. This was not an auth failure or a permissions failure. The hostname genuinely did not exist.

The root cause: the Bicep provisioned the older Azure AI Foundry resource model, a Microsoft.MachineLearningServices/workspaces hub plus a project workspace, which is the original Azure AI Studio approach. Meanwhile, the app code used Azure.AI.Projects.AIProjectClient, which only understands the newer, unified Foundry resource model: a Microsoft.CognitiveServices/accounts resource with kind: 'AIServices' and allowProjectManagement: true, plus a projects child resource. That newer model is reachable at a https://<account>.services.ai.azure.com/api/projects/<project> URL that only exists for that resource type. So two things share the “Azure AI Foundry” name, yet neither is a drop-in substitute for the other from an SDK’s point of view. As a result, it is worth explicitly checking which resource model your SDK version actually expects before wiring up IaC for it, because the error you get when you guess wrong will not obviously point at “you deployed the wrong kind of resource.”

The smaller stuff that added up

A few recurring, less dramatic gotchas worth a line each:

Double-check the resource group you think you are querying. More than once I pulled exceptions from the wrong pattern’s Application Insights instance, either because I was still in the wrong working directory, or because I grabbed the first App Insights component Azure CLI happened to return rather than the one for the resource group I actually cared about. So always sanity-check the resource group name before trusting a KQL result.
A previous session’s PowerShell variable can look like a fresh result. If a command throws partway through an assignment, the variable keeps its old value. Then, if that old value happens to look plausible (in my case, a valid-looking Durable Functions status payload from an entirely different pattern), it becomes easy to mistake it for real output from the command you just ran.
“No results” and “the backing store does not exist yet” are different failure modes. Code that gracefully handles an empty index or table does not automatically handle a missing one. For example, one specialist service in the Orchestrator pattern caught SqlException broadly and degraded gracefully when its sample table lacked seed data. By contrast, the AI Search-backed specialist had no equivalent guard for a 404 on a not-yet-created index, so it took the whole request down with it. Therefore, if you document “unseeded stores degrade gracefully,” actually test the unseeded case for every specialist, not just the one you happened to build first.

The takeaway: what building agentic AI on Azure actually requires

None of these were exotic failures. In fact, every one of them turned out to be a documented, well-understood Azure behavior once I found the right doc page. What made them expensive was that the error messages, on their own, rarely pointed straight at the fix: a DNS failure that was actually a resource-model mismatch, a 401 that was actually about endpoint type rather than credentials, and a 403 that was actually a second, unrelated RBAC system. So the practical lesson is not “Azure is fragile.” Instead, building agentic AI on Azure means dealing with more independently moving parts than the getting-started docs suggest, and the fastest way through is treating every unexplained error as worth one layer deeper of investigation before you assume the obvious fix is the real one.

I have now deployed and tested all nine patterns end to end. If you want the full breakdown of each one, including what it is for, the Azure architecture, and when to reach for it, start with the design patterns overview and go from there. The repo itself is public, so you can see exactly what each pattern provisions.

Multi-Agent Patterns in Azure Logic Apps: Handoffs, Orchestrators, and Sequential Loops

Posted on June 22, 2026 by steefjan1970

Part 5 of 7 in the Logic Apps Agent Loop series

Part 4 covered the three tooling layers available to an Azure Logic Apps agent. A single agent with well-defined tools handles a wide range of integration scenarios, but some workloads are too complex for one agent to handle well. Azure Logic Apps multi-agent patterns let you compose multiple agent loops into a coordinated system, where each agent has a single focused responsibility and the output of one feeds directly into the next. This post covers the four patterns Microsoft has defined and includes a working demo that builds a two-agent sequential loop.

This post covers the four patterns Microsoft has defined for multi-agent composition in Azure Logic Apps: prompt chaining, routing, handoff, and orchestrator-workers and includes a demo that builds a two-agent sequential loop: a triage agent that classifies a customer request and hands off to a specialist agent.

Why Azure Logic Apps multi-agent patterns matter

A single agent loop works well when the task is bounded and the instructions can cover every case. The problem comes when a task has multiple distinct phases that require different expertise, different tools, or different models. Packing all of that into one agent’s instructions creates a sprawling, hard-to-maintain prompt. The model has to context-switch between roles in a single loop, which degrades quality and makes the run history harder to interpret.

Multi-agent patterns solve this by giving each agent a single, clear responsibility. The agents are composed at the workflow level: one agent’s output becomes another agent’s input, and each agent can have its own model, its own tools, and its own focused instructions.

The four Azure Logic Apps multi-agent patterns explained

Microsoft’s documentation defines four patterns for multi-agent composition in Logic Apps. They are ordered by complexity.

Prompt chaining

The simplest pattern. A sequence of agent loops runs one after another, where the output of each loop becomes the input to the next. Each agent has a single focused task: extract, then format, then sort, then summarise. The chain is linear and predictable.

Use prompt chaining when the workload can be decomposed into sequential steps with clear handover points and when the output of each step is well-defined. A business report processing chain, raw data in, executive summary out, is the canonical example from the Microsoft documentation.

Routing

A classification agent examines the incoming request and routes it to one of several specialist agent loops based on what it finds. The routing agent does not do the work itself it decides which agent should do the work and passes control there.

Use routing when incoming requests fall into distinct categories that need different handling: a customer service triage agent that routes billing queries to a billing agent loop, technical questions to a technical support agent loop, and general inquiries to a general response agent loop. The routing pattern prevents optimization conflicts, allowing a billing specialist agent to be tuned for billing tasks without being distracted by technical support scenarios.

Handoff

Similar to routing but more dynamic. Instead of a central classifier making an upfront routing decision, each agent loop decides during its own execution whether it needs to hand off to another agent. The handoff preserves conversation context and state across the transition the receiving agent knows the full history of what the previous agent did and said.

Use handoff when the trigger for transferring control depends on what emerges during the conversation: a general support agent that escalates to a technical specialist when it detects a complex issue, or a research agent that hands off to a writer agent once it has gathered enough material. The handoff pattern mimics human escalation patterns: a front-line agent handles what it can and passes on what it cannot.

Orchestrator-workers

The most sophisticated pattern. A central orchestrator agent dynamically decomposes a task into subtasks and delegates each subtask to a worker agent loop. The worker agents operate as tools that the orchestrator can invoke, exactly the tool provider pattern from Part 4, applied to agents rather than connectors.

Use orchestrator-workers when you cannot predict the required subtasks in advance. A coding agent that needs to make changes to an unpredictable number of files, a research agent that gathers information from multiple dynamic sources, or a content pipeline with a writer, reviewer, and publisher working together, these are all orchestrator-worker scenarios. The orchestrator dynamically determines what needs to be done; the workers execute it.

Demo: Building a sequential agent loop — Extract and Summarise

This demo builds a two-agent prompt chaining workflow in a new sequential-agents workflow inside la-agent-loop. The scenario is a business report processing chain: Agent 1 extracts key facts and metrics from a raw text input, Agent 2 takes those facts and writes a concise executive summary. The output of Agent 1 feeds directly into Agent 2 — this is the prompt chaining pattern in its simplest form.

Prerequisites

The la-agent-loop Standard logic app from previous posts
An Azure OpenAI / Foundry Models connection already configured

Step 1: Create the workflow

In la-agent-loop, click Create and name the workflow sequential-agents. Select Autonomous Agents as the workflow type. Logic Apps creates the workflow with an HTTP trigger and an empty Agent action.

Step 2: Configure the HTTP trigger

Click the When an HTTP request is received trigger and paste this request body schema:

{ "type": "object", "properties": { "report": { "type": "string" } }, "required": ["report"] }

Step 3: Configure the Extract Agent

Click the first Agent action and rename it Extract Agent. Configure it:

AI model: your GPT-4o / Foundry Models connection
Instructions: You are a data extraction specialist. Extract all numerical values, metrics, and key facts from the provided text. Return them as a clean bulleted list. Do not summarise or interpret — only extract.
User instructions item – 1: select report from the HTTP trigger dynamic content

Step 4: Add a Compose action

This is a critical step. The Extract Agent output is a JSON object containing a messages array — not a plain string. The Summarize Agent cannot process it directly. A Compose action between the two agents extracts the plain text content.

Click + below the Extract Agent container and add Add an action → Simple Operations → Compose. Set the Inputs expression to:

outputs('Extract_Agent')?['body']?['messages'][0]['content']

This extracts the bulleted list text from the Extract Agent’s output object and passes it as a clean string to the next agent.

Step 5: Add the Summarize Agent

Click + below the Compose action and select Add an agent. Rename it Summarize Agent. Configure it:

AI model: your GPT-4o / Foundry Models connection
Instructions: You are an executive communications specialist. Take the provided list of facts and metrics and write a concise three-sentence executive summary suitable for a board report. Be professional and direct.
User instructions item – 1: select the Outputs of the Compose action from the dynamic content picker

Step 6: Add a Response action

Click + below the Summarize Agent container and add a Response action:

Status Code: 200
Content-Type header: application/json
Body: set the expression to outputs('Summarize_Agent')?['body']?['messages'][0]['content']

Step 7: Save and test

Save the workflow and POST this to the trigger URL:

{ "report": "Q3 revenue was €4.2M, up 18% year on year. Customer acquisition cost dropped to €142, down from €198. Net promoter score reached 67. Headcount grew from 43 to 51. Churn rate fell to 2.3%." }

The workflow runs in approximately 16 seconds and returns a clean executive summary:

In Q3, revenue reached €4.2M, reflecting an 18% year-on-year increase, supported by a significant reduction in customer acquisition cost from €198 to €142. The company saw operational growth with headcount rising from 43 to 51, while maintaining strong customer satisfaction, evidenced by a Net Promoter Score of 67 and a low churn rate of 2.3%. These metrics highlight sustained growth and improved efficiency across key areas.

The run history shows two distinct agent iterations, Extract Agent and Summarize Agent, each with their own Think → Observe cycle, confirming the prompt chaining pattern is working end to end.

Practitioner note: The Compose action between the two agents is not optional. Logic Apps Agent actions return a structured JSON object not a plain string, so the second agent cannot consume the first agent’s output directly from dynamic content. The Compose expression outputs('Extract_Agent')?['body']?['messages'][0]['content'] bridges this gap. This is not documented clearly by Microsoft at the time of writing and is the most common point of failure when building sequential agent loops.

Choosing the right pattern

Pattern	Complexity	Use when
Prompt chaining	Low	Sequential steps with clear handover points
Routing	Low–medium	Distinct input categories needing different handling
Handoff	Medium	Dynamic escalation based on conversation content
Orchestrator-workers	High	Unpredictable subtasks requiring dynamic decomposition

The patterns are not mutually exclusive. A production customer service system might use routing to direct initial requests, handoff for mid-conversation escalations, and prompt chaining within each specialist agent to process the request through multiple steps.

What comes next

Part 6 covers securing agentic workflows, the expanded caller surface introduced by multi-agent and conversational patterns, Easy Auth setup for production, and Managed Identity for backend connections.

Agentic Orchestration: The Evolution of SOA

Posted on January 25, 2026 by steefjan1970

For decades, integration professionals have shaped the digital backbone of enterprises from EAI to SOA to microservices. Today, agentic orchestration marks the next step in that evolution: transforming how we compose, coordinate, and reason across enterprise services. This isn’t a replacement for what we know; it’s an intelligent upgrade to it.

We built the bridges, the highways, and the intricate railway networks of the digital world. Yet, let’s be honest—for all our sophistication, our orchestrations often felt like a meticulous, rigid dance.

Enter Agentic Orchestration. This isn’t just another buzzword. It’s a profound shift, an evolution that takes the core principles of SOA and infuses them with intelligence, dynamism, and a remarkable degree of autonomy. For the seasoned integration architect and engineer, this isn’t about replacing what we know—it’s about enhancing it, elevating it to a new plane of capability.

How SOA Composites Differ from Agentic Orchestration

Cast your mind back to the golden age of SOA. For those of us in the Microsoft ecosystem, this meant nearly two and a half decades with BizTalk Server as our workhorse, our battleground, our canvas. We diligently crafted composite services using orchestration designers, adapters, and pipelines. Others wielded BPEL and ESBs, but the principle was the same. Our logic was clear, explicit, and, crucially, deterministic.

If a business process required validating a customer, then checking inventory, and finally processing an order, we laid out that sequence with unwavering precision—whether in BizTalk’s visual orchestration designer or in BPEL code:

XML

			
<bpel:sequence name="OrderFulfillmentProcess">
  <bpel:invoke operation="validateCustomer" partnerLink="CustomerService"/>
  <bpel:invoke operation="checkInventory" partnerLink="InventoryService"/>
  <bpel:invoke operation="processPayment" partnerLink="PaymentService"/>
</bpel:sequence>

		

Those of us who spent years with BizTalk know this dance intimately: the Receive shapes, the Decision shapes, the carefully constructed correlation sets, the Scope shapes wrapped around every potentially fragile operation. We debugged orchestrations at 2 AM, optimized dehydration points, and became masters of the Box-Line-Polygon visual language.

This approach delivered immense value. It brought order to chaos, reused services, and provided a clear, auditable trail. However, its strength was also its weakness: rigidity. Any deviation or unforeseen circumstance required a developer to step in, modify the orchestration, and redeploy. The system couldn’t “think” its way around a problem it merely executed a predefined script a well-choreographed ballet, beautiful but utterly inflexible to improvisation.

Agentic Orchestration: From Fixed Scripts to Intelligent Collaboration

Now, imagine an orchestration that doesn’t just execute a script, but reasons. An orchestration where the “participants” are not passive services waiting for an instruction, but intelligent agents equipped with goals, memory, and a suite of “tools”—which, for us, are often our existing services and APIs.

This is the essence of agentic orchestration. It shifts from a predefined, top-down command structure to a more collaborative, goal-driven paradigm. Instead of meticulously charting every step, we define the desired outcome and empower intelligent agents to find the best path to it.

Think of it as moving from a detailed project plan (SOA) to giving a highly skilled project manager (the Orchestrator Agent) a clear objective and a team of specialists (worker agents, each with specific skills/tools).

Key Differences that Matter

From Fixed Sequence to Dynamic Planning:

Traditional SOA executes a predetermined sequence: Step A, then Step B, then Step C. Agentic orchestration takes a different approach — agents dynamically construct their plan based on current context and available resources, asking: “What tools do I have, and which best serve this step?”

From Explicit Error Handling to Self-Correction:

In SOA, elaborate try-catch blocks covered every potential failure. BizTalk veterans will remember wrapping Scope shapes inside Scope shapes, each carrying its own exception handler. With agentic systems, a failing tool triggers reasoning rather than a halt — the agent may retry with a different tool, consult another agent, or revise its plan entirely.

From API Contracts to Intent-Based Communication:

Traditional SOA services communicate via strict, often verbose XML or JSON contracts — schema design and message transformation consumed countless engineering hours. Agentic systems shift to intent-based communication instead. An “Order Fulfillment Agent” can instruct a “Shipping Agent” with a clear goal: “Ship this package to customer X by date Y.” The Shipping Agent then determines which underlying tools, FedEx API, DHL API, best achieve that outcome, abstracting away the complexity of individual service calls.

From Static Connectors to Smart Tools:

Connectors and adapters in SOA are fixed pathways, each requiring explicit configuration per integration point. BizTalk veterans know this well from hours spent configuring adapters for every specific endpoint. In agentic architectures, existing APIs, databases, message queues, and even legacy systems are reframed as tools that agents can discover and wield intelligently. A Logic App connector to SAP is no longer just a connector; it becomes a capable SAP tool that an agent can invoke when the situation calls for it. The Model Context Protocol (MCP) is making this kind of dynamic tool discovery increasingly seamless.

A Concrete Example

Consider an order that fails the inventory check in our traditional BPEL or BizTalk orchestration. In SOA: hard stop, send error notification, await human intervention, and process redesign.

In an agentic system, the orchestrator agent might dynamically query alternate suppliers, adjust delivery timelines based on customer priority, suggest product substitutions, or even negotiate partial fulfillment—all without hardcoded logic for each scenario. The agent reasons about the business goal (fulfill the customer order) and uses available tools to achieve it, adapting to circumstances we never explicitly programmed for.

Azure Logic Apps: The Bridge to the Agentic Future

Azure Logic Apps demonstrates this evolution in practice, and it’s particularly compelling for integration professionals. For those of us coming from the BizTalk world, Logic Apps already felt familiar—the visual designer, the connectors, the enterprise reliability. Now, we’re not throwing away our decades of experience with these patterns. Instead, we’re adding an “intelligence layer” on top.

The Agent Loop within Logic Apps, with its “Think-Act-Reflect” cycle, transforms our familiar integration canvas into a dynamic decision-making engine. We can build multi-agent patterns—agent “handoffs” in which one agent completes a task and passes it to another, or “evaluator-optimizer” setups in which one agent generates a solution and another critiques and refines it.

All this, while leveraging the robust, enterprise-ready connectors we already depend on. Our existing investments in integration infrastructure don’t become obsolete; they become more powerful. The knowledge we gained from debugging BizTalk orchestrations, understanding message flows, and designing for reliability? All of that remains valuable. Microsoft is simply upgrading our toolkit.

Adopting Agentic Orchestration: The Path Forward for Integration Architects

For integration engineers and architects, this is not a threat but an immense opportunity. We are uniquely positioned to lead this charge. We understand the nuances of enterprise systems, the criticality of data integrity, and the challenges of connecting disparate technologies. Those of us who survived the BizTalk years are battle-tested, we know what real-world integration demands.

Agentic orchestration frees us from the burden of explicit, step-by-step programming for every conceivable scenario. It allows us to design systems that are more resilient, more adaptive, and ultimately, more intelligent. It enables us to build solutions that not only execute business processes but also actively contribute to achieving business outcomes.

Start small: Identify one rigid orchestration in your current architecture that would benefit from adaptive decision-making. Perhaps it’s an order-fulfillment process with too many exception handlers, or a customer-onboarding workflow that breaks when regional requirements change. That’s your first candidate for agentic enhancement.

Let’s cast aside the notion of purely deterministic choreography. Let us instead embrace the era of intelligent collaboration, where our meticulously crafted services become the powerful tools in the hands of autonomous, reasoning agents.

The evolution is here. It’s time to orchestrate a smarter future.