Azure Functions Behind API Management: What the Happy Path Diagram Leaves Out

Posted on August 26, 2026 by steefjan1970

Recently, I noticed another Azure architecture infographic on LinkedIn. Four boxes, left to right: clients, API Management, Function App, backend services. Underneath it, a tidy line. Function App handles the code, API Management handles the control. Together they deliver powerful and secure APIs.

(Source: LinkedIn post)

Nothing in that diagram is wrong. However, it’s not the whole story.

I have reviewed this pattern several times, in reference architectures, in project designs, and in my own work. The four boxes are always right and the design behind them is often not. The gap sits in what the arrows imply rather than in what the boxes say. So let us redraw it, and then walk through the six things the four box version quietly leaves out.

What the original gets right

Credit where it belongs. The separation of concerns in that infographic is sound, and plenty of teams still get it backwards.

API Management owns the contract. It publishes the API, applies policy, meters consumption, and gives consumers something stable to build against. Azure Functions owns the work. It runs your business logic, scales with demand, and bills for execution rather than for uptime.

That split matters because the alternative is worse. Teams that skip the gateway end up implementing authentication, throttling, and versioning inside every function, in slightly different ways, maintained by whoever touched it last. I wrote about a related boundary problem in Azure Functions, Logic Apps, and Power Automate: choosing the right tool, and the same principle applies here. The value of a gateway is not the features it lists. The value is the code it lets you delete.

So the boxes are right. Now for the arrows.

Gap one: a gateway is not a firewall

The original diagram lists “Security (OAuth, API Key)” inside the API Management box and leaves it there. That single bullet does a lot of quiet work, because it invites you to treat the gateway as your perimeter.

API Management applies policy. It validates tokens, enforces quotas, transforms payloads, and rejects malformed requests against a schema. It does not run an OWASP rule set, and it is not designed to absorb a volumetric attack aimed at your public hostname.

For anything internet facing, put Azure Front Door Premium or Application Gateway in front, with a WAF policy attached. Then lock the origin so the gateway only accepts traffic that arrived through the edge. Otherwise you have bought a policy engine and called it a perimeter.

This is also the cheapest gap to close, which makes it the most annoying one to find missing during a penetration test.

Gap two: subscription keys meter, tokens authenticate

Here is the conflation that causes the most damage in practice. A subscription key and an access token appear side by side in most diagrams, as if they were two ways of doing the same thing.

They are not. A subscription key identifies a product. It answers the question “which consumer agreement does this call belong to”, which is a billing and quota question. It does not tell you who the caller is, it travels in a header that ends up in scripts and log files, and it is shared across everyone using that product.

Authentication happens in the validate-jwt policy. That is where you check the signature, the audience, the issuer, and the claims that decide whether this particular caller may create an order.

			
<validate-jwt header-name="Authorization" failed-validation-httpcode="401">
  <openid-config url="https://login.microsoftonline.com/{{entra-tenant-id}}/v2.0/.well-known/openid-configuration" />
  <audiences>
    <audience>{{orders-api-audience}}</audience>
  </audiences>
  <required-claims>
    <claim name="roles" match="any">
      <value>Orders.Write</value>
    </claim>
  </required-claims>
</validate-jwt>

		

Once that is in place, the rate limit should follow the same identity. Throttling by IP address punishes everyone behind a corporate NAT and protects you from nobody who has a laptop and patience. Throttling by a claim from the validated token gives you a quota per caller, which is what the consumer actually agreed to.

			
<rate-limit-by-key calls="60" renewal-period="60"
  counter-key="@(context.Request.Headers.GetValueOrDefault("Authorization","").AsJwt()?.Claims.GetValueOrDefault("oid", "anonymous"))" />

Gap three: the arrow nobody draws

Now for the one that matters most.

Every version of this diagram shows one arrow into the Function App. That arrow implies the gateway is the way in. It is not. It is a way in.

An HTTP triggered function sits on a public hostname by default. Anyone holding that hostname and a function key can call it directly, and that call skips the WAF, the token validation, the rate limit, the audit trail, and every policy you carefully wrote. The gateway becomes a convention rather than a control, and conventions do not survive contact with an incident.

Teams close this in three stages, usually in this order.

Layer one, the function key. Store the host key as a secret named value in API Management, backed by Key Vault, and let the backend inject it as x-functions-key. The key never appears in your policy file or your repository. This stops crawlers and casual discovery. It does not stop anyone who has ever seen the key, and keys have a way of ending up in Postman collections, log files, and support tickets.

Layer two, token validation on the function itself. Turn on the built-in authentication, point it at the app registration that represents your API, and an untokened call now fails at the platform before your code runs. Better. Still not closed, because a valid token replayed straight at the function hostname bypasses everything the gateway added on top.

Layer three, take it off the internet. Set publicNetworkAccess to Disabled, put a private endpoint in front of the app, and integrate the gateway into the same virtual network. The hostname stops resolving from outside. Now there is genuinely one route in.

			
resource functionApp 'Microsoft.Web/sites@2023-12-01' = {
  properties: {
    publicNetworkAccess: networkIsolation ? 'Disabled' : 'Enabled'
    virtualNetworkSubnetId: networkIsolation ? functionSubnetId : null
  }
}

		

Layer three is the only one that turns your architecture diagram into a statement about reality. It also has a prerequisite that catches people out, which is the next gap.

Gap four: the tier decides the architecture

Private networking is not a checkbox you add at the end. It is gated by the API Management tier and by the Functions hosting plan, and those two choices constrain everything else in the design.

The v2 tiers changed the arithmetic here. Outbound virtual network integration used to mean Premium, which put private connectivity out of reach for a lot of internal APIs. Standard v2 brought it within reach of an ordinary project budget. On the compute side, the Flex Consumption plan brought virtual network integration to a consumption billing model, which used to mean choosing between cost and connectivity.

The same is true of time. Both the gateway and the function cap how long a request may run, and those caps differ by tier and plan. If your design assumes a two minute synchronous call, you have made a tier decision without realising it.

Pick the tier and the hosting plan before you pick the features, and verify the current limits on Microsoft Learn rather than trusting a diagram. Both sides of this moved several times during 2025 and 2026, and anything I write here has a shelf life.

Gap five: the dashed response arrow assumes synchronous

Look at the original infographic again. The response arrow is dashed and runs straight back to the client. It quietly assumes every call finishes inside the request.

Plenty do not. Bulk imports, report generation, anything that fans out to a slow downstream system. The instinct is to raise the timeout, first on the function, then on the gateway, until the whole chain waits for the slowest possible caller. That is not a fix. It is a queue with worse ergonomics, and it fails under load rather than under test.

The pattern that works is the asynchronous HTTP API. Accept the request, start the work, and answer immediately.

			
var instanceId = await durableClient.ScheduleNewOrchestrationInstanceAsync(
    nameof(BulkImportOrchestrator), orders);
var response = request.CreateResponse(HttpStatusCode.Accepted);
response.Headers.Add("Location", $"{request.PublicBaseUrl()}/bulk/{instanceId}");
response.Headers.Add("Retry-After", "5");

		

One detail deserves attention, because it connects back to gap three. Durable Functions ships a helper called CreateCheckStatusResponse that builds the polling URLs for you. Those URLs point at the function hostname. That leaks your backend to every caller, and it breaks the moment you disable public network access.

So build the status URL from a setting that holds the gateway address instead. Callers should never learn the name of your function app, and they certainly should not be given it in a header.

Gap six: one request, two telemetry stores

Both boxes in the diagram write to Application Insights, usually to two separate resources with two separate sampling configurations. Neither picture shows that, because telemetry is drawn as a property rather than as a system.

The result shows up during your first real incident. You have a gateway trace that ends at the backend call and a function trace that starts somewhere in the middle, and no reliable way to join them. Then somebody discovers that the gateway sampled at one rate and the function at another, so half the pairs do not exist at all.

Two habits fix this. Set a correlation id at the gateway when the caller did not supply one, and propagate it through the function into every log line.

			
<set-variable name="correlationId"
  value="@(context.Request.Headers.GetValueOrDefault("x-correlation-id", context.RequestId.ToString()))" />
<set-header name="x-correlation-id" exists-action="override">
  <value>@((string)context.Variables["correlationId"])</value>
</set-header>

		

Then align the sampling on both sides, and write the query that joins them before you need it rather than during an outage. I made a similar argument about governed telemetry in the Foundry Citadel Platform series, where an SDK routed its calls around the gateway entirely and the only reason we noticed was that the traces did not add up.

Where this is the wrong answer

Everything above assumes you need API Management. Often you do not.

A gateway earns its place when there is a portfolio to govern. Several APIs, consumers outside your own team, versions that have to coexist, quota that differs per consumer, one place where policy and audit live. Under those conditions API Management pays for itself in the coordination it removes.

For a single internal API with one consumer, it is a monthly bill wrapped around a proxy. Built-in authentication with Entra ID on the function, a private endpoint if it is internal, and you are done. Add the gateway when the second consumer actually appears, because that is when versioning and per consumer quota start to matter.

There is also a middle answer that gets skipped. One public API that needs a WAF but has no consumer lifecycle should sit behind Front Door and nothing else. You get the perimeter without buying governance you are not using yet.

A sample you can deploy

I put the whole thing in a repository, because policy fragments in a blog post are easy to agree with and harder to run.

One azd up deploys API Management Standard v2, a Flex Consumption function app on .NET 8 isolated, a storage account with shared key access disabled, and a shared Application Insights resource. The Orders API has a synchronous endpoint, an asynchronous bulk import built on Durable Functions, and a health endpoint the gateway can probe.

The three layers from gap three are deployment switches rather than prose:

Setting	What it turns on
default	Layer one, function key held as a secret named value
`ENTRA_TENANT_ID` and `API_AUDIENCE`	Layer two, `validate-jwt` and quota by `oid` claim
`NETWORK_ISOLATION=true`	Layer three, private endpoint and public access disabled

Deploy it with the default settings, then call the function hostname directly and watch it answer. That single curl makes the argument better than this whole post does.

Closing

The four box diagram is a good summary and a bad specification. Function App handles the code, API Management handles the contract, and neither of them handles the network. That last part is where these designs usually fail, and it is the part no infographic ever draws.

Production Readiness: Closing the Execution Gap

Posted on August 25, 2026 by steefjan1970

Every post in this series so far has covered a layer: compute, messaging and orchestration, data patterns, governance and identity, observability and FinOps. This one isn’t a layer. It’s a question that cuts across all of them: when is the platform actually ready for production?

That question is harder than it looks, because “we built it” and “we can run it” are different claims. I’ve watched more than one integration platform pass every technical checkpoint and still fall short of production-ready, not because the design missed anything, but because nobody had turned that design into something enforceable, operable, and provable. So this post is about the gap between those two states, and how you close it. It’s the through-line under every layer, and it’s the thing that turns a strong architecture into a platform you can responsibly put load on.

The shift: from “is it built?” to “can we operate it?”

Here’s the single most useful reframe I know for this stage. Stop asking “is the platform technically built?” and start asking “can we operate it safely, recoverably, auditably, and predictably?”

Those are not the same question. The first is about whether the components exist and connect. The second is about whether, when something goes wrong at 2 am, someone can see what happened, understand it, recover from it, and prove afterward that they handled it correctly. A platform can pass the first test comfortably and fail the second completely. And the second test is the one that actually determines whether you should go live. So the moment you catch yourself saying “it works,” push on: does it work in a demo, or does it work under a failure you didn’t plan for?

The core move: make the implicit explicit

Most integration platforms at this stage share the same shape. The design is good, and someone has largely written it down. But a lot of what matters lives in documentation, in code that’s still evolving, or in the heads of the people who built it. That works fine while a small team builds the foundation. It stops working the moment the first real production use case lands, because implicit choices become whatever the first team happens to decide.

The fix is a production baseline: an explicit, enforceable statement of what production use demands. It names the mandatory components and patterns, pins down the environment profiles, fixes the security controls and monitoring standards, and settles the recovery agreements and release criteria. Its job is to pull those decisions out of documents and habits and into something the platform enforces, so the first production use case inherits the decisions rather than reinventing them.

Without that baseline, every early integration is free to make its own choices, and you accumulate inconsistency, technical debt, and a future re-platforming you didn’t budget for.

Design versus demonstrable: the recurring gap

The same gap shows up in every layer, and once you see it, you can’t unsee it. On security, the design names Zero Trust, least privilege, and pipeline-driven change, yet permanent broad access rights sit in the environment, quietly contradicting it. Observability design specifies OpenTelemetry and required fields, but the alert rules and dashboards never make it into the infrastructure. And the CI/CD design describes a full release chain with quality gates, while the pipelines really only cover the dev environment.

In each case the design is right and the demonstrable working is missing. That’s the pattern to hunt for when you assess readiness: not “did someone design this?” but “does the platform actually enforce this design somewhere it can prove?” Anything that lives only as intent is a gap, however good the intent.

What operational readiness actually covers

Technical function is necessary but not sufficient. Operational readiness asks a distinct set of questions, and that set decides go-live. In practice, it comes down to whether you can demonstrably answer these:

Can you see what’s happening: monitoring, chain-level tracing, message-level insight? When an incident hits, does someone triage it, and do they have the information they need to do so? For recovery, can the platform handle errors, replay from a known point, and fall back on a real Business Continuity and Disaster Recovery plan rather than an RTO written in a document? Afterward, can you prove what happened through an audit trail, an access log, and change history? On access control, do you separate permanent from elevated rights, and dev from production? For cost, can you attribute it, budget against it, and tier retention? And finally, can you hand it over — have you defined operational ownership, or does every incident route back to the people who built it?

If any of those answers is “only in the design,” please fix it before go-live, not after the first incident teaches you the hard way.

The boundary that decides scale: central versus decentral

The other thing production-readiness has to settle is a boundary, not just a checklist. Most modern integration platforms want value-stream teams to deliver independently within central guardrails. That’s the right ambition. But it only works when you draw the line between central platform ownership and team autonomy deliberately, because that line runs through every layer: API governance, messaging configuration, RBAC, pipeline use, monitoring, error handling, lifecycle, support.

Get the line wrong and you recreate the exact problem the platform set out to solve. The platform team becomes the bottleneck again — the single point through which every change, approval, and incident has to pass. So team autonomy isn’t just a tooling question. It needs explicit ownership agreements, release paths, access models, quality controls, and operational responsibilities. The tooling enables autonomy; the agreements make it safe.

Shared components are where this bites hardest. A shared API gateway, a shared message broker, a shared logging workspace these touch technology, security, governance, operations, cost, and autonomy all at once. They make the platform economical, and they carry the biggest scaling risk. For each one, decide deliberately: why does it stay shared rather than isolated, who owns it, who may change it, how do you monitor it, and how do you attribute its cost? Leave those implicit and the shared component quietly becomes everyone’s dependency and no one’s responsibility.

Readiness isn’t one moment — it’s three

The last reframe worth making: “ready” isn’t a single bar. It’s three different bars at three different moments, and conflating them is how platforms either over-build early or under-prepare for scale.

First go-live. The bar here is the minimum production baseline and demonstrable operational readiness. Not every capability has to be complete, but the ones that are preconditions for running safely in production do. This is where the baseline, the recovery plan, and the release criteria have to be real.

First team onboarding. The bar shifts to whether the federated model actually works in practice. Can one real team deliver independently within the central guardrails, without quality, security, or consistency buckling under the first real use? This is a practice test, and it’s better, even, to let some things get concrete here rather than designing them fully in the abstract.

Scaling to many teams. Now the bar is repeatability. Anything that worked at one team through direct conversation now has to become standardised, documented, and reproducible: lifecycle policy, cost allocation, onboarding, support model, versioning, exception handling. Direct alignment doesn’t scale; product-steering does.

Naming which moment a given concern belongs to is half the battle. It stops you from demanding scale-grade rigour before first go-live, and from discovering at team five that nobody built the repeatable version.

Where this thinking gets over-applied

Consistent with the series, the honesty section. “Production baseline” thinking is right, but it can tip into paralysis.

Not everything has to be complete before first go-live. The three-moments split exists precisely so you don’t. Demanding full lifecycle policy, mature FinOps, and a complete federation model before a single use case runs is how a platform never ships. Match the rigour to the moment.

A baseline that only flags is a baseline that gets ignored. The whole point of the production baseline is that the platform enforces it. A pile of documented-but-unenforced standards manufactures the appearance of readiness without the substance, which is more dangerous than an honest gap, because it invites false confidence.

You can make a decision deliberately without making it central. Drawing the central-versus-decentral line carefully doesn’t mean pulling everything central. Sometimes the deliberate call is “this is team-owned,” and recording that reasoning is the point, not the direction.

The shape of it

For an integration architect, production-readiness isn’t a technical checkpoint; it’s the shift from a platform that’s built to one you can operate responsibly. Make the implicit explicit in a production baseline. Hunt the gap between what’s designed and what’s demonstrable. Answer the operational-readiness questions before go-live, not after. Draw the central-versus-decentral line deliberately, especially for shared components. And treat “ready” as three moments, not one. Do that, and the layers from this series stop being a good architecture on paper and become a platform you can actually run.

This is the lens that ties the series together. The Azure PaaS map has the layer-by-layer foundation; this post is the question you hold every layer up against before you put production load on it.

Azure Functions vs. Logic Apps vs. Power Automate: When to Use What

Posted on August 21, 2026 by steefjan1970

If you work anywhere near the Microsoft ecosystem, you have probably run into all three of these services. You have probably also run into the confusion around them. They all “automate” something. They all show up in architecture conversations. On the surface, their marketing pages sound almost interchangeable.

This post continues the Cloud Perspectives Azure PaaS series. It follows recent entries on Azure Functions as a serverless agents runtime and managed identity in Logic Apps Standard. Those posts went deep on one service. This one steps back and compares all three. The usual shorthand, Functions for code, Logic Apps for integration, Power Automate for business users, is cleaner than reality.

In practice, Functions is a capable integration tool in its own right. Logic Apps’ headline B2B/EDI capability comes bundled with an extra resource and its own bill. Power Automate is not even an Azure product. Picking the wrong tool does not just produce a clunkier solution either. It can mean months of maintenance pain, licensing costs nobody budgeted for, or a workflow that cannot scale. Here is a more honest breakdown of how the three differ, and when each one earns its place in your architecture.

Where each service actually lives

Before comparing capabilities, it helps to see where these tools sit organizationally. That placement drives billing, governance, and who owns the resource day to day.

Azure Functions and Logic Apps are Azure resources. You provision them in the Azure portal, under an Azure subscription, next to your virtual machines and storage accounts. Platform teams building governance models, like the ones described in Azure governance and identity for integration architects, treat them accordingly.

Power Automate is different. It is licensed through Microsoft 365 and the Power Platform admin center. That difference is not a footnote. It determines which admin center you open when something breaks, and which budget line absorbs the cost.

Azure Functions: built for developers who want full control

What it is

Azure Functions is Microsoft’s serverless compute service. You write code in C#, Python, JavaScript, TypeScript, Java, PowerShell, and more. It runs in response to an event. That event might be an HTTP request, a new file landing in Blob Storage, a message hitting a queue, or a timer firing. You never manage the underlying servers. You simply ship functions and let Azure handle the scaling.

It is also a first-class integration tool

Functions gets typecast as “the compute one” while Logic Apps gets credited with “integration.” That framing sells Functions short. Its HTTP triggers and rich set of bindings include Service Bus, Event Grid, Cosmos DB, and Blob Storage. Those let a function sit in the middle of a system-to-system exchange just as naturally as a Logic App can.

For stateful, long-running orchestration across multiple systems, the exact scenario people usually reach for Logic Apps for, Durable Functions provides that same pattern in code. You get full testability and source control with it. For developers building the kind of serverless orchestration covered in Azure Functions as a serverless agents runtime, Functions is a legitimate path for integration work, not a fallback.

When to use it

Reach for Functions when you need custom logic, complex calculations, or heavy data transformation that a visual designer cannot express cleanly. Reach for it when you want total control over code, dependencies, and third-party libraries. It also fits when you are building microservices and APIs that must perform well under load. It also fits when you are doing systems integration and would rather express it in code than in a visual designer.

Typical use cases: processing images uploaded to Blob Storage, powering a custom REST API for a mobile or web app, running scheduled jobs, handling real-time IoT telemetry, and orchestrating multi-step integration workflows through Durable Functions.

Trade-offs

Functions requires real programming skills. Cold starts on the Consumption plan can add latency to infrequent workloads. You also own more of the maintenance and security surface than a managed workflow tool would give you.

On the upside, Functions is usually the cheapest of the three at scale. The Consumption plan includes a substantial monthly free grant, around one million executions and 400,000 GB-seconds. You only pay for what you actually run.

Logic Apps: built for enterprise-grade integration

What it is

Logic Apps is Azure’s platform-as-a-service for orchestrating workflows across systems. Think of it as the enterprise integration layer. It gives you a visual designer, so it sits at a lower code level than Functions. Even so, it targets IT and integration teams rather than casual business users. Logic Apps shines when you need to connect many systems reliably, at scale, with proper DevOps practices wrapped around it. That is the kind of governance discussed in Azure data patterns for integration architects.

When to use it

Reach for Logic Apps when you need to integrate multiple systems, spanning cloud, on-premises, and SaaS, with formal reliability and monitoring requirements. It also fits B2B or EDI-style exchanges over AS2, X12, or EDIFACT. And it fits any scenario where you want a pay-per-execution model that supports high-volume enterprise workflows without managing infrastructure yourself.

Typical use cases: syncing a CRM like Salesforce with an on-premises SQL database, exchanging invoices with trading partners using industry-standard protocols, and orchestrating responses to Azure alerts or resource deployments.

The Integration Account catch

One nuance is easy to gloss over. The B2B/EDI capability is not something a plain Logic App gives you out of the box. It requires provisioning a separate resource, an Integration Account, to store trading partners, agreements, schemas, and certificates. You then link that account to your Logic App.

Integration Accounts carry their own tiered pricing across Free, Basic, Standard, and Premium levels. In other words, “use Logic Apps for B2B/EDI” really means “use Logic Apps plus an Integration Account.” That adds both cost and an extra resource to manage, something a lot of comparisons leave out entirely.

Trade-offs

Logic Apps requires an active Azure subscription and has a steeper learning curve than Power Automate. Unlike Power Automate, it also has no built-in desktop or RPA automation. Billing runs on trigger, action, and connector executions, which can add up faster than an equivalent Functions workload.

Still, you get native Visual Studio and Git integration, strong monitoring, and no hard execution limits. All of that matters at enterprise scale.

Power Automate: built for business users who need speed

What it is

Power Automate is the low-code, no-code member of the trio, built for productivity rather than infrastructure. It lets business users, not developers, automate day-to-day tasks such as approvals, notifications, report generation, and data syncing between Microsoft 365 apps.

It is not actually Azure

Here is a distinction worth making explicit, since the three tools so often get lumped together as “Azure services.” Power Automate is not an Azure service. It belongs to the Microsoft Power Platform, licensed alongside Power Apps, Power BI, and Copilot Studio. That licensing typically runs through Microsoft 365 plans, standalone per-user or per-flow licenses, or a free tier, not an Azure subscription.

The one exception is pay-as-you-go licensing, which lets you bill Power Automate usage against an Azure subscription as an alternative payment mechanism. Even so, that is a billing convenience, not evidence that Power Automate lives in Azure.

Functions and Logic Apps are Azure resources you provision in the Azure portal, under an Azure subscription, alongside your VMs and storage accounts. Power Automate, by contrast, is a Microsoft 365 offering that happens to interoperate with Azure resources through connectors. That is a real architectural distinction, not just a licensing footnote. It affects who owns the resource, where governance sits, and which admin center you troubleshoot in when something breaks.

When to use it

Reach for Power Automate when you want to boost individual or team productivity without writing code. It also fits when you need to automate UI-based tasks on legacy software through desktop flows, essentially RPA. It fits too when your workflow lives mostly inside Microsoft 365, across Outlook, Teams, SharePoint, and similar apps.

Typical use cases: routing document approvals through Teams and Outlook, using desktop flows to pull data out of an old legacy application, and automatically saving email attachments to a SharePoint folder.

Trade-offs

Power Automate is the fastest option to deploy for non-developers, and it integrates tightly with the Power Platform. That said, licensing can get expensive at scale, debugging and version control stay limited compared to code-first tools, and performance throttles kick in on high-volume runs.

A quick rule of thumb, caveats included

If the job needs raw coding power, fine-grained control, or code-first integration, reach for Azure Functions. That includes integration work. Do not rule it out just because Logic Apps carries the “integration” label.
If the job needs visual, governed workflow orchestration across systems, reach for Logic Apps. Budget for an Integration Account on top if EDI or B2B is involved.
If the job needs a fast, no-code fix for a Microsoft 365-centric business process, reach for Power Automate. Account for it as Power Platform or M365 licensing rather than an Azure cost.

The real power comes from combining them

These three are not really competitors. They are layers. A common pattern looks like this: Power Automate handles the front-end business process, say a Teams approval flow. That triggers a Logic App to orchestrate the broader integration across systems. The Logic App, in turn, calls an Azure Function to run the custom logic or heavy computation that neither low-code tool expresses well.

Used this way, each tool does the part it is actually good at. Power Automate handles speed and accessibility. Logic Apps handles governed integration at scale. Azure Functions handles anything that needs real code. The mistake is not choosing one of these. It is assuming you have to choose only one.

Azure Observability and FinOps for Integration Architects

Posted on August 20, 2026 by steefjan1970

In the Azure PaaS map post, observability was folded into the governance layer, with a note that Application Insights and Azure Monitor are non-negotiable. That was true, but it undersold the topic. For an integration platform specifically, observability isn’t a sub-bullet of governance. It’s the layer that decides whether you can actually run the thing in production.

So this post pulls observability out and gives it room. And it brings FinOps along, because the two share a root: you can’t manage what you can’t see. One makes system behaviour visible; the other makes cost visible. Both turn a platform from “it runs” into “we can run it responsibly.” Azure observability and FinOps, treated together, are what separate a platform that works in a demo from one you can operate under real load.

The gap between design and demonstrable operation

Here’s the pattern I see most often on integration platforms. The observability design is excellent. There’s a logging standard, a tracing approach, a set of required fields. Then you look at the actual infrastructure, and none of it is enforced. The alert rules aren’t there. The dashboards aren’t built. The diagnostic settings aren’t wired. The design lives in a document; the platform doesn’t know about it.

That gap matters more than it sounds. A monitoring standard that depends on discipline and review isn’t a platform capability; it’s a hope. The moment a team ships an integration without the dashboards, the standard quietly failed. So the real work in this layer isn’t designing observability. It’s making observability demonstrable: wired into the infrastructure, enforced in the pipeline, and impossible to skip.

Let’s walk what that means in practice.

OpenTelemetry as a platform contract, not a suggestion

Most mature integration platforms land on OpenTelemetry as the instrumentation standard. That’s the right call. W3C Trace Context propagates a trace across services, traces and metrics and logs share a model, and you avoid inventing your own correlation scheme. So far, so good.

The catch is that “we use OpenTelemetry” is a design statement, not an enforced one. For it to be a contract, three things must be true. First, the required fields, resource attributes, trace fields, and domain identifiers have to be defined explicitly, not left to each team’s judgment. Second, that definition has to be validated somewhere automatically, ideally at pull request. Third, the platform components themselves have to emit the standard, so a trace actually runs unbroken from the API gateway through messaging to the backend. Miss any of those, and you have telemetry that mostly correlates, which is worse than none, because it looks trustworthy right up until the incident where it isn’t.

Tracing the chain, not just the components

Azure gives you per-resource monitoring for free. You can see API Management’s metrics, Service Bus’s queue depth, and a Function’s execution count. That’s component monitoring, and it’s necessary but not sufficient. An integration platform’s job is to move a message across those components, so the question that matters is whether you can follow a single message or transaction through the entire chain.

That end-to-end view has to map onto the layers of your integration architecture, because each layer asks a different question. The consumer-facing layer cares about availability, latency, error rates, and throttling per channel. The process layer cares about routing, transformations, retries, and failures in async steps. The system-facing layer cares about dependencies on backends’ response times, timeouts, and contract breaks. Without that layered, chain-aware view, you get plenty of technical detail per Azure resource and almost no ability to reason about the integration as a whole.

Message-level insight and the async recovery problem

Component metrics tell you the platform is busy. They don’t help the person who has to answer “what happened to order 47821?” For that, an operator needs message-level insight: business identifiers, error categories, chain status, the last successful step, and retry state. Structured logging with domain attributes a flow ID, a message ID, a route key, and an error category is what makes that possible. And it has to come with explicit data classification, masking, retention, and access rules, because business identifiers in logs are exactly the kind of data a regulator asks about.

Then there’s recovery, which is where the compute choice comes back to bite. Async, message-driven processing needs a replay story: when something fails partway through, you need to know how far it got and re-drive it from there. A workflow engine often gives you some of this out of the box. Raw compute like Functions doesn’t, so you have to design the replay mechanism yourself, as part of the integration pattern rather than an afterthought.

The pattern that works: treat the message on the bus as a reference, not the full payload. Pair it with the claim-check pattern, in which the bus carries technical and functional metadata: trace ID, flow ID, message ID, route key, error category, retry count, and a pointer to the payload, safely stored in storage. Define checkpoints along the flow. Then, on failure, you can determine where processing succeeded and re-drive from the right point, with idempotency (from the data patterns post) making the re-drive safe. For fully synchronous request-response, re-driving belongs with the caller; the platform’s job there is clear error codes and traceability.

Monitoring as a Definition of Done

The single highest-leverage move in this layer costs almost nothing: make monitoring a Definition of Done for every integration. No integration ships without its dashboard, its alerts, its trace-context propagation, its required log fields, its retention setting. And this is the part that turns it from aspiration into capability: the checklist runs as a quality gate in the pipeline, not as a line in a review someone might skip.

That one change moves observability from “depends on the discipline of whoever built it” to “the platform won’t let you skip it.” It’s the difference between a standard and an enforced standard, and it’s the cheapest high-value thing on this entire list.

FinOps: cost is just another signal you can’t yet see

Everything above is about making system behavior visible. FinOps is the same discipline applied to cost. On an integration platform, it fails in the same way because cost visibility typically ends at the subscription or resource group boundary. That’s too coarse. It can’t tell you what an individual integration costs, or an API, or a queue, or a team’s share of a shared component.

Three FinOps problems come up on every integration platform:

Attribution needs a taxonomy. Without a consistent tagging scheme for value stream, team, environment, integration, API, owner, and cost category, cost remains a lump sum. With one, you can steer on cost per integration product rather than cost per subscription. This is the foundation; nothing else works without it.
Shared components are the hard part. Compute is easy to attribute when each team runs its own. But a shared API Management instance, a shared Service Bus namespace, a shared Log Analytics workspace those get used by everyone and billed centrally, and if you never build a distribution model, nobody owns the cost. The shared components that make the platform economical are exactly the ones whose cost is hardest to place. That’s not a reason to isolate everything; it’s a reason to deliberately decide the split.
Storage and retention are FinOps levers hiding within a compliance requirement. Observability generates data logs, traces, payloads held for replay, and dead-lettered messages. Compliance dictates how long you keep it. But retention length and storage tier are separate decisions. Data you must keep for audit doesn’t have to sit in a hot, queryable tier the whole time. Tie retention to data classification, then move cold data to cheaper tiers. The requirement is “keep it”; the FinOps move is “keep it cheaply.”

The through-line: FinOps on an integration platform isn’t financial reporting after the fact. It’s a design and governance concern, sitting right next to observability, because both are about seeing what the platform is actually doing.

Where this layer gets over-applied

Consistent with the series, the honesty section. Observability and cost control both have a failure mode of doing too much.

Not every signal deserves an alert. An alert that fires on something nobody acts on trains people to ignore alerts. Alert on what changes a decision; leave the rest on a dashboard. Alert fatigue is a real operational risk, not a sign of thoroughness.

Not every message needs full payload logging. Metadata-first is the right default. Payload logging belongs where there’s functional need and explicit consent, with masking and retention — not everywhere, because “log everything” is how sensitive data ends up somewhere it shouldn’t, and how your storage bill quietly triples.

Not every cost needs fine-grained attribution. Building per-message cost tracking for a low-volume internal integration spends more effort than the insight is worth. Match the granularity of attribution to the scale of the spend.

The shape of it

For an integration architect, observability and FinOps answer the same question in two currencies: what is the platform actually doing, and what is it actually costing? Wire OpenTelemetry in as an enforced contract. Trace the chain, not just the components. Give operators message-level insight and a real replay story. Make monitoring a Definition of Done the pipeline enforces. Then apply the same visibility to cost: a tagging taxonomy, a distribution model for shared components, and retention tiered by classification. Get both right, and the platform stops being a black box you hope is behaving and becomes one you can actually operate.

Want the layer this sits inside? The Azure PaaS map puts observability and governance in context against compute, integration, and data, and walks the five-question framework across all of them.

Building RAG Pipelines with Azure Functions: Event-Driven Data Retrieval at Scale

Posted on August 19, 2026 by steefjan1970

This is the fifth and final post in the series on AI and Azure Functions. The first post mapped all four AI-enabled patterns. Posts two through four went deep on MCP server hosting, the serverless agents runtime, and Durable Functions for directed agentic workflows. This post covers the fourth pattern: retrieval-augmented generation.

RAG is where Azure Functions earns its place through a capability that is easy to underestimate: the ability to handle multiple events from multiple data sources simultaneously. A RAG pipeline is only as good as its retrieval layer, and retrieval latency in production comes down to two things: how fast you can query your data sources, and how fast you can scale when demand spikes. Azure Functions on Flex Consumption handles both.

What RAG on Azure Functions actually looks like

The Microsoft Learn documentation is deliberately brief on this pattern: “RAG systems require fast data retrieval and processing. Functions can interact with multiple data sources simultaneously and provide the rapid scale required by RAG scenarios.”

That one-liner contains three architectural decisions worth unpacking.

Multiple data sources simultaneously. A retrieval layer that queries one source at a time is a bottleneck. A function app can fan out — trigger parallel calls to Azure AI Search, a Cosmos DB vector store, a blob-indexed knowledge base, and a case history API — collect all results, and pass the aggregated context to the language model in a single call. This is the fan-out/fan-in pattern from the Durable Functions post, applied to retrieval rather than to a directed workflow.

Event-driven. The most common RAG pattern in tutorials is synchronous: the user sends a query, retrieval runs, and the LLM responds. But production RAG often has an asynchronous dimension — new documents arrive, get processed, get indexed. Azure Functions handles both sides: HTTP-triggered retrieval for the synchronous path, and blob/Event Hubs/Cosmos DB change feed triggers for the ingestion pipeline that keeps the index current. The same function app can serve both.

Rapid event-driven scaling. The Azure OpenAI binding extension adds stateful chat session support — the function maintains conversation history across turns without you writing session management code. Combined with Flex Consumption’s scale-to-zero billing, that means a RAG-enabled chat endpoint costs nothing when idle and scales in seconds when traffic arrives.

The Azure OpenAI binding extension

The binding extension is the most concrete piece of infrastructure the docs call out for RAG. It provides three bindings relevant to RAG scenarios:

Text completion input binding — calls an Azure OpenAI deployment and returns the response. Used for one-shot prompt/response patterns.

Chat completion input binding — maintains a stateful chat session across function invocations. The session history is stored externally (Azure Table Storage by default) and automatically injected into each prompt. This is what the “Custom chat bot” sample in the docs demonstrates.

Embeddings input binding — generates vector embeddings from text. Used in the ingestion pipeline to embed documents before writing them to a vector store.

A simple RAG function using the chat completion binding looks like this in C#:

			
[Function(nameof(RagChat))]
public static async Task<IActionResult> RagChat(
    [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req,
    [TextCompletion(
        "{query}",
        Model = "%OPENAI_DEPLOYMENT%",
        SystemPrompt = "You are a helpful assistant. Answer using only the context provided."
    )] TextCompletionResponse completion)
{
    return new OkObjectResult(completion.Content);
}

		

The binding handles the Azure OpenAI call, authentication via managed identity, and response parsing. You write the retrieval logic — what context to inject into {query} — and the binding handles the LLM call.

The production version adds a retrieval step before the binding executes. In practice, this means the HTTP trigger extracts the user query, a call to Azure AI Search (or your vector store of choice) retrieves relevant passages, those passages are injected into the prompt template, and then the binding submits the augmented prompt to the LLM.

APIM as the LLM gateway layer

The binding extension simplifies the function code, but it routes directly to Azure OpenAI — there is no governance layer between the function and the model by default. For production RAG pipelines, that matters: you need rate limiting per client, token quota management, cost attribution across tenants or use cases, and a single point for monitoring and logging LLM calls.

APIM fills that role. Place it between the function app and Azure OpenAI, configure an inbound policy to validate the request and apply per-subscription token quotas, and configure an outbound policy to log usage. The function app calls the APIM endpoint rather than the Azure OpenAI endpoint directly.

One practical constraint from real deployment experience: if you use the Azure AI Foundry Agent Service SDK alongside Azure Functions, the SDK routes LLM calls directly to Azure OpenAI and bypasses APIM entirely. The standard OpenAI SDK doesn’t behave this way—calls flow through APIM as expected. The Citadel Platform series covers this in detail, including how to structure APIM policies for the token quota patterns that matter in production.

For RAG pipelines specifically, two APIM policies are worth implementing:

Token quota by subscription key. Set a per-day or per-hour token budget for each API consumer. RAG queries are often significantly larger than simple completions because the retrieved context inflates the prompt — a query with 2,000 tokens of retrieved passages consumes far more quota than a direct LLM call. Factor this into your quota design.

Semantic caching. APIM’s semantic caching policy stores LLM responses and returns cached results for semantically similar queries. For a RAG pipeline serving a knowledge base that changes infrequently, this can dramatically reduce token consumption and latency.

The ingestion pipeline

The retrieval side of RAG gets most of the attention, but the ingestion pipeline- how documents get into the index is where Azure Functions adds less-discussed value.

When a new document arrives in blob storage, the function extracts text, generates a vector embedding, and writes the result to Azure AI Search. An Event Hubs trigger handles high-throughput ingestion where many documents arrive simultaneously. A Cosmos DB change feed trigger keeps a vector index in sync with a transactional database.

The key advantage over a dedicated ingestion service is that the same managed identity, Application Insights workspace, and azd deployment pipeline cover both retrieval and ingestion. Operational complexity stays low even as the pipeline grows.

One constraint to know before building: blob triggers on Flex Consumption require EventGrid as the source. Standard polling blob triggers are not supported on the Flex Consumption plan. This means you must configure an EventGrid system topic and event subscription after provisioning — a two-step post-provision operation that cannot be included in the same Bicep deployment as the function app.

A minimal ingestion function using the Azure OpenAI SDK for embeddings — the recommended approach until the embeddings binding extension reaches GA:

			
[Function(nameof(IngestDocument))]
public async Task Run(
    [EventGridTrigger] EventGridEvent eventGridEvent,
    CancellationToken cancellationToken)
{
    // Read blob from storage, generate embedding, write to Azure AI Search
    var blobContent = await ReadBlobAsync(eventGridEvent.Subject, cancellationToken);
    var embeddingClient = openAiClient.GetEmbeddingClient(embeddingsDeployment);
    var embedding = await embeddingClient.GenerateEmbeddingAsync(
        blobContent, cancellationToken: cancellationToken);
    var document = new SearchDocument { ContentVector = embedding.Value.ToFloats().ToArray() };
    await searchClient.IndexDocumentsAsync(
        IndexDocumentsBatch.Upload([document]), cancellationToken: cancellationToken);
}

		

The EventGrid trigger fires on MicrosoftStorage.BlobCreated events; the function reads the blob, generates the vector, and writes to the index. EventGrid delivery includes retry logic —failed events are retried with exponential backoff for up to 24 hours.

Connecting the four patterns

This is the final post in the series, so it is worth mapping how the four AI patterns relate to each other in a production architecture. They are not mutually exclusive.

A realistic enterprise RAG system might use all four:

Azure Functions RAG pipeline handles retrieval — HTTP-triggered queries fan out to multiple sources, ingestion functions process new documents via blob and Event Hubs triggers.
MCP binding extension exposes the retrieval function as an MCP tool, so AI agents and Copilot can call it directly
Durable Functions orchestrates multi-step retrieval workflows, including approve-before-index, parallel ingestion with fan-out, andandlong-running document processing with human-in-the-loop review.
Serverless agents runtime wraps the whole thing in an agent that decides which data sources to query based on the user’s intent, using the retrieval function as one of its tools

APIM sits in front of the Azure OpenAI calls regardless of which pattern makes the LLM request. The function app’s managed identity authenticates to both APIM and the downstream Azure services. Application Insights provides a unified view across all four patterns.

The spectrum from the Durable Functions post applies here too: RAG retrieval is mostly directed (the function knows which sources to query), but the agent layer that decides what to retrieve and how to use the results is autonomous. Azure Functions provides the event-driven, scalable retrieval infrastructure for both ends of that spectrum.

What to know before building

Choose your vector store before choosing your binding. The Azure OpenAI binding extension integrates cleanly with Azure AI Search. If you are using Cosmos DB for MongoDB vCore, a Postgres pgvector extension, or an external vector database, you write the retrieval logic directly using the SDK — the binding handles the LLM call, not the retrieval.

The embeddings binding and the chat completion binding have different cost profiles. Embeddings are cheap per token; completions are expensive. The ingestion pipeline (embeddings) can run at high volume with low cost. The retrieval + completion path (chat) is where token costs accumulate. Design your APIM quota policies accordingly.

Stateful chat sessions need external storage. The chat completion binding stores session history in Azure Table Storage by default. For multi-tenant RAG, partition session keys by user or tenant — the default key scheme does not enforce isolation automatically.

Retrieval quality determines output quality. Azure Functions handles the retrieval infrastructure; what you retrieve is your responsibility. This distinction matters more than it might seem.

With keyword search alone, a query like “List all Metallica albums” retrieves documents based on term frequency across the index. An index containing Radiohead (48 releases) and Porcupine Tree alongside Metallica will surface Radiohead and Porcupine Tree documents ahead of Metallica ones, because those bands have more indexed content matching common terms. The LLM then filters the irrelevant context and may miss albums that simply did not appear in the retrieved passages.

Hybrid search combining keyword retrieval with vector retrieval addresses this directly. Generate an embedding for the user query, add a VectorizedQuery targeting the content_vector field, and Azure AI Search fuses both signals using reciprocal rank fusion. Semantically relevant documents surface regardless of term frequency:

			
var vectorQuery = new VectorizedQuery(queryVector)
{
    KNearestNeighborsCount = 20,
    Fields = { "content_vector" }
};
var searchOptions = new SearchOptions
{
    Size = 20,
    VectorSearch = new VectorSearchOptions { Queries = { vectorQuery } }
};
// Hybrid: keyword query string + vector query sent together
var results = await searchClient.SearchAsync<MusicDocument>(
    userMessage, searchOptions, cancellationToken);

		

The companion repo implements hybrid search in MusicChatAgent.cs. Chunk size, embedding model choice, and whether you use keyword, semantic, or hybrid search have more impact on output quality than any infrastructure decision.

Testing the RAG pipeline

The companion repo supports several query patterns that exercise different aspects of the pipeline. These are worth running after ingesting a few bands to verify both the retrieval and the grounding behaviour.

Artist-specific ranking — tests hybrid search surfacing the right band:

			
{"message": "List all Metallica albums with their ratings, ordered highest to lowest"}

Date filtering via LLM reasoning — the index has no date filter; the model reasons over retrieved content:

{"message": "What albums did Tool release in the 1990s?"}

Cross-band comparison — tests whether retrieval surfaces relevant documents from multiple bands:

			
{"message": "Compare Opeth and Porcupine Tree — which has more critically acclaimed albums?"}

Rating threshold query:

{"message": "Which albums across all bands have a rating above 4.5?"}

Not-in-index grounding test — the agent should say it does not have information rather than hallucinate:

{"message": "Tell me about a band called Coldplay"}

Validation — should return HTTP 400:

{"message": ""}

The hybrid search implementation means artist-specific queries return only that artist’s documents. With keyword-only search, a query for Metallica in an index containing Radiohead (48 releases) would return mostly Radiohead documents — the vector component ensures semantic relevance wins over term frequency.

The companion repo — Music RAG Agent

The music-rag-agent companion repo implements the full two-pipeline RAG system described in this post. It scrapes band and album data from SputnikMusic, generates embeddings with the Azure OpenAI SDK, indexes into Azure AI Search, and exposes an HTTP chat endpoint that retrieves relevant passages and returns grounded answers.

After ingesting Tool, Opeth, Porcupine Tree, and Radiohead you can ask questions like:

“Which band has the highest rated albums — Tool, Opeth, Porcupine Tree or Radiohead?”

The retrieval correctly surfaces Sputnik ratings and vote counts for each album. The ingestion pipeline handles special characters in album titles (Ænima, Opiate²) via URL-safe Base64 document keys, and the EventGrid trigger ensures reliable delivery with 24-hour retry backoff.

Known Sputnik artist IDs: Tool = 83, Opeth = 932, Porcupine Tree = 328, Radiohead = 86.

Series wrap-up

This post completes the series on AI and Azure Functions. The five posts covered:

The four AI-enabled patterns and how to choose between them
Hosting MCP servers — binding extension (GA), self-hosted SDK (preview), and queue-based tools
The serverless agents runtime — markdown agents, the three configuration files, and dynamic workflows
Durable Functions for directed agentic workflows — five patterns with working C# samples verified on Azure
RAG pipelines — event-driven retrieval, the Azure OpenAI binding extension, and APIM governance

The companion code repositories are at github.com/steefjan1.

Durable Functions as the Orchestration Layer for Directed Agentic Workflows

Posted on August 18, 2026 by steefjan1970

Not all AI orchestration should be autonomous. Some scenarios need predictable, directed steps, and that is where Durable Functions fit in the Azure Functions AI stack.

The previous post in this series covered the serverless agents runtime, where you give the agent instructions and tools and Microsoft Agent Framework determines the execution path. This post covers the opposite end of the spectrum: Durable Functions agentic workflows, where you define the steps and the model executes them in a known sequence. The workflow is deterministic. The AI is a participant, not the orchestrator.

If your AI-driven process has fixed, ordered steps and you need auditability, fault tolerance, and long-running execution, Durable Functions is the right tool. If you want a model to determine the steps dynamically, go back to the serverless agents runtime.

What Durable Functions brings to agentic scenarios

Durable Functions is an extension of Azure Functions that lets you build stateful workflows in a serverless environment. The runtime manages state, checkpoints, retries, and recovery so workflows can run reliably for long periods minutes, hours, or days.

The doc positions it clearly for agentic use: “Some scenarios require a higher level of predictability or well-defined steps. These directed agentic workflows orchestrate separate tasks or interactions that agents must follow.”

Three capabilities make Durable Functions particularly well-suited for AI orchestration:

State persistence across steps. A workflow that calls an LLM, waits for a human decision, calls another service, and then writes a result can span hours or days without holding a connection open or burning compute. Durable Functions checkpoints state after every activity function completes.
Built-in retry and fault tolerance. LLM calls fail. External services time out. Durable Functions handles retries at the activity level with configurable backoff. You define the retry policy once, and every step in the workflow inherits it.
Human-in-the-loop support. The external events pattern lets a workflow pause and wait for human input — an approval, a correction, a classification decision. The workflow resumes when the event arrives, with full state intact.

The four patterns that map to agentic AI

Durable Functions has several application patterns documented in the Microsoft Learn docs. Four map directly to common agentic AI scenarios.

The patterns below are illustrated in Python, which maps clearly to the Durable Functions programming model. The companion repository implements all five patterns in C# (.NET 8, isolated worker). The orchestrator and activity structure are identical, but the syntax differs.

Function chaining — sequential AI pipeline

The simplest pattern: step A completes, then step B runs, then step C. Each step takes the previous step’s output as input.

In agentic terms, this is the document processing pipeline: extract text → classify intent → call LLM with classification context → write structured result. Each activity function is independently retryable. If the LLM call fails, Durable Functions retries it without re-running extraction.

			
@app.orchestration_trigger(context_name="context")
def document_pipeline_orchestrator(context: df.DurableOrchestrationContext):
    text = yield context.call_activity("extract_text", context.get_input())
    classification = yield context.call_activity("classify_intent", text)
    result = yield context.call_activity("call_llm", {
        "text": text,
        "classification": classification
    })
    yield context.call_activity("write_result", result)
    return result

		

The orchestrator function contains no business logic; it coordinates. The activity functions contain the work. This separation makes each step independently testable and observable.

Fan-out/fan-in — parallel retrieval and aggregation

The orchestrator starts multiple activity functions simultaneously and waits for all to complete before continuing. This is the RAG retrieval pattern: query multiple data sources in parallel, collect all results, and pass the aggregated context to the LLM.

			
@app.orchestration_trigger(context_name="context")
def parallel_retrieval_orchestrator(context: df.DurableOrchestrationContext):
    query = context.get_input()
    # Fan out — all three retrieval calls run in parallel
    tasks = [
        context.call_activity("search_knowledge_base", query),
        context.call_activity("search_policy_documents", query),
        context.call_activity("search_case_history", query),
    ]
    results = yield context.task_all(tasks)
    # Fan in — aggregate and call LLM once with full context
    response = yield context.call_activity("call_llm_with_context", {
        "query": query,
        "context": results
    })
    return response

		

The Durable Functions tutorial in the Learn docs demonstrates this pattern with parallel text file analysis: multiple files are processed simultaneously, results are aggregated, and a single output is returned.

Human interaction — approval and correction loops

The external events pattern lets a workflow pause indefinitely waiting for human input. This is the compliance review pattern: the AI produces a draft or classification, a human reviews it, and the workflow continues with the human’s decision.

			
@app.orchestration_trigger(context_name="context")
def approval_orchestrator(context: df.DurableOrchestrationContext):
    input_data = context.get_input()
    # AI produces initial classification
    ai_result = yield context.call_activity("classify_with_ai", input_data)
    # Notify reviewer and wait — the workflow pauses here
    yield context.call_activity("notify_reviewer", {
        "result": ai_result,
        "instance_id": context.instance_id
    })
    # Wait for human decision — could be minutes or days
    human_decision = yield context.wait_for_external_event("ReviewDecision")
    # Continue with human-approved or corrected result
    final_result = yield context.call_activity("process_decision", {
        "ai_result": ai_result,
        "human_decision": human_decision
    })
    return final_result

		

The workflow pauses at wait_for_external_event without consuming resources. When the reviewer submits their decision, the Durable Functions client sends the event and the workflow resumes immediately.

Monitor — polling until a condition is met

The monitor pattern runs a check on a schedule until a condition is satisfied, then exits or escalates. In agentic terms: poll an external system until a document is processed, a model inference job completes, or a status changes.

			
@app.orchestration_trigger(context_name="context")
def monitor_orchestrator(context: df.DurableOrchestrationContext):
    input_data = context.get_input()
    expiry = context.current_utc_datetime + timedelta(hours=24)
    while context.current_utc_datetime < expiry:
        status = yield context.call_activity("check_processing_status", input_data)
        if status == "completed":
            return yield context.call_activity("retrieve_result", input_data)
        elif status == "failed":
            return yield context.call_activity("handle_failure", input_data)
        # Wait before next poll — no compute consumed during wait
        next_check = context.current_utc_datetime + timedelta(minutes=5)
        yield context.create_timer(next_check)
    return yield context.call_activity("handle_timeout", input_data)

		

A real pattern: risk-class-driven routing

A pattern I work with in integration architecture is risk-class-driven routing — an AI classification step followed by different downstream workflows depending on the risk class assigned.

The AI classifies an incoming request as low, medium, or high risk. The orchestrator branches based on the classification:

Low risk — automated processing, result written directly
Medium risk — automated processing with human notification and override window
High risk — human review required before any processing continues

			
@app.orchestration_trigger(context_name="context")
def risk_routing_orchestrator(context: df.DurableOrchestrationContext):
    request = context.get_input()
    # AI classification step
    risk_class = yield context.call_activity("classify_risk", request)
    if risk_class == "LOW":
        return yield context.call_activity("process_automated", request)
    elif risk_class == "MEDIUM":
        result = yield context.call_activity("process_automated", request)
        yield context.call_activity("notify_supervisor", {
            "result": result,
            "override_window_minutes": 30
        })
        try:
            override = yield context.wait_for_external_event(
                "SupervisorOverride",
                timeout=timedelta(minutes=30)
            )
            return override
        except TimeoutError:
            return result
    else:  # HIGH
        yield context.call_activity("notify_reviewer", request)
        human_decision = yield context.wait_for_external_event("ReviewDecision")
        return yield context.call_activity("process_with_decision", {
            "request": request,
            "decision": human_decision
        })

		

This pattern appears in healthcare authorization, financial transaction review, and any domain where the cost of an incorrect automated decision varies by risk level. Durable Functions is the right runtime because it handles the human-in-the-loop wait without consuming compute, and it checkpoints state so a mid-workflow restartdoesn’t lose the AI classification result.

The companion repository implements all five patterns in C# with full azd deployment. The TESTING.md file contains the exact curl commands to exercise each pattern, including submitting external events for the approval and risk-routing workflows, all verified on Azure.

When Durable Functions is not the right answer

The directed vs autonomous distinction is the primary decision. But two other constraints matter.

Avoid it for very short workflows. If your workflow completes in under a second and has no human-in-the-loop steps, the Durable Functions overhead storage writes and checkpoint reads adds latency you don’t need. A simple function chain without orchestration is faster and cheaper.
Avoid it when the steps are not known in advance. If the AI needs to decide dynamically which tools to call and in what order, Durable Functions cannot model that. That is the serverless agents runtime; the AI is the orchestrator, not a participant.
Consider Logic Apps Agent Loop for low-code scenarios. If the workflow involves mostly connector-based integrations rather than custom code, Logic Apps with its Agent Loop pattern may be the right choice. Durable Functions earns its place when you need custom code logic at each step, tight control over retry behavior, or the ability to unit test each activity function independently.

The storage backend — Durable Task Scheduler

One operational detail worth knowing before you deploy: Durable Functions needs a storage backend to persist workflow state. The recommended option is Durable Task Scheduler, a managed service that handles task hub storage without requiring you to manage Azure Storage queues and tables manually.

For agentic workflows, which may run for hours and involve many checkpoints, the Durable Task Scheduler is worth the setup once it is fully available. At the time of writing, the azureManaged storage provider requires a preview extension bundle that isn’t included in the current release. The default Azure Storage backend works correctly for all patterns and is what the companion repo uses. Check the Durable Task Scheduler quickstart for the current availability status before planning a production deployment.

Dynamic workflows — the bridge between the two runtimes

The spectrum diagram at the top of this post places Durable Functions on the directed end and the serverless agents runtime on the autonomous end. Dynamic workflows, an experimental feature in the serverless agents runtime, sits between the two, and it is worth knowing about before you commit to one pattern.

The concept: flip workflows.enabled: true in a .agent.md file’s front matter, and the agent gains five built-in tools, including start_workflow. When the agent decides the work is workflow-shaped, a multi-step plan, a fan-out across data sources, a wait; it calls start_workflow with a DAG of tasks. The runtime validates the DAG and launches it as a Durable Functions orchestration. The agent gets back a workflow_id immediately and ends its turn. The Durable orchestration runs the plan in the background.

The AI authors the plan. Durable Functions guarantees the execution

This is a meaningful architectural shift. With the patterns in this post, you write the orchestrator code; you define the steps. With dynamic workflows, the LLM authors the DAG at runtime based on the task it is given. The execution is still deterministic and fault-tolerant because Durable Functions is running it, but the plan itself is emergent. Three concrete advantages the docs cite over chaining tool calls in conversation:

Lower token cost. Intermediate task results stay inside the orchestration. The agent sees only the final completion envelope, not every fan-out result. The docs reference roughly a 10× reduction on multi-tool workflows.
Lower latency. Each direct tool call is a model round-trip. A 20-step plan is one model turn to author the workflow, not 20.
Context-window discipline. Hundreds of kilobytes of intermediate data log lines, search hits, and line items never reach the model’s context. The agent reasons over the summary.

How it works in practice. Workflow tools are Python functions decorated with @workflow_tool rather than the standard @tool. The agent authors a DAG of tool tasks and wait tasks with depends_on edges for sequencing. ${node_id.result} templates let upstream outputs flow into downstream task arguments; the resolution happens inside the orchestrator, not in the agent’s context.

			
{
  "tasks": [
    { "id": "fetch_a", "type": "tool", "tool": "fetch_logs", "args": {"service": "auth"} },
    { "id": "fetch_b", "type": "tool", "tool": "fetch_logs", "args": {"service": "api"} },
    { "id": "summarize", "type": "tool", "tool": "summarize",
      "args": {"sources": ["${fetch_a.result}", "${fetch_b.result}"]},
      "depends_on": ["fetch_a", "fetch_b"] }
  ]
}

		

The incident triage sample in the azure-functions-agents-runtime repo shows this working end to end: an agent that fans out log fetches across multiple services, waits, then summarises the evidence.

What to know before using it. Dynamic workflows are experimental v1 with real constraints:

workflows.enabled: true is currently only honored on main.agent.md — dedicated agents can’t use it yet. That is flagged as a v2 constraint to lift.
v1 handlers must be synchronous. No per-task retry or timeout policies yet — those are v2.
The plan cap is 50 nodes, 10 parallel tasks, and a 24-hour maximum wait duration.
Completion is poll-based. The chat UI polls GET /agents/{slug}/workflows and injects a synthetic user message when a workflow reaches a terminal state, which triggers the agent to call get_workflow_status and summarise.

What comes next

The next post covers the fourth AI pattern: building RAG pipelines with Azure Functions, event-driven data retrieval at scale, the Azure OpenAI binding extension, and where APIM fits as the LLM gateway layer.

Up next: Building RAG Pipelines with Azure Functions: Event-Driven Data Retrieval at Scale

Agentic AI Design Patterns on Azure

Posted on August 15, 2026 by steefjan1970

Like a lot of people on LinkedIn right now, I have seen no shortage of infographics laying out agentic AI design patterns: tidy boxes and arrows for Tool Use, ReAct, Reflection, Planning, Orchestrator, Sequential Chain, Parallel Fan-out/Fan-in, Hierarchical, and P2P Mesh. These diagrams are a fine way to get the shape of an idea across. However, a diagram cannot tell you whether the thing actually works, or what breaks when you try to stand it up for real. So I spent the last stretch actually building agentic AI on Azure: all nine patterns, for real, on Azure’s PaaS offerings. That meant writing Bicep IaC, running azd up, making actual HTTP calls against the deployed endpoints, and running actual Application Insights queries when things went wrong. In the end I had nine independently deployable reference samples, not nine slides.

Writing the Bicep was the easy part. Getting all nine actually to stand up and respond correctly was where the real lessons were. This is a retrospective on what broke, why, and what I’d tell someone else to do to avoid it.

Lesson 1: model availability is regional, and the error message won’t always tell you clearly

The first failure mode hit me repeatedly, across multiple patterns. azd up would warn that gpt-4.1 “was not found” in the target region, then fail validation outright with InvalidResourceProperties: The specified SKU 'Standard' for model 'gpt-4.1' is not supported in this region.

This is not a bug in the Bicep. It is genuinely true that not every Azure OpenAI model and SKU combination is available in every region, and availability also varies by subscription. centralus and westeurope both rejected gpt-4.1 for me at different points, while swedencentral consistently worked across every pattern in the repo. So if you are scripting or templating Azure OpenAI deployments, do not hardcode a region and assume it will work everywhere. Instead, treat azd env set AZURE_LOCATION swedencentral as a reflexive first move whenever a fresh pattern’s deploy attempt fails.

Lesson 2: Logic Apps Standard is powerful, and standing it up correctly is its own skill

The Sequential Chain pattern (input → extract → draft → validate → output, a fixed pipeline) is a textbook fit for a low-code workflow engine. So I built it on Logic Apps Standard first, and it ended up costing me, by a wide margin, the most debugging time of the whole project. Here is what went wrong, roughly in order:

ServiceProvider-type triggers cannot have a recurrence property. If you copy a recurrence block over from an ApiConnection-style trigger example, you get a WorkflowProcessingFailed error, and the message does not obviously point at the fix.
Connector parameter names are not always what you would guess. Azure literally names the built-in OpenAI connector’s connection parameter openAIEndpoint, not endpoint. I only discovered this from Azure’s own runtime error message, not by guessing at the schema.
Windows versus Linux hosting changes what app settings you need, and not symmetrically. Windows-hosted Premium plans require WEBSITE_CONTENTAZUREFILECONNECTIONSTRING and WEBSITE_CONTENTSHARE, but carrying those same settings over to a Linux plan actively breaks content sync.
WorkflowStandard (WS1) is a hard requirement, not a suggestion. Azure explicitly rejects an ElasticPremium (EP1) substitution with “Logic Apps can be deployed only to ‘WorkflowStandard’ Sku or App Service Environments,” even though the two SKUs share the same underlying compute family.
Regional SKU quota is real, and it will not show up until you deploy. SubscriptionIsOverQuotaForSku for “WS1 VMs” hit me in two different regions. I eventually fixed it by decoupling the Logic App’s region from the rest of the pattern’s resources through a separate Bicep parameter, which let it land somewhere with quota headroom.
Deterministic role-assignment names collide across resource recreation. If you name a role assignment with guid(scope, resourceId, roleName) and then delete and recreate the resource whose managed identity receives that role, Azure refuses to update the existing assignment to point at the new principal ID. So you first have to find and delete the stale assignment by its exact resource ID.
Finally, after fixing all of the above, I ran into a Logic App instance that had simply wedged itself: Sequence contains no elements on startup, Kudu unreachable or crawling, and no combination of RBAC or config fixes helping. The only real fix was deleting and recreating the whole site resource.

Durable Functions

After all that, I rebuilt the same pattern on Durable Functions instead. An orchestrator calls three sequential activities, each a plain Azure OpenAI chat completion call, and the built-in RetryOptions does the retry work that Logic Apps’ declarative retry policies would otherwise have done. It deployed in under two minutes and worked on the first real test. The conceptual case for a low-code workflow engine still holds here, since a fixed sequence of stages is exactly what that kind of tool is for. However, if you want to move fast while building agentic AI on Azure, and you already have a working Durable Functions convention elsewhere in your stack, do not underestimate how much extra surface area Logic Apps Standard adds compared with staying in code you already know how to debug.

Lesson 3: RBAC is not one system, and Cosmos DB has its own

The P2P Mesh pattern (three Azure Functions reacting to each other’s events through Event Grid, with Cosmos DB tracking completion state) threw a 403 Forbidden: "Request blocked by Auth... does not have required RBAC permissions to perform action Microsoft.DocumentDB/databaseAccounts/readMetadata". This happened even though the function app’s managed identity already had every role I expected under Microsoft.Authorization/roleAssignments.

Here is the catch: Cosmos DB runs its own, separate, native RBAC system. A normal Azure role assignment against a Cosmos account grants control-plane access, but it does nothing for data-plane operations like reading or writing items. Instead, you need a Microsoft.DocumentDB/databaseAccounts/sqlRoleAssignments resource, scoped specifically to one of Cosmos’s own built-in role definitions. (The “Cosmos DB Built-in Data Contributor” role has a fixed, well-known GUID: 00000000-0000-0000-0000-000000000002.) So if your Bicep only includes the generic role-assignment resource type pointed at a Cosmos account, it silently grants nothing useful.

(Once I fixed that, the very next error was a 404 reading “Owner resource does not exist” on the database and container the app code expected. Provisioning the Cosmos account is not the same as provisioning the database and container inside it. Remember that if your Bicep stops at the account level.)

Lesson 4: pick the “native” endpoint type when Azure offers one

Still on P2P Mesh: wiring Event Grid subscriptions to Azure Functions endpoints through a raw webhook URL (https://<app>.azurewebsites.net/runtime/webhooks/eventgrid?functionName=X&code=<key>) produced a persistent 401 Unauthorized, Webhook endpoint validation failed. It survived two fixes I tried first: swapping the master key for the eventgrid_extension system key, and then directly inspecting the deployed function, which confirmed its eventGridTrigger binding matched what Event Grid expected. Neither fix mattered, because a plain authenticated GET against that same URL came back 400 Bad Request. That proved the key itself was fine, so the actual problem had to be something specific to Event Grid’s own validation handshake against a webhook-typed endpoint.

The actual fix was switching az eventgrid event-subscription create from a raw webhook URL to --endpoint-type azurefunction --endpoint <function-resource-id>. That is the native destination type Event Grid offers for Azure Functions, and it authenticates through the ARM resource ID instead of a function key embedded in a URL. It worked immediately. So the broader lesson: if a platform offers a first-class integration type for a specific target, prefer it over hand-rolling the equivalent with a generic webhook. That way, you inherit the platform’s own handling of edge cases you would not think of yourself.

Lesson 5: “Azure AI Foundry” isn’t one resource type

The Orchestrator pattern, a central agent hosted on Azure AI Foundry Agent Service that delegates to specialist tools, failed with a raw DNS error: Name or service not known, against a hostname the app was trying to reach. This was not an auth failure or a permissions failure. The hostname genuinely did not exist.

The root cause: the Bicep provisioned the older Azure AI Foundry resource model, a Microsoft.MachineLearningServices/workspaces hub plus a project workspace, which is the original Azure AI Studio approach. Meanwhile, the app code used Azure.AI.Projects.AIProjectClient, which only understands the newer, unified Foundry resource model: a Microsoft.CognitiveServices/accounts resource with kind: 'AIServices' and allowProjectManagement: true, plus a projects child resource. That newer model is reachable at a https://<account>.services.ai.azure.com/api/projects/<project> URL that only exists for that resource type. So two things share the “Azure AI Foundry” name, yet neither is a drop-in substitute for the other from an SDK’s point of view. As a result, it is worth explicitly checking which resource model your SDK version actually expects before wiring up IaC for it, because the error you get when you guess wrong will not obviously point at “you deployed the wrong kind of resource.”

The smaller stuff that added up

A few recurring, less dramatic gotchas worth a line each:

Double-check the resource group you think you are querying. More than once I pulled exceptions from the wrong pattern’s Application Insights instance, either because I was still in the wrong working directory, or because I grabbed the first App Insights component Azure CLI happened to return rather than the one for the resource group I actually cared about. So always sanity-check the resource group name before trusting a KQL result.
A previous session’s PowerShell variable can look like a fresh result. If a command throws partway through an assignment, the variable keeps its old value. Then, if that old value happens to look plausible (in my case, a valid-looking Durable Functions status payload from an entirely different pattern), it becomes easy to mistake it for real output from the command you just ran.
“No results” and “the backing store does not exist yet” are different failure modes. Code that gracefully handles an empty index or table does not automatically handle a missing one. For example, one specialist service in the Orchestrator pattern caught SqlException broadly and degraded gracefully when its sample table lacked seed data. By contrast, the AI Search-backed specialist had no equivalent guard for a 404 on a not-yet-created index, so it took the whole request down with it. Therefore, if you document “unseeded stores degrade gracefully,” actually test the unseeded case for every specialist, not just the one you happened to build first.

The takeaway: what building agentic AI on Azure actually requires

None of these were exotic failures. In fact, every one of them turned out to be a documented, well-understood Azure behavior once I found the right doc page. What made them expensive was that the error messages, on their own, rarely pointed straight at the fix: a DNS failure that was actually a resource-model mismatch, a 401 that was actually about endpoint type rather than credentials, and a 403 that was actually a second, unrelated RBAC system. So the practical lesson is not “Azure is fragile.” Instead, building agentic AI on Azure means dealing with more independently moving parts than the getting-started docs suggest, and the fastest way through is treating every unexplained error as worth one layer deeper of investigation before you assume the obvious fix is the real one.

I have now deployed and tested all nine patterns end to end. If you want the full breakdown of each one, including what it is for, the Azure architecture, and when to reach for it, start with the design patterns overview and go from there. The repo itself is public, so you can see exactly what each pattern provisions.

The Azure Functions Serverless Agents Runtime Explained

Posted on August 13, 2026 by steefjan1970

Let’s discuss the Azure Functions serverless agents runtime. Most conversations about agent runtimes on Azure land on Azure AI Foundry Agent Service. That is the right starting point for teams that want a fully managed, enterprise-grade agent host. But it is not the only option, and for event-driven scenarios, it is often not the best one.

The Azure Functions serverless agents runtime is a programming model that lets you define agents as function apps. Events, schedules, messages, or HTTP requests trigger agents. They run on Flex Consumption with scale-to-zero, managed identity, and Application Insights. And they are deployed with azd like any other function app.

This post explains what the runtime actually is, how its three configuration files work together, and where it fits relative to Foundry Agent Service and Durable Functions. If you are new to Azure Functions as an AI platform, start with Azure Functions AI Integration: The Quiet Powerhouse, which maps all four AI-enabled patterns. If you are looking specifically at MCP server hosting, Hosting MCP Servers on Azure Functions covers the three hosting options in detail.

How the Azure Functions serverless agents runtime works

The runtime is a programming model built on top of Azure Functions. When an event fires a timer, an HTTP request, or a queue message, the runtime starts the agent, runs it through Microsoft Agent Framework, and handles the trigger registration and endpoint wiring automatically.

You do not write trigger code or implement an agent loop. You define three files, deploy a function app, and the runtime does the rest.

Those three files are:

.agent.md — defines the agent: its instructions, its trigger, and the tools it can use
agents.config.yaml — app-wide runtime defaults, including the model deployment and any shared infrastructure (such as an Azure Container Apps dynamic session pool for sandboxed code execution)
mcp.json — lists the remote MCP servers available to the agents in the app

The runtime discovers these files at startup, registers the required triggers and endpoints, and wires the agent to Microsoft Agent Framework. You can have multiple agents in a single function app, each defined in its own .agent.md file, all sharing the app-wide configuration.

What the Azure Functions serverless agents runtime deploys

The Microsoft Learn quickstart deploys two agents from a single function app:

Chat agent (main.agent.md) — an HTTP-triggered agent that exposes a debug chat UI in the browser. It can execute sandboxed Python code via an Azure Container Apps dynamic session pool and browse the web. No email tooling.

Blog summary agent (daily_microsoft_blog_summary.agent.md) — a timer-triggered agent. The YAML front matter in the file declares the schedule; the markdown body contains the agent instructions. On each timer fire, the agent gathers recent Microsoft blog posts, summarises them, and emails the digest via a managed MCP server connected to Microsoft 365 Outlook.

What gets provisioned by azd up for this template:

Resource	Purpose
Flex Consumption function app	Hosts the agents
Azure AI Foundry project + model deployment	LLM for agent reasoning
Azure Container Apps dynamic session pool	Sandboxed Python code execution
Storage account	Function app state
Application Insights	Monitoring
Connector Namespace + M365 Outlook connection	Email delivery (optional)
Managed MCP server	Exposes the Outlook connector to agents

The provisioning is handled entirely by Bicep via azd — you do not configure any of this manually.

How the agent definition files work

.agent.md is a markdown file with YAML front matter. The front matter declares the trigger and any agent-level configuration. The markdown body is the system prompt — the instructions the agent follows when it runs.

The timer-triggered blog summary agent front matter looks roughly like:

			
---
trigger:
  type: timer
  schedule: "0 0 8 * * *"
tools:
  - mcp_server: outlook
---

		

The markdown body below contains the agent’s instruction set; it tells the agent what to gather, how to summarise it, and how to format the email. You write it in plain English.

agents.config.yaml sets defaults that apply across all agents in the app. The model deployment lives here, so every agent uses the same Azure AI Foundry model unless you override it. This setting also defines the session pool endpoint for sandboxed code execution.

mcp.json lists the remote MCP servers the agents can call. The quickstart template includes a managed MCP server for the Microsoft 365 Outlook connector when you enable email delivery. The runtime reads this file at startup and makes those servers available to all agents in the app.

How Azure Functions Serverless Agents Runtime differs from Foundry Agent Service

The distinction matters for architecture decisions.

Foundry Agent Service is a fully managed service. Microsoft operates the agent host. You configure agents through the Foundry portal or SDK, connect tools, and the service handles orchestration, state, and scaling. It has enterprise SLAs, built-in tooling, and a managed lifecycle.

The serverless agents runtime is a programming model you deploy yourself. You own the function app. You manage the deployment, the model connection, and the infrastructure. In return, you get the full Azure Functions hosting model: event-driven triggers, Flex Consumption billing, VNet integration, managed identity, and azd-based deployment pipelines.

The decision table:

Situation	Use
Need a fully managed agent host with enterprise SLAs	Foundry Agent Service
Agents triggered by events, schedules, or queue messages	Serverless agents runtime
Need VNet integration or custom deployment pipelines	Serverless agents runtime
Want scale-to-zero billing for bursty agent workloads	Serverless agents runtime
Need agents embedded in an existing function app	Serverless agents runtime
Prefer not to manage the agent host infrastructure	Foundry Agent Service

These are not mutually exclusive. The serverless agents runtime can call tools hosted in Foundry via MCP servers, and Foundry agents can call tools hosted in Azure Functions. The two runtimes can coexist in the same architecture.

How Azure Functions Serverless Agents Runtime differs from Durable Functions

Post 4 in this series covers Durable Functions for directed agentic workflows in detail, but the short version is:

Durable Functions is for directed, deterministic workflows: you define the steps, the model executes them in order, and Durable Functions handles state, retry, and fault tolerance. The workflow is predictable.

The serverless agents runtime is for autonomous agents. You give the agent instructions and tools, and Microsoft Agent Framework determines how to use them to accomplish the goal. The execution path is not predetermined.

If your AI-driven process has fixed, ordered steps and you need auditability, use Durable Functions. If you want the agent to figure out the steps, use the serverless agents runtime.

What to know before you build

It is preview. The programming model, file format, and configuration details are subject to change. Do not build production-critical workloads on this today without a plan for the preview-to-GA migration.

It requires a Foundry project and model deployment. The azd template provisions both automatically, but you need an Azure subscription with permissions to create Foundry resources and model deployments. Some organizations have restrictions on which model deployments are permitted.

The azd template provisions real Azure resources with real costs. The Flex Consumption plan keeps costs very low for low-traffic agents, but the Foundry model deployment, Container Apps session pool, and Connector Namespace resources still incur costs. Review the Bicep templates in infra/ before running azd up in a production subscription.

Custom Python tools are how you add app-specific logic. The runtime provides the agent loop and the MCP connections. For anything that requires your own code — calling internal APIs, reading proprietary data sources, applying business rules — you write Python tool functions and register them in the agent definition.

Getting started

The quickstart template is the right starting point:

			
azd init --template Azure-Samples/functions-quickstart-serverless-agents-azd -e serverless-agents
azd env set TO_EMAIL <your-email>
azd up

Review the three configuration files in src/ before deploying. They are short and readable, and understanding them before the first deployment saves debugging time later.

The email delivery step (setting TO_EMAIL and authorizing the Microsoft 365 Outlook connection) is optional. If you skip it, the timer agent still runs and returns its digest in the final response, which you can verify in Application Insights logs.

Try it with a weather sample.

If you want to see the runtime in action with a minimal, self-contained example before committing to the full quickstart, I built a companion sample: a weather chat agent that fetches live conditions and 3-day forecasts for any location using Open-Meteo, no API key, no M365 connector, no email setup required.

The agent is defined in a single main.agent.md file. It uses Python code execution via the Container Apps session pool to call the Open-Meteo API and returns structured weather data in the chat UI. Deploy it in three commands:

			
git clone https://github.com/steefjan1/weather-agents
cd weather-agents
azd up

Select Central US when prompted for location — the runtime is in preview and region availability is limited. The chat UI is at https://<function-app-name>.azurewebsites.net/api/agents/main/ once deployment completes.

The README documents two known issues you will hit if you try to build from scratch rather than the official quickstart: a broken transitive dependency in azurefunctions-agents-runtime that pins a yanked version of github-copilot-sdk, and the region constraint. Both are worth knowing before you invest time in a custom deployment.

Up next: Durable Functions as the Orchestration Layer for Directed Agentic Workflows

The Four Things Naive RAG Diagrams Leave Out

Posted on August 12, 2026 by steefjan1970

You might have seen the diagrams like four boxes, left to right, with indexing, retrieval, augmentation, and generation. Parse the PDF, chunk the text, embed the chunks, store the vectors. Then embed the question, search, stuff the results into a prompt, and generate. I have seen it circulating every few weeks with a fresh coat of branding and a caption promising an end to hallucination.

The diagram is not wrong. It is a decent first explanation of naive RAG. The problem, however, starts when someone treats it as a design. TThe version I saw recently ended its augmentation box with three words: zero hallucination guaranteed.

That claim is where I want to start, because it is the tell. Retrieval-augmented generation reduces fabrication. It does not eliminate it. Anyone promising zero has not yet run an evaluation against their own system.

So here are the four things the four-box picture leaves out, in the order they will hurt you. I have put runnable samples for each one in a companion repository.

Gap 1: Retrieval is not the same thing as vector search

Naive RAG diagrams draw a single arrow from question to embedding to vector database. That works beautifully in demos, because demo questions are written in the same register as the source documents.

Production questions are not. They contain product codes, policy numbers, abbreviations, proper nouns, and negations. Embeddings capture meaning, and a product code has no meaning to capture.

I built a small corpus to measure this rather than assert it: eight synthetic Dutch policy documents, two consecutive years of the same policy, a collective variant with a structurally identical pricing table under different codes, and a separate reglement for medical aids. Thirty-three chunks. Then eleven questions with a known correct chunk for each.

What vector-only missed

Vector-only retrieval got seven of the eleven right. The failures were not random:

Which discount applies to code BAS-VR-400? returned the document’s changes section, which names the code but never prices it. Right document, wrong section, and the retrieved chunk looks relevant enough to answer from.
Does medical acceptance apply to package AANV-CO-03 in 2026? returned the 2025 document, which says the opposite: right topic, wrong year, inverted answer.
How many physiotherapy treatments are in the Extra package in 2025? returned a chunk from the basic policy entirely.

That second one is the one that should worry you. It is not a near miss. The prose in the two years is nearly identical, the answer is reversed, and nothing downstream can tell. An assessor reading a fluent, cited, confidently wrong answer about acceptance criteria has no signal that anything went sideways.

Keyword search, meanwhile, nails the code lookups. Okapi BM25 has been solving this problem since before any of us had an opinion about transformers.

Worth being precise about what this does and does not prove. My first version of this corpus had three documents and eight chunks, and vector-only scored four out of five, because with eight chunks there is nothing to confuse. The gap only appears once the corpus contains things that genuinely resemble each other. If your own evaluation shows dense retrieval doing fine, check whether your test set is hard before concluding your pipeline is.

Fusion widens the pool, reranking picks the answer

Therefore, the answer is not to pick a side. Azure AI Search will run both and fuse the result lists with reciprocal rank fusion. Then a semantic reranker reorders the fused list using a cross-encoder that actually reads the query against each candidate.

Here is where my expectations were wrong, and where the measurement earned its keep. Across the eleven questions:

Strategy	Top-1 correct	MRR@5
Vector-only	7 / 11	0.77
Hybrid (BM25 + vector, RRF)	6 / 11	0.72
Hybrid + semantic reranker	11 / 11	1.00

Adding keyword search made it worse. Hybrid lost a case that vector-only got right, and fixed none.

Why fusion alone went backwards

The reason is visible in the failures. BM25 matches the literal string BAS-VR-350, and that code appears in the document’s changes section, which names codes without pricing them. Lexical matching therefore promoted chunks that contain the code and cannot answer the question. Reciprocal rank fusion then faithfully merged two ranked lists, because RRF has no notion of whether a chunk answers anything. It fuses positions, not relevance.

The cross-encoder is what fixed it. It reads the question against each candidate and understands that a question about a discount needs the row with a price in it, not the sentence announcing that the code exists. That took the same candidate set from six correct to eleven.

So the lesson is sharper than “use hybrid search”. Hybrid retrieval widens the candidate pool; reranking is what converts a wider pool into better answers. Ship the first without the second, and you may go backward quietly, because nothing in the pipeline reports that it happened.

One clean question per turn is an assumption

Query handling is the other half of this gap. The diagram assumes one clean question per turn. Real questions arrive compound: we switched to the Compleet package in March, does my son’s dental work fall under that or under the basic policy, and does the deductible apply? That is three questions. Embed the whole sentence, and you retrieve the average of three intents, which is nothing in particular.

Agentic retrieval in Azure AI Search handles that by decomposing the query into subqueries, running them in parallel, reranking each, and merging. Extractive retrieval went generally available in API version 2026-04-01. Query planning and answer synthesis remain preview. Worth knowing which half you are depending on before you promise it to a steering committee.

Gap 2: Chunking is most of the work

“Chunk text for sharp recall.” One bullet. In practice, this single decision determines more of your answer quality than your choice of model.

Fixed-size splitting is what every quickstart does and what almost nothing should do. Run a 400-character window with 50 characters of overlap over a document containing a pricing table, and the splitter lands mid-row. This is the actual output from the sample, not an illustration:

                        | EUR 3,00          | BAS-VR-100  |
| EUR 200                 | EUR 6,50          | BAS-VR-200  |
| EUR 300                 | EUR 10,00         | BAS-VR-300  |
| EUR 400                 | EUR 14,00         | BAS-VR-400  |
| EUR 500                 | EUR 19,00         | BAS-VR-500  |

## 2. Fysiotherapie

Fysiotherapie wordt vanaf de 21e

Look at what survived. No header row, so nothing says which column is the deductible, which is the monthly discount, and which is the product code. The first row is cut mid-cell: its deductible tier is gone, leaving a discount attached to nothing. No document title, so nothing says this is the 2026 basic policy rather than the 2025 one or the collective variant, all three of which carry a table of exactly this shape with different numbers. And the chunk runs on into an unrelated section about physiotherapy, ending mid-sentence.

Retrieve that, and the model has to guess. It will guess. It will sound certain. And the citation attached to it will make the wrong answer more credible, not less.

What structure-aware chunking does differently

The sample runs three chunkers over the same document and counts how many chunks ended up holding table rows with no header. Fixed-size produces one out of three. Recursive paragraph splitting produces none, but leaves every chunk without a section heading. Structure-aware produces four chunks, none headerless, none context-free.

The difference is three rules, and none of them is clever: split on headings rather than character counts, never split a table, and prepend the document title and section heading to every chunk so an isolated chunk still says what it is.

That last rule is what makes the three near-identical pricing tables in this corpus distinguishable at all. Without it, retrieval has to tell them apart on the numbers alone.

Gap 3: Retrieval without authorization is a breach with a chat interface

This is the gap that should worry you most, and it is absent from every version of the diagram I have seen.

Put every document in one index. Wire up a chat interface. Now every user can reach every document, because semantic search does not know about your authorization model. The retrieval layer will happily surface an internal work instruction, an HR file, or a legal memo to whoever asks a question shaped roughly like its contents.

The filter is a query construct, not a prompt instruction

Two things follow. First, the filter is a server-side query construct, not a prompt instruction. Telling the model “only use documents the user may see” is not a control; it is a suggestion to a system that has already been handed the text. Second, the filter must derive from validated token claims, never from anything the user typed.

Two mechanisms, and the one that fails open

Azure AI Search gives you two mechanisms. The durable one is an explicit filterable collection of group identifiers on each document plus an OData filter built from the caller’s claims, which works today on the stable API and which you own end to end. The managed one ingests RBAC scopes, ACLs, or Purview sensitivity labels alongside the content and enforces them at query time when you pass the user’s token in the x-ms-query-source-authorization header.

The managed route has a sharp edge worth memorizing. If the knowledge source was created without ingestionPermissionOptions, the index holds no permission metadata, and results come back unfiltered regardless of the header. It fails open quietly, and the only way to fix it is to recreate the knowledge source. As of the current GA release, document-level permissions on indexed sources remain in preview.

Whichever you choose, write the leak test. The sample repository includes one: a query, an unauthorized caller, and an assertion that fails the build if the restricted document comes back. Twelve lines. Run it in CI.

Gap 4: “Zero hallucination” is a claim, and claims get measured

Grounding the prompt does not guarantee a grounded answer. Three failure modes survive the diagram intact.

The model can prefer what it already knows over what you retrieved. Ask about a monthly premium that appears nowhere in your corpus, and a model trained on the open internet has plausible Dutch premiums available. It will produce one.

The model can blend two chunks into a claim neither of them makes. This is the subtle one, because every individual fact traces back to a source.

And the model can answer confidently when retrieval returned nothing relevant at all, because nothing in the naive RAG pipeline tells it that “I do not know” is an available output.

Three rules that make it measurable

Consequently, the fix is threefold and unexciting: an explicit refusal string in the prompt so refusal is detectable rather than inferred, mandatory citation of a reference identifier after every claim so each statement is checkable, and an evaluation set that contains questions your corpus cannot answer.

That last point is the one most teams skip. Everyone builds a golden set of questions the documents answer well. Almost nobody includes the withdrawn product code, the topic that was never documented, or the answer that lives in a file the user may not see. Those are exactly the cases that generate the incident report.

Score retrieval and generation separately, too. A wrong answer with good retrieval is a generation problem. A wrong answer with bad retrieval is a retrieval problem. Without both numbers, you will spend a week tuning the wrong half of the system.

What the numbers actually said

Then the result, which surprised me: fourteen out of fourteen on groundedness and valid citations, and five out of five refusals on the unanswerable questions. The model corrected false premises rather than accepting them; asked whether medical acceptance applied in 2026, it answered yes and cited; asked about a product code withdrawn before the corpus begins, it refused outright rather than interpolating a plausible price from the neighboring rows.

I want to be careful about what that number means, because it is easy to oversell in the other direction. Retrieval hit fourteen out of fourteen in the same run, so generation was working from good material throughout. This is not evidence that hallucination is solved. It is evidence that the three unexciting mechanisms above are sufficient when retrieval is doing its job, which is the argument for building all four gaps rather than any one of them.

One caveat I would want stated if someone showed me this number: the judge was the same model as the generator. A model scoring its own output shares its own blind spots, and the groundedness figure is inflated to an unknown degree by that. Use a different model for judging if the number needs to carry weight.

Where this is the wrong answer

If your corpus is fifty pages of prose, with no product codes, no tables, one audience, and low stakes, then the four-box diagram is enough. Hybrid retrieval, structure-aware chunking, security trimming, and a groundedness harness are all overhead you do not need yet. Build the simple thing, ship it, and see what breaks.

The four gaps become urgent at specific, recognizable moments: the first exact-match question that returns the wrong table row, the first document that should not be visible to everyone, and the first stakeholder who asks how you know the answers are right. If none of those have happened, you are fine.

The part the diagram gets right

Context beats prompt engineering. That much is true, and it is the reason the picture keeps circulating. But “give the model the right information at the right time” restates the problem. It is not a solution. The right information depends on hybrid retrieval and reranking. The right time depends on query decomposition. Whether the user was entitled to that information depends on trimming. And whether the model actually used it depends on evaluation.

Do all four and the numbers hold up — mine did. Skip any one of them, and you will not find out which one you skipped until someone asks a question about last year’s policy and gets this year’s answer.

Four boxes, four gaps. The gaps are where the engineering lives.

The samples are at steefjan1/naive-rag-gap: hybrid retrieval and reranking, chunking, security trimming, and groundedness evaluation, provisioned with a single azd up. Four of the five run end to end against a live service; the agentic retrieval sample needs a knowledge base that none of the scripts create, so treat that one as a sketch rather than a worked example.

They target Azure AI Search API versions current as of August 2026. Agentic retrieval and document-level permissions are both moving quickly, so check the docs before assuming a preview flag is still a preview flag.

Citadel Grew Up: What the citadel-v1 Release Means If You Built on the AI Hub Gateway.

Posted on August 12, 2026 by steefjan1970

The Citadel Governance Hub accelerator that sits underneath my entire five-part Citadel Platform series just had a significant release. In addition, the citadel-v1 branch of the AI Hub Gateway Solution Accelerator repositions the project from “a solid APIM gateway pattern” to the official reference implementation of Layer 1 in Microsoft’s AI Citadel Blueprint.

I cloned the branch and went through it with one question in mind: what does this change for anyone who, like me, deployed and built on the earlier iteration? The answer starts with one finding. As a practitioner, the whole point of this blog is honesty: the API surface I used throughout the series is now explicitly labeled legacy.

Note that citadel-v1 has not yet been merged to main; if you deployed from the main branch without specifying --branch citadel-v1, you are on the earlier architecture.

Let’s start with the bigger picture, then get to that.

The 4-layer AI Citadel Blueprint

The README now frames the accelerator as one layer of a larger architecture. The AI Citadel Blueprint describes four interlocking layers, each with its own responsibility and implementation:

Layer 1, the Governance Hub, is this accelerator: runtime enforcement through a unified AI gateway, policy-as-code, identity validation, token rate limiting, content filtering, and cost attribution—everything my series built and tested lives in this layer.
Next, layer 2, AI Control Plane, covers the agent runtime, observability, and compliance: agent traces, AI evaluations, and fleet operations, implemented through the Microsoft Foundry control plane.
Subsequently, layer 3, Agent Identity, handles agent identity and lifecycle governance through Agent 365: unique agent identities, blueprints, shadow agent detection, and a sponsorship model.
And finally, layer 4, the Security Fabric, provides unified protection through Microsoft Defender for AI threat intelligence, Purview for data governance, and Entra for authentication and authorization.

Looking back at the series through this lens, my five posts covered Layer 1 thoroughly, and the registry work with Azure API Center reached into Layer 2 territory before the layer had that name. The kill switch from Part 4 sits squarely in Layer 1 as runtime enforcement. What the series never touched, and what I now have vocabulary for, is Layers 3 and 4. That’s useful: it turns “what’s missing from my platform” from a vague feeling into a named checklist.

What citadel-v1 Changes in the AI Hub Gateway: The API Surface

Here’s the finding that matters most if you followed the series. The new LLM Access Guide defines three API surfaces on the gateway, and it’s blunt about which one you should use.

The Azure OpenAI API surface, at /openai/deployments/{deployment-id}/*, preserves the exact URL shape the Azure OpenAI SDK expects. This is what Part 2 of my series wired the weather agent against, and it’s what every code sample in the series uses. The guide now labels it “legacy integration only,” for existing code that pins that URL shape. Not the target state for new work.

The Universal LLM API, at /models/*, exposes a clean OpenAI v1-compatible surface across many models and providers through a single stable path.

The Unified AI API, at /unified-ai/*, is the recommended surface: a single wildcard endpoint that serves OpenAI-compatible calls and every provider-native pattern with dynamic routing behind it.

LLM ACCESS guide

The citadel-v1 branch documents these three surfaces in its LLM access guide. Check the guides folder in the branch for the current filename, as the documentation is actively evolving.

I want to be precise about what this does and doesn’t mean. Nothing broke. Code targeting /openai/deployments/... keeps working, and the surface exists precisely because migrations take time. But the arrow points one way: new integrations should target /unified-ai/v1/*, and my series should be read with that footnote attached. If I started the series today, Part 2 would look different.

This doesn’t invalidate the architectural argument, and I’d argue it strengthens it. The reason the series routed the standard OpenAI SDK through APIM was to keep every call on a governed path. The Unified AI API is that same principle with a better front door: one endpoint, every provider, every pattern, all governed. The lesson survived the release; only the URL changed. Citadel is evolving fast, and this is what evolving looks like from the inside.

Contract-driven everything

The second big theme in citadel-v1 is contracts, and if you read my registry post about the AI Publish Contract, this will feel familiar in the best way.

The accelerator now ships a Citadel Access Contract package: declarative, version-controlled .bicepparam files that onboard an AI use case end-to-end. One contract deployment creates the APIM product (with naming like LLM-Healthcare-PatientAssistant-DEV), the subscription with its key, optional Key Vault secret storage, and optionally an APIM connection for Microsoft Foundry agents. I described this as a pattern worth building in the registry post, and the access contract is now live while the publish contract remains upcoming in the current release.

Alongside it sits a backend onboarding contract (llmBackendConfig) for declaratively registering LLM backends, and the whole thing is versioned through a release.json manifest at the repository root. That manifest is worth a moment of appreciation: instead of one monolithic version number, it tracks independent, component-scoped versions for the routing logic, the backend contract shape, the access contract shape, and the usage ingestion pipeline. A change to routing doesn’t force a re-version of contracts that didn’t change. That’s a small design decision that signals the project expects to be operated, not just deployed once.

The parallel to the AI Publish Contract from my registry post is direct. Both encode the same conviction: onboarding an AI workload should be a reviewed, versioned artifact in a repository, not a sequence of portal clicks someone half-remembers. The access contract governs how a workload reaches the gateway. The publish contract governs registration and description. A mature platform wants both.

Multi-provider routing, briefly

The gateway is no longer an Azure OpenAI front door with ambitions. AWS Bedrock, Google Gemini, and Anthropic Claude are first-class citizens, each available through OpenAI-compatible access, provider-native access, or both.

The design that makes this work without chaos is a fragment-based routing architecture, and one detail from the onboarding guide shows how much operational scar tissue is encoded in it. Every API type declares its own compatible pool types, and the Universal LLM API restricts pool selection to OpenAI-compatible pools before backend selection runs. Why? Because if the same model ID is registered against both a native Bedrock pool and an OpenAI-compatible one, a naive router could send an unrewritten OpenAI-shaped path to the native provider, which answers with something as friendly as com.amazon.coral.service#UnknownOperationException. The guide documents the failure mode by name. Someone hit that error so you don’t have to, which is exactly what a good accelerator encodes.

For a platform team, the practical consequence is real: model choice becomes a routing decision instead of an architecture decision. Adding Claude or Gemini to an estate governed by the hub doesn’t create a second governance perimeter. It adds a backend behind the one you already operate.

What I’d do differently starting today

Distilling this into advice for anyone deploying now:

Target the Unified AI API from day one. Start at /unified-ai/v1/* with the standard OpenAI SDK. You get the same governed path my series argued for, plus provider reach and a native-access upgrade path you’ll eventually want.

Adopt the access contract instead of hand-rolling onboarding. The .bicepparam contract per use case gives you reviewable, repeatable onboarding with product, subscription, and secrets in one deployment. I built a weaker version of this by hand during the series; you don’t have to.
Pin your contract versions consciously. release.json gives you independent version tracks. Treat contract shape changes as reviewable events in your own repo, the same way you’d treat an API schema change.
Look at the PII blocking mode. The PII framework now supports managed identity authentication to the Language Services, regex pre-processing before NLP detection, and a strict mode that rejects requests containing PII with a 400 instead of masking. For regulated industries, that hard-fail option changes the compliance conversation: some data should never reach the model, masked or not.

What’s next

The obvious follow-up experiment: migrating the weather agent from the legacy /openai/deployments/... path to the Unified AI API, documenting whatever breaks along the way. If the routing architecture delivers on its promise, that migration should be a base-URL change. If it isn’t, that’s a post worth writing too.

The accelerator that started this series as a useful pattern is now the reference implementation of a named layer in a published blueprint, with contracts, multi-provider routing, and a defined seam toward agent-runtime governance. Preview or not, the direction is clear, and it’s the direction the series has been arguing for all along: one governed front door, everything registered, nothing invisible.

If you’ve deployed citadel-v1 or migrated from the earlier iteration, I’d like to hear what surprised you.