Azure Functions AI Integration: The Quiet Powerhouse

Posted on by
Reply

When people talk about running AI workloads on Azure, the conversation usually lands on Azure AI Foundry, Azure OpenAI Service, or maybe Azure Container Apps. Azure Functions tends to get mentioned as the glue the thing you bolt on to handle a webhook. But this overlooks the real story: Azure Functions AI integration has quietly evolved from simple glue into a powerhouse for running production-grade AI.

That framing is outdated. Azure Functions is now a first-class runtime for AI workloads, with four distinct patterns that the Microsoft Learn documentation lays out explicitly. This post walks through each of them and helps you decide which one fits your situation.

The four AI-enabled scenarios

Microsoft groups Azure Functions AI integration into four scenarios:

  1. Serverless agents runtime — event-driven agents that run on serverless infrastructure
  2. Tools and MCP servers — hosting remote Model Context Protocol servers and AI tools
  3. Agentic workflows — multistep, long-running directed agent operations via Durable Functions
  4. Retrieval-augmented generation (RAG) — fast, parallel data retrieval for knowledge-augmented AI
Structural diagram showing the Azure Functions platform envelope containing four patterns in a two-by-two grid — serverless agents runtime, MCP servers and tools, agentic workflows via Durable Functions, and RAG pipeline — with Foundry Agent Service, Azure OpenAI, APIM, and AI Search shown below as connected surrounding services.
The four AI-enabled patterns inside the Azure Functions platform and the surrounding Azure services they integrate with.

These are not just marketing buckets. Each one reflects a different architectural decision. Let me unpack them.

Azure Functions AI integration: Serverless agents runtime

The serverless agents runtime is a preview programming model for building event-driven agents as function apps. Moreover, Agents are defined in .agent.md files, app-wide runtime defaults live in agents.config.yaml, and remote MCP server connections are listed in mcp.json. The runtime discovers these files, registers the required triggers and endpoints, and runs the agent through the Microsoft Agent Framework when an event fires.

That is a meaningfully different model from what you get in Azure AI Foundry Agent Service. In Foundry, the managed service hosts and orchestrates your agents. In addition, in the serverless agents runtime, your function app is the agent host running on Flex Consumption, with built-in managed identity, monitoring, and scale-to-zero. Furthermore, you write custom Python tools for app-specific logic, and the platform wires in MCP-enabled connections based on Azure connectors and remote MCP servers.

Use this when: you want agents triggered by events, schedules, messages, or HTTP requests and you need the familiar Functions deployment and hosting model rather than a managed agent service.

Avoid it when: you need a fully managed, enterprise-grade agent service with built-in tooling and long-term Microsoft support guarantees today. That is what Foundry Agent Service is built for.

Azure Functions AI integration: Tools and MCP servers

The Model Context Protocol (MCP) has become the industry standard for how AI models and agents interact with external systems. Azure Functions has first-class support for hosting remote MCP servers, and this is already generally available.

There are two hosting options:

OptionStatusHow it works
MCP binding extensionGAUses Functions triggers and bindings; supports stateful execution
Self-hosted MCP servers (MCP SDK)PreviewUses standard MCP SDKs via custom handlers; requires Streamable HTTP transport
Flowchart showing an AI agent at the top fanning out via L-shaped connectors to three columns: MCP binding extension (GA, synchronous, stateful, all major languages), self-hosted MCP SDK server (preview, Streamable HTTP, no stateful execution yet), and queue-based tool (async, decoupled, fault-tolerant delivery via Service Bus or Storage Queue). Each column lands on a function app box at the bottom.
An AI agent reaches Azure Functions via three distinct paths, each with different trade-offs on availability, transport, and execution model.

The binding extension is the right default. It supports C#, Python, TypeScript, JavaScript, and Java, and it integrates with the Functions programming model you already know. Self-hosted MCP servers offer portability: you can use the official MCP SDKs and bring in existing server code. However, stateful execution is not yet supported, and the configuration is still changing during preview.

There is also a third option worth knowing about: queue-based Azure Functions tools, where AI agents interact with your code through message queues rather than direct MCP calls. Microsoft Foundry provides specific Azure Functions tooling for this pattern. It is ideal when you need reliable delivery, built-in retry, and decoupling between agent and function execution.

Use MCP servers when: you are exposing tools to AI clients and you want the industry-standard protocol with serverless hosting.

Use queue-based tools when: you need asynchronous, fault-tolerant communication between an agent and your function code.

Agentic workflows with Durable Functions

Not all AI orchestration should be autonomous. Some scenarios need predictable, directed steps and that is where Durable Functions fits.

The Microsoft Learn documentation makes the distinction clearly: Durable Functions is positioned as the runtime for directed agentic workflows, not for emergent agent reasoning. Think of it this way: when you know the sequence of steps and you need fault tolerance, auditability, and long-running execution, Durable Functions is the right tool. When you want a model to figure out the steps dynamically, you want an agent runtime.

The documentation gives a clean example: a trip planning workflow that gathers user requirements, searches for options, waits for approval, and makes bookings. Each step is a function; Durable Functions coordinates them with built-in retry, state persistence, and human-in-the-loop support.

Use this when: your AI-driven process has well-defined, ordered steps,authorization flows, multi-stage approval chains, or orchestrated data pipelines where you cannot afford unpredictable execution paths.

Avoid it when: you want a model to determine steps dynamically. That is the serverless agents runtime or Foundry Agent Service territory.

RAG with Azure Functions

Because Functions handles multiple events from various data sources simultaneously, it scales well for real-time AI scenarios, particularly RAG systems where fast, parallel retrieval is the bottleneck.

The Azure OpenAI binding extension lets you integrate RAG directly into your function code. Functions can pull data from multiple sources simultaneously, feed it through Azure AI Search or other retrieval layers, and pass the results to your language model, all within the event-driven, scale-to-zero model that keeps costs down when load is low.

The Azure Functions RAG pattern also pairs naturally with APIM, which handles routing, rate limiting, and token quota management a pattern the Citadel Platform series covers in detail, including the discovery that the Foundry Agent Service SDK bypasses APIM for LLM calls.

Use this when you have event-driven retrieval requirements new documents arriving in blob storage, database change feeds, or streaming IoT data that needs to inform model responses.

How the scenarios relate to other Azure services

It helps to think of Azure Functions AI integration as filling the compute and integration layer between your AI services and your data sources. Here is roughly how that maps:

  • Azure AI Foundry Agent Service — fully managed agent orchestration with enterprise security and built-in tools. Functions integrates into Foundry via MCP servers and queue-based tools.
  • Azure Logic Apps — low-code orchestration for business process automation. Functions is the right choice when you need custom code, complex event processing, or lower latency.
  • Azure Container Apps — container-based hosting for long-running services. Functions on Flex Consumption beats it on cost for bursty, event-driven AI workloads that spend time idle.
  • Durable Functions — lives inside Functions and adds stateful, long-running orchestration. Use it for directed agentic workflows; use the serverless agents runtime for event-driven agents.
Four-tier layered diagram. Bottom tier: data sources and messaging (Blob Storage, Cosmos DB, Event Hubs, Service Bus, Azure SQL, IoT Hub). Second tier: Azure Functions compute and integration layer containing four pattern pills — agents runtime, MCP servers, Durable workflows, and RAG pipeline. Third tier: managed AI services (Azure OpenAI, AI Search, APIM, Microsoft Agent Framework). Top tier: AI clients and agents (Foundry Agent Service, Semantic Kernel, custom SDK agents, chat apps). Arrows show data flowing upward and tool calls flowing downward.
Azure Functions fills the compute and integration layer between your data sources and your managed AI services.

The underlying platform advantage

Across all four scenarios, the same hosting model applies: Flex Consumption. It offers fast, event-driven scaling, virtual network integration, and pay-as-you-go billing. For AI workloads, which tend to be bursty rather than continuous, this is a significant cost advantage over always-on hosting.

Managed identity, Application Insights integration, and azd-based deployment are consistent across all four patterns. That means your security posture, observability, and deployment pipeline do not have to change when you move from a simple timer trigger to hosting a remote MCP server.

Azure Functions AI integration: Choosing the right pattern

Here is a simple decision table:

I want to…Use…
Build event- or schedule-triggered agents with MCP toolsServerless agents runtime (preview)
Expose tools to AI clients via the industry-standard protocolMCP binding extension (GA)
Orchestrate predictable, multistep AI-driven processesDurable Functions
Build a RAG pipeline with fast, parallel data retrievalAzure Functions + Azure OpenAI binding
Fully managed agent hosting with enterprise SLAsAzure AI Foundry Agent Service

What comes next

The rest of this series goes deep on each pattern. The next post covers the two MCP server hosting options in detail binding extension versus self-hosted SDK servers, including where the current preview constraints matter in practice.

Up next: Hosting Remote MCP Servers in Azure Functions: GA vs. Preview Options

Azure Messaging and Orchestration for Integration Architects

Posted on by
Reply

In the Azure PaaS map post, the integration layer got a single paragraph. It named four services: Logic Apps, API Management, Service Bus, and Event Grid, and moved on. This post takes Azure messaging and orchestration apart into the decisions underneath that paragraph.

I won’t tour features here. Two of these services already have their own deep series on this blog, so re-covering them would waste your time. Instead, I’ll stay at the decision layer. When do you reach for which? And why do teams so often reach wrong? Those are the questions that actually cost you in production.

Azure messaging and orchestration: two axes decide almost everything

Four services sound like four choices. In practice, though, only two questions matter, and they cut across the whole layer.

Decision diagram with two axes. The first splits messaging from orchestration. For messaging, the cost of a lost message chooses between Service Bus and Event Grid. For orchestration, complexity and ownership choose between Logic Apps and code. A band across the bottom shows API Management as the control point in front of all of it.
You don’t choose between four services; you answer two questions: messaging or orchestration, then failure cost or ownership. The service falls out from there, with API Management governing the front.
  • First: is this messaging or orchestration? Messaging moves events and data between systems. Orchestration coordinates a multi-step process toward an outcome. The two look similar on a whiteboard, but they fail differently, scale differently, and belong to different services. So separate them before anything else.
  • Second: what does failure cost, and what shape is the work? Once you know whether you’re moving messages or coordinating steps, the follow-up question splits the choice further. For messaging, the cost of a lost message decides it. For orchestration, the complexity of the flow and the team who owns it decide it.

Get those two axes clear, and the service almost picks itself. Skip them, and you end up with Event Grid where you needed guarantees, or a Logic App doing work that belonged in code.

Messaging: Service Bus vs Event Grid

Both move things between systems. That’s where the similarity ends.

  • Service Bus is the durable, ordered, transactional backbone. Reach for it when delivery has to be guaranteed. It gives you sessions for ordered processing, dead-lettering for messages that can’t be handled, and transactional handling across multiple operations. Topics and subscriptions add pub/sub without a separate broker. So Service Bus fits business messages: an order, a payment, a claim, where losing one is an incident.
  • Event Grid is a lightweight, high-volume router. It broadcasts events to whoever cares: resource state changes, custom application events, and telemetry. It’s built for throughput and fire-and-forget delivery, not guaranteed processing. Therefore, it fits notifications and reactive triggers, where a missed event is a shrug rather than a page.

Here’s the rule of thumb I give teams new to Azure messaging. If losing a message would be a business incident, it belongs on Service Bus. If losing it would just mean a missed notification, Event Grid is fine.

And often you use both. A common pattern pairs them: Event Grid fans out a notification, and a subscriber drops a durable message onto Service Bus for guaranteed processing. That way you get Event Grid’s reach and Service Bus’s reliability in one flow, each doing the job it’s good at.

Pattern diagram showing a source emitting an event to Event Grid, which fans out to a notification subscriber, a logging subscriber, and a bridge subscriber. The bridge subscriber drops a durable message onto Service Bus, where a processor handles it with guaranteed ordered delivery and dead-lettering.
Event Grid fans an event out to multiple subscribers; the one that needs a guarantee drops a durable message onto Service Bus for ordered, dead-lettered processing.

The honest note, though, is that this is where teams get burned. Event Grid looks simpler, so teams default to it. Then, weeks later, they discover the workload actually needed ordering or delivery guarantees. Now they’re bolting reliability onto a service that was never designed for it. So decide on the message-loss cost first, before the “which feels easier” instinct takes over.

Orchestration: Logic Apps vs code

Messaging moves things. Orchestration coordinates them. The decision here isn’t about reliability; it’s about complexity and ownership.

  • Logic Apps is the designer-first route. It shines when you need enterprise connectors SAP, IBM MQ, mainframe hosts, the long tail of line-of-business systems without a modern REST API. Standard Logic Apps also closes the old gaps that made it hard in regulated environments: VNet integration, built-in state, per-workflow scaling. So for a workflow that a less code-heavy team will own and maintain, Logic Apps is often the right call even when a Function would be more elegant.
  • Code is the route once complexity climbs. Designer workflows are fast to build and easy to read at first. Past a certain size, though, they get hard to reason about and harder to code-review. In my experience, the practical ceiling sits around a dozen actions with a couple of branches. Beyond that, do one of two things. Either decompose the workflow into smaller ones, or move the logic into a Function where a proper language and real tests take over.

The deciding questions, then, are simple. Who maintains this: a low-code team or engineers? How complex is the flow really? And can you review it a year from now? For the deeper mechanics of building agentic workflows in Logic Apps, I covered that ground in the Logic Apps Agent Loop series so that I won’t repeat it here.

Where API Management fits

Azure API Management (APIM) isn’t messaging or orchestration. Instead, it’s the control point in front of both.

It sits between consumers and whatever does the real work: a Logic App, a Function, an App Service backend. From there, it enforces rate limits, authentication, transformation, and policy-based routing. So when multiple consumers hit a shared set of backend capabilities, APIM lets you change the implementation behind them without breaking anyone, and lets you enforce policy without touching application code.

That’s all I’ll say here, because APIM earns a series of its own. I went deep on it in the APIM for AI workloads series, including how it behaves as an AI gateway. For this layer, treat it as the front door that governs whatever messaging and orchestration sit behind it.

Where each is the wrong answer

Every service here has a failure mode when you reach for it by reflex. So, to keep this honest:

  • Service Bus is wrong for high-volume telemetry: If you’re routing millions of fire-and-forget events and none of them individually matter, Service Bus is expensive overkill. Use Event Grid.
  • Event Grid is wrong for anything needing order: The moment sequence or guaranteed delivery matters, Event Grid stops fitting. Move to Service Bus before the gap bites.
  • Logic Apps is wrong past its complexity ceiling: A workflow with thirty actions and nested branches is a maintenance liability in the designer. Decompose it, or move it to code.
  • Code is wrong for something a citizen developer should own: Not every integration belongs in a repo. If a low-code team can own and maintain a simple connector-driven flow, hand-writing it in a Function just centralizes work that didn’t need to be centralized.

The shape of it

Azure gives you four integration services, but you don’t choose between four things. You answer two questions. Is this messaging or orchestration? And then what does failure cost, and who owns the work? Answer those, and Service Bus, Event Grid, Logic Apps, or a Function each falls out naturally, with APIM governing the front.

In practice, real platforms use several together: APIM, fronting Logic Apps, and Functions, Event Grid fanning out to Service Bus for reliable processing. So the craft of Azure messaging and orchestration isn’t picking a winner. It’s drawing clean boundaries between them.

Want the layer above this one? The Azure PaaS map puts integration in context against compute, data, and governance, and walks the five-question framework for choosing across all of them.

Citadel APIM Kill Switch: Stop a Governed Agent Cold

Posted on by
Reply

In the previous post we added conversation persistence to the Microsoft Foundry Citadel Platform on Azure. As a result, every agent run now produces a structured document in the spoke’s Cosmos DB conversations container. The agent is fully operational: it routes through the APIM governance hub, executes tool calls, stores its history, and returns grounded responses. However, the question that every enterprise AI architect eventually faces remains: what happens when it needs to stop?

Not a graceful shutdown. Not a redeployment. An immediate, operator-triggered containment the kind you need when an agent is behaving unexpectedly, consuming runaway tokens, or has been flagged by your security team. In a Microsoft Foundry Citadel Platform on Azure deployment, the answer is the Kill Switch: a layered containment system built into the APIM hub that stops agent traffic cold without touching the agent code, the spoke, or the Azure OpenAI deployment.

This post implements three of the five Citadel kill switch layers against the hub we deployed in Sweden Central:

  • Layer 1 — Named Value flip: instant global block via a single boolean
  • Layer 2 — JWT claim block: identity-based containment via header validation
  • Layer 3 — Agent ID blocklist: surgical per-agent blocking

The Scenario

The weather agent (agent_with_memory.py) is running in production. Specifically, it is routing through apim-wpvlimv4ngkns.azure-api.net, storing conversations in the spoke Cosmos DB, and generating token usage events in the hub Cosmos DB. Everything is working. Then your security team flags it. The agent needs to stop immediately while the incident is investigated. You have seconds, not minutes. For example, redeploying the spoke takes too long. Rotating the APIM subscription key is irreversible and affects all consumers. Therefore, the Kill Switch is the right tool.

The Kill Switch is the right tool. The APIM hub has built-in pre-wiring, requires no code changes, and can trigger actions in under 30 seconds.

To ensure reliability, always pre-wire the kill switch as Layer 1 before you need it. Remember, you can’t flip a Named Value that doesn’t exist. In addition, the inbound policy must already be in place, checking the Named Value on every request, before any incident occurs.

Prerequisites

From the previous posts you should have:

  • Hub deployed in rg-ai-hub-gateway-dev with APIM instance apim-wpvlimv4ngkns
  • agent_with_memory.py running and saving to Cosmos DB
  • Azure CLI authenticated

Citadel Kill Switch Layer 1 — Named Value Flip

How It Works

A Named Value called kill-switch-enabled is created in APIM and set to false. An inbound policy on the OpenAI API checks this value on every request. When the value is flipped to true, all requests through the gateway immediately return HTTP 403 — no code changes, no redeployment, no spoke involvement.

Step 1.1 — Create the Named Value

az apim nv create `
  --resource-group rg-ai-hub-gateway-dev `
  --service-name apim-wpvlimv4ngkns `
  --named-value-id kill-switch-enabled `
  --display-name "kill-switch-enabled" `
  --value "false" `
  --secret false

Verify it was created:

az apim nv show `
  --resource-group rg-ai-hub-gateway-dev `
  --service-name apim-wpvlimv4ngkns `
  --named-value-id kill-switch-enabled `
  --query "value" -o tsv

Should return false.

Step 1.2 — Add the Inbound Policy

In the Azure Portal:

  1. Navigate to apim-wpvlimv4ngkns → APIs → Azure OpenAI Service API → All operations
  2. Click Policies → Inbound processing → Edit
  3. Add this policy inside the <inbound> section, before any other policies:
<!-- Kill Switch Layer 1: Named Value flip -->
<set-variable name="killSwitchActive" value="@("{{kill-switch-enabled}}" == "true")" />
<choose>
    <when condition="@((bool)context.Variables["killSwitchActive"])">
        <return-response>
            <set-status code="403" reason="Agent Suspended" />
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-header name="x-kill-switch-layer" exists-action="override">
                <value>1-named-value</value>
            </set-header>
            <set-body>{"error": {"code": "KillSwitchActive", "message": "Agent access has been suspended by the governance hub.", "layer": 1}}</set-body>
        </return-response>
    </when>
</choose>

The dedicated Named Value check policy provides a cleaner approach.

<!-- Kill Switch Layer 1: Named Value flip -->
<!-- Pre-wire this BEFORE any incident. Set kill-switch-enabled=true to activate. -->
<set-variable name="killSwitchActive" value="@("{{kill-switch-enabled}}" == "true")" />
<choose>
  <when condition="@((bool)context.Variables["killSwitchActive"])">
    <return-response>
      <set-status code="403" reason="Agent Suspended" />
      <set-header name="Content-Type" exists-action="override">
        <value>application/json</value>
      </set-header>
      <set-header name="x-kill-switch-layer" exists-action="override">
        <value>1-named-value</value>
      </set-header>
      <set-body>{"error": {"code": "KillSwitchActive", "message": "Agent access has been suspended by the governance hub. Contact your administrator.", "layer": 1}}</set-body>
    </return-response>
  </when>
</choose>

Click Save.

Step 1.3 — Confirm Agent Runs Normally

With kill-switch-enabled set to false, the agent should still work:

python agent_with_memory.py

Expected output: normal run, conversation saved, answer returned.

Step 1.4 — Trigger the Kill Switch

az apim nv update `
  --resource-group rg-ai-hub-gateway-dev `
  --service-name apim-wpvlimv4ngkns `
  --named-value-id kill-switch-enabled `
  --value "true"

Now run the agent:

python agent_with_memory.py

Expected output:

The agent stops. No spoke changes occur. No code changes happen. One CLI command executes.

Step 1.5 — Reset

az apim nv update `
  --resource-group rg-ai-hub-gateway-dev `
  --service-name apim-wpvlimv4ngkns `
  --named-value-id kill-switch-enabled `
  --value "false"

Citadel Kill Switch Layer 2 — Agent Approval Header

How It Works

The agent is required to pass a custom header x-agent-token containing a signed JWT with a specific claim (agt-approved: true). The APIM inbound policy validates this claim. If the claim is absent or the token is invalid, the system blocks the request with a 401 status. This action simulates identity-based containment, revoking the agent’s token or invalidating its claim at the identity provider level.

Step 2.1 — Update the Agent to Send a Header

Add the x-agent-token header to agent_with_memory.py. In this demo, we simulate the token by using a simple header value. IIn a production environment, Entra ID issues a JWT.

Modify the AzureOpenAI client creation in agent_with_memory.py:

client = AzureOpenAI(
    azure_endpoint=apim_base,
    api_key=cfg["APIM_SUBSCRIPTION_KEY"],
    api_version="2024-02-01",
    default_headers={
        "x-agent-id": "citadel-weather-agent-v1"
    }
)

Step 2.2 — Add the JWT Claim Check Policy

In the portal, add this policy after the Layer 1 block in the inbound section:

<!-- Kill Switch Layer 2: Agent approval header check -->
<choose>
  <when condition="@(context.Request.Headers.GetValueOrDefault("x-agent-approved", "false") != "true")">
    <return-response>
      <set-status code="401" reason="Agent Not Approved" />
      <set-header name="Content-Type" exists-action="override">
        <value>application/json</value>
      </set-header>
      <set-header name="x-kill-switch-layer" exists-action="override">
        <value>2-agent-approval</value>
      </set-header>
      <set-body>{"error": {"code": "AgentNotApproved", "message": "Agent identity could not be verified. Approval header missing or invalid.", "layer": 2}}</set-body>
    </return-response>
  </when>
</choose>

Step 2.3 — Trigger Layer 2

Remove the x-agent-approved header from the agent (or set it to false) and run:

python agent_with_memory.py

Expected output:

Note the response header x-kill-switch-layer: 2-agent-approval this indicates which containment layer fired and is critical for incident triage.

Pitfall: Policy Order Matters

Layer 1 must appear before Layer 2 in the policy document. APIM evaluates inbound policies top to bottom and stops at the first <return-response>. If Layer 2 appears before Layer 1, a globally suspended agent would return a 401 (identity error) instead of a 403 (suspended), obscuring the true containment reason in your incident log.

Citadel Kill Switch Layer 3 — Agent ID Blocklist in APIM

How It Works

A Named Value called blocked-agent-ids holds a comma-separated list of agent IDs. The inbound policy checks the x-agent-id header against this list. When agents match, the system blocks them with a 403 status code. Non-matching agents continue operating normally. This approach allows for surgical containment, stopping one specific agent while allowing all others to function.

Step 3.1 — Create the Blocklist Named Value

az apim nv create `
  --resource-group rg-ai-hub-gateway-dev `
  --service-name apim-wpvlimv4ngkns `
  --named-value-id blocked-agent-ids `
  --display-name "blocked-agent-ids" `
  --value "none" `
  --secret false

Start with an empty value — no agents blocked.

Step 3.2 — Add the Blocklist Policy

Add this policy after Layer 2 in the inbound section:

<!-- Kill Switch Layer 3: Agent ID blocklist -->
<set-variable name="agentId" value="@(context.Request.Headers.GetValueOrDefault("x-agent-id", ""))" />
<set-variable name="blockedIds" value="@("{{blocked-agent-ids}}")" />
<choose>
    <when condition="@{
        var agentId = (string)context.Variables["agentId"];
        var blockedIds = (string)context.Variables["blockedIds"];
        if (string.IsNullOrEmpty(agentId) || string.IsNullOrEmpty(blockedIds)) { return false; }
        return blockedIds.Split(',').Any(id => id.Trim() == agentId.Trim());
    }">
        <return-response>
            <set-status code="403" reason="Agent Blocked" />
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-header name="x-kill-switch-layer" exists-action="override">
                <value>3-agent-blocklist</value>
            </set-header>
            <set-body>{"error": {"code": "AgentBlocked", "message": "Agent has been added to the governance blocklist.", "layer": 3}}</set-body>
        </return-response>
    </when>
</choose>

Step 3.3 — Add the Agent to the Blocklist

az apim nv update `
  --resource-group rg-ai-hub-gateway-dev `
  --service-name apim-wpvlimv4ngkns `
  --named-value-id blocked-agent-ids `
  --value "citadel-weather-agent-v1"

Run the agent:

python agent_with_memory.py

Expected output:

Step 3.4 — Surgical Validation

The power of Layer 3 is specificity. If you had a second agent with a different x-agent-id say citadel-docs-agent-v1 it would pass through Layer 3 unaffected while citadel-weather-agent-v1 remains blocked. One agent stopped, all others running. This is the enterprise AI governance pattern: granular control without broad disruption.

Remove the agent from the blocklist:

az apim nv update --resource-group rg-ai-hub-gateway-dev
--service-name apim-wpvlimv4ngkns --named-value-id blocked-agent-ids
--value "none"

Validating the Citadel Kill Switch in Application Insights

Rather than using the CLI — which has a 5–10 minute Log Analytics ingestion lag — go directly to Application Insights in the portal for immediate results:

  1. Portal → appi-apim-wpvlimv4ngkns in rg-ai-hub-gateway-dev
  2. Left sidebar → Logs
  3. Paste and run this query:
requests
| where timestamp > ago(2h)
| where resultCode in ("200", "401", "403")
| project timestamp, resultCode, duration, name
| order by timestamp desc

The results table tells the complete kill switch story in two columns — resultCode and duration:

Azure Application Insights Logs results table showing POST /openai/deployments/chat/chat/completions requests — two 401 responses at 41ms and 0.9ms from Layer 2 kill switch activation, and four 200 responses at 683ms to 2010ms from normal governed agent runs through the Microsoft Foundry Citadel APIM hub in Sweden Central.
Application Insights Logs query on the Citadel APIM hub showing the kill switch in action, 401 responses at under 1ms confirm Layer 2 (agent approval header) blocking requests at the gateway before any LLM call is made, contrasted with normal 200 responses taking 683ms–2010ms for a full Azure OpenAI round trip.

The duration contrast is the definitive proof that the kill switch works as designed. The 401s and 403s resolve in under 50ms, stopped cold at the APIM inbound policy before a single token is sent to Azure OpenAI. The 200s take 683ms–2010ms because they made the full round trip through the governance hub to Azure OpenAI and back.

Zero tokens consumed on blocked requests, zero cost, and zero Cosmos DB writes in the spoke. The agent is stopped at the perimeter.

For a sharper view that highlights exactly which kill switch layer fired on each blocked request, add the response header to the query. Unfortunately APIM response headers are not automatically projected into the requests table in Application Insights — but you can distinguish the layers by combining result code and timing:

requests
| where timestamp > ago(2h)
| where resultCode in ("200", "401", "403")
| extend killSwitchLayer = case(
    resultCode == "401", "Layer 2 — agent approval",
    resultCode == "403" and duration < 10, "Layer 1 or 3 — gateway block",
    resultCode == "200", "Normal — LLM call completed",
    "Unknown"
  )
| project timestamp, resultCode, duration, killSwitchLayer
| order by timestamp desc
Azure Application Insights Logs results table for the Citadel APIM hub showing six requests — two 401 responses labeled Layer 2 agent approval at 41ms and 0.9ms duration, and four 200 responses labeled Normal LLM call completed at 683ms to 2010ms duration, confirming the Microsoft Foundry Citadel kill switch blocks requests at the gateway before any Azure OpenAI call is made.
Application Insights Logs query on the Citadel APIM hub showing the kill switch incident log — Layer 2 agent approval blocks resolving in under 1ms with zero LLM calls made, contrasted with normal governed runs completing in 683ms–2010ms. The killSwitchLayer column identifies exactly which containment layer fired on each request.

This gives you a readable incident log showing which containment layer was active at each point in time, directly useful for DORA incident post-mortem documentation and EU AI Act Article 17 risk management records.

The Complete Three-Layer Kill Switch Policy

Here is the complete inbound policy block containing all three layers, ready to paste into APIM:

 <!-- Kill Switch Layer 1: Named Value flip -->
        <set-variable name="killSwitchActive" value="@("{{kill-switch-enabled}}" == "true")" />
        <choose>
            <when condition="@((bool)context.Variables["killSwitchActive"])">
                <return-response>
                    <set-status code="403" reason="Agent Suspended" />
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/json</value>
                    </set-header>
                    <set-header name="x-kill-switch-layer" exists-action="override">
                        <value>1-named-value</value>
                    </set-header>
                    <set-body>{"error": {"code": "KillSwitchActive", "message": "Agent access has been suspended by the governance hub.", "layer": 1}}</set-body>
                </return-response>
            </when>
        </choose>
        <!-- Kill Switch Layer 2: Agent approval header -->
        <choose>
            <when condition="@(context.Request.Headers.GetValueOrDefault("x-agent-approved", "false") != "true")">
                <return-response>
                    <set-status code="401" reason="Agent Not Approved" />
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/json</value>
                    </set-header>
                    <set-header name="x-kill-switch-layer" exists-action="override">
                        <value>2-agent-approval</value>
                    </set-header>
                    <set-body>{"error": {"code": "AgentNotApproved", "message": "Agent identity could not be verified. Approval header missing or invalid.", "layer": 2}}</set-body>
                </return-response>
            </when>
        </choose>
        <!-- Kill Switch Layer 3: Agent ID blocklist -->
        <set-variable name="agentId" value="@(context.Request.Headers.GetValueOrDefault("x-agent-id", ""))" />
        <set-variable name="blockedIds" value="@("{{blocked-agent-ids}}")" />
        <choose>
            <when condition="@{
        var agentId = (string)context.Variables["agentId"];
        var blockedIds = (string)context.Variables["blockedIds"];
        if (string.IsNullOrEmpty(agentId) || string.IsNullOrEmpty(blockedIds)) { return false; }
        return blockedIds.Split(',').Any(id => id.Trim() == agentId.Trim());
    }">
                <return-response>
                    <set-status code="403" reason="Agent Blocked" />
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/json</value>
                    </set-header>
                    <set-header name="x-kill-switch-layer" exists-action="override">
                        <value>3-agent-blocklist</value>
                    </set-header>
                    <set-body>{"error": {"code": "AgentBlocked", "message": "Agent has been added to the governance blocklist.", "layer": 3}}</set-body>
                </return-response>
            </when>
        </choose>

Pitfalls Summary

PitfallFix
Named Value doesn’t exist at incident timePre-wire Layer 1 during normal operations — never during an incident
Policy evaluation error on {{kill-switch-enabled}}Named Value must exist before the policy referencing it is saved
Layer 2 fires before Layer 1 in policyPolicy order matters — Layer 1 must be first in the inbound block
Agent ID header not sentAdd x-agent-id to default_headers in AzureOpenAI client
Blocklist with trailing spaces blocks nothingUse .Trim() in the policy C# expression when splitting
Kill switch left active after testAlways reset Named Values after testing — kill-switch-enabled=false, blocked-agent-ids=""

What the Kill Switch Demonstrates About Citadel

The three layers reveal something important about the Citadel architecture: governance lives in the hub, not the agent. The agent code has no knowledge of the kill switch. The spoke has no kill switch configuration. The Azure OpenAI deployment is untouched. All containment logic is in the APIM hub’s inbound policy — one place, centrally managed, instantly effective.

This is the enterprise AI control plane pattern in practice. When an incident occurs:

  • Layer 1 stops everything immediately while you triage
  • Layer 2 enforces identity verification once normal operations resume
  • Layer 3 surgically targets the offending agent while other agents continue

The x-kill-switch-layer response header ensures your incident log captures exactly which containment mechanism fired, giving you a clean audit trail for post-mortem analysis — directly relevant for DORA incident reporting and EU AI Act Article 17 risk management documentation.

What’s Next

The next post in this series takes the dev setup and hardens it for non-prod: networkIsolation=true, APIM Premium SKU, per-spoke subscription keys with independent quotas, and Azure Policy at the management group level. The kill switch policies we built here carry forward unchanged governance in the hub environment, which is environment-agnostic.

Azure App Service Architecture: A Deeper Look for Integration Architects

Posted on by
Reply

In the Azure PaaS map post, App Service got just one paragraph. I called it the default for synchronous REST APIs that front a backend system. That summary is right, but Azure App Service architecture hides far more than one paragraph can carry.

When you stand up an App Service in a regulated enterprise, the interesting decisions sit around the app, not inside it. How does traffic reach it? And how does it authenticate outbound? Furthermore, how does it scale? And how do you ship changes without downtime? So this post takes a deeper look. I’ll walk the request path from the user to the data tier and flag the decisions that matter specifically for integration work.

Comprehensive diagram of Azure App Service architecture. Users reach the app through Azure DNS, Front Door with edge WAF, and Application Gateway with regional WAF. The App Service Plan runs multiple instances across availability zones with built-in features and application components. Surrounding tiers show CI/CD with deployment slots, security and identity, data services, networking, and monitoring.
The full picture request path across the top, the App Service Plan at the center, and the surrounding tiers of CI/CD, security, data, networking, and monitoring that make an integration platform work.

Azure App Service architecture: the request path, front to back

Before a request touches your code, it passes through a chain of services. Moreover, each one is a design decision rather than a default.

Linear diagram of the App Service request path: user to Azure DNS to Front Door with edge WAF to Application Gateway with regional WAF to App Service to the private data tier.
Every hop from user to data tier is a design decision, not a default, and for a single-region regulated platform, Application Gateway alone often does the job.
  • First, DNS resolves the URL. Azure DNS points the hostname at whatever sits in front of App Service. That step is trivial, but it’s worth naming, because the next hop depends on it.
  • Next, a global entry point handles routing and Web Application Firewall (WAF). Here the first real choice appears. Azure Front Door gives you global routing, CDN-style caching, and a Web Application Firewall at the edge. Therefore, you’d pick it when you serve a distributed audience or want TLS termination close to the user. Application Gateway, by contrast, performs Layer 7 load balancing and WAF regionally within your VNet. So you’d reach for it when traffic stays regional, and you want the firewall inside your network boundary. Plenty of designs use both Front Door globally, Application Gateway behind it. For a single-region platform in a regulated environment, though, Application Gateway alone often does the job. Better still, it keeps everything inside the VNet where your security team wants it.
  • Finally, App Service receives the request. By now, the traffic has been routed, load-balanced, and WAF-filtered. What App Service adds is a managed platform that runs your code, patches the underlying OS, and handles TLS for you.

Azure App Service architecture: Inside the App Service Plan

The App Service Plan catches people out. After all, this is where the platform allocates and bills are computed, not the app itself. Multiple apps can share one plan, so they also share its CPU and memory. That arrangement saves money until two apps contend under load. Then the “why is my API slow when the other app gets busy” investigation begins.

Diagram of an App Service Plan showing load balancing across instances spread over two availability zones, with a highlighted note that multiple apps sharing the same plan contend for CPU and memory under load. A legend explains scale-out, scale-up, and metric-based autoscale triggers.
The plan is the billed compute unit. It load-balances instances across availability zones, and because multiple apps can share one plan, they also contend for its CPU and memory under load.

The plan defines a few things that matter architecturally:

  • Instances and scaling: The plan runs one or more instances, and App Service load-balances across them. Scale-out adds instances, which drives throughput. Scale-up swaps in bigger instances, which adds per-request headroom. Autoscale rules fire on metrics like CPU, memory, and HTTP queue length. That’s exactly why App Service suits steadily loaded request workloads rather than bursty, event-driven ones. So if your load is spiky and event-driven, consider Functions or Container Apps instead.
  • Availability zones: On tiers that support it, you can spread plan instances across zones. As a result, “highly available” ceases to be a claim and becomes an actual design property. For regulated production, treat this as table stakes rather than an upgrade.
  • Isolation: The isolated tiers run your plan in a dedicated environment inside your VNet, away from shared infrastructure. Therefore, you’d reach for them when compliance demands network isolation that shared tiers can’t provide a common requirement in health and finance.
Current image: Azure App Service architecture diagram with users, networking, compute, storage, management, and DevOps components.

The platform features integration architects actually use

App Service ships built-in capabilities that often do more work than the application code. Three of them earn their keep in every integration design:

Sequence diagram of a deployment slot swap: deploy the new version to a staging slot, warm up and validate against production configuration, then swap staging and production instantly, with a dashed swap-back path shown for rolling back on failure.
Deploy and validate a new version in staging against production config, then swap it in instantly with an equally instant swap-back if something breaks.
  • Deployment slots are the single most useful feature for shipping without downtime. A staging slot lets you deploy, warm up, and validate a new version against production configuration. Then you swap it into production instantly, and swap back just as fast if something breaks. For a platform where a bad deploy takes down downstream consumers, that swap turns a potential incident into a controlled release.
  • Managed identity is the one I’d insist on. App Service can carry a system-assigned or user-assigned identity. Consequently, it authenticates to Key Vault, SQL, Service Bus, and Storage without a single connection string in the configuration. My first post made the same point about the governance layer. App Service is where you implement it for the compute tier.
  • VNet integration and private endpoints close the network. VNet integration lets the app call into your private network. Private endpoints let consumers reach the app privately, without a public address. In a regulated environment, you’ll usually want both. That way, the app communicates with backends over private links, and consumers access it through the gateway rather than a public URL.

The tiers around it: data, identity, observability

App Service never runs alone. In fact, the architecture around it is where an integration platform lives or dies:

  • Data services repeat the choices from the first post. Pick Azure SQL for relational integrity, Cosmos DB for flexible scaling, Blob Storage for files, and Redis Cache to absorb read load. App Service connects to all of them over private endpoints and authenticates through managed identity.
  • Security and identity means Entra ID for authentication, Key Vault for the secrets that can’t be an identity, and managed identities threading through everything. App Service’s built-in “Easy Auth” can offload the whole OIDC flow to the platform. That helps for internal APIs. Still, understand it before you lean on it for anything with complex authorization logic.
  • Monitoring and observability mean Application Insights and Azure Monitor. For an integration platform, this isn’t optional. When a request fails somewhere across the gateway, app, backend, and data tier, distributed tracing shows you where. So wire it in on day one, not after the first production incident.

Where App Service is the wrong answer

Let me keep this honest: the same point I make in every post. App Service isn’t always right, and reaching for it by reflex causes as many problems as it solves.

Does your workload run event-driven and bursty? Then App Service’s metric-based autoscale will lag the load or leave you overprovisioned. Functions or Container Apps fit better. Do you need long-running orchestration? App Service will host it, but you’re building a workflow engine on top of a request-serving platform. Logic Apps or Durable Functions exist for exactly that. Do you have real Kubernetes-native requirements? App Service won’t stretch that far, so that’s an AKS conversation.

App Service shines for steadily-loaded, request-driven APIs and backends. It gives you managed availability, easy TLS, slot-based deployment, and clean managed identity auth to the rest of the platform, for an integration platform that describes a large share of the synchronous surface. That’s exactly why it earns its place as the default compute tier as long as you know when to reach past it.

Azure App Service architecture: The shape of it

For an integration architect, Azure App Service architecture is mostly about what surrounds the app. Put a gateway with WAF in front, with private connectivity to backends and data. Use managed identity everywhere. Add slots for safe deployment. Trace the whole path. Get those right, and the app in the middle becomes almost boring, which, for a production platform, is the highest compliment there is.

Want the layer above this one? The Azure PaaS map puts App Service in context against Functions, Container Apps, and AKS. It also walks the five-question framework for choosing between them.

AI Foundry Spoke Model Deployment: Why It Still Happens

Posted on by
Reply

A comment on the first post in this series asked why AI Foundry Spoke model deployment happens at all in the Citadel pattern, a question worth answering properly in its own post rather than buried in a reply thread.

Great article as usual! Just wondering, in the given Governance Hub & Agent Spoke architecture, what is the purpose of deploying the models both to the Spoke and the Hub? Shouldn’t they only be deployed to the Hub and provided from there?

That’s a sharp question, and it points at a real tension in the Citadel pattern that the original post didn’t call out explicitly enough. Here’s the direct answer, followed by the reasoning behind it.

The short answer

No, they shouldn’t both serve your application’s inference needs. Only the Hub deployment should. The model deployment sitting in the Spoke exists today because of how Azure AI Foundry’s Agent Service currently works, not because the architecture intends a second, governance-free inference path.

That distinction matters, so it’s worth walking through why the Spoke deployment is there at all.

Why AI Foundry Spoke Model Deployment Happens at All

In the ideal version of the Citadel pattern, every model call flows through the Hub’s APIM gateway. That’s the entire point of centralizing governance in one place. It’s what the rest of the series demonstrated: every agent call routed through apim-wpvlimv4ngkns, with token tracking, content safety, cost attribution, and the kill switch all enforced at that single choke point.

The Spoke still ends up with its own local model deployment because the AI Foundry Agent Service needs one for two reasons. This is a direct consequence of how the AI Landing Zone Bicep templates provision the Spoke, not a choice made anywhere in this series.

It powers the Agent Service’s own internal capabilities. Thread management, agent orchestration, and built-in tools like Code Interpreter or File Search (when enabled) call the model directly, through Foundry’s own runtime, rather than through any external endpoint you control. That runtime doesn’t route through APIM. It talks to whatever model deployment sits alongside it in the same project.

It satisfies the Foundry project’s provisioning requirements. A Foundry project currently expects an associated model deployment to exist as part of setting up the project, even if your actual application traffic never calls that deployment directly.

Neither of these is a governance decision. They’re artifacts of how the Agent Service is architected right now.

Two paths, not one

The practical result is that a Citadel deployment ends up with two separate paths to a model, and they serve different purposes.

Diagram showing two paths to a model in the Citadel architecture. Application code routes through APIM in the Hub to the Hub's Azure OpenAI deployment, a governed path shown with a solid green arrow. Foundry's internal Agent Service runtime calls a local model deployment in the Spoke directly, bypassing APIM, shown with a dashed red arrow labeled as ungoverned.
Application traffic should only ever reach a model through APIM in the Hub. The Spoke’s local deployment exists for Foundry’s internal agent runtime, not for your code to call directly.

The Spoke’s local deployment exists for Foundry’s own internal agent runtime. It’s not meant to see your production traffic, and if it does, none of the governance you built in the Hub applies to those calls.

The Hub’s deployment, reached through APIM, is what your application code should use. That’s what we wired up explicitly in Part 2 of this series, with the standard OpenAI SDK pointed at the gateway rather than directly at the Foundry endpoint. The Hub itself is built on the AI Hub Gateway Solution Accelerator, and its AI gateway capabilities are exactly what give APIM the token metering, content safety, and audit trail features this series has leaned on throughout.

The second path exists precisely because of a limitation the series already documented. The Agent Service SDK, in its current preview state, doesn’t route its own LLM calls through APIM. It bypasses the gateway entirely, which means using it directly would mean giving up token metering, policy enforcement, and audit trails on every call the agent makes. That’s why Part 2 used the standard OpenAI SDK pointed at APIM instead of the native Agent Service SDK, and it’s the same underlying issue this reader’s question is really about.

What this means in practice

If you’re building on this pattern today, treat the Spoke’s model deployment as infrastructure the platform needs to exist, not as a second inference endpoint your application is allowed to call. Point your application code at the Hub, through APIM, every time. Leave the Spoke deployment alone to do the job Foundry needs it for internally, and don’t build anything that calls it directly for your own traffic.

If you’re reviewing someone else’s Citadel-pattern deployment, this is worth checking explicitly. A model deployment sitting in a Spoke isn’t wrong by itself, but it’s worth confirming nothing in the application is quietly calling it and skipping the gateway.

Where this is heading

I’d expect this to tighten up as the Agent Service SDK matures out of preview and gains native APIM routing support. When that happens, the two-path situation described here becomes a one-path situation, and the Spoke deployment stops being something you need to actively route around.

Side-by-side comparison diagram. Left panel, labeled Today, shows application code and the Agent Service each reaching separate model deployments, one through APIM in the Hub, one bypassing it in the Spoke. Right panel, labeled Future, shows both application code and the Agent Service routing through a single APIM gateway to one model deployment in the Hub.
Once the Agent Service SDK supports native APIM routing, both application and agent traffic converge on a single governed path.

Until then, the answer to the original question stands: deploy to the Hub, govern everything through APIM, and treat the Spoke’s model deployment as plumbing the platform needs rather than a second front door.

Thanks to the reader who asked the original question. It’s exactly the kind of detail that’s easy to leave implicit in an architecture diagram and much more useful said out loud.

Microsoft Foundry Citadel Platform Azure: Conversation Persistence with Cosmos DB

Posted on by
1

In the previous post, we connected a real tool-calling agent to the Microsoft Foundry Citadel Platform on Azure, routing every LLM call through the APIM governance hub in Sweden Central. The agent answered weather questions; the hub captured usage events in Cosmos DB; and Application Insights confirmed that both LLM calls were governed. The agent worked, but it had no memory. Every run started fresh, with no record of what was asked or answered.

This post adds conversation persistence to the Microsoft Foundry Citadel Platform on Azure. Every agent run now produces a structured document in the spoke’s Cosmos DB conversations container: the user’s question, the tool call made, the tool result, the agent’s answer, token counts, model version, and timestamp. The agent gains a memory layer, marking the transition as the spoke’s data tier becomes active.

What We Build

Each agent run writes one document to the spoke Cosmos DB:

{
  "id": "run-20260625-143022-stockholm",
  "principal_id": "steefjan@msn.com",
  "timestamp": "2026-06-25T14:30:22.441Z",
  "question": "What is the weather like in Stockholm right now?",
  "tool_calls": [
    {
      "name": "get_weather",
      "arguments": {"location": "Stockholm"},
      "result": {
        "location": "Stockholm, Sweden",
        "temperature_celsius": 22.5,
        "wind_speed_kmh": 7.2,
        "condition": "Overcast"
      }
    }
  ],
  "answer": "The weather in Stockholm is overcast with a temperature of 22.5°C...",
  "model": "gpt-4o-2024-11-20",
  "prompt_tokens": 234,
  "completion_tokens": 67,
  "total_tokens": 301,
  "apim_gateway": "apim-wpvlimv4ngkns.azure-api.net"
}

The partition key is /principal_id matching the container definition deployed by the spoke Bicep template. In addition, this arrangement ensures that all conversations for a given user are grouped into the same logical partition, making per-user history queries efficient.

The agent writes the document after completing the run, so a failed or incomplete run leaves no record.Moreover, it’s clean, simple, and auditable.

Prerequisites

From the previous two posts you should have:

  • Hub deployed in rg-ai-hub-gateway-dev
  • Spoke deployed in rg-ai-spoke-dev with Cosmos DB cosmos-tggi2gmkw22w4, database cosmos-dbtggi2gmkw22w4, container conversations
  • App Config appcs-tggi2gmkw22w4 populated with COSMOS_DB_ENDPOINT and CONVERSATIONS_DATABASE_CONTAINER
  • agent.py, config.py, and tools.py from the previous post
  • Virtual environment activated with openai, azure-appconfiguration, azure-identity, and requests installed

Step 1 — Install the Cosmos DB SDK

With your virtual environment activated:

pip install azure-cosmos

Pitfall: Cosmos DB Public Network Access

If your Cosmos DB has firewall rules enabled (which the spoke Bicep template sets by default), your local IP needs to be in the allowed list or public access needs to be set to All networks for dev. Check via the portal: cosmos-tggi2gmkw22w4 (your instance) → Networking → Public access → All networks → Save. In production this would be networkIsolation=true with private endpoints only.

Step 2 — Extend Config to Read Cosmos DB Settings

The spoke App Config already contains COSMOS_DB_ENDPOINT and CONVERSATIONS_DATABASE_CONTAINER — populated automatically during deployment. Extend config.py to pull these:

$lines = @(
    "from azure.appconfiguration import AzureAppConfigurationClient",
    "from azure.identity import DefaultAzureCredential",
    "",
    "APP_CONFIG_ENDPOINT = 'https://appcs-tggi2gmkw22w4.azconfig.io'",
    "LABEL = 'ai-lz'",
    "",
    "def get_config() -> dict:",
    "    credential = DefaultAzureCredential()",
    "    client = AzureAppConfigurationClient(",
    "        base_url=APP_CONFIG_ENDPOINT,",
    "        credential=credential",
    "    )",
    "    keys = [",
    "        'AI_FOUNDRY_PROJECT_ENDPOINT',",
    "        'CHAT_DEPLOYMENT_NAME',",
    "        'APIM_GATEWAY_URL',",
    "        'APIM_SUBSCRIPTION_KEY',",
    "        'COSMOS_DB_ENDPOINT',",
    "        'CONVERSATIONS_DATABASE_CONTAINER',",
    "        'DATABASE_NAME',",
    "    ]",
    "    config = {}",
    "    for key in keys:",
    "        setting = client.get_configuration_setting(key=key, label=LABEL)",
    "        config[key] = setting.value",
    "    return config",
    "",
    "if __name__ == '__main__':",
    "    cfg = get_config()",
    "    for k, v in cfg.items():",
    "        print(f'{k}: {v[:40]}...')"
)
[System.IO.File]::WriteAllLines("$PWD\config.py", $lines, [System.Text.UTF8Encoding]::new($false))

Test it:

python config.py

You should now see seven keys including COSMOS_DB_ENDPOINT pointing to https://cosmos-tggi2gmkw22w4.documents.azure.com:443/ and CONVERSATIONS_DATABASE_CONTAINER set to conversations.

Step 3 — Create cosmos.py

Create a dedicated Cosmos DB module:

$code = 'import os
cosmos = """from azure.cosmos import CosmosClient
from azure.identity import DefaultAzureCredential
import uuid
from datetime import datetime, timezone
def get_cosmos_client(endpoint):
    return CosmosClient(url=endpoint, credential=DefaultAzureCredential())
def save_conversation(endpoint, database_name, container_name, principal_id, question, tool_calls, answer, model, prompt_tokens, completion_tokens, total_tokens, apim_gateway):
    container = get_cosmos_client(endpoint).get_database_client(database_name).get_container_client(container_name)
    now = datetime.now(timezone.utc)
    fmt = \"%Y%m%d-%H%M%S\"
    doc_id = f\"run-{now.strftime(fmt)}-{str(uuid.uuid4())[:8]}\"
    document = {\"id\": doc_id, \"principal_id\": principal_id, \"timestamp\": now.isoformat(), \"question\": question, \"tool_calls\": tool_calls, \"answer\": answer, \"model\": model, \"prompt_tokens\": prompt_tokens, \"completion_tokens\": completion_tokens, \"total_tokens\": total_tokens, \"apim_gateway\": apim_gateway}
    container.create_item(body=document)
    print(f\"Saved conversation: {doc_id}\")
    return document
def get_conversation_history(endpoint, database_name, container_name, principal_id, limit=5):
    container = get_cosmos_client(endpoint).get_database_client(database_name).get_container_client(container_name)
    query = f\"SELECT TOP {limit} c.id, c.timestamp, c.question, c.answer, c.total_tokens FROM c WHERE c.principal_id = @principal_id ORDER BY c._ts DESC\"
    return list(container.query_items(query=query, parameters=[{\"name\": \"@principal_id\", \"value\": principal_id}], partition_key=principal_id))
"""
agent = """import json
from openai import AzureOpenAI
from config import get_config
from tools import get_weather, WEATHER_TOOL_DEFINITION
from cosmos import save_conversation, get_conversation_history
PRINCIPAL_ID = \"steefjan@msn.com\"
def run_agent_with_memory(user_question):
    cfg = get_config()
    apim_base = cfg[\"APIM_GATEWAY_URL\"].rstrip(\"/\").replace(\"/openai\", \"\")
    client = AzureOpenAI(azure_endpoint=apim_base, api_key=cfg[\"APIM_SUBSCRIPTION_KEY\"], api_version=\"2024-02-01\")
    messages = [{\"role\": \"user\", \"content\": user_question}]
    print(f\"Sending request via APIM: {apim_base}\")
    response = client.chat.completions.create(model=cfg[\"CHAT_DEPLOYMENT_NAME\"], messages=messages, tools=[WEATHER_TOOL_DEFINITION], tool_choice=\"auto\")
    msg = response.choices[0].message
    messages.append(msg)
    tool_calls_log = []
    answer = msg.content or \"\"
    total_prompt = response.usage.prompt_tokens
    total_completion = response.usage.completion_tokens
    if msg.tool_calls:
        for tool_call in msg.tool_calls:
            args = json.loads(tool_call.function.arguments)
            print(f\"  -> Tool call: get_weather({args})\")
            result_str = get_weather(**args)
            result_json = json.loads(result_str)
            print(f\"  -> Tool result: {result_str}\")
            tool_calls_log.append({\"name\": tool_call.function.name, \"arguments\": args, \"result\": result_json})
            messages.append({\"role\": \"tool\", \"tool_call_id\": tool_call.id, \"content\": result_str})
        response2 = client.chat.completions.create(model=cfg[\"CHAT_DEPLOYMENT_NAME\"], messages=messages)
        answer = response2.choices[0].message.content
        total_prompt += response2.usage.prompt_tokens
        total_completion += response2.usage.completion_tokens
    save_conversation(endpoint=cfg[\"COSMOS_DB_ENDPOINT\"], database_name=cfg[\"DATABASE_NAME\"], container_name=cfg[\"CONVERSATIONS_DATABASE_CONTAINER\"], principal_id=PRINCIPAL_ID, question=user_question, tool_calls=tool_calls_log, answer=answer, model=cfg[\"CHAT_DEPLOYMENT_NAME\"], prompt_tokens=total_prompt, completion_tokens=total_completion, total_tokens=total_prompt+total_completion, apim_gateway=apim_base.replace(\"https://\", \"\"))
    return answer
if __name__ == \"__main__\":
    cfg = get_config()
    print(\"=== Recent conversation history ===\")
    history = get_conversation_history(cfg[\"COSMOS_DB_ENDPOINT\"], cfg[\"DATABASE_NAME\"], cfg[\"CONVERSATIONS_DATABASE_CONTAINER\"], PRINCIPAL_ID, 3)
    if history:
        for h in history:
            print(f\"  [{h[\"timestamp\"]}] Q: {h[\"question\"][:60]}...\")
    else:
        print(\"  No previous conversations found.\")
    print()
    question = \"What is the weather like in Amsterdam right now?\"
    print(f\"Question: {question}\")
    answer = run_agent_with_memory(question)
    print(f\"Answer: {answer}\")
"""
with open("cosmos.py", "w", encoding="utf-8") as f:
    f.write(cosmos)
with open("agent_with_memory.py", "w", encoding="utf-8") as f:
    f.write(agent)
print("Done")
'
[System.IO.File]::WriteAllText("$PWD\write_files.py", $code, [System.Text.UTF8Encoding]::new($false))
python write_files.py

Pitfall: Managed Identity RBAC for Cosmos DB

The CosmosClient with DefaultAzureCredential uses your Azure CLI identity locally. That identity needs the Cosmos DB Built-in Data Contributor role on the Cosmos DB account — not a standard Azure RBAC role, but a Cosmos DB data plane role. The spoke deployment should have assigned this automatically via the assignCosmosDBCosmosDbBuiltInDataContributorExecutor deployment. If you get a 403, verify:

az cosmosdb sql role assignment list `
  --account-name cosmos-tggi2gmkw22w4 `
  --resource-group rg-ai-spoke-dev `
  --output table

Your principal ID (8e856fa1-f4c4-4a02-91a5-a6ccc6afc6b3) should appear with role definition ID ending in 00000000-0000-0000-0000-000000000002 (Built-in Data Contributor). If not, assign it:

az cosmosdb sql role assignment create `
  --account-name cosmos-tggi2gmkw22w4 `
  --resource-group rg-ai-spoke-dev `
  --role-definition-id /subscriptions/dc0f4d72-3734-4b03-8884-ccfb9c2c4cc7/resourceGroups/rg-ai-spoke-dev/providers/Microsoft.DocumentDB/databaseAccounts/cosmos-tggi2gmkw22w4/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002 `
  --principal-id 8e856fa1-f4c4-4a02-91a5-a6ccc6afc6b3 `
  --scope /subscriptions/dc0f4d72-3734-4b03-8884-ccfb9c2c4cc7/resourceGroups/rg-ai-spoke-dev/providers/Microsoft.DocumentDB/databaseAccounts/cosmos-tggi2gmkw22w4

Pitfall: No Connection Strings

Never use Cosmos DB connection strings or account keys in the agent code. The pattern here uses DefaultAzureCredential throughout — locally it picks up your az login identity, in production it uses the spoke’s Managed Identity. This is the NEN 7510 and cVGZ security baseline compliant approach.

Step 4 — Create agent_with_memory.py

$lines = @(
    "import json",
    "from openai import AzureOpenAI",
    "from config import get_config",
    "from tools import get_weather, WEATHER_TOOL_DEFINITION",
    "from cosmos import save_conversation, get_conversation_history",
    "",
    "PRINCIPAL_ID = 'steefjan@msn.com'",
    "",
    "def run_agent_with_memory(user_question: str) -> str:",
    "    cfg = get_config()",
    "    apim_base = cfg['APIM_GATEWAY_URL'].rstrip('/').replace('/openai', '')",
    "",
    "    client = AzureOpenAI(",
    "        azure_endpoint=apim_base,",
    "        api_key=cfg['APIM_SUBSCRIPTION_KEY'],",
    "        api_version='2024-02-01',",
    "    )",
    "",
    "    messages = [{'role': 'user', 'content': user_question}]",
    "    print(f'Sending request via APIM: {apim_base}')",
    "",
    "    # First LLM call - tool decision",
    "    response = client.chat.completions.create(",
    "        model=cfg['CHAT_DEPLOYMENT_NAME'],",
    "        messages=messages,",
    "        tools=[WEATHER_TOOL_DEFINITION],",
    "        tool_choice='auto',",
    "    )",
    "",
    "    msg = response.choices[0].message",
    "    messages.append(msg)",
    "    first_usage = response.usage",
    "",
    "    tool_calls_log = []",
    "    answer = msg.content or ''",
    "    total_prompt_tokens = first_usage.prompt_tokens",
    "    total_completion_tokens = first_usage.completion_tokens",
    "",
    "    # Handle tool calls",
    "    if msg.tool_calls:",
    "        for tool_call in msg.tool_calls:",
    "            args = json.loads(tool_call.function.arguments)",
    "            print(f'  -> Tool call: get_weather({args})')",
    "            result_str = get_weather(**args)",
    "            result_json = json.loads(result_str)",
    "            print(f'  -> Tool result: {result_str}')",
    "",
    "            tool_calls_log.append({",
    "                'name': tool_call.function.name,",
    "                'arguments': args,",
    "                'result': result_json,",
    "            })",
    "",
    "            messages.append({",
    "                'role': 'tool',",
    "                'tool_call_id': tool_call.id,",
    "                'content': result_str,",
    "            })",
    "",
    "        # Second LLM call - synthesis",
    "        response2 = client.chat.completions.create(",
    "            model=cfg['CHAT_DEPLOYMENT_NAME'],",
    "            messages=messages,",
    "        )",
    "        answer = response2.choices[0].message.content",
    "        total_prompt_tokens += response2.usage.prompt_tokens",
    "        total_completion_tokens += response2.usage.completion_tokens",
    "",
    "    total_tokens = total_prompt_tokens + total_completion_tokens",
    "",
    "    # Save to Cosmos DB conversations container in the spoke",
    "    save_conversation(",
    "        endpoint=cfg['COSMOS_DB_ENDPOINT'],",
    "        database_name=cfg['DATABASE_NAME'],",
    "        container_name=cfg['CONVERSATIONS_DATABASE_CONTAINER'],",
    "        principal_id=PRINCIPAL_ID,",
    "        question=user_question,",
    "        tool_calls=tool_calls_log,",
    "        answer=answer,",
    "        model=cfg['CHAT_DEPLOYMENT_NAME'],",
    "        prompt_tokens=total_prompt_tokens,",
    "        completion_tokens=total_completion_tokens,",
    "        total_tokens=total_tokens,",
    "        apim_gateway=apim_base.replace('https://', ''),",
    "    )",
    "",
    "    return answer",
    "",
    "if __name__ == '__main__':",
    "    # Show last 3 conversations before running",
    "    from config import get_config",
    "    cfg = get_config()",
    "    print('=== Recent conversation history ===')",
    "    history = get_conversation_history(",
    "        endpoint=cfg['COSMOS_DB_ENDPOINT'],",
    "        database_name=cfg['DATABASE_NAME'],",
    "        container_name=cfg['CONVERSATIONS_DATABASE_CONTAINER'],",
    "        principal_id=PRINCIPAL_ID,",
    "        limit=3,",
    "    )",
    "    if history:",
    "        for h in history:",
    "            print(f'  [{h[chr(116)+chr(105)+chr(109)+chr(101)+chr(115)+chr(116)+chr(97)+chr(109)+chr(112)]}] Q: {h[chr(113)+chr(117)+chr(101)+chr(115)+chr(116)+chr(105)+chr(111)+chr(110))[:60]}...')",
    "    else:",
    "        print('  No previous conversations found.')",
    "    print()",
    "",
    "    question = 'What is the weather like in Amsterdam right now?'",
    "    print(f'Question: {question}')",
    "    answer = run_agent_with_memory(question)",
    "    print(f'Answer: {answer}')"
)
[System.IO.File]::WriteAllLines("$PWD\agent_with_memory.py", $lines, [System.Text.UTF8Encoding]::new($false))

Run it:

python agent_with_memory.py

A successful run looks like this:

Run it a second time and the history section will show the previous exchange:

Step 5 — Validate in Cosmos DB Data Explorer

Go to the Azure Portal → cosmos-tggi2gmkw22w4 → Data Explorer → cosmos-dbtggi2gmkw22w4 → conversations → Items.

You should see your conversation document with all fields populated. The partition key /principal_id should match steefjan@msn.com.

To query all conversations for a user:

SELECT * FROM c WHERE c.principal_id = 'steefjan@msn.com' ORDER BY c._ts DESC

To get a summary of all runs with token totals:

SELECT c.id, c.timestamp, c.question, c.total_tokens, c.model
FROM c
WHERE c.principal_id = 'steefjan@msn.com'
ORDER BY c._ts DESC

Step 6 — What’s in the Document

Looking at a stored conversation document, every field serves a purpose:

FieldPurpose
idUnique run identifier — traceable back to a specific agent invocation
principal_idPartition key — enables per-user history queries and RBAC scoping
timestampISO 8601 UTC — audit trail, correlatable with APIM logs
questionOriginal user input — searchable for pattern analysis
tool_callsFull tool call log including arguments and results — debugging and audit
answerFinal agent response — quality review and feedback loops
modelModel version — tracks which model version answered which questions
prompt_tokens / completion_tokensCumulative across both LLM calls — accurate per-conversation cost
total_tokensSum of both calls — FinOps input per user per conversation
apim_gatewayGateway used — identifies which hub instance served the request

The token counts here are cumulative across both LLM calls (tool decision and synthesis), yielding a true per-conversation costrather than a per-call figure. This is more useful for FinOps reporting you care about the cost of answering a question, not the cost of individual API calls within that answer.

Pitfalls Summary

PitfallFix
Cosmos DB firewall blocks local IPPortal → Networking → All networks for dev, or add specific IP
403 on Cosmos DB writeAssign Cosmos DB Built-in Data Contributor data plane role to your principal
CosmosResourceNotFoundErrorVerify database name (cosmos-dbtggi2gmkw22w4) and container name (conversations) match exactly
Partition key mismatchContainer was created with /principal_id — every document must include this field
DefaultAzureCredential fails locallyRun az login and ensure the correct subscription is selected
Never use connection stringsUse DefaultAzureCredential throughout — locally via az login, in production via Managed Identity

What the Full Citadel Data Layer Now Looks Like

After this post, the spoke’s data tier is fully active:

StoreWhat it holdsWho writes it
Hub Cosmos DB ai-usage-containerPer-LLM-call usage events (tokens, model, gateway, IP)APIM gateway automatically
Spoke Cosmos DB conversationsPer-run conversation documents (question, tools, answer, cumulative tokens)Agent code explicitly
App Config appcs-tggi2gmkw22w4All configuration keys for the spokeSpoke deployment automatically

The hub’s ai-usage-container captures the infrastructure view of every API call, governed and logged. The spoke’s conversations container captures the application view of every user interaction, structured and queryable. Together, they give you both compliance evidence and application telemetry from a single agent run.

What’s Next

The next post in this series showcases the Citadel Kill Switch and explains how it stops a governed agent when necessary. It details how the five-layer containment system in APIM effectively shuts down the process without affecting the spoke or agent code. The conversation history you’ve created illustrates the clear before-and-after contrast: requests flow to Cosmos DB and then abruptly halt at the gateway layer.

Azure PaaS Integration for Architects: A Practitioner’s Map

Posted on by
2

If you’ve spent any time in the Azure portal lately, you’ll know the problem isn’t a lack of PaaS services; it’s too many of them, with overlapping capabilities and just enough marketing gloss to make every option look like the right one. App Service or Container Apps? Logic Apps or Functions? Service Bus or Event Grid? The Azure PaaS catalog has grown quickly, and for integration architects specifically, the decisions compound: pick the wrong option at the compute layer, and you’re fighting the platform every time you add a connector, a retry policy, or a compliance control.

This post is a map, not a comparison matrix. I’m grouping the Azure PaaS services that matter to integration work into four layers: compute, integration, data, and governance, and walking through the decision points that arise when designing for a regulated enterprise environment rather than a greenfield demo. If you’ve followed my Logic Apps Agent Loop series or the APIM for AI workloads series, this sits underneath both the platform primer and the deeper posts, which assume you already have read.

Why Azure PaaS still matters to integration architects

IaaS provides you with a VM and asks you to manage everything above it. SaaS gives you the finished product and asks for nothing. PaaS sits in between: the platform owns patching, scaling, and availability, and you own the application logic and configuration. For integration workloads specifically, that trade-off is usually the right one: you rarely need to control the OS of a message broker, but you do need fine-grained control over routing, transformation, and policy enforcement.

The practical test I use: if a service requires you to think about instance sizing, OS patch cycles, or cluster upgrades, it’s leaning IaaS regardless of what the marketing page calls it. If it requires you to think about triggers, bindings, connectors, and scaling rules, it’s PaaS. AKS sits deliberately on that boundary; more on that below.

Diagram showing five stacked Azure PaaS layers for integration architects: Compute (App Service, Functions, Container Apps, AKS), Integration (Logic Apps, API Management, Service Bus, Event Grid), Data (Azure SQL Database, Cosmos DB, Cache for Redis), Governance and identity (Entra ID, Key Vault, Azure Policy, Monitor), and Governance and resilience — the agentic gap (per-action authorization, compensating actions, evaluation and drift detection).
The four core PaaS layers for integration work compute, integration, data, and governance/identity plus the fifth layer, agentic workloads, expose: per-action authorization, compensating actions, and evaluation.

Layer 1: Compute in Azure PaaS Integration

Azure App Service

Still, the default is web APIs and backend services that don’t need event-driven scaling. App Service gives you deployment slots, built-in autoscale, and managed TLS with minimal ceremony. For integration architects, the main use case is hosting synchronous REST APIs that front a backend system the kind of thing that used to be a WCF service or an on-prem IIS site.

The limitation that catches people out: App Service scales on CPU/memory/queue-length rules, not on arbitrary event volume. If your workload is bursty and event-driven rather than steadily loaded, you’ll either overprovision or look elsewhere.

Azure Functions

The event-driven counterpart. Functions are the right choice when the unit of work is a discrete event: a message landing in a queue, a file arriving in Blob Storage, or an HTTP call that needs to fan out. The Consumption plan gives true scale-to-zero, which matters for cost in low-traffic integration scenarios; the Premium and Flex Consumption plans trade some of that elasticity for warm instances and VNet integration, which most enterprise integration platforms need anyway because you’re rarely allowed to expose a public endpoint without a private link in front of it.

Where Functions get uncomfortable: long-running orchestrations. A single function execution has a timeout, and while Durable Functions solves the orchestration problem, you’re now managing a stateful workflow engine on top of a stateless compute primitive. That’s usually the point where I ask whether Logic Apps would do the job with less code.

Azure Container Apps

The newer entrant is increasingly my default recommendation for anything that needs to run a container without the operational overhead of Kubernetes. Container Apps gives you KEDA-based event-driven scaling, Dapr integration for service-to-service calls and pub/sub, and revision-based traffic splitting all without you touching a node pool. For integration architects building agent-based or microservice-style integration components, this is often the sweet spot: you get container portability (useful if the workload might move, or if you’re standardizing on containers for other reasons) without inheriting cluster lifecycle management.

Azure Kubernetes Service (AKS)

Worth naming even though it’s not strictly Azure PaaS: Microsoft manages the control plane, yet you still own node pool upgrades, networking configuration, and workload scheduling. AKS earns its place when you have genuine Kubernetes-native requirements: custom operators, a multi-team platform where Kubernetes is the common substrate, or workloads that need capabilities Container Apps doesn’t expose yet. For most integration teams, reaching for AKS by default is over-engineering. Reach for it when a specific requirement forces your hand, not because it’s the more “serious” option.

Layer 2: Integration with Azure PaaS

Azure Logic Apps

The workflow orchestration layer with Azure PaaS remains the most direct route to enterprise connectors: SAP, IBM MQ, mainframe hosts, and the long tail of line-of-business systems that lack a modern REST API. Standard Logic Apps (running on the single-tenant model) close most of the gaps that made Consumption Logic Apps hard to use in regulated environments: VNet integration, built-in state management, and per-workflow scaling.

The honest trade-off: Logic Apps designer-first workflows are fast to build and easy for less code-heavy teams to maintain, but they get harder to reason about and harder to code-review once a workflow grows past a certain complexity. I’ve found the practical ceiling is somewhere around “a dozen actions with a couple of branches.” Past that, either decompose into smaller workflows or move the logic into a Function.

Azure API Management

Not just a gateway for integration architects, APIM is where governance actually gets enforced. Rate limiting, authentication, request/response transformation, and policy-based routing all live here, in front of whatever compute layer is doing the real work. If you’re building any platform where multiple consumers hit a shared set of backend capabilities, APIM is the control point that lets you change backend implementations without breaking consumers and enforce policy without touching application code.

The thing worth planning for early: policy authoring in APIM is a distinct skill, separate from the languages your team already knows. Please budget time for the team to learn the policy XML dialect rather than treating it as an afterthought. Badly written policies are a common source of latency and hard-to-diagnose failures.

Azure Service Bus

The durable, ordered, transactional messaging backbone. Reach for Service Bus when you need guaranteed delivery, sessions for ordered processing, or transactional message handling across multiple operations. Topics and subscriptions give you pub/sub without standing up a separate broker.

Azure Event Grid

The lightweight, high-throughput event router. Where Service Bus is about reliable delivery of business messages, Event Grid is about routing high-volume, fire-and-forget events resource state changes, custom application events, IoT telemetry to whichever subscriber cares about them. The two are frequently used together: Event Grid fans out a notification, and a subscriber puts a durable message on Service Bus for guaranteed processing.

A rule of thumb I use with teams new to Azure integration: if losing a message would be a business incident, it belongs on Service Bus. If losing a message would just mean a missed notification, Event Grid is fine.

Layer 3: Data in Azure PaaS

Integration architecture lives and dies by what’s underneath it, and the PaaS data services matter as much as the compute and messaging layers.

Azure SQL Database remains the default for relational, transactional workloads with a need for strong consistency think reference data, transactional state, anything with real foreign-key relationships. In addition, Azure Cosmos DB earns its place when you need global distribution, flexible schema, or the kind of horizontal scale that a single SQL instance won’t give you cheaply; it’s also increasingly the default choice for conversation and state storage in agentic workloads, given its low-latency reads and flexible document model. Finally, Azure Cache for Redis sits in front of both, absorbing read load and giving you a fast, ephemeral store for session state or short-lived coordination data.

The mistake I see most often: teams default to Cosmos DB because it’s the “modern” choice, then discover they actually needed relational integrity and end up hand-rolling consistency checks that SQL would have given them for free. Pick based on the access pattern, not the reputation.

Layer 4: Governance and identity for Azure PaaS

This is the layer that separates a proof of concept from something you can run in a regulated industry. Microsoft Entra ID and managed identities remove the need for connection strings and API keys scattered across configuration files. Each of the Azure PaaS services above should authenticate via managed identity. Key Vault holds what can’t be a managed identity (third-party API keys, certificates). Azure Policy and Microsoft Defender for Cloud provide the guardrails and posture visibility that an auditor or security team will ask for. Azure Monitor and Application Insights are non-negotiable for integration platforms, especially when a message fails somewhere in a chain of five services. Distributed tracing is the difference between a five-minute diagnosis and a day of log archaeology.

Layer 5: The gap that agentic workloads expose

Everything above holds for conventional integration platforms. Agentic AI workloads add a wrinkle that a recent round of discussion on LinkedIn I saw around an enterprise agent architecture diagram put well: the model is arguably the least differentiated part of a production agent deployment. Identity, permissions, observability, governance, and reliable orchestration are what separate a working demo from something you can run against real systems, and a few of those deserve to be called out specifically for integration architects, because they don’t map cleanly onto the governance layer above.

Diagram showing a caller validated at a green dashed identity perimeter (Entra ID / RBAC) before entering an agent's reasoning loop of plan, call tool, observe result, and act. A coral dashed arrow shows a poisoned result from an untrusted tool or RAG source entering the loop directly, bypassing the perimeter. A per-action authorization control sits inside the loop, asking whether each specific call is allowed for the tenant.
The identity perimeter validates the caller once, at the edge. A poisoned tool result enters the reasoning loop from the data the agent requested and never crosses that boundary. Per-action authorization is the control that reaches inside the loop where the threat actually lives.

Idenitity

Identity secures who the agent is, not what it does. Entra ID and managed identity answer “Is this caller who it claims to be?” They don’t address what happens when a poisoned tool result or a manipulated retrieved document changes the agent’s next action mid-reasoning loop. Prompt injection rides in through the RAG layer and tool outputs inside the loop, where identity checks at the perimeter don’t reach. The practical implication for PaaS design is that authorization needs to occur per action, not just per identity. This is exactly what APIM policy scoping and per-tool consent in Logic Apps and AI Foundry connectors are for: to treat each tool call as its own authorization decision, rather than an inherited privilege from a validated caller.

Recovery

Recovery means compensating actions, not just retries. Agent actions have side effects across systems: a ticket got created, a record got updated, an email went out. A failed step three actions into an agent loop can’t just retry from the top; it needs a saga-style compensating action to undo what already happened. Service Bus sessions and Logic Apps’ native support for scoped try/catch-with-compensation are the building blocks here — but the compensation logic has to be designed in explicitly, because neither service provides it by default.

Evaluation

Evaluation and drift detection are a first-class layer, not an afterthought. Application Insights and Azure Monitor provide operational observability into latency, error rates, and throughput. They don’t tell you if the agent’s outputs are quietly degrading in quality over time. That’s a separate concern, and one worth budgeting for from the start rather than bolting on after the first bad production incident.

The questions worth asking before an agent goes anywhere near a real system: what did it access, what tool did it call, why did it act, what policy constrained it, what happened when it failed, and who owns the outcome. If a PaaS architecture can’t answer all six, the gap isn’t in the model; it’s in the platform around it.

Azure PaaS Integration decisions: a framework, not a decision tree for integration architects

None of these layers are picked in isolation; the compute choice constrains the integration pattern, and the integration pattern constrains what the data layer needs to support. When I’m working through this with a team, the questions I ask in order are:

  1. Is the trigger an event or a schedule/request? Event-driven points toward Functions or Container Apps with KEDA; request-driven points toward App Service or APIM-fronted compute.
  2. Does a human or a low-code team need to maintain this workflow? If yes, Logic Apps earns serious consideration even if a Function would be more “elegant.”
  3. What’s the cost of losing a message? Business-critical → Service Bus. Best-effort notification → Event Grid.
  4. Does the data need strong relational integrity, or flexible scale? SQL for the former, Cosmos DB for the latter — and don’t let the “modern” label make the decision for you.
  5. Is everything behind managed identity and traced end to end? If the answer is no anywhere in the chain, that’s the next thing to fix, not the last.
Decision flowchart for choosing Azure PaaS services: question one splits event-driven compute (Functions, Container Apps) from request-driven compute (App Service, APIM-fronted); question two routes low-code-maintained workflows to Logic Apps; question three splits business-critical messaging (Service Bus) from best-effort events (Event Grid); question four chooses Azure SQL for relational integrity or Cosmos DB for scale; question five is a gate requiring managed identity and end-to-end tracing before deployment.
The five questions as a branching flow trigger type, workflow ownership, message-loss cost, data access pattern, and the managed-identity gate that comes before anything ships.

Azure PaaS Integration Conclusion

That’s the shape of it. In practice, most enterprise integration platforms end up using several of these services together: API Management fronting a mix of Logic Apps and Functions, backed by Service Bus for reliable delivery and Cosmos DB or SQL for state and the architecture work is less about picking a single winner than about drawing clean boundaries between them.

Microsoft Foundry Citadel Platform Azure: Connecting a Tool-Calling Agent

Posted on by
Reply

In the previous post we deployed a working Microsoft Foundry Citadel Platform on Azure Sweden Central, a Governance Hub built on Azure API Management and an Agent Spoke built on Azure AI Foundry. We validated the setup with a raw chat completion call through the APIM gateway. That proved the plumbing works. This post takes the next step: connecting a real tool-calling agent to the Microsoft Foundry Citadel Platform on Azure, using the Open-Meteo weather API as a tool, and showing that every LLM call flows through the hub’s governance layer.

The agent is built with the standard Azure OpenAI SDK pointed directly at the Citadel APIM gateway. It uses a custom function tool that calls the Open-Meteo API to retrieve real current weather data for any location. The governance hub intercepts all traffic: content safety policies fire, token usage is tracked, and telemetry flows into Application Insights. This is the Microsoft Foundry Citadel Platform doing what it is designed to do.

What We Build

The flow looks like this:

Flow diagram showing a Python agent using the OpenAI SDK making an initial request to the Citadel APIM gateway, which forwards to Azure OpenAI gpt-4o. The model requests a tool call to get_weather, the agent calls the Open-Meteo API for real weather data, submits the result back through APIM for a second LLM call, receives a grounded response, and telemetry flows to Application Insights and Cosmos DB.
End-to-end flow of a tool-calling agent on the Microsoft Foundry Citadel Platform in Azure Sweden Central, the Python agent routes both LLM calls through the APIM Governance Hub, executes the get_weather tool against Open-Meteo, and receives a grounded response, with all traffic captured in Application Insights and Cosmos DB.

Two LLM calls flow through APIM per agent run: the tool decision call and the synthesis call. Both are governed, appear in Application Insights, and contribute to Cosmos DB usage tracking.

Why Open-Meteo and Why the Standard OpenAI SDK

The original plan was to use the Azure AI Foundry Agent Service SDK with Bing Search grounding. Two blockers emerged:

Bing Search SKU eligibility: The Grounding with Bing Search resource (G1 SKU) requires Pay-As-You-Go or EA subscriptions and is not available on MVP or MSDN subscriptions.

AI Foundry Agent Service routing: The azure-ai-projects SDK routes LLM calls through the AI Foundry project’s internal endpoint (aif-tggi2gmkw22w4.openai.azure.com) rather than through APIM, bypassing the governance layer. In addition, even after adding APIM as a connected resource in the AI Foundry portal, the Agent Service does not honor it for model routing in the current preview version.

The solution, therefore, is to use the standard OpenAI Python SDK pointed directly at the APIM gateway endpoint. This guarantees that all traffic flows through the hub; consequently, the tool-calling loop is implemented explicitly in Python, and the governance telemetry is fully captured in Application Insights.

Open-Meteo is a free, open-source weather API; therefore, it requires no API key and returns structured JSON weather data. Additionally, it serves as a clean stand-in for any external API your agents might call in production.

Prerequisites

From the previous post you should have:

  • Hub deployed in rg-ai-hub-gateway-dev with APIM gateway URL https://apim-wpvlimv4ngkns.azure-api.net and subscription key
  • Spoke deployed in rg-ai-spoke-dev with App Config appcs-tggi2gmkw22w4 containing APIM_GATEWAY_URL and APIM_SUBSCRIPTION_KEY
  • Your principal ID with App Configuration Data Reader role on the spoke App Config

For this post you additionally need Python 3.11 or later installed locally.

Step 1 — Set Up the Python Environment

mkdir citadel-agent && cd citadel-agent
python -m venv .venv
# Windows
.venv\Scripts\activate
pip install openai
pip install azure-appconfiguration
pip install azure-identity
pip install requests

Step 2 — Read Configuration from App Config

Create config.py using Set-Content to avoid BOM issues on Windows:

$lines = @(
    "from azure.appconfiguration import AzureAppConfigurationClient",
    "from azure.identity import DefaultAzureCredential",
    "",
    "APP_CONFIG_ENDPOINT = 'https://appcs-tggi2gmkw22w4.azconfig.io'",
    "LABEL = 'ai-lz'",
    "",
    "def get_config() -> dict:",
    "    credential = DefaultAzureCredential()",
    "    client = AzureAppConfigurationClient(",
    "        base_url=APP_CONFIG_ENDPOINT,",
    "        credential=credential",
    "    )",
    "    keys = [",
    "        'AI_FOUNDRY_PROJECT_ENDPOINT',",
    "        'CHAT_DEPLOYMENT_NAME',",
    "        'APIM_GATEWAY_URL',",
    "        'APIM_SUBSCRIPTION_KEY',",
    "    ]",
    "    config = {}",
    "    for key in keys:",
    "        setting = client.get_configuration_setting(key=key, label=LABEL)",
    "        config[key] = setting.value",
    "    return config",
    "",
    "if __name__ == '__main__':",
    "    cfg = get_config()",
    "    for k, v in cfg.items():",
    "        print(f'{k}: {v[:30]}...')"
)
[System.IO.File]::WriteAllLines("$PWD\config.py", $lines, [System.Text.UTF8Encoding]::new($false))

Test it:

python config.py

All four keys should return truncated values. If you get a 403, wait 2–5 minutes for role assignment propagation and retry.

Pitfall: Always Use WriteAllLines for Python Files on Windows

Out-File -Encoding utf8NoBOM and @"..."@ | Out-File both add a BOM on some Windows PowerShell versions, causing Python to throw SyntaxError: Non-UTF-8 code starting with '\xff'. Use [System.IO.File]::WriteAllLines with [System.Text.UTF8Encoding]::new($false) to write files without BOM.

Step 3 — Define the Weather Tool

Create tools.py:

$lines = @(
    "import json",
    "import requests",
    "",
    "def get_weather(location: str) -> str:",
    "    try:",
    "        geo = requests.get(",
    "            'https://geocoding-api.open-meteo.com/v1/search',",
    "            params={'name': location, 'count': 1, 'language': 'en', 'format': 'json'},",
    "            timeout=10",
    "        )",
    "        geo.raise_for_status()",
    "        geo_data = geo.json()",
    "        if not geo_data.get('results'):",
    "            return json.dumps({'error': f'Location not found: {location}'})",
    "        r = geo_data['results'][0]",
    "        weather = requests.get(",
    "            'https://api.open-meteo.com/v1/forecast',",
    "            params={'latitude': r['latitude'], 'longitude': r['longitude'], 'current_weather': True, 'wind_speed_unit': 'kmh', 'timezone': 'auto'},",
    "            timeout=10",
    "        )",
    "        weather.raise_for_status()",
    "        c = weather.json()['current_weather']",
    "        codes = {0:'Clear sky',1:'Mainly clear',2:'Partly cloudy',3:'Overcast',45:'Foggy',61:'Slight rain',63:'Moderate rain',65:'Heavy rain',71:'Slight snow',80:'Showers',95:'Thunderstorm'}",
    "        return json.dumps({'location': f'{r[chr(110)+(chr(97)+chr(109)+chr(101))]}, {r.get(chr(99)+chr(111)+chr(117)+chr(110)+chr(116)+chr(114)+chr(121),chr(32))}', 'temperature_celsius': c['temperature'], 'wind_speed_kmh': c['windspeed'], 'wind_direction_degrees': c['winddirection'], 'condition': codes.get(c['weathercode'],'Unknown'), 'is_day': bool(c['is_day'])})",
    "    except Exception as e:",
    "        return json.dumps({'error': str(e)})",
    "",
    "WEATHER_TOOL_DEFINITION = {",
    "    'type': 'function',",
    "    'function': {",
    "        'name': 'get_weather',",
    "        'description': 'Get current weather for a location. Returns temperature in Celsius, wind speed, condition.',",
    "        'parameters': {",
    "            'type': 'object',",
    "            'properties': {'location': {'type': 'string', 'description': 'City name e.g. Stockholm'}},",
    "            'required': ['location']",
    "        }",
    "    }",
    "}"
)
[System.IO.File]::WriteAllLines("$PWD\tools.py", $lines, [System.Text.UTF8Encoding]::new($false))

Test it:

python -c "from tools import get_weather; print(get_weather('Stockholm'))"

Step 4 — Create the Agent

Create agent.py using the standard openai SDK pointed directly at the APIM gateway:

$lines = @(
    "import json",
    "from openai import AzureOpenAI",
    "from config import get_config",
    "from tools import get_weather, WEATHER_TOOL_DEFINITION",
    "",
    "def run_agent(user_question: str) -> str:",
    "    cfg = get_config()",
    "",
    "    # Strip /openai suffix - AzureOpenAI SDK adds it automatically",
    "    apim_base = cfg['APIM_GATEWAY_URL'].rstrip('/').replace('/openai', '')",
    "",
    "    client = AzureOpenAI(",
    "        azure_endpoint=apim_base,",
    "        api_key=cfg['APIM_SUBSCRIPTION_KEY'],",
    "        api_version='2024-02-01',",
    "    )",
    "",
    "    messages = [{'role': 'user', 'content': user_question}]",
    "    print(f'Sending request via APIM: {apim_base}')",
    "",
    "    # First LLM call - agent decides whether to use the tool",
    "    response = client.chat.completions.create(",
    "        model=cfg['CHAT_DEPLOYMENT_NAME'],",
    "        messages=messages,",
    "        tools=[WEATHER_TOOL_DEFINITION],",
    "        tool_choice='auto',",
    "    )",
    "",
    "    msg = response.choices[0].message",
    "    messages.append(msg)",
    "",
    "    # Handle tool calls if the agent decided to use get_weather",
    "    if msg.tool_calls:",
    "        for tool_call in msg.tool_calls:",
    "            args = json.loads(tool_call.function.arguments)",
    "            print(f'  -> Tool call: get_weather({args})')",
    "            result = get_weather(**args)",
    "            print(f'  -> Tool result: {result}')",
    "            messages.append({",
    "                'role': 'tool',",
    "                'tool_call_id': tool_call.id,",
    "                'content': result,",
    "            })",
    "",
    "        # Second LLM call - synthesise grounded response",
    "        response = client.chat.completions.create(",
    "            model=cfg['CHAT_DEPLOYMENT_NAME'],",
    "            messages=messages,",
    "        )",
    "        return response.choices[0].message.content",
    "",
    "    return msg.content",
    "",
    "if __name__ == '__main__':",
    "    question = 'What is the weather like in Stockholm right now?'",
    "    print(f'Question: {question}')",
    "    answer = run_agent(question)",
    "    print(f'Answer: {answer}')"
)
[System.IO.File]::WriteAllLines("$PWD\agent.py", $lines, [System.Text.UTF8Encoding]::new($false))

Run it:

python agent.py

A successful run looks like this:

Pitfall: APIM Endpoint Format

The AzureOpenAI SDK constructs the full path as {azure_endpoint}/openai/deployments/{model}/chat/completions. If your APIM_GATEWAY_URL in App Config contains /openai at the end, strip it before passing to the client; otherwise, the SDK builds a doubled path (/openai/openai/...) that returns a 500 from APIM. The line apim_base = cfg['APIM_GATEWAY_URL'].rstrip('/').replace('/openai', '') handles this automatically.

After running the agent, check Application Insights in the hub:

az monitor app-insights query `
  --app <Your APIM instance Name> `
  --resource-group rg-ai-hub-gateway-dev `
  --analytics-query "requests | where timestamp > ago(10m) | project timestamp, name, resultCode, duration | order by timestamp desc" `
  --output table

Pitfall: CLI vs Portal Ingestion Lag

The CLI query hits the Log Analytics store; however, it has a 5–10-minute ingestion lag. In contrast, the Azure Portal Application Insights blade uses a live metrics path and shows results immediately. Therefore, if the CLI returns an empty response, it’s a good idea to check the portal directly, go to the APIM instance → Performance to view requests in real time.

What Governed Traffic Looks Like in the Portal

The Application Insights Performance blade shows two operation types per agent run:

  • azure-openai-service-api:rev=1 - ChatCompletions_Create — the APIM policy-matched operation, showing the governed calls with content safety applied
  • POST /openai/openai/deployments/chat/chat/completions — the raw endpoint calls

Each agent run generates two successful requests (tool decision + synthesis), both with response code 200 and latency around 900ms–1.2s for gpt-4o. Failed attempts from earlier endpoint format issues show as 500s and are clearly distinguishable.

Azure Application Insights Performance blade showing 9 requests to the Citadel APIM gateway including the azure-openai-service-api ChatCompletions_Create operation with 6 calls at 1.09 seconds average and POST /openai/deployments/chat/chat/completions with 3 calls, confirming the tool-calling agent traffic flows through the Microsoft Foundry Citadel governance hub in Sweden Central.
Application Insights Performance blade for the Citadel Governance Hub, confirming agent traffic routed through APIM: 9 requests captured, with the governed ChatCompletions_Create operation averaging 1.09 seconds, and all successful calls returning response code 200.

The Azure AI Foundry Agent Service SDK — What We Learned

For completeness, here is a summary of what we discovered when attempting to use the azure-ai-projects SDK before switching to the standard OpenAI SDK:

IssueDetail
FunctionTool import pathMust import from azure.ai.agents.models, not azure.ai.projects.models
create_thread does not existUse create_thread_and_process_run instead
list_messages does not existUse client.agents.messages.list(thread_id=...)
MessageRole.ASSISTANT does not existUse the string "assistant" directly
enable_auto_function_calls(toolset=...) failsParameter is tools=, not toolset=
Function not found errorCall client.agents.enable_auto_function_calls(tools=toolset) before create_agent
Agent traffic bypasses APIMAI Foundry Agent Service uses its own endpoint resolution — use standard OpenAI SDK pointed at APIM instead

The Agent Service SDK is in active beta development (azure-ai-agents==1.2.0b6 at the time of writing). Expect these APIs to stabilise and the APIM routing issue to be addressed in future versions.

Pitfalls Summary

PitfallFix
Grounding with Bing Search G1 SKU not eligibleRequires Pay-As-You-Go or EA subscription
Bing.Search.v7 CLI creation failsResource type moved to Microsoft.Bing/accounts
BOM in Python files on WindowsUse [System.IO.File]::WriteAllLines with UTF8Encoding($false)
APIM endpoint doubles /openai pathStrip /openai from URL before passing to AzureOpenAI client
App Config 403 on first runWait 2–5 minutes for role assignment propagation
CLI Application Insights query empty5–10 minute ingestion lag — check portal Performance blade instead
AI Foundry Agent Service bypasses APIMUse standard openai SDK pointed directly at APIM gateway

What the Full Citadel Loop Delivers

With the agent running through APIM, every LLM call in the tool-calling loop is governed:

Content Safety — both the user question and the synthesised response pass through Azure AI Content Safety policies configured in APIM.

Token tracking — each of the two LLM calls contributes to the token usage log in Cosmos DB, giving you per-call cost attribution by APIM subscription key. The Cosmos DB ai-usage-container in the hub captures a structured document for each LLM call, including the model version, token counts, gateway region, request IP, APIM subscription name, backend routing, and timestamp. In production, the productName field maps to the APIM subscription key. Aggregating documents by this field gives you direct FinOps reporting per AI initiative.

Azure Cosmos DB Data Explorer showing a usage event document in the ai-usage-container of the Citadel hub, with fields including model gpt-4o-2024-11-20, promptTokens 17, responseTokens 53, totalTokens 70, gatewayRegion Sweden Central, productName Portal-Admin, and timestamp 6/24/2026, confirming token tracking and cost attribution via the Microsoft Foundry Citadel APIM governance hub.
The Citadel hub Cosmos DB ai-usage-container showing a usage document captured from the tool-calling agent run model gpt-4o-2024-11-20, 70 total tokens, gateway region Sweden Central, routed via apim-wpvlimv4ngkns. Every LLM call through APIM generates a document like this, which serves as the cost attribution and audit trail for enterprise AI governance.

Latency observability — Application Insights captures the duration of every call, making it easy to identify slow tool calls or model latency spikes.

Audit trail — every request is logged with timestamp, operation name, response code, and duration. For a healthcare or financial services context, this is your compliance evidence.

What’s Next

This post wires a tool-calling agent to the Citadel hub using the standard OpenAI SDK. The natural next steps:

Azure AI Foundry Agent Service routing — as the SDK matures, the azure-ai-projects client will likely gain proper APIM gateway support. Watch the azure-ai-agents release notes for updates on connection-based routing.

Conversation persistence — store conversation history in the Cosmos DB conversations container already deployed in the spoke. The App Config key CONVERSATIONS_DATABASE_CONTAINER points to it.

Network isolation — re-enable networkIsolation=true in the spoke parameters to route all traffic through private endpoints.

Multiple tools — extend the agent with additional function tools (document lookup, product catalog, claims system) using the same pattern. Each tool call flows through APIM and is governed identically.

Conclusion

Connecting a real tool-calling agent to the Microsoft Foundry Citadel Platform on Azure requires three components: the standard OpenAI SDK configured to point to the APIM gateway, a function tool with a JSON schema definition, and an explicit tool-call-handling loop. Everything else, governance, content safety, token tracking, and cost attribution, is handled by the Citadel hub automatically.

The path to get here involved navigating several SDK beta rough edges and discovering that the AI Foundry Agent Service bypasses APIM in its current preview form. These are expected friction points with a platform in active development. The governance architecture underneath is sound, the APIM policies work, and the Application Insights telemetry confirms it.

Two LLM calls. Both governed. Both visible. That is what the Citadel hub delivers.

Azure Logic Apps Agent Loop Production Operations

Posted on by
Reply

Part 7 of 7 in the Logic Apps Agent Loop series

Part 6 covered the security stack for agentic workflows: Easy Auth, Managed Identity, and Key Vault. This final post closes the series with Azure Logic Apps agent loop production operations: how to monitor agent loops with Application Insights, what the pricing model looks like across Standard and Consumption, the key platform limits to be aware of, and how to deploy agentic workflows through a repeatable DevOps pipeline.

By the end of this post, you will have a complete picture of what it takes to run an agentic workflow in production, not just to build one.

Azure Logic Apps agent loop production monitoring with Application Insights

The run history you have used throughout this series is the starting point for understanding what an agent loop did and why. For production workloads you need more: aggregated metrics across multiple runs, structured log queries, alerting on failures, and tracing across distributed systems. Application Insights provides all of this for Standard logic apps.

Enabling Application Insights

If you did not enable Application Insights when you created la-agent-loop, you can add it after deployment:

  1. In the Azure portal, open your la-agent-loop logic app resource
  2. Navigate to Application Insights under Settings in the left sidebar
  3. Click Turn on Application Insights
  4. After the pane updates, click Apply → Yes
  5. Click View Application Insights data to open the dashboard

Application Insights begins collecting telemetry from that point forward; it does not backfill historical run data.

What Application Insights captures for agent loops

For Standard agentic workflows, Application Insights captures enhanced telemetry beyond what the run history provides. Key data points include:

Requests — each workflow trigger appears as an incoming request, with duration, success/failure status, and HTTP response code.

Dependencies — each tool call the agent makes appears as a dependency call, with the target service, duration, and result. Moreover, for an agent loop that invokes Azure OpenAI and Azure AI Search, you will see both as dependency entries, making it straightforward to identify which tool call is slowest.

Exceptions — any workflow failure surfaces as an exception with a full stack trace, correlated to the specific run and iteration where it occurred.

Custom metrics — Logic Apps emits custom metrics for agent loop iterations, token usage, and tool invocation counts. These are queryable via Kusto (KQL) in the Logs blade.

Useful KQL queries for agent loops

You can query agent loop run durations for, let’s say, over the last 72 hours:

requests | where timestamp > ago(72h) | where name contains "agent" | summarize avg(duration), max(duration), count() by bin(timestamp, 1h) | render timechart

To identify failed agent loop runs:

requests | where timestamp > ago(72) | where success == false | project timestamp, name, duration, resultCode, cloud_RoleInstance | order by timestamp desc

To track tool call durations:

dependencies | where timestamp > ago(24h) | where type == "HTTP" | summarize avg(duration), count() by target | order by avg_duration desc

Reading the run history for agent loops

The run history in the Logic Apps portal is the fastest way to debug a specific agent loop run. For agentic workflows it shows more than a conventional run history — each agent action expands to show its iterations, and each iteration shows the model’s reasoning, the tool calls it made, and the results it received.

The Agent activity tab is the most useful view for agentic workflows. It shows the conversation between the model and the tools in chronological order, every message the model generated, every tool it invoked, and every result it received. The agent loop reveals its chain of thought.

Key things to look for in the run history:

  • Iteration count — how many Think → Act → Observe cycles the loop ran. A loop that runs the maximum number of iterations (default 100) without completing is a signal that the instructions are ambiguous or the tools are not returning usable results.
  • Tool call inputs and outputs — expand each tool call to see exactly what the model passed as parameters and what the tool returned. This is the fastest way to diagnose a tool that is returning unexpected data.
  • Token usage — the metadata output of each agent action shows total tokens, prompt tokens, and completion tokens. High prompt token counts indicate the conversation history is growing large — consider enabling agent history reduction.

Azure Logic Apps agent loop production pricing: Standard versus Consumption

The pricing model for agentic workflows differs between Standard and Consumption, and it differs significantly from conventional Logic Apps pricing.

Standard

Standard logic apps use a fixed App Service Plan pricing model — you pay for the compute capacity whether the workflow is running or not. Agentic workflows on Standard do not incur extra charges beyond the base App Service Plan cost. However, every Azure OpenAI call the agent makes is billed separately against your Azure OpenAI resource at standard token rates.

For the la-agent-loop workflows in this series:

  • The Standard logic app itself: App Service Plan (Workflow Standard WS1 or higher)
  • Each GPT-4o call: billed to aoai-demo-ptu at your PTU reservation rate
  • Azure AI Search queries (if used): billed separately at Search tier rates

The practical implication is that Standard agentic workflow costs scale with model usage, not with workflow execution count. A loop that runs five iterations and calls GPT-4o five times costs five times more in model tokens than a loop that resolves in one iteration.

Consumption

Consumption agentic workflows use a pay-as-you-go model. Agent loop pricing is based on the number of tokens each agent action uses and appears as Enterprise Units on your bill. This is a different billing unit from the standard Consumption action executions — each token consumed by the agent is metered separately.

The Consumption agent loop is also subject to throttling based on token usage — unlike Standard, which is constrained only by the App Service Plan compute capacity.

For production workloads with predictable, high-volume agent loop usage, Standard with a PTU Azure OpenAI deployment is the more cost-predictable option. For low-volume or experimental workloads, Consumption pay-as-you-go avoids the fixed App Service Plan cost.

Known limits for agentic workflows

Before going to production, be aware of the current platform limits:

Tool constraints — tools can only contain actions, not triggers. A tool must start with an action and always contains at least one action. Control flow actions (conditions, loops, switches) are not supported inside tools. A tool only works inside the agent loop where it is defined — it cannot be shared across agent actions.

Consumption-specific limits — Consumption agentic workflows can only be created in the Azure portal, not Visual Studio Code. The AI model can come from any region, so data residency for a specific region is not guaranteed for data the model handles. The agent action is throttled based on token usage.

Agent history — by default the agent loop accumulates the full conversation history across iterations. For long-running loops this can push the context length toward the model’s limit. Enable agent history reduction in the agent action’s Settings tab to manage this. The default strategy is token count reduction with a ceiling of 128,000 tokens — adjust this based on your model’s context window and your scenario’s complexity.

Deploying agentic workflows through a DevOps pipeline

Standard logic apps are built on the Azure Functions runtime and deploy the same way as any other Standard logic app — via zip deploy, Azure Pipelines, or GitHub Actions. The workflow definitions are JSON files on disk, making them version-controllable and deployable through standard CI/CD patterns.

What to include in source control

For an agentic workflow project, the key files to version-control are:

  • sequential-agents/workflow.json — the sequential agent loop definition
  • sample/workflow.json — the autonomous agent from Post 2
  • mcp-research/workflow.json — the MCP research workflow from Post 4
  • connections.json — connection references (without credentials — those go in Key Vault)
  • host.json — Logic Apps host configuration
  • local.settings.json — local development settings (excluded from source control, .gitignore)

Deploying with Azure CLI

The simplest production deployment from a CI/CD pipeline uses the Azure CLI:

# Zip the logic app project zip -r la-agent-loop.zip . -x "*.git*" "local.settings.json"

# Deploy to Azure az logicapp deployment source config-zip \ --name la-agent-loop \ --resource-group rg-ai-solutions \ --src la-agent-loop.zip

Environment-specific configuration

Agent connections and app settings differ between development and production environments. Use Azure CLI or Bicep to set environment-specific app settings as part of the deployment pipeline:

az logicapp config appsettings set \ --name la-agent-loop \ --resource-group rg-ai-solutions \ --settings \ agent_openAIEndpoint="https://aoai-prod.openai.azure.com/" \ OPENAI__endpoint="https://aoai-prod.openai.azure.com/"

This keeps environment-specific values out of source control and injected at deploy time — the standard twelve-factor app pattern applied to Logic Apps.

Closing the series

This post closes a seven-part series on Azure Logic Apps agent loop production operations, from first principles through to observability, pricing, and DevOps deployment. The series covered:

  1. Why the agent loop is a different design paradigm from conventional workflow automation
  2. The anatomy of a single agent loop — trigger, instructions, model, and tools
  3. Autonomous versus conversational agentic workflows: when to use each
  4. Building tools: connectors, custom connectors, and MCP servers
  5. Multi-agent patterns: prompt chaining, routing, handoff, and orchestrator-workers
  6. Securing agentic workflows: Easy Auth, Managed Identity, and Key Vault
  7. Observability, pricing, and production operations — this post

The agent loop is still a rapidly evolving capability in Azure Logic Apps. The platform limitations documented throughout this series Foundry Models connection persistence, API Center MCP wizard regional constraints, Foundry OpenAPI tool network restrictions will be addressed in future platform releases. The architectural patterns, however, are stable: the four building blocks of an agent loop, the three tooling layers, the four multi-agent patterns, and the two-concern security model will remain the right mental model for this platform regardless of how the surface-level tooling evolves.

Azure Logic Apps Agentic Workflow Security in Production

Posted on by
2

Part 6 of 7 in the Logic Apps Agent Loop series

Part 5 covered multi-agent patterns in the Azure Logic Apps agentic workflow series. Each pattern extends your agent’s reach, but that reach comes with a security cost. The more capable and connected your agent, the more important it is to understand who can call it and under what conditions. This post covers the expanded caller surface, the developer key’s limitations, and the full production security stack.

Conventional Logic Apps workflows have a bounded caller surface. The callers are known systems: a scheduler, a service bus, and an HTTP client you control. The authentication model is straightforward: SAS tokens, Managed Identity, and IP filtering. Agentic workflows fundamentally change this, particularly conversational ones. When you expose a chat interface to external callers, those callers can be people, other agents, MCP servers, or automation clients from networks you do not control. The security model has to change with the threat model.

Two-column diagram showing the security model for Azure Logic Apps agentic workflows. Left column shows the caller surface: human users via external chat client, external agents with dynamic unknown callers, MCP servers on untrusted networks, automation clients for CI/CD, and a developer key marked as portal testing only and not for production. Arrows from all caller types point toward the right column. Right column shows the security stack from top to bottom: Entry via Easy Auth with Microsoft Entra ID and Conditional Access, Logic app Standard running agentic workflows and agent loops, Managed Identity for backend authentication to Azure OpenAI, AI Search, and Storage, Azure Key Vault for secrets that cannot use Managed Identity, and Consumption OAuth 2.0 with Entra ID agent auth policy at the bottom. Legend shows teal for auth layers, purple for workflow and caller, coral for avoid in production.
Figure 1 — The two security concerns for Azure Logic Apps agentic workflows. The caller surface (left) expands significantly compared to conventional workflows. Human users, external agents, MCP servers, and automation clients can all reach the workflow endpoint from networks you do not control. The developer key used during portal development is explicitly not suitable for any of these caller types. The security stack (right) addresses the expanded surface area in two directions: Easy Auth with Microsoft Entra ID secures who can invoke the workflow, while Managed Identity and Key Vault secure what the workflow can call, without storing credentials in app settings.

The expanded caller surface

The shift from nonagentic to agentic workflows introduces a qualitatively different caller population. In a nonagentic workflow the trigger is called by a known system at a known time for a known reason. In a conversational agentic workflow the trigger is called by:

  • Human users interacting through an external chat client
  • External agents invoking the workflow as a tool
  • MCP servers routing requests through the workflow
  • Automation clients from untrusted or unknown networks

Each of these caller types introduces different identity, trust, and access control requirements. A billing system calling a webhook is easy to reason about. An external agent calling your workflow from an unknown network at unpredictable intervals is not.

This expanded surface area is why Microsoft’s documentation draws a sharp distinction between the developer key used during design and testing in the Azure portal and proper production authentication. Understanding that distinction is the starting point for securing any agentic workflow.

The developer key: what it is and what it is not

Understanding the developer key’s limitations is the starting point for any serious Azure Logic Apps agentic workflow security implementation. When you test a conversational agentic workflow in the Logic Apps designer, the Azure portal authenticates your test calls using a developer key. The developer key is a convenience mechanism that lets you skip manual authentication setup during development. It fires automatically when you run a workflow, call a Request trigger, or interact with the integrated chat interface.

The developer key has five hard limitations that make it unsuitable for production:

  • It is not a substitute for Easy Auth, Managed Identity, federated credentials, or signed SAS callback URLs.
  • In addition, it is designed for large or untrusted caller populations, agent tools, or automation clients.
  • It is also not a per-user authorization mechanism; it has no granular scopes or roles.
  • And finally, it is not governed by Conditional Access policies at the request execution layer, only at the portal sign-in layer. And it is not intended for programmatic or CI/CD usage.

The developer key is linked to a specific user and tenant based on an Azure Resource Manager bearer token. Because of that binding, you cannot distribute it externally. It is, in the Microsoft documentation’s own framing, a mechanism for quick testing before you formalize authentication, not a path to production.

Azure Logic Apps agentic workflow security: Standard versus Consumption

The right production authentication mechanism depends on your Logic Apps hosting model.

Setting up Managed Identity for backend connections

Easy Auth secures who can call your agentic workflow. Managed Identity secures what your workflow can call. These are two distinct security concerns and both need to be addressed in production.

When your agent invokes a tool, Azure OpenAI, Azure AI Search, a storage account, or a Service Bus namespace, that call needs to be authenticated. The default approach during development is often to store an API key or connection string in app settings. In production, replace these with Managed Identity connections wherever possible. This removes credentials from app settings entirely. The logic app authenticates to backend services using its Azure AD identity, which is governed by RBAC, auditable, and revocable without rotating keys.

  1. Go to your la-agent-loop resource → Identity → System assigned → turn Status to On
  2. Save — Azure assigns a service principal to the logic app
  3. In each target resource (Azure OpenAI, AI Search, Storage), go to Access control (IAM) → Add role assignment
  4. Assign the appropriate role to the logic app’s Managed Identity:
    • Azure OpenAI: Cognitive Services OpenAI User
    • Azure AI Search: Search Index Data Reader
    • Azure Blob Storage: Storage Blob Data Reader
  5. In the Logic Apps connections, switch from API key authentication to Managed Identity for each backend service where possible.

Note: Managed Identity authentication for the agent model connection is only supported when the model type is AzureOpenAI. If your workflows use the MicrosoftFoundry model type, as in this series, the agent connection must use Key authentication. Managed Identity remains the right choice for all other backend connections such as Azure AI Search, Blob Storage, and Service Bus.

Azure portal Identity blade for the la-agent-loop Standard logic app. The System assigned tab is selected, Status is set to On, and the Object principal ID is shown as 8db32242-e936-4d84-a44a-6b39d37f24f7. An Azure role assignments button is visible under Permissions.
Figure 2 — System-assigned Managed Identity enabled on the la-agent-loop Standard logic app. Once enabled, Azure registers the logic app as a service principal in Microsoft Entra ID. Click Azure role assignments to assign the appropriate RBAC roles to each backend resource: Cognitive Services OpenAI User for Azure OpenAI and Search Index Data Reader for Azure AI Search, so the agent can authenticate to those services without storing any credentials in app settings.

Setting up Easy Auth for your Azure Logic Apps agentic workflow

For Standard logic apps, the production authentication path is Easy Auth, also known as App Service Authentication. Easy Auth is an App Service platform feature that sits in front of your logic app and enforces identity-based authentication on every incoming request before it reaches your workflow.

When you enable Easy Auth on a Standard logic app, external callers, whether human users, external agents, or MCP servers, must present a valid identity token. Easy Auth validates the token against Microsoft Entra ID before allowing the request through. This gives you full Conditional Access policy enforcement, per-user identity, token revocation, and audit logging, the full production security stack.

To set up Easy Auth on a Standard logic app:

  1. In the Azure portal, open your la-agent-loop logic app resource
  2. Navigate to Authentication in the left sidebar under Settings
  3. Click Add identity provider
  4. Select Microsoft as the identity provider
  5. Under App registration, select an existing registration or choose Create new app registration and name it la-agent-loop-auth
  6. Under Supported account types, select Current tenant — single tenant for internal workloads
  7. Set Unauthenticated requests to HTTP 401 Unauthorized: recommended for APIs
  8. Leave Token store enabled
  9. Click Add

Note: Easy Auth operates at the App Service host level, before the Logic Apps runtime processes the request. Authentication failures are rejected at the infrastructure layer with a 401 the workflow never executes and no run history entry is created for unauthenticated calls.

Azure portal Authentication blade for the la-agent-loop Standard logic app. Authentication settings show App Service authentication as Enabled, Restrict access set to Require authentication, and Unauthenticated requests set to Return HTTP 401 Unauthorized. The Identity provider section shows Microsoft with app registration la-agent-loop-auth and client ID bc8d8407-a79e-4a18-be48-f0fc54fa4966.
Figure 3 — Easy Auth configured on the la-agent-loop Standard logic app. App Service authentication is enabled, unauthenticated requests return HTTP 401 Unauthorized, and Microsoft Entra ID is registered as the identity provider via the la-agent-loop-auth app registration. Any external caller, human user, external agent, or MCP servermust now present a valid Entra ID token before the Logic Apps runtime processes the request.

Consumption: OAuth 2.0 with Microsoft Entra ID

For Consumption logic apps, configure an agent authorization policy on the logic app resource using OAuth 2.0 with Microsoft Entra ID. This provides equivalent identity enforcement to Easy Auth for the Consumption hosting model. For the full configuration steps, see Create conversational agent workflows in Azure Logic Apps on Microsoft Learn.

Key Vault for secrets that cannot use Managed Identity

Not every connection in an Azure Logic Apps agentic workflow supports Managed Identity. Where API keys or connection strings are unavoidable, store them in Azure Key Vault and reference them from Logic Apps app settings using the Key Vault reference syntax:

@Microsoft.KeyVault(SecretUri=https://your-keyvault.vault.azure.net/secrets/your-secret/)

This keeps credentials out of app settings in plain text, provides centralized rotation, and gives you audit logs of every secret access. The Standard logic app accesses Key Vault using its Managed Identity; no separate credentials are needed for the vault itself.

Network controls for Standard workflows

Standard logic apps run on the App Service infrastructure, which gives you network-level controls that Consumption workflows do not have:

Private endpoints allow your logic app to receive inbound traffic only from within a virtual network, removing public internet exposure entirely. This is the recommended configuration for production agentic workflows that serve internal users or agents.

VNet integration allows your logic app to make outbound calls to services within a virtual network, including on-premises systems, private Azure services, and internal APIs, without exposing those services to the internet.

IP access restrictions let you restrict inbound traffic to specific IP ranges at the App Service level, providing a lighter-weight alternative to private endpoints for scenarios where full network isolation is not required.

For production agentic workflows processing sensitive data, patient records, financial data, internal business intelligence, and private endpoints with VNet integration is the right starting point.

Azure Logic Apps agentic workflow security checklist

Before going live with any agentic workflow:

  • Easy Auth configured with Microsoft Entra ID (Standard) or OAuth 2.0 agent authorisation policy (Consumption)
  • Developer key not used or referenced in any production caller
  • Managed Identity enabled on the logic app and assigned to all backend services
  • API keys and connection strings moved to Key Vault references
  • Private endpoints configured for Standard workflows handling sensitive data
  • Conditional Access policies applied to the Entra ID app registration backing Easy Auth
  • Run history access restricted to authorised operations personnel

What comes next

The final post in this series concludes with operations: Application Insights integration, agent loop pricing, run history analysis, and deployment of agentic workflows through a CI/CD pipeline. Part 7 covers everything you need to run agent loops confidently in production.