Registering a Citadel Agent in the AI Registry: Azure API Center

This is the final post in a five-part series on the Microsoft Foundry Citadel Platform about Azure API Center AI agent registration. If you landed here first, a quick catch-up helps.

In Part 1, I deployed the Citadel Governance Hub (APIM, Azure OpenAI, Content Safety, Cosmos DB, Logic App) and an Agent Spoke (AI Foundry, Cosmos DB, Key Vault, App Config) in Sweden Central. Next in Part 2, I connected a tool-calling agent through APIM using the standard OpenAI SDK, with the Open-Meteo weather API as its only tool. In Part 3, the agent started writing every run to a conversations container in the spoke’s Cosmos DB. In Part 4, I built a three-layer kill switch so the platform team could shut an agent down fast when something went wrong.

By the end of Part 4, I had a working agent with a gateway, an audit trail, and an off switch. But I still had a gap that only shows up once you stop thinking about one agent and start thinking about a fleet of them: nobody outside my own head knew this agent existed.

That’s the problem this post solves.

Why register agents at all

One agent is easy to track. You know its name, its resource group, and roughly what it does, because you built it last week. Ten agents get harder. A hundred agents, spread across teams, spokes, and environments, becomes a governance problem rather than an inconvenience.

Platform teams eventually ask the same three questions about every agent in the estate: what does it do, what data does it touch, and who owns it. Without a registry, the answer lives in Slack threads, README files, and the memory of whoever built the thing. That doesn’t scale, and it definitely doesn’t survive an audit.

Azure API Center gives the Citadel Platform a place to answer those questions consistently, for every agent, in a format a platform team (or a compliance officer) can actually query. And as you’ll see later in this post, that same discipline lines up neatly with a transparency obligation that’s no longer theoretical: Article 49 of the EU AI Act.

Azure API Center 101

API Center and Azure API Management solve different problems, even though they sound similar and live in the same governance hub.

APIM is a runtime gateway. It sits in the request path, enforces policies, and does the actual work of routing, throttling, and inspecting traffic. That’s where the kill switch from Part 4 lives, and it’s where the five governance policies from earlier in the series get enforced.

API Center is a catalog. It doesn’t sit in the request path at all. Its job is to hold structured metadata about APIs and, in our case, agents: what they are, what version they’re on, who owns them, and what environment they run in. Think of APIM as the checkpoint and API Center as the registry office.

In the hub resource group rg-ai-hub-gateway-dev, the API Center instance is already deployed as apic-wpvlimv4ngkns, alongside APIM. That placement matters. The registry belongs in the hub because it needs to see across every spoke, not just the one running our weather agent.

The AI Publish Contract pattern

A generic API definition (an OpenAPI spec, for example) tells you the shape of the requests and responses. It doesn’t tell you whether the thing behind that shape is calling a third-party model, touching personal data, or making decisions that affect a person’s rights.

For agents, that gap matters more than it does for a plain REST API. So the Citadel Platform defines a small, structured metadata contract that every agent has to carry before it gets registered. I call it the AI Publish Contract. It rides alongside the API definition in API Center as custom metadata, rather than replacing the API definition entirely.

Here’s what it looks like for the weather agent:

json

{
"contractVersion": "1.0",
"agent": {
"id": "citadel-weather-agent-v1",
"displayName": "Citadel Weather Agent",
"description": "Tool-calling agent that answers weather queries using Open-Meteo as its only external tool.",
"owner": "platform-team@example.com",
"environment": "dev",
"spokeResourceGroup": "rg-ai-spoke-dev",
"aiFoundryProject": "aifp-tggi2gmkw22w4"
},
"modelBacking": {
"provider": "azure-openai",
"routedThrough": "apim-wpvlimv4ngkns",
"model": "gpt-4o-mini"
},
"tools": [
{
"name": "open-meteo-forecast",
"type": "external-api",
"endpoint": "https://api.open-meteo.com/v1/forecast",
"dataClassification": "public"
}
],
"dataClassification": "public",
"personalDataProcessed": false,
"highRiskCategory": false,
"killSwitchLayer": "named-value-flip",
"transparencyDisclosure": {
"aiActArticle49Applicable": false,
"userFacingDisclosureRequired": true,
"disclosureText": "This response was generated by an AI system."
}
}

A few fields deserve a comment, because I went back and forth on them.

dataClassification and personalDataProcessed exist because a registry that doesn’t record data sensitivity is only half a registry. The weather agent is a deliberately boring example: no personal data, no high-risk category, public weather data in and out. That’s exactly why it’s a good agent to demonstrate the pattern on before you register something that touches real user data.

transparencyDisclosure maps directly to the Article 49 discussion later in this post. Even for a low-risk agent, I keep the field in the contract rather than making it conditional. Consistency across every agent in the registry is the whole point.

Step-by-step: registering the weather agent in Azure API Center

With the contract defined, registration itself is a handful of steps. I’ll show the CLI first, then the portal equivalent, because I use both depending on whether I’m scripting a pipeline or walking a colleague through it live.

Prerequisites

You need an existing API Center instance (ours is apic-wpvlimv4ngkns in rg-ai-hub-gateway-dev) and Contributor access to it. If you’re following along, confirm the instance is visible first:

az apic show --resource-group rg-ai-hub-gateway-dev \
--name apic-wpvlimv4ngkns \
--subscription dc0f4d72-3734-4b03-8884-ccfb9c2c4cc7

Step 1: define the custom metadata schema

API Center lets you extend its metadata model with custom fields. Before registering any agent, define a schema for the AI Publish Contract so every future registration validates against the same structure.

az apic metadata create --resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--metadata-name aiPublishContract \
--schema "{\"type\":\"string\",\"title\":\"AI Publish Contract\"}"
--assignments "[{\"entity\":\"api\",\"required\":true}]"

I keep ai-publish-contract-schema.json as a JSON Schema file that mirrors the structure shown above. Doing this once means every agent registered afterward gets validated the same way, instead of drifting field by field as different teams register their own agents.

Step 2: register the agent as an API

az apic api create --resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--api-id citadel-weather-agent-v1 \
--title "Citadel Weather Agent" \
--type rest \
--custom-properties @C:\Users\steef\ai-citadel-agent\citadel-agent\ai-publish-contract.json

The --type rest flag looks like a mismatch at first, since this is an agent, not a REST API in the traditional sense. API Center doesn’t yet have a native “agent” type, so I register it as a REST API and let the AI Publish Contract metadata carry the agent-specific detail. That’s a workaround, not a limitation I’d defend as elegant, and I expect this to get cleaner as Microsoft extends API Center for agentic workloads.

Step 3: create the API version

az apic api version create --resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--api-id citadel-weather-agent-v1 \
--version-id v1-0 \
--title "v1" \
--lifecycle-stage development

Step 4: attach the definition

az apic api definition create --resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--api-id citadel-weather-agent-v1 \
--version-id v1-0 \
--definition-id openapi \
--title "OpenAPI"

Then import the actual spec, which describes the agent’s HTTP-facing invocation contract (the endpoint APIM exposes, not the internal tool-calling logic):

az apic api definition import-specification --resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--api-id citadel-weather-agent-v1 \
--version-id v1-0 \
--definition-id openapi \
--format link
--value "https://apim-wpvlimv4ngkns.azure-api.net/weather-agent/openapi.json"
--specification "{\"name\":\"openapi\",\"version\":\"3.0.0\"}"

That failed immediately:

(ValidationError) Could not download the API specification from
the specified URL due to an error:
Could not reach the provided URL.
Please make sure that the specification file
is publicly available and that the link is valid and up-to-date.

My first assumption was a network or auth issue between API Center and APIM. It wasn’t. Curling the same URL directly returned a plain 404 from APIM itself:

{ "statusCode": 404, "message": "Resource not found" }

That 404 is the real signal. There is nothing published at that path, because there was never a dedicated API resource for the weather agent in APIM to begin with. Part 2 wired the agent up through APIM as a passthrough to Azure OpenAI, using the standard OpenAI SDK. That gets you routing, policy enforcement, and governance on every call, but it doesn’t give you a discoverable, documented API surface. A passthrough proxies requests. It doesn’t publish an OpenAPI spec describing them.

This is worth sitting with for a second, because it’s not just a command-line gotcha. It means the OpenAPI definition API Center wants isn’t something you can export from a running agent. You have to author it, the same way you’d write documentation, because the system underneath genuinely has nothing to hand you.

So the real fix has two parts. First, write a minimal OpenAPI spec that documents the agent’s actual invocation contract as it exists through the passthrough:

{
"openapi": "3.0.0",
"info": {
"title": "Citadel Weather Agent",
"version": "1.0.0",
"description": "Tool-calling agent invocation contract, routed through APIM to Azure OpenAI."
},
"servers": [
{ "url": "https://apim-wpvlimv4ngkns.azure-api.net" }
],
"paths": {
"/openai/deployments/{deploymentId}/chat/completions": {
"post": {
"summary": "Invoke the weather agent",
"parameters": [
{
"name": "deploymentId",
"in": "path",
"required": true,
"schema": { "type": "string" }
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"messages": { "type": "array" },
"tools": { "type": "array" }
},
"required": ["messages"]
}
}
}
},
"responses": {
"200": {
"description": "Chat completion response, including any tool calls the agent made.",
"content": {
"application/json": {
"schema": { "type": "object" }
}
}
}
}
}
}
}
}

Save that locally as openapi.json, then import it inline instead of by link, since there’s nothing at the other end of a link to point to:

az apic api definition import-specification --resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--api-id citadel-weather-agent-v1 \
--version-id v1-0 \
--definition-id openapi \
--format inline \
--value @C:\Users\steef\ai-citadel-agent\citadel-agent\openapi.json \
--specification "{\"name\":\"openapi\",\"version\":\"3.0.0\"}

That worked. But notice what actually happened here. The spec isn’t extracted from the system, it’s a description you write and commit to keeping accurate. If you add a second tool to the agent, or change the invocation shape, this file goes stale unless you treat it as part of the change, not an afterthought to registration.

Step 5: register the environment and deployment

az apic environment create \
--resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--environment-id dev \
--title "Development" \
--type development
az apic api deployment create --resource-group rg-ai-hub-gateway-dev \
--service-name apic-wpvlimv4ngkns \
--api-id citadel-weather-agent-v1 \
--deployment-id dev-deployment \
--title "Dev deployment" \
--environment-id "/workspaces/default/environments/dev" \
--definition-id "/workspaces/default/apis/citadel-weather-agent-v1/versions/v1-0/definitions/openapi" \
--server "{\"runtimeUri\":[\"https://apim-wpvlimv4ngkns.azure-api.net/weather-agent\"]}"

The portal equivalent FOR Azure API Center

If you’d rather click through it, the same five steps map onto the portal like this. Open the API Center resource in the Azure portal, and go to APIs. Select “Register API,” fill in the title and type, and paste the AI Publish Contract JSON into the custom metadata panel that appears once the metadata schema is assigned. From there, add a version, upload or link the OpenAPI definition, and finish by adding an environment and a deployment under the API’s Deployments tab.

I default to the CLI for anything I’ll repeat across agents, and the portal when I’m registering something once or explaining the process to someone new to the platform.

What it looks like in the fleet dashboard

Once registration finishes, the weather agent shows up in the Azure API Center catalog alongside anything else registered in the hub. The catalog view lists each API (agent) with its title, type, and lifecycle stage, and clicking into it surfaces the custom metadata panel with the full AI Publish Contract attached.

This is where the payoff becomes visible. Instead of a spreadsheet someone updates twice a year, the platform team gets a live, filterable view. Filter by dataClassification: personal and you instantly see which agents touch sensitive data. In addition, filter by environment: production and you get the actual production fleet, not the fleet someone remembers deploying. And finally, filter by owner, and you know exactly who to page when an agent misbehaves.

For a single weather agent, this looks like overkill. It isn’t the weather agent that justifies the pattern. It’s the fortieth agent, registered by a team you’ve never met, that you’ll be glad has the same contract structure as this one.

Connecting to EU AI Act Article 49

Article 49 of the EU AI Act introduces a registration obligation, primarily for providers and deployers of high-risk AI systems, who must register certain information in an EU database before those systems go into service or are used. The exact scope depends on the system’s risk classification, and I’m not going to pretend to give legal advice here. Talk to your compliance team about whether a specific system falls under that obligation.

What I can talk about is the architecture. The AI Publish Contract pattern doesn’t make an agent compliant with Article 49. What it does is put your organization in a position to answer an Article 49-style question quickly: which AI systems exist, what do they do, what data do they process, and are any of them high-risk. That’s the same question a platform team asks internally for entirely operational reasons, and it turns out to be the same question a regulator asks for entirely different reasons.

Building the registry now, before any single agent forces the issue, means the answer already exists when someone asks. Retrofitting a registry across dozens of agents that were never designed with one in mind is a much worse project to inherit.

The weather agent in this post sets aiActArticle49Applicable: false, because it’s a low-risk, non-personal-data agent. The field exists precisely so the next agent, the one that does touch personal data or make consequential decisions, has a place to say true and trigger whatever process your organization builds around that flag.

Pitfalls

A few things caught me off guard while working through this. I’m flagging these as the kind of issues I’d expect to hit on a real rollout, so treat them as a checklist to verify against your own environment rather than a transcript of my exact errors.

  • A passthrough agent has no OpenAPI spec to export, even though it looks like it should. This is the one that actually got me. Importing by link failed with a vague reachability error from az apic, which pointed me toward a networking explanation. Curling the URL directly gave the real answer: a plain 404 Resource not found from APIM. There was nothing published at that path, because Part 2 wired the agent up as a passthrough to Azure OpenAI, not as a dedicated API resource with its own documented surface. Passthroughs proxy requests, they don’t publish specs. The fix is to hand-author a minimal OpenAPI spec describing the invocation contract and import it inline, and then treat that file as something you update deliberately when the agent’s interface changes, not something the platform keeps in sync for you.
  • Custom metadata schema assigned after APIs already exist. If you register an API before creating the metadata schema, the custom properties won’t validate retroactively. Define the schema first, every time, or you’ll end up patching APIs one by one afterward.
  • RBAC gaps across the hub-spoke boundary. The identity registering the agent needs Contributor (or a custom role with equivalent permissions) on the API Center instance in the hub, which is a different resource group than where the agent itself runs. Teams that only have access to their own spoke will hit a permissions wall here, and that’s worth deciding on deliberately rather than discovering it during a rollout.

Other Pitfalls

  • Confusing API Center registration with APIM configuration. These are separate systems that happen to sit in the same hub. Registering an agent in API Center doesn’t change how APIM routes or governs it. I’ve seen (and made) the assumption that registration alone would trigger some policy change in APIM. It doesn’t. They’re linked by convention and shared metadata, not by any automatic sync.
  • Definition drift. The OpenAPI definition describes the HTTP surface of the agent, but the AI Publish Contract describes its behavior and data handling. Nothing enforces that these stay in sync when the agent changes. Treat both as part of your deployment pipeline, updated together, rather than the contract as a one-time registration artifact.
  • Lifecycle stage mismatches. Marking an API’s --lifecycle-stage as development when it’s actually running in a shared dev/test environment used by multiple teams creates confusion during the fleet review. Match lifecycle stages to how the environment is actually used, not just what resource group it happens to sit in.

What’s next

This closes out the deployment arc of the series: hub and spoke, a working agent, persistence, a kill switch, governance policies, and now a registry entry. What I haven’t covered yet, and what comes next, is hardening this for anything beyond a single dev environment. That means promotion paths from dev to staging to production, cost controls that scale with a growing agent fleet, and staging environment parity so what you test actually matches what you ship.

If there’s interest, that hardening work is a natural follow-up series rather than a single post, given how much ground it covers.

Wrapping up the series

Five posts ago, this started as an empty resource group in Sweden Central. It’s now a governed platform: a hub that enforces policy and holds the registry, a spoke that runs the agent, a kill switch that can shut things down fast, and a contract-based registration pattern that keeps the fleet auditable as it grows.

The weather agent was never really the point. It was the simplest possible agent I could use to prove out every layer of the platform without the complexity of a real production workload getting in the way. The patterns here, hub-spoke isolation, layered kill switches, policy-driven governance, and contract-based registration, are the parts meant to outlast this specific agent.

Thanks for following along through the series. If you’re building something similar, I’d like to hear what broke for you.

Logic Apps Automation Preview

Microsoft announced Azure Logic Apps Automation at Build 2026 and put it straight into public preview at auto.azure.com. The launch framing was “automation just became a team sport“. The product framing is a managed SaaS experience: you sign in, and compute, connectors, model endpoints, and knowledge services are already there.

I have spent the last months in the Logic Apps agent loop on Standard, and I wrote a seven-part series about what actually breaks there. So my first reaction to Automation was not “new designer, nice”. It was a governance question: who owns the workflow, who can read it, and what happens when the person who built it leaves?

That question turns out to be the most interesting thing about this release, and almost nobody is writing about it.

This post covers what Automation is, how it works, a scenario you can build and demo in about half an hour, and an honest list of what preview does not do yet.

Logic Apps Automation: What it actually is

Strip the marketing and Automation is four things at once:

  • A new SKU on the same engine. The Logic Apps runtime is unchanged. The connector catalogue, 1,400 plus, is the same one you already use. Expressions, control flow, stateful and stateless workflows, draft and published versions: all familiar. Adam Marczak put it well after testing it: Automation is new, but agentic Logic Apps is not.
  • A new hosting model. Consumption is multitenant and scales to zero. Standard is single tenant but you provision and pay for capacity. Automation is single tenant with a dedicated runtime boundary, Microsoft manages the hosting capacity, and it scales 0 to N. That combination did not exist before.
  • A new resource hierarchy. Project, then Application, then Workflow. Sandboxes sit at project level. This is the part that changes how you govern.
  • A new portal. auto.azure.com is not a replacement for the Azure portal. It sits alongside it. Projects show up as Azure resources in your resource group, but you build in the new experience.
Diagram of the Logic Apps Automation resource hierarchy: a Project contains an Application, which contains Workflows with draft and published versions, alongside project-level Sandboxes and Project settings. A second column shows the two permission scopes, project and app, plus Owner as a property and the boundary where project admins see app metadata only.
Figure 1. Project, Application, Workflow, plus the two permission scopes that sit on top of them. Project admins see app metadata for governance, not workflow contents, connections, or run history.

Logic Apps Sandbox

One capability worth pausing is the Logic Apps Sandbox: built-in Python code execution within the agent loop, running in a secure, isolated environment with no external compute resource required. In the Standard agent loop series, I covered tools as connector actions the model can invoke at runtime. Sandbox adds a different category entirely: the agent writes code, executes it, observes the output, and iterates. Data transformation, chart generation, CSV processing, and dynamic API calls tasks that connectors alone cannot handle and that previously required an Azure Function or a custom action sitting outside the workflow. In Automation, Sandbox is already provisioned alongside the model endpoints and connectors. You do not configure it. You enable the Code Interpreter toggle in the agent action, and the capability is there. That is the managed SaaS promise made concrete, and it is the scenario I plan to cover in a dedicated follow-up post.

Where it fits next to the SKUs you already run

Positioning is now clearer than it was in the first 48 hours after Build. Power Automate is personal and team productivity inside Microsoft 365. Logic Apps are the enterprise integration platform: SAP, EDI, B2B, high-throughput API orchestration, predictable 24×7 load. Consumption remains excellent for sporadic, event-driven work.

Automation aims at the gap in the middle. Teams that need enterprise grade infrastructure but want a SaaS experience and AI assisted creation.

Bubble chart plotting four Microsoft workflow products by builder profile, from business team to integration developer, against infrastructure ownership, from fully managed to customer provisioned. Power Automate and Logic Apps Consumption sit in the managed lower area, Logic Apps Standard in the customer-provisioned upper right, and Logic Apps Automation is highlighted in the middle as single-tenant isolation with Microsoft-managed capacity.
Figure 2. Automation is not a replacement for Standard. It fills the gap between a productivity tool and an integration platform: single-tenant isolation, but Microsoft manages the capacity.

A short decision table, and I am reading direction here rather than quoting a Microsoft matrix:

ScenarioWhere I would put it today
SAP, EDI, B2B, high throughput API orchestrationStandard
Central integration platform, predictable 24×7 loadStandard
Simple event driven automation, low volumeConsumption
AI and agent workloads with bursty trafficAutomation
Long idle periods with traffic spikesAutomation
Departmental workflow owned by a business teamAutomation, with the caveats in section 5
Personal productivity inside Microsoft 365Power Automate

How an agentic workflow actually runs

An agent in Automation is a workflow action backed by a model. You give it a system prompt, a toolset, an input, and downstream actions consume its output. There are two flavours.

Native agents run the loop inside the workflow runtime. Every iteration appears in the execution log. Tools are the action nodes you place inside the agent boundary, plus a Code Interpreter toggle.

Foundry agents hand off to Azure AI Foundry Agent Service. The workflow sees one call and a final output. You pick this when the assistant already exists in Foundry.

The switch between them is a config change in the AI model dropdown. The rest of the workflow does not care. That is a genuinely good design decision.

Diagram of a native agent action at runtime. A trigger feeds an agent boundary in which the model loops with its tools, issuing tool calls and receiving tool results, grounded by knowledge bases and a sandbox. The final structured output flows to deterministic downstream actions. An observability lane below shows that every step writes to real-time run history.
Figure 3. The loop, the tools, the grounding, and the lane where you debug all of it. Branch on the agent’s structured output, not on its prose answer.

Two expression details worth memorising, because they are the seams where things break:

@outputs('AgentName')['lastAssistantMessage']
@outputs('AgentName')['structuredOutput']?['severity']

and inside a tool, to read what the model decided to pass you:

@agentParameters('errorCode')

Build on structuredOutput, not on the final message. In my blog series, I hit this the hard way: agent output arrives as a structured payload, and reaching for the prose answer produces workflows that pass a demo and fail on the third real message.

Logic Apps Automation: A scenario you can build and demo

Here is a demo that survives contact with an audience. It uses an HTTP trigger, which matters: HTTP and manual triggers fire from a draft, so you can iterate with Test your draft without publishing. Schedule and event driven triggers only fire against the published workflow.

The scenario: integration failure triage. A message lands on a dead letter queue. Instead of paging a human with a JSON blob, an agent classifies the failure, checks the runbooks, decides whether it is retryable, and either posts a structured summary to the on call channel or raises a ticket. The deterministic parts stay deterministic.

Flow diagram of the integration failure triage demo. An HTTP trigger receives a dead-letter payload, Parse JSON types it, and a triage agent with lookup, recent-failures, code-interpreter, and runbook-knowledge tools returns a structured verdict. Three deterministic branches follow: high severity raises a ticket, retryable requeues, and everything else goes to a digest.
Figure 4. A demo that shows the loop, the grounding, and the deterministic guardrails around both. The convincing run is the one where the agent declines to invent an answer.

CREATE Project

  • Create a project at auto.azure.com, then create an application inside it. Wait for the status to flip from Building to Ready before you open it. This will take some time.
  • Start the workflow. You can type the intent into the assistant box, or click Build from scratch. For a demo I build from scratch, because every step stays visible and explainable.
  • Add the trigger: search for Request, pick When an HTTP request is received. Give it a schema so downstream tokens are typed.
  • Add Parse JSON. Yes, the trigger schema already types things. Doing it explicitly makes the failure mode visible when you demo a malformed payload.

Building the Agent Steps

  • Drop in the agent action. System message, roughly: You triage failed integration messages. Classify severity as high, medium or low. Decide whether the failure is retryable. Use the runbook knowledge source before you answer. If the runbooks do not cover this error code, say so explicitly and set severity to medium. Never invent a runbook reference. Return structured output only.
  • Attach a knowledge base. Agent panel, Knowledge tab, Add knowledge source, Document Upload, upload three or four runbook pages. Knowledge bases are private preview, so this may not be enabled on your project yet. Azure AI Search is the fallback if you already maintain an index.
  • Add tools inside the agent boundary. Keep it to three or four. Toolsets of three to seven beat toolsets of twenty. Write the descriptions like docstrings, because the model’s reasoning is only as good as they are.
    • lookup_error_code as an HTTP action against your error catalogue. Inside it, read the argument with @agentParameters('errorCode').
    • get_recent_failures as a SQL or HTTP action, so the agent can see whether this is a one off or the fortieth today.
    • code_interpreter toggled on in the Parameters tab, for counting and grouping. It is JavaScript only, has no network access and no filesystem, so do not ask it to fetch anything.
  • Set the iteration bound in the Settings tab. Six is plenty here. This is your cost fuse.
  • Branch on the structured output, not the prose:
   @outputs('Triage_Agent')['structuredOutput']?['severity']

High goes to a ticket and an on call ping. Retryable goes back on the queue. Everything else goes into a digest.

Then click Test your draft, and watch the monitoring tab stream the run live. Real time run history is the single biggest day to day improvement over Standard, and it is the thing your audience will notice first.

The preview reality check

Every launch post is written in the present tense about a future state. Here is the current state, as of this writing.

AreaStatus today
CI/CD and deployment pipelinesMissing. There is no deployment story yet.
Export the whole solution as codeMissing. Versioning exists, but at workflow level.
ARM exportWorkflow code does not appear in the ARM template, and Export Template fails in the Azure portal. You can copy and paste workflow code from the Automation portal.
Conversational workflowsNot supported in Automation today, although they are documented for Consumption and Standard in Logic Apps Labs.
VNet integration and private endpointsContested. See below.
Knowledge basesPrivate preview. Document upload limits, per source token budgets and granular permissions are all still moving.
Sandbox input files.txt and .md only in private preview. CSV needs a contentType set by hand in code view.
Code InterpreterJavaScript only. No network. No filesystem. Per execution timeout. Sized for transformation, not compute.
Model supportNot every model works yet. Marczak reports workflows failing on GPT 5.1 that ran on 4.1, and the same workflows working on Standard.
PricingNot finalised. Third party posts quoting specific meters are running ahead of what Microsoft has published.
RegionsAn initial set at launch, with more rolling out. Check before you promise anything.

The VNet discrepancy, because it matters

Microsoft’s launch post describes virtual network integration and private endpoints as day zero enterprise capability. The same post lists “VNet support and private endpoints” in its coming soon section. An MVP testing the preview reports it as missing.

I am not calling anyone wrong. I am saying that if you work in a regulated sector, this is the one line item you must verify in your own tenant before it appears on any roadmap slide. For a health insurer, “reaching internal systems without exposing them to the internet” is not a feature. It is the precondition for the conversation.

The governance question I opened with

Apps are private by default. That is deliberate and, for personal automations connected to someone’s own mailbox or OneDrive, it is correct. Project Owners and Contributors see app name, owner, creation date and last modified. They cannot read workflow contents, connections or run history.

Now put that in an enterprise. Three consequences follow.

  • Auditability. Your compliance function cannot answer “what does this automation do and what does it touch” from the governance view. Someone has to be granted app scope access, per app, by the owner. That is a manual process with no obvious escalation path short of the Project Owner deleting the app.
  • Orphaned apps. When an owner leaves, existing collaborators keep access and only the Project Owner can delete the app. Nobody inherits the ability to read it. In a large organisation, that is a slow accumulation of automations nobody can review and nobody dares remove.
  • Shadow integration. The value proposition is that business teams build their own automations. The governance model means the platform team cannot see what they built. Both statements are true at once. That is not a bug, but it is a policy decision your organisation has to make deliberately rather than discover eighteen months in.

My working position: treat projects as the tenancy unit and map them to a business domain with a named owner and a named deputy. Require that anything touching regulated data lives in a project where the platform team holds app scope Contributor from day one. Write that down before the first workflow ships, not after.

Logic Apps Automation: What I would ask the product team for

Short list, in priority order, from someone who wants to put this in production.

  1. A deployment story. CI/CD and a full project definition as code. Without it, Automation is a place to build, not a place to run a regulated workload. This is the single blocker.
  2. An audit read role. A project scope role that can read workflow definitions and connection metadata across apps, without run history or payloads. Privacy by default and auditability are not in conflict if the role model is granular enough.
  3. Ownership transfer. Reassigning an owner should not require deleting the app.
  4. A published model support matrix for the Automation SKU, with the failure mode visible in the designer rather than at runtime.
  5. Clarity on VNet and private endpoints in preview, stated once, in one place, with the region list.
  6. Project level connector policy shipped early. Being able to block a connector class across a project is what lets a platform team say yes to self service.

Points 1 and 2 are the difference between an interesting preview and something an enterprise architecture board approves.

Logic Apps Automation: Where this is the wrong answer

Because it will not be the right answer everywhere, and pretending otherwise helps nobody.

  • You already run a mature Standard estate. Do not migrate. The engine is the same, the value is the experience, and you would trade a working CI/CD pipeline for one that does not exist yet.
  • SAP, EDI or B2B. Standard. Not close.
  • Predictable, sustained, high throughput load. Standard is more cost efficient and you keep capacity control.
  • Strict data residency or network isolation requirements that you cannot verify today. Wait for the VNet story to settle.
  • Your problem is deterministic. If the steps are known up front and you need exact repeatable behaviour, an agent adds latency, cost and a non deterministic failure mode in exchange for flexibility you do not need. Use a plain workflow.
  • You need conversational agents. Not in Automation yet. Consumption and Standard have that documented today.

What I would do on Monday

If you are a practitioner: get a project, build the triage scenario above or another scenario, and spend your time in the Chat tab and the retrieval view rather than the designer. The designer is pleasant and unsurprising. The observability is where the new value actually is.

If you are a decision maker: do not fund a migration. Fund one team, one project, one non regulated workflow, and a written answer to the ownership and audit question before the second team asks for access. The technology is further along than the operating model, and the operating model is the part you own.

Automation is in public preview. Treat it exactly like one, and it is the most interesting thing to happen to Logic Apps since Standard.

Sources

Knowledge Base as a Service in Azure Logic Apps

The Logic Apps Agent Loop series published here between May and June 2026 documented the agentic capabilities of Azure Logic Apps in depth. Notably, one recurring limitation across that series was the complexity of knowledge retrieval: building a proper Retrieval-Augmented Generation pipeline required a separately configured Azure AI Search index, an indexer, a data source, and a significant setup overhead before the agent could answer a question from enterprise content.

Consequently, the Knowledge Base-as-a-Service (KBaaS) capability in Azure Logic Apps, announced at Integrate 2026 and now in preview, addresses this directly. This post, as a follow-up to the previous post on all the announcements, walks through what KBaaS is, how it works, and how to build a practical HR policy agent that answers employee questions from uploaded policy documents, with a complete sample available on GitHub.

What is Knowledge Base as a Service?

KBaaS is a Logic Apps-native RAG pipeline that sits in front of Azure Cosmos DB and Azure OpenAI. Instead of building and managing the ingestion and retrieval infrastructure yourself, you upload documents and KBaaS handles the rest.

Specifically, the service has two pipelines:

  • Ingestion pipeline โ€” when you upload a document, KBaaS automatically parses, chunks, summarises, and vectorises the content using your Azure OpenAI embeddings model. The results are stored in four Cosmos DB containers that KBaaS creates and manages on your behalf.
  • Retrieval pipeline โ€” when an agent loop queries the knowledge base, KBaaS rewrites the query if needed, generates a vector representation, performs a semantic search against Cosmos DB, and returns the most relevant chunks to the language model for response generation.
Structural diagram showing the two KBaaS pipelines. The ingestion pipeline flows left to right: document upload, parse, chunk, summarise, embed, then into Cosmos DB across four containers. A dashed arrow connects the stored vectors in Cosmos DB down to the retrieval pipeline. The retrieval pipeline flows left to right: agent loop query, rewrite, vector search against Cosmos DB, top ranked chunks, LLM response. The bottom row shows the Logic Apps agent loop wrapper: HTTP trigger, knowledge base tool, Response action.
Figure 1 โ€” The two pipelines that power Knowledge Base as a Service in Azure Logic Apps. The ingestion pipeline (top) runs automatically when you upload a document: it parses and extracts text, chunks it into segments, summarises using Azure OpenAI, and embeds the content using an embeddings model before storing everything in four Cosmos DB containers. The retrieval pipeline (middle) runs when the agent loop queries the knowledge base: the query is rewritten if needed, vectorized, and matched against the stored embeddings in Cosmos DB via semantic search. The top-ranked chunks are passed to the LLM to generate a grounded answer. The agent loop wrapper (bottom) shows how these two pipelines sit inside a standard Logic Apps autonomous workflow.

In contrast, the key difference from a manually configured RAG pipeline is the abstraction layer. You do not configure chunking strategies, embedding dimensions, index schemas, or retrieval parameters. Instead, you upload documents and add the knowledge base as a tool. The agent loop does the rest.

Knowledge Base as a Service: Deploying the sample

The complete sample is available at steefjan1/logic-apps-kbaas-sample. It includes the workflow definition, Bicep infrastructure template, sample HR policy documents, and connection configuration.

Step 1: Provision infrastructure with azd

cd C:\Dev\logic-apps-kbaas-sample
azd auth login
azd init
azd env new <your-env-name>
azd provision

Naming note: avoid hyphens in your environment name. The Bicep template appends store to the prefix for the storage account name, and hyphens are not valid in storage account names. Use a short alphanumeric name such as kbaas rather than kbaas-preview.

Specifically, azd provision creates the resource group, a Logic App Standard, a Cosmos DB with vector search enabled, a storage account, an App Service Plan, and RBAC role assignments.

Step 2: Set app settings

$cosmosKey = az cosmosdb keys list `
--name <cosmos-name> `
--resource-group <resource-group> `
--query primaryMasterKey -o tsv
$storageKey = az storage account keys list `
--account-name <storage-name> `
--resource-group <resource-group> `
--query "[0].value" -o tsv
az logicapp config appsettings set `
--name <logic-app-name> `
--resource-group <resource-group> `
--settings `
AzureWebJobsStorage="DefaultEndpointsProtocol=https;AccountName=<storage-name>;AccountKey=$storageKey;EndpointSuffix=core.windows.net" `
FUNCTIONS_EXTENSION_VERSION="~4" `
FUNCTIONS_WORKER_RUNTIME="dotnet" `
APP_KIND="workflowapp" `
OPENAI__endpoint="https://<your-aoai-resource>.openai.azure.com/" `
OPENAI__key="<your-aoai-key>" `
agent_openAIKey="<your-aoai-key>" `
agent_openAIEndpoint="https://<your-aoai-resource>.openai.azure.com/" `
COSMOS__endpoint="https://<cosmos-name>.documents.azure.com:443/" `
COSMOS__key="$cosmosKey"

Step 3: Deploy the workflow

Compress-Archive `
-Path connections.json, host.json, hr-policy-agent `
-DestinationPath ..\kbaas-deploy.zip `
-Force
az logicapp deployment source config-zip `
--name <logic-app-name> `
--resource-group <resource-group> `
--subscription "<your-subscription>" `
--src ..\kbaas-deploy.zip
az logicapp restart `
--name <logic-app-name> `
--resource-group <resource-group>

Verify the workflow deployed cleanly go to the Logic App Overview โ†’ Notifications tab and confirm no WorkflowProcessingFailed errors appear before proceeding.

Step 4: Create the knowledge base connection

In the Azure portal, open your Logic App. Under Agents in the left sidebar, select Knowledge base then + Set up.

Basics tab:

  • Display name: hr-knowledge-base
  • Authentication type: Key-based
  • Database: select your Cosmos DB account โ€” URL endpoint and key auto-populate
  • Click Next

Model tab:

  • Authentication type: URL and key-based authentication
  • Azure OpenAI resource: select your Azure OpenAI resource
  • Completions model: enter your GPT-4o deployment name exactly as it appears in Azure OpenAI
  • Embeddings model: text-embedding-3-small
  • Click Create

Preview limitation: The knowledge base connection does not persist across page refreshes in the current preview โ€” this is a known portal UI bug. Do not refresh the page after clicking Create. Proceed immediately to adding files.

Note: Furthermore, after creating the connection, you can only edit display names. Authentication type, endpoint, and model deployment names cannot be changed. Get these right before clicking Create.

Step 5: Upload knowledge sources

Immediately after creating the connection, without refreshing, click New โ†’ Add files.

  • Group name: type hrpolicies
  • Upload hr-leave-policy.md from the sample’s docs/ folder
  • Artifact name: hr-leave-policy
  • Click Add

Once the status changes to Completed, then repeat for hr-expense-policy.md.

Preview limitation: The portal uploads one file at a time. Wait for each file to reach Completed before uploading the next.

Preview limitation: The Add button may remain greyed out if the artifact name field does not register keyboard input. If this happens, try dragging and dropping the file onto the upload area rather than using the file picker, then type the artifact name. If the button remains inactive, try a private/incognito browser window.

Azure portal Knowledge base page for the kbass-preview-la Standard logic app. The page shows the hrpolicies group containing two files: hr-leave-policy and hr-expense-policy, both showing Completed status after KBaaS ingestion processing.
Figure 2 โ€” Both HR policy documents successfully ingested into the knowledge base. The KBaaS ingestion pipeline parsed, chunked, summarised, and vectorised each document automatically no manual configuration of Cosmos DB containers, indexers, or embedding pipelines required. The Completed status confirms the content is ready for semantic retrieval by the agent loop.

KBaaS creates the following Cosmos DB containers during ingestion:

ContainerPurpose
KnowledgeHubsKnowledge base metadata
KnowledgeArtifactsSource metadata and document references
KnowledgeArtifactChunksFull-text document chunks
KnowledgeArtifactChunkSummariesSummarised chunks with vector embeddings

Step 6: Add the knowledge base as a tool

Open the hr-policy-agent workflow in the Logic Apps designer. Select the HR Policy Agent action and scroll to the Knowledge base section in the parameters pane. Select Create and choose hr-knowledge-base from the list. Save the workflow.

Testing the agent

Get the trigger URL from hr-policy-agent โ†’ Overview โ†’ Run trigger โ†’ copy the callback URL.

Test 1 โ€” grounded answer from the leave policy:

In postman execute: https://kbass-preview-la.azurewebsites.net:443/api/hr-policy-agent/triggers/When_an_HTTP_request_is_received/invoke?api-version=2022-05-01&sp=%2Ftriggers%2FWhen_an_HTTP_request_is_received%2Frun&sv=1.0&sig=<sig>

With payload (application\json): {“question”: “How many days of annual leave am I entitled to?”}

Postman request showing a POST to the hr-policy-agent trigger URL with body containing question: How many days of annual leave am I entitled to? The response shows 200 OK with a JSON body containing the question and a grounded answer citing the annual leave policy document.
Figure 3 โ€” The HR policy agent returns a grounded answer to a leave entitlement question. The agent queried the knowledge base, retrieved the relevant section from the annual leave policy document, and returned a cited response, all within a single autonomous agent loop run. The answer is grounded in the uploaded policy content rather than the model’s general training knowledge.

Test 2 โ€” grounded answer from the expense policy:

In postman execute: https://kbass-preview-la.azurewebsites.net:443/api/hr-policy-agent/triggers/When_an_HTTP_request_is_received/invoke?api-version=2022-05-01&sp=%2Ftriggers%2FWhen_an_HTTP_request_is_received%2Frun&sv=1.0&sig=<sig>

With payload: (application\json):{“question”: “What is the maximum I can claim per night for hotel accommodation?”}

Test 3 โ€” correct fallback when knowledge base has no answer:

In postman execute: https://kbass-preview-la.azurewebsites.net:443/api/hr-policy-agent/triggers/When_an_HTTP_request_is_received/invoke?api-version=2022-05-01&sp=%2Ftriggers%2FWhen_an_HTTP_request_is_received%2Frun&sv=1.0&sig=<sig>

With payload: {“question”: “What is the company policy on remote working?”}

As a result, the agent answers the first two questions from the uploaded documents and correctly declines the third exactly as the system prompt instructs.

Logic Apps run history for the hr-policy-agent workflow showing a successful run. The log panel shows the HTTP trigger, a user chat message, the agent action completing with tool invocations including the knowledge base retrieval tool, a sent chat message, and the Response action. The canvas shows the workflow steps with green success indicators.
Figure 4 โ€” The run history of the HR policy agent showing the full agent loop execution. The knowledge base tool invocation is visible as a discrete step within the agent iterations: the agent reasoned over the question, decided to query the knowledge base, received the relevant policy chunks, and composed a grounded answer before returning the Response action result. This is the same Think โ†’ Act โ†’ Observe loop covered in the Logic Apps Agent Loop series, now with KBaaS providing the retrieval step automatically.

Knowledge Base as a Service:: Practitioner notes

Several things in the current preview are worth documenting for anyone following along:

Connection persistence bug โ€” the knowledge base connection disappears on page refresh in the portal. The connection is correctly stored in connections.json on disk and the workflow runs correctly โ€” but the portal UI does not display it after refresh. Work around this by not refreshing after creating the connection and uploading files in the same browser session.

knowledgeHubConnections format โ€” the connections.json format for knowledge base connections is not publicly documented. If you include a knowledgeHubConnections block with incorrect structure in your deployed connections.json, the Logic App runtime will fail to start with The 'knowledgeHubConnections' property in connection.json has a value that cannot be parsed. The safest approach is to omit knowledgeHubConnections from your deployed connections.json entirely and let the portal write it after deployment.

Embeddings model region availability โ€” text-embedding-3-small with Standard SKU is not available in all Azure regions. Use GlobalStandard SKU as shown in the prerequisites command above.

Managed Identity for agent connections โ€” KBaaS supports Managed Identity authentication for the Cosmos DB connection. However, Managed Identity is not supported for the agent model connection when using the MicrosoftFoundry model type. If your workflows use Foundry Models, the agent connection must use Key authentication. Managed Identity remains the right choice for the Cosmos DB connection and all other backend services.

workflow.json schema โ€” the Agent action requires a limit property at the action level (not inside inputs) and a modelConfigurations block with a referenceName pointing to the agent connection. The Microsoft Learn documentation does not show these as required, but the runtime rejects the workflow without them.

Preview limitations

  • Only uploaded files are supported as knowledge sources โ€” live API connectors and SharePoint libraries are not yet available
  • Supported formats: DOC, DOCX, HTML, MD, PDF, PPT, PPTX, TXT, XLS, XLSX
  • Text-based content only โ€” images within documents are not parsed
  • Default chunking only โ€” custom chunk size and overlap configuration is not yet available
  • Azure portal only โ€” VS Code is not yet supported for knowledge base configuration
  • One file upload at a time โ€” wait for each file to reach Completed before uploading the next

GitHub sample

The complete sample workflow definition, Bicep infrastructure template, sample HR policy documents, and connection configuration are available on my GitHub.

What comes next

KBaaS is the most significant reduction in RAG pipeline complexity that Logic Apps has offered to date. For scenarios where uploaded documents are the primary knowledge source, it removes the Azure AI Search setup entirely and replaces it with a portal-native upload flow. The preview limitations, particularly the connection persistence bug and the lack of live source connectors, will be addressed in subsequent releases.

The natural next step is combining KBaaS with the multi-agent patterns from Part 5 of the Logic Apps Agent Loop series, an orchestrator agent that routes questions to specialist knowledge bases, each containing documents for a specific domain.

Azure Logic Apps at Integrate 2026: The Announcements

Integrate 2026 took place on June 8โ€“9 and brought the Microsoft integration product group together with the community for the first time since the platform’s agentic capabilities became generally available. For Azure Logic Apps, the announcements from Divya Swarnkar and Wagner Silveira’s session “What’s New and What’s Next in Azure Logic Apps” signal something more than a feature release cycle. They signal a platform repositioning.

Historically, Logic Apps has always occupied the integration and workflow orchestration layer of the Azure stack. Consequently, it is also firmly in the AI orchestration layer, connecting systems, knowledge, and intelligent agents in ways that were not possible twelve months ago. As a result, this post unpacks the five announcements that matter most for integration architects and connects them to the work covered in the Logic Apps Agent Loop series published here over the past two months.

Azure Logic Apps Integrate: The announcements

1. Azure Logic Apps Automation

The headline announcement is Logic Apps Automation, a new managed offering that sits alongside Logic Apps Standard and Consumption. It introduces a dedicated automation portal, AI-assisted workflow authoring, and a fully managed infrastructure model that removes the App Service Plan configuration and management required by Standard.

For integration architects in particular, this is significant in two ways. First, AI-assisted authoring lowers the barrier to building workflows; natural language descriptions of what a workflow should do can generate a starting point for the designer. Second, the fully managed model means organizations can adopt Logic Apps at scale without dedicated infrastructure expertise for every deployment.

In practice, Logic Apps Automation targets the enterprise automation use case: the high-volume, repeatable processes that currently live in RPA tools, home-grown scripts, or overly complex BPMN platforms. Importantly, it retains Logic Apps’ governance and security capabilities while making the platform accessible to a broader audience within the organization.

2. Knowledge as a Service

Microsoft announced Knowledge as a Service for Logic Apps, a capability that simplifies how organizations prepare enterprise data for AI-driven scenarios. Rather than building and maintaining complex data ingestion, chunking, embedding, and retrieval pipelines, teams can upload content, and Logic Apps handles the orchestration required to make that data available to AI agents.

Notably, this is directly relevant to the agentic workflows covered in this series. In Post 4, the agent tool layer relied on Azure AI Search as the retrieval mechanism, which required a separately configured search index, an indexer, a data source, and a skill set. Instead, Knowledge as a Service abstracts that complexity into a Logic Apps-native capability, reducing the setup time for a retrieval-augmented generation pattern from hours to minutes.

Significantly, the capability will be available across both Logic Apps Automation and Logic Apps Standard.

3. Azure AI Foundry Agent Integration

Logic Apps now supports invoking Azure AI Foundry Agents directly from workflows. Organisations can build, evaluate, and govern agents within Azure AI Foundry and use Logic Apps to orchestrate those agents as part of broader business processes.

Crucially, this closes a gap that the agent loop series ran into directly. Moreover, in Post 4, the attempt to call a Logic Apps workflow as an OpenAPI tool from Foundry hit network restrictions between the two platforms. As a result, the native Foundry Agent Integration announced at Integrate 2026 addresses this at the platform level; the connection between Logic Apps and Foundry is a first-class integration, not a custom OpenAPI workaround.

In practice, for multi-agent architectures, this means the orchestrator-worker pattern from Post 5 can now span both platforms: a Foundry agent as the orchestrator, Logic Apps autonomous workflows as the workers, with native connectivity between them rather than the SAS token-based HTTP invocation used in the demo.

4. Logic Apps Standard SDK

Microsoft introduced the Logic Apps Standard SDK, enabling workflows to be authored directly in C#. Developers gain access to familiar .NET tooling, type safety, NuGet packaging, and proper source control practices, while continuing to use the existing Logic Apps runtime and operational infrastructure.

Of all the announcements, this is the most relevant to the DevOps content in Post 7 of this series. The JSON-on-disk deployment model covered there remains valid, but the SDK adds a code-first authoring path that developer teams will strongly prefer for complex workflows. Type-safe workflow definitions, unit testability, and IDE integration (Visual Studio, VS Code) address the most common developer friction points with the current designer-first model.

For integration architects evaluating Logic Apps for new projects, the SDK changes the “who builds this” conversation. Workflows no longer need to be designer-authored by integration specialists; they can be written by developers using the tools they already know.

5. Azure Connector Namespace

Microsoft unveiled Azure Connector Namespace, which decouples Logic Apps connectors from Logic Apps workflows. The connector ecosystem, which includes over 1,400 connectors covering Microsoft and third-party services, can now be used from custom applications, Azure Functions, Container Apps, and AI agent platforms without the workflow runtime.

Of the five, this is architecturally the most significant for the longer term. Previously, the three-layer tool model covered in Post 4 (built-in connectors, custom connectors, MCP servers) assumed that connectors lived inside Logic Apps workflows. Azure Connector Namespace removes that constraint. Now, an Azure Function or a Foundry agent can now consume a Logic Apps connector directly, accessing Office 365, Service Bus, SAP, or any of the other 1,400+ services without a workflow in between.

Side-by-side diagram showing the architectural shift introduced by Azure Connector Namespace. Left side labelled Before shows connectors and agent tools locked inside a Logic Apps workflow boundary, with the workflow runtime required for all connector use, and a coral box at the bottom indicating connectors are locked to the Logic Apps runtime only. Right side labelled After shows a shared Azure Connector Namespace layer at the top containing Office 365, SAP, Service Bus, and 1,400 plus connectors, with four consumers below it connected by arrows: Logic Apps workflows, Azure Functions serverless compute, Container Apps custom apps, and AI agents including Foundry, MCP, and custom. A teal box at the bottom indicates connectors are decoupled from the workflow runtime and usable anywhere.
Figure 1 โ€” The architectural shift introduced by Azure Connector Namespace. Before the announcement (left), connectors were tightly coupled to the Logic Apps workflow runtime, accessible only from within a workflow, with the runtime always in the execution path. After (right), the connector ecosystem becomes a shared infrastructure layer. Logic Apps workflows, Azure Functions, Container Apps, and AI agents, whether running in Azure AI Foundry, via MCP, or as custom implementations, can all consume the same 1,400+ connectors independently of the workflow runtime.

For enterprise AI architectures, this means the connectivity layer and the orchestration layer are now separable. An AI agent can reach any enterprise system through the connector ecosystem without Logic Apps being the runtime that executes the connection.

Azure Logic Apps Integrate: The direction of travel

Taken together, the five announcements describe a platform moving in a consistent direction: Logic Apps is becoming the connectivity and orchestration substrate for enterprise AI, not just enterprise integration.

The diagram below maps the five announcements against the platform layers they affect: authoring, orchestration, knowledge, connectivity, and developer experience.

Structural diagram showing five Integrate 2026 Logic Apps announcements organised into three platform layers. Layer 1 authoring and developer experience contains Logic Apps Automation with AI-assisted authoring and fully managed infrastructure, and the Logic Apps Standard SDK with C# workflow authoring and .NET tooling. Layer 2 orchestration and AI contains Azure AI Foundry Agent Integration spanning the full width, enabling native invocation of Foundry agents from workflows without an OpenAPI workaround. Layer 3 knowledge and connectivity contains Knowledge as a Service with RAG pipeline abstraction, and Azure Connector Namespace giving access to 1,400 plus connectors without the workflow runtime. A series connection row at the bottom links the announcements to Posts 4, 5, and 7 of the Logic Apps Agent Loop series.
Figure 2 โ€” Five announcements from the Logic Apps product group session at Integrate 2026, mapped to the platform layers they affect. Layer 1 addresses the authoring and developer experience gap. Logic Apps Automation brings AI-assisted workflow creation and a fully managed infrastructure model, while the Standard SDK opens a code-first C# path for development teams. Layer 2 closes the orchestration gap between Logic Apps and Azure AI Foundry with a native agent integration that removes the OpenAPI workaround documented in Post 4 of this series. Layer 3 extends the platform’s reach: Knowledge as a Service abstracts RAG pipeline complexity, and Azure Connector Namespace decouples the 1,400+ connector ecosystem from the workflow runtime entirely.

The agent loop series documented the platform as it stood at the general availability of the agentic capabilities. Encouragingly, several of the limitations called out in that series the Foundry network restrictions, the complexity of knowledge retrieval setup, and the JSON-only authoring model are directly addressed by the Integrate 2026 announcements. That is a healthy sign: the platform team is hearing the practitioner feedback and moving quickly.

Azure Logic Apps Integrate: What this means for integration architects

Three practical implications for architects evaluating or already using Logic Apps:

  • First, revisit your hosting model decision. Logic Apps Automation changes the Standard-versus-Consumption decision for new projects. If the fully managed model meets your governance requirements, the App Service Plan overhead goes away.
  • Secondly, reconsider your knowledge retrieval approach. If you are building RAG patterns on Azure today using manually configured AI Search indexes, Knowledge as a Service is worth evaluating as a simpler path, particularly for projects where the data preparation pipeline is more complex than the agent itself.
  • Third, plan for SDK adoption. If your organization has strong .NET development capability, the Logic Apps Standard SDK should be on the evaluation list for any new workflow project. The designer-first model remains valid, but the code-first path will be preferred by development teams working in existing C# codebases.

Azure Logic Apps Integrate: Series connection

The Logic Apps Agent Loop series published here between May and June 2026 covered the agentic capabilities of Logic Apps in depth, from the anatomy of a single agent loop through to multi-agent patterns, security, and production operations. The Integrate 2026 announcements build directly on that foundation. Post 4’s MCP server pattern connects to the Azure Connector Namespace. Subsequently, Post 5’s orchestrator-worker pattern connects to the Foundry Agent Integration. Post 7’s DevOps section connects to the Standard SDK.

Citadel APIM Gateway Policies Azure: A Deep Dive

The Citadel APIM gateway policies on Azure are doing the heavy lifting silently in every post in this series. We deployed the hub, connected a tool-calling agent, added conversation persistence, and demonstrated the kill switch. Throughout all of that, every chat completion request passed through these policies. They counted every token. They produced all usage telemetry. This post opens the hood and examines all five Citadel APIM gateway policy layers in Azure token rate limiting, semantic caching, content safety routing, cost attribution, and PII redaction with the actual policy XML, live demonstration results, and honest debugging sessions included.

Where the Citadel APIM Gateway Policies Live in Azure

The Citadel hub deploys two types of policies.

API-level policies apply to all operations on the Azure OpenAI API for every chat completion, embedding, and batch request. These contain the token-limiting, usage-tracking, and routing logic.

Policy fragments serve as reusable blocks that you include by reference. The hub uses fragments for AAD authorization (aad-auth), load balancing (openai-backend-pool), and dynamic throttling (dynamic-throttling-assignment).

In the portal, find them at apim-wpvlimv4ngkns โ†’ APIs โ†’ Azure OpenAI Service API โ†’ All operations โ†’ Policies.

The full policy XML is also in the repo at github.com/Azure-Samples/ai-hub-gateway-solution-accelerator under infra/modules/apim/policies/.

Citadel APIM Gateway Policy 1 โ€” Token Rate Limiting

What It Does

APIM natively supports rate limiting on the number of requests per time window. The hub repurposes this to manage capacity based on tokens โ€” the azure-openai-token-limit policy limits how many tokens per minute a subscription can consume, using a counter keyed to the APIM subscription.

The Policy XML

Token rate limiting โ€” inbound section:

<azure-openai-token-limit
counter-key="@(context.Subscription.Id)"
tokens-per-minute="10000"
estimate-prompt-tokens="true"
tokens-consumed-variable-name="TotalConsumedTokens"
remaining-tokens-variable-name="RemainingTokens"
/>

Line by line:

  • counter-key=”@(context.Subscription.Id)” โ€” the counter is per APIM subscription, so each spoke gets its own independent token budget.
  • tokens-per-minute=”10000″ โ€” 10,000 TPM limit per subscription; adjust per spoke based on workload.
  • estimate-prompt-tokens=”true” โ€” APIM estimates prompt tokens before the response arrives, enabling proactive rate limiting rather than post-hoc tracking.
  • tokens-consumed-variable-name=”TotalConsumedTokens” โ€” stores the running count for use by downstream policies (cost attribution reads this variable).

Demonstration

The Citadel hub actually applies two independent token limit policies at different scopes, and tracing the real behaviour reveals an important lesson about how APIM evaluates them.

Scope 1 โ€” subscription-level, per deployment. The product policy on oai-retail-assistant assigns a different tokens-per-minute value based on the targeted deployment.

<when condition="@((string)context.Variables["target-deployment"] == "chat")">
<azure-openai-token-limit
counter-key="@(context.Subscription.Id + "-" + context.Variables["target-deployment"])"
tokens-per-minute="50"
token-quota="10000"
token-quota-period="Weekly"
tokens-consumed-header-name="consumed-tokens"
remaining-tokens-header-name="remaining-tokens"
retry-after-header-name="retry-after"
/>
</when>

Scope 2 โ€” product-level, across all deployments. A second, independent counter applies a higher ceiling across the entire product:

<azure-openai-token-limit
counter-key="@(context.Product?.Name?.ToString() ?? "Portal-Admin")"
tokens-per-minute="15000"
token-quota="150000"
token-quota-period="Monthly"
tokens-consumed-header-name="consumed-tokens"
remaining-tokens-header-name="remaining-tokens"
/>

Both policies execute on every request, each maintaining its own counter. Whichever limit you hit first governs the response, regardless of which one you configured.

Live test โ€” three rapid requests against the chat deployment with the subscription-level limit set to 50 TPM:

$response1 = Invoke-WebRequest -Uri $uri -Method POST -Headers $headers -Body $body
Write-Host "Status: $($response1.StatusCode)"
Write-Host "Remaining-Tokens: $($response1.Headers['remaining-tokens'])"

Results:

REQUEST 1 โ€” Status: 200, Remaining-Tokens: 14940, Consumed-Tokens: 15

REQUEST 2 โ€” Status: 429, Body: “Token limit is exceeded. Try again in 51 seconds.”

REQUEST 3 โ€” Status: 429, Body: “Token limit is exceeded. Try again in 51 seconds.”

Notice that Remaining-Tokens: 14940 on the first request comes from the product-level counter (15000 TPM, barely touched). The 429 on request 2, however, comes from the subscription-level counter for the chat deployment, which only allows 50 tokens per minute, exhausted after a single 15-token call. The visible remaining-tokens header reflects whichever counter last wrote to it, which can be misleading if you assume there is only one limit in play.

Pitfall: Multiple Token Limit Scopes Stack Independently

When multiple azure-openai-token-limit policy elements exist at the API level, product level, and per deployment in a choose block, they are all evaluated for each request. In addition, the most restrictive policy that triggers first decides the outcome. Furthermore, to debug rate limit issues, check all policy scopes: API-level, product-level, and deployment-specific branches. A Named Value in one scope doesn’t affect a hardcoded limit in another.

Citadel APIM Gateway Policy 2 โ€” Semantic Caching

What It Does

Semantic caching intercepts requests before they reach Azure OpenAI and checks for previously answered semantically similar questions. If it finds a match above the configured similarity threshold, it returns the cached response immediately, using zero tokens and ensuring near-zero latency.

The Policy XML

The lookup and store directives belong in different policy sections โ€” lookup runs on the inbound request, store runs on the outbound response after a successful call.

Semantic caching โ€” inbound section:

<azure-openai-semantic-cache-lookup
score-threshold="0.8"
embeddings-backend-id="openai-backend-0"
embeddings-backend-auth="system-assigned"
ignore-system-messages="true"
max-message-count="5"
/>

Semantic caching โ€” outbound section:

<azure-openai-semantic-cache-store duration="600" />

Line by line:

  • score-threshold=”0.8″ โ€” similarity must be 80% or higher for a cache hit. Lower values increase hit rate but risk returning mismatched responses.
  • embeddings-backend-id=”openai-backend-0″ โ€” points to the backend that hosts the embeddings deployment used for cache lookup. The embeddings model itself is configured on the backend, not as a policy attribute โ€” embeddings-model is not a valid attribute on this policy element and will fail schema validation if added.
  • embeddings-backend-auth=”system-assigned” โ€” the embeddings backend call is authenticated via the APIM instance’s system-assigned managed identity.
  • ignore-system-messages=”true” โ€” only user messages are used for cache key generation, not system prompts.
  • max-message-count=”5″ โ€” only the last 5 messages in a conversation are used for cache lookup.
  • duration=”600″ โ€” cached responses expire after 10 minutes.

Demonstration

First request โ€” cache miss:

python agent.py
# Question: What is the weather like in Stockholm right now?
# Response time: 1.2 seconds

Second request โ€” cache hit:

python agent.py
# Question: What is the weather in Stockholm today?
# Response time: 45ms

The second question resembles the first semantically, but it’s not an exact match. The 0.8 threshold identifies it as a cache hit, allowing the system to return the response from the cache in 45ms without making an Azure OpenAI call or using any tokens. For a conversational agent that often encounters similar questions, semantic caching can lower token consumption by 20โ€“40%.

Production Consideration

The cache shares its data across all subscribers to the same APIM product. If two spokes utilize the same APIM gateway, Spoke A’s cached response can serve Spoke B for a similar inquiry. To protect sensitive data, configure the cache key to be scoped per subscription by adding @(context.Subscription.Id).

Pitfall: Policy Section Placement and Schema Validation

When adding semantic caching for the first time, you may encounter two common schema errors. First, placing azure-openai-semantic-cache-store in the inbound section, along with the lookup policy, results in the error: “Policy is not allowed in this section.” Second, youโ€™ll find that embeddings-model is not a declared attribute on azure-openai-semantic-cache-lookup. Instead of being specified directly in the policy, the embeddings deployment is retrieved from the backend referenced by embeddings-backend-id. You can quickly identify both errors when you save the policy in the portal, which provides the fastest validation of your XML before deployment.

Citadel APIM Gateway Policy 3 โ€” Content Safety Routing

What It Does

The hub routes every request through Azure AI Content Safety before it reaches Azure OpenAI. It inspects both the user prompt and the model response for harmful content in four categories: hate, self-harm, sexual, and violence. The system blocks any requests or responses that exceed the configured severity thresholds.

The Policy XML

Content safety โ€” inbound section:

<llm-content-safety backend-id="content-safety-backend" shield-prompt="true">
<categories output-type="FourSeverityLevels">
<category name="Hate" threshold="2" />
<category name="SelfHarm" threshold="2" />
<category name="Sexual" threshold="2" />
<category name="Violence" threshold="2" />
</categories>
</llm-content-safety>

Line by line:

  • backend-id=”content-safety-backend” โ€” routes to the cog-consafety-wpvlimv4ngkns Azure AI Content Safety instance deployed in the hub.
  • shield-prompt=”true” โ€” enables Prompt Shields, which additionally detects jailbreak attempts and prompt injection on top of standard content categories.
  • categories output-type=”FourSeverityLevels” โ€” selects the four-level severity scale (0, 2, 4, 6) rather than the eight-level scale; each category child element sets its own threshold independently.
  • category name=”…” threshold=”2″ โ€” one element per category (Hate, SelfHarm, Sexual, Violence), each can have a different threshold. A threshold of 2 blocks low severity and above; omitting a category leaves it unchecked.

You can reference custom Azure AI Content Safety blocklist IDs in an optional blocklists element to always block organization-specific terms, regardless of their severity scoring.

Pitfall: Element Name and Schema

The element is llm-content-safety, not azure-content-safety โ€” using the wrong name produces “There is no policy matching element name” on save. The schema is also structural rather than attribute-based: categories are child category elements inside a categories block, not a flat comma-separated categories=”…” attribute. Both errors surface immediately in the portal policy editor, which validates the XML before allowing a save.

Demonstration

Normal request โ€” content safety passes silently:

python agent.py
# Question: What is the weather like in Stockholm?
# Answer: The weather in Stockholm is overcast...

The Policy

The llm-content-safety policy does not add a confirmation header on a passing request โ€” the absence of a block response is the only signal that the prompt and response cleared all four category thresholds. To confirm the policy actually ran, check Application Insights:

requests
| where timestamp > ago(10m)
| where resultCode == "200"
| project timestamp, name, resultCode, duration
| order by timestamp desc

A 200 with normal duration confirms the request passed through llm-content-safety without being blocked.

Harmful content

Blocked request โ€” harmful content detected:

# Modify agent.py to send a harmful prompt, then run:
python agent.py
# openai.BadRequestError: Error code: 400
# 'Request failed content safety check.'

Requests block at the gateway before they reach Azure OpenAI, ensuring that these blocked requests consume no tokens. In Application Insights, you observe the block as a 400 response with near-zero duration, reflecting the same signature pattern seen with the kill switch layers mentioned in the previous post.

Citadel APIM Gateway Policy Policy 4 โ€” Cost Attribution

What It Does

Every request that completes successfully generates a usage event sent to Azure Event Hub. A Logic App deployed in the hub consumes these events and writes structured documents to the Cosmos DB ai-usage-container. Each document contains token counts, model version, gateway region, APIM subscription name, and timestamp, giving you per-subscription cost attribution without any agent-side code changes.

The Policy XML

The hub deploys three named loggers, visible via az rest against the APIM management API: appinsights-logger for Application Insights telemetry, usage-eventhub-logger for the cost attribution pipeline described here, and a separate pii-usage-eventhub-logger for PII-specific event logging tied to the redaction policy described later in this post. The logger-id attribute on log-to-eventhub must match one of these exactly โ€” using a placeholder or guessed name produces “Logger not found” when saving the policy.

Cost attribution โ€” outbound section:

<log-to-eventhub logger-id="usage-eventhub-logger" partition-id="0">
@{
var response = context.Response;
var request = context.Request;
var usage = response.Body.As<JObject>(true)?["usage"];
return new JObject(
new JProperty("id", context.Response.Headers.GetValueOrDefault("x-request-id", Guid.NewGuid().ToString())),
new JProperty("timestamp", DateTime.UtcNow.ToString("o")),
new JProperty("subscriptionId", context.Subscription?.Id),
new JProperty("productName", context.Product?.Name),
new JProperty("targetService", "chat.completion"),
new JProperty("model", response.Body.As<JObject>(true)?["model"]?.ToString()),
new JProperty("gatewayName", context.Deployment.ServiceName),
new JProperty("gatewayRegion", context.Deployment.Region),
new JProperty("RequestIp", request.IpAddress),
new JProperty("promptTokens", usage?["prompt_tokens"]?.ToObject<int>() ?? 0),
new JProperty("responseTokens", usage?["completion_tokens"]?.ToObject<int>() ?? 0),
new JProperty("totalTokens", usage?["total_tokens"]?.ToObject<int>() ?? 0),
new JProperty("backendId", context.Variables.GetValueOrDefault<string>("backendId")),
new JProperty("deploymentName", context.Request.MatchedParameters["deployment-id"])
).ToString();
}
</log-to-eventhub>

Key fields:

  • context.Subscription?.Id and context.Product?.Name โ€” the APIM subscription and product name used for the request. In a multi-spoke setup, each spoke has its own APIM product making cost attribution per initiative automatic.
  • usage?[“prompt_tokens”] / usage?[“completion_tokens”] โ€” extracted directly from the Azure OpenAI response body.
  • context.Deployment.Region โ€” the gateway region (Sweden Central) for data residency auditing.
  • context.Variables.GetValueOrDefault<string>(“backendId”) โ€” which Azure OpenAI backend served the request, critical for multi-region deployments.

Demonstration

Run the agent:

python agent.py

Then query the hub Cosmos DB in the portal (cosmos-wpvlimv4ngkns โ†’ Data Explorer โ†’ ai-usage-container โ†’ Items):

SELECT TOP 5 c.timestamp, c.productName, c.model,
c.promptTokens, c.responseTokens, c.totalTokens,
c.gatewayRegion, c.deploymentName
FROM c ORDER BY c._ts DESC

A real document from the deployed hub looks like this:

{
"timestamp": "6/30/2026 11:26:49 AM",
"productName": "OAI-HR-Assistant",
"model": "gpt-4o-2024-11-20",
"promptTokens": 93,
"responseTokens": 56,
"totalTokens": 149,
"gatewayRegion": "Sweden Central",
"deploymentName": "chat"
}

Each agent run produces two documents, one for the tool decision call and one for the synthesis call. The totalTokens across both documents is the true per-conversation cost. The productName field maps directly to the APIM product the calling subscription belongs to, in this example OAI-HR-Assistant, distinct from any other product sharing the same hub.

FinOps in Practice

In production with multiple spokes, filter by productName to get per-initiative cost:

SELECT c.productName,
SUM(c.totalTokens) as totalTokens,
COUNT(1) as requestCount
FROM c
WHERE c.timestamp >= "2026-06-01T00:00:00Z"
GROUP BY c.productName

Run against the live hub, this returns a clean per-product summary:

[
{
"productName": "Portal-Admin",
"totalTokens": 3865,
"requestCount": 32
},
{
"productName": "OAI-HR-Assistant",
"totalTokens": 3035,
"requestCount": 31
}
]

Two distinct products, each with an independently aggregated token total and request count, no additional instrumentation required beyond the log-to-eventhub policy already in place. This is your direct FinOps input per AI initiative, per month, ready to feed into a PowerBI report or a monthly cost allocation process.

Citadel APIM Gateway Policy 5 โ€” PII Redaction

What It Does

The hub detects personally identifiable information โ€” names, email addresses, phone numbers, IBAN numbers, and other entity types โ€” in conversation content and logs a redacted version to a dedicated Event Hub logger. The original request still reaches Azure OpenAI unmodified; only the logged version is clean.

There Is No Built-In Policy Element for This

The blog draft for this post originally referenced an azure-openai-pii-removal-logging policy element, on the assumption that APIM ships a built-in PII redaction policy the same way it ships azure-openai-token-limit and azure-openai-semantic-cache-lookup. It does not. Saving a policy referencing that element name fails immediately with “There is no policy matching element name,” the same class of error encountered earlier with azure-content-safety versus the correct llm-content-safety.

Unlike content safety, however, there is no equivalent built-in alternative for PII detection at the time of writing. The hub’s pii-usage-eventhub-logger, visible alongside usage-eventhub-logger and appinsights-logger when listing loggers via the APIM management API, exists as infrastructure for this purpose, but the policy that populates it has to be built explicitly using send-request to call Azure AI Language Service directly, then log-to-eventhub to write the result.

The Policy XML

This belongs entirely in outbound, immediately after the cost attribution log-to-eventhub block. PII detection runs on the response side rather than inbound because the goal is to log a redacted record of the conversation, not to block or alter the request itself, context.Request.Body remains accessible in the outbound pipeline, so the original user message can still be analysed at this stage.

PII detection โ€” outbound section, after cost attribution:

<send-request mode="new" response-variable-name="piiDetectionResponse" timeout="10" ignore-error="true">
<set-url>@("https://" + "{{language-service-name}}" + ".cognitiveservices.azure.com/language/:analyze-text?api-version=2023-04-01")</set-url>
<set-method>POST</set-method>
<set-header name="Ocp-Apim-Subscription-Key" exists-action="override">
<value>{{language-service-key}}</value>
</set-header>
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-body>@{
var requestBody = context.Request.Body.As<JObject>(true);
var userMessage = requestBody["messages"]?.Last?["content"]?.ToString() ?? "";
return new JObject(
new JProperty("kind", "PiiEntityRecognition"),
new JProperty("analysisInput", new JObject(
new JProperty("documents", new JArray(
new JObject(
new JProperty("id", "1"),
new JProperty("language", "en"),
new JProperty("text", userMessage)
)
))
))
).ToString();
}</set-body>
</send-request>
<log-to-eventhub logger-id="pii-usage-eventhub-logger" partition-id="0">
@{
var piiResult = ((IResponse)context.Variables["piiDetectionResponse"]).Body.As<JObject>(true);
var redactedText = piiResult?["results"]?["documents"]?[0]?["redactedText"]?.ToString() ?? "[pii-detection-unavailable]";
var entityCount = piiResult?["results"]?["documents"]?[0]?["entities"]?.Count() ?? 0;
return new JObject(
new JProperty("timestamp", DateTime.UtcNow.ToString("o")),
new JProperty("subscriptionId", context.Subscription?.Id),
new JProperty("productName", context.Product?.Name),
new JProperty("redactedText", redactedText),
new JProperty("piiEntitiesFound", entityCount)
).ToString();
}
</log-to-eventhub>

Line by line:

  • send-request mode=”new” โ€” fires an independent outbound call to Azure AI Language Service rather than reusing the current request/response context. ignore-error=”true” means a Language Service outage does not fail the agent’s actual response, only the PII logging step is skipped.
  • requestBody[“messages”]?.Last?[“content”] โ€” extracts the most recent user message from the original request body for analysis, since that is where PII is most likely to appear.
  • The Language Service PiiEntityRecognition endpoint returns both a redactedText field (PII replaced with asterisks) and an entities array listing every detected entity and its category.
  • log-to-eventhub logger-id=”pii-usage-eventhub-logger” โ€” writes the redacted text and entity count to the dedicated PII logger, kept separate from the general usage logger so PII-related audit records can be access-controlled independently.

Two Named Values must exist before this policy validates:

az apim nv create \
--resource-group rg-ai-hub-gateway-dev \
--service-name apim-wpvlimv4ngkns \
--named-value-id language-service-name \
--display-name "language-service-name" \
--value "cog-language-wpvlimv4ngkns" \
--secret false
az apim nv create \
--resource-group rg-ai-hub-gateway-dev \
--service-name apim-wpvlimv4ngkns \
--named-value-id language-service-key \
--display-name "language-service-key" \
--value "<your-language-service-key>" \
--secret true

Retrieve the Language Service key:

az cognitiveservices account keys list \
--name cog-language-wpvlimv4ngkns \
--resource-group rg-ai-hub-gateway-dev \
--query key1 -o tsv

Demonstration

Request containing PII:

question = "What is the weather near Jan Janssen who lives at Keizersgracht 123 Amsterdam?"

First attempt, the policy ran but logged the wrong data. The send-request call to Azure AI Language Service fired correctly and returned 200 every time, confirmed via Application Insights dependency tracking:

dependencies
| where timestamp > ago(10m)
| where target contains "cognitiveservices"
| project timestamp, name, target, resultCode, duration
POST /language/:analyze-text โ†’ 200 โ†’ 68.62ms
POST /language/:analyze-text โ†’ 200 โ†’ 48.26ms
POST /language/:analyze-text โ†’ 200 โ†’ 106.96ms

Yet the documents landing in pii-usage-container showed no redactedText field at all, only promptTokens, gatewayName, backendId, and the rest of the cost attribution schema:

{
"id": "30db8c4b-ce45-4695-9078-e357405845bc",
"subscriptionId": "oai-hr-assistant-sub-01",
"productName": "OAI-HR-Assistant",
"model": "gpt-4o-2024-11-20",
"promptTokens": 109,
"responseTokens": 21,
"gatewayName": "apim-wpvlimv4ngkns.azure-api.net",
"backendId": "openai-backend-0"
}

The cause turned out to be a copy-paste error in the log-to-eventhub block itself. The comment above it correctly read “PII detection and redacted logging,” but the JObject construction inside was a verbatim duplicate of the cost attribution payload from usage-eventhub-logger, it never referenced context.Variables[“piiDetectionResponse”] at all. The Language Service call succeeded and its result sat in a context variable, completely unused, while the logger faithfully wrote the wrong document every time. Every dependency call returned 200; every Cosmos DB write succeeded; nothing in the telemetry indicated a problem. The only way to catch it was reading the document schema in Cosmos DB and noticing it matched the usage container rather than containing redacted text.

The fix was correcting the log-to-eventhub body to actually read the stored piiDetectionResponse variable:

<log-to-eventhub logger-id="pii-usage-eventhub-logger" partition-id="0">@{
var piiResult = ((IResponse)context.Variables["piiDetectionResponse"]).Body.As<JObject>(true);
var redactedText = piiResult?["results"]?["documents"]?[0]?["redactedText"]?.ToString() ?? "[pii-detection-unavailable]";
var entityCount = piiResult?["results"]?["documents"]?[0]?["entities"]?.Count() ?? 0;
return new JObject(
new JProperty("id", context.Response.Headers.GetValueOrDefault("x-request-id", Guid.NewGuid().ToString())),
new JProperty("timestamp", DateTime.UtcNow.ToString("o")),
new JProperty("subscriptionId", context.Subscription?.Id),
new JProperty("productName", context.Product?.Name),
new JProperty("redactedText", redactedText),
new JProperty("piiEntitiesFound", entityCount)
).ToString();
}</log-to-eventhub>

After the fix, the same question produced the correct redacted record:

{
"id": "419321ce-0544-4908-a2ac-9ce10f80aaba",
"timestamp": "2026-06-30T13:12:24.8989851Z",
"subscriptionId": "oai-hr-assistant-sub-01",
"productName": "OAI-HR-Assistant",
"redactedText": "What is the weather near *********** who lives at ***************************?",
"piiEntitiesFound": 2
}

Two entities detected and redacted, the person’s name and the street address, while the sentence structure remains intact for log readability. A second request, where the agent’s tool call itself failed to resolve the address, still produced a correctly redacted record of the error message:

{
"redactedText": "{\"error\": \"Location '***************************' not found.\"}",
"piiEntitiesFound": 1
}

This confirms PII redaction applies consistently regardless of whether the underlying tool call succeeds, the policy operates on the original user message, independent of how the agent’s downstream logic handles it.

In Cosmos DB usage documents, unaffected: the cost attribution document written by usage-eventhub-logger still contains only token counts and metadata, never request body content, so PII redaction applies exclusively to the dedicated pii-usage-container stream.

Pitfall: A Logger Existing Does Not Mean It Logs the Right Thing

The most instructive failure in this section was not a missing policy element or a schema validation error, it was a policy that validated, deployed, and executed successfully while silently logging the wrong payload. Every signal that normally indicates “this is working” was green: the send-request dependency call returned 200, the log-to-eventhub write succeeded, and documents appeared in Cosmos DB on schedule. The only way to catch the bug was to inspect the actual field names in the logged document and notice they matched a different policy’s output entirely. When wiring up custom logging policies, always verify the logged document shape directly in the data store, a successful HTTP status code on a dependency call says nothing about whether its result was ever used downstream.

Pitfall: Two send-request Round Trips Add Latency

Every request now makes an additional outbound call to Azure AI Language Service before the agent’s response is returned to the caller, since send-request blocks until it completes (or times out at 10 seconds with ignore-error=”true”). For latency-sensitive workloads, consider moving PII detection to an asynchronous pattern, log raw request IDs to Event Hub immediately, then run PII detection as a separate downstream process reading from the Event Hub stream, rather than inline in the request path.

The Complete Policy Execution Order

Understanding the order in which policies execute is critical for troubleshooting. APIM processes policies in this sequence.

Inbound, request processing, top to bottom:

  1. Kill switch check (Named Value flip)
  2. Agent approval header validation
  3. Agent ID blocklist check
  4. AAD authorization (aad-auth fragment)
  5. Token rate limiting (azure-openai-token-limit)
  6. Semantic cache lookup (azure-openai-semantic-cache-lookup)
  7. Content safety check (inbound prompt, llm-content-safety)
  8. Backend routing (load balancer)

Outbound, response processing, top to bottom:

  1. Usage event capture (cost attribution fragment)
  2. Semantic cache store (azure-openai-semantic-cache-store)
  3. PII detection request (send-request to Language Service)
  4. Cost attribution logging (log-to-eventhub, usage-eventhub-logger)
  5. PII redaction logging (log-to-eventhub, pii-usage-eventhub-logger)
  6. Response header enrichment

On-error:

  1. Error response formatting
  2. Retry logic (if configured)

This order means a request blocked by the kill switch never reaches token rate limiting, content safety, or Azure OpenAI. A request blocked by content safety never reaches cost attribution or PII logging, no usage document and no redacted-text document are created for blocked requests.

Validating All Five Policies in Application Insights

Use this KQL query to see the policy execution evidence in one view:

requests
| where timestamp > ago(24h)
| extend
cacheHit = tostring(customDimensions["x-cache"]),
tokensConsumed = toint(customDimensions["TotalConsumedTokens"]),
remainingTokens = toint(customDimensions["RemainingTokens"])
| project timestamp, resultCode, duration, cacheHit, tokensConsumed, remainingTokens
| order by timestamp desc

Each column corresponds to a policy layer.

  • resultCode โ€” 200 (all policies passed, including content safety), 400 (content safety block), 429 (token rate limit), 403/401 (kill switch).
  • cacheHit โ€” hit or miss from semantic cache.
  • tokensConsumed โ€” running token count from rate limiter.
  • remainingTokens โ€” remaining budget for this subscription.

There is no dedicated content safety column because llm-content-safety does not emit a custom telemetry property, its outcome is entirely reflected in resultCode. A 400 with near-zero duration is the signature of a content safety block, the same pattern used to identify kill switch activations in the previous post.

For PII detection specifically, dependency tracking shows whether the Language Service call fired:

dependencies
| where timestamp > ago(24h)
| where target contains "cognitiveservices" and name contains "analyze-text"
| project timestamp, name, target, resultCode, duration
| order by timestamp desc

A 200 here only confirms the call succeeded, it does not confirm the result was logged correctly downstream. Cross-check against the actual documents in pii-usage-container to verify the redactedText field is present and populated, not just that the dependency call returned successfully.

Pitfalls Summary

Token rate limiting doesn’t work for streaming. Fix: use non-streaming endpoints for accurate token counting.

Semantic cache returns stale weather data. Fix: reduce duration to 60 seconds for time-sensitive tool calls.

Content safety blocks valid medical terminology. Fix: raise threshold to 4 for healthcare-specific deployments and validate against your use case.

azure-content-safety element does not exist. Fix: use llm-content-safety with categories and category child elements, not flat attributes.

azure-openai-pii-removal-logging element does not exist. Fix: no built-in policy exists, implement via send-request to Language Service plus a custom log-to-eventhub.

The logger successfully writes but records incorrect data. To fix this, a 200 status on the send-request and a successful write to Event Hub do not guarantee that the result was actually used. Instead, verify the logged document schema directly in Cosmos DB.

The system misses redacting Dutch-language names. To fix this issue, optimize the analyze-text request by setting the language to “nl” and testing it with representative Dutch inputs, instead of using “en,” which is optimized for English.

Cost attribution missing for some requests. Fix: check Event Hub to Logic App pipeline; ingestion lag of 2 to 5 minutes is normal.

To ensure high compliance in production, always surface failures in content safety and PII detection instead of silently bypassing them with the on-error-action or ignore-error=”true” settings.

Conclusion

The five APIM policies in the Citadel hub, token rate limiting, semantic caching, content safety, cost attribution, and PII redaction, collectively implement enterprise AI governance at the gateway layer. None of them require changes to agent code. None of them require spoke involvement. All of them apply to every request from every agent that routes through the hub, regardless of which spoke deployed it.

Furthermore, they compose cleanly: a request that hits the semantic cache never reaches token rate limiting, content safety, or Azure OpenAI. As a result, cached responses consume zero tokens, zero content safety budget, and zero Azure OpenAI quota. Similarly, a request blocked by content safety produces no cost attribution event and no PII redaction record.

This composability is what makes the Citadel APIM hub a governance layer rather than just a proxy. The policies work together, in a defined order, to enforce the enterprise AI control plane pattern across every governed workload, though, as the PII redaction section makes clear, composability and correct execution are not the same thing. A policy chain can validate, deploy, and run green across every dependency call while still logging the wrong data. The only reliable verification is checking the actual data that lands in the store, not the status codes along the way.

Azure Functions AI Integration: The Quiet Powerhouse

When people talk about running AI workloads on Azure, the conversation usually lands on Azure AI Foundry, Azure OpenAI Service, or maybe Azure Container Apps. Azure Functions tends to get mentioned as the glue the thing you bolt on to handle a webhook. But this overlooks the real story: Azure Functions AI integration has quietly evolved from simple glue into a powerhouse for running production-grade AI.

That framing is outdated. Azure Functions is now a first-class runtime for AI workloads, with four distinct patterns that the Microsoft Learn documentation lays out explicitly. This post walks through each of them and helps you decide which one fits your situation.

The four AI-enabled scenarios

Microsoft groups Azure Functions AI integration into four scenarios:

  1. Serverless agents runtime โ€” event-driven agents that run on serverless infrastructure
  2. Tools and MCP servers โ€” hosting remote Model Context Protocol servers and AI tools
  3. Agentic workflows โ€” multistep, long-running directed agent operations via Durable Functions
  4. Retrieval-augmented generation (RAG) โ€” fast, parallel data retrieval for knowledge-augmented AI
Structural diagram showing the Azure Functions platform envelope containing four patterns in a two-by-two grid โ€” serverless agents runtime, MCP servers and tools, agentic workflows via Durable Functions, and RAG pipeline โ€” with Foundry Agent Service, Azure OpenAI, APIM, and AI Search shown below as connected surrounding services.
The four AI-enabled patterns inside the Azure Functions platform and the surrounding Azure services they integrate with.

These are not just marketing buckets. Each one reflects a different architectural decision. Let me unpack them.

Azure Functions AI integration: Serverless agents runtime

The serverless agents runtime is a preview programming model for building event-driven agents as function apps. Moreover, Agents are defined in .agent.md files, app-wide runtime defaults live in agents.config.yaml, and remote MCP server connections are listed in mcp.json. The runtime discovers these files, registers the required triggers and endpoints, and runs the agent through the Microsoft Agent Framework when an event fires.

That is a meaningfully different model from what you get in Azure AI Foundry Agent Service. In Foundry, the managed service hosts and orchestrates your agents. In addition, in the serverless agents runtime, your function app is the agent host running on Flex Consumption, with built-in managed identity, monitoring, and scale-to-zero. Furthermore, you write custom Python tools for app-specific logic, and the platform wires in MCP-enabled connections based on Azure connectors and remote MCP servers.

Use this when: you want agents triggered by events, schedules, messages, or HTTP requests and you need the familiar Functions deployment and hosting model rather than a managed agent service.

Avoid it when: you need a fully managed, enterprise-grade agent service with built-in tooling and long-term Microsoft support guarantees today. That is what Foundry Agent Service is built for.

Azure Functions AI integration: Tools and MCP servers

The Model Context Protocol (MCP) has become the industry standard for how AI models and agents interact with external systems. Azure Functions has first-class support for hosting remote MCP servers, and this is already generally available.

There are two hosting options:

OptionStatusHow it works
MCP binding extensionGAUses Functions triggers and bindings; supports stateful execution
Self-hosted MCP servers (MCP SDK)PreviewUses standard MCP SDKs via custom handlers; requires Streamable HTTP transport
Flowchart showing an AI agent at the top fanning out via L-shaped connectors to three columns: MCP binding extension (GA, synchronous, stateful, all major languages), self-hosted MCP SDK server (preview, Streamable HTTP, no stateful execution yet), and queue-based tool (async, decoupled, fault-tolerant delivery via Service Bus or Storage Queue). Each column lands on a function app box at the bottom.
An AI agent reaches Azure Functions via three distinct paths, each with different trade-offs on availability, transport, and execution model.

The binding extension is the right default. It supports C#, Python, TypeScript, JavaScript, and Java, and it integrates with the Functions programming model you already know. Self-hosted MCP servers offer portability: you can use the official MCP SDKs and bring in existing server code. However, stateful execution is not yet supported, and the configuration is still changing during preview.

There is also a third option worth knowing about: queue-based Azure Functions tools, where AI agents interact with your code through message queues rather than direct MCP calls. Microsoft Foundry provides specific Azure Functions tooling for this pattern. It is ideal when you need reliable delivery, built-in retry, and decoupling between agent and function execution.

Use MCP servers when: you are exposing tools to AI clients and you want the industry-standard protocol with serverless hosting.

Use queue-based tools when: you need asynchronous, fault-tolerant communication between an agent and your function code.

Agentic workflows with Durable Functions

Not all AI orchestration should be autonomous. Some scenarios need predictable, directed steps and that is where Durable Functions fits.

The Microsoft Learn documentation makes the distinction clearly: Durable Functions is positioned as the runtime for directed agentic workflows, not for emergent agent reasoning. Think of it this way: when you know the sequence of steps and you need fault tolerance, auditability, and long-running execution, Durable Functions is the right tool. When you want a model to figure out the steps dynamically, you want an agent runtime.

The documentation gives a clean example: a trip planning workflow that gathers user requirements, searches for options, waits for approval, and makes bookings. Each step is a function; Durable Functions coordinates them with built-in retry, state persistence, and human-in-the-loop support.

Use this when: your AI-driven process has well-defined, ordered steps,authorization flows, multi-stage approval chains, or orchestrated data pipelines where you cannot afford unpredictable execution paths.

Avoid it when: you want a model to determine steps dynamically. That is the serverless agents runtime or Foundry Agent Service territory.

RAG with Azure Functions

Because Functions handles multiple events from various data sources simultaneously, it scales well for real-time AI scenarios, particularly RAG systems where fast, parallel retrieval is the bottleneck.

The Azure OpenAI binding extension lets you integrate RAG directly into your function code. Functions can pull data from multiple sources simultaneously, feed it through Azure AI Search or other retrieval layers, and pass the results to your language model, all within the event-driven, scale-to-zero model that keeps costs down when load is low.

The Azure Functions RAG pattern also pairs naturally with APIM, which handles routing, rate limiting, and token quota management a pattern the Citadel Platform series covers in detail, including the discovery that the Foundry Agent Service SDK bypasses APIM for LLM calls.

Use this when you have event-driven retrieval requirements new documents arriving in blob storage, database change feeds, or streaming IoT data that needs to inform model responses.

How the scenarios relate to other Azure services

It helps to think of Azure Functions AI integration as filling the compute and integration layer between your AI services and your data sources. Here is roughly how that maps:

  • Azure AI Foundry Agent Service โ€” fully managed agent orchestration with enterprise security and built-in tools. Functions integrates into Foundry via MCP servers and queue-based tools.
  • Azure Logic Apps โ€” low-code orchestration for business process automation. Functions is the right choice when you need custom code, complex event processing, or lower latency.
  • Azure Container Apps โ€” container-based hosting for long-running services. Functions on Flex Consumption beats it on cost for bursty, event-driven AI workloads that spend time idle.
  • Durable Functions โ€” lives inside Functions and adds stateful, long-running orchestration. Use it for directed agentic workflows; use the serverless agents runtime for event-driven agents.
Four-tier layered diagram. Bottom tier: data sources and messaging (Blob Storage, Cosmos DB, Event Hubs, Service Bus, Azure SQL, IoT Hub). Second tier: Azure Functions compute and integration layer containing four pattern pills โ€” agents runtime, MCP servers, Durable workflows, and RAG pipeline. Third tier: managed AI services (Azure OpenAI, AI Search, APIM, Microsoft Agent Framework). Top tier: AI clients and agents (Foundry Agent Service, Semantic Kernel, custom SDK agents, chat apps). Arrows show data flowing upward and tool calls flowing downward.
Azure Functions fills the compute and integration layer between your data sources and your managed AI services.

The underlying platform advantage

Across all four scenarios, the same hosting model applies: Flex Consumption. It offers fast, event-driven scaling, virtual network integration, and pay-as-you-go billing. For AI workloads, which tend to be bursty rather than continuous, this is a significant cost advantage over always-on hosting.

Managed identity, Application Insights integration, and azd-based deployment are consistent across all four patterns. That means your security posture, observability, and deployment pipeline do not have to change when you move from a simple timer trigger to hosting a remote MCP server.

Azure Functions AI integration: Choosing the right pattern

Here is a simple decision table:

I want toโ€ฆUseโ€ฆ
Build event- or schedule-triggered agents with MCP toolsServerless agents runtime (preview)
Expose tools to AI clients via the industry-standard protocolMCP binding extension (GA)
Orchestrate predictable, multistep AI-driven processesDurable Functions
Build a RAG pipeline with fast, parallel data retrievalAzure Functions + Azure OpenAI binding
Fully managed agent hosting with enterprise SLAsAzure AI Foundry Agent Service

What comes next

The rest of this series goes deep on each pattern. The next post covers the two MCP server hosting options in detail binding extension versus self-hosted SDK servers, including where the current preview constraints matter in practice.

Up next: Hosting Remote MCP Servers in Azure Functions: GA vs. Preview Options

Azure Messaging and Orchestration for Integration Architects

In the Azure PaaS map post, the integration layer got a single paragraph. It named four services: Logic Apps, API Management, Service Bus, and Event Grid, and moved on. This post takes Azure messaging and orchestration apart into the decisions underneath that paragraph.

I won’t tour features here. Two of these services already have their own deep series on this blog, so re-covering them would waste your time. Instead, I’ll stay at the decision layer. When do you reach for which? And why do teams so often reach wrong? Those are the questions that actually cost you in production.

Azure messaging and orchestration: two axes decide almost everything

Four services sound like four choices. In practice, though, only two questions matter, and they cut across the whole layer.

Decision diagram with two axes. The first splits messaging from orchestration. For messaging, the cost of a lost message chooses between Service Bus and Event Grid. For orchestration, complexity and ownership choose between Logic Apps and code. A band across the bottom shows API Management as the control point in front of all of it.
You don’t choose between four services; you answer two questions: messaging or orchestration, then failure cost or ownership. The service falls out from there, with API Management governing the front.
  • First: is this messaging or orchestration? Messaging moves events and data between systems. Orchestration coordinates a multi-step process toward an outcome. The two look similar on a whiteboard, but they fail differently, scale differently, and belong to different services. So separate them before anything else.
  • Second: what does failure cost, and what shape is the work? Once you know whether you’re moving messages or coordinating steps, the follow-up question splits the choice further. For messaging, the cost of a lost message decides it. For orchestration, the complexity of the flow and the team who owns it decide it.

Get those two axes clear, and the service almost picks itself. Skip them, and you end up with Event Grid where you needed guarantees, or a Logic App doing work that belonged in code.

Messaging: Service Bus vs Event Grid

Both move things between systems. That’s where the similarity ends.

  • Service Bus is the durable, ordered, transactional backbone. Reach for it when delivery has to be guaranteed. It gives you sessions for ordered processing, dead-lettering for messages that can’t be handled, and transactional handling across multiple operations. Topics and subscriptions add pub/sub without a separate broker. So Service Bus fits business messages: an order, a payment, a claim, where losing one is an incident.
  • Event Grid is a lightweight, high-volume router. It broadcasts events to whoever cares: resource state changes, custom application events, and telemetry. It’s built for throughput and fire-and-forget delivery, not guaranteed processing. Therefore, it fits notifications and reactive triggers, where a missed event is a shrug rather than a page.

Here’s the rule of thumb I give teams new to Azure messaging. If losing a message would be a business incident, it belongs on Service Bus. If losing it would just mean a missed notification, Event Grid is fine.

And often you use both. A common pattern pairs them: Event Grid fans out a notification, and a subscriber drops a durable message onto Service Bus for guaranteed processing. That way you get Event Grid’s reach and Service Bus’s reliability in one flow, each doing the job it’s good at.

Pattern diagram showing a source emitting an event to Event Grid, which fans out to a notification subscriber, a logging subscriber, and a bridge subscriber. The bridge subscriber drops a durable message onto Service Bus, where a processor handles it with guaranteed ordered delivery and dead-lettering.
Event Grid fans an event out to multiple subscribers; the one that needs a guarantee drops a durable message onto Service Bus for ordered, dead-lettered processing.

The honest note, though, is that this is where teams get burned. Event Grid looks simpler, so teams default to it. Then, weeks later, they discover the workload actually needed ordering or delivery guarantees. Now they’re bolting reliability onto a service that was never designed for it. So decide on the message-loss cost first, before the “which feels easier” instinct takes over.

Orchestration: Logic Apps vs code

Messaging moves things. Orchestration coordinates them. The decision here isn’t about reliability; it’s about complexity and ownership.

  • Logic Apps is the designer-first route. It shines when you need enterprise connectors SAP, IBM MQ, mainframe hosts, the long tail of line-of-business systems without a modern REST API. Standard Logic Apps also closes the old gaps that made it hard in regulated environments: VNet integration, built-in state, per-workflow scaling. So for a workflow that a less code-heavy team will own and maintain, Logic Apps is often the right call even when a Function would be more elegant.
  • Code is the route once complexity climbs. Designer workflows are fast to build and easy to read at first. Past a certain size, though, they get hard to reason about and harder to code-review. In my experience, the practical ceiling sits around a dozen actions with a couple of branches. Beyond that, do one of two things. Either decompose the workflow into smaller ones, or move the logic into a Function where a proper language and real tests take over.

The deciding questions, then, are simple. Who maintains this: a low-code team or engineers? How complex is the flow really? And can you review it a year from now? For the deeper mechanics of building agentic workflows in Logic Apps, I covered that ground in the Logic Apps Agent Loop series so that I won’t repeat it here.

Where API Management fits

Azure API Management (APIM) isn’t messaging or orchestration. Instead, it’s the control point in front of both.

It sits between consumers and whatever does the real work: a Logic App, a Function, an App Service backend. From there, it enforces rate limits, authentication, transformation, and policy-based routing. So when multiple consumers hit a shared set of backend capabilities, APIM lets you change the implementation behind them without breaking anyone, and lets you enforce policy without touching application code.

That’s all I’ll say here, because APIM earns a series of its own. I went deep on it in the APIM for AI workloads series, including how it behaves as an AI gateway. For this layer, treat it as the front door that governs whatever messaging and orchestration sit behind it.

Where each is the wrong answer

Every service here has a failure mode when you reach for it by reflex. So, to keep this honest:

  • Service Bus is wrong for high-volume telemetry: If you’re routing millions of fire-and-forget events and none of them individually matter, Service Bus is expensive overkill. Use Event Grid.
  • Event Grid is wrong for anything needing order: The moment sequence or guaranteed delivery matters, Event Grid stops fitting. Move to Service Bus before the gap bites.
  • Logic Apps is wrong past its complexity ceiling: A workflow with thirty actions and nested branches is a maintenance liability in the designer. Decompose it, or move it to code.
  • Code is wrong for something a citizen developer should own: Not every integration belongs in a repo. If a low-code team can own and maintain a simple connector-driven flow, hand-writing it in a Function just centralizes work that didn’t need to be centralized.

The shape of it

Azure gives you four integration services, but you don’t choose between four things. You answer two questions. Is this messaging or orchestration? And then what does failure cost, and who owns the work? Answer those, and Service Bus, Event Grid, Logic Apps, or a Function each falls out naturally, with APIM governing the front.

In practice, real platforms use several together: APIM, fronting Logic Apps, and Functions, Event Grid fanning out to Service Bus for reliable processing. So the craft of Azure messaging and orchestration isn’t picking a winner. It’s drawing clean boundaries between them.

Want the layer above this one? The Azure PaaS map puts integration in context against compute, data, and governance, and walks the five-question framework for choosing across all of them.

Citadel APIM Kill Switch: Stop a Governed Agent Cold

In the previous post we added conversation persistence to the Microsoft Foundry Citadel Platform on Azure. As a result, every agent run now produces a structured document in the spoke’s Cosmos DB conversations container. The agent is fully operational: it routes through the APIM governance hub, executes tool calls, stores its history, and returns grounded responses. However, the question that every enterprise AI architect eventually faces remains: what happens when it needs to stop?

Not a graceful shutdown. Not a redeployment. An immediate, operator-triggered containment the kind you need when an agent is behaving unexpectedly, consuming runaway tokens, or has been flagged by your security team. In a Microsoft Foundry Citadel Platform on Azure deployment, the answer is the Kill Switch: a layered containment system built into the APIM hub that stops agent traffic cold without touching the agent code, the spoke, or the Azure OpenAI deployment.

This post implements three of the five Citadel kill switch layers against the hub we deployed in Sweden Central:

  • Layer 1 โ€” Named Value flip: instant global block via a single boolean
  • Layer 2 โ€” JWT claim block: identity-based containment via header validation
  • Layer 3 โ€” Agent ID blocklist: surgical per-agent blocking

The Scenario

The weather agent (agent_with_memory.py) is running in production. Specifically, it is routing through apim-wpvlimv4ngkns.azure-api.net, storing conversations in the spoke Cosmos DB, and generating token usage events in the hub Cosmos DB. Everything is working. Then your security team flags it. The agent needs to stop immediately while the incident is investigated. You have seconds, not minutes. For example, redeploying the spoke takes too long. Rotating the APIM subscription key is irreversible and affects all consumers. Therefore, the Kill Switch is the right tool.

The Kill Switch is the right tool. The APIM hub has built-in pre-wiring, requires no code changes, and can trigger actions in under 30 seconds.

To ensure reliability, always pre-wire the kill switch as Layer 1 before you need it. Remember, you canโ€™t flip a Named Value that doesnโ€™t exist. In addition, the inbound policy must already be in place, checking the Named Value on every request, before any incident occurs.

Prerequisites

From the previous posts you should have:

  • Hub deployed in rg-ai-hub-gateway-dev with APIM instance apim-wpvlimv4ngkns
  • agent_with_memory.py running and saving to Cosmos DB
  • Azure CLI authenticated

Citadel Kill Switch Layer 1 โ€” Named Value Flip

How It Works

A Named Value called kill-switch-enabled is created in APIM and set to false. An inbound policy on the OpenAI API checks this value on every request. When the value is flipped to true, all requests through the gateway immediately return HTTP 403 โ€” no code changes, no redeployment, no spoke involvement.

Step 1.1 โ€” Create the Named Value

az apim nv create `
--resource-group rg-ai-hub-gateway-dev `
--service-name apim-wpvlimv4ngkns `
--named-value-id kill-switch-enabled `
--display-name "kill-switch-enabled" `
--value "false" `
--secret false

Verify it was created:

az apim nv show `
--resource-group rg-ai-hub-gateway-dev `
--service-name apim-wpvlimv4ngkns `
--named-value-id kill-switch-enabled `
--query "value" -o tsv

Should return false.

Step 1.2 โ€” Add the Inbound Policy

In the Azure Portal:

  1. Navigate to apim-wpvlimv4ngkns โ†’ APIs โ†’ Azure OpenAI Service API โ†’ All operations
  2. Click Policies โ†’ Inbound processing โ†’ Edit
  3. Add this policy inside the <inbound> section, before any other policies:
<!-- Kill Switch Layer 1: Named Value flip -->
<set-variable name="killSwitchActive" value="@("{{kill-switch-enabled}}" == "true")" />
<choose>
<when condition="@((bool)context.Variables["killSwitchActive"])">
<return-response>
<set-status code="403" reason="Agent Suspended" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-header name="x-kill-switch-layer" exists-action="override">
<value>1-named-value</value>
</set-header>
<set-body>{"error": {"code": "KillSwitchActive", "message": "Agent access has been suspended by the governance hub.", "layer": 1}}</set-body>
</return-response>
</when>
</choose>

The dedicated Named Value check policy provides a cleaner approach.

<!-- Kill Switch Layer 1: Named Value flip -->
<!-- Pre-wire this BEFORE any incident. Set kill-switch-enabled=true to activate. -->
<set-variable name="killSwitchActive" value="@("{{kill-switch-enabled}}" == "true")" />
<choose>
<when condition="@((bool)context.Variables["killSwitchActive"])">
<return-response>
<set-status code="403" reason="Agent Suspended" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-header name="x-kill-switch-layer" exists-action="override">
<value>1-named-value</value>
</set-header>
<set-body>{"error": {"code": "KillSwitchActive", "message": "Agent access has been suspended by the governance hub. Contact your administrator.", "layer": 1}}</set-body>
</return-response>
</when>
</choose>

Click Save.

Step 1.3 โ€” Confirm Agent Runs Normally

With kill-switch-enabled set to false, the agent should still work:

python agent_with_memory.py

Expected output: normal run, conversation saved, answer returned.

Step 1.4 โ€” Trigger the Kill Switch

az apim nv update `
--resource-group rg-ai-hub-gateway-dev `
--service-name apim-wpvlimv4ngkns `
--named-value-id kill-switch-enabled `
--value "true"

Now run the agent:

python agent_with_memory.py

Expected output:

The agent stops. No spoke changes occur. No code changes happen. One CLI command executes.

Step 1.5 โ€” Reset

az apim nv update `
--resource-group rg-ai-hub-gateway-dev `
--service-name apim-wpvlimv4ngkns `
--named-value-id kill-switch-enabled `
--value "false"

Citadel Kill Switch Layer 2 โ€” Agent Approval Header

How It Works

The agent is required to pass a custom header x-agent-token containing a signed JWT with a specific claim (agt-approved: true). The APIM inbound policy validates this claim. If the claim is absent or the token is invalid, the system blocks the request with a 401 status. This action simulates identity-based containment, revoking the agent’s token or invalidating its claim at the identity provider level.

Step 2.1 โ€” Update the Agent to Send a Header

Add the x-agent-token header to agent_with_memory.py. In this demo, we simulate the token by using a simple header value. IIn a production environment, Entra ID issues a JWT.

Modify the AzureOpenAI client creation in agent_with_memory.py:

client = AzureOpenAI(
azure_endpoint=apim_base,
api_key=cfg["APIM_SUBSCRIPTION_KEY"],
api_version="2024-02-01",
default_headers={
"x-agent-id": "citadel-weather-agent-v1"
}
)

Step 2.2 โ€” Add the JWT Claim Check Policy

In the portal, add this policy after the Layer 1 block in the inbound section:

<!-- Kill Switch Layer 2: Agent approval header check -->
<choose>
<when condition="@(context.Request.Headers.GetValueOrDefault("x-agent-approved", "false") != "true")">
<return-response>
<set-status code="401" reason="Agent Not Approved" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-header name="x-kill-switch-layer" exists-action="override">
<value>2-agent-approval</value>
</set-header>
<set-body>{"error": {"code": "AgentNotApproved", "message": "Agent identity could not be verified. Approval header missing or invalid.", "layer": 2}}</set-body>
</return-response>
</when>
</choose>

Step 2.3 โ€” Trigger Layer 2

Remove the x-agent-approved header from the agent (or set it to false) and run:

python agent_with_memory.py

Expected output:

Note the response header x-kill-switch-layer: 2-agent-approval this indicates which containment layer fired and is critical for incident triage.

Pitfall: Policy Order Matters

Layer 1 must appear before Layer 2 in the policy document. APIM evaluates inbound policies top to bottom and stops at the first <return-response>. If Layer 2 appears before Layer 1, a globally suspended agent would return a 401 (identity error) instead of a 403 (suspended), obscuring the true containment reason in your incident log.

Citadel Kill Switch Layer 3 โ€” Agent ID Blocklist in APIM

How It Works

A Named Value called blocked-agent-ids holds a comma-separated list of agent IDs. The inbound policy checks the x-agent-id header against this list. When agents match, the system blocks them with a 403 status code. Non-matching agents continue operating normally. This approach allows for surgical containment, stopping one specific agent while allowing all others to function.

Step 3.1 โ€” Create the Blocklist Named Value

az apim nv create `
--resource-group rg-ai-hub-gateway-dev `
--service-name apim-wpvlimv4ngkns `
--named-value-id blocked-agent-ids `
--display-name "blocked-agent-ids" `
--value "none" `
--secret false

Start with an empty value โ€” no agents blocked.

Step 3.2 โ€” Add the Blocklist Policy

Add this policy after Layer 2 in the inbound section:

<!-- Kill Switch Layer 3: Agent ID blocklist -->
<set-variable name="agentId" value="@(context.Request.Headers.GetValueOrDefault("x-agent-id", ""))" />
<set-variable name="blockedIds" value="@("{{blocked-agent-ids}}")" />
<choose>
<when condition="@{
var agentId = (string)context.Variables["agentId"];
var blockedIds = (string)context.Variables["blockedIds"];
if (string.IsNullOrEmpty(agentId) || string.IsNullOrEmpty(blockedIds)) { return false; }
return blockedIds.Split(',').Any(id => id.Trim() == agentId.Trim());
}">
<return-response>
<set-status code="403" reason="Agent Blocked" />
<set-header name="Content-Type" exists-action="override">
<value>application/json</value>
</set-header>
<set-header name="x-kill-switch-layer" exists-action="override">
<value>3-agent-blocklist</value>
</set-header>
<set-body>{"error": {"code": "AgentBlocked", "message": "Agent has been added to the governance blocklist.", "layer": 3}}</set-body>
</return-response>
</when>
</choose>

Step 3.3 โ€” Add the Agent to the Blocklist

az apim nv update `
--resource-group rg-ai-hub-gateway-dev `
--service-name apim-wpvlimv4ngkns `
--named-value-id blocked-agent-ids `
--value "citadel-weather-agent-v1"

Run the agent:

python agent_with_memory.py

Expected output:

Step 3.4 โ€” Surgical Validation

The power of Layer 3 is specificity. If you had a second agent with a different x-agent-id say citadel-docs-agent-v1 it would pass through Layer 3 unaffected while citadel-weather-agent-v1 remains blocked. One agent stopped, all others running. This is the enterprise AI governance pattern: granular control without broad disruption.

Remove the agent from the blocklist:

az apim nv update --resource-group rg-ai-hub-gateway-dev
--service-name apim-wpvlimv4ngkns --named-value-id blocked-agent-ids
--value "none"

Validating the Citadel Kill Switch in Application Insights

Rather than using the CLI โ€” which has a 5โ€“10 minute Log Analytics ingestion lag โ€” go directly to Application Insights in the portal for immediate results:

  1. Portal โ†’ appi-apim-wpvlimv4ngkns in rg-ai-hub-gateway-dev
  2. Left sidebar โ†’ Logs
  3. Paste and run this query:
requests
| where timestamp > ago(2h)
| where resultCode in ("200", "401", "403")
| project timestamp, resultCode, duration, name
| order by timestamp desc

The results table tells the complete kill switch story in two columns โ€” resultCode and duration:

Azure Application Insights Logs results table showing POST /openai/deployments/chat/chat/completions requests โ€” two 401 responses at 41ms and 0.9ms from Layer 2 kill switch activation, and four 200 responses at 683ms to 2010ms from normal governed agent runs through the Microsoft Foundry Citadel APIM hub in Sweden Central.
Application Insights Logs query on the Citadel APIM hub showing the kill switch in action, 401 responses at under 1ms confirm Layer 2 (agent approval header) blocking requests at the gateway before any LLM call is made, contrasted with normal 200 responses taking 683msโ€“2010ms for a full Azure OpenAI round trip.

The duration contrast is the definitive proof that the kill switch works as designed. The 401s and 403s resolve in under 50ms, stopped cold at the APIM inbound policy before a single token is sent to Azure OpenAI. The 200s take 683msโ€“2010ms because they made the full round trip through the governance hub to Azure OpenAI and back.

Zero tokens consumed on blocked requests, zero cost, and zero Cosmos DB writes in the spoke. The agent is stopped at the perimeter.

For a sharper view that highlights exactly which kill switch layer fired on each blocked request, add the response header to the query. Unfortunately APIM response headers are not automatically projected into the requests table in Application Insights โ€” but you can distinguish the layers by combining result code and timing:

requests
| where timestamp > ago(2h)
| where resultCode in ("200", "401", "403")
| extend killSwitchLayer = case(
resultCode == "401", "Layer 2 โ€” agent approval",
resultCode == "403" and duration < 10, "Layer 1 or 3 โ€” gateway block",
resultCode == "200", "Normal โ€” LLM call completed",
"Unknown"
)
| project timestamp, resultCode, duration, killSwitchLayer
| order by timestamp desc
Azure Application Insights Logs results table for the Citadel APIM hub showing six requests โ€” two 401 responses labeled Layer 2 agent approval at 41ms and 0.9ms duration, and four 200 responses labeled Normal LLM call completed at 683ms to 2010ms duration, confirming the Microsoft Foundry Citadel kill switch blocks requests at the gateway before any Azure OpenAI call is made.
Application Insights Logs query on the Citadel APIM hub showing the kill switch incident log โ€” Layer 2 agent approval blocks resolving in under 1ms with zero LLM calls made, contrasted with normal governed runs completing in 683msโ€“2010ms. The killSwitchLayer column identifies exactly which containment layer fired on each request.

This gives you a readable incident log showing which containment layer was active at each point in time, directly useful for DORA incident post-mortem documentation and EU AI Act Article 17 risk management records.

The Complete Three-Layer Kill Switch Policy

Here is the complete inbound policy block containing all three layers, ready to paste into APIM:

 <!-- Kill Switch Layer 1: Named Value flip -->
        <set-variable name="killSwitchActive" value="@("{{kill-switch-enabled}}" == "true")" />
        <choose>
            <when condition="@((bool)context.Variables["killSwitchActive"])">
                <return-response>
                    <set-status code="403" reason="Agent Suspended" />
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/json</value>
                    </set-header>
                    <set-header name="x-kill-switch-layer" exists-action="override">
                        <value>1-named-value</value>
                    </set-header>
                    <set-body>{"error": {"code": "KillSwitchActive", "message": "Agent access has been suspended by the governance hub.", "layer": 1}}</set-body>
                </return-response>
            </when>
        </choose>
        <!-- Kill Switch Layer 2: Agent approval header -->
        <choose>
            <when condition="@(context.Request.Headers.GetValueOrDefault("x-agent-approved", "false") != "true")">
                <return-response>
                    <set-status code="401" reason="Agent Not Approved" />
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/json</value>
                    </set-header>
                    <set-header name="x-kill-switch-layer" exists-action="override">
                        <value>2-agent-approval</value>
                    </set-header>
                    <set-body>{"error": {"code": "AgentNotApproved", "message": "Agent identity could not be verified. Approval header missing or invalid.", "layer": 2}}</set-body>
                </return-response>
            </when>
        </choose>
        <!-- Kill Switch Layer 3: Agent ID blocklist -->
        <set-variable name="agentId" value="@(context.Request.Headers.GetValueOrDefault("x-agent-id", ""))" />
        <set-variable name="blockedIds" value="@("{{blocked-agent-ids}}")" />
        <choose>
            <when condition="@{
        var agentId = (string)context.Variables["agentId"];
        var blockedIds = (string)context.Variables["blockedIds"];
        if (string.IsNullOrEmpty(agentId) || string.IsNullOrEmpty(blockedIds)) { return false; }
        return blockedIds.Split(',').Any(id => id.Trim() == agentId.Trim());
    }">
                <return-response>
                    <set-status code="403" reason="Agent Blocked" />
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/json</value>
                    </set-header>
                    <set-header name="x-kill-switch-layer" exists-action="override">
                        <value>3-agent-blocklist</value>
                    </set-header>
                    <set-body>{"error": {"code": "AgentBlocked", "message": "Agent has been added to the governance blocklist.", "layer": 3}}</set-body>
                </return-response>
            </when>
        </choose>

Pitfalls Summary

PitfallFix
Named Value doesn’t exist at incident timePre-wire Layer 1 during normal operations โ€” never during an incident
Policy evaluation error on {{kill-switch-enabled}}Named Value must exist before the policy referencing it is saved
Layer 2 fires before Layer 1 in policyPolicy order matters โ€” Layer 1 must be first in the inbound block
Agent ID header not sentAdd x-agent-id to default_headers in AzureOpenAI client
Blocklist with trailing spaces blocks nothingUse .Trim() in the policy C# expression when splitting
Kill switch left active after testAlways reset Named Values after testing โ€” kill-switch-enabled=false, blocked-agent-ids=""

What the Kill Switch Demonstrates About Citadel

The three layers reveal something important about the Citadel architecture: governance lives in the hub, not the agent. The agent code has no knowledge of the kill switch. The spoke has no kill switch configuration. The Azure OpenAI deployment is untouched. All containment logic is in the APIM hub’s inbound policy โ€” one place, centrally managed, instantly effective.

This is the enterprise AI control plane pattern in practice. When an incident occurs:

  • Layer 1 stops everything immediately while you triage
  • Layer 2 enforces identity verification once normal operations resume
  • Layer 3 surgically targets the offending agent while other agents continue

The x-kill-switch-layer response header ensures your incident log captures exactly which containment mechanism fired, giving you a clean audit trail for post-mortem analysis โ€” directly relevant for DORA incident reporting and EU AI Act Article 17 risk management documentation.

What’s Next

The next post in this series takes the dev setup and hardens it for non-prod: networkIsolation=true, APIM Premium SKU, per-spoke subscription keys with independent quotas, and Azure Policy at the management group level. The kill switch policies we built here carry forward unchanged governance in the hub environment, which is environment-agnostic.

Azure App Service Architecture: A Deeper Look for Integration Architects

In the Azure PaaS map post, App Service got just one paragraph. I called it the default for synchronous REST APIs that front a backend system. That summary is right, but Azure App Service architecture hides far more than one paragraph can carry.

When you stand up an App Service in a regulated enterprise, the interesting decisions sit around the app, not inside it. How does traffic reach it? And how does it authenticate outbound? Furthermore, how does it scale? And how do you ship changes without downtime? So this post takes a deeper look. I’ll walk the request path from the user to the data tier and flag the decisions that matter specifically for integration work.

Comprehensive diagram of Azure App Service architecture. Users reach the app through Azure DNS, Front Door with edge WAF, and Application Gateway with regional WAF. The App Service Plan runs multiple instances across availability zones with built-in features and application components. Surrounding tiers show CI/CD with deployment slots, security and identity, data services, networking, and monitoring.
The full picture request path across the top, the App Service Plan at the center, and the surrounding tiers of CI/CD, security, data, networking, and monitoring that make an integration platform work.

Azure App Service architecture: the request path, front to back

Before a request touches your code, it passes through a chain of services. Moreover, each one is a design decision rather than a default.

Linear diagram of the App Service request path: user to Azure DNS to Front Door with edge WAF to Application Gateway with regional WAF to App Service to the private data tier.
Every hop from user to data tier is a design decision, not a default, and for a single-region regulated platform, Application Gateway alone often does the job.
  • First, DNS resolves the URL. Azure DNS points the hostname at whatever sits in front of App Service. That step is trivial, but it’s worth naming, because the next hop depends on it.
  • Next, a global entry point handles routing and Web Application Firewall (WAF). Here the first real choice appears. Azure Front Door gives you global routing, CDN-style caching, and a Web Application Firewall at the edge. Therefore, you’d pick it when you serve a distributed audience or want TLS termination close to the user. Application Gateway, by contrast, performs Layer 7 load balancing and WAF regionally within your VNet. So you’d reach for it when traffic stays regional, and you want the firewall inside your network boundary. Plenty of designs use both Front Door globally, Application Gateway behind it. For a single-region platform in a regulated environment, though, Application Gateway alone often does the job. Better still, it keeps everything inside the VNet where your security team wants it.
  • Finally, App Service receives the request. By now, the traffic has been routed, load-balanced, and WAF-filtered. What App Service adds is a managed platform that runs your code, patches the underlying OS, and handles TLS for you.

Azure App Service architecture: Inside the App Service Plan

The App Service Plan catches people out. After all, this is where the platform allocates and bills are computed, not the app itself. Multiple apps can share one plan, so they also share its CPU and memory. That arrangement saves money until two apps contend under load. Then the “why is my API slow when the other app gets busy” investigation begins.

Diagram of an App Service Plan showing load balancing across instances spread over two availability zones, with a highlighted note that multiple apps sharing the same plan contend for CPU and memory under load. A legend explains scale-out, scale-up, and metric-based autoscale triggers.
The plan is the billed compute unit. It load-balances instances across availability zones, and because multiple apps can share one plan, they also contend for its CPU and memory under load.

The plan defines a few things that matter architecturally:

  • Instances and scaling: The plan runs one or more instances, and App Service load-balances across them. Scale-out adds instances, which drives throughput. Scale-up swaps in bigger instances, which adds per-request headroom. Autoscale rules fire on metrics like CPU, memory, and HTTP queue length. That’s exactly why App Service suits steadily loaded request workloads rather than bursty, event-driven ones. So if your load is spiky and event-driven, consider Functions or Container Apps instead.
  • Availability zones: On tiers that support it, you can spread plan instances across zones. As a result, “highly available” ceases to be a claim and becomes an actual design property. For regulated production, treat this as table stakes rather than an upgrade.
  • Isolation: The isolated tiers run your plan in a dedicated environment inside your VNet, away from shared infrastructure. Therefore, you’d reach for them when compliance demands network isolation that shared tiers can’t provide a common requirement in health and finance.
Current image: Azure App Service architecture diagram with users, networking, compute, storage, management, and DevOps components.

The platform features integration architects actually use

App Service ships built-in capabilities that often do more work than the application code. Three of them earn their keep in every integration design:

Sequence diagram of a deployment slot swap: deploy the new version to a staging slot, warm up and validate against production configuration, then swap staging and production instantly, with a dashed swap-back path shown for rolling back on failure.
Deploy and validate a new version in staging against production config, then swap it in instantly with an equally instant swap-back if something breaks.
  • Deployment slots are the single most useful feature for shipping without downtime. A staging slot lets you deploy, warm up, and validate a new version against production configuration. Then you swap it into production instantly, and swap back just as fast if something breaks. For a platform where a bad deploy takes down downstream consumers, that swap turns a potential incident into a controlled release.
  • Managed identity is the one I’d insist on. App Service can carry a system-assigned or user-assigned identity. Consequently, it authenticates to Key Vault, SQL, Service Bus, and Storage without a single connection string in the configuration. My first post made the same point about the governance layer. App Service is where you implement it for the compute tier.
  • VNet integration and private endpoints close the network. VNet integration lets the app call into your private network. Private endpoints let consumers reach the app privately, without a public address. In a regulated environment, you’ll usually want both. That way, the app communicates with backends over private links, and consumers access it through the gateway rather than a public URL.

The tiers around it: data, identity, observability

App Service never runs alone. In fact, the architecture around it is where an integration platform lives or dies:

  • Data services repeat the choices from the first post. Pick Azure SQL for relational integrity, Cosmos DB for flexible scaling, Blob Storage for files, and Redis Cache to absorb read load. App Service connects to all of them over private endpoints and authenticates through managed identity.
  • Security and identity means Entra ID for authentication, Key Vault for the secrets that can’t be an identity, and managed identities threading through everything. App Service’s built-in “Easy Auth” can offload the whole OIDC flow to the platform. That helps for internal APIs. Still, understand it before you lean on it for anything with complex authorization logic.
  • Monitoring and observability mean Application Insights and Azure Monitor. For an integration platform, this isn’t optional. When a request fails somewhere across the gateway, app, backend, and data tier, distributed tracing shows you where. So wire it in on day one, not after the first production incident.

Where App Service is the wrong answer

Let me keep this honest: the same point I make in every post. App Service isn’t always right, and reaching for it by reflex causes as many problems as it solves.

Does your workload run event-driven and bursty? Then App Service’s metric-based autoscale will lag the load or leave you overprovisioned. Functions or Container Apps fit better. Do you need long-running orchestration? App Service will host it, but you’re building a workflow engine on top of a request-serving platform. Logic Apps or Durable Functions exist for exactly that. Do you have real Kubernetes-native requirements? App Service won’t stretch that far, so that’s an AKS conversation.

App Service shines for steadily-loaded, request-driven APIs and backends. It gives you managed availability, easy TLS, slot-based deployment, and clean managed identity auth to the rest of the platform, for an integration platform that describes a large share of the synchronous surface. That’s exactly why it earns its place as the default compute tier as long as you know when to reach past it.

Azure App Service architecture: The shape of it

For an integration architect, Azure App Service architecture is mostly about what surrounds the app. Put a gateway with WAF in front, with private connectivity to backends and data. Use managed identity everywhere. Add slots for safe deployment. Trace the whole path. Get those right, and the app in the middle becomes almost boring, which, for a production platform, is the highest compliment there is.

Want the layer above this one? The Azure PaaS map puts App Service in context against Functions, Container Apps, and AKS. It also walks the five-question framework for choosing between them.

AI Foundry Spoke Model Deployment: Why It Still Happens

A comment on the first post in this series asked why AI Foundry Spoke model deployment happens at all in the Citadel pattern, a question worth answering properly in its own post rather than buried in a reply thread.

Great article as usual! Just wondering, in the given Governance Hub & Agent Spoke architecture, what is the purpose of deploying the models both to the Spoke and the Hub? Shouldn’t they only be deployed to the Hub and provided from there?

That’s a sharp question, and it points at a real tension in the Citadel pattern that the original post didn’t call out explicitly enough. Here’s the direct answer, followed by the reasoning behind it.

The short answer

No, they shouldn’t both serve your application’s inference needs. Only the Hub deployment should. The model deployment sitting in the Spoke exists today because of how Azure AI Foundry’s Agent Service currently works, not because the architecture intends a second, governance-free inference path.

That distinction matters, so it’s worth walking through why the Spoke deployment is there at all.

Why AI Foundry Spoke Model Deployment Happens at All

In the ideal version of the Citadel pattern, every model call flows through the Hub’s APIM gateway. That’s the entire point of centralizing governance in one place. It’s what the rest of the series demonstrated: every agent call routed through apim-wpvlimv4ngkns, with token tracking, content safety, cost attribution, and the kill switch all enforced at that single choke point.

The Spoke still ends up with its own local model deployment because the AI Foundry Agent Service needs one for two reasons. This is a direct consequence of how the AI Landing Zone Bicep templates provision the Spoke, not a choice made anywhere in this series.

It powers the Agent Service’s own internal capabilities. Thread management, agent orchestration, and built-in tools like Code Interpreter or File Search (when enabled) call the model directly, through Foundry’s own runtime, rather than through any external endpoint you control. That runtime doesn’t route through APIM. It talks to whatever model deployment sits alongside it in the same project.

It satisfies the Foundry project’s provisioning requirements. A Foundry project currently expects an associated model deployment to exist as part of setting up the project, even if your actual application traffic never calls that deployment directly.

Neither of these is a governance decision. They’re artifacts of how the Agent Service is architected right now.

Two paths, not one

The practical result is that a Citadel deployment ends up with two separate paths to a model, and they serve different purposes.

Diagram showing two paths to a model in the Citadel architecture. Application code routes through APIM in the Hub to the Hub's Azure OpenAI deployment, a governed path shown with a solid green arrow. Foundry's internal Agent Service runtime calls a local model deployment in the Spoke directly, bypassing APIM, shown with a dashed red arrow labeled as ungoverned.
Application traffic should only ever reach a model through APIM in the Hub. The Spoke’s local deployment exists for Foundry’s internal agent runtime, not for your code to call directly.

The Spoke’s local deployment exists for Foundry’s own internal agent runtime. It’s not meant to see your production traffic, and if it does, none of the governance you built in the Hub applies to those calls.

The Hub’s deployment, reached through APIM, is what your application code should use. That’s what we wired up explicitly in Part 2 of this series, with the standard OpenAI SDK pointed at the gateway rather than directly at the Foundry endpoint. The Hub itself is built on the AI Hub Gateway Solution Accelerator, and its AI gateway capabilities are exactly what give APIM the token metering, content safety, and audit trail features this series has leaned on throughout.

The second path exists precisely because of a limitation the series already documented. The Agent Service SDK, in its current preview state, doesn’t route its own LLM calls through APIM. It bypasses the gateway entirely, which means using it directly would mean giving up token metering, policy enforcement, and audit trails on every call the agent makes. That’s why Part 2 used the standard OpenAI SDK pointed at APIM instead of the native Agent Service SDK, and it’s the same underlying issue this reader’s question is really about.

What this means in practice

If you’re building on this pattern today, treat the Spoke’s model deployment as infrastructure the platform needs to exist, not as a second inference endpoint your application is allowed to call. Point your application code at the Hub, through APIM, every time. Leave the Spoke deployment alone to do the job Foundry needs it for internally, and don’t build anything that calls it directly for your own traffic.

If you’re reviewing someone else’s Citadel-pattern deployment, this is worth checking explicitly. A model deployment sitting in a Spoke isn’t wrong by itself, but it’s worth confirming nothing in the application is quietly calling it and skipping the gateway.

Where this is heading

I’d expect this to tighten up as the Agent Service SDK matures out of preview and gains native APIM routing support. When that happens, the two-path situation described here becomes a one-path situation, and the Spoke deployment stops being something you need to actively route around.

Side-by-side comparison diagram. Left panel, labeled Today, shows application code and the Agent Service each reaching separate model deployments, one through APIM in the Hub, one bypassing it in the Spoke. Right panel, labeled Future, shows both application code and the Agent Service routing through a single APIM gateway to one model deployment in the Hub.
Once the Agent Service SDK supports native APIM routing, both application and agent traffic converge on a single governed path.

Until then, the answer to the original question stands: deploy to the Hub, govern everything through APIM, and treat the Spoke’s model deployment as plumbing the platform needs rather than a second front door.

Thanks to the reader who asked the original question. It’s exactly the kind of detail that’s easy to leave implicit in an architecture diagram and much more useful said out loud.