The Citadel APIM gateway policies on Azure are doing the heavy lifting silently in every post in this series. We deployed the hub, connected a tool-calling agent, added conversation persistence, and demonstrated the kill switch. Throughout all of that, every chat completion request passed through these policies. They counted every token. They produced all usage telemetry. This post opens the hood and examines all five Citadel APIM gateway policy layers in Azure token rate limiting, semantic caching, content safety routing, cost attribution, and PII redaction with the actual policy XML, live demonstration results, and honest debugging sessions included.

Where the Citadel APIM Gateway Policies Live in Azure

The Citadel hub deploys two types of policies.

API-level policies apply to all operations on the Azure OpenAI API for every chat completion, embedding, and batch request. These contain the token-limiting, usage-tracking, and routing logic.

Policy fragments serve as reusable blocks that you include by reference. The hub uses fragments for AAD authorization (aad-auth), load balancing (openai-backend-pool), and dynamic throttling (dynamic-throttling-assignment).

In the portal, find them at apim-wpvlimv4ngkns → APIs → Azure OpenAI Service API → All operations → Policies.

The full policy XML is also in the repo at github.com/Azure-Samples/ai-hub-gateway-solution-accelerator under infra/modules/apim/policies/.

Citadel APIM Gateway Policy 1 — Token Rate Limiting

What It Does

APIM natively supports rate limiting on the number of requests per time window. The hub repurposes this to manage capacity based on tokens — the azure-openai-token-limit policy limits how many tokens per minute a subscription can consume, using a counter keyed to the APIM subscription.

The Policy XML

Token rate limiting — inbound section:

<azure-openai-token-limit
    counter-key="@(context.Subscription.Id)"
    tokens-per-minute="10000"
    estimate-prompt-tokens="true"
    tokens-consumed-variable-name="TotalConsumedTokens"
    remaining-tokens-variable-name="RemainingTokens"
/>

Line by line:

• counter-key=”@(context.Subscription.Id)” — the counter is per APIM subscription, so each spoke gets its own independent token budget.
• tokens-per-minute=”10000″ — 10,000 TPM limit per subscription; adjust per spoke based on workload.
• estimate-prompt-tokens=”true” — APIM estimates prompt tokens before the response arrives, enabling proactive rate limiting rather than post-hoc tracking.
• tokens-consumed-variable-name=”TotalConsumedTokens” — stores the running count for use by downstream policies (cost attribution reads this variable).

Demonstration

The Citadel hub actually applies two independent token limit policies at different scopes, and tracing the real behaviour reveals an important lesson about how APIM evaluates them.

Scope 1 — subscription-level, per deployment. The product policy on oai-retail-assistant assigns a different tokens-per-minute value based on the targeted deployment.

<when condition="@((string)context.Variables["target-deployment"] == "chat")">
    <azure-openai-token-limit
        counter-key="@(context.Subscription.Id + "-" + context.Variables["target-deployment"])"
        tokens-per-minute="50"
        token-quota="10000"
        token-quota-period="Weekly"
        tokens-consumed-header-name="consumed-tokens"
        remaining-tokens-header-name="remaining-tokens"
        retry-after-header-name="retry-after"
    />
</when>

Scope 2 — product-level, across all deployments. A second, independent counter applies a higher ceiling across the entire product:

<azure-openai-token-limit
    counter-key="@(context.Product?.Name?.ToString() ?? "Portal-Admin")"
    tokens-per-minute="15000"
    token-quota="150000"
    token-quota-period="Monthly"
    tokens-consumed-header-name="consumed-tokens"
    remaining-tokens-header-name="remaining-tokens"
/>

Both policies execute on every request, each maintaining its own counter. Whichever limit you hit first governs the response, regardless of which one you configured.

Live test — three rapid requests against the chat deployment with the subscription-level limit set to 50 TPM:

$response1 = Invoke-WebRequest -Uri $uri -Method POST -Headers $headers -Body $body
Write-Host "Status: $($response1.StatusCode)"
Write-Host "Remaining-Tokens: $($response1.Headers['remaining-tokens'])"

Results:

REQUEST 1 — Status: 200, Remaining-Tokens: 14940, Consumed-Tokens: 15

REQUEST 2 — Status: 429, Body: “Token limit is exceeded. Try again in 51 seconds.”

REQUEST 3 — Status: 429, Body: “Token limit is exceeded. Try again in 51 seconds.”

Notice that Remaining-Tokens: 14940 on the first request comes from the product-level counter (15000 TPM, barely touched). The 429 on request 2, however, comes from the subscription-level counter for the chat deployment, which only allows 50 tokens per minute, exhausted after a single 15-token call. The visible remaining-tokens header reflects whichever counter last wrote to it, which can be misleading if you assume there is only one limit in play.

Pitfall: Multiple Token Limit Scopes Stack Independently

When multiple azure-openai-token-limit policy elements exist at the API level, product level, and per deployment in a choose block, they are all evaluated for each request. In addition, the most restrictive policy that triggers first decides the outcome. Furthermore, to debug rate limit issues, check all policy scopes: API-level, product-level, and deployment-specific branches. A Named Value in one scope doesn’t affect a hardcoded limit in another.

Citadel APIM Gateway Policy 2 — Semantic Caching

What It Does

Semantic caching intercepts requests before they reach Azure OpenAI and checks for previously answered semantically similar questions. If it finds a match above the configured similarity threshold, it returns the cached response immediately, using zero tokens and ensuring near-zero latency.

The Policy XML

The lookup and store directives belong in different policy sections — lookup runs on the inbound request, store runs on the outbound response after a successful call.

Semantic caching — inbound section:

<azure-openai-semantic-cache-lookup
    score-threshold="0.8"
    embeddings-backend-id="openai-backend-0"
    embeddings-backend-auth="system-assigned"
    ignore-system-messages="true"
    max-message-count="5"
/>

Semantic caching — outbound section:

<azure-openai-semantic-cache-store duration="600" />

Line by line:

• score-threshold=”0.8″ — similarity must be 80% or higher for a cache hit. Lower values increase hit rate but risk returning mismatched responses.
• embeddings-backend-id=”openai-backend-0″ — points to the backend that hosts the embeddings deployment used for cache lookup. The embeddings model itself is configured on the backend, not as a policy attribute — embeddings-model is not a valid attribute on this policy element and will fail schema validation if added.
• embeddings-backend-auth=”system-assigned” — the embeddings backend call is authenticated via the APIM instance’s system-assigned managed identity.
• ignore-system-messages=”true” — only user messages are used for cache key generation, not system prompts.
• max-message-count=”5″ — only the last 5 messages in a conversation are used for cache lookup.
• duration=”600″ — cached responses expire after 10 minutes.

Demonstration

First request — cache miss:

python agent.py 

# Question: What is the weather like in Stockholm right now?
# Response time: 1.2 seconds

Second request — cache hit:

python agent.py 
# Question: What is the weather in Stockholm today?
# Response time: 45ms

The second question resembles the first semantically, but it’s not an exact match. The 0.8 threshold identifies it as a cache hit, allowing the system to return the response from the cache in 45ms without making an Azure OpenAI call or using any tokens. For a conversational agent that often encounters similar questions, semantic caching can lower token consumption by 20–40%.

Production Consideration

The cache shares its data across all subscribers to the same APIM product. If two spokes utilize the same APIM gateway, Spoke A’s cached response can serve Spoke B for a similar inquiry. To protect sensitive data, configure the cache key to be scoped per subscription by adding @(context.Subscription.Id).

Pitfall: Policy Section Placement and Schema Validation

When adding semantic caching for the first time, you may encounter two common schema errors. First, placing azure-openai-semantic-cache-store in the inbound section, along with the lookup policy, results in the error: “Policy is not allowed in this section.” Second, you’ll find that embeddings-model is not a declared attribute on azure-openai-semantic-cache-lookup. Instead of being specified directly in the policy, the embeddings deployment is retrieved from the backend referenced by embeddings-backend-id. You can quickly identify both errors when you save the policy in the portal, which provides the fastest validation of your XML before deployment.

Citadel APIM Gateway Policy 3 — Content Safety Routing

What It Does

The hub routes every request through Azure AI Content Safety before it reaches Azure OpenAI. It inspects both the user prompt and the model response for harmful content in four categories: hate, self-harm, sexual, and violence. The system blocks any requests or responses that exceed the configured severity thresholds.

The Policy XML

Content safety — inbound section:

<llm-content-safety backend-id="content-safety-backend" shield-prompt="true">
    <categories output-type="FourSeverityLevels">
        <category name="Hate" threshold="2" />
        <category name="SelfHarm" threshold="2" />
        <category name="Sexual" threshold="2" />
        <category name="Violence" threshold="2" />
    </categories>
</llm-content-safety>

Line by line:

• backend-id=”content-safety-backend” — routes to the cog-consafety-wpvlimv4ngkns Azure AI Content Safety instance deployed in the hub.
• shield-prompt=”true” — enables Prompt Shields, which additionally detects jailbreak attempts and prompt injection on top of standard content categories.
• categories output-type=”FourSeverityLevels” — selects the four-level severity scale (0, 2, 4, 6) rather than the eight-level scale; each category child element sets its own threshold independently.
• category name=”…” threshold=”2″ — one element per category (Hate, SelfHarm, Sexual, Violence), each can have a different threshold. A threshold of 2 blocks low severity and above; omitting a category leaves it unchecked.

You can reference custom Azure AI Content Safety blocklist IDs in an optional blocklists element to always block organization-specific terms, regardless of their severity scoring.

Pitfall: Element Name and Schema

The element is llm-content-safety, not azure-content-safety — using the wrong name produces “There is no policy matching element name” on save. The schema is also structural rather than attribute-based: categories are child category elements inside a categories block, not a flat comma-separated categories=”…” attribute. Both errors surface immediately in the portal policy editor, which validates the XML before allowing a save.

Demonstration

Normal request — content safety passes silently:

python agent.py
# Question: What is the weather like in Stockholm?
# Answer: The weather in Stockholm is overcast...

The Policy

The llm-content-safety policy does not add a confirmation header on a passing request — the absence of a block response is the only signal that the prompt and response cleared all four category thresholds. To confirm the policy actually ran, check Application Insights:

requests
| where timestamp > ago(10m)
| where resultCode == "200"
| project timestamp, name, resultCode, duration
| order by timestamp desc

A 200 with normal duration confirms the request passed through llm-content-safety without being blocked.

Harmful content

Blocked request — harmful content detected:

# Modify agent.py to send a harmful prompt, then run:
python agent.py
# openai.BadRequestError: Error code: 400
# 'Request failed content safety check.'

Requests block at the gateway before they reach Azure OpenAI, ensuring that these blocked requests consume no tokens. In Application Insights, you observe the block as a 400 response with near-zero duration, reflecting the same signature pattern seen with the kill switch layers mentioned in the previous post.

Citadel APIM Gateway Policy Policy 4 — Cost Attribution

What It Does

Every request that completes successfully generates a usage event sent to Azure Event Hub. A Logic App deployed in the hub consumes these events and writes structured documents to the Cosmos DB ai-usage-container. Each document contains token counts, model version, gateway region, APIM subscription name, and timestamp, giving you per-subscription cost attribution without any agent-side code changes.

The Policy XML

The hub deploys three named loggers, visible via az rest against the APIM management API: appinsights-logger for Application Insights telemetry, usage-eventhub-logger for the cost attribution pipeline described here, and a separate pii-usage-eventhub-logger for PII-specific event logging tied to the redaction policy described later in this post. The logger-id attribute on log-to-eventhub must match one of these exactly — using a placeholder or guessed name produces “Logger not found” when saving the policy.

Cost attribution — outbound section:

<log-to-eventhub logger-id="usage-eventhub-logger" partition-id="0">
@{
    var response = context.Response;
    var request = context.Request;
    var usage = response.Body.As<JObject>(true)?["usage"];
    
    return new JObject(
        new JProperty("id", context.Response.Headers.GetValueOrDefault("x-request-id", Guid.NewGuid().ToString())),
        new JProperty("timestamp", DateTime.UtcNow.ToString("o")),
        new JProperty("subscriptionId", context.Subscription?.Id),
        new JProperty("productName", context.Product?.Name),
        new JProperty("targetService", "chat.completion"),
        new JProperty("model", response.Body.As<JObject>(true)?["model"]?.ToString()),
        new JProperty("gatewayName", context.Deployment.ServiceName),
        new JProperty("gatewayRegion", context.Deployment.Region),
        new JProperty("RequestIp", request.IpAddress),
        new JProperty("promptTokens", usage?["prompt_tokens"]?.ToObject<int>() ?? 0),
        new JProperty("responseTokens", usage?["completion_tokens"]?.ToObject<int>() ?? 0),
        new JProperty("totalTokens", usage?["total_tokens"]?.ToObject<int>() ?? 0),
        new JProperty("backendId", context.Variables.GetValueOrDefault<string>("backendId")),
        new JProperty("deploymentName", context.Request.MatchedParameters["deployment-id"])
    ).ToString();
}
</log-to-eventhub>

Key fields:

• context.Subscription?.Id and context.Product?.Name — the APIM subscription and product name used for the request. In a multi-spoke setup, each spoke has its own APIM product making cost attribution per initiative automatic.
• usage?[“prompt_tokens”] / usage?[“completion_tokens”] — extracted directly from the Azure OpenAI response body.
• context.Deployment.Region — the gateway region (Sweden Central) for data residency auditing.
• context.Variables.GetValueOrDefault<string>(“backendId”) — which Azure OpenAI backend served the request, critical for multi-region deployments.

Demonstration

Run the agent:

python agent.py

Then query the hub Cosmos DB in the portal (cosmos-wpvlimv4ngkns → Data Explorer → ai-usage-container → Items):

SELECT TOP 5 c.timestamp, c.productName, c.model,
c.promptTokens, c.responseTokens, c.totalTokens,
c.gatewayRegion, c.deploymentName
FROM c ORDER BY c._ts DESC

A real document from the deployed hub looks like this:

{
  "timestamp": "6/30/2026 11:26:49 AM",
  "productName": "OAI-HR-Assistant",
  "model": "gpt-4o-2024-11-20",
  "promptTokens": 93,
  "responseTokens": 56,
  "totalTokens": 149,
  "gatewayRegion": "Sweden Central",
  "deploymentName": "chat"
}

Each agent run produces two documents, one for the tool decision call and one for the synthesis call. The totalTokens across both documents is the true per-conversation cost. The productName field maps directly to the APIM product the calling subscription belongs to, in this example OAI-HR-Assistant, distinct from any other product sharing the same hub.

FinOps in Practice

In production with multiple spokes, filter by productName to get per-initiative cost:

SELECT c.productName,
    SUM(c.totalTokens) as totalTokens,
    COUNT(1) as requestCount
FROM c
WHERE c.timestamp >= "2026-06-01T00:00:00Z"
GROUP BY c.productName

Run against the live hub, this returns a clean per-product summary:

[
  {
    "productName": "Portal-Admin",
    "totalTokens": 3865,
    "requestCount": 32
  },
  {
    "productName": "OAI-HR-Assistant",
    "totalTokens": 3035,
    "requestCount": 31
  }
]

Two distinct products, each with an independently aggregated token total and request count, no additional instrumentation required beyond the log-to-eventhub policy already in place. This is your direct FinOps input per AI initiative, per month, ready to feed into a PowerBI report or a monthly cost allocation process.

Citadel APIM Gateway Policy 5 — PII Redaction

What It Does

The hub detects personally identifiable information — names, email addresses, phone numbers, IBAN numbers, and other entity types — in conversation content and logs a redacted version to a dedicated Event Hub logger. The original request still reaches Azure OpenAI unmodified; only the logged version is clean.

There Is No Built-In Policy Element for This

The blog draft for this post originally referenced an azure-openai-pii-removal-logging policy element, on the assumption that APIM ships a built-in PII redaction policy the same way it ships azure-openai-token-limit and azure-openai-semantic-cache-lookup. It does not. Saving a policy referencing that element name fails immediately with “There is no policy matching element name,” the same class of error encountered earlier with azure-content-safety versus the correct llm-content-safety.

Unlike content safety, however, there is no equivalent built-in alternative for PII detection at the time of writing. The hub’s pii-usage-eventhub-logger, visible alongside usage-eventhub-logger and appinsights-logger when listing loggers via the APIM management API, exists as infrastructure for this purpose, but the policy that populates it has to be built explicitly using send-request to call Azure AI Language Service directly, then log-to-eventhub to write the result.

The Policy XML

This belongs entirely in outbound, immediately after the cost attribution log-to-eventhub block. PII detection runs on the response side rather than inbound because the goal is to log a redacted record of the conversation, not to block or alter the request itself, context.Request.Body remains accessible in the outbound pipeline, so the original user message can still be analysed at this stage.

PII detection — outbound section, after cost attribution:

<send-request mode="new" response-variable-name="piiDetectionResponse" timeout="10" ignore-error="true">
    <set-url>@("https://" + "{{language-service-name}}" + ".cognitiveservices.azure.com/language/:analyze-text?api-version=2023-04-01")</set-url>
    <set-method>POST</set-method>
    <set-header name="Ocp-Apim-Subscription-Key" exists-action="override">
        <value>{{language-service-key}}</value>
    </set-header>
    <set-header name="Content-Type" exists-action="override">
        <value>application/json</value>
    </set-header>
    <set-body>@{
        var requestBody = context.Request.Body.As<JObject>(true);
        var userMessage = requestBody["messages"]?.Last?["content"]?.ToString() ?? "";
        return new JObject(
            new JProperty("kind", "PiiEntityRecognition"),
            new JProperty("analysisInput", new JObject(
                new JProperty("documents", new JArray(
                    new JObject(
                        new JProperty("id", "1"),
                        new JProperty("language", "en"),
                        new JProperty("text", userMessage)
                    )
                ))
            ))
        ).ToString();
    }</set-body>
</send-request>
<log-to-eventhub logger-id="pii-usage-eventhub-logger" partition-id="0">
@{
    var piiResult = ((IResponse)context.Variables["piiDetectionResponse"]).Body.As<JObject>(true);
    var redactedText = piiResult?["results"]?["documents"]?[0]?["redactedText"]?.ToString() ?? "[pii-detection-unavailable]";
    var entityCount = piiResult?["results"]?["documents"]?[0]?["entities"]?.Count() ?? 0;
    return new JObject(
        new JProperty("timestamp", DateTime.UtcNow.ToString("o")),
        new JProperty("subscriptionId", context.Subscription?.Id),
        new JProperty("productName", context.Product?.Name),
        new JProperty("redactedText", redactedText),
        new JProperty("piiEntitiesFound", entityCount)
    ).ToString();
}
</log-to-eventhub>

Line by line:

• send-request mode=”new” — fires an independent outbound call to Azure AI Language Service rather than reusing the current request/response context. ignore-error=”true” means a Language Service outage does not fail the agent’s actual response, only the PII logging step is skipped.
• requestBody[“messages”]?.Last?[“content”] — extracts the most recent user message from the original request body for analysis, since that is where PII is most likely to appear.
• The Language Service PiiEntityRecognition endpoint returns both a redactedText field (PII replaced with asterisks) and an entities array listing every detected entity and its category.
• log-to-eventhub logger-id=”pii-usage-eventhub-logger” — writes the redacted text and entity count to the dedicated PII logger, kept separate from the general usage logger so PII-related audit records can be access-controlled independently.

Two Named Values must exist before this policy validates:

az apim nv create \
  --resource-group rg-ai-hub-gateway-dev \
  --service-name apim-wpvlimv4ngkns \
  --named-value-id language-service-name \
  --display-name "language-service-name" \
  --value "cog-language-wpvlimv4ngkns" \
  --secret false
az apim nv create \
  --resource-group rg-ai-hub-gateway-dev \
  --service-name apim-wpvlimv4ngkns \
  --named-value-id language-service-key \
  --display-name "language-service-key" \
  --value "<your-language-service-key>" \
  --secret true

Retrieve the Language Service key:

az cognitiveservices account keys list \
  --name cog-language-wpvlimv4ngkns \
  --resource-group rg-ai-hub-gateway-dev \
  --query key1 -o tsv

Demonstration

Request containing PII:

question = "What is the weather near Jan Janssen who lives at Keizersgracht 123 Amsterdam?"

First attempt, the policy ran but logged the wrong data. The send-request call to Azure AI Language Service fired correctly and returned 200 every time, confirmed via Application Insights dependency tracking:

dependencies
| where timestamp > ago(10m)
| where target contains "cognitiveservices"
| project timestamp, name, target, resultCode, duration

POST /language/:analyze-text  →  200  →  68.62ms
POST /language/:analyze-text  →  200  →  48.26ms
POST /language/:analyze-text  →  200  →  106.96ms

Yet the documents landing in pii-usage-container showed no redactedText field at all, only promptTokens, gatewayName, backendId, and the rest of the cost attribution schema:

{
  "id": "30db8c4b-ce45-4695-9078-e357405845bc",
  "subscriptionId": "oai-hr-assistant-sub-01",
  "productName": "OAI-HR-Assistant",
  "model": "gpt-4o-2024-11-20",
  "promptTokens": 109,
  "responseTokens": 21,
  "gatewayName": "apim-wpvlimv4ngkns.azure-api.net",
  "backendId": "openai-backend-0"
}

The cause turned out to be a copy-paste error in the log-to-eventhub block itself. The comment above it correctly read “PII detection and redacted logging,” but the JObject construction inside was a verbatim duplicate of the cost attribution payload from usage-eventhub-logger, it never referenced context.Variables[“piiDetectionResponse”] at all. The Language Service call succeeded and its result sat in a context variable, completely unused, while the logger faithfully wrote the wrong document every time. Every dependency call returned 200; every Cosmos DB write succeeded; nothing in the telemetry indicated a problem. The only way to catch it was reading the document schema in Cosmos DB and noticing it matched the usage container rather than containing redacted text.

The fix was correcting the log-to-eventhub body to actually read the stored piiDetectionResponse variable:

<log-to-eventhub logger-id="pii-usage-eventhub-logger" partition-id="0">@{
    var piiResult = ((IResponse)context.Variables["piiDetectionResponse"]).Body.As<JObject>(true);
    var redactedText = piiResult?["results"]?["documents"]?[0]?["redactedText"]?.ToString() ?? "[pii-detection-unavailable]";
    var entityCount = piiResult?["results"]?["documents"]?[0]?["entities"]?.Count() ?? 0;
    return new JObject(
        new JProperty("id", context.Response.Headers.GetValueOrDefault("x-request-id", Guid.NewGuid().ToString())),
        new JProperty("timestamp", DateTime.UtcNow.ToString("o")),
        new JProperty("subscriptionId", context.Subscription?.Id),
        new JProperty("productName", context.Product?.Name),
        new JProperty("redactedText", redactedText),
        new JProperty("piiEntitiesFound", entityCount)
    ).ToString();
}</log-to-eventhub>

After the fix, the same question produced the correct redacted record:

{
  "id": "419321ce-0544-4908-a2ac-9ce10f80aaba",
  "timestamp": "2026-06-30T13:12:24.8989851Z",
  "subscriptionId": "oai-hr-assistant-sub-01",
  "productName": "OAI-HR-Assistant",
  "redactedText": "What is the weather near *********** who lives at ***************************?",
  "piiEntitiesFound": 2
}

Two entities detected and redacted, the person’s name and the street address, while the sentence structure remains intact for log readability. A second request, where the agent’s tool call itself failed to resolve the address, still produced a correctly redacted record of the error message:

{
  "redactedText": "{\"error\": \"Location '***************************' not found.\"}",
  "piiEntitiesFound": 1
}

This confirms PII redaction applies consistently regardless of whether the underlying tool call succeeds, the policy operates on the original user message, independent of how the agent’s downstream logic handles it.

In Cosmos DB usage documents, unaffected: the cost attribution document written by usage-eventhub-logger still contains only token counts and metadata, never request body content, so PII redaction applies exclusively to the dedicated pii-usage-container stream.

Pitfall: A Logger Existing Does Not Mean It Logs the Right Thing

The most instructive failure in this section was not a missing policy element or a schema validation error, it was a policy that validated, deployed, and executed successfully while silently logging the wrong payload. Every signal that normally indicates “this is working” was green: the send-request dependency call returned 200, the log-to-eventhub write succeeded, and documents appeared in Cosmos DB on schedule. The only way to catch the bug was to inspect the actual field names in the logged document and notice they matched a different policy’s output entirely. When wiring up custom logging policies, always verify the logged document shape directly in the data store, a successful HTTP status code on a dependency call says nothing about whether its result was ever used downstream.

Pitfall: Two send-request Round Trips Add Latency

Every request now makes an additional outbound call to Azure AI Language Service before the agent’s response is returned to the caller, since send-request blocks until it completes (or times out at 10 seconds with ignore-error=”true”). For latency-sensitive workloads, consider moving PII detection to an asynchronous pattern, log raw request IDs to Event Hub immediately, then run PII detection as a separate downstream process reading from the Event Hub stream, rather than inline in the request path.

The Complete Policy Execution Order

Understanding the order in which policies execute is critical for troubleshooting. APIM processes policies in this sequence.

Inbound, request processing, top to bottom:

1. Kill switch check (Named Value flip)
2. Agent approval header validation
3. Agent ID blocklist check
4. AAD authorization (aad-auth fragment)
5. Token rate limiting (azure-openai-token-limit)
6. Semantic cache lookup (azure-openai-semantic-cache-lookup)
7. Content safety check (inbound prompt, llm-content-safety)
8. Backend routing (load balancer)

Outbound, response processing, top to bottom:

1. Usage event capture (cost attribution fragment)
2. Semantic cache store (azure-openai-semantic-cache-store)
3. PII detection request (send-request to Language Service)
4. Cost attribution logging (log-to-eventhub, usage-eventhub-logger)
5. PII redaction logging (log-to-eventhub, pii-usage-eventhub-logger)
6. Response header enrichment

On-error:

1. Error response formatting
2. Retry logic (if configured)

This order means a request blocked by the kill switch never reaches token rate limiting, content safety, or Azure OpenAI. A request blocked by content safety never reaches cost attribution or PII logging, no usage document and no redacted-text document are created for blocked requests.

Validating All Five Policies in Application Insights

Use this KQL query to see the policy execution evidence in one view:

requests
| where timestamp > ago(24h)
| extend
    cacheHit = tostring(customDimensions["x-cache"]),
    tokensConsumed = toint(customDimensions["TotalConsumedTokens"]),
    remainingTokens = toint(customDimensions["RemainingTokens"])
| project timestamp, resultCode, duration, cacheHit, tokensConsumed, remainingTokens
| order by timestamp desc

Each column corresponds to a policy layer.

• resultCode — 200 (all policies passed, including content safety), 400 (content safety block), 429 (token rate limit), 403/401 (kill switch).
• cacheHit — hit or miss from semantic cache.
• tokensConsumed — running token count from rate limiter.
• remainingTokens — remaining budget for this subscription.

There is no dedicated content safety column because llm-content-safety does not emit a custom telemetry property, its outcome is entirely reflected in resultCode. A 400 with near-zero duration is the signature of a content safety block, the same pattern used to identify kill switch activations in the previous post.

For PII detection specifically, dependency tracking shows whether the Language Service call fired:

dependencies
| where timestamp > ago(24h)
| where target contains "cognitiveservices" and name contains "analyze-text"
| project timestamp, name, target, resultCode, duration
| order by timestamp desc

A 200 here only confirms the call succeeded, it does not confirm the result was logged correctly downstream. Cross-check against the actual documents in pii-usage-container to verify the redactedText field is present and populated, not just that the dependency call returned successfully.

Pitfalls Summary

Token rate limiting doesn’t work for streaming. Fix: use non-streaming endpoints for accurate token counting.

Semantic cache returns stale weather data. Fix: reduce duration to 60 seconds for time-sensitive tool calls.

Content safety blocks valid medical terminology. Fix: raise threshold to 4 for healthcare-specific deployments and validate against your use case.

azure-content-safety element does not exist. Fix: use llm-content-safety with categories and category child elements, not flat attributes.

azure-openai-pii-removal-logging element does not exist. Fix: no built-in policy exists, implement via send-request to Language Service plus a custom log-to-eventhub.

The logger successfully writes but records incorrect data. To fix this, a 200 status on the send-request and a successful write to Event Hub do not guarantee that the result was actually used. Instead, verify the logged document schema directly in Cosmos DB.

The system misses redacting Dutch-language names. To fix this issue, optimize the analyze-text request by setting the language to “nl” and testing it with representative Dutch inputs, instead of using “en,” which is optimized for English.

Cost attribution missing for some requests. Fix: check Event Hub to Logic App pipeline; ingestion lag of 2 to 5 minutes is normal.

To ensure high compliance in production, always surface failures in content safety and PII detection instead of silently bypassing them with the on-error-action or ignore-error=”true” settings.

Conclusion

The five APIM policies in the Citadel hub, token rate limiting, semantic caching, content safety, cost attribution, and PII redaction, collectively implement enterprise AI governance at the gateway layer. None of them require changes to agent code. None of them require spoke involvement. All of them apply to every request from every agent that routes through the hub, regardless of which spoke deployed it.

Furthermore, they compose cleanly: a request that hits the semantic cache never reaches token rate limiting, content safety, or Azure OpenAI. As a result, cached responses consume zero tokens, zero content safety budget, and zero Azure OpenAI quota. Similarly, a request blocked by content safety produces no cost attribution event and no PII redaction record.

This composability is what makes the Citadel APIM hub a governance layer rather than just a proxy. The policies work together, in a defined order, to enforce the enterprise AI control plane pattern across every governed workload, though, as the PII redaction section makes clear, composability and correct execution are not the same thing. A policy chain can validate, deploy, and run green across every dependency call while still logging the wrong data. The only reliable verification is checking the actual data that lands in the store, not the status codes along the way.