API Design Patterns with Azure

Your API works on the happy path. The interesting part starts when a request times out, a client retries, or a dependency goes down. So I looked at various common API design patterns: RESTful resources, versioning, pagination, consistent errors, idempotency keys, conditional requests, rate limiting, async request-reply, API gateway, webhooks, circuit breaker, and aggregation. Moreover, I wanted to see how they behave on Azure, so I built all in one repo and wrote four implementations of idempotency keys. Three of them pass the test most teams write. One of them is correct. Then I ran all four against the deployed app and counted the charges.

The code is on GitHub: steefjan1/api-design-patterns-azure.

Twelve patterns, three kinds of decision

The sample runs Azure API Management (BasicV2) in front of one Azure Functions app on Flex Consumption (.NET 8 isolated), with Table Storage and Storage queues behind it. Managed identity everywhere, no storage keys.

Placing each pattern on the diagram sorts the list into three groups:

GroupPatternsThe question it answers
Contract01 resources, 02 versioning, 03 pagination, 04 errors, 06 conditional requestsWhat can the client rely on?
Retries and time05 idempotency, 08 async request-reply, 10 webhooksWhat happens when one request outlives one round trip?
Capacity and failure07 rate limiting, 09 gateway, 11 circuit breaker, 12 aggregationWhere do we say no, or answer with less?

The gateway commenter is right. In the repo, rate limiting (07) is one rate-limit-by-key policy on the gateway (09), keyed on the APIM subscription. The same policy file renders the gateway’s 429 as application/problem+json (04), so a client parses a throttled request exactly like a validation error from the function. A pattern list draws these as separate boxes. In a deployment, they are one policy.

One measured caveat: 70 simultaneous calls against a limit of 60 per minute let 69 through and throttled one, with Retry-After: 9. The gateway’s counters are synchronized asynchronously, so a burst overshoots before the limit bites. Treat a gateway rate limit as capacity protection, not as an exact quota.

The behavior-first commenter is right too, and the repo shows why. Look at what the “retries and time” patterns do underneath. The idempotency store claims a key with an insert that fails if the row exists. The webhook inbox dedups webhook-id with the same insert. The PATCH handler rejects lost updates with a compare-and-swap on the ETag, and so does the idempotency store when it completes a claim. Twelve names, two primitives: insert-if-absent and compare-and-swap. Table Storage gives you both (AddEntity, and UpdateEntity with an ETag). So do Cosmos DB, SQL Server, and Redis.

The timeout that tells you nothing

A payment call times out. Did it succeed? As one commenter wrote, a timeout tells you almost nothing. You retry because you think it failed, and later you find out the first payment went through too.

Idempotency keys fix this by turning “do it again” into “give me the result of the first one”. The client sends Idempotency-Key: K and reuses K on every retry of that operation. The server remembers K and replays the stored response. That description fits on a slide. The bugs live in the three things it does not say.

Three ways to implement idempotency keys without the semantics

The check is racy. The handler reads the store (“have I seen K?”), finds nothing, charges, then saves K. Two retries that arrive within the same 400 milliseconds both read nothing and both charge. Both callers get a 201.

The key is scoped to the endpoint. The sample exposes the same payment through POST /payments and POST /orders/{id}/pay. If the key is stored as route plus K, a retry through the second route is new work. This is what you get from middleware that keys on the request path, or from one key table per service.

The result is not durable. The claim sits in Table Storage; the response sits in process memory or in a cache with eviction. After a restart, or when the retry lands on another instance, the key exists, and the response does not. The handler answers “already processed” without a payment id. The client is back to the timeout problem, and its likely next move is a new key.

The fourth implementation does three things differently:

// 1. Claim the key atomically. Exactly one concurrent caller gets an ETag back.
var claim = new IdempotencyRecord(scope, key, Pending, fingerprint, now.Add(lease));
var etag = await store.TryInsertAsync(claim, ct); // AddEntity: 409 if it exists
// 2. Pass the key downstream. Every retry asks the provider for the same charge.
var receipt = await provider.ChargeAsync(command, ChargeIdFor(scope, key), ct);
// 3. Store the response durably, only if we still own the claim.
await store.TryReplaceAsync(claim with { Status = Completed, ResponseBody = body }, etag, ct);

When the insert fails, the handler reads the row and decides. A different payload under the same key is a 422. A completed claim is a replay with Idempotent-Replayed: true. A pending claim within its lease is a 409 with Retry-After: 1. An expired lease is taken over with a conditional update. The scope is the caller plus the key, and behind the gateway the caller comes from the APIM subscription, not from a header the client picked.

Step 2 is the one most write-ups skip. If the provider charged the card and the answer got lost, the handler releases its claim and returns 503. The client retries with the same key, the handler runs again, and the provider recognizes the charge id. Without a key that travels downstream, the most careful upstream store still double charges on that path. The racy version fails it for exactly that reason.

Why the tests stay green

The usual test sends the same request three times, one after another, and asserts one charge. All four implementations pass it. Each broken one fails a condition the sequential test never creates: concurrency, a second route, a restart, a lost reply. The xUnit project asserts those failures on purpose, so the evidence stays in the build.

scripts/race.ps1 runs the same twenty cells against the deployed app and reads the ledger. On Azure, the matrix came out exactly as predicted. The one number worth quoting: a burst of twenty identical requests with one key made the racy version charge the card 16 times, and all twenty callers received a 201.

Building the race script produced its own example. Windows PowerShell 5.1 runs on .NET Framework, which allows two connections per host by default. A burst of twenty requests quietly becomes ten pairs, the race window never opens, and the racy implementation looks fine. One line, ServicePointManager.DefaultConnectionLimit = 64, separates a passing test from a real one. In-process tests have the same trap in another form: a lock around a dictionary works on one instance, and Flex Consumption can run the app on forty.

What idempotency keys do not solve

Idempotency keys make retries safe for one operation. They leave three gaps; the sample names in code comments:

  • The response must fit. A table property holds 64 KiB. Large responses go to Blob Storage with a pointer in the row.
  • Side effects after the claim are still two writes. The sample enqueues a payment. captured webhook after storing the response. A crash between those writes loses the event. A transactional outbox closes that gap.
  • Per-instance state stays per instance. The in-code circuit breaker learns about an outage once per instance. In the demo it rejected calls in about 40 ms once open, against about 150 ms per failing call before. The APIM backend breaker sees the whole app but trips for every route. Neither is wrong; pick per dependency.

Where this is the wrong answer

Not every write needs an Idempotency-Key header.

  • PUT and DELETE are idempotent by definition. The sample’s DELETE answers 204 twice. Use conditional requests (If-Match) for lost updates instead.
  • If the business already has a natural unique id, use it. A unique constraint on “one payment per order” is simpler and stronger than a generic key store.
  • For long-running, multi-step work, Durable Functions already gives you instance ids, replay and the 202 status shape of pattern 08.
  • If your payment provider does not accept an idempotency key, no upstream design gives you exactly-once charging. Reconcile against the provider’s ledger and make that part of the design.

Try it

azd up # Functions, Storage, APIM BasicV2 in swedencentral; postdeploy seeds data
dotnet test # the failure matrix, no Azure needed
./scripts/race.ps1 # the same matrix against Azure, counted on the ledger
./scripts/demo.ps1 # all twelve patterns: status codes and headers
azd down --purge # APIM BasicV2 bills by the hour

One deploy lesson for free: the seed function first lived on admin/seed. The Functions host reserves routes that start with admin for its own API. The function still appears in az functionapp function list, and every call returns an empty 404. It now lives on ops/seed.

Conclusion

The list was right that idempotency keys matter. The comment was right about why: the key is the easy part. The atomic claim, the scope, the durable result and the key that travels downstream are the pattern. Test those under concurrency, or the green build is telling you about a different system than the one in production.

Building RAG Pipelines with Azure Functions: Event-Driven Data Retrieval at Scale

This is the fifth and final post in the series on AI and Azure Functions. The first post mapped all four AI-enabled patterns. Posts two through four went deep on MCP server hosting, the serverless agents runtime, and Durable Functions for directed agentic workflows. This post covers the fourth pattern: retrieval-augmented generation.

RAG is where Azure Functions earns its place through a capability that is easy to underestimate: the ability to handle multiple events from multiple data sources simultaneously. A RAG pipeline is only as good as its retrieval layer, and retrieval latency in production comes down to two things: how fast you can query your data sources, and how fast you can scale when demand spikes. Azure Functions on Flex Consumption handles both.

What RAG on Azure Functions actually looks like

The Microsoft Learn documentation is deliberately brief on this pattern: “RAG systems require fast data retrieval and processing. Functions can interact with multiple data sources simultaneously and provide the rapid scale required by RAG scenarios.”

That one-liner contains three architectural decisions worth unpacking.

Multiple data sources simultaneously. A retrieval layer that queries one source at a time is a bottleneck. A function app can fan out — trigger parallel calls to Azure AI Search, a Cosmos DB vector store, a blob-indexed knowledge base, and a case history API — collect all results, and pass the aggregated context to the language model in a single call. This is the fan-out/fan-in pattern from the Durable Functions post, applied to retrieval rather than to a directed workflow.

Event-driven. The most common RAG pattern in tutorials is synchronous: the user sends a query, retrieval runs, and the LLM responds. But production RAG often has an asynchronous dimension — new documents arrive, get processed, get indexed. Azure Functions handles both sides: HTTP-triggered retrieval for the synchronous path, and blob/Event Hubs/Cosmos DB change feed triggers for the ingestion pipeline that keeps the index current. The same function app can serve both.

Rapid event-driven scaling. The Azure OpenAI binding extension adds stateful chat session support — the function maintains conversation history across turns without you writing session management code. Combined with Flex Consumption’s scale-to-zero billing, that means a RAG-enabled chat endpoint costs nothing when idle and scales in seconds when traffic arrives.

The Azure OpenAI binding extension

The binding extension is the most concrete piece of infrastructure the docs call out for RAG. It provides three bindings relevant to RAG scenarios:

Text completion input binding — calls an Azure OpenAI deployment and returns the response. Used for one-shot prompt/response patterns.

Chat completion input binding — maintains a stateful chat session across function invocations. The session history is stored externally (Azure Table Storage by default) and automatically injected into each prompt. This is what the “Custom chat bot” sample in the docs demonstrates.

Embeddings input binding — generates vector embeddings from text. Used in the ingestion pipeline to embed documents before writing them to a vector store.

A simple RAG function using the chat completion binding looks like this in C#:

[Function(nameof(RagChat))]
public static async Task<IActionResult> RagChat(
[HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req,
[TextCompletion(
"{query}",
Model = "%OPENAI_DEPLOYMENT%",
SystemPrompt = "You are a helpful assistant. Answer using only the context provided."
)] TextCompletionResponse completion)
{
return new OkObjectResult(completion.Content);
}

The binding handles the Azure OpenAI call, authentication via managed identity, and response parsing. You write the retrieval logic — what context to inject into {query} — and the binding handles the LLM call.

The production version adds a retrieval step before the binding executes. In practice, this means the HTTP trigger extracts the user query, a call to Azure AI Search (or your vector store of choice) retrieves relevant passages, those passages are injected into the prompt template, and then the binding submits the augmented prompt to the LLM.

APIM as the LLM gateway layer

The binding extension simplifies the function code, but it routes directly to Azure OpenAI — there is no governance layer between the function and the model by default. For production RAG pipelines, that matters: you need rate limiting per client, token quota management, cost attribution across tenants or use cases, and a single point for monitoring and logging LLM calls.

APIM fills that role. Place it between the function app and Azure OpenAI, configure an inbound policy to validate the request and apply per-subscription token quotas, and configure an outbound policy to log usage. The function app calls the APIM endpoint rather than the Azure OpenAI endpoint directly.

One practical constraint from real deployment experience: if you use the Azure AI Foundry Agent Service SDK alongside Azure Functions, the SDK routes LLM calls directly to Azure OpenAI and bypasses APIM entirely. The standard OpenAI SDK doesn’t behave this way—calls flow through APIM as expected. The Citadel Platform series covers this in detail, including how to structure APIM policies for the token quota patterns that matter in production.

For RAG pipelines specifically, two APIM policies are worth implementing:

Token quota by subscription key. Set a per-day or per-hour token budget for each API consumer. RAG queries are often significantly larger than simple completions because the retrieved context inflates the prompt — a query with 2,000 tokens of retrieved passages consumes far more quota than a direct LLM call. Factor this into your quota design.

Semantic caching. APIM’s semantic caching policy stores LLM responses and returns cached results for semantically similar queries. For a RAG pipeline serving a knowledge base that changes infrequently, this can dramatically reduce token consumption and latency.

The ingestion pipeline

The retrieval side of RAG gets most of the attention, but the ingestion pipeline- how documents get into the index is where Azure Functions adds less-discussed value.

When a new document arrives in blob storage, the function extracts text, generates a vector embedding, and writes the result to Azure AI Search. An Event Hubs trigger handles high-throughput ingestion where many documents arrive simultaneously. A Cosmos DB change feed trigger keeps a vector index in sync with a transactional database.

The key advantage over a dedicated ingestion service is that the same managed identity, Application Insights workspace, and azd deployment pipeline cover both retrieval and ingestion. Operational complexity stays low even as the pipeline grows.

One constraint to know before building: blob triggers on Flex Consumption require EventGrid as the source. Standard polling blob triggers are not supported on the Flex Consumption plan. This means you must configure an EventGrid system topic and event subscription after provisioning — a two-step post-provision operation that cannot be included in the same Bicep deployment as the function app.

A minimal ingestion function using the Azure OpenAI SDK for embeddings — the recommended approach until the embeddings binding extension reaches GA:

[Function(nameof(IngestDocument))]
public async Task Run(
[EventGridTrigger] EventGridEvent eventGridEvent,
CancellationToken cancellationToken)
{
// Read blob from storage, generate embedding, write to Azure AI Search
var blobContent = await ReadBlobAsync(eventGridEvent.Subject, cancellationToken);
var embeddingClient = openAiClient.GetEmbeddingClient(embeddingsDeployment);
var embedding = await embeddingClient.GenerateEmbeddingAsync(
blobContent, cancellationToken: cancellationToken);
var document = new SearchDocument { ContentVector = embedding.Value.ToFloats().ToArray() };
await searchClient.IndexDocumentsAsync(
IndexDocumentsBatch.Upload([document]), cancellationToken: cancellationToken);
}

The EventGrid trigger fires on MicrosoftStorage.BlobCreated events; the function reads the blob, generates the vector, and writes to the index. EventGrid delivery includes retry logic —failed events are retried with exponential backoff for up to 24 hours.

Connecting the four patterns

This is the final post in the series, so it is worth mapping how the four AI patterns relate to each other in a production architecture. They are not mutually exclusive.

A realistic enterprise RAG system might use all four:

  • Azure Functions RAG pipeline handles retrieval — HTTP-triggered queries fan out to multiple sources, ingestion functions process new documents via blob and Event Hubs triggers.
  • MCP binding extension exposes the retrieval function as an MCP tool, so AI agents and Copilot can call it directly
  • Durable Functions orchestrates multi-step retrieval workflows, including approve-before-index, parallel ingestion with fan-out, andandlong-running document processing with human-in-the-loop review.
  • Serverless agents runtime wraps the whole thing in an agent that decides which data sources to query based on the user’s intent, using the retrieval function as one of its tools

APIM sits in front of the Azure OpenAI calls regardless of which pattern makes the LLM request. The function app’s managed identity authenticates to both APIM and the downstream Azure services. Application Insights provides a unified view across all four patterns.

The spectrum from the Durable Functions post applies here too: RAG retrieval is mostly directed (the function knows which sources to query), but the agent layer that decides what to retrieve and how to use the results is autonomous. Azure Functions provides the event-driven, scalable retrieval infrastructure for both ends of that spectrum.

What to know before building

Choose your vector store before choosing your binding. The Azure OpenAI binding extension integrates cleanly with Azure AI Search. If you are using Cosmos DB for MongoDB vCore, a Postgres pgvector extension, or an external vector database, you write the retrieval logic directly using the SDK — the binding handles the LLM call, not the retrieval.

The embeddings binding and the chat completion binding have different cost profiles. Embeddings are cheap per token; completions are expensive. The ingestion pipeline (embeddings) can run at high volume with low cost. The retrieval + completion path (chat) is where token costs accumulate. Design your APIM quota policies accordingly.

Stateful chat sessions need external storage. The chat completion binding stores session history in Azure Table Storage by default. For multi-tenant RAG, partition session keys by user or tenant — the default key scheme does not enforce isolation automatically.

Retrieval quality determines output quality. Azure Functions handles the retrieval infrastructure; what you retrieve is your responsibility. This distinction matters more than it might seem.

With keyword search alone, a query like “List all Metallica albums” retrieves documents based on term frequency across the index. An index containing Radiohead (48 releases) and Porcupine Tree alongside Metallica will surface Radiohead and Porcupine Tree documents ahead of Metallica ones, because those bands have more indexed content matching common terms. The LLM then filters the irrelevant context and may miss albums that simply did not appear in the retrieved passages.

Hybrid search combining keyword retrieval with vector retrieval addresses this directly. Generate an embedding for the user query, add a VectorizedQuery targeting the content_vector field, and Azure AI Search fuses both signals using reciprocal rank fusion. Semantically relevant documents surface regardless of term frequency:

var vectorQuery = new VectorizedQuery(queryVector)
{
KNearestNeighborsCount = 20,
Fields = { "content_vector" }
};
var searchOptions = new SearchOptions
{
Size = 20,
VectorSearch = new VectorSearchOptions { Queries = { vectorQuery } }
};
// Hybrid: keyword query string + vector query sent together
var results = await searchClient.SearchAsync<MusicDocument>(
userMessage, searchOptions, cancellationToken);

The companion repo implements hybrid search in MusicChatAgent.cs. Chunk size, embedding model choice, and whether you use keyword, semantic, or hybrid search have more impact on output quality than any infrastructure decision.

Testing the RAG pipeline

The companion repo supports several query patterns that exercise different aspects of the pipeline. These are worth running after ingesting a few bands to verify both the retrieval and the grounding behaviour.

Artist-specific ranking — tests hybrid search surfacing the right band:

{"message": "List all Metallica albums with their ratings, ordered highest to lowest"}

Date filtering via LLM reasoning — the index has no date filter; the model reasons over retrieved content:

{"message": "What albums did Tool release in the 1990s?"}

Cross-band comparison — tests whether retrieval surfaces relevant documents from multiple bands:

{"message": "Compare Opeth and Porcupine Tree — which has more critically acclaimed albums?"}

Rating threshold query:

{"message": "Which albums across all bands have a rating above 4.5?"}

Not-in-index grounding test — the agent should say it does not have information rather than hallucinate:

{"message": "Tell me about a band called Coldplay"}

Validation — should return HTTP 400:

{"message": ""}

The hybrid search implementation means artist-specific queries return only that artist’s documents. With keyword-only search, a query for Metallica in an index containing Radiohead (48 releases) would return mostly Radiohead documents — the vector component ensures semantic relevance wins over term frequency.

The companion repo — Music RAG Agent

The music-rag-agent companion repo implements the full two-pipeline RAG system described in this post. It scrapes band and album data from SputnikMusic, generates embeddings with the Azure OpenAI SDK, indexes into Azure AI Search, and exposes an HTTP chat endpoint that retrieves relevant passages and returns grounded answers.

After ingesting Tool, Opeth, Porcupine Tree, and Radiohead you can ask questions like:

“Which band has the highest rated albums — Tool, Opeth, Porcupine Tree or Radiohead?”

The retrieval correctly surfaces Sputnik ratings and vote counts for each album. The ingestion pipeline handles special characters in album titles (Ænima, Opiate²) via URL-safe Base64 document keys, and the EventGrid trigger ensures reliable delivery with 24-hour retry backoff.

Known Sputnik artist IDs: Tool = 83, Opeth = 932, Porcupine Tree = 328, Radiohead = 86.


Series wrap-up

This post completes the series on AI and Azure Functions. The five posts covered:

  1. The four AI-enabled patterns and how to choose between them
  2. Hosting MCP servers — binding extension (GA), self-hosted SDK (preview), and queue-based tools
  3. The serverless agents runtime — markdown agents, the three configuration files, and dynamic workflows
  4. Durable Functions for directed agentic workflows — five patterns with working C# samples verified on Azure
  5. RAG pipelines — event-driven retrieval, the Azure OpenAI binding extension, and APIM governance

The companion code repositories are at github.com/steefjan1.