Five RAG Architectures in Real Azure Code

Posted on September 13, 2026 by steefjan1970

Over the past few months I kept running into the similar looking infographics, in one form or another: five or six boxes, each a named RAG architecture, arrows showing how a query flows through it. Hybrid RAG. GraphRAG. Agentic RAG. Corrective RAG. Multimodal RAG. They’re useful as vocabulary. They are not implementation guides. None of them show you the part that actually takes the time.

So I built all five RAG architectures, on Azure, against one shared corpus and one shared set of test questions, and measured what came out. This post is the result: what each diagram leaves out, what the equivalent Azure code actually looks like, where the real deployment pitfalls were, and a comparison table built from real runs, not from argument.

The corpus is a fictional Dutch health insurer, Zorgverzekeraar Meridiaan, the same one I’ve used in a couple of other posts in this series. Nine documents: dental and physiotherapy policies, a provider network, an authorization process, a member complaint and the quarterly report that restates it, a stale FAQ sitting next to the current policy, a reimbursement table, and a scanned claim form. Fifteen questions, tagged by which pattern they were designed to stress. All five patterns answer all fifteen questions, so the comparison is apples to apples.

The repo is at github.com/steefjan1/five-rag-patterns if you want to run it yourself.

What the diagram shows vs. what the Azure code does

Hybrid RAG

The diagram draws dense and sparse retrieval as two separate paths that merge into a box labeled Reciprocal Rank Fusion. That box is mostly a non-event on Azure. Azure AI Search’s hybrid query type takes a vector query and a text query together and fuses them server-side. There is no RRF code to write.

What actually takes engineering effort is the index schema: chunk granularity (I chunk by document section, not by a fixed token window, so a retrieval unit is a coherent answer, not an arbitrary slice), which fields are filterable versus searchable versus vector, and whether semantic ranking is worth its cost on top of the fusion you already get for free.

Measured: recall 1.00 across all fifteen questions, the best of any pattern on pure coverage. Precision sits at 0.38, diluted by a fixed top-5 retrieval regardless of how many documents a question actually needs. Cheapest sane baseline in the set: $0.0026 and 2.00 seconds per query.

GraphRAG

The diagram draws one static graph: entities, edges, a subgraph retrieval step, a box for community summaries. What it doesn’t draw is that the graph has a maintenance cost. Community detection (Louvain, via networkx, which runs fine at this corpus’s scale without a dedicated graph database) and community summarization are real compute and real Azure OpenAI spend, paid once at setup and again every time the graph changes enough to shift community boundaries. Nothing about that shows up in the box-and-arrow version.

Entity linking here is a cheap substring match against entity names, not an embedding call, which is part of why this pattern is the cheapest per query in the whole set. Retrieval is a graph walk: two hops, both directions, so a question like “which hospital did this referral come from, and which GP group refers into that hospital” resolves correctly even though no single document states the answer. It’s two separate edges, walked in sequence.

Measured: recall 1.00 on its own three relational questions, the two-hop case included, and 0.21 on the other twelve. No other pattern swings that hard between its own territory and everything else. $0.0013 per query, the cheapest pattern here, in the narrowest lane.

Agentic RAG

The diagram shows a planner routing to tools and a reasoner that loops “until confident.” There is no upper bound drawn anywhere on that loop. Left alone, that is a cost leak, not a reliability feature, so the actual implementation caps it at five iterations and reports hitting the cap as its own outcome rather than quietly forcing an answer and calling it clean.

The other thing worth knowing if you’re building this on Azure: the AI Foundry Agent Service SDK bypasses API Management for its own LLM calls. I found this the hard way on an earlier project in this series. If your governance model depends on APIM, that means routing tool-calling agents through the standard OpenAI SDK pointed at the gateway, not through the framework’s own agent runtime, or every rate limit and kill switch you built stops applying the moment the agent framework makes the call instead of your code.

Two of this pattern’s four tools aren’t retrieval at all. The dental waiting-period and annual-maximum arithmetic is transcribed from the policy documents as plain code, not left for a language model to compute from prose. Insurance eligibility math is exactly the kind of thing an LLM gets subtly wrong under pressure, and exactly the kind of thing code gets right every time.

Measured: precision 1.00, recall 1.00 on its own two questions, and it’s the only pattern that doesn’t collapse elsewhere: recall 0.85 on the other thirteen, because it always has a general search tool as a fallback when nothing more specific fits. That’s the real finding here. It’s not that Agentic RAG is “better,” it’s that it hedges.

Corrective RAG

The diagram shows retrieve, grade, then three branches: answer, rewrite the query and loop back, or fall back to a web search. The rewrite loop has an arrow pointing backward and no stated exit condition. A closed corpus also has no web to fall back to, so “incorrect” here means declining to answer rather than guessing.

The corpus has a document built specifically to test the grading step: an archived FAQ with a plausible, wrong number sitting right next to the current policy with the right one. A pattern with no grading step retrieves both and may cite either. This one grades the retrieval, asks the model to identify which passage is authoritative using published dates and explicit supersession language, and only feeds the authoritative passages to the final answer. What got fetched and what got used are tracked separately on purpose, so a working grader shows zero distractor citations even though the distractor was retrieved.

Measured: precision 1.00, recall 1.00 on the three distractor questions, confirmed live, not just in the design. The more interesting number is that its other twelve questions score better (precision 0.79) than its own target slice (0.67). The grading discipline isn’t just catching the one distractor it was built to catch, it generalizes. That comes at a real cost: 3.97 seconds average latency, roughly double every other pattern, and the highest cost per query in the set, because a full run can mean three model calls instead of one.

Multimodal RAG

The diagram’s box says “shared multimodal embedding model (e.g. CLIP or ColPali),” which means self-hosting an embedding model. That’s a heavier operational commitment than anything else in this comparison needs, and it’s avoidable. This uses caption-then-embed instead: Document Intelligence extracts the actual structure of the reimbursement table (tables are exactly where a vision model hallucinates a plausible-looking row that isn’t in the source, so that step doesn’t get skipped), the vision-capable chat deployment captions the scanned claim form directly, and both captions get embedded with the same text-embedding-3-large deployment every other pattern uses. Same index Hybrid RAG built, two more documents in it, no new index and no new field.

One implementation note that cost real iteration: a first version of the captioning prompt asked for verbatim transcription, which correctly produced the form’s Dutch date format and Dutch status text. A validation step checking the caption against a hand-written ground truth flagged that as a mismatch, because the ground truth expected ISO dates and English. That’s not a captioning bug, it’s a prompt that needed to ask for normalization, not transcription. Worth deciding on purpose, since a shared index with mixed date formats and mixed languages retrieves worse than a normalized one.

Measured: recall 1.00 across the board and groundedness 1.00 on its own two questions, with no degradation on the other thirteen. That composability is the finding: this pattern is Hybrid RAG’s exact retrieve-and-answer loop plus two documents, and the numbers confirm that composition was free.

The comparison table

Same corpus, same fifteen questions, one pass, all five RAG architectures.

Pattern	Avg latency	Cost/query	Overall precision	Overall recall	Own-target recall
Hybrid	2.00s	$0.0026	0.38	1.00	1.00 (n=5)
GraphRAG	2.32s	$0.0013	0.20	0.37	1.00 (n=3)
Agentic	2.09s	$0.0045	0.45	0.87	1.00 (n=2)
Corrective	3.97s	$0.0051	0.77	0.97	1.00 (n=3)
Multimodal	2.18s	$0.0027	0.38	1.00	1.00 (n=2)

“Own-target” means the small subset of the fifteen questions each pattern was actually designed to answer (GraphRAG’s two-hop provider questions, Corrective RAG’s stale-document case, and so on). Every pattern hits recall 1.00 in its own lane. What separates them is what happens outside it: GraphRAG falls to 0.21 recall on the other twelve questions, Agentic RAG only falls to 0.85, and Hybrid, Corrective, and Multimodal don’t fall at all, because their retrieval isn’t scoped to a narrow entity set in the first place.

Two honest caveats on this table. Precision across every pattern is capped low by a fixed top-5 retrieval regardless of how many documents a question actually needs, so precision here measures retrieval breadth more than answer quality, read recall and the own-target column as the more meaningful columns. And the cost figures come from a placeholder price table, not a live Azure billing export, useful for comparing patterns against each other, not for a procurement conversation.

Deployment pitfalls

Every one of these was a real failure against a live Azure subscription, not a hypothetical.

Pinned model versions rot. A deployment written against gpt-4o-mini version 2024-07-18 failed eight months later with ServiceModelDeprecated. The fix wasn’t a newer pin, it was to stop pinning: leave the deployment’s model version empty and let Azure resolve the current default, and check az cognitiveservices model list -l <region> -o table before assuming a model name is still offered at all.

The account kind changed. Azure OpenAI is now provisioned through Foundry as kind: 'AIServices', not the older kind: 'OpenAI'. Same deployment mechanism underneath, different account kind and a newer API version. A template written against the old kind fails Cognitive Services preflight validation, not at compile time.

A malformed policy XML fails at ARM validation, not at Bicep build time. An APIM policy embedded as a Bicep string had a raw double-quoted path literal sitting inside an already double-quoted XML attribute. bicep build compiled it clean, because Bicep has no way to know a string is meant to be well-formed XML. The actual break only showed up against the live ARM validation API, after Azure AI Search, Cosmos DB, and the Foundry account had already finished provisioning. A small script that compiles the template and separately parses every embedded policy string as XML catches this before the next azd up, not during one.

RBAC role assignments alone don’t turn on Azure AD authentication. Azure AI Search kept returning a flat 403 on every data-plane call despite two correctly scoped role assignments, because the service still only accepted API-key authentication. Nothing had told it to accept AAD tokens at all. The fix is a separate property, disableLocalAuth: true, on the search service itself. If a resource with roles that look correct still refuses an authenticated caller, check the resource’s own auth settings before re-checking the role assignment.

Where none of this is the answer

None of these five patterns is the right first move for a small, stable knowledge base. Plain vector search, no fusion, no graph, no grading, no agent loop, is the correct answer until you can name the specific failure mode you’re buying insurance against. Every pattern here is a bet against one kind of failure, and every bet has a cost attached whether or not you ever collect on it.

Don’t build all five for one real system either. Pick based on the failure mode your domain actually has. GraphRAG only pays for itself if your questions are genuinely relational, multi-hop, the kind no single document answers. If they’re not, you’re paying setup cost and getting a narrower Hybrid RAG. Agentic RAG’s flexibility costs a planning call before any retrieval happens at all, worth it if your questions genuinely vary in shape, wasted overhead if they don’t. Corrective RAG’s discipline costs roughly double the latency of everything else in this comparison. That’s a fine trade when a wrong answer is expensive and a two-second wait isn’t. It’s a bad trade for a chat widget where speed is the product.

What this actually proves

The infographic’s taxonomy is real. These five RAG architectures are genuinely different, with genuinely different failure modes, and that part of the diagram holds up. What doesn’t hold up is the implication that the hard part is choosing between them. The hard part, in every case, was the piece the diagram didn’t draw: RRF turned out to be free because Azure AI Search already does it, but community detection is not free and has to be redone as the graph changes. An agent loop needs a hard cap or it’s an open-ended bill. A query rewrite loop needs the same cap for the same reason. A self-hosted multimodal embedding model turned out to be avoidable entirely, caption-then-embed onto infrastructure you already have gets you most of the way there.

The single most useful number in this whole exercise might be the smallest one: Corrective RAG’s grading step scored better on questions it wasn’t built for than on the one it was. That’s a pattern worth paying attention to. The things that make a RAG system more disciplined in one specific place often make it more disciplined everywhere, not just in the place you were testing for.

If you want to see the actual failure modes up close rather than the aggregate table, the earlier posts in this series go deeper on two of them: what naive RAG diagrams leave out covers the hybrid retrieval and groundedness gaps in more detail, and choosing between RAG, GraphRAG, and Agentic RAG when auditability is the constraint makes the conceptual case this post backs with numbers.

The full repo, including the corpus, the eval harness, and every pattern’s implementation, is at github.com/steefjan1/five-rag-patterns.

Don’t Build Around Today’s Model. Build for the AI Control Plane

Posted on August 31, 2026 by steefjan1970

Microsoft isn’t just shipping AI models. It is building toward a model-agnostic AI control plane. The bigger story behind the recent Microsoft Foundry updates is not another model launch. It is the architecture direction. Once you see it, the individual announcements fall into place as layers of one stack:

AI Control Plane → Model Router → Agents → RAG/Knowledge → MCP/Tools → Enterprise Systems

The AI control plane takes shape

Two August updates make the direction concrete. First, the model router update expanded the router to 28 regions for global standard and 21 data zone regions. The supported pool now includes Anthropic Claude Opus 4.8 and the GPT-5.6 family, while deprecated models such as DeepSeek-V3.1 and the gpt-5-chat line were pruned. Second, Foundry added DeepSeek-V4-Flash-0731 and NVIDIA Nemotron 3.5 Lightning to the catalog, each available through multiple deployment paths: Direct from Azure, Fireworks on Foundry, or the Hugging Face collection on managed compute.

I covered the router update in more detail on InfoQ, including the caveats around behavioral stability. Here, I want to focus on what the updates mean for your architecture.

Neither update is spectacular on its own. Together, however, they show a platform where models arrive, improve, and retire underneath a stable endpoint. Your application keeps calling the same integration while the pool refreshes. That is control plane behavior, not model shipping.

Agent ≠ Model

Here is the key idea for enterprise architects: an agent is not a model. An agent is an orchestration unit with instructions, knowledge, and tools. The model is a swappable dependency underneath it.

The model router makes that separation operational. It selects a model per request, optimizing for quality, cost, and latency within the regions your governance allows. You can run it in balanced, quality, or cost mode, and you can restrict routing to an approved subset of models. Moreover, every response includes a model field that shows which model handled the request, so the routing decisions leave an auditable trail.

Microsoft frames this as a hill climb: model selection as a continuous, measured loop rather than a one-time decision. In an ecosystem where the frontier moves monthly, a hardcoded model choice goes stale fast. An agent bound to the control plane instead of a specific model can evolve without a rebuild every time the leaderboard changes.

What this means for your architecture

For enterprise architects, the practical guidance follows directly. Treat model selection as configuration, not code. Put governance, observability, and policy at the control plane layer, because that is where they survive model churn. Ground agents in your own knowledge through RAG, and connect them to enterprise systems through MCP and tools. Consequently, each layer can evolve at its own pace. The stack outlives any single model.

Where this is the wrong answer

Model-agnostic routing is not free, and it is not always right. If your workload requires reproducible behavior, for example in regulated decision flows, a pinned model version beats a router that refreshes its pool automatically. Your evaluations were run against a specific model; a silent pool refresh invalidates them until you re-run. There are operational caveats too: Anthropic models still need to be deployed separately before the router can reach them, and routing modes take time to propagate. Finally, a single well-tuned small model per agent remains the simpler option for narrow, high-volume tasks. Routing adds a layer you must monitor. Only accept that cost when model diversity actually pays for itself.

Closing thoughts

The direction is clear even where the details will shift. Models are becoming interchangeable parts. The durable investment is the control plane: routing, governance, evaluation, and the connective tissue to your enterprise systems. So don’t build around today’s model. Build for the control plane, and let the models come and go.

What’s your approach: one model per agent, or model-agnostic agents?

Building RAG Pipelines with Azure Functions: Event-Driven Data Retrieval at Scale

Posted on August 19, 2026 by steefjan1970

This is the fifth and final post in the series on AI and Azure Functions. The first post mapped all four AI-enabled patterns. Posts two through four went deep on MCP server hosting, the serverless agents runtime, and Durable Functions for directed agentic workflows. This post covers the fourth pattern: retrieval-augmented generation.

RAG is where Azure Functions earns its place through a capability that is easy to underestimate: the ability to handle multiple events from multiple data sources simultaneously. A RAG pipeline is only as good as its retrieval layer, and retrieval latency in production comes down to two things: how fast you can query your data sources, and how fast you can scale when demand spikes. Azure Functions on Flex Consumption handles both.

What RAG on Azure Functions actually looks like

The Microsoft Learn documentation is deliberately brief on this pattern: “RAG systems require fast data retrieval and processing. Functions can interact with multiple data sources simultaneously and provide the rapid scale required by RAG scenarios.”

That one-liner contains three architectural decisions worth unpacking.

Multiple data sources simultaneously. A retrieval layer that queries one source at a time is a bottleneck. A function app can fan out — trigger parallel calls to Azure AI Search, a Cosmos DB vector store, a blob-indexed knowledge base, and a case history API — collect all results, and pass the aggregated context to the language model in a single call. This is the fan-out/fan-in pattern from the Durable Functions post, applied to retrieval rather than to a directed workflow.

Event-driven. The most common RAG pattern in tutorials is synchronous: the user sends a query, retrieval runs, and the LLM responds. But production RAG often has an asynchronous dimension — new documents arrive, get processed, get indexed. Azure Functions handles both sides: HTTP-triggered retrieval for the synchronous path, and blob/Event Hubs/Cosmos DB change feed triggers for the ingestion pipeline that keeps the index current. The same function app can serve both.

Rapid event-driven scaling. The Azure OpenAI binding extension adds stateful chat session support — the function maintains conversation history across turns without you writing session management code. Combined with Flex Consumption’s scale-to-zero billing, that means a RAG-enabled chat endpoint costs nothing when idle and scales in seconds when traffic arrives.

The Azure OpenAI binding extension

The binding extension is the most concrete piece of infrastructure the docs call out for RAG. It provides three bindings relevant to RAG scenarios:

Text completion input binding — calls an Azure OpenAI deployment and returns the response. Used for one-shot prompt/response patterns.

Chat completion input binding — maintains a stateful chat session across function invocations. The session history is stored externally (Azure Table Storage by default) and automatically injected into each prompt. This is what the “Custom chat bot” sample in the docs demonstrates.

Embeddings input binding — generates vector embeddings from text. Used in the ingestion pipeline to embed documents before writing them to a vector store.

A simple RAG function using the chat completion binding looks like this in C#:

			
[Function(nameof(RagChat))]
public static async Task<IActionResult> RagChat(
    [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequestData req,
    [TextCompletion(
        "{query}",
        Model = "%OPENAI_DEPLOYMENT%",
        SystemPrompt = "You are a helpful assistant. Answer using only the context provided."
    )] TextCompletionResponse completion)
{
    return new OkObjectResult(completion.Content);
}

		

The binding handles the Azure OpenAI call, authentication via managed identity, and response parsing. You write the retrieval logic — what context to inject into {query} — and the binding handles the LLM call.

The production version adds a retrieval step before the binding executes. In practice, this means the HTTP trigger extracts the user query, a call to Azure AI Search (or your vector store of choice) retrieves relevant passages, those passages are injected into the prompt template, and then the binding submits the augmented prompt to the LLM.

APIM as the LLM gateway layer

The binding extension simplifies the function code, but it routes directly to Azure OpenAI — there is no governance layer between the function and the model by default. For production RAG pipelines, that matters: you need rate limiting per client, token quota management, cost attribution across tenants or use cases, and a single point for monitoring and logging LLM calls.

APIM fills that role. Place it between the function app and Azure OpenAI, configure an inbound policy to validate the request and apply per-subscription token quotas, and configure an outbound policy to log usage. The function app calls the APIM endpoint rather than the Azure OpenAI endpoint directly.

One practical constraint from real deployment experience: if you use the Azure AI Foundry Agent Service SDK alongside Azure Functions, the SDK routes LLM calls directly to Azure OpenAI and bypasses APIM entirely. The standard OpenAI SDK doesn’t behave this way—calls flow through APIM as expected. The Citadel Platform series covers this in detail, including how to structure APIM policies for the token quota patterns that matter in production.

For RAG pipelines specifically, two APIM policies are worth implementing:

Token quota by subscription key. Set a per-day or per-hour token budget for each API consumer. RAG queries are often significantly larger than simple completions because the retrieved context inflates the prompt — a query with 2,000 tokens of retrieved passages consumes far more quota than a direct LLM call. Factor this into your quota design.

Semantic caching. APIM’s semantic caching policy stores LLM responses and returns cached results for semantically similar queries. For a RAG pipeline serving a knowledge base that changes infrequently, this can dramatically reduce token consumption and latency.

The ingestion pipeline

The retrieval side of RAG gets most of the attention, but the ingestion pipeline- how documents get into the index is where Azure Functions adds less-discussed value.

When a new document arrives in blob storage, the function extracts text, generates a vector embedding, and writes the result to Azure AI Search. An Event Hubs trigger handles high-throughput ingestion where many documents arrive simultaneously. A Cosmos DB change feed trigger keeps a vector index in sync with a transactional database.

The key advantage over a dedicated ingestion service is that the same managed identity, Application Insights workspace, and azd deployment pipeline cover both retrieval and ingestion. Operational complexity stays low even as the pipeline grows.

One constraint to know before building: blob triggers on Flex Consumption require EventGrid as the source. Standard polling blob triggers are not supported on the Flex Consumption plan. This means you must configure an EventGrid system topic and event subscription after provisioning — a two-step post-provision operation that cannot be included in the same Bicep deployment as the function app.

A minimal ingestion function using the Azure OpenAI SDK for embeddings — the recommended approach until the embeddings binding extension reaches GA:

			
[Function(nameof(IngestDocument))]
public async Task Run(
    [EventGridTrigger] EventGridEvent eventGridEvent,
    CancellationToken cancellationToken)
{
    // Read blob from storage, generate embedding, write to Azure AI Search
    var blobContent = await ReadBlobAsync(eventGridEvent.Subject, cancellationToken);
    var embeddingClient = openAiClient.GetEmbeddingClient(embeddingsDeployment);
    var embedding = await embeddingClient.GenerateEmbeddingAsync(
        blobContent, cancellationToken: cancellationToken);
    var document = new SearchDocument { ContentVector = embedding.Value.ToFloats().ToArray() };
    await searchClient.IndexDocumentsAsync(
        IndexDocumentsBatch.Upload([document]), cancellationToken: cancellationToken);
}

		

The EventGrid trigger fires on MicrosoftStorage.BlobCreated events; the function reads the blob, generates the vector, and writes to the index. EventGrid delivery includes retry logic —failed events are retried with exponential backoff for up to 24 hours.

Connecting the four patterns

This is the final post in the series, so it is worth mapping how the four AI patterns relate to each other in a production architecture. They are not mutually exclusive.

A realistic enterprise RAG system might use all four:

Azure Functions RAG pipeline handles retrieval — HTTP-triggered queries fan out to multiple sources, ingestion functions process new documents via blob and Event Hubs triggers.
MCP binding extension exposes the retrieval function as an MCP tool, so AI agents and Copilot can call it directly
Durable Functions orchestrates multi-step retrieval workflows, including approve-before-index, parallel ingestion with fan-out, andandlong-running document processing with human-in-the-loop review.
Serverless agents runtime wraps the whole thing in an agent that decides which data sources to query based on the user’s intent, using the retrieval function as one of its tools

APIM sits in front of the Azure OpenAI calls regardless of which pattern makes the LLM request. The function app’s managed identity authenticates to both APIM and the downstream Azure services. Application Insights provides a unified view across all four patterns.

The spectrum from the Durable Functions post applies here too: RAG retrieval is mostly directed (the function knows which sources to query), but the agent layer that decides what to retrieve and how to use the results is autonomous. Azure Functions provides the event-driven, scalable retrieval infrastructure for both ends of that spectrum.

What to know before building

Choose your vector store before choosing your binding. The Azure OpenAI binding extension integrates cleanly with Azure AI Search. If you are using Cosmos DB for MongoDB vCore, a Postgres pgvector extension, or an external vector database, you write the retrieval logic directly using the SDK — the binding handles the LLM call, not the retrieval.

The embeddings binding and the chat completion binding have different cost profiles. Embeddings are cheap per token; completions are expensive. The ingestion pipeline (embeddings) can run at high volume with low cost. The retrieval + completion path (chat) is where token costs accumulate. Design your APIM quota policies accordingly.

Stateful chat sessions need external storage. The chat completion binding stores session history in Azure Table Storage by default. For multi-tenant RAG, partition session keys by user or tenant — the default key scheme does not enforce isolation automatically.

Retrieval quality determines output quality. Azure Functions handles the retrieval infrastructure; what you retrieve is your responsibility. This distinction matters more than it might seem.

With keyword search alone, a query like “List all Metallica albums” retrieves documents based on term frequency across the index. An index containing Radiohead (48 releases) and Porcupine Tree alongside Metallica will surface Radiohead and Porcupine Tree documents ahead of Metallica ones, because those bands have more indexed content matching common terms. The LLM then filters the irrelevant context and may miss albums that simply did not appear in the retrieved passages.

Hybrid search combining keyword retrieval with vector retrieval addresses this directly. Generate an embedding for the user query, add a VectorizedQuery targeting the content_vector field, and Azure AI Search fuses both signals using reciprocal rank fusion. Semantically relevant documents surface regardless of term frequency:

			
var vectorQuery = new VectorizedQuery(queryVector)
{
    KNearestNeighborsCount = 20,
    Fields = { "content_vector" }
};
var searchOptions = new SearchOptions
{
    Size = 20,
    VectorSearch = new VectorSearchOptions { Queries = { vectorQuery } }
};
// Hybrid: keyword query string + vector query sent together
var results = await searchClient.SearchAsync<MusicDocument>(
    userMessage, searchOptions, cancellationToken);

		

The companion repo implements hybrid search in MusicChatAgent.cs. Chunk size, embedding model choice, and whether you use keyword, semantic, or hybrid search have more impact on output quality than any infrastructure decision.

Testing the RAG pipeline

The companion repo supports several query patterns that exercise different aspects of the pipeline. These are worth running after ingesting a few bands to verify both the retrieval and the grounding behaviour.

Artist-specific ranking — tests hybrid search surfacing the right band:

			
{"message": "List all Metallica albums with their ratings, ordered highest to lowest"}

Date filtering via LLM reasoning — the index has no date filter; the model reasons over retrieved content:

{"message": "What albums did Tool release in the 1990s?"}

Cross-band comparison — tests whether retrieval surfaces relevant documents from multiple bands:

			
{"message": "Compare Opeth and Porcupine Tree — which has more critically acclaimed albums?"}

Rating threshold query:

{"message": "Which albums across all bands have a rating above 4.5?"}

Not-in-index grounding test — the agent should say it does not have information rather than hallucinate:

{"message": "Tell me about a band called Coldplay"}

Validation — should return HTTP 400:

{"message": ""}

The hybrid search implementation means artist-specific queries return only that artist’s documents. With keyword-only search, a query for Metallica in an index containing Radiohead (48 releases) would return mostly Radiohead documents — the vector component ensures semantic relevance wins over term frequency.

The companion repo — Music RAG Agent

The music-rag-agent companion repo implements the full two-pipeline RAG system described in this post. It scrapes band and album data from SputnikMusic, generates embeddings with the Azure OpenAI SDK, indexes into Azure AI Search, and exposes an HTTP chat endpoint that retrieves relevant passages and returns grounded answers.

After ingesting Tool, Opeth, Porcupine Tree, and Radiohead you can ask questions like:

“Which band has the highest rated albums — Tool, Opeth, Porcupine Tree or Radiohead?”

The retrieval correctly surfaces Sputnik ratings and vote counts for each album. The ingestion pipeline handles special characters in album titles (Ænima, Opiate²) via URL-safe Base64 document keys, and the EventGrid trigger ensures reliable delivery with 24-hour retry backoff.

Known Sputnik artist IDs: Tool = 83, Opeth = 932, Porcupine Tree = 328, Radiohead = 86.

Series wrap-up

This post completes the series on AI and Azure Functions. The five posts covered:

The four AI-enabled patterns and how to choose between them
Hosting MCP servers — binding extension (GA), self-hosted SDK (preview), and queue-based tools
The serverless agents runtime — markdown agents, the three configuration files, and dynamic workflows
Durable Functions for directed agentic workflows — five patterns with working C# samples verified on Azure
RAG pipelines — event-driven retrieval, the Azure OpenAI binding extension, and APIM governance

The companion code repositories are at github.com/steefjan1.

Azure Functions AI Integration: The Quiet Powerhouse

Posted on July 19, 2026 by steefjan1970

When people talk about running AI workloads on Azure, the conversation usually lands on Azure AI Foundry, Azure OpenAI Service, or maybe Azure Container Apps. Azure Functions tends to get mentioned as the glue the thing you bolt on to handle a webhook. But this overlooks the real story: Azure Functions AI integration has quietly evolved from simple glue into a powerhouse for running production-grade AI.

That framing is outdated. Azure Functions is now a first-class runtime for AI workloads, with four distinct patterns that the Microsoft Learn documentation lays out explicitly. This post walks through each of them and helps you decide which one fits your situation.

The four AI-enabled scenarios

Microsoft groups Azure Functions AI integration into four scenarios:

Serverless agents runtime — event-driven agents that run on serverless infrastructure
Tools and MCP servers — hosting remote Model Context Protocol servers and AI tools
Agentic workflows — multistep, long-running directed agent operations via Durable Functions
Retrieval-augmented generation (RAG) — fast, parallel data retrieval for knowledge-augmented AI

These are not just marketing buckets. Each one reflects a different architectural decision. Let me unpack them.

Azure Functions AI integration: Serverless agents runtime

The serverless agents runtime is a preview programming model for building event-driven agents as function apps. Moreover, Agents are defined in .agent.md files, app-wide runtime defaults live in agents.config.yaml, and remote MCP server connections are listed in mcp.json. The runtime discovers these files, registers the required triggers and endpoints, and runs the agent through the Microsoft Agent Framework when an event fires.

That is a meaningfully different model from what you get in Azure AI Foundry Agent Service. In Foundry, the managed service hosts and orchestrates your agents. In addition, in the serverless agents runtime, your function app is the agent host running on Flex Consumption, with built-in managed identity, monitoring, and scale-to-zero. Furthermore, you write custom Python tools for app-specific logic, and the platform wires in MCP-enabled connections based on Azure connectors and remote MCP servers.

Use this when: you want agents triggered by events, schedules, messages, or HTTP requests and you need the familiar Functions deployment and hosting model rather than a managed agent service.

Avoid it when: you need a fully managed, enterprise-grade agent service with built-in tooling and long-term Microsoft support guarantees today. That is what Foundry Agent Service is built for.

Azure Functions AI integration: Tools and MCP servers

The Model Context Protocol (MCP) has become the industry standard for how AI models and agents interact with external systems. Azure Functions has first-class support for hosting remote MCP servers, and this is already generally available.

There are two hosting options:

Option	Status	How it works
MCP binding extension	GA	Uses Functions triggers and bindings; supports stateful execution
Self-hosted MCP servers (MCP SDK)	Preview	Uses standard MCP SDKs via custom handlers; requires Streamable HTTP transport

The binding extension is the right default. It supports C#, Python, TypeScript, JavaScript, and Java, and it integrates with the Functions programming model you already know. Self-hosted MCP servers offer portability: you can use the official MCP SDKs and bring in existing server code. However, stateful execution is not yet supported, and the configuration is still changing during preview.

There is also a third option worth knowing about: queue-based Azure Functions tools, where AI agents interact with your code through message queues rather than direct MCP calls. Microsoft Foundry provides specific Azure Functions tooling for this pattern. It is ideal when you need reliable delivery, built-in retry, and decoupling between agent and function execution.

Use MCP servers when: you are exposing tools to AI clients and you want the industry-standard protocol with serverless hosting.

Use queue-based tools when: you need asynchronous, fault-tolerant communication between an agent and your function code.

Agentic workflows with Durable Functions

Not all AI orchestration should be autonomous. Some scenarios need predictable, directed steps and that is where Durable Functions fits.

The Microsoft Learn documentation makes the distinction clearly: Durable Functions is positioned as the runtime for directed agentic workflows, not for emergent agent reasoning. Think of it this way: when you know the sequence of steps and you need fault tolerance, auditability, and long-running execution, Durable Functions is the right tool. When you want a model to figure out the steps dynamically, you want an agent runtime.

The documentation gives a clean example: a trip planning workflow that gathers user requirements, searches for options, waits for approval, and makes bookings. Each step is a function; Durable Functions coordinates them with built-in retry, state persistence, and human-in-the-loop support.

Use this when: your AI-driven process has well-defined, ordered steps,authorization flows, multi-stage approval chains, or orchestrated data pipelines where you cannot afford unpredictable execution paths.

Avoid it when: you want a model to determine steps dynamically. That is the serverless agents runtime or Foundry Agent Service territory.

RAG with Azure Functions

Because Functions handles multiple events from various data sources simultaneously, it scales well for real-time AI scenarios, particularly RAG systems where fast, parallel retrieval is the bottleneck.

The Azure OpenAI binding extension lets you integrate RAG directly into your function code. Functions can pull data from multiple sources simultaneously, feed it through Azure AI Search or other retrieval layers, and pass the results to your language model, all within the event-driven, scale-to-zero model that keeps costs down when load is low.

The Azure Functions RAG pattern also pairs naturally with APIM, which handles routing, rate limiting, and token quota management a pattern the Citadel Platform series covers in detail, including the discovery that the Foundry Agent Service SDK bypasses APIM for LLM calls.

Use this when you have event-driven retrieval requirements new documents arriving in blob storage, database change feeds, or streaming IoT data that needs to inform model responses.

How the scenarios relate to other Azure services

It helps to think of Azure Functions AI integration as filling the compute and integration layer between your AI services and your data sources. Here is roughly how that maps:

Azure AI Foundry Agent Service — fully managed agent orchestration with enterprise security and built-in tools. Functions integrates into Foundry via MCP servers and queue-based tools.
Azure Logic Apps — low-code orchestration for business process automation. Functions is the right choice when you need custom code, complex event processing, or lower latency.
Azure Container Apps — container-based hosting for long-running services. Functions on Flex Consumption beats it on cost for bursty, event-driven AI workloads that spend time idle.
Durable Functions — lives inside Functions and adds stateful, long-running orchestration. Use it for directed agentic workflows; use the serverless agents runtime for event-driven agents.

The underlying platform advantage

Across all four scenarios, the same hosting model applies: Flex Consumption. It offers fast, event-driven scaling, virtual network integration, and pay-as-you-go billing. For AI workloads, which tend to be bursty rather than continuous, this is a significant cost advantage over always-on hosting.

Managed identity, Application Insights integration, and azd-based deployment are consistent across all four patterns. That means your security posture, observability, and deployment pipeline do not have to change when you move from a simple timer trigger to hosting a remote MCP server.

Azure Functions AI integration: Choosing the right pattern

Here is a simple decision table:

I want to…	Use…
Build event- or schedule-triggered agents with MCP tools	Serverless agents runtime (preview)
Expose tools to AI clients via the industry-standard protocol	MCP binding extension (GA)
Orchestrate predictable, multistep AI-driven processes	Durable Functions
Build a RAG pipeline with fast, parallel data retrieval	Azure Functions + Azure OpenAI binding
Fully managed agent hosting with enterprise SLAs	Azure AI Foundry Agent Service

What comes next

The rest of this series goes deep on each pattern. The next post covers the two MCP server hosting options in detail binding extension versus self-hosted SDK servers, including where the current preview constraints matter in practice.

Up next: Hosting Remote MCP Servers in Azure Functions: GA vs. Preview Options

Cloud Perspectives

Steef-Jan Wiggers

Tag Archives: RAG