I spent some time observing what’s inside the Microsoft AI stack. Foundry, Agent Framework, Logic Apps, AI Search, Purview, Entra. After a while, I had a picture of how the pieces fit and decided to draw one myself to share.
Then recently Microsoft published AI-500, an expert certification for multi-agent systems. That made me curious whether Microsoft’s view of the platform matches my own. So I plotted the certification trail against my diagram. This post shows the result.
The Microsoft AI stack as I see it
Six layers. Five of them stack vertically. The sixth runs down the side, through all the others.
Models sit at the bottom: GPT-5, Claude, Mistral, Grok, Microsoft’s own MAI and Phi, Llama, DeepSeek, and the open catalog. This is the least differentiated layer. In Foundry, swapping one model for another is a configuration change. It gets the most attention and deserves the least.
Infrastructure comes next. Foundry is the hub, alongside Azure OpenAI, Azure ML, AKS, Container Apps, App Service, and Foundry Local for the edge. This is hosting, serving, and compute. Solid and well understood.
Data and context: the real moat
I split the data layer in two, because it hides the most important part of the platform. Layer 3a holds the sources of truth: Microsoft Graph, SharePoint, Exchange, Fabric and OneLake, Dataverse, and the vector stores in Cosmos DB, Azure SQL, and PostgreSQL.
Layer 3b is context. Work IQ, Fabric IQ, Foundry IQ, and Azure AI Search turn enterprise data into grounding that respects who is asking. Permission-trimmed retrieval must honor Entra ACLs at query time, not filter after ranking. AI Search supports this through document-level access control. Get it wrong and answers leak, or recall collapses.
This is where the hard engineering hours go. Swapping a model is a config change. Graph-grounded context is months of work.
Agents, Copilots, and the layer vendors leave out
The agentic platform sits above the data. Microsoft Agent Framework merged Semantic Kernel and AutoGen. Next to it sit Foundry Agent Service, Copilot Studio, Logic Apps, Azure Functions, Service Bus, and Event Grid. MCP and A2A handle tools and agent-to-agent communication.
The Copilot layer is on top: Microsoft 365, GitHub, Security, Dynamics 365, Power Platform, and Teams. This is distribution. These are the surfaces people already live in. As a result, enterprise AI adoption is easier when data, permissions, infrastructure, and applications already exist in one ecosystem.
Then comes the sixth layer of the Microsoft AI stack, the one vendor diagrams leave out: governance and evidence. Entra ID and Entra Agent ID handle identity. Purview covers labels, DLP, and audit. Content Safety and API Management provide guardrails and the AI gateway. Defender, Sentinel, Azure Monitor, Log Analytics, and Azure Policy deliver traces, evaluations, and control evidence.
I work for a regulated organization. Before anything goes live, Legal and Internal Audit ask four questions. Who approved the agent? What data did it access? Which controls applied? What happened when it made a wrong decision? Distribution makes deployment easier. However, without traceable evidence it does not make the system production ready. That is why this layer runs vertically through my drawing.
The Microsoft AI certification trail
I knew AI-900. I had not followed what replaced it. This week I learned that Microsoft now offers a full set of AI certifications, from fundamentals to an expert exam. The expert exam, AI-500, is in beta. I read the study guide. Its scope says a lot: orchestration patterns, agent-to-agent protocols, observability, guardrails, and cost control. Architecture work, end to end.
What made it click was laying the whole trail side by side. Each step has its own verb.
- AI-901, Azure AI Fundamentals. Understand the concepts and services.
- AI-103, AI Apps and Agents Developer. Build applications and agentic solutions on Foundry.
- AI-200, Azure AI Cloud Developer. Engineer cloud-native AI properly.
- AI-300, ML Operations Engineer. Operate models in production.
- GH-300 and GH-600, Copilot and Agentic AI Developer. Ship with agents working beside you.
- AI-500, Multi-Agent AI Solutions Expert. Orchestrate systems of agents at scale.
Plotting the trail on the Microsoft AI stack
Six exams, six layers of the Microsoft AI stack. I mapped each exam to its layers, in its study guide exercises, and its center of gravity.
The fundamentals exam touches every layer at concept depth. AI-103 lives in models, infrastructure, context, and the agentic platform. AI-200 moves down into infrastructure and data. AI-300 sits on infrastructure and the evidence plane, because operating models in production is mostly monitoring and evaluation. Meanwhile, the GitHub exams live at the top, where developers meet agents in the editor.
AI-500 is the interesting one. It spans context, the agentic platform, and governance. It does not test models at all. In fact, three of its five headline topics belong to the plane most stack diagrams leave out.
That was the moment the two pictures agreed. My diagram says the hard part of the Microsoft AI stack is context and evidence, not model choice. Microsoft’s expert exam tests context and evidence, not model choice. I did not expect a certification roadmap to confirm an architecture opinion, but here we are.
Where this is the wrong answer
Do not read the trail as a ladder you must climb in order. If you already run agents in production, AI-500 reflects your work, and AI-901 will teach you nothing. Platform engineers should look at AI-200 and AI-300 first. Developers should start with the GitHub exams.
Also, do not read my layer mapping as Microsoft’s. It is my reading of the study guides, and beta study guides move.
Finally, do not confuse the certificate with the evidence. Passing AI-500 shows you know what a control looks like. It does not produce the audit trail for your agent. That still takes Purview configured, Entra Agent ID issued, traces flowing to Log Analytics, and someone signing off. The exam is a map of the work. The work is still the work!
Credits
The Azure icons come from the official Azure architecture icon set. Product marks belong to their owners. Microsoft’s announcements are on the Skills Hub blog: Multi-Agent AI Solutions Expert, AI Apps and Agents Developer Associate, and GitHub Agentic AI Developer. The stack diagram builds on a five-layer picture that circulated on LinkedIn; the split data layer and the governance plane are my additions.