Implementing Multi-Agent System for Your Fabric Data Agents
Ever wished you could have a team of specialized advisors at your fingertips, each ready to answer your toughest data questions? With Microsoft Fabric Data Agents and Microsoft Foundry, you can make this a reality. By leveraging multi-agent routing, you’ll have a smart, all-purpose knowledge agent, similar to a hotline to a board of experts, ready to assist with insights across your business domains, but they never rest, they never complain, and complement your highly skilled workforce.
Data First
Start by ensuring your data landscape is ready. This can range from “simple” to “advanced”: creating semantic models to organize your key business data, or standing up a full-fledged data platform within Microsoft Fabric. The goal is to make your data discoverable and actionable for agents. If you’re just getting started, focus on building clean, well-structured semantic models. For more advanced needs, consider integrating data lakes, warehouses, and pipelines. Make sure that every column, every table, every data model includes a description and as much metadata as possible. In the agentic era, this is even more important than data itself.
Your Crew: Fabric Data Agents per Business Domain
Fabric Data Agents are your domain experts. Set up one agent for each major business domain — think Sales, Finance, Operations, HR, and so on, and connect each domain with its related semantic model. System prompt, role, and agent description are key in this part. Each agent will have access to their own relevant datasets and models, allowing them to answer questions specific to their area of expertise. Consider that RLS (Row Level Security) and other profile security settings can be incorporated into agents, hence into the final solution.
Deploy Microsoft Foundry in Azure
Set up Microsoft Foundry in your Azure environment. This involves creating a resource group, selecting a subscription, and establishing a new Foundry resource or project. Foundry acts as your agent platform, enabling orchestration, routing, and advanced AI-powered capabilities that tie your data agents together.
Foundry for Multi-Agent Features
Here’s a pro tip: The new Foundry doesn’t yet support the connected multi-agent features needed for this solution (yes, I know…). So, make sure you’re working with the “Old” Foundry version. You can easily toggle to this in the Azure portal or Foundry interface. This unlocks the multi-agent routing capabilities you’ll need.
Within Foundry, add each of your Fabric Data Agents as a connection tool. You can do this directly from the Create Agent UI or by configuring each agent’s connection settings. This step allows Foundry to interact with each domain agent and route queries appropriately.
For every Fabric Data Agent, create a corresponding Foundry agent. Assign the relevant Fabric agent as its knowledge base and instruct it to always use its associated Fabric agent for all incoming questions. This ensures each Foundry agent acts as a proxy, leveraging the deep knowledge contained in your Fabric data models.
Orchestrator Agent
Create a top-level orchestrator agent in Foundry. This is a new agent, sometimes called the ‘Manager Agent’ or the ‘Orchestrator Agent’. This agent is responsible for directing the question to the appropriate agent based on the user prompt. Add all your domain-level Foundry agents as “Connected agents.” This is where the multi-agent routing comes into play. The orchestrator receives incoming questions, determines which domain experts are best suited to each query, and routes them accordingly.
Make sure to instruct your orchestrator to always involve at least one domain agent for every question, and to prompt the underlying agents to use their Fabric knowledge tools. This hierarchical setup ensures robust, context-aware responses, enabling your orchestrator to draw on specialized expertise as needed.
Publish and Start Using Your Solution
Once your agents and orchestrator are in place, publish your Foundry solution. It is possible to publish as a standalone web application or as a Teams Agent. You now have a powerful, multi-agent knowledge system at your disposal! Users can interact with the orchestrator as they would with a human advisor, and the orchestrator will seamlessly route questions to the appropriate domain experts.
Conclusion: A Low-Effort, High-Value Solution
By following these steps, you’ve implemented a versatile, scalable multi-agent routing system on top of Microsoft Fabric and Foundry. This approach offers an effective way to deliver expert-level insights across your organization, with far less effort and complexity than custom-coding similar solutions from scratch. It’s a smart, adaptable foundation for knowledge management, giving you a hotline to a virtual boardroom of AI-powered advisors.
