Azure Foundry IQ Deep Dive: One Retrieval Endpoint to Replace Your Custom RAG Pipeline
Azure Foundry IQ reached General Availability at Microsoft Build 2026 as the dedicated knowledge and retrieval layer for Foundry agents. The core proposition is a single SLA-backed retrieval endpoint that unifies five distinct knowledge sources: Work IQ (Microsoft 365 data), Fabric IQ (Power BI and OneLake data), Azure AI Search, custom data sources, and web grounding. One endpoint. One SLA. One SDK call.
For the enterprise teams I work with, the dominant approach to grounding agents in organisational knowledge has been custom RAG pipelines — custom chunking, custom embedding, custom retrieval, custom reranking. Foundry IQ is a direct challenge to that approach. Here is the technical breakdown of what it offers and where it falls short.
The Retrieval Architecture
Foundry IQ operates through three internal components that developers do not manage directly:
- Query planner: Analyses the incoming query and determines which of the five knowledge sources are relevant. A question about a SharePoint document routes to Work IQ. A question about sales figures routes to Fabric IQ. A question about a recent news event routes to Web IQ.
- Parallel source retrieval: Relevant sources are queried concurrently, not sequentially. This is the primary latency optimisation — retrieving from three sources in parallel is faster than retrieving from each in sequence.
- Unified ranker: Results from all sources are merged and reranked by a single model that understands relevance across source types. This is the component that makes Foundry IQ more than a routing layer — it produces a single ranked result set rather than separate ranked lists per source.
The Five Knowledge Sources
- Work IQ: Microsoft 365 data — SharePoint, Teams messages, Outlook, OneDrive. Scoped by Microsoft Entra identity.
- Fabric IQ: Power BI semantic models, OneLake datasets, Fabric warehouses. Analytics and structured data.
- Azure AI Search: Custom vector and keyword indexes — your existing AI Search indexes are first-class citizens in the unified endpoint.
- Custom data sources: Any MCP-exposed data source can be registered as a Foundry IQ source.
- Web IQ: Live web grounding with sub-200ms latency, achieved through cached crawls and partnership indexing agreements.
Why the Unified Ranker Is the Real Differentiation
Custom RAG pipelines that pull from multiple sources face a fundamental problem: how do you merge ranked lists from sources with different relevance scoring models? A cosine similarity score from an Azure AI Search vector index is not directly comparable to a BM25 score from a keyword search. Most custom implementations handle this with ad-hoc heuristics or separate retrieval paths that the model must reconcile in its context window.
Foundry IQ's unified ranker is trained specifically to produce a single coherent relevance ranking across all source types. This means developers get a clean, prioritised result set without building cross-source reconciliation logic.
Web IQ Sub-200ms Architecture
The sub-200ms web grounding claim deserves scrutiny. Traditional web search grounding involves a live crawl-and-index cycle that cannot achieve sub-200ms latency. Foundry IQ achieves this through a combination of pre-cached crawls of high-frequency domains, partnership agreements with Bing indexing, and result caching for recent queries. Live crawl for novel URLs will not hit sub-200ms — the latency guarantee applies to the indexed web content, not arbitrary URLs.
Honest Trade-offs
- Foundry lock-in: Foundry IQ is deeply integrated with Azure — migrating retrieval logic to another platform requires rebuilding the pipeline
- Opaque ranker: The unified ranker is a black box — you cannot inspect or customise its relevance signals
- MCP supply chain: Custom data sources via MCP introduce third-party dependencies into your retrieval architecture
- Cold starts for serverless tier: The serverless tier (in public preview) has cold start latency that makes it unsuitable for latency-sensitive production workloads
- Web IQ limitations: Sub-200ms applies to indexed content, not arbitrary live web retrieval
Key Takeaways
- Foundry IQ unifies five enterprise knowledge sources behind a single retrieval endpoint — the most significant reduction in RAG pipeline complexity available on Azure today
- The unified ranker is the architectural differentiator — cross-source relevance ranking without custom reconciliation logic
- Web IQ sub-200ms applies to indexed web content; live crawl for novel URLs will not achieve this latency
- For organisations already invested in Microsoft 365 and Fabric, the Work IQ and Fabric IQ integration delivers immediate value without custom pipeline work
- Teams with highly customised retrieval requirements should evaluate whether the opaque ranker and Foundry lock-in are acceptable trade-offs before committing
