With services like Azure AI Foundry, you can not only access generative AI, but build your own agents, tailor-made for your use cases. Creating these custom AI experiences requires data—lots of it. Data is the foundation on which AI is built, and the simple fact is AI is only as good as the data it’s based on. As you enter a future built on AI, you need a data estate capable of fueling AI innovation across your organization. This can be a challenging prospect for most organizations whose data environments have grown organically over time with specialized and fragmented solutions.

That’s why we introduced Microsoft Fabric and Microsoft OneLake, Fabric’s unified data lake. With OneLake, you can access your entire multi-cloud data estate from a single data lake that spans the entire organization. OneLake can act as the central, accessible location for comprehensive data access and management.​ And once connected to OneLake, your teams can use the array of data and analytics tools in Fabric to integrate, transform, model, and prepare your data for any AI project—all in a pre-integrated and optimized SaaS environment.

Today we are going to focus on why Fabric and OneLake are the ideal data tools to fuel your AI projects in AI Foundry. First, we will talk through how you can unify your data estate on OneLake, then cover how Fabric’s workloads can help you prepare your data for AI projects. Finally, we’ll show you how easy it is to connect OneLake to Azure AI Foundry so you can start building data-driven agents in seconds.

Unifying your data estate on OneLake

For teams tasked with building new AI solutions, finding and accessing the necessary data across a sea of disconnected data services can be challenging at the best of times. To lay the foundation for long-term success, organizations need a more unified, flexible data estate based on a lake-centric approach. The right data lake foundation can help you unify all of your multi-cloud sources and allow your data professionals to work from the same data—reducing data duplication, improving collaboration, and streamlining analysis.

OneLake is designed as the single point to discover and explore data for everyone in your entire organization. You can unify all of your multi-cloud and on-premise sources using zero ETL shortcuts and mirroring in OneLake without data duplication or movement. Alternatively, you can leverage the 180+ connectors in Fabric Data Factory to move your data in from any other source. OneLake is automatically wired into every Fabric workload and since data is stored in an open format, you can use data in OneLake for all your data projects, no matter the vendor or service. You can also save time and reduce data duplication by loading data into OneLake only once and using a single copy across every Fabric engine and even other engines like from Snowflake.

Once enabled in OneLake, you can use domains and the OneLake catalog to organize your data into a logical data mesh and empower everyone to easily explore, manage, and govern their data. Take a look at the OneLake catalog:

Preparing and curating your data for AI projects

For generative AI solutions to be as accurate as possible, they need to be built with clean data and in a semi-structured way. You’ll need an analytics platform that can help you prepare your data before building custom AI experiences. ​With your data in OneLake, you can use Fabric’s various workloads to make the data AI-ready. Fabric has tools for data integration and engineering, data warehousing, data science, real-time analytics, data modeling and visualization, and even has native, industry-specific and partner-created workloads to help you accelerate your data projects.

All Fabric workloads work together seamlessly out-of-the-box without the myriad of infrastructure and configuration settings you typically find in data platforms, so you can focus on getting results. Advanced security, governance, and continuous integration and continuous delivery (CI/CD) capabilities are woven into the platform with personalized experiences for admins and users alike. Copilot in Fabric and other AI capabilities are built into every layer of Fabric to help data professionals and business users automate routine tasks and get more done. Fabric also comes with category-leading performance, instant scalability, shared resilience, and built-in security, governance, and compliance so you can feel confident using Fabric for your mission-critical workloads.

Connecting OneLake data to AI products

Now that your data is AI-ready, you need to connect it to your AI platforms like Azure AI Foundry to build and scale data-driven GenAI apps. We’ve built native integration between OneLake and Azure AI Foundry to make this as seamless as possible. Azure AI Foundry can operate directly on OneLake, opening endless possibilities for AI and app developers, data engineers, data scientists, and business users to interact using natural language to uncover insights from their data.

Azure AI Foundry

Azure AI Foundry is a platform designed to empower your developers, AI engineers, and IT professionals to customize, host, run, and manage AI solutions with greater ease and confidence. Similar to Fabric, Azure AI Foundry’s unified approach simplifies the development and management process, helping all stakeholders focus on driving innovation and achieving strategic goals. It’s designed to help your developers build more technical, customized AI solutions.

The integration between Azure AI Foundry and OneLake is built on the same shortcut technology that allows you to virtualize data in OneLake from your cloud sources like Amazon S3 and Google Cloud without having to move and duplicate the data. You can immediately work with your structured and unstructured data from OneLake in Azure AI Foundry without creating copies and adding more data sprawl. OneLake also directly integrates with Azure AI Search, which can store, index, and retrieve data, including vector embeddings, from your data sources including OneLake. 

Finally, you can ground your Azure AI Agent’s responses with data from Fabric using Fabric data agents to unlock powerful data analysis capabilities. Data agents (formally known as AI skills) in Fabric are AI-powered assistants that can learn, adapt, and deliver insights, allowing users to interact with the data through chat. With out-of-the-box authorization, this integration simplifies access to enterprise data in Fabric while maintaining robust security, ensuring proper access control and enterprise-grade protection. Check out this full demo:

https://www.youtube.com/embed/SBsErGew1yE?feature=oembedUsing data agents in Fabric as knowledge sources in Azure AI Foundry

Popular applications for data-driven agents

This seamless integration offers many opportunities for generative AI use cases across various industries, including:

• Enhancing data insights: Build agents that can help your business users explore and better understand critical data using natural language from structured, unstructured, and real-time data.
• Analyzing customer interactions: Build agents trained on your customer interaction data to enhance customer service, tailor support responses, and make data-driven decisions. These agents can detect language, summarize content, analyze sentiment, and convert insights into vector embeddings for future access in search queries.
• Customizing machine learning models: Tailor models to specific business needs, whether it’s predictive maintenance, fraud detection, or customer sentiment analysis. Azure AI Foundry, Azure Machine Learning, and Microsoft Fabric empower developers and data scientists to create custom models that fit their business requirements, grounded on their enterprise data in OneLake.
• Department-specific agents: Build agents that automate budget and expenses, increase up-sell and conversion opportunities, and improve operational efficiency
• Industry-specific agents: Build data-driven agents to streamline operations and manage OEE in manufacturing, optimize logistics and interact with customers in retail, and reduce patient-practitioner contact time in healthcare.

Ready to learn more?

Unlock a realm of new possibilities for your organization in the era of AI with the integration of Microsoft Fabric and Azure AI. Explore the potential, innovate, and thrive in the new digital landscape.

 If you want to learn more about these tools, consider:

• Signing up for the Microsoft Fabric free trial.
• Visiting the Microsoft Fabric website.
• Visiting the Azure AI Foundry website.

The post Build data-driven agents with curated data from OneLake appeared first on Microsoft Fabric Blog.

Microsoft Fabric has embraced this mission by integrating multiple products and services needed for an end-to-end analytics and AI solution, redefining existing processes to make them simpler and more intuitive. It has significantly simplified how one interfaces with such a comprehensive solution by creating a turnkey software-as-a-service experience that is easy to use with a much simpler and singular capacity usage model.

At Ignite 2024, Fabric took another bold step forward by adding operational databases to the Fabric portfolio with SQL database in Fabric. Adding operational data alongside Fabric’s analytical OLAP (Online Analytics Processing) data and real-time streaming data (RTI) opens a host of new scenarios for AI agentic applications. It also provides our customers with a unified data estate where consistent security and governance policies can be applied.

SQL database in Fabric leverages the proven mission-critical SQL Server database engine. It applies the core principles of Fabric to make deploying and managing an operational database simpler, more autonomous, secure by default, and optimized for AI. For example, deploying and configuring a database only requires a name, and the database is ready in seconds. It is secure by default with encryption at rest and in transit enabled. Networking security is also enabled via Private Link, and high availability and zone redundancy are automatically configured. 

SQL in Fabric includes native AI capabilities like support for vector and RAG (Retrieval-augmented Generation). You can also make calls directly to Azure AI services from the database and connect your database to Azure AI Foundry, VSCode, and GitHub for an integrated developer experience. In addition, you will find Microsoft Copilot integrated into every workload in Fabric including SQL in Fabric, simplifying administrative and management tasks for the databases. 

Beyond just the OLTP (Online Transaction Processing) database, Fabric introduces new agentic AI application scenarios by providing access to real-time streaming data from IoT sensors, alongside your system of record with SQL in Fabric and other data sets securely stored in OneLake. 

OneLake is at the heart of enabling a unified data estate. OneLake is built on top of Azure Data Lake Storage (ADLS) Gen2 and can support any type of file, structured or unstructured. All Fabric data items like data warehouses and lakehouses store their data automatically in OneLake in Delta Parquet format. With the addition of SQL database in Fabric you now also have access to your mirrored SQL data in OneLake and mirroring data in Fabric is free. 

Fabric also provides a rich ecosystem to support agentic AI applications using your operational data. Changes from your data can be seamlessly sent to Azure OpenAI for business recommendations using Fabric Real-time Intelligence Eventstream, Spark, OneLake, and Power BI. 

This unification of data is incredibly powerful, enabling dynamic improvements to customer prompt responses and proactive, personalized offers. From a security standpoint, Fabric can enable consistent data protection from when the data is born to business insights via PowerBI. The same goes for data governance. 

This is just the tip of the iceberg when it comes to the number of new scenarios that Fabric can enable by creating a unified data estate. SQL database in Fabric is just the first Azure Database to be added to Fabric, with more Azure Databases to follow, so stay tuned.

Get started today

SQL database in Fabric is simple, autonomous, secure, and optimized for AI. We highly encourage you to try it today and see how you can build new AI apps faster and easier than ever! 

Learning with Fabric

We have multiple resources to help you and your teams swiftly ramp up on SQL database in Fabric: 

• Explore the documentation and build with our end-to-end tutorials.

• Skill up with the new Fabric Databases learn modules.

• Watch new Data Exposed episodes.

• Watch the new Microsoft Mechanics episode.

• Get the latest product news here on the Fabric updates blog.

• Check out an AI chatbot application example built on SQL database in Fabric. 

Fabric Community Conference Vegas: A must-attend event for database professionals! Be sure to take advantage of the discount code MSCUST for $150 off the registration price. 

FabCon Vegas is the perfect opportunity to connect with experts and data leaders to build your skills with Fabric Databases and Azure Databases and see how your peers are implementing their solutions. 

• Microsoft Fabric Community Conference March 31^st – April 2^nd, in Vegas! Workshops will also be available on March 29th, 30th, and April 3rd, making this the most comprehensive Microsoft Fabric learning experience to date.

• SQL pros can take advantage of a dedicated track for SQL in Fabric Databases and Azure Databases. 

• Connect with product specialists for 1:1 support in the Ask the Experts area. 

• You’ll get endless opportunities all week to engage with the Fabric and data communities through sessions, thoughtful discussions, attendee mixers, and interactive activations. 

• In touch with your Microsoft account team? Ask them if they have any special discounts to share.

Database experts at FabCon

• CVP of Azure Databases: Shireesh Thota, speaking at the event¹.

• Sessions from the Microsoft Databases Product team: Rie Merritt, Bob Ward, Mazuma Zahid, Erin Stellato, Davide Mauri, and more.

• Sessions from Database Community MVPs: Joey D’Antoni, John Morehouse, Monica Rathbun, Denny Cherry, Karen Lopez, Anthony Nocentino, Erwin de Kreuk, Warwick Rudd, Kelly Broekstra, Heidi Hasting, and Hamish Watson.

Microsoft Fabric

Unify your teams and data to accelerate AI innovation with a complete data platform

Try today

¹speakers subject to change

The post The art of simplifying the complex: Microsoft Fabric’s superpower appeared first on Microsoft Fabric Blog.

A Leader in Data Integration 

We feel that Microsoft’s acknowledgment in the Gartner Magic Quadrant reflects our dedication to innovation and customer-centric solutions. This stems from our relentless drive to advance technology and address the ever-evolving needs of modern organizations.

Our vision for data integration is to deliver seamless, intuitive experiences that empower businesses to unlock the full potential of their data and achieve transformative results. This recognition reinforces our dedication to leading the evolution of data integration and delivering unparalleled value to our customers and partners worldwide.

Microsoft Fabric

Give your teams the AI-powered tools they need for any data project—including workloads tailored to your industry

Try today

Microsoft Fabric: Unified Data Platform for the Era of AI 

At the core of our data integration strategy is Microsoft Fabric. Built to navigate the complexities of modern data ecosystems, Microsoft Fabric provides an all-in-one, software-as-a-service (SaaS) platform with AI-powered services to handle any data project—all within a pre-integrated and optimized environment. It enables organizations to unlock their data’s full potential, drive innovation, and make smarter decisions. Features like Copilot and other generative AI tools introduce new ways to transform and analyze data, generate insights, and create visualizations and reports in Microsoft Fabric.

Microsoft OneLake: The heart of our Data Integration journey 

At the center of our Fabric is OneLake, the unified, open data lake that simplifies and accelerates data integration across diverse systems. OneLake, with the data integration capabilities of Fabric, is designed to help you simplify data management and reduce data duplication. OneLake’s open data format means you only need to load the data into the lake once and you can use the single copy across every Fabric workload and engine. It acts as the central hub, ensuring seamless connectivity, accessibility, and collaboration for all your data needs. 

OneLake has four innovative pathways for integrating data depending on your needs: 

1. Fabric Data Factory 

Fabric Data Factory integrates seamlessly with OneLake, offering powerful cloud-scale services for data movement, orchestration, transformation, deployment, and monitoring. These capabilities enable organizations to tackle even the most complex ETL (Extract, Transform, and Load) scenarios, unifying data estates, streamlining operations, and unlocking the full potential of their data.

2. Multi-Cloud Shortcuts

OneLake shortcuts allow you to virtualize data into OneLake from across clouds, accounts, and domains—all without duplication, movement, or changes to metadata or ownership. This capability allows organizations to access and analyze their data in place, without the need for complex data migration processes. By maintaining a live connection to the source, OneLake ensures real-time data availability and consistency across all integrated environments. You can shortcut data from Azure Data Lake Service, S3-compatible sources, Iceberg-compatible sources, Google Cloud Platform, Dataverse, and more.

3. Database Mirroring 

OneLake offers an innovative zero-ETL approach to database mirroring, simplifying the replication of operational databases into the lake. This capability minimizes the effort required to synchronize databases, supporting real-time changes and ensuring that data is always current and ready for analytics and reporting.

4. Real-Time Intelligence 

Real-time intelligence in Microsoft Fabric empowers organizations to ingest and process streaming and high granularity data instantaneously, driving real-time insights and automating decision-making. This solution is ideal for applications requiring immediate data updates, such as IoT analytics, fraud detection, and operational dashboards. The capability extends to highly granular data analytics, allowing businesses to track a single package within a global delivery network or monitor a specific component in a manufacturing machine across a fleet of factories worldwide, enabling precise insights and optimized operations. Leveraging cutting-edge data processing frameworks, Eventhouse ensures scalability, reliability, and low-latency performance, making it suitable for high-volume streaming scenarios.

With these innovative pathways, Fabric empowers organizations to break down data silos, optimize workflows, and unlock the full potential of their data. Whether it’s through seamless data integration, real-time insights, or multi-cloud collaboration, Fabric is designed to meet the demands of modern data ecosystems. These enriched features position Fabric as a critical tool for organizations aiming to unlock the full potential of their data while maintaining simplicity, security, and scalability.

Customer success stories 

Our customers’ success stories are a testament to the impact of Microsoft Fabric. Organizations across various industries have leveraged our data integration capabilities to unlock new opportunities, drive innovation, and achieve their business goals. By streamlining data processes and improving data quality, Microsoft Fabric has enabled these businesses to make data-driven decisions with confidence. 

Read UST Global’s case study to learn how they leveraged the power of Fabric to migrate over 20 years of data, integrating disparate data sources to facilitate better collaboration and innovation among employees. 

Looking ahead: The future of Data Integration with Microsoft Fabric 

As we celebrate being recognized as a Leader in the Gartner Magic Quadrant for the fourth consecutive year in a row, we are motivated to push the boundaries of what’s possible in data integration. To us, this is a milestone that reflects not only our commitment to innovation but also our dedication to empowering our customers to turn their data into actionable insights.

Looking forward, the roadmap for Microsoft Fabric is filled with exciting enhancements and new features. These advancements are designed to tackle the complexities of modern data ecosystems, making it even easier for organizations to unify, transform, and harness their data at scale. Continuous improvement is at the core of our strategy. We aim to remain at the forefront of the data integration landscape and redefine the possibilities of what a comprehensive data platform can achieve. 

We believe this recognition by Gartner is a validation of the trust our customers place in us and a reflection of our relentless drive to deliver world-class solutions. As we continue this journey, we remain committed to collaborating with our community and partners, building on this success to achieve even greater outcomes together.

Resources 

• Read the full complimentary report from Gartner. 
• Data Factory in Microsoft Fabric  
• Data Factory Overview 
• Real Time Intelligence 
• Transforming Your Business With Real-Time Intelligence 
• Getting Started with Microsoft Fabric  
• What’s new and planned for Data Factory in Microsoft Fabric  
• FAQ 

Gartner, Magic Quadrant for Data Integration Tools, By Thornton Craig, Sharat Menon, Robert Thanaraj, Michele Launi, Nina Showell, 3 December 2024 

Gartner does not endorse any vendor, product, or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. 

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, Magic Quadrant is a registered trademark of Gartner, Inc. and/or its affiliates and is used herein with permission. All rights reserved. 

This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. Available here.

The post Microsoft: A leader in the 2024 Gartner Magic Quadrant report appeared first on Microsoft Fabric Blog.

Consider this: over 97% of Fortune 500 companies rely on Microsoft Power BI and many of these organizations seek to enhance Power BI’s capabilities by extending it for planning and Hybrid Transactional/Analytical Processing (HTAP) workloads. However, traditional approaches to planning and reporting exacerbate silos: 

1. Historical insights feeding future plans: Organizations rely on historical performance data to inform future plans. This process requires transferring actual transactional data into separate planning systems, adding time and complexity. 

2. From planning back to reporting: Once budgets and forecasts are finalized, they must be reloaded into Power BI for variance analysis and performance reporting—a tedious and redundant process. 

Planning is rarely a one-time exercise. It evolves through multiple iterations, scenarios, and assumptions, requiring inputs from diverse teams, departments, and geographies. This fragmented approach necessitates back-and-forth data orchestration, creating inefficiencies and compounding the issue of data silos. 

Craig Schiff, Founder and CEO of BPM Partners, summarized it perfectly: “Planning solutions fully integrated with existing BI software is an underserved area that is growing in importance.” 

Microsoft’s vision for planning 

At Microsoft, our vision is clear: eliminate silos and empower organizations to plan and report seamlessly within a unified platform—Microsoft Fabric. 

With Fabric, enterprises no longer need to replicate data into separate software as a service (SaaS) or legacy planning systems. Instead, users can build plans and forecasts directly on top of the semantic models in Power BI, ensuring immediate availability for reporting and analysis across Fabric. 

Microsoft Fabric and Lumel: partnering for success 

We are excited to announce a deep collaboration with Lumel that brings Enterprise Performance Management (EPM) for planning applications to Power BI and Microsoft Fabric. 

Lumel’s no-code, self-service EPM solution enables business users to create sophisticated planning and reporting applications within Power BI, tightly integrated with Fabric. This approach delivers: 

• Seamless integration: Build and modify plans directly in Power BI, leveraging Fabric semantic models. 

• Collaborative capabilities: Commenting, notifications, scheduling, and approval workflows streamline team collaboration. 

• Broader use cases: Support for transactional and analytical scenarios expands Fabric’s utility for modern HTAP workloads and planning use cases. 

Lumel’s solution redefines the boundaries of what organizations can achieve with Power BI and Fabric. For example, with Lumel’s Inforiver Write-Back Matrix for Planning, businesses can effortlessly create their 2025 plans, save them to OneLake, and integrate them instantly into reporting and analysis workflows. 

Our strong collaboration with Lumel reinforces Microsoft’s commitment to providing a single, unified platform where planning and analytics coexist, helping organizations make better, faster decisions. 

Looking ahead with Microsoft Fabric 

Take the next step towards connected planning—streamlining  workflows, eliminating silos, and unlocking new possibilities with Microsoft Fabric and Lumel. 

Lumel and Microsoft Fabric

Enterprise planning and performance management in Power BI and Fabric

Watch here

Stay tuned for more updates as we continue to empower organizations with cutting-edge tools for the future of planning and analytics. 

The post Planning in Microsoft Fabric: A shared vision through collaboration with Lumel  appeared first on Microsoft Fabric Blog.

With Fabric, we focused on simplicity, openness, and autonomy. All Fabric workloads work together seamlessly out-of-the-box without the myriad of infrastructure and configuration settings you typically find in data platforms so you can focus on getting results. You can ingest structured and unstructured data in any format into OneLake’s open Delta Parquet format and even access third-party tools from industry leading software companies built directly into Fabric. Advanced security, governance, and continuous integration and continuous delivery (CI/CD) capabilities are woven into the platform with personalized experiences for admins and users alike. Microsoft Copilot and other AI capabilities are built into every layer of Fabric to help data professionals and business users automate routine tasks and get more done. In fact, we’ve found that users were 52% faster in completing standard data analysis tasks and uncovered insights 36% more accurately when using Copilot in Fabric, with 90% saying they were likely to adopt Copilot in Fabric.¹  

Fabric’s vision for a data platform has highly resonated with the industry, and more than 17,000 customers, including 70% of the Fortune 500, are already using Fabric to empower their data teams.

• Melbourne Airport, the second busiest in Australia, used Fabric to analyze their operational data in real-time and gained 30% increased performance efficiency across data-related operations. “It’s a radical and powerful new technology that can feel just like using Microsoft Excel or Power BI. But once in the hands of the user, it doesn’t feel like a new, complex technology at all,” Irfan Khan, Head of Data and Analytics.  
• Chanel, a world leader in luxury fashion, adopted Fabric not only to drive more value from its data and support their AI innovation, but also safeguard its data at rest and in-transit with Fabric’s end-to-end, built-in security, governance, and reliability. “We chose Microsoft Fabric as the foundation of this platform, driven by its ability to implement a data mesh approach,” Olivier Barbonnat, Chief Information Officer Europe.  
• Our own Microsoft IDEAS (insights, data, engineering, analytics, systems) team, one of the largest data teams in the world, transitioned to Fabric to support its AI ambitions. Its solution now encompasses 27,000 data sources, 420 petabytes of data, 35,000 data pipelines, 38,000 semantic models, and more than 600 teams relying on its models. The IDEAS team estimated it has received a 50% efficiency boost from consolidating assets in OneLake, using modern tools such as Spark and Python, Direct Lake mode in Microsoft Power BI, and AI-assisted coding through IDEAS Copilot.

Schaeffler, Hitachi Solutions, KPMG, Epic, and many other customers have seen a transformational impact to how they process data. You can explore all these Fabric stories on the Microsoft Customer Stories page. One of the reasons Fabric caught the imagination of so many is because, with Fabric, you can simplify and future-proof your data estate. Fabric’s capabilities and workloads will continue to expand and be seamlessly infused into our pre-integrated platform, helping you keep up with the technology trends without added work.  

And since launching Fabric, we’ve added new ways to bring data into Fabric with capabilities like mirroring and new shortcut sources. We’ve expanded Copilot in Fabric across almost every experience to help everyone automate routine tasks. We’ve added a multitude of security and governance features to help you make sure your data is secure at every step of its journey. We’ve added the ability to extend Fabric further with native, industry-specific workloads from Microsoft and other software developers. And most impactfully, we launched a new workload to help organizations make better decisions from Internet of Things (IoT), logs, and telemetry data—Real-Time Intelligence.  

With Fabric Real-Time Intelligence, we transformed Fabric into a platform equipped to support your operational scenarios and data in motion. And now, we’re helping you bring transactional scenarios to Fabric with the introduction of Fabric Databases. 

Introducing a unified data platform with Fabric Databases

Currently, the data and AI technology market is massively fragmented with hundreds of vendors and thousands of services. We believe the future of data and AI is the convergence of all your data services into a unified, open, and extensible platform, so you no longer have to manually stitch together disconnected services.  

Today, we’re thrilled to announce a major leap toward this goal with Fabric Databases, now in preview. Fabric Databases represent a new class of cloud databases that brings a world-class transactional database natively to Microsoft Fabric for app developers. With the addition of Fabric Databases, Fabric now brings together both transactional and analytical workloads, creating a truly unified data platform. Developers can streamline application development with simple, autonomous, and AI-optimized databases that provision in seconds and are secured by default with features like cloud authentication and database encryption. Built-in vector search, retrieval augmented generation (RAG) support, and Azure AI integration simplify AI app development, and your data is instantly available in OneLake for advanced analytics. Developers can even use Copilot in Fabric to translate natural language queries into SQL and get inline code completion alongside code fixes and explanations.

SQL database, the first available in Fabric, was built on our industry-leading SQL Server engine and the simple and intuitive SaaS platform of Fabric. In fact, data professionals who’ve tried SQL database in Fabric were able to complete common database tasks up to 71% faster and with 63% more effective task completion. They reported feeling 84% more confident in these tasks and finding the tasks up to 91% less difficult.²  These results were even more pronounced for people who were newer to cloud. Those with less than two years of cloud platform experience benefited the most in terms of efficiency and effectiveness, highlighting the simplicity and intuitiveness of Fabric Databases. 

SQL database is just the beginning for Fabric Databases, with more databases on the roadmap. Whether you’re an experienced data professional or just getting started, you can build AI apps faster and more confidently on Fabric Databases.

To learn more, read the Fabric Databases blog post, watch the Microsoft Mechanics deep dive video, and watch the following sizzle video:

General availability of Fabric Real-Time Intelligence

We’re also thrilled to announce Real-Time Intelligence is now generally available. With Real-Time Intelligence, you get both pro-dev and no-code tools to ingest high-volume streaming data with high granularity, dynamically transform streaming data, query data in real-time for instant insights, and trigger automated actions based on the data. The Real-time hub provides a central place to discover and manage all your streaming data. Dener Motorsport, a participant in the annual Porsche Carrera Cup Brasil event, used Real-Time Intelligence for in-race analytics, and their CEO, Dener Pires, said “Before we used Microsoft Fabric and Real-Time Intelligence, it was probably 30 minutes before the engineers knew that something was wrong with a car, could get the data, analyze it, and provide a solution. Today that process is done in minutes.” Check out this blog post and the following demo to see Real-Time Intelligence in action: 

OneLake catalog—a complete catalog for discovery, management, and governance

No matter what data project you’re trying to accomplish, it starts with the right foundation. OneLake, Fabric’s unified, multi-cloud data lake, is built for everyone in your entire organization as the single point to discover and explore your data. With OneLake shortcuts and mirroring, you can unify all of your multi-cloud and on-premise sources and enable your people to work from the same data—meaning fewer copies of data, better collaboration between your teams, and easier, more streamlined analysis. And since data is stored in an open format, you can use data in OneLake for all your data projects, no matter the vendor or service.  

Today, we’re excited to announce the OneLake catalog, a complete solution to explore, manage, and govern your entire Fabric data estate. The OneLake catalog comes with two tabs, Explore and Govern, that can help all Fabric users discover and manage trusted data, as well as provide governance for data owners with valuable insights, recommended actions, and tooling. Since the OneLake catalog is an evolution of the OneLake data hub, it already shows up in Microsoft 365, such as in Excel and Microsoft Teams and many other products in the Microsoft cloud for easy data consumption. OneLake catalog value can be extended to the Microsoft Purview data governance solution, Unified Catalog, which offers the data office, data stewards, and data owners advanced governance capabilities, including data quality and a global catalog for the heterogeneous data estate. The Explore tab is now generally available, and the Govern tab will be coming soon in preview.  

Learn more about the OneLake catalog by reading this blog post and by watching the following demo:

More Fabric innovation

The introduction of Fabric Databases and the growing opportunity with generative AI in accelerating data projects has encouraged us to reimagine the pillars of Fabric. We are now focused on making sure Fabric can provide you with: 

• An AI-powered data platform. Fabric can give your teams the AI-powered tools needed for any data project in a pre-integrated and optimized SaaS environment. You can even extend Fabric further by adding other native workloads from the Workload Hub, created by industry-leading partners.  
• An open and AI-ready data lake. Fabric can help you access your entire multi-cloud data estate from a single data lake, work from the same copy of data across analytics engines, and ensure your data is ready to power AI innovation.  
• AI-enabled business users. Fabric can empower everyone to better understand your data with AI-enhanced Q&A experiences and visuals embedded in the Microsoft 365 apps they use every day. 
• A mission-critical foundation. You can confidently deploy and manage Fabric with category-leading performance, instant scalability, shared resilience, and built-in security, governance, and compliance. 

Check out the new Fabric sizzle video to see these pillars in action: 

We’re excited to share a huge slate of announcements designed to help us better accomplish each goal above. These enhancements include: 

Fabric workload enhancements

• The general availability of sustainability data solutions in Microsoft Fabric, a single place for all your environmental, social, and governance (ESG) data needs. Julie Nikulina, IT Solutions Engineer at Schaeffler AG, a global automotive and industrial supplier, mentioned that, “thanks to Microsoft Fabric, we’ll be able to answer lots of questions about climate neutrality and decarbonization company-wide via a single platform—and we can implement new use cases in short sprints within two to six weeks.”  
• Coming soon, the preview of AI functions in Fabric notebooks, which provide a simplified API for common AI text enrichments like summarization, translation, sentiment analysis, and more. 
• The general availability of API for GraphQL, which is an API to help you access data from multiple sources in Fabric with a single query API. 
• The preview of several enhancements to Fabric Real-Time Intelligence, which include new Fabric events, enhancements to Eventstreams and Eventhouses, and easier real-time dashboard sharing. 
• The preview of the Copilot in Fabric experience for data pipelines in Fabric Data Factory. 
• The preview of our integration with Esri ArcGIS for advanced spatial analytics. 

Microsoft OneLake enhancements

• The preview of open mirroring, a feature that allows any application or data provider to write change data directly into a mirrored database within Fabric. 
• The general availability of Azure SQL DB mirroring and the preview of SQL MI mirroring. 

New AI capabilities in Fabric

• Coming soon, the preview of AI skill enhancements, including a more conversational experience and support for semantic models and Eventhouse KQL databases. 
• Coming soon, the preview of AI skill integration with Agent Service in the newly announced Azure AI Foundry, allowing developers to use AI skills as a core knowledge source. 

Platform-wide enhancements

• The preview of workspace monitoring, which provides detailed diagnostic logs for workspaces to troubleshoot performance issues, capacity performance, and data downtime. 
• The general availability of the Workload Development Kit, created to help software developers design, build, and interoperate applications within Fabric. We’re excited to see many of our industry-leading partners announce preview of their workload hub offerings, including Quantexa, SAS, Teradata, Osmos, Esri, and Profisee. 
• The preview of further integration with Microsoft Purview including extending Protection policies to enforce access permissions to more sources and using Data Loss Prevention policies to restrict access to semantic models with sensitive data. 
• The general availability of external data sharing allows you to directly share OneLake tables and folders with other Fabric tenants in an easy, quick, and secure manner. 
• Fabric is FedRAMP High certified for the Azure Commercial cloud, the highest level of compliance and security standards required by the federal government for cloud service providers. Now government agencies can run Fabric on the Azure Commercial cloud while maintaining strict compliance. 

You can learn more about all of these announcements and so much more in the Fabric November 2024 Update blog post and the numerous blog posts that will go live throughout this week on the Fabric blog channel.  

Fabric billing and consumption updates

Finally, we’re making some important changes to Fabric’s billing model. First, coming soon, organizations with multiple capacities can now direct Copilot in Fabric consumption and billing to a specific capacity, no matter where the Copilot in Fabric usage actually takes place. Admins can assign specific members of their organization to the specified F64 or higher capacity for all of their Copilot requests. These requests will be consumed and billed on that assigned F64+ capacity, ensuring Copilot in Fabric usage doesn’t impact priority jobs while expanding Copilot access to any workspace regardless of its capacity.

Additionally, we’re providing capacity admins with more control over the Fabric jobs running in their capacities. Surge protection, now in preview, helps protect capacities from unexpected surges in background workload consumption. Admins can use surge protection to set a limit on background activity consumption, which will prevent background jobs from starting when reached. Admins can configure different limits for each capacity in your organization to give you the flexibility to meet your needs.

Watch Fabric in action at Microsoft Ignite

Join us at Microsoft Ignite 2024 from November 19 to November 21, 2024 to see all of these announcements in action across the following sessions:  

• “Microsoft Fabric: What’s new and what’s next” with Arun Ulagaratchagan and Amir Netz
• “Power AI apps with unified data and insights in Microsoft Fabric” with Panos Antonopoulos, Anna Hoffman, and Asad Khan 
• “Ingest, govern, and secure your data with OneLake” with Josh Caplan, Wilson Lee, and Adi Regev 
• “Unlock the power of Real-Time Intelligence in the era of AI” with Yitzhak Kesselman, Tessa Kloster, Niraj Revankar, Vijay Sankaran, and Ksenia Suresh. 

And six other Fabric breakout sessions. You can also join us at labs and theater sessions throughout the event. Find all the data-related sessions at Ignite. You can also learn about other announcements across our Azure portfolio by reading these blogs by Jessica Hawk and Omar Khan. 

• The next wave of Azure innovation: Azure AI Foundry, intelligent data, and more: We’re bringing a huge range of innovation to our AI platform including the new Azure AI Foundry, a unified platform designed to empower both developers and business decision-makers in the age of AI. 
• Scale your AI transformation with a powerful, secure, and adaptive cloud infrastructure: We’re also announcing cloud and AI infrastructure announcements, from advancements in chips and liquid cooling to new data integrations and more flexible cloud deployments.  

Finally, if you want more strategic guidance to help you along your data and analytics journey in the era of AI, you should watch the recent Data and Analytics Forum.

Getting started with Microsoft Fabric

New customers can try out everything Fabric has to offer by signing up for a free 60-day trial—no credit card information required. Learn how to start your free trial.  

If you’re considering purchasing Fabric and need help deciding on a SKU, we’re excited to share a new Fabric SKU estimator, now in private preview. You can sign up to try out this tool as part of the early adopter program—try the SKU estimator.

Start building your Fabric skills

Be one of the first to start using Fabric Databases

Ready to build reliable, highly scalable applications where cloud authentication and encryption are secured by default? Starting December 3, 2024, join live sessions with database experts and see just how easy it is to get started. View the schedule and register for the series.

Get certified in Microsoft Fabric—for free

Get ready to fast-track your career by earning your Microsoft Certified: Fabric Analytics Engineer Associate certification. For a limited time, we’re offering 5,000 free DP-600 exam vouchers to eligible Fabric community members. Complete your exam by the end of the year and join the ranks of certified experts. Don’t miss this opportunity to get certified. 

A new Fabric certification for data engineers

We’re excited to announce a brand-new certification for data engineers. The new Microsoft Certified: Fabric Data Engineer Associate certification will help you demonstrate your skills with data ingestion, transformation, administration, monitoring, and performance optimization in Fabric. To earn this certification, pass Exam DP-700: Implementing Data Engineering Solutions Using Microsoft Fabric, currently in beta.

Join us at the 2025 Microsoft Fabric Community Conference

Looking to gain hands-on experience with Fabric and learn directly from the people who created it? If so, join us from March 29 to April 3, 2025, at the Microsoft Fabric Community Conference in Las Vegas, Nevada. Register today. 

Explore additional resources for Microsoft Fabric

If you want to learn more about Fabric:  

• Sign up for the Fabric free trial. 
• Register for the Microsoft Data and Analytics Forum. 
• View the updated Fabric Roadmap. 
• Visit the Fabric website. 
• Join the Fabric community. 
• Read other in-depth, technical blogs on the Microsoft Fabric Updates Blog. 
• Explore the Fabric technical documentation.

Read additional blogs by industry-leading partners:

• “Data Governance in Microsoft Fabric with Microsoft Purview,” by CloudThat Technologies Pvt. Ltd.
•  “Key Fabric Announcements and Overview of Fabric Databases,” by Telefónica Tech.
• “Ignite 2024 – Microsoft Fabric announcements: the continued convergence of data platform and analytics to deliver intelligent insights,” by BJSS. 
• “Architecting tomorrow—A Microsoft Fabric Blueprint to intelligent enterprise covering Fabric Databases, AI Skills, Fabric and Microsoft Purview, Apache Iceberg support in OneLake,” by Avasoft Inc.
• “Microsoft Fabric Databases,” “Microsoft Fabric’s AI Skills,” and “Data Engineering: Liquid Clustering in Fabric,” by Verne Tech.
• “A First Look at Fabric Mirroring for SQL Managed Instance,” by Coeo Ltd.
• “Leveraging Microsoft Fabric for Dynamics 365 with Dataverse,” and “Automating Data Workflows in Microsoft Fabric: A Low-Code/No-Code Approach,” by PreludeSys.
• “Enabling self-service for supply chain analytics with Microsoft Fabric and Direct Lake,” “Transforming data operations with Microsoft Fabric and CI/CD for ISVs,” and “How Microsoft Fabric transformed data analytics for an ISV with Real-Time Intelligence,” by MAQ Software.
• “Real Time Intelligence for the Rest of Us with Microsoft Fabric,” by iLink Digital.       

¹Based upon n=209 user studies conducted by Microsoft Corporation in October 2024 that measured four common metrics associated with the consumption experience of Power BI in Microsoft Fabric. Qualitative sentiment gathered upon task completion. The actual results may vary. 

²Based upon n=210 user studies conducted with technical practitioners by Microsoft Corporation in October 2024 that measured time to complete four common tasks associated with AI application development on a SQL database in Microsoft Fabric and on Azure SQL Database. Actual results may vary based upon individual performance and sentiment.

The post Accelerate app innovation with an AI-powered data platform appeared first on Microsoft Fabric Blog.

The European Microsoft Fabric Community Conference 2024 has officially come to a close and we hope it left attendees, organizers, and volunteers feeling inspired and excited to apply their learnings. This year’s conference was held in the beautiful city of Stockholm and brought together our amazing community of Microsoft Fabric users, practitioners, and industry leaders from across Europe and beyond for four days of immersive learning and networking. The conference was a celebration of innovation, collaboration, and the power of community. With more than 3,000 attendees from 61 countries, the conference was a celebration of innovation, collaboration, and the power of community.

Let’s take a look at some of the top moments from the event. 

Wednesday Keynote 

The conference kicked off with a warm attendee welcome and an action-packed keynote delivered by Arun Ulag, Amir Netz, and experts from across the Fabric team. Arun and Amir set the stage for an exciting event and shared the latest capabilities coming to the Fabric platform. These announcements included OneLake shortcuts for Iceberg tables (preview), Azure Databricks Unity Catalog mirroring into OneLake (preview), new Copilot in Fabric experiences across workloads (preview and general availability), deeper integration with Microsoft Purview, and so much more. Read Arun’s Fabric Community Conference blog for more information on the announcements.  

Thursday Keynote

Each day of the conference began with a new Microsoft-led keynote, and in the Day 2 session, I led off with our inspiring vision for how Microsoft is making it easier to apply AI at scale with an end-to-end stack across data, analytics, and AI. I then introduced Marco Casalaina, who shared a vision for the future of AI and Fabric’s role in revolutionizing how users interface with their data to drive actionable insights and hold two-way conversations with their data. 

Friday Keynote

I was honored to deliver the first part of the Day 3 keynote, where I was joined onstage by Fabric customers to uncover their real-world challenges and opportunities in establishing a data culture that will allow them to take full advantage of AI innovations at scale. In part two, Rudra Mitra took the stage to discuss security and governance and demonstrated how Microsoft Purview and Fabric are enabling users to secure their data across a single, unified data estate.  

Community-led content 

The conference offered a diverse range of sessions and workshops for beginners through to seasoned professionals. Speakers guided attendees through topics spanning data integration, machine learning and AI, security and governance, and advanced analytics. During the pre-conference hands-on workshops, participants had the opportunity to gain practical experience with the latest tools and techniques. 

Connections and networking 

At the heart of the event was of course the Fabric community. Over the course of the conference, there were endless opportunities for attendees to connect with like-minded professionals to share experiences, challenges, and collaborate on new ideas.  

The highlight for many was getting to meet other Fabric users in the community lounge, exhibit hall, networking events, or even casually in the convention center hallways going to and from sessions. There was a feeling of community that underscored the event, making it a truly memorable experience for those in attendance. 

Continue your learning experience 

Just because European Microsoft Fabric Community Conference 2024 has ended, it doesn’t mean your learning has to stop there. 

Data and AI Forum (digital event) 

Join us at an upcoming event designed to help data leaders navigate the current data and AI landscape—the Data and AI Forum. You will hear from Microsoft experts, partners, and customers on the latest trends and industry best practices to help you stay ahead of the curve.  

This digital event will include a keynote session hosted by Jessica Hawk that outlines the current data landscape now with AI and a fireside chat with customers to see how they have been able to prepare their data estate to take advantage of AI. After the keynote there will be three breakout sessions on establishing an AI-ready data foundation, governing data and AI, and a session hosted by me on how you can fuel your data culture with generative AI.

Microsoft Ignite 2024 (in-person and online) 

One of the biggest events on the Microsoft calendar is right around the corner and you can join us either in person in Chicago, Illinois or online. Stay one step ahead of the newest Microsoft innovations and join fellow professionals who are ready to upskill and collaborate.

Microsoft Fabric Community Conference 2025 (in-person) 

Missed the European Microsoft Fabric Community Conference 2024? Join us in Las Vegas, Nevada in 2025 for the next Microsoft Fabric Community Conference event and reconnect with the Fabric community for a week of sessions and networking.

Explore additional resources for Microsoft Fabric  

Microsoft Fabric

Reshape how everyone accesses, manages, and acts on data with a single, AI-powered platform

Explore solutions

If you want to learn more about Microsoft Fabric:  

• Sign up for the Microsoft Fabric free trial.
• Visit the Microsoft Fabric website.
• Join the Fabric community. 
• Read other in-depth, technical blogs on the Microsoft Fabric Updates Blog.
• Explore the Fabric technical documentation.
• Register for the upcoming Microsoft Data and Analytics Forum.
• View the updated Microsoft Fabric Roadmap.
• Explore the Fabric technical documentation.

The post Highlights from the European Microsoft Fabric Community Conference 2024 appeared first on Microsoft Fabric Blog.

Microsoft is excited to announce an expanded partnership with Snowflake, marking a significant step forward in our commitment to providing customers with a seamless data experience. This will enable bi-directional data access between Snowflake and Microsoft Fabric, ensuring more efficient and flexible data management. 

With the launch of Fabric, we introduced OneLake—a single, unified, software as a service (SaaS) open data foundation, establishing our commitment to open standards. We are now taking our commitment to open standards further by including support for Iceberg, in addition to Delta Lake in Microsoft Fabric OneLake. This means that applications that support Iceberg (including Snowflake) or Delta Lake can interoperate with Fabric. This empowers our customers with diverse environments to work across applications and platforms, with OneLake providing an open data lake foundation for unifying their data estate.

Microsoft Fabric

Reshape how everyone accesses, manages, and acts on data with a single, AI-powered platform

Explore the benefits

What are we announcing? 

Microsoft and Snowflake are working to enable the following capabilities, which will light up over the coming months: 

• OneLake will support Iceberg, alongside Delta Lake. This will be done using the Apache XTable project that both Microsoft and Snowflake are committed to. 
• Snowflake will be able to store data in Iceberg format in OneLake. 
• Data written by either platform, Snowflake or Fabric, will be available in both Iceberg and Delta Lake formats through XTable translation in OneLake. 
• Snowflake will be able to read any Fabric data artifact in OneLake, stored physically or virtually through shortcuts. 
• Snowflake data items will light up experiences in any of the Fabric engines as well as Microsoft 365, Microsoft Azure AI Studio, and Copilot for Microsoft 365. 

What does it mean for you? 

With this integration, you will be able to work with a single copy of your data across Snowflake and Fabric, regardless of whether the data is stored using Delta Lake or Iceberg format. This enables a number of compelling scenarios: 

• Bi-directional data access across Fabric and Snowflake ensures that data can be analyzed across any of the engines in Fabric, or in Snowflake, depending on the customer’s needs.  
• You will also be able to access your data through your favorite apps such as Microsoft 365, Copilot for Microsoft 365, and Studio. 
• With OneLake data accessible in both Snowflake and Fabric, you can reduce the time spent integrating applications and organizing your data estate, reduce the costs of duplicating data across various locations, and dedicate more time to discovering insights. 

“We’ve heard from many of our mutual customers that they want increased interoperability across Snowflake and Fabric in a seamless, cost-efficient manner. As both Snowflake and Microsoft are firm believers in open standards, we see our joint support for Apache Iceberg as an opportunity to offer customers choice when it comes to how and where they interact with their data.” 

—Christian Kleinerman, Executive Vice President of Product at Snowflake  

Unify your data estate 

The rationale behind this expanded partnership is clear: to give our customers an edge in harnessing their digital investments by helping them simplify and unify their data estate with OneLake in Fabric. By combining the strengths of Microsoft Fabric and Snowflake, two leading data platforms, we are further expanding our commitment to open standards, and empowering our customers to do more with their data with the Microsoft Intelligent Data Platform. 

Learn the latest about Fabric in our blog “Unlock real-time insights with AI-powered analytics in Microsoft Fabric.”

The post Snowflake and Microsoft announce expansion of their partnership appeared first on Microsoft Fabric Blog.

With Microsoft Fabric, we are simplifying and future-proofing your data estate with an ever-evolving, AI-powered data analytics platform. Fabric will keep up with the trends for you and seamlessly integrate each new capability so you can spend less time integrating and managing your data estate and more time unlocking value from your data.  

Get started with Microsoft Fabric

Set up Fabric for your business and discover resources that help you take the first steps

Try for free

Aurizon, Australia’s largest rail freight operator, turned to Fabric to modernize their data estate and analytics system.

“With Microsoft Fabric, we’ve answered many of our questions about navigating future growth, to remove legacy systems, and to streamline and simplify our architecture. A trusted data platform sets us up to undertake complex predictive analytics and optimizations that will give greater surety for our business and drive commercial benefits for Aurizon and our customers in the very near future.”

—Tammy Wigg, Chief Data Analytics Officer at Aurizon

Aurizon is just one among thousands of customers who have already used Fabric to revolutionize how they connect to and analyze their data. In fact, a 2024 commissioned Total Economic Impact™ (TEI) study conducted by Forrester Consulting found that Microsoft Fabric customers saw a three-year 379% return on investment (ROI) with a payback period of less than six months. We are thrilled to share a huge range of new capabilities coming to Fabric. These innovations will help you more effectively uncover insights and keep you at the forefront of the trends in data and analytics. Check out a quick overview of the biggest changes coming to Fabric.

Fabric is a complete data platform

Fabric is a complete data platform—giving your data teams the ability to unify, transform, analyze, and unlock value from data from a single, integrated software as a service (SaaS) experience. We are excited to announce additions to the Fabric workloads that will make Fabric’s capabilities even more robust and even customizable to meet the unique needs of each organization. These enhancements include: 

1. A completely redesigned workload, Real-Time Intelligence, that brings together and enhances Synapse Real-Time Analytics and Data Activator to help you analyze and act on high-volume, high-granular event streaming data and even explore your organization’s real-time data in the new Real-time hub.

2. New tools like the Fabric Workload Development Kit, Application Programming Interface (API) for GraphQL™, and “user data functions” that can help developers build powerful solutions on the Fabric platform. 

3. A new feature in the Microsoft Azure Data Factory experience called Data workflow, powered by Apache Airflow runtime, that can help you author, schedule, and monitor workflows or data pipelines using Python. 

Unlock continuous insights with Real-Time Intelligence and the Real-time hub

When we introduced Fabric, it launched with seven core workloads which included Synapse Real-time Analytics for data streaming analysis and Data Activator for monitoring and triggering actions in real-time. We are unveiling an enhanced workload called Real-Time Intelligence that combines these workloads and brings an array of additional new features, in preview, to help organizations make better decisions with up-to-the-minute insights. From ingestion to transformation, querying, and taking immediate action, Real-Time Intelligence is an end-to-end experience that enables seamless handling of real-time data without the need to land it first. With Real-Time Intelligence, you can ingest streaming data with high granularity, dynamically transform streaming data, query data in real-time for instant insights, and trigger actions like alerting a production manager when equipment is overheating or rerunning jobs when data pipelines fail. And with both simple, low-code or no-code, and powerful, code-rich interfaces, Real-Time Intelligence empowers every user to work with real-time data. 

Behind this powerful workload is the Real-time hub, a single place to discover, manage, and use event streaming data from Fabric and other data sources from Microsoft, third-party cloud providers, and other external data sources. Just like the OneLake data hub makes it easy to discover, manage, and use the data at rest, the Real-time hub can help you do the same for data in motion. All events that flow through the Real-time hub can be easily transformed and routed to any Fabric data store and users can create new streams that can be discovered and consumed. From the Real-time hub, users can gain insights through the data profile, configure the right level of endorsement, set alerts on changing conditions and more, all without leaving the hub. While the existing Real-Time Analytics capabilities are still generally available, the Real-time hub and the other new capabilities coming to the Real-Time Intelligence workload are currently in preview. Watch this demo video to check out the redesigned Real-Time Intelligence experience:  

Elcome, one of the world’s largest marine electronics companies, built a new service on Fabric called “Welcome” that helps maritime crews stay connected to their families and friends.

Microsoft Fabric Real-Time Intelligence has been the essential building block that’s enabled us to monitor, manage, and enhance the services we provide. With the help of the Real-time hub for centrally managing data in motion from our diverse sources and Data Activator for event-based triggers, Fabric’s end-to-end cloud solution has empowered us to easily understand and act on high-volume, high-granularity events in real-time with fewer resources.”

—Jimmy Grewal, Managing Director of Elcome

Real-time insights are becoming increasingly critical across industries like route optimization in transportation and logistics, grid monitoring in energy and utilities, predictive maintenance in manufacturing, and inventory management in retail. And since Real-Time Intelligence comes fully optimized and integrated in a SaaS platform, adoption is seamless. Strathan Campbell, Channel Environment Technology Lead at One NZ—the largest mobile carrier in New Zealand—said they “…went from a concept to a delivered product in just two weeks.” To learn more about the Real-Time Intelligence workload, watch the “Ingest, analyze and act in real time with Microsoft Fabric” Microsoft Build session or read the Real-Time Intelligence blog.  

Extend Fabric with your own, custom workloads and experiences

Fabric was built from the ground up to be extensible, customizable, and open. Now, we are making it even easier for software developers and customers to design, build, and interoperate applications within Fabric with the new Fabric Workload Development Kit—currently in preview. Applications built with this kit will appear as a native workload within Fabric, providing a consistent experience for users directly in their Fabric environment without any manual effort. Software developers can publish and monetize their custom workloads through Azure Marketplace. And, coming soon, we are creating a workload hub experience in Fabric where users can discover, add, and manage these workloads without ever leaving the Fabric environment. We already have industry-leading partners building on Fabric including SAS, Esri, Informatica, Teradata, and Neo4j.

You can also learn more about the Workload Development Kit by watching the “Extend and enhance your analytics applications with Microsoft Fabric” Microsoft Build session.

We are also excited to announce two new features, both in preview, created with developers in mind: API for GraphQL and user data functions in Fabric. API for GraphQL is a flexible and powerful RESTful API that allows data professionals to access data from multiple sources in Fabric with a single query API. With API for GraphQL, you can streamline requests to reduce network overheads and accelerate response rates. User data functions are user-defined functions built for Fabric experiences across all data services, such as notebooks, pipelines, or event streams. These features enable developers to build experiences and applications using Fabric data sources more easily like lakehouses, data warehouses, mirrored databases, and more with native code ability, custom logic, and seamless integration. You can watch these features in action in the “Introducing API for GraphQL and User Data Functions in Microsoft Fabric” Microsoft Build session.

You can also learn more about the Workload Development Kit, the API for GraphQL, user data functions, and more by reading the Integrating ISV apps with Microsoft Fabric blog.

Orchestrate complex data workflows in the Fabric Data Factory workload

We are also announcing the preview of Data workflows in Fabric as part of the Data Factory experience. Data workflows allow customers to define Directed Acyclic Graphs (DAG) files for complex data workflow orchestration in Fabric. Data workflows is powered by the Apache Airflow runtime and designed to help you author, schedule and monitor workflows or data pipelines using python. Learn more by reading the data workflows blog.  

Fabric is lake-centric and open

The typical data estate has grown organically over time to span multiple clouds, accounts, databases, domains, and engines with a multitude of vendors and specialized services. OneLake, Fabric’s unified, multi-cloud data lake built to span an entire organization, can connect to data from across your data estate and reduce data duplication and sprawl.  

We are excited to announce the expansion of OneLake shortcuts to connect to data from on-premises and network-restricted data sources beyond just Azure Data Lake Service Gen2, now in preview. With an on-premises data gateway, you can now create shortcuts to Google Cloud Storage, Amazon S3, and S3 compatible storage buckets that are either on-premises or otherwise network-restricted. To learn more about these announcements, watch the Microsoft Build session “Unify your data with OneLake and Microsoft Fabric.”  

Empower business users with Fabric

Insights drive impact only when they reach those who can use them to inform actions and decisions. Professional and citizen analysts bridge the gap between data and business results, and with Fabric, they have the tools to quickly manage, analyze, visualize, and uncover insights that can be shared with the entire organization. We are excited to help analysts work even faster and more effectively by releasing the model explorer and the DAX query view in Microsoft Power BI Desktop into general availability.

The model explorer in Microsoft Power BI provides a rich view of all the semantic model objects in the data pane—helping you find items in your data fast. You can also use the model explorer to create calculation groups and reduce the number of measures by reusing calculation logic and simplifying semantic model consumption. 

The DAX query view in Power BI Desktop lets users discover, analyze, and see the data in their semantic model using the DAX query language. Users working with a model can validate data and measures without having to build a visual or use an additional tool—similar to the Explore feature. Changes made to measures can be seamlessly updated directly back to the semantic model. 

To learn more about these announcements and others coming to Power BI, check out the Power BI blog.  

Fabric is AI-powered

When ChatGPT was launched, it had over 100 million users in just over two months—the steepest adoption curve in the history of technology.¹ It’s been a year and a half since that launch, and organizations are still trying to translate the benefit of generative AI from novelty to actual business results. By infusing generative AI into every layer of Fabric, we can empower your data professionals to employ its benefits, in the right context and in the right scenario to get more done, faster.  

Use Copilot in Fabric, now generally available 

Copilot in Fabric was designed to help users unlock the full potential of their data by assisting data professionals to be more productive and business users to explore their data more easily. With Copilot in Fabric, you can use conversational language to create dataflows, generate code and entire functions, build machine learning models, or visualize results. We are excited to share that Copilot in Fabric is now generally available, starting with the Power BI experience. This includes the ability to create stunning reports and summarize your insights into narrative summaries in seconds. Copilot in Fabric is also now enabled on-by-default for all eligible tenants including Copilot in Fabric experiences for Data Factory, Data Engineering, Data Science, Data Warehouse, and Real-Time Intelligence, which are all still in preview. The general availability of Copilot in Fabric for the Power BI experience will be rolling out over the coming weeks to all customers with Power BI Premium capacity (P1 or higher) or Fabric capacity (F64 or higher). 

We are also thrilled to announce a new Copilot in Fabric experience for Real-Time Intelligence, currently in preview, that enables users to explore real-time data with ease. Starting with a Kusto Query Language (KQL) Queryset connected to a KQL Database in an Eventhouse or a standalone Azure Data Explorer database, you can type your question in conversational language and Copilot will automatically translate it to a KQL query you can execute. This experience is especially powerful for users less familiar with writing KQL queries but still want to get the most from their time-series data stored in Eventhouse. 

Create custom Q&A experiences with your data with AI skills 

We are also thrilled to release a new AI capability in preview called AI skills—an innovative experience designed to provide any user with a conversational Q&A experience about their data. AI skills allow you to simply select the data source in Fabric you want to explore and immediately start asking questions about your data—even without any configuration. When answering questions, the generative AI experience will show the query it generated to find the answer and you can enhance the Q&A experience by adding more tables, setting additional context, and configuring settings. AI skills can empower everyone to explore data, build and configure AI experiences, and get the answers and insights they need.  

AI skills will honor existing security permissions and can be configured to respect the unique language and nuances of your organization, ensuring that responses are not just data-driven but steeped in the context of your business operations. And, coming soon, it can also enrich the creation of new copilots in Microsoft Copilot Studio and be interacted with from Copilot for Microsoft for 365. It’s about making your data not just accessible but approachable, inviting users to explore insights through natural dialogue, and shortening the time to insight.

New partnerships with Microsoft Fabric

Snowflake and Microsoft Fabric

With the launch of Fabric, we’ve committed to open data formats, standards, and interoperability with our partners to give our customers the flexibility to do what makes sense for their business. We are taking this commitment a step further by expanding our existing partnership with Snowflake to expand interoperability between Snowflake and Fabric’s OneLake. We are excited to announce future support for Apache Iceberg in Fabric OneLake and bi-directional data access between Snowflake and Fabric. This integration will enable users to analyze their Fabric and Snowflake data written in Iceberg format in any engine within either platform, and access data across apps like Microsoft 365, Microsoft Power Platform, and Microsoft Azure AI Studio.

With the upcoming availability of shortcuts for Iceberg in OneLake, Fabric users will be able to access all data sources in Iceberg format, including the Iceberg sources from Snowflake, and translate metadata between Iceberg and Delta formats. This means you can work with a single copy of your data across Snowflake and Fabric. Since all the OneLake data can be accessed in Snowflake as well as in Fabric, this integration will enable you to spend less time stitching together applications and your data estate, and more time uncovering insights. To learn more about this announcement, read the Fabric and Snowflake partnership blog.

Adobe and Microsoft Fabric 

We are also excited to announce we are expanding our existing relationship with Adobe. Adobe Experience Platform (AEP) and Adobe Campaign will have the ability to federate enterprise data from Fabric. Our joint customers will soon have the capability to connect to Fabric and use the Fabric Data Warehouse for query federation to create and enrich audiences for engagement, without having to transfer or extract the data from Fabric. 

Combine Fabric and Microsoft Azure Databricks to get the best of both worlds

We are excited to announce that we are expanding the integration between Fabric and Azure Databricks—allowing you to have a truly unified experience across both products and pick the right tools for any scenario. 

Azure Databricks Unity Catalog integration with Fabric 

Coming soon, you will be able to access Azure Databricks Unity Catalog tables directly in Fabric, making it even easier to unify Azure Databricks with Fabric. From the Fabric portal, you can create and configure a new Azure Databricks Unity Catalog item in Fabric with just a few clicks. You can add a full catalog, a schema, or even individual tables to link and the management of this Azure Databricks item in OneLake—a shortcut connected to Unity Catalog—is automatically taken care of for you.  

This data acts like any other data in OneLake—you can write SQL queries or use it with any other workloads in Fabric including Power BI through Direct Lake mode. When the data is modified or tables are added, removed, or renamed in Azure Databricks, the data in Fabric will remain always in sync. This new integration makes it simple to unify Azure Databricks data in Fabric and seamlessly use it across every Fabric workload. 

Federate OneLake as a Remote Catalog in Azure Databricks 

Also coming soon, Fabric users will be able to access Fabric data items like lakehouses as a catalog in Azure Databricks. While the data remains in OneLake, you can access and view data lineage and other metadata in Azure Databricks and leverage the full power of Unity Catalog. This includes extending Unity Catalog’s unified governance over data and AI into Azure Databricks Mosaic AI. In total, you will be able to combine this data with other native and federated data in Azure Databricks, perform analysis assisted by generative AI, and publish the aggregated data back to Power BI—making this integration complete across the entire data and AI lifecycle. 

Watch these announcements in action at Microsoft Build 2024

Join us at Microsoft Build from May 21 to 23, 2024 to see all of these announcements in action across the following sessions: 

• “Microsoft Build opening keynote” with Satya Nadella, Kevin Scott, and Rajesh Jha.

• “Next generation AI for developers with the Microsoft Cloud” with Scott Guthrie.

• “Microsoft Fabric: What’s new and what’s next” with Arun Ulagaratchagan and Amir Netz.

• “Ingest, analyze and act in real time with Microsoft Fabric” with Yitzhak Kesselman and Tessa Kloster.

• “Extend and enhance your analytics applications with Microsoft Fabric” with Dipti Borkar and Teddy Bercovitz.

• Introducing API for GraphQL and User Data Functions in Microsoft Fabric” with Monica Boris and Dipti Borkar.

• “Unify your data with OneLake and Microsoft Fabric” with Josh Caplan and Charles Webb.

• “Accelerate insights with real-time Azure SQL Data and Microsoft Fabric” with Anna Hoffman and Bob Ward.
• “Intergrating Azure AI and Microsoft Fabric for Next-Gen AI Solutions” with Nellie Gustafsson and Facundo Santiago.
• “Activate enterprise data in AI-enabled business applications” with Rakesh Krishnan and Srikumar Nair.

You can also try out these new capabilities and everything Fabric has to offer yourself by signing up for a free 60-day trial—no credit card information required. To start your free trial, sign up for a free account (Power BI customers can use their existing account), and once signed in, select start trial within the account manager tool in the Fabric app. Existing Power BI Premium customers can already access Fabric by simply turning on Fabric in their Fabric admin portal. Learn more on the Fabric get started page. 

Join us at the European Microsoft Fabric Community Conference 

We are excited to announce a European Microsoft Fabric Community Conference that will be held in Stockholm, Sweden from September 23 to 26, 2024. You can see firsthand how Fabric and the rest of the data and AI products at Microsoft can help your organization prepare for the era of AI. You will hear from leading Microsoft and community experts from around the world and get hands on experiences with the latest features from Fabric, Power BI, Azure Databases, Azure AI, Microsoft Purview, and more. You will also have the opportunity to learn from top data experts and AI leaders while having the chance to interact with your peers and share your story. We hope you will join us and see how cutting-edge technologies from Microsoft can enable your business success with the power of Fabric.   

Explore additional resources for Microsoft Fabric

If you want to learn more about Microsoft Fabric: 

• Sign up for the Microsoft Fabric free trial.
• View the updated Microsoft Fabric Roadmap.
• Visit the Microsoft Fabric website.
• Join the Fabric community.
• Read about other Azure announcements:  

• • From code to production: New ways Azure helps your build transformational AI experiences.

• • Unleashing innovation: The new era of compute powering Azure AI solutions.

• Read other in-depth, technical blogs on the Microsoft Fabric Updates Blog.
• Read the new Microsoft Fabric security whitepaper.
• Watch the Real-Time Intelligence Microsoft Mechanics video.
• Explore the Fabric technical documentation.

Microsoft Fabric

Experience the next generation in analytics 

Explore capabilities

¹ChatGPT sets record for fastest-growing user base – analyst note, Reuters.

The post Unlock real-time insights with AI-powered analytics in Microsoft Fabric appeared first on Microsoft Fabric Blog.

With the maturation of cloud-native big data platforms and the exciting revolution in generative AI, the potential for data-driven decisions and operational optimizations has never been greater, raising the urgency of solving the longstanding problem of how to enable organizations to bring together estate-wide data for analytics.

The vision of open lakes for analytics

Unlocking data usage for the age of AI

Read the whitepaper

Optimizing processes by simplifying data

We believe that the emergence of open, updatable table formats presents us with a unique opportunity to solve this problem by standardizing these formats across all analytic engines, and by simplifying data replication. In fact, as an increasing number of engines adopt open data formats, we can minimize data replication by instead using references to data sources.

Further, as the value of data is recognized, we are seeing corresponding emphasis on right-use and increasing regulation. Thus, it is important that we be able to govern the entire data estate in a compliant manner, and in particular, evolve current best practices for aggregating estate-wide data to reflect the emerging world of cloud-native data lakes that bring together a diverse range of analytic capabilities, from exploratory tools, to AI models, to tools for serving data and rich business reports reliably, securely, and at scale.

Shaping the future of data analytics

This vision of the future of analytics is at the heart of OneLake design in Microsoft Fabric. We have striven to make it the “one place to bring all data for analytics”, making it easy to virtualize and aggregate data from all sources. Fabric itself then democratizes access to the wealth of insights that can be unlocked, thanks to a Microsoft 356-like simplicity in bringing analytic tools to bear on the data through intelligent software as a service, and by infusing AI copilot experiences to assist with complex tasks in-stride. The entire life cycle of analytics, from aggregating data to unlocking rich insights for appropriately authorized users, can be managed using the data governance capabilities of Fabric and the integrated estate-wide governance capabilities of Microsoft Purview.

Read the whitepaper to learn more!

The post Microsoft’s vision of an open data lake ecosystem: Open lakes, not walled gardens appeared first on Microsoft Fabric Blog.

Bring your data into the era of AI

Reshape how everyone accesses, manages, and acts on data with a single, AI-powered platform.

Start today

Take PricewaterhouseCoopers (PwC) for example, a leader in the professional services sector and a long-time technology innovator. PwC is applying generative AI to acquire, transform, and analyze data faster to better support its employees and provide better audit experiences for clients.

“We’re implementing Microsoft infrastructure to help future-proof NGA design, increasing the potential future adaptability of our assurance services and processes. Generative AI capabilities within the Azure OpenAI Service open up possibilities for us to enable natural language interfaces for enterprise data.” 

Winnie Cheng, Director of AI in Products and Technology at PwC

Connect your data with Fabric

PwC is not alone. Increasingly, organizations are turning to AI to transform their data cultures for better business outcomes. Traditionally, building this culture requires a few key ingredients:

1. Organizing your data into a logical data mesh to make it easier for users to discover, reuse, and enhance the best data available.

2. Creating a seamless analytics engine that can meet the demands of the business to uncover insights in real-time.

3. Infusing those insights into the applications your people use every day so they can make data-driven decisions.  

These steps are still vital, but now you can employ generative AI to accelerate the path to a data-rich culture by enhancing the productivity of your data teams and making analytics tools more accessible to everyone. In my webinar, Infusing AI into your Data Culture: A Guide for Data Leaders I walk through exactly how Microsoft can help you accomplish each step along this journey.

First, we’ll explore the blockers preventing users from discovering, accessing, and using data to innovate and make better decisions. I will also show you the promise of Microsoft Fabric’s single, SaaS, multi-cloud data lake, OneLake, designed to connect to any data across the organization and serve everyone who needs access to data in an organized, intuitive data hub. Your data teams can use the OneLake data hub to manage your data, endorse high-quality data to encourage use, and manage access. Users can easily find, explore, and use the data items that they have access to—from inside data tools like Fabric or even applications like Teams and Excel. For more on OneLake, read our e-book, Lakehouse Analytics with Microsoft Fabric and Azure Databricks. 

With your data accessible to those who need it, you also need to equip them with powerful analytics tools that can help them scale to the needs of the business. That’s where Microsoft Fabric further shines. With Fabric, data teams can use a single product with a unified experience and architecture that provides all the capabilities required for analysts to extract insights from data and present them to the business user. Each role in the analytics process has the tools they need, so data engineers, data scientists, data analysts, business users, and data stewards feel right at home. By delivering the experience as a SaaS platform, everything is automatically integrated and optimized, and users can sign up within seconds and unlock significant business value within minutes.  

With your data in a single place and your data teams empowered to uncover insights faster than ever, the next step is to get insights into the hands of everyone in your organization. I’ll show you how Power BI can infuse reports and insights into your apps like Microsoft 365, Dynamics 365, Power Platform, and even third-party apps like Salesforce and SAP.  

And now, as we enter a future built on AI, I’ll walk you through three key ways generative AI can help foster a more data-rich culture: 

1. Take advantage of out-of-the-box experiences like Copilot in Fabric which helps you accelerate the productivity of your data teams.  
2. Employ powerful AI models right from Fabric to draw deeper insights from your data. 
3. Create custom AI experiences, grounded on your data, with native integration between Microsoft Fabric and Azure AI Studio.

See these capabilities in action 

If you’re interested in learning more, I’d encourage you to join my webinar on Infusing AI Into Your Data Culture: A Guide for Data Leaders. I’d also encourage you to try Microsoft Fabric and Azure AI Studio out for yourself and learn more about how to adopt and foster a data-driven culture by reading the Microsoft Fabric adoption roadmap on data culture.

The post Foster a more data-driven culture with generative AI in Microsoft Fabric appeared first on Microsoft Fabric Blog.