{"id":10631,"date":"2017-12-13T14:21:23","date_gmt":"2017-12-13T22:21:23","guid":{"rendered":"https:\/\/www.microsoft.com\/insidetrack\/blog\/?p=10631"},"modified":"2023-06-09T11:18:13","modified_gmt":"2023-06-09T18:18:13","slug":"azure-data-lake-connects-supply-chain-data-for-advanced-analytics","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/","title":{"rendered":"Azure Data Lake connects supply chain data for advanced analytics"},"content":{"rendered":"<div class=\"m-alert f-error\" role=\"alert\">\n<div>\n<div class=\"c-glyph glyph-incident-triangle\" aria-label=\"Error message\"><\/div>\n<p class=\"c-paragraph\">This content has been archived, and while it was correct at time of publication, it may no longer be accurate or reflect the current situation at Microsoft.<\/p>\n<\/div>\n<\/div>\n<p>Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect our processes, we built a centralized, big data architecture on Azure Data Lake. Now, we\u2019ve improved data quality and visibility into the end-to-end supply chain, and we can use advanced analytics, predictive analytics, and machine learning for deep insights and effective, data-driven decision-making across teams.<\/p>\n<p>The power of data\u2014data that shows patterns and predicts what\u2019s ahead\u2014helps to fuel digital transformation in the Microsoft supply chain. One building block of digital transformation is a modern data and analytics platform. Specifically, it\u2019s a platform that allows you to gather, store, and process data of all sizes and types, from any data source, and that enables you to access it seamlessly to gain valuable insights.<\/p>\n<p>In the end-to-end supply chain process, the right insights make it easier to identify trends and risks that help you ship on time, provide a quality product, save costs, and optimize inventory. The digital transformation journey is one that Microsoft has already begun.<\/p>\n<p>The supply chain engineering team at Microsoft provides solutions for core supply-chain functions related to hardware and devices like Microsoft Surface, Xbox, and HoloLens. Examples of these functions include sourcing of materials, planning, manufacturing, delivering products to consumers, and managing the customer care process.<\/p>\n<p>These supply chain activities are highly interconnected and interdependent, yet we\u2019ve often operated in silos\u2014with data in disparate systems and supply chain teams making isolated decisions. While a common result of organic growth over time, this approach can make it difficult to connect the pieces and optimize the whole. This, in turn, makes it even more challenging to apply advanced analytics and machine learning for deeper insights.<\/p>\n<p>An early step in this transformation was to bring this disparate supply chain data into one location\u2014a data lake\u2014rather than managing multiple data warehouses or manually co-locating data on an ad hoc basis. While a data lake doesn\u2019t solve all problems, and may create a few new ones, it allows more immediate focus on value-added activities and innovation with data.<\/p>\n<h2>Solving business problems with Azure<\/h2>\n<p>To tackle these challenges and bring our data together, we moved from a traditional relational database-centric solution to a big data platform with Azure Data Lake Store as its foundation. Data Lake Store, the Microsoft hyperscale repository for big data analytic workloads, is essentially Hadoop for the cloud made simple. Although SQL Server continues to play an important role in the presentation of data, it\u2019s just no longer a central storage component. This move to Data Lake Store has:<\/p>\n<ul class=\"c-list\">\n<li>Enabled us to say \u201cyes\u201d and take on more business requests, by giving us flexibility, scale, and speed. For example, we don\u2019t have to worry about whether our infrastructure can accommodate ever-increasing amounts of data that we need to store and process.<\/li>\n<li>Enabled application-independent compute engines to be provisioned over a shared data foundation.<\/li>\n<li>Allowed us to quickly onboard new data assets\u2014what used to take weeks now takes days.<\/li>\n<li>Provided simpler support for large-scale encryption at rest and in transit. We have out-of-the-box support for enterprise security and compliance, including Azure Active Directory authentication, and folder-level authorization via role-based and POSIX-style access control lists (ACLs).<\/li>\n<\/ul>\n<h2>Consolidating our data is key<\/h2>\n<p>Someday the industry may realize the dream of data federation, but today there\u2019s big value in centralizing data and then connecting that data to our supply chain processes. In a\u00a0blog post\u00a0from 2016, Josh Klahr\u2014VP of Product Management at AtScale\u2014proposed six architectural principles that influenced our solution. The decision to implement our data lake in Data Lake Store meant we could realize the benefits of three of these principles, right out of the box.<\/p>\n<ul class=\"c-list\">\n<li>Treat data as a shared asset. The co-location and scalability of Data Lake Store enables us to use the lake for a wide range of use cases and diverse target audiences.<\/li>\n<li>Ensure security and access controls. Data Lake Store includes encryption and auditing, in addition to role- and POSIX-based controls at the folder level.<\/li>\n<li>Eliminate data copies and movement. Data Lake Store can store and enable analysis of all our data in a single layer. While we still move small subsets of data to a database or reporting tool, we can meet many of our use cases by simply layering our compute engine in Spark over Data Lake Store. With this layering, we enable SQL-like experiences with Hive or any other supported capabilities in Spark, like machine learning, all with zero data movement.<\/li>\n<\/ul>\n<h3>Our key business scenarios<\/h3>\n<p>While there are obvious business benefits to adopting a big data platform, this was largely about technology modernization in the short-term with a focus on preserving business continuity. Even so, we defined some key scenarios that would help us prioritize what to go after on the new platform:<\/p>\n<ul class=\"c-list\">\n<li>Use data to identify why a device failed, predict manufacturing yield rates (the number of manufactured products that pass inspection the first time), and predict or even prevent future failures.<\/li>\n<li>Find the best prices for parts from suppliers and predict supplier prices at a component level for future buying optimization.<\/li>\n<li>Create accurate demand forecasting for hardware and devices for consumers, online stores, and frequent orders.<\/li>\n<\/ul>\n<h3>Engineering and business: better together<\/h3>\n<p>We help the supply chain by providing intelligence and data to make our planning and daily operations run more smoothly\u2014and to optimize costs. Manufacturers, for example, need real-time\u2014or earlier\u2014information when defects occur. They need data about what\u2019s failing and why. Day-to-day operations involve gathering, storing, processing, and visualizing data by:<\/p>\n<ul class=\"c-list\">\n<li>Gathering and storing the data in Data Lake Store.<\/li>\n<li>Applying a computing engine to process the data by using Spark.<\/li>\n<li>Visualizing the data with tools like\u00a0<a class=\"c-hyperlink\" href=\"https:\/\/powerbi.microsoft.com\/en-us\/\" target=\"_blank\" rel=\"noopener\" aria-label=\"Read more about Microsoft Power BI\">Microsoft Power BI<\/a>\u00a0dashboards and reports.<\/li>\n<li>Creating predictive and prescriptive models.<\/li>\n<\/ul>\n<h4>Our end-to-end team process<\/h4>\n<p>From start to finish, our team process works as follows:<\/p>\n<ul class=\"c-list\">\n<li>We collaborate with our business partners to determine what data is needed to move the business forward.<\/li>\n<li>The platform team defines the architecture, designs solutions, performs data engineering, and automates and supports the solution from code to consumption.<\/li>\n<li>Our engineering team also builds denormalized information assets including Kimball-style dimension and fact tables housed in Azure SQL Data Warehouse and OLAP cubes. We\u2019ve also built a robust data quality engine by using all Azure components, which serves as the centerpiece of our live-site process.<\/li>\n<li>We partner with the data science team who uses the platform to assist the business with analysis, and to gain deep insights through econometric modeling and machine learning.<\/li>\n<li>Our business partners build and maintain most of the presentation layer, including Power BI dashboards and SQL\u00a0Server Reporting Services reports.<\/li>\n<\/ul>\n<h2>Using the intelligent cloud<\/h2>\n<p>To achieve our business goals, we needed a modern architecture that could serve as a foundation for digital transformation in the supply chain. In addition to the primary storage layer in Data Lake Store, we use a broad array of building blocks in Azure, with the key components being:<\/p>\n<ul class=\"c-list\">\n<li><a class=\"c-hyperlink\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/data-lake-store\/\" target=\"_blank\" rel=\"noopener\" aria-label=\"Read more about Azure Data Lake Store\">Azure Data Lake Store<\/a>\u00a0as the primary means of storing our data.<\/li>\n<li><a class=\"c-hyperlink\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/data-factory\/\" target=\"_blank\" rel=\"noopener\" aria-label=\"Read more about Azure Data Factory\">Azure Data Factory<\/a>\u00a0for the pipelines we use to ingest data from various sources.<\/li>\n<li><a class=\"c-hyperlink\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/hdinsight\/apache-spark\/\" target=\"_blank\" rel=\"noopener\" aria-label=\"Read more about Apache Spark for HDInsight\">Apache Spark for HDInsight<\/a>\u00a0for transforming and querying data stored in the lake.<\/li>\n<li><a class=\"c-hyperlink\" href=\"https:\/\/azure.microsoft.com\/en-us\/services\/sql-data-warehouse\/\" target=\"_blank\" rel=\"noopener\" aria-label=\"Read more about Azure SQL Data Warehouse\">Azure SQL Data Warehouse<\/a>, for curated data assets, supporting read-only queries.<\/li>\n<\/ul>\n<p>Figure 1 highlights the architecture that surrounds Data Lake Store for our end-to end-solution.<\/p>\n<div><\/div>\n<div class=\"m-image\">\n<figure id=\"attachment_10633\" aria-describedby=\"caption-attachment-10633\" style=\"width: 624px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-10633 size-full\" src=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-fig1.png\" alt=\"Shows the architectural components we used including Azure Data Lake Store, Apache Spark on HDInsight, Azure Data Factory, and Power BI.\" width=\"624\" height=\"285\" srcset=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-fig1.png 624w, https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-fig1-300x137.png 300w\" sizes=\"(max-width: 624px) 100vw, 624px\" \/><figcaption id=\"caption-attachment-10633\" class=\"wp-caption-text\">Figure 1. Our technical architecture<\/figcaption><\/figure>\n<\/div>\n<h3>Technical and design considerations<\/h3>\n<p>If you\u2019re planning to implement a similar solution for your supply chain data, here are a few points to consider:<\/p>\n<ul class=\"c-list\">\n<li>To co-locate data in a data lake effectively, determine how you\u2019ll design and organize it. Data Lake Store has a file system like the one on your personal computer. For example, you wouldn\u2019t store all your documents in the C:\u00a0root directory. It\u2019s similar with a data lake\u2014everything is a file, with other options that can be tailored to a given requirement.<\/li>\n<li>The folder structure in Data Lake Store reflects the levels of governance and ownership. For example, business partners might own everything that\u2019s stored in a particular location and have authority over it. Data in another location might be managed by the engineering team and could have source-control mechanisms in place. Careful design choices in the folder structure will make governance much easier. The data in high-level folder paths can be user-owned, where individuals and teams can set their own policies locally. Or, the engineering teams can manage it.<\/li>\n<\/ul>\n<p>Figure 2 shows an example of our approach. The arrow from bottom to top demonstrates increasing levels of data curation.<\/p>\n<p>&nbsp;<\/p>\n<figure id=\"attachment_10634\" aria-describedby=\"caption-attachment-10634\" style=\"width: 300px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-10634\" src=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-fig2.png\" alt=\"Data assets in folder paths, which reflect levels of governance and ownership.\" width=\"300\" height=\"238\" \/><figcaption id=\"caption-attachment-10634\" class=\"wp-caption-text\">Figure 2. Data governance and ownership<\/figcaption><\/figure>\n<ul class=\"c-list\">\n<li>Data assets are categorized as raw, gold, or cooking:\n<ul class=\"c-list\">\n<li><strong>Raw.<\/strong>\u00a0Data assets are onboarded (full loads, delta enabled, timeseries) one by one as they come from the source, with no changes.<\/li>\n<li><strong>Gold.<\/strong>\u00a0Largely mirrored structure of raw data with assertions about data quality, de-duplication, and merging. Data is also exposed as Hive tables and available for consumption by individuals and tools.<\/li>\n<li><strong>Cooking.<\/strong>\u00a0Contains reporting assets that are usually the result of an extraction, transformation, and load (ETL) process. This includes all our dimensions, facts, and other information assets that are moved to Azure SQL\u00a0Data Warehouse.<\/li>\n<\/ul>\n<\/li>\n<li>With hardware, you have dedicated servers for environments like development, testing, and production. Often, you need data in your test environment that\u2019s similar in size and quality to production data. When this happens, you must physically move it from production to testing and back. With a data lake, your folder structure can be your environment, with roots for testing, development, and production. To move data from one environment to another, point your source to the desired location. Rather than moving data between environments, you\u2019re just reading the data using a different path.<\/li>\n<\/ul>\n<h3>How do we get there from here?<\/h3>\n<p>Figure 3 shows the process we followed to move from a data-warehouse-centric platform to our modern platform.<\/p>\n<div><\/div>\n<div class=\"m-image\">\n<figure id=\"attachment_10635\" aria-describedby=\"caption-attachment-10635\" style=\"width: 652px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-10635 size-full\" src=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-fig3.png\" alt=\"Our four-phased approach for moving from a legacy data warehouse to using Azure Data Lake.\" width=\"652\" height=\"176\" srcset=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-fig3.png 652w, https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-fig3-300x81.png 300w\" sizes=\"(max-width: 652px) 100vw, 652px\" \/><figcaption id=\"caption-attachment-10635\" class=\"wp-caption-text\">Figure 3. Our approach for moving to a data lake<\/figcaption><\/figure>\n<\/div>\n<p>Our approach consists of four phases:<\/p>\n<ul class=\"c-list\">\n<li><strong>Phase 1.<\/strong>\u00a0This stage is the starting point for most modernization projects with a relational data warehouse in a central role.<\/li>\n<li><strong>Phase 2.<\/strong>\u00a0This stage mostly focuses on adding the data lake and ingesting data sources in parallel. Optionally, we added Azure SQL Data Warehouse and copied processed dimensions and facts from our legacy data warehouse.<\/li>\n<li><strong>Phase 3.<\/strong>\u00a0This stage represents the bulk of the project effort. In most data platforms, the core ETL processing is the long pole, and moving that around is a lot of effort and risk. The dotted lines in the graphic denote processes being incrementally turned off in the legacy flow once migrated. This repeats until all are moved. Also, you\u2019ll want to run these processes in parallel for some time for regression testing.<\/li>\n<li><strong>Phase 4.<\/strong>\u00a0This stage involves turning off the legacy data warehouse, with full processing occurring through the data lake.<\/li>\n<\/ul>\n<h2>Implementation challenges<\/h2>\n<p>A few of the challenges we\u2019ve faced, but have overcome, include:<\/p>\n<ul class=\"c-list\">\n<li><strong>Initial misalignment on which technology to use.\u00a0<\/strong>As you might expect in a company as large as Microsoft, there are many options, and very good internal ones, that aren\u2019t available to the public. We worked with our business partners to refine our options to two candidates, and public Azure was the clear choice.<\/li>\n<li><strong>Aggressive timeline.\u00a0<\/strong>For a process that might typically be a multi-year effort, our core implementation was done in less than one year. We have about 150 terabytes of raw data over many data sources, and lots of dependencies. We were able to complete this transition rapidly because of the power of Azure and a determined engineering team.<\/li>\n<li><strong>Simple, performant consumption layer.\u00a0<\/strong>Using the Spark execution engine over Data Lake Store solved many of our problems, especially when paired with Jupyter Notebooks. One of our core architectural principals was to minimize data movement after we had moved to the data lake. Microsoft has exciting developments related to Azure SQL Data Warehouse, Azure Analysis Services, and possibly some other surprises. There is more work to do in achieving parity with traditional approaches for simple and performant consumption of data in this crucial last mile.<\/li>\n<\/ul>\n<h2>Recommendations based on our lessons learned<\/h2>\n<p>Some of the lessons we\u2019ve learned thus far include:<\/p>\n<ul class=\"c-list\">\n<li><strong>The more you can invest in learning new technologies, the better.\u00a0<\/strong>Most Microsoft customers who have worked with a relational database are comfortable writing T-SQL, and the same was true for us. But there is a learning curve, for example, to pivot from SQL Server to Hive (HQL), and even more so for our engineers writing complex code in languages like Scala.<\/li>\n<li><strong>Understand the various Azure roadmaps<\/strong>. The various components in Azure have different levels of maturity. Take time to understand how the various components in Azure work together, what gaps if any there may be for your specific requirements, proceed in an Agile fashion, and plan or the future.<\/li>\n<li><strong>Data is power, but it can exist in different forms.\u00a0<\/strong>Have a clear strategy for master data management, so that you can provide the most accurate information and a single data source. Proactively plan for and manage data quality. We decided to build our own custom tooling for big data quality management.<\/li>\n<\/ul>\n<h2>Benefits we\u2019ve seen<\/h2>\n<p>So far, we\u2019ve seen the following benefits:<\/p>\n<ul class=\"c-list\">\n<li><strong>A home for all data.\u00a0<\/strong>We no longer have to be concerned that the next project will be the tipping point for running out of SAN space or hitting a hidden performance limit.<\/li>\n<li><strong>Less latency than our previous BI system.\u00a0<\/strong>We have reduced onboarding time-to-value across the board and, on some critical datasets, we\u2019ve reduced the time that it takes to load information from days to hours.<\/li>\n<li><strong>Improved data consistency.\u00a0<\/strong>We built a data dictionary that spans the supply chain. The data dictionary keeps track of standardized KPI definitions so that teams have a common taxonomy.<\/li>\n<\/ul>\n<h2>Next steps<\/h2>\n<p>We\u2019re still in the early stages of digitally transforming our supply chain, but we\u2019re already seeing the benefit of having centralized data, a more flexible data processing model that scales as needed, and connected processes. Our roadmap includes using what we\u2019ve already done as a foundation and layering other capabilities including:<\/p>\n<ul class=\"c-list\">\n<li><strong>Expand our use of predictive and prescriptive analytics.\u00a0<\/strong>For example, our data science team is currently building on this foundation to deliver a dynamic regression model for better prediction of sales through demand. Also, we\u2019re planning a component \u201crecommender system\u201d to optimize part selection during design and sourcing.<\/li>\n<li><strong>Get new insights into the customer life cycle.\u00a0<\/strong>For example, how does the supply chain contribute to customers\u2019 and partners\u2019 perception of doing business with \u201cOne Microsoft\u201d? How do we measure the lifetime value of a customer?<\/li>\n<li><strong>Continue to modernize our compute architecture.\u00a0<\/strong>We\u2019ve applied Azure AD ACLs against folders in Azure Data\u00a0Lake. We\u2019ll continue moving toward a seamless governed experience as users move between the various query and visualization tools. Additionally, we\u2019re in the process of delivering capabilities that build upon other foundational work in our \u201chot path\u201d for data, including near real-time statistical process control for better yield management and smart services.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>This content has been archived, and while it was correct at time of publication, it may no longer be accurate or reflect the current situation at Microsoft. Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect [&hellip;]<\/p>\n","protected":false},"author":146,"featured_media":10636,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"_hide_featured_on_single":false,"_show_featured_caption_on_single":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[1],"tags":[],"coauthors":[674],"class_list":["post-10631","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","m-blog-post"],"jetpack_publicize_connections":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Azure Data Lake connects supply chain data for advanced analytics - Inside Track Blog<\/title>\n<meta name=\"description\" content=\"Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect our processes, we built a centralized, big data architecture on Azure Data Lake. Now, we\u2019ve improved data quality and visibility into the end-to-end supply chain, and we can use advanced analytics, predictive analytics, and machine learning for deep insights and effective, data-driven decision-making across teams.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Azure Data Lake connects supply chain data for advanced analytics - Inside Track Blog\" \/>\n<meta property=\"og:description\" content=\"Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect our processes, we built a centralized, big data architecture on Azure Data Lake. Now, we\u2019ve improved data quality and visibility into the end-to-end supply chain, and we can use advanced analytics, predictive analytics, and machine learning for deep insights and effective, data-driven decision-making across teams.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/\" \/>\n<meta property=\"og:site_name\" content=\"Inside Track Blog\" \/>\n<meta property=\"article:published_time\" content=\"2017-12-13T22:21:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-06-09T18:18:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1040\" \/>\n\t<meta property=\"og:image:height\" content=\"585\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Inside Track \u2013 retired stories\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Inside Track \u2013 retired stories\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/\",\"url\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/\",\"name\":\"Azure Data Lake connects supply chain data for advanced analytics - Inside Track Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg\",\"datePublished\":\"2017-12-13T22:21:23+00:00\",\"dateModified\":\"2023-06-09T18:18:13+00:00\",\"author\":{\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/#\/schema\/person\/59e5f7b07dae629412c990cc1a63b575\"},\"description\":\"Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect our processes, we built a centralized, big data architecture on Azure Data Lake. Now, we\u2019ve improved data quality and visibility into the end-to-end supply chain, and we can use advanced analytics, predictive analytics, and machine learning for deep insights and effective, data-driven decision-making across teams.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#primaryimage\",\"url\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg\",\"contentUrl\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg\",\"width\":1040,\"height\":585,\"caption\":\"Small business owner working on Samsung Galaxy TabPro.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Azure Data Lake connects supply chain data for advanced analytics\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/#website\",\"url\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/\",\"name\":\"Inside Track Blog\",\"description\":\"How Microsoft does IT\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/#\/schema\/person\/59e5f7b07dae629412c990cc1a63b575\",\"name\":\"Inside Track \u2013 retired stories\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/#\/schema\/person\/image\/ee0de87c339052d5d84852473bd7f213\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/24a8c329ab32afd1bc23fd1658d1acc2?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/24a8c329ab32afd1bc23fd1658d1acc2?s=96&d=mm&r=g\",\"caption\":\"Inside Track \u2013 retired stories\"},\"description\":\"The content on this page was crafted to highlight a specific moment in time or the solutions that have led us to where we are today. It offers valuable insights into our journey and the progress made over the years. Check out the Inside Track blog page for our up-to-date stories around Microsoft.\",\"url\":\"https:\/\/www.microsoft.com\/insidetrack\/blog\/author\/insidetrackarchive\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Azure Data Lake connects supply chain data for advanced analytics - Inside Track Blog","description":"Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect our processes, we built a centralized, big data architecture on Azure Data Lake. Now, we\u2019ve improved data quality and visibility into the end-to-end supply chain, and we can use advanced analytics, predictive analytics, and machine learning for deep insights and effective, data-driven decision-making across teams.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/","og_locale":"en_US","og_type":"article","og_title":"Azure Data Lake connects supply chain data for advanced analytics - Inside Track Blog","og_description":"Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect our processes, we built a centralized, big data architecture on Azure Data Lake. Now, we\u2019ve improved data quality and visibility into the end-to-end supply chain, and we can use advanced analytics, predictive analytics, and machine learning for deep insights and effective, data-driven decision-making across teams.","og_url":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/","og_site_name":"Inside Track Blog","article_published_time":"2017-12-13T22:21:23+00:00","article_modified_time":"2023-06-09T18:18:13+00:00","og_image":[{"width":1040,"height":585,"url":"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg","type":"image\/jpeg"}],"author":"Inside Track \u2013 retired stories","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Inside Track \u2013 retired stories","Est. reading time":"12 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/","url":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/","name":"Azure Data Lake connects supply chain data for advanced analytics - Inside Track Blog","isPartOf":{"@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#primaryimage"},"image":{"@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#primaryimage"},"thumbnailUrl":"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg","datePublished":"2017-12-13T22:21:23+00:00","dateModified":"2023-06-09T18:18:13+00:00","author":{"@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/#\/schema\/person\/59e5f7b07dae629412c990cc1a63b575"},"description":"Our supply chain engineering team at Microsoft used to store and process data in disparate systems, which made data sharing and forecasting harder. To aggregate data and connect our processes, we built a centralized, big data architecture on Azure Data Lake. Now, we\u2019ve improved data quality and visibility into the end-to-end supply chain, and we can use advanced analytics, predictive analytics, and machine learning for deep insights and effective, data-driven decision-making across teams.","breadcrumb":{"@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#primaryimage","url":"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg","contentUrl":"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg","width":1040,"height":585,"caption":"Small business owner working on Samsung Galaxy TabPro."},{"@type":"BreadcrumbList","@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/azure-data-lake-connects-supply-chain-data-for-advanced-analytics\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.microsoft.com\/insidetrack\/blog\/"},{"@type":"ListItem","position":2,"name":"Azure Data Lake connects supply chain data for advanced analytics"}]},{"@type":"WebSite","@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/#website","url":"https:\/\/www.microsoft.com\/insidetrack\/blog\/","name":"Inside Track Blog","description":"How Microsoft does IT","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.microsoft.com\/insidetrack\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/#\/schema\/person\/59e5f7b07dae629412c990cc1a63b575","name":"Inside Track \u2013 retired stories","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.microsoft.com\/insidetrack\/blog\/#\/schema\/person\/image\/ee0de87c339052d5d84852473bd7f213","url":"https:\/\/secure.gravatar.com\/avatar\/24a8c329ab32afd1bc23fd1658d1acc2?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/24a8c329ab32afd1bc23fd1658d1acc2?s=96&d=mm&r=g","caption":"Inside Track \u2013 retired stories"},"description":"The content on this page was crafted to highlight a specific moment in time or the solutions that have led us to where we are today. It offers valuable insights into our journey and the progress made over the years. Check out the Inside Track blog page for our up-to-date stories around Microsoft.","url":"https:\/\/www.microsoft.com\/insidetrack\/blog\/author\/insidetrackarchive\/"}]}},"jetpack_featured_media_url":"https:\/\/www.microsoft.com\/insidetrack\/blog\/uploads\/prod\/2023\/05\/7855-hero.jpg","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9hcZA-2Lt","_links":{"self":[{"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/posts\/10631"}],"collection":[{"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/users\/146"}],"replies":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/comments?post=10631"}],"version-history":[{"count":8,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/posts\/10631\/revisions"}],"predecessor-version":[{"id":10644,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/posts\/10631\/revisions\/10644"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/media\/10636"}],"wp:attachment":[{"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/media?parent=10631"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/categories?post=10631"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/tags?post=10631"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.microsoft.com\/insidetrack\/blog\/wp-json\/wp\/v2\/coauthors?post=10631"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}