{"id":197,"date":"2016-02-14T11:40:43","date_gmt":"2016-02-14T11:40:43","guid":{"rendered":"https:\/\/www.microsoft.com\/en-gb\/industry\/blog\/industry\/2016\/02\/14\/fifty-shades-of-data\/"},"modified":"2016-02-14T11:40:43","modified_gmt":"2016-02-14T11:40:43","slug":"fifty-shades-of-data","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-gb\/industry\/blog\/financial-services\/2016\/02\/14\/fifty-shades-of-data\/","title":{"rendered":"Fifty Shades of Data"},"content":{"rendered":"

Don\u2019t lose control of your analytics<\/em><\/h2>\n

Maybe you\u2019re a bit curious. Or a bit scared. Maybe it\u2019s a bit of both, and that\u2019s what makes it exciting. You know there are people around you that are into it\u2026 like really<\/em> into it, and from the outside, they seem so normal. But dig a little deeper beneath their spreadsheets, and you find yourself stumbling into the strange and scintillating world of data.<\/p>\n

Fair warning, the preceding intro is about as juicy as this blog post is going to get (at least from a pulp romance novel perspective), and in case you need it, just remember the safe-word \u201ccloud confidence\u201d. From here on out, we\u2019ll be taking a closer look at the kinds of data we collect and share, how we transform that data into insight, and how data is changing just about every aspect of business \u2013 which, come to think of it, is actually pretty juicy in its own right.<\/p>\n

Big Data: a rose by any other name?<\/h2>\n

About five years ago, the term \u201cbig data\u201d started to appear \u2013 almost out of nowhere \u2013 on every CIOs list of their top technology priorities. Five years on, it\u2019s still on their lists (though it\u2019s fallen off the Gartner Hype Curve this year). And while it may have slipped from being the<\/em> top priority for many CIOs (in favour of issues such as cloud computing and cyber security), it still is a perennial favourite when it comes to issues that dominate discussions of technology strategy. Where did <\/em>this big data phenomenon come from? And why do so many organisations seem unable to harness the power of big data? In short, \u201cbig data\u201d is a set of data so large that current technology and understanding struggles to process it. It\u2019s not new. The idea of being able to collect data that taxed processing power can be traced back to the creation of the Hollerith machine in the late 1890s. We\u2019ll examine these issues throughout the post, but first let\u2019s take a look the different names we\u2019ve used to describe the idea of turning your data into insight.<\/p>\n

    \n
  1. Data mining<\/li>\n
  2. Analytics<\/li>\n
  3. Business analytics<\/li>\n
  4. Business intelligence<\/li>\n
  5. Big data<\/li>\n<\/ol>\n

    How does<\/em> data become insight?<\/h2>\n

    For a lot of business executives, the idea of analytics can be daunting. There may be a vague understanding of how databases work (those are the cylinder-shaped things in presentations about IT, right?) Even for IT executives who aren\u2019t directly involved in analytics, the structure of a data program can be confusing. For the most part, it\u2019s fairly straightforward: you\u2019ve got (usually) separate systems to capture your data, to arrange that data into a useable format, to store that formatted data, to structure that data in a way that\u2019s easy to \u201cask it the right question,\u201d to present the answers to those questions logical and compelling ways, and ultimately to apply those answers to solve business problems. So from end to end (from data to insight, if you will) those systems are:<\/p>\n

      \n
    1. ETL: allows you to extract, transform and load your data<\/li>\n
    2. Data warehouse: where your enterprise data is stored<\/li>\n
    3. OLAP: lets you select bits of data for comparison (elements like time, revenue, costs & location)<\/li>\n
    4. Data Visualisation: presents data in charts, graphs and other visual ways<\/li>\n
    5. Analytics: the interpretation (human or machine) of that visual data to gain insights and make business decisions.<\/li>\n<\/ol>\n

      Styles of analytics<\/h2>\n

      So now that the journey your data take (from extraction through to analytics) is clear, how are you presented with this information? Below, are styles of analytics you\u2019re likely to encounter from the most basic reporting (e.g. sales figures by region) to the most advanced analytics with machine learning (e.g., incorporating real-time environmental data from multiple sources to enable a driverless car). It\u2019s worth digging a bit deeper into each of the styles as each has a set of use cases for which it\u2019s best suited, but in order of complexity, they allow you to see what has happened, what is<\/em> happening, what\u2019s happening against expected or desired measures, what\u2019s likely to happen, and how to interpret and act on what\u2019s happening.<\/p>\n

        \n
      1. Reporting<\/li>\n
      2. Adhoc query<\/li>\n
      3. OLAP (slice and dice)<\/li>\n
      4. Alerts and notifications<\/li>\n
      5. Dashboards<\/li>\n
      6. Scorecards<\/li>\n
      7. Predictive Analytics<\/li>\n
      8. Machine learning<\/li>\n<\/ol>\n

        Sources of data<\/h2>\n

        One of the reasons that big data has become such a large issue to organisations in recent years is the simple fact that the sources of data keep changing and getting more complex. Before the internet, data came from fairly fixed systems with fairly structured and standardised data. Information from ERP, CRM or POS systems didn\u2019t really change the type of data they captured. But in the past decade, there\u2019s been an explosion in sources of data. Companies are capturing unstructured data from places such as social media and call centres. Loads of data is being generated by tags and sensors in smart devices. As this variety of data continues to grow, the challenges of being able to make sense of all of it continues to grow as well. Here are the main sources of data your organisation is likely to encounter:<\/p>\n

          \n
        1. ERP systems<\/li>\n
        2. CRM systems<\/li>\n
        3. Social Media<\/li>\n
        4. Call centre logs<\/li>\n
        5. RFID tags<\/li>\n
        6. Sensors<\/li>\n<\/ol>\n

          Location of data<\/h2>\n

          So where does all this data reside? The short answer is all over the place \u2013 which in and of itself is a problem when you\u2019re trying to bring together all your company\u2019s data for a single version of the truth. With the rise of cloud computing though, \u201call over the place\u201d isn\u2019t necessarily a bad thing. And while there\u2019s much to be said about cloud computing and data storage, we\u2019re just about halfway through the list. So if you need a break (or just want to read more about the cloud), remember the safe-word: Cloud Confidence<\/a><\/em>.<\/p>\n

            \n
          1. Desktop (Excel files, Access database)<\/li>\n
          2. On-premises<\/li>\n
          3. Cloud<\/li>\n
          4. Hybrid<\/li>\n<\/ol>\n

            Top people to listen to on Social Media<\/h2>\n

            If you\u2019re still reading\u2026 I\u2019m glad you stayed. Because while the first half of this post focuses on the basic structures that make up analytics, the second half focuses on the juicier bits: the bigger questions about where data and analytics are going, and what barriers need to be overcome to get there. The best way to keep a finger on the pulse of the analytics community is to get connected with the top minds in the space. From data science, to machine learning, to prescriptive analytics, and more, here are some of the big names in big data:<\/p>\n

              \n
            1. @claudia_imhoff<\/a>: Claudia Imhoff, Founder of the Boulder BI Brain Trust<\/li>\n
            2. @idigdata<\/a>: Jen Underwood, Microsoft BI & Analytics guru and thought leader<\/li>\n
            3. @jameskobielus<\/a>: James Kobielus, IBM Big Data Evangelist<\/li>\n
            4. @jenstirrup<\/a>: Jen Stirrup, PASS Board of Directors for Business Analytics<\/li>\n
            5. @jlwoodward<\/a>: Jonathan Woodward, Microsoft UK Lead for BI and Analytics<\/li>\n
            6. @josephsirosh<\/a>: Joseph Sirosh, Microsoft machine learning expert<\/li>\n
            7. @kdnuggets<\/a>: Gregory Piatetsky, big data and data science legend<\/li>\n
            8. @kirkdborne<\/a>: Kirk Borne, Principal Data Scientist at Booz Allen<\/li>\n
            9. @stevedunbar<\/a>: Steve Dunbar, Microsoft UK Lead for IoT<\/li>\n<\/ol>\n

              Real world examples of data in action<\/h2>\n

              Can you use data as a transformational force for your organisation? Absolutely. How it\u2019s used is entirely up to you. We\u2019ve got customer stories showcasing big data from a wide variety of industries from a football club that have used analytics to improve fan engagement across the globe, to a manufacturer using data from IoT to improve lettuce production, to a charity that uses advanced analytics to connect people with causes that matter the most to them. Here are just a handful of our customer stories:<\/p>\n

                \n
              1. Financial Services: <\/strong>MetroBank uses data to improve the customer experience<\/a>.<\/li>\n
              2. Wholesale and Distribution:<\/strong> JJ Foods uses predictive analytics to improve sales.<\/a><\/li>\n
              3. Manufacturing:<\/strong> Fujistu uses IoT data to improve crop yield<\/a>.<\/li>\n
              4. Sports:<\/strong> Real Madrid tracks social data to improve fan engagement. <\/a><\/li>\n
              5. Charity:<\/strong> Just Giving uses advanced analytics to increase giving<\/a>.<\/li>\n<\/ol>\n

                Top challenges of data<\/h2>\n

                While there are a lot of great examples of companies leveraging data in all its forms, quite frankly there aren\u2019t enough. Only a fraction of companies feel as though they are successfully leveraging their data. This happens for a number of reasons. Some are technological challenges (e.g. having legacy systems and data or just having the wrong portfolio of analytics tools in place), but more are human challenges. There\u2019s a dearth of qualified data scientists available in the market (it\u2019s why they command top salaries), but more importantly leveraging data to make business decisions isn\u2019t part of the business culture for many companies.<\/p>\n

                  \n
                1. Historical estate and legacy data<\/li>\n
                2. Not having the right technology portfolio<\/li>\n
                3. Dearth of available talent<\/li>\n
                4. Data-driven decisions aren\u2019t part of business culture and strategy<\/li>\n
                5. Not knowing the right questions to ask (or how to ask them) of data<\/li>\n<\/ol>\n

                  Future Trends of Data:<\/h2>\n

                  So where do we go from here? For those organisations who have overcome the current challenges of data, they\u2019ve got their eye on the next phase of analytics. While we could write a separate post on 50 trends of data, many of these ideas fall under a few umbrella concepts. The cloud will be the source of significantly more data as more devices and systems are connected, but the cloud \u2013 with its infinite scalability and flexibility will also be crucial in being able to make sense of all these new sources of data. While traditional analytics (what happened and why) will still have an important part to play, we\u2019ll start to move beyond predicting what will happen to being prescriptive with analytics \u2013 systems automatically taking action on insights gleaned through data. And our systems continue to improve their ability to anticipate, learn and respond, we will reach a point \u2013 aka the singularity \u2013 when this technology surpasses human control or understanding. Depending on your point of view, this could be thrilling or terrifying. Maybe a bit of both.<\/p>\n

                    \n
                  1. The Cloud<\/li>\n
                  2. Prescriptive Analytics<\/li>\n
                  3. The singularity<\/li>\n<\/ol>\n

                    Now with all this talk of data, are you curious? Is your data rewarding you or punishing you? If you want to find out more, have a look at our Data Culture Series. Whether you\u2019re a data expert or a business professional, we\u2019ve got an event to help you take control of your data. Laters.<\/p>\n

                    Register now to attend the Data Culture Summit<\/strong><\/a><\/p>\n