The short happy life of the data scientist
There’s no better time to understand data: to really be able to look at a vast sea of data and, like Neo from The Matrix, see the bigger picture. It’s a thrilling time for hardcore “dataphiles” because there has never been as much data to play around with in order to solve real-world problems. It’s not just the amount of data, but in the ever-increasing types of data from an ever-increasing number of sources. It’s also a thrilling time because, well, with so few true data scientists out there, and with such a global demand for their services, they really can pick and choose where they want to work. “Data scientist” continue to be the hottest job role globally, with some folks (especially in tech startup hotbeds like Silicon Valley) earning starting salaries over $240K (£166K). But how long will the good times really last?
Remember the telephone?
Before we answer the question about the future of data science, lets step back in time a bit. Remember when the telephone was invented? Unless you’re a Galapagos turtle (with a 177-year lifespan), probably not. Most likely, the telephone has always been a bit of ubiquitous technology that allowed you to directly dial anyone in the world (or in the country at least). But if we go back about 100 years, to when telephones were still relatively new (when less than 30% of the population owned one), they required a switchboard operator. The technology at the time was so complex that someone physically had to connect the phones of the two parties who wanted to speak via phone plugs. Fast forward a few years, and the process became much easier as phone users learned how to input the right sequence of numbers to reach whomever they wished to automatically. Fast forward again to today, and the functionality of our phones has evolved so far that there’s very little we can’t do with them.
The end of the data scientist?
That’s the same story we’ve had, until very recently, with our data. The system for communication had been so complex, it required an expert to connect up our business questions with the ever-increasing ocean of data constantly flowing into businesses: hence the rise data scientist. But the days of the data scientist as the key connector may be coming to an end for two simple reasons.
First, business decision-makers are becoming much more savvy in knowing the kinds of questions that data can answer: moving from reactive reporting (“how did we do…”) to predictive analytics (“what’s likely to happen if…”). They’re what Gartner calls the citizen data scientist: business or IT managers, or public sector officials whose primary function isn’t focused on managing data. Many of them have become lay-experts because of the limited supply of full-time data scientists.
The second reason for this coming shift is the fact that the productivity tools business decision makers are using continue to have more sophisticated analytics features built in. Machine learning is continuing to get better at understanding how we work, and providing easily understandable answers to fairly complex data questions that are asked in “normal” language. If you’re an 3PL operations manager, for instance, you’re able to ask questions such as “What’s a correlation between weather conditions and on-time delivery?” If you’re an emergency first-responder to a fire, you may be able to ask “where’s the next-greatest risk as this fire spreads.”
Rise of the data storyteller
So if businesses and government departments are becoming more savvy on how to use data, and productivity tools are becoming better at anticipating our data needs, does this mean that we’re not going to need data scientists anymore? Not at all. We will continue to need human intelligence to understand which new sources of data can be used, and how that should be organised. We’ll also need people who understand the subtleties and complexities of their particular business/civic landscape and who can bring the stories that matter to their audience to life with data. What’s more, as the role of data continues to evolve from predictive to prescriptive, (i.e., allowing for more autonomous digital agents, like self-driving cars) data scientists will be crucial in making sure that we’re looking at the right set of data, and in incorporating that knowledge into the right kind of action.
What does the future hold for data scientists?
For the talented ones, the future looks quite bright if they’re able to adapt to the changing expectations of our data. How those expectations evolve are certainly up for debate, but a great place to take part in the conversation is through our Data Culture series. This exclusive Microsoft emersion workshop helps empower your organisation to gain a competitive edge through data driven insight. This series of customer events is delivered in conjunction with a host of leading Microsoft partners to provide a series of half day and full day sessions specific to your needs.
Take part in the data conversation at the next Data Culture Summit
