Big takeaways from big data
Today’s digital oil field encompasses a comprehensive list of transformative technologies that deliver increased productivity in upstream, midstream and downstream operations. Of all these technologies, big data remains one of the most disruptive and elusive competitive advantages that an oil and gas company can achieve.
Big data capabilities are becoming a key differentiating factor in the hunt for reserves of oil and natural gas. The U.S. Energy Information Administration projects that world energy consumption will increase 56% by 2040. To meet this demand, oil and gas companies must produce more from the conventional fields via EOR methods as well as developing techniques to maximize production from unconventional reserves. Competition is fierce, and the companies that are best able to adapt and begin a digital transformation of their businesses are the ones who will succeed.
The need to make strategic and tangible decisions from massive sets of raw data is becoming more important for tapping into both of these production efforts, particularly as the conventional and easy-to-produce hydrocarbon sources become more depleted.
Increasing recovery rates require improving operational excellence and using digital oilfield approaches that integrate practices such as seismic processing, reservoir modeling, drilling optimization and real-time production monitoring. Exploration and extraction of unconventional reserves further introduces advanced physical technologies as well as new information management and decision-making challenges.
What is big data?
Big data is a difficult term to grasp, in large part because everyone seems to have their own definition. A significant part of the complexity with big data is attributed to the fact that this term has become an expression for all advanced data analytics.
At Microsoft the term big data refers to several IT concepts and tools, spanning from strategic planning and advanced mathematical analysis to collaborative human interaction and reporting. Working together, these technologies provide tangible takeaways from massive amounts of data. The process of collecting operational insights from massive amounts of data is a difficult feat and the key ingredient to what makes big data a transformative technology.
When properly implemented and used, big data gives companies the power to find value in data that otherwise would yield few results. In oil and gas this could mean anything from finding new reserves of gas to improving operational integrity.
Big data has a lot of moving parts. In order for the technology to provide a positive effect on an organization, it needs to be able to provide real-time, actionable insights that add to what the organization already knows. The real struggle behind a successful big data solution is the need to manage several technologies using a variety of both structured and unstructured data. Microsoft’s approach toward big data is to bring together all these moving components and extrapolate business value by leveraging underlying data. In executing this mission, Microsoft’s approach is simple: Democratize big data with commonly used tools that let organizations interpret the results to better understand and drive their company direction.
Democratizing big data
Oil and gas organizations need a consolidated approach toward big data that includes not only the high-tech and mathematically cutting-edge tools associated with the technology but an easy-to-learn and -use human interface that allows decision-makers to view, assess and take action anytime from anywhere.
Microsoft’s approach toward big data involves a whole platform of devices and services designed to bring data to everyone. This includes a front-end interface using PowerBI and Power Pivot for Office 365 and Excel. As a self-service business intelligence (BI) product, Power Pivot is intended to allow users with no specialized BI or analytics training to develop data models and calculations. This universal platform enables the end user to not only visualize and analyze the information but also search, manipulate, analyze and develop new models that could be adopted by the enterprise without the need for already limited IT specialists and developers.
This “democratization of big data” starts at the back-end with the analytics platform system (APS). This appliance, comprised of the SQL-server parallel-data warehouse Azure HDInsight and PolyBase, is how structured data are combined with semistructured data that reside on Hadoop, an open-source storage software for large-scale processing of datasets on clusters of computers.
Azure HDInsight is Microsoft’s Hadoop-based solution for the cloud. It was architected to handle any amount of data and can scale from terabytes to petabytes on demand. Companies can deploy Hadoop in the cloud without buying new hardware or incurring other upfront costs. There’s also no time-consuming installation or set up since Azure HDInsight is part of Azure, an open and flexible cloud platform that enables users to quickly build, deploy and manage applications across a global network of managed datacenters. Being a part of Azure allows organizations the ability to launch new clusters in minutes. For customers wanting a lot of the same cloud benefits in an on-premises solution, Microsoft also offers HDInsight as part of the APS appliance so Hadoop can still be deployed on-premise.
Azure HDInsight’s ability to process data from web clickstreams, server logs, devices and sensors makes it vital in oil and gas, where operational technology is a significant portion of an enterprise’s infrastructure. Its compatibility with Hadoop also allows organizations to unleash new and more profitable business possibilities.
Getting tactical
Big data can be defined as high volume, velocity and variety. This is particularly important in oil and gas, where effectively reacting to available data can differentiate successful organizations from the others. In enhanced E&P, for example, big data can reduce the nonproductive time of assets by predictive maintenance of critical components such as electric submersible pumps. Big data also can help reduce HSE incidents within drilling and production and provide end-to-end views of hydrocarbon reservoirs through advanced pattern recognition.
Big data also can improve overall asset performance by managing real-time metrics across different subsystems. The machine learning capabilities of Microsoft’s big data solutions provide predictive analytics in areas like condition-based and predictive maintenance, which is a predominant theme for every successful oil and gas company.
Big data encompasses many technologies and systems that, working in concert, truly enable today’s digital oil field. Big data empowers oil and gas companies to deliver profitable results, critical in an era when costly EOR and ever-changing unconventional extraction techniques are needed to meet rising production demands.
