
Edit Story

Reinventing Energy Exploration & Power Generation With Extreme Data Analytics


Europe is all in on smart cities. Copenhagen collects and analyzes sensor data to control pollution, Amsterdam is integrating home energy storage into the smart grid, and Deutsche Telekom in Germany will more than double its electric car charging infrastructure by converting their power distribution boxes. From real-time management to resource optimization to predictive maintenance, data-driven technology is reshaping the urban landscape.


But what if data could reinvent energy exploration and generation itself?

In the Extreme Data Economy, it can. Extreme data isn’t just bigger “big data.” It runs our modern economy. Essentially, the Manufacturing Economy used data for validation, the Service Economy used it to make informed decisions, and the Extreme Data Economy runs on it.

Editor’s Note:  To learn more about how you can leverage extreme data analytics, download the whitepaper “BI Meets AI: Augmenting Analytics with Artificial Intelligence to Beat the Extreme Data Economy”

As we shift into the Extreme Data Economy, the energy sector is facing new challenges: data now comes from unpredictable sources and analysis becomes much more complex. Data can be big or small, static or streaming, structured or unstructured, human or machine. On top of that, it may be perishable. Data from sensors and smart devices might only be useful for hours or minutes, rather than weeks or months.

But if an organization is able to process and analyze that data immediately and effectively, it can cut costs, optimize investments, and reduce risk–from exploration to drilling, production, and energy generation.  

To illustrate, oil and gas companies need to be able to leverage real-time pipeline, oil well, and spatial data to determine the most viable oil fields for exploration, to remotely monitor drilling and production performance, and to prevent safety and environmental issues. If companies can perform faster and richer analysis of seismic, drilling, and production data, they can enhance geologic analysis, reduce exploration risk, increase drill and production performance, and improve oil market forecasting.

An oil and gas research firm in North America now offers a screening tool that can visualize and zoom into well production, and change and view the data in real time. This provides instant insight into the most economical place to invest resources. With spreadsheets, this took weeks; in the Extreme Data Economy–with speedy, concurrent analysis–it takes seconds. When profit comes down to how fast you can get the raw material out of the ground, accelerated analytics and instant insight make or break the business.

Energy service providers involved in the transportation, storage, and wholesale marketing of crude and refined oil and gas products also face extreme data challenges. In order to successfully build the vast networks of oil pipelines, these companies need to analyze comprehensive data sets for pipeline oil pressure monitoring, supply/demand forecasting, and real-time visualization of pipeline operations.

Discovering new value from high volumes of streaming IoT data is one of the biggest opportunities in energy today. Energy companies can utilize smart meters to measure how energy resources, such as electricity, water, and natural gas, are used. If you can process and analyze streaming smart meter data and multi-billion row datasets in real time, you can optimize energy generation and uptime, predict and prevent power outages, chart energy trends, understand customer energy usage, and improve load forecasts.  

One large American electric utility is ingesting real-time data streaming from smart meters to plot billions of data points, including meters, power plants, distribution infrastructure, weather, service vehicles, outages, and customer feedback–all in real time. By consolidating data silos and synthesizing vast sets of location-aware data, utilities can optimize energy generation and uptime.

The analysis of streaming data also plays a key role in monitoring and maintaining equipment in the energy and petroleum industries. Oil and gas companies can analyze sensor data from equipment and wells to avoid the downtime, costs, and safety issues that stem from equipment failures. Utilities can analyze electrical transmission, electrical distribution, gas distribution, gas transmission, and power pole data to predict, analyze, and minimize failures. In addition, data from maintenance personnel devices and service vehicles can be tracked to find the closest crew to a trouble call, identify the most efficient route, predict vehicle maintenance issues, and reduce emissions.

The entire energy sector is rife with opportunities to apply extreme data strategies. But most businesses are still using technology running on central processing units (CPUs) to store, manage, and analyze data linearly. Problem is, this technology is just too slow to extract value from data in real-time, to make operational decisions on the go, or to quickly and accurately assess risk. Traditional serial processing isn’t designed to handle unpredictable data sources or complex analysis. CPUs take a long time, often require decisions about what data to analyze in advance, and cannot produce real-time visualizations. They’re holding businesses back.

Energy companies, from top of the funnel to the bottom, require the speed of parallel processing and the capacity to manage unpredictable data sources to succeed in the Extreme Data Economy. They need solutions with NVIDIA (NVDA) GPUs–rather than CPUs–to perform complex, real-time analysis and produce instant visualizations. With the freedom to analyze data whenever it’s most valuable, businesses, utilities, and governments can gain the instant insights that revolutionize the way they do business–and the way we produce and consume energy.


Rebecca leads Kinetica's Global Content & Creative Marketing efforts, developing messaging and materials that convey the value of data-driven insights across industries.

Rebecca leads Kinetica's Global Content & Creative Marketing efforts, developing messaging and materials that convey the value of data-driven insights across industries. Prior to Kinetica, Rebecca held various marketing roles at ForgeRock, where she became Senior Manager of Content & Brand, heading up messaging, branding, and video. Prior to ForgeRock, Rebecca worked at Blue State Digital, the firm behind President Obama’s 2008 digital campaign, creating digital campaigns for diverse clients from Lady Gaga’s Born This Way Foundation to the US Holocaust Memorial Museum. She has also worked for The Center for the Study of the Presidency and Congress and Di Novi Pictures and holds a B.A. in History from Pomona College.