Duke Energy has been involved in data mining its smart grid system and device data for the last two years. One major issue that emerged through this work was extracting data from the various siloes it resided in for analysis and new value development. Therefore Duke Energy developed a “Sandbox” – a data model and dataset that combined a finite set of data elements from a variety of systems – to analyze and identify new value opportunities from utilizing the smart grid system.
To accelerate this process, Duke Energy sponsored the Data Modeling and Analytics Initiative (DMAI), an innovative forum by which big data experts were given a slice of the dataset to analyze for opportunities and insights that it could incorporate into its big data analytics strategy and activities.
Seventeen vendors submitted final reports that discussed data issues, models, and tools used to analyze the data, and use cases that could be developed for new value opportunities. Responses varied considerably based on the skills and expertise of each vendor. Vendors provided over 150 unique use cases for consideration. Duke Energy asked for general financial information regarding the potential benefits of the use cases; however, these results were limited. Therefore, a follow-up exercise and interview was implemented that allowed the vendors to provide value scores to major use case categories, and provided qualitative input as to where the value may be discovered within each category.
Significant insights were generated from information on how vendors constructed and applied systems and analytics to develop and model the use cases. This information can be incorporated into Duke Energy’s big data and analytics strategy activities. Below are key observations and insights that came out of the DMAI: There is significant potential value that can come from implementing a big data platform across a variety of use cases areas.
We encountered a variety of problems extracting data from Duke Energy’s systems. Consolidation and integration of data elements are required to perform the analytics necessary to identify and realize the value identified above. Issues with data include missing data, no common information model, problems linking data sets from different systems, and challenges from extracting data.
Vendors had few problems ingesting Duke Energy data. While the systems and tools for data ingestion varied among vendors, the results for ingestion across vendor platforms was generally consistent.
To understand big data implications for other areas of Duke Energy, more data is needed than what was included in the initial data set. The vendors provided numerous examples of use cases that could be implemented with the inclusion of additional data. Social media, asset attributes (age, type) and event alerts were the three most common data elements most often identified by vendors.