Big data and analytics are necessary for the energy transition according to Pieter den Hamer of Alliander. Alliander is using big data from smart meters, renewable energy sources, electric vehicles, and more to improve grid management, asset management, and customer care. Examples include using big data to predict solar panel adoption, detect fraud, monitor transmission power quality, and simulate energy sharing. Organizing for big data requires skills, data quality, awareness, integration, and multidisciplinary collaboration. Big data can optimize renewable energy use, minimize central production, incentivize customer behavior, and allow for a self-healing grid.

1. Big Data: a necessity for the energy transition
Big Data Expo
September 20, 2017
Pieter den Hamer
Digital Strategy, Alliander
Copernicus Institute, University of Utrecht
pieter.den.hamer@alliander.com
+31 6 55134229
@PieterDenHamer

2. Alliander is an energy network group, including the largest energy
grid operator in The Netherlands and (parts of) Germany
2

4. 4
Big Data & IoT
Energy

5. 5
+
Green
but
Volatile
Energy
Demand
& Supply
-
Grey
but
Predictable
Energy
Demand
& Supply
Source: CE Delft

6. 6
Smart Homes
Smart Meters
Smart Appliances
Dynamic Pricing
Electric
Vehicles
+
Charging
Stations
Smart Grid
Offshore
Wind
Substation
Automation
Energy
Storage
Waste heat
distribution / city
warming
Local (solar)
energy production
Solar Farms
Smart Power Plants
EU Super Grid
Hydro
Power
Communciations Grid
(mobile + fiber)
MicroGrids
Virtual Power Plants
Onshore Wind
Biogas
Tidal
Energy
Smart Buildings & Cities
Dynamic
Demand/Supply
Balancing
Power Quality Mgt
CO2 emission reduction
+ CCS

7. 7
ENTERPRISE
DATA
(Finance,
Customers,
Workforce)
GEOSPATIAL
DATA
(Assets,
Network)
STREAMING
DATA
(Energy D/S,
Grid Status,
Social Media)
CONNECTED
(BIG) DATA
& Analytics
Connected customers & partners
Connected
employees
Connected smart grid
Connected assets
(Internet of Things)
Building the Connected Utility

8. Example
Distributed Data Processing & Blockchain for
Flexible Electric Vehicle Charging

9. Connected Data - example E: adoption of residential PV (1/2)
To predict the spread of residential PV over our service area, we built a model that calculates the current
probability of PV-adoption for all our customers.
9
Example
Big Data Analytics for Predicting Solar Panel Adoption

10. 10
Example
Big Data Analytics for Predicting Solar Panel Adoption

11. 11
CONFIDENTIAL
CONFIDENTIAL
CONFIDEN
TIAL
CONFIDENTIAL
CONFIDENTIAL
CONFIDENTIAL
CONFIDENTIAL
Example
Big Data Analytics for Fraud Detection

12. Example
Event Stream Processing for Transmission Power Quality Monitoring
Datum
Titel van de presentatie
12

13. 13
Example
Real-time Multi Agent System for Energy Sharing

14. 14
Example – Real-time analytics on the smart grid (or: the energy IoT)

15. 15
Main features:
• Dynamic reconfiguration of net
topology for resiliency, net loss
reduction, incident impact
minimization & graceful degradation
• Strong support for Microgrid / local
prosumer ‘energy sharing’ initiatives
• Optimize use of (local) renewable
production & storage, minimize
central energy production
• Dynamic energy pricing for
prosumer behaviour incentivizing &
(local) D/S balancing
• Power quality management
SmartCap:
MultiAgent Grid Simulation
Example
Simulations & real-time data analytics for autonomous / self-healing grids

16. Data-driven innovations require …
a common data platform (target architecture)
Data Platform (HANA+Dynamic Tiering)
Data
Consumption
ERP
Ware
house
Smart Grid –Data Lake (Hadoop)
Data
Provisioning
Geodata
ERP, CRM, … GIS GRID
Data
Sources
EXTERNAL
Process Mining
Logical Data
Warehouse
Open/Linked Data
provisioning
Workspace
(+ high performance in-database analytics)
Predictive
Analytics (SPSS, R)
Unstructured
Data
Geospatial & grid
Analytics (Esri)
(Self Service)
Reporting &
dashboards (Design
Studio)
Prescriptive
Analytics
Model / Rule
development
(Self Service)
Exploration (Lumira)
Data quality
(InformationStudio)
Advanced Visualization
Virtual / Extract Transform Load / Replication
Streaming Data &
Event Detection

17. Organizing for big data & analytics
17
1. People & Organization: we work in a
multidisciplinary, federated (central + LobS)
structure + collaboration with academic/research
institutes
2. Process: we apply the CRISP-DM process model as a
best practice, in the context of agile/devops methodology
3. Portfolio & Governance: enterprise-wide
analytics & data management activities are
categorized according to business readiness
and centrally governed.
Top 5 sucess factors
1. Data Science skills
2. Data quality
3. Big data awareness
4. Data integration & semantic
reconciliation
5. Multidisciplinary collaboration
(IT, data scientists, business experts)
Phi
• Analytical
Research
projects
• Academic
collaboration
• Uncertain
business case
Beta
• Analytical Pilot
projects
• Feasibilty
check
• Business case
validation
Omega
• Analytical
Business
implementation
projects
• Business case
realization
• Periodic
recalibration

18. Conclusion: big data for the energy
transition… we’re just getting started !
18
Asset Management & Operations
• Investment planning &
optimization
• Predictive, condition based
asset maintenance
• Outage risk analysis
• Maintenance scenario
simulations
• Augmented reality for workforce
support
Grid Management
• Outage detection, localization &
control
• Realtime load (demand &
supply) forecasting
• Power quality monitoring
• Grid configuration simulations
• Technical net loss reductions
• Self healing grids
Customer Care
• Communication localization &
personalization
• Fraud/theft detection
• Social media outage detection
• Energy prosumer behaviour analysis
• Energy saving potential analysis
• Customer Energy Insight services
• Open & linked data sharing
• Dynamic energy pricing

19. 19