An overview of the new ESRC Transformative research project given by Andy Newing to MRS Census and Geodemographic Group (CGG) hosted by GFK NOP, 19th November 2013 at Ludgate House, London

1. Census 2022: Transforming
Small Area Socio-Economic
Indicators through ‘Big Data’
An ESRC Transformative Research Project
Andy Newing and Ben Anderson
a.newing@soton.ac.uk
Sustainable Energy Research Group (SERG)
University of Southampton

2. Outline
2

3. Who are we?
3

4. Transforming Small Area Socio-
Economic Indicators through ‘Big Data’
4

5. Transforming Small Area Socio-
Economic Indicators through ‘Big Data’
5
 Potential demise of decennial census
– Challenge: robust alternative methods for
creating small area socio-demographic and
socio-economic indicators
– Opportunity: transform the nature of these
indicators using new analytic methods and
datasets
 Annual or sub-annual production?

6. Commercial transactional data
 Geo-coded household data from utility
service providers - collected as part of
routine service provision
 Can we derive traditional and novel small
area population estimates and social
indicators
 Census-like and Census-plus
6

7. Owen. 2006. The rise of the machines—a
review of energy using products in the
home from the 1970s to today., Energy
Saving Trust, London.
7

8. What can utilities data reveal about
household characteristics?
 Established links between household and
householder characteristics and energy
consumption (e.g. see AECOM 2011,
Druckman and Jackson, 2008)
 Based on understanding relationship
between end use and end users
 Consumption profiles reflect ownership and
use of appliances, household size and
characteristics and routines 8

9. What can utilities data reveal about
household characteristics?
 Elexon. 2013. Load profiles and their use in Electricity Settement, Elexon, London.
9

10. What can utilities data reveal about
household characteristics?
10
Druckman, A. and Jackson, T. 2008.
Household energy consumption in the
UK: A highly geographically and socio-
economically disaggregated model.
Energy Policy, 36(8), pp.3177-3192

11.  Onzo. 2012. Onzo Application Detection Technology Onzo Ltd., London. 11

12. One-Minute Resolution Domestic
Electricity Use Data (2008-2009)
 Household study
 Sample of 22 households
 1 minute resolution mean power import
 Linked to (limited) survey data on
household characteristics
 This dataset alone produces over 17m
recorded observations
12

13. 13Original dataset: Richardson, I. and Thomson, M., One-Minute Resolution Domestic Electricity Use Data, 2008-
2009 [computer file]. Colchester, Essex: UK Data Archive [distributor], October 2010. SN: 6583,
http://dx.doi.org/10.5255/UKDA-SN-6583-1.

14. 14Original dataset: Richardson, I. and Thomson, M., One-Minute Resolution Domestic Electricity Use Data,
2008-2009 [computer file]. Colchester, Essex: UK Data Archive [distributor], October 2010. SN: 6583,
http://dx.doi.org/10.5255/UKDA-SN-6583-1.

15. 15Original dataset: Richardson, I. and Thomson, M., One-Minute Resolution Domestic Electricity Use Data,
2008-2009 [computer file]. Colchester, Essex: UK Data Archive [distributor], October 2010. SN: 6583,
http://dx.doi.org/10.5255/UKDA-SN-6583-1.

16. 16
Original dataset: Richardson, I. and Thomson,
M., One-Minute Resolution Domestic Electricity Use
Data, 2008-2009 [computer file]. Colchester, Essex:
UK Data Archive [distributor], October 2010. SN:
6583, http://dx.doi.org/10.5255/UKDA-SN-6583-1.

17. UoS ‘Smart Meter’ Data
 Household study in two areas (case and
control)
 Up to 180 households over extended
period
 Detailed survey of household
characteristics, behaviors and attitudes
 Energy use (and other attributes) at one
second resolution 17

18. 18

19. 19

20. ‘Big Data’
 Huge datasets
– UoS one second data produces over 500m
records per month!
– Aggregate and summarise but need to
understand missing data
– Nationally at 30 minute resolution – 94bn
readings per month
 Need to sample
– Spatially representative and/or temporal
– New ways of thinking about census data 20

21. Scope
 Not seeking to produce robust nationally
representative small area indicators ……
 Identify available datasets
 Focused exploratory work to demonstrate
the potential of these datasets
 Develop methodologies and algorithms
 Build up an interested expert stakeholder
group 21

22. Approaches
 Regression – using median household
consumption
 Cluster analysis – driven by data 22

23. Approaches
 Time series and
periodicity –
repeating patterns
23
 All can be explored using standard
settlement periods (30min) and alternative
temporal resolution – and disaggregation
by day, time of year etc.

24. Indicators
 Traditional ‘census-type’
– household occupancy,
– household age structure
– household economic activity
– OAC classification?
….. based on energy consumption
 Novel census-plus indicators
– energy inequality - energy consumption
gini coefficient? 24

25. Commercial Value
25
 Almost all households consume electricity
 New source of geo-coded address point
data with household attributes
 Commercial transaction-driven ‘big data’
from utilities providers
 Cannot mask consumption – reveals actual
habits and routines!

26. Impacts
 Commercial data owners
 Commercial data aggregators
 Local and national authorities and similar
organisations
 Future collaborations
26

27. 27

28. Questions and discussion
 To what extent is this type of thinking
already part of your business or
organisation?
 What datasets of this nature do you have
access to or might be able to share?
 What is your specific interest in work of this
nature?
 Would you be interested in contributing
further?
28

29. Census 2022: Transforming
Small Area Socio-Economic
Indicators through ‘Big Data’
www.energy.soton.ac.uk
/category/research/energy-behaviour/census-
/
Andy Newing
a.newing@soton.ac.uk
Sustainable Energy Research Group (SERG)