1. Using Market Segmentation to
Track Program Success
Amanda Dwelley
AESP EM&V Online Conference
December 4, 2013

2. About Opinion Dynamics
Established in 1987
Leader in market research
for utilities
Offices in Massachusetts,
California & Wisconsin
Energy Efficiency Evaluation
Energy Advising
Smart Grid, DR, and Behavior
Market Research
Custom approach —
We work with utilities and
implementers to use all
available data to develop
tailored solutions
AESP EM&V Online Conference
2

3. Key Points
 There are many ways to segment utility customer populations
 Some are more meaningful than others for program design, portfolio
planning and/or EM&V
 Implementers are already using segmentation to improve program
targeting (and uptake)
 The EM&V community (us!) does analyze results by customer
group/segment
 …But often not in a cohesive or consistent way
 Consistently integrating segmentation in to EM&V will:
 Deliver insights that help programs improve faster
 Get stakeholders thinking about (a) how results can be used/extrapolated,
and (b) if/how programs should be tailored/targeted to different segments
AESP EM&V Online Conference
3

4. Program implementers use segmentation all the time
Segmentation defines and divides a large population into identifiable groups
based on similar characteristics
Summer kWh
25%
20%
15%
10%
5%
0%
• High summer usage
targeted for HVAC rebate
• High annual usage
targeted for behavioral
programs
Experian Mosaic Segment
Multi-family
middle-income
targeted for
audits /
weatherization
1
AESP EM&V Online Conference
Urbanites targeted for HEMS / IHD

5. Historical approach of “equal access” to programs, and
undifferentiated marketing, hasn’t yielded equal impacts
For this utility, there’s a strong relationship between wealth quintile
(measured three ways) and long-term EE program participation:
10%
8%
6%
4%
2%
0%
1
2
3
4
Income Quintile
5
12%
Cumulative EE Participation vs.
Assessed Home Value (among the
50% of customers with assessor data)
EE Participation Rate
12%
Cumulative EE Participation vs.
Pct of Neighborhood with Income
>$75k (from secondary data)
EE Participation Rate
EE Participation Rate
Cumulative EE Participation vs.
Per Capita Income as % Poverty
Line (modeled value)
10%
8%
6%
4%
2%
0%
1
2
3
4
Income Quintile
5
16%
14%
12%
10%
8%
6%
4%
2%
0%
1
2
3
4
5
Home Value Quintile
What were the drivers of these differences? Targeted
marketing? Awareness/knowledge? Qualification
criteria? Interest?
AESP EM&V Online Conference
5

6. We’re leaving opportunity on the table, but don’t know where or how
much
“Our customers are
unique – So we can’t
reach statewide
goals”
Three-Year Plan vs. Statewide Goals
3.5%
3.0%
2.5%
2.50%
2.55%
2.60%
PY 2013
PY 2014
Segmented program
evaluation and opportunity
studies can uncover how/why:
• Moderate income status?
• House type (SF/MF)?
• Seasonal/vacation homes?
• Channel preferences vs.
implementation channels?
• Baseline efficiencies
already high?
PY2015
2.0%
1.5%
1.0%
0.5%
0.0%
AESP EM&V Online Conference
6

7. Evaluators do report on differences by customer group, but sometimes
we only look within a program
Annual Percent
Savings
2.5%
Annual Percent Savings by
Consumption Tertile
2.0%
1.5%
1.6%
1.8%
1.2%
1.0%
0.5%
0.0%
Low
Medium
High
Consumption Consumption Consumption
Top 2030%
Top 1020%
• Misleading to report,
because the program
targeted high users!
• Difficult for
planners/evaluators to
understand how to use
findings
Top 10%
Make sure segment “membership” we report is relative to the
customer population; use the same data source
AESP EM&V Online Conference
7

8. Segment-level insights are useful across the program
lifecycle
Metrics
Measurement Opportunities
Awareness / Knowledge
 General population and non-part surveys
Intention
 Inquiries, leads, incomplete applications that link
to customer database by account #
Qualification
 Ex ante: Filter database by qualifying criteria
 Ex post: Program qualification rates
Participation
 Program participation rates
 Portfolio-level participation: What % of all segment
members have participated in any EE?
Engagement
 Online / HEMS / IHD device tracking
 Participant surveys
Impacts
 Realization rates by segment
 Savings “depth” by segment (% savings)
 Measure mix by segment
AESP EM&V Online Conference
8

10. So, what segmentation is “good” for EM&V purposes?
1
1. Segment membership must be identifiable ex ante for all customers
 Rate code (Low income, SF/MF, Small/large commercial)
 Psychographic “lifestyle segment” available through data providers
(e.g., Experian)
 Usage characteristics (L/M/H; summer load; load shape)
2
1. Segments should distinguish between meaningful differences that affect
program outcomes
 Energy opportunity
 Barriers to participation (own/rent; income)
 Motivation to participate
 Channel/communication preferences (on-bill, web, phone)
 Impacts!
3
1. Segments should be “consumable” by readers/regulators:
 Easy to understand / well-named
 Manageable number
AESP EM&V Online Conference
10

11. 1
Identification: Tracking by segment requires defining segments
based on readily-available data – And we have a lot!
Secondary demographic/
housing data – e.g., age,
income, home value
Past program participation –
DSM and non-DSM
TOU
Account Rate
A

B
C
Energy
Audit
Ref.
Rebate


Customer characteristics from
CIS data – e.g., rate class,
time-as-customer
New
Customer engagement
– e.g., online activity,
payment preferences
Energy indicators –
e.g., seasonal
usage, load shape
1-4 yrs 5-9 yrs 10-19 20+ yrs
yrs
0 2 4 6 8 10 12 14 16 18 20 22
AESP EM&V Online Conference
11

12. 2
Meaningful Differences: Segment membership should correlate with
savings opportunities, program propensity, barriers and preferences
Demographically-Based
“Lifestyle” Segmentation
Custom Psychographi
Segmentation
Energy Usage Patterns
Past Participation
Highest
Medium
Dim. 1
Lowest
Dim. 2
• May correlate well with:
• Ability to
participate
• Channel/
marketing affinity
• Heterogeneous in
terms of:
• Savings
opportunities
• May correlate well with:
• Ability to
participate
• Motivation
• Heterogeneous in
terms of:
• Savings
opportunities
• Channel/
marketing affinity
0
2
4
6
8 10 12 14 16 18 20 22
• May correlate well with:
• Savings
opportunities
• Heterogeneous in
terms of:
• Ability to
participate
• Channel/
marketing affinity

13. Have AMI data? Clustering customers into Load Shape
Segments could enable long-term impact tracking
Best target for DR
and conservation
programs?
cluster similar
patterns
Relatively high
baseload - many
EE/Wx opportunities
Whole‐House Load Shapes
4000
3500
High Peak / Low Baseload
3500
3000
3000
2500
2500
Extended Peak
High Baseload
Low Users
Non-HVAC EE
and behavioral
interventions
2000
2000
1500
1500
1000
1000
500
500
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
0
0 1 2 3 4 5 6 7 8
Low-cost
conservation and
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23behavior
Identify highest-impact
equipment, envelope and
behavioral opportunities for
each segment
AESP EM&V Online Conference
13

14. 3
“Consumable” segments: Easy to explain and
interpret; manageable number
 Single dimensions (single-family / multi-family) or 2X2
matrices have merit
 But they leave a lot of heterogeneity undescribed
 Complex segmentation schemes quickly go un-used
 Reviewers don’t have background/knowledge of approach
 Imagine 70 Experian lifestyle segments!
 Cost implications to what we choose
 Segment quotas
AESP EM&V Online Conference
14

15. We can start by reporting savings at a segment level
Segment
Percent of
Customers
Percent of Wx
Participants
Wx Savings per
Household
(kWh)
Wx Savings
Total
(MWh)
A
25%
28%
180
81.0
B
15%
14%
150
33.8
C
40%
34%
100
56.0
D
20%
24%
80
32
Total
100%
100%
124
202.8
AESP EM&V Online Conference
15

16. End game: Identify and track program opportunities and
success metrics specific to each segment
Participation rate
among encouraged
Savings depth or
realization rate
n Targeted
for Wx
Wx Uptake
(among
those
targeted)
Wx Savings
per Household
(kWh)
Wx Opportunity
per Household
(kWh)
% of
Opportunity
Achieved
28%
5,000
9%
180
200
90%
15%
14%
3,000
7.5%
150
300
50%
C
40%
34%
8,000
7%
100
150
75%
D
20%
24%
4,000
10%
80
100
80%
Total
100%
100%
20,000
8.2%
124
172
72%
Segment
Percent of
Customers
Percent of
Wx
Participants
A
25%
B
AESP EM&V Online Conference
16

17. Thank You!
Amanda Dwelley
Associate Director
617-301-4629
adwelley@opiniondynamics.com
Visit us at www.opiniondynamics.com
AESP EM&V Online Conference
17