One of the most challenging evaluation questions for residential lighting energy efficiency programs in the U.S. is the identification and correction for net-to-gross (NTG) effects such as free ridership and spillover. Over the last twenty years, considerable effort and financial resources have been directed toward accurately measuring these effects. Furthermore, the correction for these NTG effects has direct, and sometimes, drastic impact on program savings.
APT and Opinion Dynamics discuss a new framework for the estimation of free ridership in upstream lighting programs grounded in sound, economically rational decision making on the part of retail partners. This approach, built on functional retail behavior, provides a clearer more insightful look into the elements comprising the retail sales environment thus providing program implementers with a more predictable outcome of end results – up front.
MEEA Technical Webinar: A New Approach to Estimating Free Ridership in Upstream Lighting Programs
1. MEEA Technical Webinar Series:
A New Approach to Estimating
Free Ridership for Upstream
Lighting Programs
Presenters:
Stan Mertz - Applied Proactive Technologies &
Tami Buhr - Opinions Dynamics
Thursday, October 18th, 2012
2. MEEA’s Role in the Midwest
• Nonprofit serving 13 Midwest states
• 10+ years serving states, energy offices, utilities
and communities
• Staff of 25 in Chicago
• Actions
– Designing & Administering Energy Efficiency Programs
– Evaluating & Promoting Emerging Technologies
– Regional Voice for DOE/EPA & ENERGY STAR
– Coordinating Utility Program Efforts
– Delivering Training & Workshops
– Advancing Energy Efficiency Policy
– Promoting Best Practices
3. REVENUE NEUTRAL MODEL
A New Approach to Estimating Free
Ridership for Upstream Lighting Programs
October 2012
4. Agenda
1. Challenges associated with estimating lighting program NTG
2. Theoretical background underlying Revenue Neutral Sales Model
3. Example of how the model works
4. Questions
Revenue Neutral Model 2
6. Estimating Lighting Program Free Ridership is Challenging
• Programs usually delivered through an upstream markdown method
• Customers purchase discount lighting, walk out the store and disappear
• Often unaware of the discount
• Note that this does not mean customer would have purchased bulbs at full price
• Retailers are fully aware of their participation
• Traditional methods use data from both customers and retailers to estimate
free ridership
Revenue Neutral Model 4
7. What Evaluators Really Want: Full Sales Data
• Data that tracks sales of EE lighting with and without program pricing would
provide the best estimate of program impact
• Could see the actual lift in sales when program in effect
• Depending on how long program has been running, measurement approaches
could include:
• Pre-program sales, sales when programs turn pricing on and off, sales of like products
that are not discounted, sales in comparison areas that do not have programs
• Unfortunately, retailers will not provide sales of EE lighting at regular price. Will
only provide sales of products lighting programs discount.
Revenue Neutral Model 5
8. Sales Data Alternatives for Estimating Free Ridership
• Participant self-reports
• Common method for estimating program free ridership for rebate programs
• Difficult with upstream programs where customers purchase program incented product
and disappear
• Two survey methods commonly used:
• General population telephone surveys
• Call utility customers and ask detailed questions about past lighting purchases
• Results are of questionable validity due to timing of survey , small nature of purchase, and difficulty
identifying program purchasers
• In-store customer interviews
• Interview customers in store immediately after they make purchase decision
• Greater confidence in self-report results
• Usually make use of non-probability samples
• Expensive and challenging to conduct
Revenue Neutral Model 6
9. Sales Data Alternatives for Estimating Free Ridership (2)
• Retailer Interviews
• Conduct interviews with corporate or store level retailers and ask for
estimate of program impact on sales
ti t f g i t l
• Are no more likely to give up the numbers in an interview than a request for
data.
• Store level staff often do not know sales
• Corporate level do not know for a specific utility territory
• At best, get rough estimates
• May have vested interest in seeing programs continue
• Modeling Techniques
• Many require use of self-report data in addition to other data (e.g. multi-
state model revealed preference models)
model,
• Suffer from the same validity problems
Revenue Neutral Model 7
10. Another Look at Sales Data
• Existing FR methods are challenging, expensive and produce questionable
results
• We keep going back and asking for complete sales data.
• Maybe if we ask again, or ask nicer, or ask a different person, we’ll get it.
• We have asked everyone for sales data including program implementers
• Implementers also h l evaluators g t access t stores t conduct intercept
I l t l help l t get to t to d ti t t
interviews
• We started talking about the challenges associated with our existing free
ridership methods and alternatives
p
• Is there something we can do with the sales data we do have?
• Have program sales data. Also have program prices, regular prices, and sales goals
for each retailer for each product
• Estimate non-program sales using data we do have and a model of retailer
behavior
Revenue Neutral Model 8
12. Retailer Behavior
• Retailers will only participate in utility lighting programs if their participation is
revenue neutral
• Their “top line sales” remain the same or increase; cannot decrease
• But why are “top line sales” so important to a retailer?
• How exactly do retailers factor topline sales into their decision to participate in
utility programs?
Revenue Neutral Model 10
13. What is Topline Sales?
• “Top Line Sales” is a reference to the gross sales or revenues of a
company.
• The "top" reference relates to the fact that on a company's income
statement, the first line at the top of the page is generally reserved for
gross sales or revenue.
• Program reimbursements for sales are not included in revenue
• A company that increases its revenues is said to be "growing its top line", or
"generating top-line growth".
• This contrasts with net income (or net earnings per share), which is usually the
bottom line of the company's income statement.
Revenue Neutral Model 11
14. Topline Sales Impact Example
• Retailer must sell a minimum of 254 additional units just to get back to $697
total sales dollars generated before program
$697.00 $697.00
Incremental
Top Line sales lift
Sales $
Sales $ +254 units
+254 units
Generated $197.00
Regular Retail $6.97
R l R il $6 97
Discount $5.00 100 Units 100 Units 354 Units
Promo Retail $1.97 sold sold sold
Revenue Neutral Model 12
15. Gross Margin Impact Example
• Retailer is made whole on the Gross Margin line after discounts have been
reimbursed on first 100 units and all additional units provide incremental GM $
Growth.
G th
• Unfortunately, the rebate dollars are not able to be credited to Top Line Sales
dollars.
Incremental
Incremental
Gross Gross
Margin $ Margin lift
Generated on
+254 units
100 Units If only 100 354 Units
sold before Units sold sold
Program during
Program
Revenue Neutral Model 13
16. Retailer Decision Tree: Year 1 (SKU Level)
Decline to participate OR
Less than
Revise Strategy of Incentives to
gy
prior to
i t
meet Necessary Revenue Dollar
How many units can Program
level
be sold in the
Promotional time
p
period at the
reduced price?
Same or Above
prior to Program
Same Above
Participate at Participate at
Agreed Upon Incentive Agreed Upon Incentive level
level and Allocation and Negotiate Additional
Amount Allocation Amount
Revenue Neutral Model 14
17. Retailer Decision Tree: Additional Years (SKU Level)
Was the
No Discontinue
Previous Year
OR
promotion revenue
promotion revenue
Participate with Revised Incentive
neutral or better?
Discontinue OR
Yes Less than Revise Strategy of Incentives to
Previous Year meet Previous Year Revenue or
How many units can Additional Promotion Opportunity
be sold in the
Next Year?
Same or Above
Previous Year
Same Above
Continue at Continue at
Current Incentives Current Incentives
and
d and Negotiate
d
Same Allocation Additional Allocation
Revenue Neutral Model 15
19. Required Data
• Need program tracking data at sku level for:
• Sales goals
• MSRP
• Incentive amount (and any changes in incentive amount over the promotional
timeframe)
• Program sales
• Use the first three to estimate sales without program discount pricing
• Use program sales to calculate free ridership
Revenue Neutral Model 17
20. Model Implementation
• Can estimate the program’s likely free ridership before the program year
• Likely free ridership is what the program will receive if each sku meets its sales
goal and the program does not allow sales to exceed goals
• Actual free ridership is based on final program sales
• Will be higher for skus that do not meet sales goal
• Will be lower for those that exceed goals
Revenue Neutral Model 18
21. Use in Actual Evaluations
• Have used the model in two evaluations so far
• Results are not yet publicly available
• Used multiple methods for one evaluation and all came up with similar
p p
numbers
• We are pursuing a patent on the model.
Revenue Neutral Model 19
22. Model Benefits
• Can calculate free ridership by
• Bulb type (standard, specialty, fixtures, CFLs, LEDs)
• Retailer
• During special promotions compared to regular program pricing
• None of the existing lighting free ridership methods provide this level of detail
Revenue Neutral Model 20
23. Program Design
• Can use results to identify changes to program implementation to minimize free
ridership within existing program budgets.
• Tailor the product mix by bulb type
• Vary incentive levels by retailer and bulb type
• Some retailer and bulb types have higher FR rates
• Need greater incentives to encourage purchase by non-free riders
• Maximize incentive dollars by matching incentive amount to bulb and retailer
types
Revenue Neutral Model 21
24. Questions or Comments?
Tami Buhr
Director of Survey Research
Opinion Dynamics
tbuhr@opiniondynamics.com
617-301-4654
Stan Mertz
Director of Retail Operations
Applied Proactive Technologies
stanm@appliedproactive.com
413-731-6546
413 731 6546
Revenue Neutral Model 22
25. The EE Story (Future)
• Future: Finding a new portfolio
– Lighting savings going down
– Some program saturation
– Need ‘new’ programs
• Whole home (HPwES, air sealing, etc)
• Systems work (HVAC systems, smart homes, etc)
• Behavior programs (changing the customer habit)
• Education
• Building Energy Codes (adoption, training and
compliance)
– Challenges
• Cost effectiveness (non-energy benefits not counted)
• More complex (contractors, systems, etc)
26. Presenter Contact Information
Stan Martz, Applied Proactive Technology –
StanM@AppliedProactive.com
Tami Buhr, Opinion Dynamics –
tbuhr@opiniondynamics.com