Slides of the ICSE 2015 Technical Briefing "The Green Lab - Experimentation in Software Energy Efficiency"

1. The Green Lab
Experimentation in Software Energy Efficiency
ICSE 2015 Technical Briefing
19th May, 2015
Giuseppe Procaccianti, Patricia Lago, VU University Amsterdam, ND
Antonio Vetrò, Daniel Méndez Fernández, TU München, GER
Roel Wieringa, University of Twente, ND

2. Discover the challenges

3. Software and Energy

4. Software is energy-inefficient
Hardware
Energy-
unaware
software
Hardware optimizations are
negated by software
inefficiencies [cf. Wirth’
Law]
….
© Patricia Lago 2014

5. “Software: it's a gas”
-Nathan Myrvhold, Microsoft Research

6. Source: http://en.wikipedia.org/wiki/Koomey's_law

8. Source: A major IT manufacturing company

9. [Research survey on
Industrial Energy
Management. M. Littlefield,
LNS Research, 2013]
[The energy efficiency
potential of cloud-based
software: A US case study.
Tech. rep., Berkeley,
California, 2013]
Motivation

10. State-of-the-Art: Software Energy Impact

11. Knowledge for developers
[Procaccianti et al., Empirical Evaluation of Best Practices for Energy-Efficient Software
Development, Springer Empirical Software Engineering, under preparation, 2015]
[Gude & Lago, Best Practices for Energy-Efficient Software, wiki.cs.vu.nl/green_software]
© Patricia Lago 2014

12. Knowledge for designers and architects /1
[Procaccianti et al., A Systematic Literature Review on Energy Efficiency in Cloud Software Architectures, Sustainable Computing:
Informatics and Systems, Elsevier, 2014]
© Patricia Lago 2014
Cloud FederationSelf Adaptation Energy Monitoring

13. Knowledge for designers and architects /2
[Procaccianti et al., A Systematic Literature Review on Energy
Efficiency in Cloud Software Architectures, Sustainable Computing:
Informatics and Systems, Elsevier, 2014]
[Procaccianti et al., Green Architectural Tactics for the Cloud,
IEEE/IFIP WICSA, 2014]
[Lewis & Lago, A Catalog of Architectural Tactics for Cyber-
Foraging, ACM SIGSOFT QoSA, 2015]
© Patricia Lago 2014

14. Open Knowledge
[Gu and Lago, Estimating the economic value of reusable green ICT
practices, ICSR, Springer, 2013]
ICT and Sustainability
http://greenpractice.few.vu.nl
http://thegreenpractitioner.amsterdamdatascience.nl
© Patricia Lago 2014

15. Software Energy Efficiency is odd

16. Software Energy Efficiency is odd
• Examples:
– Diminishing returns of adding more resources
Source: Harizopoulos, S., Shah, M. A., Meza, J. & Ranganathan, P. Energy Efficiency: The New Holy Grail of Data
Management Systems Research. CoRR abs/0909.1784, (2009).

17. Software Energy Efficiency is odd
• Examples:
– Algorithm design
Source: Harizopoulos, S., Shah, M. A., Meza, J. & Ranganathan, P. Energy Efficiency: The New Holy Grail of Data
Management Systems Research. CoRR abs/0909.1784, (2009).

18. Software Energy Efficiency is odd
• Examples:
– Third-party Software updates
Source: Cameron, K. W. Energy Oddities, Part 2: Why Green Computing Is Odd. Computer 46, 90–93 (2013)

19. Hotspot
Elements or properties, at any level of
abstraction of the system architecture,
that have a measurable and significant
impact on energy consumption.

20. Case study

21. The VMware benchmark (VMmark)

22. Example – Energy Hotspot / measurements
• VM energy estimation is quite reliable
• Systematic error of about ~10 Watts
Estimation Measure
100120140
Watt

23. Example – Energy Hotspot visualization / measurements
• If we plot the measure -10 watts you can see it approximates well
100120140160
Timestamp
Watt
15:40:00 15:46:00 15:52:00 15:58:00 16:04:00 16:10:00
Estimation
Measure

24. Example – Energy Hotspot visualization
• VMs over time: what happened in Phase 2?
010203040
Timestamp
Watt
15:56:00 15:57:00 15:58:00 15:59:00 16:00:00 16:01:00 16:02:00
DB
WS1
WS2
WS3

25. Phase 1: Hotspots Identification
• Measure energy usage in the field
• Identify patterns in the data
• Explain these by underlying mechanisms
Phase 2: Hotspots validation
• Try to reproduce these patterns by simulating these mechanisms in the lab.
• Investigate in the lab how different mechanisms may interact
• Search for evidence of those patterns in the field
• Integrate this into theories about energy usage
• From this knowledge, identify recommendations for reducing energy usage
Research Strategy

26. Generalization and
explanation
Application and
testing in the lab
and in the field
The Approach

27. Experience the complexity

28. Tool: Spotfire Tibco (http://spotfire.tibco.com/)
Hands-on session
Data: http://www.s2group.cs.vu.nl/green-lab/
(Click on ICSE Technical Briefing – online package)

29. Research Implications

30. Looking back (briefly): Principles of scientific working
Measurements
Patterns
Theories
Identify patterns
Generalisation and
explanation
Hypothetical mechanisms
Explain the patterns
Integrate into theories
Application and
testing in the lab and
in the field
Study Population
(entities from the theoretical
population)
Theoretical Population
(all entities that we want to
generalise about)

31. Empirical SE is conducted in a cyclic manner
Engineering
Cycle
Treatment
implementation
Implementation evaluation /
Problem investigation
Treatment designDesign validation

32. Scaling up by iterating through the engineering cycle
Engineering
Cycle
Engineering
Cycle
. . . . . .
Idealised
assumptions
Realistic
assumptions
• Evidence is gathered in a cyclic (step by step) manner
• With each iteration, we
– Narrow down our observations
– Increase the precision of our instruments and, thus, measurements
– Increase the validity of our conclusions

33. Levels of evidence
+
For
-
Against
Strong
evidence
Evidence
Circumstantial
evidence
Third-party claim
First or second part
claim
Strong
evidence
Circumstantial
evidence
Third-party claim
Evidence
First or second part
claim
Further reading: Wohlin. An Evidence Profile for Software Engineering Research and Practice

34. What does this mean for Experimentation
in Software Energy Efficiency?
1
1
2

35. What does this mean for Experimentation
in Software Energy Efficiency?
Objectives in experimental settings*
• Hypothesis-driven exploration
• Statistical significance
• Control
• Blocked subject assignment
• Balanced subject groups
• Contextualisation
• Cause-effect analyses
• Randomised assignments
• Replicability
• Competing alternatives
* Don’t blame us, Claes Wohlin said so… ;-)

36. Implications on Principles in Experimental SE
 Relax rigor in favour of pragmatism in initial stages
• Hypothesis-driven exploration
• Statistical significance
• Control
• Blocked subject assignment
• Balanced subject groups
• Contextualisation
• Cause-effect analyses
• Randomised assignments
• Replicability
• Competing alternatives
Relax
Stress

37. Thank you!