The availability of on-demand, utility computing via the cloud introduces a new world of flexibility but also an entirely new charging model for applications. This new model has long promised to provide metered compute, charging you for exactly the amount of processing power you need, at the points that you need it. The cloud is a large paradigm change, not just for some of the technologies involved but also for the economics and the return on investment for deploying and running a given application. Whereas traditional on-premises applications require upfront capital expenditure on hardware, cloud deployments have an ongoing operational expense. Additionally, clouds typically charge by the amount of memory used, whereas applications are typically developed and tuned to run as fast as possible using all the available (already paid for) resources. Chris Bailey explains how this new economics of the cloud is driving changes in the way applications are architected, developed, and deployed. Presented at the O'Reilly Software Architecture Conference, London 2017

1. Cloud Economics
Chris Bailey, IBM
@Chris__Bailey

2. “Cloud economics is a branch of knowledge
concerned with the principles, costs and benefits of
cloud computing.”
— searchcio.techtarget.com

5. Why Cloud?

6. Traditional Computing
§ On premise, or hosted bare metal machines
| @Chris__Bailey

7. Traditional Computing
§ On premise, or hosted bare metal machines
§ Capital Expenditure (CapEX):
- Upfront cost, followed by depreciation
| @Chris__Bailey

8. Traditional Computing
§ On premise, or hosted bare metal machines
§ Capital Expenditure (CapEX):
- Upfront cost, followed by depreciation
§ Requirement for accurate capacity planning
- Estimate hardware for given load
- Project future load requirements
| @Chris__Bailey

9. Traditional Computing
§ On premise, or hosted bare metal machines
§ Capital Expenditure (CapEX):
- Upfront cost, followed by depreciation
§ Requirement for accurate capacity planning
- Estimate hardware for given load
- Project future load requirements
§ IT is seen as a cost centre
| @Chris__Bailey

10. Capacity Planning
§ Find a representative benchmark?
| @Chris__Bailey
SPECjEnterprise 2010 (spec.org)

11. Capacity Planning
§ Find a representative benchmark?
- SPECjbb2015
- SPECjEnterprise2010
- SPECjms2007
- SPECjvm2008
| @Chris__Bailey
SPECjbb2015 (spec.org)

12. Capacity Planning
§ Find a representative benchmark?
- SPECjbb2015
- SPECjEnterprise2010
- SPECjms2007
- SPECjvm2008
| @Chris__Bailey
SPECjbb2015 (spec.org)

13. Capacity Planning
§ Find a representative benchmark?
- SPECjbb2015
- SPECjEnterprise2010
- SPECjms2007
- SPECjvm2008
| @Chris__Bailey
SPECjEnterprise2010 (spec.org)

14. Capacity Planning
§ Find a representative benchmark?
- SPECjbb2015
- SPECjEnterprise2010
- SPECjms2007
- SPECjvm2008
| @Chris__Bailey

15. Capacity Planning
§ Find a representative benchmark?
- SPECjbb2015
- SPECjEnterprise2010
- SPECjms2007
- SPECjvm2008
| @Chris__Bailey

16. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2
Jun 29, 2016 | HTML | Text
14,121.47 1 16 cores, 2 chips 1 16 cores, 2 chips
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2
Jun 29, 2016 | HTML | Text
14,400.78 1 16 cores, 2 chips 1 16 cores, 2 chip
| @Chris__Bailey

17. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2
Jun 29, 2016 | HTML | Text
14,121.47 1 16 cores, 2 chips 1 16 cores, 2 chips
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.2.1 on SPARC S7-2
Jun 29, 2016 | HTML | Text
14,400.78 1 16 cores, 2 chips 1 16 cores, 2 chip
| @Chris__Bailey

18. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle Weblogic Server Standard Edition Release 12.2.1 on Oracle Server X6-2
Mar 31, 2016 | HTML | Text
NC 1 44 cores, 2 chips 1 44 cores, 2 chips
| @Chris__Bailey

19. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle Weblogic Server Standard Edition Release 12.2.1 on Oracle Server X6-2
Mar 31, 2016 | HTML | Text
NC 1 44 cores, 2 chips 1 44 cores, 2 chips
| @Chris__Bailey

20. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

21. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

22. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

23. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

24. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
- Intel® Xeon® processor E5- 2600 v3 @3.6GHz
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

25. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
§ Calculate transactions per core
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

26. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
§ Calculate transactions per core
- 21,504.30 / 36 = 597.3 EjOPS/core
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

27. Capacity Planning
| @Chris__Bailey
Time
Capacity

28. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load

29. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Purchasing
Delay

30. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Purchasing
Delay

31. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Purchasing
Delay
Capacity Requirement

32. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Purchasing
Delay
Capacity Requirement

33. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Purchasing
Delay
Predicted Load + 10%
Capacity Requirement

34. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Purchasing
Delay
Predicted Load + 10%

35. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Purchasing
Delay
Predicted Load + 10%

36. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Purchasing
Delay

37. Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Purchasing
Delay
Excess Capacity
Excess Capacity

38. Capacity Planning
§ Size for peak demand
- Allow for Black Friday / Cyber Monday
- Or just changes in load throughout each day
| @Chris__Bailey

39. Capacity Planning
§ Size for peak demand
- Allow for Black Friday / Cyber Monday
- Or just changes in load throughout each day
§ Allow for High Availability (HA) and Disaster Recovery (DR)
- Rule of three for HA - 50% extra capacity
- Full fail-over for DR - 100% extra capacity
| @Chris__Bailey

40. Capacity Planning
§ Size for peak demand
- Allow for Black Friday / Cyber Monday
- Or just changes in load throughout each day
§ Allow for High Availability (HA) and Disaster Recovery (DR)
- Rule of three for HA - 50% extra capacity
- Full fail-over for DR - 100% extra capacity
§ Oh, and what about capacity for integration, stress and performance test?
| @Chris__Bailey

41. Most enterprises have hardware utilisation below 20%

42. Most enterprises have hardware utilisation below 20%
and below 10% for workloads with peaks

43. Capacity Planning and Agility
§ Time to obtain capacity affects operational agility
§ Market opportunities lost because of lead time for new projects
| @Chris__Bailey

44. Capacity Planning and Agility
§ Time to obtain capacity affects operational agility
§ Market opportunities lost because of lead time for new projects
§ Case Study: A fashion retailer can show measureable increase in sales if a item
similar to that seen in the media can be placed on their on-line store landing page
within 1 hr of it appearing in public.
§ Each product placement is different so they need a fast, agile, approach that does
not jeopardize their on-line stores availability and quality.
| @Chris__Bailey

45. Cloud Nirvana

46. What Cloud Promises
“a virtual, dynamic environment which maximizes use, is infinitely scalable,
always available and needs minimal upfront investment or commitment”
§ Take your code – host it on someone else's machine and pay only for the resource
you use for the time you use it
§ AND be able to do that very quickly and repeatedly in parallel
| @Chris__Bailey

47. Cloud Values
§ Much shorter “purchasing delay”
§ Return integration, stress and performance test environments after use
§ Cloud guarantees remove need for DR
| @Chris__Bailey

48. Traditional Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Purchasing
Delay
Excess Capacity
Excess Capacity

49. Cloud Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Excess Capacity
Excess Capacity

50. Cloud Capacity Planning
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Excess Capacity

51. Capacity Planning and Agility
§ Case Study: A fashion retailer can show measureable increase in sales if a item
similar to that seen in the media can be placed on their on-line store landing page
within 1 hr of it appearing in public.
§ Each product placement is different so they need a fast, agile, approach that does
not jeopardize their on-line stores availability and quality.
| @Chris__Bailey

52. Capacity Planning and Agility
§ Case Study: A fashion retailer can show measureable increase in sales if a item
similar to that seen in the media can be placed on their on-line store landing page
within 1 hr of it appearing in public.
§ Each product placement is different so they need a fast, agile, approach that does
not jeopardize their on-line stores availability and quality.
Achievement Unlocked!
| @Chris__Bailey

53. Other advantages of Cloud
§ Globally distributed data centres and locality
| @Chris__Bailey
https://aws.amazon.com/about-aws/global-infrastructure/

54. Other advantages of Cloud
§ Globally distributed data centres and locality
| @Chris__Bailey
https://aws.amazon.com/about-aws/global-infrastructure/

55. Trouble in Paradise?

56. OpEx vs CapEx…
§ Rent vs Buy
| @Chris__Bailey

57. OpEx vs CapEx…
§ Rent vs Buy
§ No upfront cost, but no end to cost
| @Chris__Bailey

58. OpEx vs CapEx…
§ Rent vs Buy
§ No upfront cost, but no end to cost
§ Drive for operational efficiency
| @Chris__Bailey

59. OpEx vs CapEx…
§ Rent vs Buy
§ No upfront cost, but no end to cost
§ Drive for operational efficiency
§ Focus on ROI for individual apps and app features
| @Chris__Bailey

60. How is Cloud cost calculated?
| @Chris__Bailey
Provider Type Memory CPUs Cost/month
Amazon EC2 Linux t2.nano 512MB 1 vCPU $4.68
Linux t2.micro 1GB 1 vCPU $9.36
Linux t2.small 2GB 1 vCPU $18.72
Linux t2.medium 4GB 2 vCPUs $37.44
Digital Ocean Standard 512MB 1 Core Processor $5.00
Standard 1GB 1 Core Processor $10.00
Standard 2GB 2 Core Processors $20.00
Pivotal Cloud Foundry App Instance 512MB 4 vCPUs $10.80
App Instance 1GB 4 vCPUs $21.60
App Instance 2GB 4 vCPUs $43.20
Heroku Standard 1x 512MB 1 Share $25.00
Standard 2x 1GB 2 Shares $50.00
IBM Bluemix Instant Runtimes 512MB 4 vCPUs $24.15
Instant Runtimes 1GB 4 vCPUs $49.35
Containers 512MB 4 vCPUs $10.22
Containers 1GB 4 vCPUs $20.59

61. $£€¥ = GB/Hr

62. Movement from CPU to Memory based costs is a
fundamental shift for Application Architectures

63. § Find a representative benchmark?
- SPECjEnterprise2010
§ Analyse the results
- Intel® Xeon® processor E5- 2600 v3 @3.6GHz
Capacity Planning
Tested By System Name Result
Java EE Server DB Server
Nodes CPU Nodes CPU
Oracle Corporation
Oracle WebLogic Server Standard Edition Release 12.1.3 on Oracle Server X5-2
Apr 1, 2015 | HTML | Text
21,504.30 1 36 cores, 2 chips 1 36 cores, 2 chips
| @Chris__Bailey

64. Reading the fine print…
Hardware Vendor: Oracle Corporation
Model Name: Oracle Server X5-2
Processor: Intel Xeon processor E5-2699 v3 (Intel Turbo Boost Technology up to 3.6 GHz)
MHz: 2300
# of CPUs: 36 cores, 2 chips, 18 cores/chip, 2 threads/core (Hyper-Threading)
Memory (MB): 262144
Disks: 2x 600GB SAS-2 HDD and 2x 400GB SSD
Network Interface: 4x 10Gbit onboard and 3x 10GBit Dual-port NICs
JVM Options:
-server -Xms18g -Xmx18g -Xmn16g -Xss256k -XX:MetaspaceSize=200M -XX:+AggressiveOpts -XX:+UseParallelOldGC -XX:ParallelGCThreads=9 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
-XX:InitialCodeCacheSize=200m -XX:ReservedCodeCacheSize=200m -XX:+UseCompressedOops -XX:-UseBiasedLocking -XX:+AlwaysPreTouch -XX:InitialSurvivorRatio=24 -XX:SurvivorRatio=24 -XX:TargetSurvivorRatio=90
-XX:-UseAdaptiveSizePolicy -XX:+UseLargePages -Djava.net.preferIPv4Stack=true -Djaxws.transport.streaming=true -XX:-TieredCompilation -Dweblogic.ProductionModeEnabled=true
-Dweblogic.management.discover=false -Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true -Doracle.jdbc.defaultRowPrefetch=200
-Djavax.xml.parsers.DocumentBuilderFactory=weblogic.xml.jaxp.RegistryDocumentBuilderFactory -Dweblogic.threadpool.MinPoolSize=30 -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0
-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory
-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=2
Server instances were started using numactl binding 1 instance per 9 cores.
| @Chris__Bailey

65. Reading the fine print…
Hardware Vendor: Oracle Corporation
Model Name: Oracle Server X5-2
Processor: Intel Xeon processor E5-2699 v3 (Intel Turbo Boost Technology up to 3.6 GHz)
MHz: 2300
# of CPUs: 36 cores, 2 chips, 18 cores/chip, 2 threads/core (Hyper-Threading)
Memory (MB): 262144
Disks: 2x 600GB SAS-2 HDD and 2x 400GB SSD
Network Interface: 4x 10Gbit onboard and 3x 10GBit Dual-port NICs
JVM Options:
-server -Xms18g -Xmx18g -Xmn16g -Xss256k -XX:MetaspaceSize=200M -XX:+AggressiveOpts -XX:+UseParallelOldGC -XX:ParallelGCThreads=9 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
-XX:InitialCodeCacheSize=200m -XX:ReservedCodeCacheSize=200m -XX:+UseCompressedOops -XX:-UseBiasedLocking -XX:+AlwaysPreTouch -XX:InitialSurvivorRatio=24 -XX:SurvivorRatio=24 -XX:TargetSurvivorRatio=90
-XX:-UseAdaptiveSizePolicy -XX:+UseLargePages -Djava.net.preferIPv4Stack=true -Djaxws.transport.streaming=true -XX:-TieredCompilation -Dweblogic.ProductionModeEnabled=true
-Dweblogic.management.discover=false -Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true -Doracle.jdbc.defaultRowPrefetch=200
-Djavax.xml.parsers.DocumentBuilderFactory=weblogic.xml.jaxp.RegistryDocumentBuilderFactory -Dweblogic.threadpool.MinPoolSize=30 -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0
-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory
-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=2
Server instances were started using numactl binding 1 instance per 9 cores.
| @Chris__Bailey

66. Reading the fine print…
Hardware Vendor: Oracle Corporation
Model Name: Oracle Server X5-2
Processor: Intel Xeon processor E5-2699 v3 (Intel Turbo Boost Technology up to 3.6 GHz)
MHz: 2300
# of CPUs: 36 cores, 2 chips, 18 cores/chip, 2 threads/core (Hyper-Threading)
Memory (MB): 262144
Disks: 2x 600GB SAS-2 HDD and 2x 400GB SSD
Network Interface: 4x 10Gbit onboard and 3x 10GBit Dual-port NICs
7.1GB / core
JVM Options:
-server -Xms18g -Xmx18g -Xmn16g -Xss256k -XX:MetaspaceSize=200M -XX:+AggressiveOpts -XX:+UseParallelOldGC -XX:ParallelGCThreads=9 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
-XX:InitialCodeCacheSize=200m -XX:ReservedCodeCacheSize=200m -XX:+UseCompressedOops -XX:-UseBiasedLocking -XX:+AlwaysPreTouch -XX:InitialSurvivorRatio=24 -XX:SurvivorRatio=24 -XX:TargetSurvivorRatio=90
-XX:-UseAdaptiveSizePolicy -XX:+UseLargePages -Djava.net.preferIPv4Stack=true -Djaxws.transport.streaming=true -XX:-TieredCompilation -Dweblogic.ProductionModeEnabled=true
-Dweblogic.management.discover=false -Dweblogic.diagnostics.debug.DebugLogger.DISABLED=true -Doracle.jdbc.defaultRowPrefetch=200
-Djavax.xml.parsers.DocumentBuilderFactory=weblogic.xml.jaxp.RegistryDocumentBuilderFactory -Dweblogic.threadpool.MinPoolSize=30 -Dsun.net.inetaddr.ttl=0 -Dnetworkaddress.cache.ttl=0
-Dcom.sun.xml.ws.spi.db.BindingContextFactory=com.sun.xml.ws.db.glassfish.JAXBRIContextFactory -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory
-Dweblogic.MuxerClass=weblogic.socket.NIOSocketMuxer -Dweblogic.SocketReaders=2
Server instances were started using numactl binding 1 instance per 9 cores.
| @Chris__Bailey

67. § Project Management Triangle
§ Fast
§ Good
§ Cheap
§ Pick any two…
Performance Optimisation
Euler Diagram
Cheap
Fast Good
| @Chris__Bailey

68. § Performance Triangle
§ Throughput
§ Latency
§ Memory footprint
§ Enterprise software has traditionally chosen
throughput and latency over memory
§ Data caching
§ Garbage Collection
§ Function Inlining
Low
Memory
High
Throughput
Low
Latency
Performance Optimisation
Euler Diagram
| @Chris__Bailey

69. Garbage Collection Tradeoff
| @Chris__Bailey
Memory
Heap Size

70. Garbage Collection Tradeoff
| @Chris__Bailey
Memory
Live in-use data
(Retained Set)
Heap Size

71. Garbage Collection Tradeoff
| @Chris__Bailey
Memory
Live in-use data
(Retained Set)
Heap Size
Temporary Data
(Garbage)
Temporary Data
(Garbage)

72. Garbage Collection Tradeoff
| @Chris__Bailey
Memory
Live in-use data
(Retained Set)
Heap Size
Temporary Data
(Garbage)
Temporary Data
(Garbage)

73. Garbage Collection Tradeoff
| @Chris__Bailey
Memory
Live in-use data
(Retained Set)
Heap Size
Temporary Data
(Garbage)
Temporary Data
(Garbage)

74. Garbage Collection Tradeoff
| @Chris__Bailey
Memory
Live in-use data
(Retained Set)
Heap Size
Temporary Data
(Garbage)
Temporary Data
(Garbage)
40 to 70%

75. Garbage Collection Tradeoff
| @Chris__Bailey
Memory
Live in-use data
(Retained Set)
Heap Size
Temporary Data
(Garbage)
Temporary Data
(Garbage)
40 to 70%
30 to 60% additional memory for additional performance

76. Optimising for the Cloud

77. Optimising for the Cloud
#1: Choose The Right Language

78. Memory Usage by Language
| @Chris__Bailey
0
17.5
35
52.5
70
C++
Mem usage (MB)
C

79. Memory Usage by Language
| @Chris__Bailey
0
17.5
35
52.5
70
C++
Go
Sw
i:
Rust
Mem usage (MB)
Modern NativeC

80. Memory Usage by Language
| @Chris__Bailey
0
17.5
35
52.5
70
C++
Go
Sw
i:
Rust
Ruby
PHP
Python
JavaScript
Erlang
Dart
Mem usage (MB)
Modern Native ScriptingC

81. Memory Usage by Language
| @Chris__Bailey
0
17.5
35
52.5
70
C++
Go
Sw
i:
Rust
Ruby
PHP
Python
JavaScript
Erlang
Dart
Java
Scala
Clojure
Mem usage (MB)
Modern Native Scripting JVM C

82. Throughput by Language
| @Chris__Bailey
C
0
25
50
75
100
C++
DuraOon

83. Throughput by Language
| @Chris__Bailey
Modern NativeC
0
25
50
75
100
C++
Go
Sw
i:
Rust
DuraOon

84. Throughput by Language
| @Chris__Bailey
Modern Native ScriptingC
0
25
50
75
100
C++
Go
Sw
i:
Rust
Ruby
PHP
Python
JavaScript
Erlang
Dart
DuraOon

85. Throughput by Language
| @Chris__Bailey
Modern Native Scripting JVM C
0
25
50
75
100
C++
Go
Sw
i:
Rust
Ruby
PHP
Python
JavaScript
Erlang
Dart
Java
Scala
Clojure
DuraOon

86. Normalised Throughput/Memory Language
| @Chris__Bailey
C
0
20
40
60
80
C++
DuraOon

87. Normalised Throughput/Memory Language
| @Chris__Bailey
Modern NativeC
0
20
40
60
80
C++
Go
Sw
i:
Rust
DuraOon

88. Normalised Throughput/Memory Language
| @Chris__Bailey
Modern Native ScriptingC
0
20
40
60
80
C++
Go
Sw
i:
Rust
Ruby
PHP
Python
JavaScript
Erlang
Dart
DuraOon

89. Normalised Throughput/Memory Language
| @Chris__Bailey
Modern Native Scripting JVM C
0
20
40
60
80
C++
Go
Sw
i:
Rust
Ruby
PHP
Python
JavaScript
Erlang
Dart
Java
Scala
Clojure
DuraOon

90. Selecting the Language
§ Ruby on Rails app, 100 instances at 500MB / instance
- 50GB of memory required at $24.15 GB/month
- = $14,490/year
| @Chris__Bailey

91. Selecting the Language
§ Ruby on Rails app, 100 instances at 500MB / instance
- 50GB of memory required at $24.15 GB/month
- = $14,490/year
§ Switching to Java provides a 7x saving:
- = $2,070/year
| @Chris__Bailey

92. Selecting the Language
§ Ruby on Rails app, 100 instances at 500MB / instance
- 50GB of memory required at $24.15 GB/month
- = $14,490/year
§ Switching to Java provides a 7x saving:
- = $2,070/year
§ Switching to Swift provides 42x saving:
- = $345/year
| @Chris__Bailey

93. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To

94. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To
C

95. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To
C
Modern
Native

96. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To
C
Modern
Native
Scripting

97. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To
C
Modern
Native
Scripting

98. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To
C
Modern
Native
Scripting

99. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To
C
Modern
Native
Scripting
JVM

100. Selecting the Language
| @Chris__Bailey
C++ Go Swift Rust Ruby PHP Python JScript Erlang Dart Java Scala Clojure
C++ 4.02 10.96 14.65 461.69 251.83 689.22 40.25 359.13 75.16 65.18 79.88 169.17
Go 0.25 2.73 3.64 114.79 62.61 171.37 10.01 89.29 18.69 16.21 19.86 42.06
Swift 0.09 0.37 1.34 42.11 22.97 62.86 3.67 32.75 6.86 5.95 7.29 15.43
Rust 0.07 0.27 0.75 31.52 17.19 47.06 2.75 24.52 5.13 4.45 5.45 11.55
Ruby 0.00 0.01 0.02 0.03 0.55 1.49 0.09 0.78 0.16 0.14 0.17 0.37
PHP 0.00 0.02 0.04 0.06 1.83 2.74 0.16 1.43 0.30 0.26 0.32 0.67
Python 0.00 0.01 0.02 0.02 0.67 0.37 0.06 0.52 0.11 0.09 0.12 0.25
JScript 0.02 0.10 0.27 0.36 11.47 6.26 17.13 8.92 1.87 1.62 1.98 4.20
Erlang 0.00 0.01 0.03 0.04 1.29 0.70 1.92 0.11 0.21 0.18 0.22 0.47
Dart 0.01 0.05 0.15 0.19 6.14 3.35 9.17 0.54 4.78 0.87 1.06 2.25
Java 0.02 0.06 0.17 0.22 7.08 3.86 10.57 0.62 5.51 1.15 1.23 2.60
Scala 0.01 0.05 0.14 0.18 5.78 3.15 8.63 0.50 4.50 0.94 0.82 2.12
Clojure 0.01 0.02 0.06 0.09 2.73 1.49 4.07 0.24 2.12 0.44 0.39 0.47
From
To
C
Modern
Native
Scripting
JVM

101. Optimising for the Cloud

102. Optimising for the Cloud
#2: Use Intelligent Scaling

103. Intelligent Scaling for Capacity
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Excess Capacity

104. Intelligent Scaling for Capacity
| @Chris__Bailey
Time
Capacity
Predicted Load
Capacity Requirement
Excess Capacity

105. Intelligent Scaling for Capacity
| @Chris__Bailey
Time
Capacity
Actual Load
Capacity Requirement
Excess Capacity

106. Intelligent Scaling for Capacity
| @Chris__Bailey
Time
Capacity
Actual Load
Capacity Requirement
Excess Capacity

107. Intelligent Scaling for Capacity
| @Chris__Bailey
Time
Capacity
Actual Load
Capacity Requirement
Excess Capacity

108. Dynamic Scaling to Load
| @Chris__Bailey
00:00
Capacity
Actual Load
Excess Capacity
12:00 18:0006:0000:00
Capacity

109. Dynamic Scaling to Load
| @Chris__Bailey
00:00
Capacity
Actual Load
Capacity
Excess Capacity
12:00 18:0006:0000:00

110. Dynamic Scaling to Load
§ Daily load cycles are common for user facing applications
- Particularly if your using localised availability zones
| @Chris__Bailey
“Most enterprises have hardware utilisation below 20%
and below 10% for workloads with peaks”

111. Dynamic Scaling to Load
§ Daily load cycles are common for user facing applications
- Particularly if your using localised availability zones
§ Intelligent Scaling can provide a 2x saving:
| @Chris__Bailey
“Most enterprises have hardware utilisation below 20%
and below 10% for workloads with peaks”

112. Dynamic Scaling to Load
§ Daily load cycles are common for user facing applications
- Particularly if your using localised availability zones
§ Intelligent Scaling can provide a 2x saving:
- Ruby on Rails app: $14,490/year
- Move to Java: $ 2,070/year
| @Chris__Bailey
“Most enterprises have hardware utilisation below 20%
and below 10% for workloads with peaks”

113. Dynamic Scaling to Load
§ Daily load cycles are common for user facing applications
- Particularly if your using localised availability zones
§ Intelligent Scaling can provide a 2x saving:
- Ruby on Rails app: $14,490/year
- Move to Java: $ 2,070/year
- Add scaling: $ 1,035/year
| @Chris__Bailey
“Most enterprises have hardware utilisation below 20%
and below 10% for workloads with peaks”

114. Optimising for the Cloud

115. Optimising for the Cloud
#3: Identify and Fix Memory Leaks

116. Look for and Fix memory leaks
§ A small leak means a very large
amount of memory over time
| @Chris__Bailey
American Society of Civil Engineers

117. Look for and Fix memory leaks
§ A small leak means a very large
amount of memory over time
§ Memory leaks act a little like
interest on a debt
| @Chris__Bailey
Live in-use data
(Principle)

118. Look for and Fix memory leaks
§ A small leak means a very large
amount of memory over time
§ Memory leaks act a little like
interest on a debt
| @Chris__Bailey
Live in-use data
(Principle)
Memory Leak
(Interest)

119. Look for and Fix memory leaks
§ A small leak means a very large
amount of memory over time
§ Memory leaks act a little like
interest on a debt
§ Made worse if you have to size the
container ahead of time to allow for
the leak
| @Chris__Bailey
Live in-use data
(Principle)
Memory Leak
(Interest)

120. Optimising for the Cloud

121. Optimising for the Cloud
#4: Memory Analysis is the New Performance Tuning

122. Memory Analysis
§ Most applications are memory in-efficient
- Duplication of data
- Oversized collections
- Empty collections
§ Initial analysis and optimisation usual has a large return on investment
| @Chris__Bailey

123. Memory Analysis
§ Java case study: PlantsByWebSphere sample in WebSphere AppServer 7
- 5 user test load, 206MB of “live data”:
| @Chris__Bailey

124. Memory Analysis
§ Java case study: PlantsByWebSphere sample in WebSphere AppServer 7
- 5 user test load, 206MB of “live data”:
| @Chris__Bailey
Collection Count Collection Size (MB)
Hashtable 262,234 26.5
WeakHashMap 19,562 12.6
HashMap 10,600 2.3
ArrayList 9,530 0.3
HashSet 1,551 1.0
Vector 1,271 0.04
LinkedList 1,148 0.1
TreeMap 299 0.03
TOTAL 306,195 42.9

125. Memory Analysis
§ Java case study: PlantsByWebSphere sample in WebSphere AppServer 7
- 5 user test load, 206MB of “live data”:
§ 42.9MB (16% of memory) are collection objects
- Just the collections themselves - excluding any data they store
| @Chris__Bailey
Collection Count Collection Size (MB)
Hashtable 262,234 26.5
WeakHashMap 19,562 12.6
HashMap 10,600 2.3
ArrayList 9,530 0.3
HashSet 1,551 1.0
Vector 1,271 0.04
LinkedList 1,148 0.1
TreeMap 299 0.03
TOTAL 306,195 42.9

126. Memory Analysis
§ Analysis of just the Hashtable instances showed:
| @Chris__Bailey

127. Memory Analysis
§ Analysis of just the Hashtable instances showed:
§ 127,016 of the Hashtables were completely empty
| @Chris__Bailey

128. Memory Analysis
§ Looking at empty collections across all collection types:
§ 52.6% of collections are empty (22.6MB)
§ Could save 11% of heap utilisation by removing empty collections
| @Chris__Bailey
Collection Count Empty % Empty
Hashtable 262,234 127,016 48.8
WeakHashMap 19,562 19,456 99.5
HashMap 10,600 7,599 71.7
ArrayList 9,530 4,588 48.1
HashSet 1,551 866 55.8
Vector 1,271 622 48.9
TOTAL 304,748 160,156 52.6

129. Memory Analysis
§ Looking at empty collections across all collection types:
§ 52.6% of collections are empty (22.6MB)
| @Chris__Bailey
Collection Count Empty % Empty
Hashtable 262,234 127,016 48.8
WeakHashMap 19,562 19,456 99.5
HashMap 10,600 7,599 71.7
ArrayList 9,530 4,588 48.1
HashSet 1,551 866 55.8
Vector 1,271 622 48.9
TOTAL 304,748 160,156 52.6

130. Optimising for the Cloud

131. Optimising for the Cloud
#5: Design for Memory Usage

132. Design for Memory
§ Practice lazy allocation of collections
| @Chris__Bailey

133. Design for Memory
§ Practice lazy allocation of collections
§ Don’t use default collection sizes
| @Chris__Bailey

134. Design for Memory
§ Practice lazy allocation of collections
§ Don’t use default collection sizes
§ Reallocate collections to shrink them if needed
| @Chris__Bailey

135. Design for Memory
§ Practice lazy allocation of collections
§ Don’t use default collection sizes
§ Reallocate collections to shrink them if needed
§ Use cache miss based strategies for cache sizes
| @Chris__Bailey

136. Summary

137. Cloud Economics