As presented at CloudOpen Japan in Tokyo in 2015. Today everyone is talking about clouds, and some are building them, but far fewer are operating successful clouds. In this session we'll examine a variety of paradigm shifts must IT make when moving from a traditional virtualization and management mindset to operating a successful cloud. For most organizations, without careful planning the hype of a cloud solution can quickly overcome its capabilities and existing best practices can combine to create the worst possible cloud scenario -- a cloud which isn't economical to operate, and which is more cumbersome to manage than a traditional virtualization farm. Key topics covered will include; transitioning the operational paradigm, the impact of VM density on operations and network management, and preventing storage cost from outpacing requirements.

1. CloudOpen Japan 2015
Controlling the cost of your first cloud

2. #whoami
Name: Tim Mackey
Current roles: XenServer Community Manager and Evangelist; occasional coder
Cool things I’ve done
• Designed laser communication systems
• Early designer of retail self-checkout machines
• Embedded special relativity algorithms into industrial control system
Find me
• Twitter: @XenServerArmy
• SlideShare: slideshare.net/TimMackey
• LinkedIn: www.linkedin.com/in/mackeytim

3. Clouds are cool
Agility
• Ease of provisioning w/o hardware latency
• Remove IT as a service delivery critical path
• Manage complete infrastructure, regardless of
scale
Lower operational expenses
• Consistent application and service deployment
• Drive reduced capital requirements
• Visibility into user and line of business usage

4. Why we want a cloud – the reality
Agility
• Ease of provisioning w/o hardware latency
• Remove IT as a service delivery critical path
• Manage complete infrastructure, regardless of
scale
Lower operational expenses
• Consistent application and service deployment
• Drive reduced capital requirements
• Visibility into user and line of business usage
Our boss bought the hype!!!

5. Why we want a cloud – the reality
Agility
• Ease of provisioning w/o hardware latency
• Remove IT as a service delivery critical path
• Manage complete infrastructure, regardless of
scale
Lower operational expenses
• Consistent application and service deployment
• Drive reduced capital requirements
• Visibility into user and line of business usage
Our boss bought the hype …
… but doesn’t want to look like an idiot

6. Today’s realities versus tomorrow’s norms
IT is a large capital expense
Work only happens in the office
IT does support and maintenance
Calling the helpdesk
IT is a monthly operating expense
Work happens wherever you are
IT drives strategic initiatives
On-demand, self-service IT
IT Today IT Tomorrow

7. Enterprise datacenter Cloud
• Architected for 100s of hosts
• Scale-up (server clusters)
• Applications assume reliability
• IT Management-centric
• Proprietary vendor stack
• Architected for 1000s of hosts
• Scale-out (multi-site server farms)
• Applications assume failure
• Autonomic [1:1,000’s]
• Open, value-added stack
Competing paradigms

8. Service offerings
Clearly define what you want to offer
• What types of applications
• Who has access, and who owns them
• What type of access
Define how templates need to be managed
• Operating system support
• Patching requirements
Define expectations around compliance and availability
• Who owns backup and monitoring

9. Define tenancy requirements
Department data local to department
• Where is the application data stored
Data and service isolation
• VM migration and host HA
• Network services
Encryption of PII/PCI
• Where do keys live when data location unknown
• Need encryption designed for the cloud
Showback to stakeholders
• More than just usage, compliance and audits

10. Let’s build a cloud ….

11. Where to start …
Deliver something easy
• Web services are a perfect start
• With experience grow the install base
Pay for what you need
• Use free hypervisors
• Use free orchestration services
Succeed early
• Remember your success is tied to a happy boss

12. Enterprise system assumptions
Redundant networking
• LACP or bonding
• Stacked switches
• Additional NICs
High performance storage
• iSCSI/ Fiber Channel
• Multiple paths
High density systems management
• Blade servers
• Data center wide operations management

13. Let’s get some shiny new gear ….
… because you can’t just reuse what you’ve got
Redundant, independent networks
• Management, storage, VM traffic  6 NICs
High performance storage
• 10 Gbps links for storage and VM traffic
• 1 Gbps for management traffic
Compute simplification
• Blade based servers
• Network fabric for network management
VM specification
• 2 vCPU, 8GB RAM, 40GB disk

14. The BOM – May 2015
Total hardware cost: 304 819 USD
Capacity 240 VMs with minimal overcommit
Component Cost
HP BladeSystem Enclosure w/Flex 10 Interconnects 68 893 USD
Compute blades (16x BL460c, dual socket, 128GB) 169 088 USD
Cisco Nexus switches (2x 5548UP) 36 620 USD
HP MSA Storage (24 drives/RAID 5/900GB) 30 218 USD

15. 1270 USD per VM
3086 USD for equivalent physical server

16. Did you build a cloud?
Answering to the Boss
Cost
• Did we just refresh some hardware?
• Have we improved anything?
• Is this just “business as usual”?
Capabilities
• Are we more agile?
• Can we take advantage of new paradigms?
• Is this easier to manage?

17. Scalability and failure
Fact
• Cloud providers have outages
• Networks go down
• Usage varies
Assertion
• Is failure a form of scalability?
• Can we leverage this in our design?
• Is our paradigm correct?

18. Attack the storage paradigm

19. Shared storage growth and provisioning time
1,000
500
VMs
Cost, AU100 200
500
VMs
Provisioning efficiency
AU – arbitrary units

20. Combined efficiency and storage evolution
Redesign
1,000
500
VMs
100 200 Cost, AU
VMs
1,000
500
Cost, AU100 200
?
Alternatives
AU – arbitrary units

21. Redesign
Efficiency and pod storage
1,000
500
VMs
100 200 Cost, AU
POD #1
POD #2
POD #3
1,000
500
VMs
100 200 Cost, AU
AU – arbitrary units
No redesign

22. What about local storage?
1,000
500
VMs
Cost, AU100 200
50
VMs
Provisioning efficiency
AU – arbitrary units

23. POD
trend
Traditional
trend
Cost-performance trends
Shared Storage Local Storage
1,000
500
VMs
Cost, AU100 200
1,000
500
VMs
100 200 Cost, AU
Local storage
Performance
trend
Local storage
trend

24. Understanding relationship between VM density and IO

25. Plan B

26. The power of local storage
Our IO requirements
• 300-400 IOPs per blade
• 3-4 SAS 10K disks, or SSD MLC
Our storage requirements
• 700-900 GB per blade
Options
• Storage blade (per pod)
• Local storage per blade
• Change to pizza boxes

27. Let’s get some shiny new gear ….
… because you can’t just reuse what you’ve got
Redundant, independent networks
• Management, VM traffic  4 NICs
No shared storage
• 10 Gbps links for VM traffic
• 1 Gbps for management traffic
Compute
• Rack based servers
VM specification
• 2 vCPU, 8GB RAM, 40GB disk

28. The BOM – May 2015
Total hardware cost: 166 652 USD
Component Cost
16x Dell R630 (dual socket, 128GB, 4x300GB SAS 10k) 130 032 USD
Cisco Nexus switches (2x 5548UP) 36 620 USD

29. 694 USD per VM
45% savings over “enterprise best practices”
77% savings over physical server

31. Understanding what you want to accomplish
Public clouds are people carriers and minibuses
YOUR cloud should be a race car
• Optimize it for your needs
Don’t rent what you can own cheaper
• Cloud operator doesn’t care about your success
• Optimized applications might be key
Ensure you have backup plans
• Usage can and does spike
• Outages can and do happen
vs.

32. Virtualization infrastructure choices
Hypervisor defined by service offerings
• Don’t select hypervisor based on “standards”
• Understand true costs of virtualization
• Multiple hypervisors are “OK”
• Bare metal can be a hypervisor
To “Pool” resources or not
• Is there a real requirement for pooled resources
• Can the cloud management solution do better?
• Real cost of shared storage

33. Understanding the operational costs of clouds
Instance provisioning
• Time from request to network configuration
• Time from request to VM start
• Time from VM start to application running
Application management
• Unified template to reduce compliance variance
• Configuration management for consistency
• Auto-scaling for cost control
Build Showback model
• Create spot opportunity cost for instances

34. Power utilization as ROI motivator
Server
Configuration
Power
Requirements
Heat Generation Cost of Power Operating Cost
(Annual)
Legacy DL360G6 18 790 W 64 072 BTU 0.17 USD/ kWh 56 000 USD
BL460 Blades 5 290 W 18 072 BTU 0.17 USD/ kWh 15 766 USD
Dell R630 5 710 W 19 420 BTU 0.17 USD/ kWh 17 018 USD
Hardware refresh ROI approximates 40 000 USD savings per year
7.57 Years for power savings using blades with shared storage
4.27 Years for power savings with rack servers

35. The ROI of a private cloud
Becoming a business partner to your organization
EC2 costs (US/East)
• m3.large (Linux) on-demand: 1352 USD per year
• m3.large (Linux) 3 yr reserved: 1609 USD for three years
• m3.large (RHEL) 3 yr reserved: 3348 USD for three years
Your cloud infrastructure costs
• 166 652 USD depreciated over 3 years: 231 USD per VM per year
• 17 018 USD power/cooling per year: 70 USD per VM per year
Become a service provider to your business
• Provide on demand services, which scale with the business
• Through business understanding, retain operational control of IT, simplify compliance monitoring
• Demonstrate opportunity costs for cloud services

36. Tying it all together
1. Define success criteria
2. Evaluate current best practices
3. Define supported configurations
4. Decide on scaling model
5. Build costing model
6. Build your cloud
7. Scale to new workloads

38. Questions?

40. WORK BETTER. LIVE BETTER.