Cloudstack at Spotify, a slightly updated version of the presentation I presented at the Cloudstack meetup hosted by WebMD in New York City.

1. Cloudstack
at Spotify
2013-04-10
/2013

2. About Spotify
Streaming music service
Bare metal, baby
Engineering offices in Stockholm, Gothenburg, NYC
Hundreds of engineers
Lots of services
Highly managed configuration

3. Challenges
Let developers be developers
Build awesome tools
Geographically distributed teams
Support different use cases
Develop, test, integrate with external parties

4. spvirtenv
KVM hypervisors
DNS as database
Warning: Bad idea!
Everything in the client
Access everywhere
No error handling

5. Spoticloud
Built on top of cloudstack
Command line tools
API Server
Async task execution
Integration with puppet, DNS
Not highly available
Something we can grow with

6. ssh key authentication
ssh-agent
Challenge/Response over http
Token gives access
LDAP key & group
github.com/spotify/crtauth

7. Grow your storage
High level of integration with guest os
Creates local storage backed block device
Script that checks for new devices
Logical volume grows
Triggers ext4 resize

8. Seamless config
Puppet class set on create
DNS generated on create
User created early

9. Cloudstack lessons
Killer feature: add hypervisors, provision guests
Moving fast
Lots of features you don’t use
Still an appliance mindset
Feedback could be improved

10. What we want to focus on
Improve the basic use cases
Ever added a hypervisor with wrong pod id?
Setting up a new cluster, what is secondary storage VM doing?
Building, packaging, system administration
Work around maven transitive dependency problems
Make less assumptions about netfilter configuration

11. Looking ahead
Large scale integration testing
Unified interface for private and public cloud
Spcloud in production
Storage

12. Thank you!
Noa Resare
noa@spotify.com
@blippie
Apr 2013

13. Yes, we’re hiring
spoti.fi/devops
Apr 2013