The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

Apache
Cloudstack :
A not-so-cloudy
future
@chiradeep
FUTURE Collab 2012

Tuesday, December 4, 2012 1

Disclaimer


Disclaimer
• Prognostications
– From experience
– But feel free to disagree


Disclaimer
– From experience
• Not laying out a roadmap
– Or timeline


Disclaimer
– From experience
• Not laying out a roadmap
– Or timeline
• I reserve
– The right to change my mind
– The right to be wildly optimistic
– The right to be biased (to AWS)
– The right to make a fool of myself


Not covered
• In the long run
…we are all dead
• In the very short term
… point features, critical bug ﬁxes
… important but


Not covered
• In the long run
…we are all dead
• In the very short term
… point features, critical bug ﬁxes
… important but
• Consult jira and the wiki and mailing list
– https://cwiki.apache.org/conﬂuence/display/CLOUDSTACK/CloudStack
+Next+Release


Inﬂuencers


Inﬂuencers

End-users


Inﬂuencers

Developers

End-users


Inﬂuencers

Developers
Vendors

End-users


Inﬂuencers

Developers
Vendors

End-users

Operators


Inﬂuencers
Contributors

Developers
Vendors

End-users

Operators


Inﬂuencers
Contributors

Developers
Vendors

End-users

Operators

These are in turn inﬂuenced by…


Outside Inﬂuences


Outside Inﬂuences

AWS


Outside Inﬂuences

OTHER STACKS

AWS


Outside Inﬂuences

OTHER STACKS
Other clouds
(GCE)

AWS


Outside Inﬂuences
Secular Trends

OTHER STACKS
Other clouds
(GCE)

AWS


Secular forces
• Moore’s Law
• DevOps / Developer as kingmaker
• OSS ecosystems
• Mobility
• Power / green computing
• etc


Secular forces
• Moore’s Law
• OSS ecosystems
• Mobility
• etc

Can’t impact, but:


Secular forces
• Moore’s Law
• OSS ecosystems
• Mobility
• etc

• Beneﬁt from it


Secular forces
• Moore’s Law
• OSS ecosystems
• Mobility
• etc

• Beneﬁt from it
• Anticipate it


End-users


End-users
• Features, Features, Features
– Clear roadmap
– Well documented guides
– Fast pace of delivery


End-users
• Features, Features, Features
– Clear roadmap
– Well documented guides
– Fast pace of delivery
• What does the future hold?
– More features, quicker delivery
– Javelin refactor should help
– https://cwiki.apache.org/conﬂuence/display/
CLOUDSTACK/Javelin


End-users


End-users
• Stability
– Just works
– Understandable error messages
– Monitoring


End-users
• Stability
– Just works
– Understandable error messages
– Monitoring
• What does the Future hold?
– Continuous improvement / bug ﬁxes
– Continuous integration, automated testing


End-users


End-users
• Smooth upgrades
– Zero – to – limited – downtime (API availability)
during upgrades


End-users
• Smooth upgrades
– Zero – to – limited – downtime (API availability)
during upgrades
– Javelin branch: Modular architecture implies
individually upgradable components
– Separate api endpoints and orchestration


End-users


End-users
• Principle of least surprise
– Looks like a cloud they are used to


End-users
– Looks like a cloud they are used to
– More AWS API ﬁdelity
• Access control like IAM
• Multiple AZ per region
– CLI mode fully supported
– GCE module anybody?


AWS Platform


AWS Platform

AWS Global Infrastructure


AWS Platform

Networking



AWS Platform

Compute Storage Database

Networking



AWS Platform

Application Services


Networking



AWS Platform
Deployment and Administration



Networking



Mapping to CloudStack
FEATURE
means
not core to
CloudStack
but
CloudStack will work
extremely well
with it


Networking Platform


Networking Platform
• VPC
– Partial implementation already in CloudStack


Networking Platform
• VPC
• Direct Connect
– Somewhat covered by VPC private gateway
– Speciﬁc to AWS infrastructure design.


Networking Platform
• VPC
• Direct Connect
• Route 53 (DNS anycast infrastructure)


Networking Platform
• VPC
• Direct Connect
• Route 53 (DNS anycast infrastructure)
• Future
– Full-ﬁdelity of VPC implementation using SDN
– ENI
– ELB and EIP in VPC
– (More) Commercial Network Devices / Appliances
– IPv6


Storage Services


Storage Services
• S3
– Essential for a Multi-AZ deployment
• Multiple backends (NFS, Cloudian, Caringo, Basho CS)


Storage Services
• S3
• EBS
– CloudStack supports EBS semantics


Storage Services
• S3
• EBS
• Glacier


Storage Services
• S3
• EBS
• Glacier
• Storage Gateway


Storage Services
• S3
• EBS
• Glacier
• Storage Gateway
– Storage refactor == More EBS back-ends
– Zone-wide visibility of volumes
– HDFS backend to S3
– NFS secondary -> S3 sync


Compute Services
• EC2
– Full-featured already
• Autoscale
– Netscaler provider
• ELB
– Virtual router
– Netscaler
– F5


Compute Services
– EC2
• Dedicated instances
• Multi-instance launch
• Better API ﬁdelity
• GCE api endpoint?
– Autoscale
• Non-netscaler provider
– ELB
• Health checks
• SSL termination
• IPV6
• Brocade ADX
• Auto-scaled virtual router


Database Services


Database Services
• Dynamo DB (NoSQL)
– Apache Cassandra, Riak, Apache HBase


Database Services
• RDS


Database Services
• RDS
• ElastiCache


Database Services
• RDS
• ElastiCache
• What does the Future Hold?
– RDS and ElastiCache could be subprojects?
– Or new Apache projects


• CloudFront
• Elastic MapReduce (EMR)
• CloudSearch
• Simple Email Service (SES)
• Simple Queue Service (SQS)
• Simple Notiﬁcation Service (SNS)
• Simple Workﬂow Service (SWF)


• CloudFront
• Elastic MapReduce (EMR)
• CloudSearch
• Simple Email Service (SES)
• Simple Queue Service (SQS)
• Simple Notiﬁcation Service (SNS)
• Simple Workﬂow Service (SWF)
• CloudStack will enable other projects / vendors
that provide this
– Like S3
– IAM a key enabler


• CloudWatch
• IAM
• Management Console
• CloudFormation
• Elastic Beanstalk (PAAS)


• CloudWatch
• IAM
• Management Console
• CloudFormation
• Elastic Beanstalk (PAAS)
– New subprojects for CloudWatch and CloudFormation ?
– Better UI/UX experience
– IAM:
• Who, what, from where, when, for how long
• Expand to include more Governance:
– Call-out to governance service, e.g., approval, verify tickets


Developers


Developer Needs


Developer Needs
• Add new features faster
– Continuous integration
– Modular / loosely coupled architecture
– Well documented interfaces
– Use well-known libraries / frameworks
– Use best tool / language for task at hand


Developer Needs
• Add new features faster
– Continuous integration
– Modular / loosely coupled architecture
– Well documented interfaces
– Use well-known libraries / frameworks
– Use best tool / language for task at hand
• What does the future hold?
– Javelin branch for loose coupled architecture
– Well known components for DI, messaging,
packaging, build
– Jenkins improvements – continuous automated tests


Developer Needs


Developer Needs
• Fast code-debug-test cycle
– DevCloud
– Bare minimum moving parts (no system vms)


Developer Needs
• Fast code-debug-test cycle
– DevCloud
– Bare minimum moving parts (no system vms)
– Improvements to DevCloud
– Deployment without system vms
• Enable new hypervisors
• Laptop-based hypervisors (vbox, v. workstation,
v.fusion)
– System VM and virtual router will be decoupled/
separate modules


Developer Needs


Developer Needs
• Process
– Visibility into roadmap
– Code submission criteria needs to be easy
and non-onerous
– Clear guidelines on what belongs in
CloudStack


Developer Needs
• Process
– Visibility into roadmap
– Code submission criteria needs to be easy
and non-onerous
– Clear guidelines on what belongs in
CloudStack
– Process improvements
– Release managers to keep on top of things


Cloud Operators


Cloud operator needs


• Easy install
– #1 predictor whether the POC goes forward
– System VMs are boon or bane?


• Easy install
– #1 predictor whether the POC goes forward
– System VMs are boon or bane?
• What does the Future Hold:
– Optional System VMs
– More forgiving install methods


• Scalable
– Already proven in # of hypervisors
– Need scalable networking (SDN)


• Scalable
– Already proven in # of hypervisors
– Need scalable networking (SDN)
– Optional System VMs
– Productize native SDN (GRE-based)
– IPv6


• Flexibility
– Choice of hypervisors, networking providers,
storage providers
– Flexibility in orchestration


• Flexibility
– Choice of hypervisors, networking providers,
storage providers
– Flexibility in orchestration
– More hypervisors supported
• Hyper-V, Oracle VM, PowerVM, Solaris Zones
– Choice of SDN and networking providers
– Choice of storage backends (volume and object)
– Customizable orchestration ﬂows?


• Resilient
– Occam’s razor. Should not be more complicated than
necessary.
• Enterprise needs different from SP needs
– Handle zone failures
– Automated, semi-automated and manual recovery


• Resilient
– Occam’s razor. Should not be more complicated than
necessary.
• Enterprise needs different from SP needs
– Handle zone failures
– Automated, semi-automated and manual recovery
– Modular architecture means components are optional
– Databases are split
– Per-zone orchestration layer
– Well documented upgrade procedures


• Adaptive
– Graceful degradation and admission control
– Easy well-deﬁned upgrades
– Continuous deployment?


• Adaptive
– Graceful degradation and admission control
– Easy well-deﬁned upgrades
– Continuous deployment?
– Independently scaled stateless API endpoints
– Well documented zero-downtime upgrades
– Individual service upgrade


• Monitoring
• Secure


• Monitoring
• Secure
– Well documented attack surface so that it can
be defended.
– Informative logs
– Lots of instrumentation
– Well integrated monitoring backends


• Supportability
• Chargeback and usage


• Supportability
• Chargeback and usage
– Troubleshoot / replicate end-user problems
– Override automated recovery processes when
required
– Powerful end-user GUI
– Basic admin GUI
– Alerts


Cloud Operator Needs
• Support for more hypervisors
• Integrate existing virtualization
deployments into CloudStack


• Support for more hypervisors
• Integrate existing virtualization
deployments into CloudStack
– Optional system vm means support for
• LXC, PowerVM, LPAR, Solaris Zones, Baremetal
• Laptop hypervisors (Workstation, Fusion,vBox)
– Virtualization Import


• Integrate with existing datacenter
infrastructure / processes


• Integrate with existing datacenter
infrastructure / processes
– DNS, DHCP, LDAP, IPAM integration hooks
– Monitoring systems hooks
– Event-based integration into
• CMDB, Ticketing systems
• Service management, change management,
service desk, capacity planning, release planning


Vendors


Vendors
• Strong community


Vendors
• Well-documented integration points
– Plugins – vs – services


Vendors
• Testability outside of full-ﬂedged CS
deployment


Vendors
• Testability outside of full-ﬂedged CS
deployment
– More documentation
– Integration test suites that can run without CS


AWS Virtuous Cycle
Reduced

Prices

Lower
More

Infra
costs Customers

More

Economie AWS

s
of
Usage
More

Infrastruct
ure


What is CloudStack’s Virtuous
Great

So>ware

More
More

features operators

More


What is CloudStack’s Virtuous
Great

So>ware More vendors
Expanding ecosystem

More
More

features operators

More


The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (6)

Semelhante a The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

Semelhante a The Future of Apache CloudStack (Not So Cloudy) (Collab 2012) (20)

Mais de Chiradeep Vittal

Mais de Chiradeep Vittal (11)

Último

Último (20)

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)