After successfully moving a large workload from a Public Cloud to an OpenStack Private Cloud, the former TubeMogul Operations Engineering team tackled its next important step toward Cloud Bursting. While experiencing hyper-growth on the Adobe Advertising Cloud, the team had to figure out a simple way to quickly provision new compute resources. Our latency critical workload need our core private resources while some workload can safely leverage public cloud. Cloud Bursting helped to ensure rapid support of the business and provide a more flexible capacity planning strategy. Being able to burst some workload back to the Public Cloud allowed the team to leverage the best of public and private cloud. In this presentation, we will detail how the team dealt with the network complexity between clouds infrastructure, the cost effectiveness of cloud bursting, the impact on performance, and the hard reality of multi-cloud environments.

1. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Adobe Advertising Cloud: The Reality Of Cloud Bursting with OpenStack
Nicolas Brousse | Director, Operations Engineering

2. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Adobe Advertising Cloud
2
Cross-Channel
End-to-End
Audience at the Core Brand Safety
And
High Performance
Data-Driven Creative

3. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 3
Programmatic Ad Buying
Now: Software based purchase
with Real Time Bidding
Before: “RFP, IO, human based
orders”

4. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 4
Technical Challenges
Latency Requirement
<50ms @ 95th percentile
High Volume Traffic
+100 billions request a day
Huge Datasets
billions of object to store

5. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 5
Public Clouds Can Be Annoying
Lack of visibility on technical
limitations or performances
Costs controls and projections
can get cumbersome
Can become a blame game
without improvement options

6. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 6
Private Clouds Rock
You know everythingYou control everything You perform the best

7. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 7
Hybrid Cloud is Better
Can use the most efficient
technology and services based on
use cases
You end up with siloed
infrastructure and can’t truly
benefit of cross solutions
... But still a mess

8. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 8
Private Cloud: Scaling Challenges
• Can’t scale quickly enough
• Impact stability and growth

9. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 9
Implement Cloud Bursting
• Quickly overflow compute
resources to Public Cloud
• Help mitigate uncontrolled peak
• Buffer for procurement delays

10. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 10
Cloud Bursting: Not quite that easy…
• Can be hard to build right
• Can be costly to operate

11. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
CloudMogul
A Journey of Infrastructure Automation

12. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 12
CloudMogul: The Journey
2011 2012 2013 2014 2015 2016
Ideation
Driven by Costs
& Performances
Eucalyptus
OpenNebula
OpenStack
CloudStack
Evaluation
Technically
driven
Evaluate
Eucalyptus
EC2
Compatibility
Vendors Help
Resources
constraints
Zynga Model
Failed
Reset with
OpenStack
DIY
Design and build
to fit our need
Stick to OS core
GlusterFS is slow
Ceph can be
painful
In Production!
A Make it work
moment
Give up on EC2
compatibilities
Tune for the
workload
Growing!
Costs and
technically driven
Deployed more
locations
Scale, automate,
iterate

13. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 13
CloudMogul: Philosophy
Lean Operations Team
Do A Lot With A Little
Cattle Not Pets
Rack And Roll
Continuous Delivery
Automate Everything

14. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 14
6 OpenStack locations
Multi Cloud Region

15. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 15
CloudMogul: Multi-Cloud Challenges
• Long learning curve with many iterations needed
• Complexity of deployments, upgrades, or patching
• Many immature options with roadblock or instabilities
• Cloud API compatibility is an utopia
• Different behaviors (Network, Disks, CPU)
• No mature, on-premises, block storage options

16. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 16
CloudMogul: Going thru the Journey
• OpenStack: Stick to the core, keep it simple
• Simple Network Designs to scale, innovate, and
iterate quickly
• Keep hardware operations lean, use a VAR
• We are not in the 90s anymore: Leverage today
technologies

17. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 17
CloudMogul: Private Cloud Outcomes
• Observed 30% cost savings and reduced server footprint.
• Improved visibility on the full stack which helped for
troubleshooting and performance improvements.
• Have a strong technical need for it. Cost shouldn’t be
the only driving factor.

18. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 18
CloudMogul: New Challenges
Hyper growth and seasonal traffic peaks
Ø Difficult capacity planning
Ø Long delays to procure new hardware

19. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 19
CloudMogul: KISS Cloud Bursting
• Identify compute candidate to burst
• POC to Validate feasibility: SSH L2 Tunnel, AWS VPC
• Measure Service Impact: Latency, Volumes, Performances

20. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 20
CloudMogul: Cloud Bursting at Scale
• Size a dedicated network pipe: AWS Direct Connect 10G
• Link directly to OpenStack Network. Configure Routes.
• Automate scaling with Terraform
• Update code deployment on CI/CD

21. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 21
CloudMogul: Bursting Limits
• Understand constraints with
Network, Latencies, and Costs
• Burst as needed. Measure
service impact.

22. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 22
CloudMogul: The Journey
2011 2012 2013 2014 2015 2016
Ideation
Driven by Costs
& Performances
Eucalyptus
OpenNebula
OpenStack
CloudStack
Evaluation
Technically
driven
Evaluate
Eucalyptus
EC2
Compatibility
Vendors Help
Resources
constraints
Zynga Model
Failed
Reset with
OpenStack
DIY
Design and build
to fit your need
Stick to OS core
GlusterFS is slow
Ceph can be
painful
In Production!
A Make it work
moment
Give up on EC2
compatibilities
Tune for the
workload
Growing!
Costs and
technically driven
Deployed more
locations
Scale, automate,
iterate
2017
Expand Footprint in US
Implement Cloud Bursting
Terraform, Containers, SDN

23. © 2017 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 23
The Bottom Line:
It’s all about Infrastructure Automation
• Automate Everything, including network
• Handle hardware like software, Rack and Roll
• What was hard before is now easier
• Don’t lose yourself in tech trends,
focus to Delight Your Customers

24. @AdobeAdCloud @orieg