Watson developer cloud delivers Watson Cognitive services as micro services on the cloud that are being used by many IBM Watson customers. The micro services were packaged in ova at the first release. There were some drawbacks in ova deployment in the cloud. We gradually switched to use docker. As a result, the service deployment time and start up time are significantly improved. It also greatly simplified our continuous delivery process since our services run on both Intel and Power platform and we have offerings on our public cloud, dedicated cloud as well as customers’ on premise cloud. With minimal deployment time and quick startup time, Docker makes our dynamic creation of service instance on the fly per customer request possible.
Use Docker to Deliver Cognitive Services Running Cross Platform and Multi Cloud Environments by Susan Diamond
1.
2. Use Docker to Deliver Cognitive
Services in Multi Cloud Environments
- The Watson Developer Cloud Use Case
Susan Diamond
Senior Software Engineer/Continuous Delivery Leader
6/20/2016
3. Agenda
• Watson/Watson Services/Watson Developer Cloud (WDC)
Introduction
• WDC Transition from VM to Docker
• WDC Continuous Delivery Process
• Things to Consider to Run Private Docker Registries
• Watson Service Demo
8. Watson Developer Cloud and Docker
• Started investigate Docker in mid 2014
• Lack of monitoring/management tooling
• Picked up Docker again in beginning of 2015 and use docker in production
environment since mid 2015
Some characteristics of Cognitive
services:
• Self-learning
• Training data/models
• Data security
• One service instance per customer
Dock Deployment
• Deployment time significantly improved
• Start up time significantly improved
• Enable dynamic service instance creation on the fly
• Enable seamless DevOps experience across multi-cloud environments in multi-GEOs
VM deployment Pain Points:
• Long deployment time
• Slow start up time
• Problematic in dynamic deployment
automation
9. Transitioning from VM to Docker
1. VM/docker image building,
bake the image once and
use for all environments.
2. Service deployment
3. Service registration
4. Service request routing
5. Full stack in private
network except DataPower
10. Canary & Red/Black Deployments/Updates
▪Design Philosophy: No destructive or in-place upgrades
▪New ASG deployed alongside existing ASG
▪#instances determine load (point in time)
▪Disable old ASG after sufficient confidence in new
– Can flip to old with a couple of clicks in case of an issue
11. Ubuntu VM
Docker
engine
Docker Registry
on Docker Container
Ubuntu VM
Docker
engine
Docker Registry
on Docker Container
Docker
container
1. Java 7
2. Tomcat 7
3. Side-car
4. Eureka
5. Zookeeper
6. Zuul
…
Imaginator
Container
1 base image
deb files
2. Services
debfiles
3. Dreamfiles
4. dockerfiles
…
SL object storage
Docker AAS
SL image
template
repository v3 VM Image template
v1 Deb files + dream file
d1 Deb files + dock file
d2 Store/retrieve deb files
v2 Store/retrieve
deb files
d3 Docker image
SL firewall
Jenkins
Git Enterprise
Ubuntu VM
Haproxy
source code
Imaginator server/Ubuntu VM
Docker
engine
Imaginator
VM AAS
• Design Philosophy: Every change should
be done via images
• Base images available to service teams
➢ Ubuntu 14.04 LTS
+ security hardening + agents (logging,
metrics)
+ JRE + Tomcat
+ Sidecar for non-Java services
• Service teams build their image on top of
base image
➢ “Dream” or Docker file describes
dependencies and build targets
➢ Gradle fpm plugin for Debian
package generation
• Output is Softlayer CCI image or Docker
image for deployment
Continuous Delivery - Image Baking Process
14. Service build Server
9.x
Docker deamon
SL object
storage
Docker images
Docker Registry
9.x.x.x:5000
SL object
storage
Docker images
Dev
Staging
Private Docker Registries in Public Cloud
Docker Registry
9.x.x.x:5001
haproxy
haproxy
Docker
deamon
Softlayer IBM intranet
Docker
deamon
push
push
pull
pull
dockerrepo-v2-01:5000 dockerrepo-v2-02:5000
dockerregistry1-v2:5000 dockerregistry2-v2:5000
SL object
storage
Docker images
Production
haproxy
Docker
deamon
pull
dockerregistry1-v2:5000 dockerregistry2-v2:5000
15. • Security
• Proprietary source code needs to be stored in a private registry
• Customer’s security requirement: fully isolated stack
• Network accessible to all docker agent in the environment.
• docker agents are in private network that is not accessible to
public network
• Docker registries need to available in each GEO for good
performance.
• Maintenance
• Maintaining the multiple private docker registries up to date
and operational needs resources.
Things to Consider to Run Private Docker Registries