Join the team behind Cisco’s Project Shipped as they take you on a rare behind the scenes tour and deep dive in to real world highly available distributed microservices architecture. Understand the challenges they faced along and the thinking behind key design decisions they made on their journey from concept to product. Learn how they leveraged Continuous Integration, Continuous Deployment, DevOps patterns, Cisco Collaboration and Cisco Cloud Services to deliver a high quality product to market less than 6 months. Here how they team plans to scale the 100x over the next year.

1. Microservices Case Study
Shipped
Neelesh Pateriya
@ Cloud Innovation Lab - CTO (Ken Owens)

2. • Introduction
• Decision Timeline
• Architectural Choices
• Micro Services – Project Management
• Conclusion
Agenda

3. Shipped Hybrid DevOps platform
enables developers to easily
Build, Deploy and Run
Containerized Microservices

4. Project
Management
Continuous
Integration
Shipped 1.0
Source
Control
Continuous
Deployment
Application
Orchestration
Developer
Tools
Service
Assurance
Consul
Kafka
Elasticsearch
InfluxDB
Issue
Management
Mesos
Marathon
ConsulTerraformDroneGitHubGitHub
Cisco Spark
Build Packs
Eclipse
Vagrant
Docker GitHub
Collaboration

5. Shipped 10,000ft View
Environment
Shipped
GitHub Drone
CCS Region 2CCS Region 1
Microservice
Infrastructure
Microservice
Infrastructure
Shipped
Deploy
MetaCloud
Microservice
Infrastructure
Marketplace
Catalog
Laptop
Sandbox
Developer

6. Microservices Architecture
Shipped-
Core
Deploy
Event
Build
Logging
Repository
Container
CCS Region 2CCS Region 1
Microservice
Infrastructure
Microservice
Infrastructure
MetaCloud
Microservice
Infrastructure
Cloud OS : Mesos-Marathon
Service Discovery: Consul
1. Registrator
2. Marathon-HAProxy.
Logging Framework:
1. Logstash, ES, Kibana
2.Logstash, HDFS, ZoomData
Market
Place
Metric

7. Microservices Incremental Development
Idea Brainstorm
• Identified key
services
components
Investigation
Prototype
• Validation of
idea by
creating
monolithic.
Development
Version 0.1
• Core API
Service
• Infrastructure
services
Development
release 0.2
• Event Service
• User Interface
• SQL to
Cassandra
Development
release 0.3
• Build Service
• Cassandra As
A Service.
• Container
repository
service
Development
release 0.4
• Logging
service
• Market Place
• ….

8. • Drone
• Builds source and runs unit tests in docker container (CI Slave).
• Very flexible build slave with developer capable of defining environment.
• Developer defines build, test, publish steps, no central admin.
• Open source product, Cisco shipped team contribute to build next gen
version.
Continuous Integration

9. Support multiple clustering in longer (first/default)
• Mesos-Marathon
• Kubernetes
• Docker Swarm
• Mesos-Kubernetes
Docker Clustering Choice

10. Micro Services Infrastructure
Marathon
Controller
Node
dnsmasq
Mesos
Leader
Consul
Server
Registrator
Zookeeper
Docker
Linux
Mesos
Slave
Resource Node
dnsmasq
Consul
Slave
Registrator HAProxy
Docker
Linux
App App

11. Multi Datacenter: Micro Services Discovery
1. Consul,Registrator, ConsulTemplate
HAProxy
2. Marathon/HAProxy, Consul

12. Service Discovery & Load Balancer
MicroService-1
MicroService-1MicroService-1
(Github repo)
MicroService-1
MicroService-1MicroService-2
(Github repo)
MicroService-1
MicroService-1Dependency-1
(Marketplace App)
MicroService-1
MicroService-1
Dependency-2
(Marketplace App)
Docker containers
Mesos – Marathon (MS-infra)
Deploy Env. (Staging)
(MS-Infra)
(MS-Infra)
Consul
Template
HAProxy
Consul
(DNS)
Registrator
1. Registrator monitors Docker events
2. Registrator adds docker instances
host:port to service-name mapping to
consul.
3. Consul exposes information via inbuilt
DNS
4. Consult templates watches changes in
Consul
5. HAProxy configuration is updated based
on changes in consul
6. Namespace naming convention
environment.project.service.shipped.com

13. Cassandra v/s SQL
Node1
Node2Node3
Node1
Node2Node3
Multiple Datacenters
DC1 DC1

14. Timeline: Microservices and Team Growth
Idea Brainstorm
• Identified key
services
components
• 1 Pizza Size
Investigation
Prototype
• Validation of
idea by
creating
monolithic.
• 1 Pizza Size
Development
Version 0.1
• Core API
Service
• Infrastructure
services
• 2 Pizza size
(teams)
Development
release 0.2
• Event Service
• User Interface
• SQL to
Cassandra
• 3 Pizza Size
Development
release 0.3
• Build Service
• Cassandra As
A Service.
• 4 Pizza size
team
• Split Scrum
Development
release 0.4
• Logging
service
• Container
repository
service
• ….
• 4-5 Pizza size
team

15. • Day 1 design consider Microservices .
• Evaluate each service independently for scalability.
• Microservice enables easily changing service internal
implementation with minimal impact to consumer.
• Microservices not only enables breaking software into
manageable size but it also make team size manageable.
Conclusion

16. Thank you