Microservices continuous delivery with mantl & shipped
•Transferir como PPTX, PDF•
0 gostou•285 visualizações
Microservices continuous delivery with MANTL & Shipped Running, building and deploying microservices is hard. Either if you try to chunk a monolith application into small pieces or want to start a project from scratch, you’ll need to figure out how to deal with: security, service discovery, networking, monitoring, persistence, orchestration and cluster management. Once you manage to have a microservices architecture in place, you’ll hit other challenges: scaling, infrastructure monitoring, building, running and shipping to your users. In this talk I’ll cover what you need to take into account when you run microservices and how those problems are addressed in MANTL I’ll also look into a continuous delivery pipeline for microservices using Shipped MANTL is an open source platform for building microservices started by Cisco. It combines the best open source technologies to deliver an out-of-the box open platform for microservices development. You can contribute to MANTL: https://github.com/CiscoCloud/mantl Shipped is a CI/CD tool that will be released later this year by Cisco and is natively integrated with MANTL. Shipped is in open beta now: ciscoshipped.io
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Microservices continuous delivery with mantl & shipped
Semelhante a Microservices continuous delivery with mantl & shipped (20)
Mais de Catalin Jora
Mais de Catalin Jora (6)
Último
Último (20)
Microservices continuous delivery with mantl & shipped
- 1. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin µservices Continuous Delivery with MANTL & Shipped Catalin Jora May 2016, Krakow Atmosphere Conference
- 2. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin About me 2 Catalin Jora engineer @ Container Solutions worked on Shipped @JoCatalin linkedin.com/in/JoCatalin catalin.jora@container-solutions.com
- 3. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Agenda 3 Software development: Monolith to microservices Rise of the containers Challenges with microservices: Putting everything together (MANTL) Make them available to your users/customers (Shipped) Q&A
- 4. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Software development : pre-DevOps 4
- 5. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Software development : DevOps 5
- 6. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Software development : Post-DevOps 6
- 7. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin7 How to move to microservices?
- 8. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin8
- 9. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin9
- 10. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin10 Security? Service discovery? Orchestration? Networking? Persistence? Monitoring? Cluster Management?
- 11. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin11 We start with Docker
- 12. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin12 Maybe add Mesos for scheduling?
- 13. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin13 And Consul for service discovery?
- 14. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin14 And Project Calico (canal) for the networking?
- 15. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin15 And ELK for the logs collection?
- 16. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin16 … 6 months later … Terraform & Ansible deployment to cloud providers etcd distributed key-value store for Calico Calico a new kind of virtual network Mesos cluster manager Marathon long running services Consul for service discovery Vault for managing secrets Docker container runtime collectd for metrics collection Logstash for log forwarding GlusterFS for container volume storage Traefik for proxying external traffic
- 17. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin17 Why are you building infrastructure and not your application?
- 18. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin18
- 19. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin What is MANTL ? 19 Mantl is a modern platform for rapidly deploying globally distributed microservices Open source Cloud agnostic Available now: github.com/ciscocloud/mantl
- 20. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin What problems does MANTL solve? 20 Networking Security Service Discovery Persistence Monitoring Orchestration Cluster Management And others...
- 21. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin What problems does MANTL solve? 21 Networking Security Service Discovery Persistence Monitoring Orchestration Cluster Management And others... Supports easier migration to microservices Solves integration conflicts Lowers configuration time More time for development Runs anywhere Avoid cloud locking End-to-End platform Customizable (open-source)
- 22. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin22 MANTL Architecture 3 types of components: ● Control ● Edge ● Resource
- 23. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin23 MANTL Architecture
- 24. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin24 MANTL Architecture
- 25. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin25 MANTL Architecture
- 26. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin26 Adding MANTL to your infrastructure
- 27. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin27 Adding MANTL to your infrastructure
- 28. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin28
- 29. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin29
- 30. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin30
- 31. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin31 Security Service discovery Orchestration Networking Persistence Monitoring Cluster Management
- 32. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin32 How to build them? Policies? How to run them? How to ship them? Cloud or on- premises? Infrastructure monitoring? Auto-scaling?
- 33. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin33
- 34. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin34 What is Shipped ? ● Shipped is a continuous delivery platform for microservices (cloud native apps) ● Commercial Cisco product ● Runs on top of MANTL ● Complements MANTL ● In the cloud or on premises ● In open Beta now: ciscoshipped.io
- 35. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin35 What problems does Shipped solve ?
- 36. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin What problems does Shipped solve? 36 CI/CD platform (build/deploy/run) Application intent engine Infrastructure monitoring Contiv (Infra Policy Enforcement)
- 37. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin37 Shipped overview
- 38. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin38 Shipped overview Application Intent
- 39. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Shipped - Implementing App Intent - Scaling
- 40. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Shipped - Implementing App Intent - Scaling
- 41. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Shipped - Implementing App Intent - Scaling
- 42. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin42
- 43. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin43 Security Service discovery Orchestration Networking Persistence Monitoring Cluster Management How to build them? How to run them? Infrastructure monitoring? Policies? How to ship them? Cloud or on- premises? Auto-scaling?
- 44. container-solutions.com | @containersolutiµservices CD with MANTL & Shipped | @JoCatalin Thanks, links & more 44 MANTL: mantl.io (contribute on ) Shipped: ciscoshipped.io (open beta) Microservices: Applications & Microservices with Docker & Containers - thenewstack.io martinfowler.com/articles/microservices.html Docker : Using Docker - Adrian Mouat Blogs: blogs.cisco.com/cloud container-solutions.com/blog
Notas do Editor
- CS is working on Shipped now and developed a few frameworks for MANTL
- Introducing the agenda for today. Why we need MANTL and Shipped? An overview of software architecture.
- How: developers create software and throw it over the wall to operations people. Who install it for them. They don’t know what it is and how it should operate. Problems: Slow cycle (Time to market slow. Hard to experiment) Ops blame Dev, Dev blame QA and Managers blame budget. Nobody has a full overview of what is happening (WALL of confusion). Monolith applications.
- DevOps movement promotes collaboration DEV and OPS start to work together. Mixed teams or just improved communication. You still have an elephant in the room. Faster time to market. Software is deployed in faster cycles. Problems: Still hard to experiment. Monolith still in place sometimes.
- Software is installed as a bundle of services that work together. Software is decoupled. Teams can deploy independent. We apply some smart logic to applications to make them ready for deployment. Ops maintain a platform (like MANTL) Developers are responsible for managing the application towards the entire cycle. FAST release cycle. Enables a CI/CD pipeline
- How can a company move to microservices architecture? How to get rid of the monolith application? How to decouple dependencies?
- Horizontal scaling (more instances) Data partitioning (split in more data formats) Split on functionality.
- OK, lets use Docker. Because this seems the way to go. Docker/containers is the new atomic unit of infrastructure.
- 3 years ago Docker emerged as a solution for packaging software in a “container”. Problems it solves: agility, portability, lighter footprint. Huge adoption from developers. 100.000+ members of docker meetups. Containers are dumb on their own.
- So we add docker. And me have a few containers. They can talk one to another. But once we need to move outside our development machine we realized we need to manage them somehow.
- We’ll use Mesos for scheduling our containers. We install marathon to schedule our containers so that we’re sure they can be restarted and scaled up/down. And we have a way to control what is running where.
- And we’ll use Consul so that one container can find another container. A web server can know where is the database and how to reach it. Because we use a distributed system now.
- And we’ll create a virtual network so that our containers can communicate in a secure way. Calico: Layer 3 Virtual networking
- We need to gather all the logs, so that we can debug everything. We use elastic search, kibana and logstash
- And we need a bunch of other tools to achieve the functionality we had in the monolith applications. And this takes time and makes adoption of microservices hard because those technologies need to be glued together.
- After six months of work, you realize that you didn’t build your application. You just build the services that are needed by your application.
- You don’t have to do this, because this is already in place. It’s called MANTL and it is developed by Cisco as an open source project. MANTL allows you to spend time on your application not on configurations.
- MANTL is a platform that allows you to create applications using microservices infrastructure. Includes best open source tools that are needed to develop an modern application architecture handles the heavy lifting for you out of the box. open sourced - can be tuned and modified, but can run anywhere (private cloud, datacenters, bare metal, cloud, openstack);
- Helps you to adopt microservices. Is configured to work out of the box. You don’t need to configure anything. Gives you more time to develop. Can run anywhere. Is not locking you on a specific cloud provider / datacenter. It has all you need so that you can start building applications on top of it. Open sourced - can be customized and improved.
- Mantle architecture contains 3 elements These are VM’s
- Control node - are managing the cluster Run Marathon for scheduling tasks, Mesos Master for management and Consul for service discovery.
- Resource nodes are running the containers, the applications.
- Edge nodes are running the Traefik service for proxying external trafic to the MANTL cluster.
- This is your current architecture: some VM’s and a few DB clusters.
- You install MANTL via some Ansible scripts, that will provision via Terraform a few VM’s.
- On every VM, there will be specific MANTL tools installed, depending on the role of the node.
- On top of that you will have the infrastructure services (logging, scheduling). Think of them as utilities in the house: water, gas, electricity.
- And on top of that you will get you cloud services: Replication, Automation. Think of them as furniture in the house. You can live without furniture, but not without electricity and water.
- MANTL resolves some of the problems and challenges you have with starting up microservices architecture.
- How to build, run and ship microservices? Policies - how to ensure some rules? Auto-scaling Where to run them? Infrastructure monitoring?
- Introducing Shipped. Let’s see how ships solves those problems and how it complements MANTL.
- CI/CD platform for microservices. Allows us to have cloud native applications (dynamically managed microservice apps packed in containers) Runs on top of MANTL and complements MANTL Can run on the cloud or on premises (because it is built on top of MANTL) Open Beta now
- Helps you run containerized applications Takes care of installation for you And assures you that they run as requested.
- So build/deploy/run can do a lot of other tools. What is special? App intent: an egine that contains rules for our application Extensive monitoring Contiv: rules for the infrastructure
- Shipped sits on top of mantl Leverages mantl plaftorm capabilities
- Shipped sits on top of mantl Leverages mantl plaftorm capabilities
- The green graph is our application. The red borders are soft and hard limits. From CPU to Storage and Security.
- As we increase the number of nodes we see how the application is behaving. We break the soft limits.
- And we go over the hard limits. And the application can’t go further. Notificatins will be send
- Shipped is going to help us to go from source code to end users via a fully automated CD pipeline. We start on the dev box. Where we have code and in the future a tool like miniMANTL. Code goes to gihub and Shipped picks it up. Code is build by shipped. We create docker images, that are deployed on various env. The code runs on top of a MANTl cluster (mesos, or kubernetes) We gather analytics out of it.
- MANTL resolves some of the problems and challenges you have with starting up microservices architecture.
- Some recommandations: Check the product websites. And some resources on microservices and devops.