CICD Pipelines for Microservices: Lessons from the Trenches

•

1 gostou•291 visualizações

You have finally split your big monolith into microservices built on top of Kubernetes! Now what? How do you validate a more complex application? And how do you make it scale? In this live talk, we look at two case studies, Expedia’s journey to microservices, and Codefresh. If you try to treat microservices like monoliths you’ll end up with thousands of broken pipelines that are impossible to maintain. Learn from the mistakes of the past and let us show you how we fought our way to something much better! This live talk has everything, tech tips, best practices, and yes, even the fabled business value that our bosses all seem to care so much about!

Tecnologia

CI/CD Pipelines for Microservices
Best Practices
DAN GARFIELD

Dan
Garﬁeld
Chief Technology Evangelist
@todaywasawesome

Agenda
1. Why microservices
2. How Expedia approached
microservice CI/CD
3. How we do it at Codefresh
@todaywasawesome @codefresh

Why Microservices?
@todaywasawesome @codefresh

Monolith 1 Monolith 2@todaywasawesome @codefresh

Multiple services with single UI&UXMultiple monoliths with diff. UI&UX
Expedia corporate travel re-architecture
Uniﬁed cars UI
Car sortCar search
Cars
(Egencia)
Cars (ECT)
Cars
(Travelforce)
Booking service
Phone apps.

Moving to Microservices at Expedia
Approach
● Consolidate code bases and
● Build shared libraries for global platform. Ex:
○ Logging service, monitoring service
● Rely on manual integration testing
● Standardize CI/CD pipelines
● Use Maven for modularity
● Migrate to cloud from on-prem
Issues faced
● Geographically distributed
● Tools consolidation was hard
● Too many pipelines as
microservices grew.
( 100 pipelines → 1000+ )
● Pipelines not modular or
re-usable
● Jenkins master-slave issues
● Copypasta causing bad patterns
● Central team could not keep up
● Plugin upgrade was a nightmare

Moving to Microservices at Expedia
Lessons learned and Recommendations
● CI/CD templates should have been prioritized higher than “business needs”
● Bootstrapping new projects should have been externalized from the microservice and
adding a new microservice should have full pipeline setup once a repo is created
● A modular pipeline approach would ease the pain caused by different versions
● Reusability in CI/CD platform is critical
@todaywasawesome @codefresh

Single pipeline per
project
Can be
complex/difﬁcult to
be maintain
Usually led by a single
team (anti-devops)
Organizing pipelines for monolithic applications

Scalability issues with microservice pipelines
Does this look
like a plan??

Requires everyone to use
same version of library
Libraries often rely on each
other in complex ways
Changes have to go to
admins
Leads to big stability
problems
Relies on proprietary API
Organizing pipelines for monolithic applications

How Codefresh does
CI/CD for Microservices
1. Container-based pipelines
2. Shared pipelines
3. Deployment testing

Codefresh Architecture Diagram
MULTI CLOUD

Each task is built into a
Docker image.
Users can self-serve
these images.
Images DO NOT rely
on each other.
Containers can be
anything: go/node/c++
Container-based pipelines

Huge open source library at codefresh.io/steps

Maintain a single
pipeline
Make microservices
uniform
Change behavior
based on context
Use a single pipeline that operates with context

Triggers carry their context
Codebase
Tests
Docker-compose
Helm chart
...
Dependencies
...
Git checkout

Why Canary?
Usefulness of early testing
Complexity of
Infrastructure
Testing early
becomes less useful
as infrastructure
complexity rises
https://codefresh.io/events/canary-deployment-helm-istio-codefresh/

Summary
Shared pipelines > libraries
Reusable Docker images > Copypasta
Deployment validation with canary
Read the blog post at
https://codefresh.io/continuous-deployment/
ci-cd-pipelines-microservices/

Questions?
Dan Garﬁeld
@todaywasawesome
Want to try it yourself?
Open a FREE account
today at
Codefresh.io Kostis Kapelonis
@codepipes

Mais conteúdo relacionado

Mais procurados

Deploying containerized applications with Kubeapps

Janakiram MSV

Watch the demo: https://youtu.be/tmGSqK3BahU Konveyor Move2Kube helps automate your migration from various platforms such as cloud foundry, docker swarm, VM based applications, or from your custom framework to Kubernetes. With the latest release of Move2Kube, v0.3.0, it has acquired extensive customization capabilities to help create the target artifacts as per your organization requirements. In this session, the Move2Kube team will demonstrate the migration of a typical application to run on Kubernetes. It will be followed by a demonstration of customization capabilities of Move2Kube, such as customizing the generated Dockerfile, the directory structure, the helm chart and custom artifacts. Website: https://move2kube.konveyor.io/ GitHub repo: https://github.com/konveyor/move2kube/

Automate The Creation/Transformation of Infrastructure as Code Artifacts with...

Konveyor Community

For on-demand recording of the webinar, click here: https://youtu.be/aN_DaNFfBx4 Why You Should Watch Application developers are well-advised to not only think about their actual programming, related time-saving tools and deployment schemes, but also about the specific needs of application operations - in particular with regards to data privacy requirements when dealing with clients in Europe and their customer data which are processed in applications. That's where the Open Telekom Cloud kicks in. As the the "new kid on the block", Open Telekom Cloud's public cloud offering features Bitnami's vast range of open source applications. Join Bitnami as we host our featured speaker, Max Guhl, from Deutsche Telekom. He will showcase the Open Telekom Cloud's intuitive user interface and how this Public Cloud does not only smoothly integrate Bitnami's application catalog, but also provides answers on how to comply with the upcoming European General Data Protection Regulation already today Register now to watch and learn: What the Open Telekom Cloud is How to launch and manage DevOps tools and instances on Open Telekom Cloud Actions to keep the new European General Data Protection Regulation (GDPR) in mind The benefits of using Bitnami with Open Telekom Cloud

Thinking One Step Further with Time-saving DevOps Tools with Open Telekom Clo...

Bitnami

A mini adventure in Minikube devopsdays CT 2017

Whitney Tennant

Watch the presentation: https://youtu.be/s-0dXfjfPkA The eternal struggle of application development is choosing to pay down technical debt or adding new features. Why not both! This meetup will explore modernization strategies enabled by open-source project KubeVirt that will help you do just that. Legacy applications are usually monolithic and run on one or more virtual machines. Some applications are easier to modernize if they have well established counterparts in the containerized world (EAP, Spring Boot, etc). Large classic .Net applications running on IIS in Windows server are a lot harder to modernize in one shot. KubeVirt allows you to import your existing VM workloads into Kubernetes and modernize your application in stages. With KubeVirt, Virtual Machines are first-class citizens in Kubernetes and have access to all the artifacts that pods do, including being able to access and be accessed using service endpoints. Once you have the VM running in your Kubernetes project, you can start to modernize and extend the functionality of your application. We will import a .Net application running on IIS on a Windows Server VM into KubeVirt. Then we will go through the stages of containerizing each of the logical layers of the application. Note that this strategy can be used with other OS and Middleware combinations. Presenter: Arvin Amirian, Principal Consultant for Container Infrastructure @ Red Hat

How to Modernize Virtualized Workloads

Konveyor Community

Helm at reddit: from local dev, staging, to production

Gregory Taylor

Create a One Click Migration (OCM) process to Automate Repeatable Infrastruct...

Quantyca - Data at Core

Watch presentation: https://youtu.be/9hDdg_Beui4 Despite the incredible pace of adoption of container orchestration platforms, the vast majority of EAP workloads are still running on VMs or bare metal. In a lot of cases enterprise operation teams are mandated to modernize and move these workloads to the cloud, and containerization and migration to Kubernetes is the natural destination. When talking about this migration path, we're often asked questions like: What's involved? How easy is it to move these workloads? How can you be sure of no code changes? What tools are there to assist with this effort? What are the benefits of moving workloads to Kubernetes? In this meetup, Philip Hayes, Runtimes Practice Lead at Red Hat, will provide answers to these questions and also include a step-by-step guide to migrating an EAP 7 application to Kubernetes.

Migrating Java JBoss EAP Applications to Kubernetes With S2I

Konveyor Community

2017 was a banner year for change and disruption at Lockheed Martin. Specifically Enterprise IT, moving to cloud and taking a chance on Containerization as a Service with Docker. As innovation accelerates and customer needs rapidly evolve, Lockheed Martin must become increasingly agile. Rapidly responding to customer requirements is key, and to facilitate overall business goals, Enterprise IT needs to be agile. We are experiencing software development lifecycles moving from waterfall to agile models of development. These changes are moving downstream toward Enterprise IT operations with the evolution of DevOps. In order to meet the demands of the customer and make significant progress on our cloud journey, we needed to look at changing how we manage infrastructure, how we change our IT culture and ultimately how we innovate at scale. That’s where the story of i2 labs begins. This presentation will focus on the creation of i2 labs at Lockheed Martin, which inspired and enabled people and processes to invest in Docker as a company, Containerization as a technology and DevOps as a methodology. How we went from labs testing to building Agile Development Teams to preparing to enter our BETA phase of our Enterprise Containerization as a Service to deploy applications in a consistent, repeatable, and reliable manner.

#bigwhale: An Unexpected Journey into Containerization @ Lockheed Martin - Pa...

Docker, Inc.

From development to production: Deploying Java and Scala apps to kubernetes

Olanga Ochieng'

Kubernetes Deployments: A "Hands-off" Approach

Rodrigo Reis

K8s debugging talk

ConSol Consulting & Solutions Software GmbH

There are 6Rs that can help you have Cloud-native workloads running in your Kubernetes deployments: Refactor, Replatform, Rehost, Retire, Retain or Repurchase. Rehosting virtual machines provides less friction than others, while still providing some advantages. One of those advantages being that you can have workloads you don't want to or cannot containerize yet sit alongside your containers through KubeVirt. In this meetup, we'll show you how Forklift 2.0 makes it easy to move them to their new home. And explain why this is a small step for your workloads but a giant leap on your path to the cloud. Presenters: Miguel Pérez Colino, Senior Principal Product Manager & Fabien Dupont, Manager, Software Engineering & Senior Principal Engineer. YouTube recording: https://youtu.be/-w4Afj5-0_g

Mass Migrate Virtual Machines to Kubevirt with Tool Forklift 2.0

Konveyor Community

Utilizing messaging systems grants us the capability to decouple services from each other: downtime of a service consuming messages does not impact the functionality of the service sending messages and vice-versa. In this talk we will discuss how to setup and use messaging systems. As practical examples, we use AMQP backed by ArtemisMQ, as well as kafka to send and receive messages, automatically as well as programmatically.

"Messaging with Quarkus"

ConSol Consulting & Solutions Software GmbH

Presentation recording: https://youtu.be/6wZhwAZWs84 There's no good way currently to make consistent, structural changes to Kubernetes manifests but still ensure YAML correctness and repeatability. OpenRewrite is somewhat unique in this space. OpenRewrite is an OSS project which can be leveraged to make large and significant changes to a Kubernetes cluster in a concise and repeatable way. Integrating Rewrite into workflows is easy, making adoption straightforward. Jon Brisbin, software engineer at OpenRewrite, will give a demonstration of this tool at this meetup. If you want to try the getting started or any of the tutorials: https://github.com/openrewrite

Cost Control and Rapid Innovation in Kubernetes with OpenRewrite

Konveyor Community

A central focus of Quarkus is developer joy and productivity. As part of this focus, Quarkus provides a powerful development mode which, among other things, allows true hot compilation of code. In this talk we will take a look at this feature in detail. Aside from allowing hot reloading locally, we can also configure our application to allow hot reloading remotely. For example, the application could be running in a docker container or on a kubernetes-/openshift-cluster. This improves Dev/Prod-parity and thus enhances the development lifecycle.

"Remote development of Quarkus applications"

ConSol Consulting & Solutions Software GmbH

From 2013-04-14 OpenShift Origin Community Day in Portland, Oregon Presenter: (William) Bill DeCoste (Principal Software Engineer for OpenShift) will introduce Red Hat's PaaS. He'll describe the several flavors of OpenShift ranging from the upstream community project Origin, the EC2-based online offering at http://openshift.redhat.com, to the corresponding commercial offering, OpenShift Enterprise. OpenShift Origin and Online will be demonstrated.

Introduction to OpenShift Origin- Private, Public and Community

OpenShift Origin

In the cloud-native workflow automation world, there are many great open-source projects such as Argo, Airflow, and Brigade. These tools require technical expertise to be used by business users, or they are aligned with a specific use case such as CI/CD. Boomerang Flow is an alternative tool that is usable by a business user but also provides an extensible framework developers can use for new use cases. It aims to replace Robotic Process Automation flows, which don’t necessarily need to be full-blown bots. And it doesn’t try to tackle use cases such as screen-scraping legacy green text applications. Instead, it tries to solve the problem where enterprises are using bots to perform tasks such as receiving a service now ticket and automatically moving it to a special status or workflow queue. These types of scenarios can be automated using Boomerang Flow. And the cost of running these workflows in Kubernetes is fractions of the cost of running an RPA bot for licensing. Presenters: Tyson W Lawrie, head of engineering - IBM, Marcus D Roy, Senior Lead Software engineer - IBM, and Benjamin Ruby, Analytics Consultant - IBM Recording link: https://youtu.be/-lOninwMoz4

Automate Workflows With The Open-source Cloud-native Tool Boomerang Flow

Konveyor Community

Operator development made easy with helm

ConSol Consulting & Solutions Software GmbH

Docker for HPC: how E4 is using it

E4 Computer Engineering SpA

Mais procurados (20)

Deploying containerized applications with Kubeapps

Automate The Creation/Transformation of Infrastructure as Code Artifacts with...

Thinking One Step Further with Time-saving DevOps Tools with Open Telekom Clo...

A mini adventure in Minikube devopsdays CT 2017

How to Modernize Virtualized Workloads

Helm at reddit: from local dev, staging, to production

Create a One Click Migration (OCM) process to Automate Repeatable Infrastruct...

Migrating Java JBoss EAP Applications to Kubernetes With S2I

#bigwhale: An Unexpected Journey into Containerization @ Lockheed Martin - Pa...

From development to production: Deploying Java and Scala apps to kubernetes

Kubernetes Deployments: A "Hands-off" Approach

K8s debugging talk

Mass Migrate Virtual Machines to Kubevirt with Tool Forklift 2.0

"Messaging with Quarkus"

Cost Control and Rapid Innovation in Kubernetes with OpenRewrite

"Remote development of Quarkus applications"

Introduction to OpenShift Origin- Private, Public and Community

Automate Workflows With The Open-source Cloud-native Tool Boomerang Flow

Operator development made easy with helm

Docker for HPC: how E4 is using it

Semelhante a CICD Pipelines for Microservices: Lessons from the Trenches

You have finally split your big monolith into microservices. Now what? How do you validate a more complex application? And how do you make it scale? Instead of having one CI/CD pipeline, you have multiple. And as the number of microservices increases so does the number of pipelines. Managing pipelines for microservice applications can quickly get out of hand, especially when you try to reuse common pipeline parts between different applications. In this webinar, we will see how you can create CI/CD pipelines designed specifically for microservices and how you can reuse the same pipeline across different applications.

CI/CD for Microservices Best Practices

DevOps.com

Microservices - How Microservices Have Changed and Why They Matter

Alexander Arda

The Case Against Microservices

aledsage

Cloud to Edge

Wesley Reisz

Dockers and kubernetes

Dr Ganesh Iyer

Intro Docker to Loire Atlantique

Julien Barbier

Enterprise level cloud CI

SQUADEX

When Cisco started envisioning the future of its application development platforms, the ability to create applications that are cloud-native with elastic services, network-aware application policies, and micro-services was strategic to the company. When the decision to build and operate a Cisco cloud service delivery platform for collaboration, video, and Internet of Things (IoT) application development was made, OpenStack and micro-services became central to our application architectures and strategic to our vision as a company. This presentation will look at the journey Cisco developers took to transform to an application-centric OpenStack platform for application development in a secure, network-centric, and completely open source manner. The importance of the platform being Red Hat Enterprise Linux OpenStack Platform and using OpenShift by Red Hat and the contribution to the community will be described. The micro-services architecture and service-oriented DevOps lessons learned for enabling massive scalable and continuous delivery of software will be presented and demoed.

Application Centric Microservices from Redhat Summit 2015

Ken Owens

This session is targeted towards teams and organizations considering to migrate their applications from monolithic to Microservice architecture by proposing Istio as an enabler. Istio is an implementation of service mesh, a technology useful for migrating to Microservices iteratively and safely. Migrating application architectures to Microservices is considered a key area of transformation in the IT world. Modernizing legacy applications to Kubernetes-based Microservices can prove to be very challenging if not planned correctly, taking into consideration the right technologies and enablers. This session explains how Istio can be used as a bridge and enabler for modernizing legacy monolithic applications to Microservices. Topics covered in the session will include: 1- Advantages of migrating to Microservices and service mesh . 2- Designing a Microservice application based on splitting an existing monolithic application. 3- Implementing Microservices iteratively as a strangler fig application with Istio. 4- Features Istio provides as a service mesh platform.

Istio as an enabler for migrating to microservices (edition 2022)

Ahmed Misbah

Tampere Docker meetup - Happy 5th Birthday Docker

Sakari Hoisko

Your developers just walked into your cube and said: "Here's the new app, I built it with Docker, and it's ready to go live." What do you do next? In this session, we'll talk about what containers are and what they are not. And we'll step through a series of considerations that need to be examined when deploying containerized workloads - VMs or Container? Bare Metal or Cloud? What about capacity planning? Security? Disaster Recovery? How do I even get started?

Docker?!?! But I'm a SysAdmin

Docker, Inc.

Executive Briefing: The Why, What, and Where of Containers

NVISIA

In celebration of Docker's 5th birthday in March, user groups all around the world hosted birthday events with an introduction to Docker presentation and hands-on-labs. We invited Docker users to recognize where they were on their Docker journey and the goal was to help them take the next step of their journey with the help of mentors. This presentation was done at the beginning of the events (this one is from the San Francisco event in HQ) and gives a run down of the birthday event series, Docker's momentum, a basic explanation of containers, the benefits of using the Docker platform, Docker + Kubernetes and more.

Docker Bday #5, SF Edition: Introduction to Docker

Docker, Inc.

Docker

Yansi Keim

Java Cloud Native Hack Nights GDL

Domingo Suarez Torres

Agile integration: Decomposing the monolith

Judy Breedlove

OpenShift Overview - Red Hat Open House 2017

Rodolfo Carvalho

Docker Birthday #5 Meetup Cluj - Presentation

Alex Vranceanu

Introduction to Microservices and Cloud Native Application Architecture

David Currie

CNCF Introduction - Feb 2018

Krishna-Kumar

Semelhante a CICD Pipelines for Microservices: Lessons from the Trenches (20)

CI/CD for Microservices Best Practices

Microservices - How Microservices Have Changed and Why They Matter

The Case Against Microservices

Cloud to Edge

Dockers and kubernetes

Intro Docker to Loire Atlantique

Enterprise level cloud CI

Application Centric Microservices from Redhat Summit 2015

Istio as an enabler for migrating to microservices (edition 2022)

Tampere Docker meetup - Happy 5th Birthday Docker

Docker?!?! But I'm a SysAdmin

Executive Briefing: The Why, What, and Where of Containers

Docker Bday #5, SF Edition: Introduction to Docker

Docker

Java Cloud Native Hack Nights GDL

Agile integration: Decomposing the monolith

OpenShift Overview - Red Hat Open House 2017

Docker Birthday #5 Meetup Cluj - Presentation

Introduction to Microservices and Cloud Native Application Architecture

CNCF Introduction - Feb 2018

Mais de Codefresh

In today’s fast-paced and dynamic development environment, minutes matter. How do you know if the changes that you’re making to your microservices are having the desired impact on your end-users? With Codefresh and Lightstep, you’re able to more quickly, more accurately, and more easily ensure that any commit and build results in a better experience for your end-users. Join us for this live workshop where you’ll learn: how to easily set up continuous delivery of your code with Codefresh, and how to measure the impact of a deployment in Lightstep (both free accounts!). You'll also learn how to easily instrument an application with OpenTelemetry in order to view metrics and traces of every request. In addition, you’ll see how easy it is to detect regressions and roll back to a healthy and stable state. You’ll be amazed at how easy it is!

Detect, debug, deploy with Codefresh and Lightstep

Codefresh

WATCH THE WEBINAR HERE:https://codefresh.io/events/simplify-code-helmfile/ You’ve got Helm deploying Charts via Codefresh but now how do you deal with Helm chart dependencies? There has to be a better solution than a Chart of Charts? How do I DRY out my Helm charts and simplify my Codefresh Pipelines for multiple environments? How do I create secrets in Codefresh? Is there another way than using Encrypted Secrets / Environment Variables? Helmfile has solutions to all these common problems and more. In this talk, we will go over some of the common problems you can solve with Helmfile and how you can use it to simplify your code and Codefresh Pipelines.

Simplify Your Code with Helmfile

Codefresh

Watch the webinar here: https://codefresh.io/events/5-simple-tips-troubleshooting-kubernetes-pods/ Once you start working with Kubernetes, and you deployed some applications to a cluster, you may have encountered issues where the application has not been deployed correctly or is not working as it should. In this webinar, we will explain the different statuses and error messages, and offer troubleshooting tips to resolve such situations swiftly!

5 Simple Tips for Troubleshooting Your Kubernetes Pods

Codefresh

For years people have been using VM-based CI platforms where they are managing build nodes that run their CI workflows. A few years ago, Codefresh revolutionized the CI/CD world and became the first container-native CI/CD platform. **WATCH THE WEBINAR AT https://Codefresh.io/events ** In this webinar, we will look at the differences between VM-based CI pipelines and Docker-based CI pipelines, in terms of maintenance, upgrades, pipeline creation, caching, and speed!

VM vs Docker-Based Pipelines

Codefresh

Watch the webinar at: https://codefresh.io/events/devsecops-deploy-secure-cloud-native-apps-fast/ Modern enterprises are constantly evolving their development practices by automating manual processes to deliver applications to market more quickly. This new “DevOps” model of application development enables organizations to more effectively serve their customers by bringing products to market faster. When it comes to security, however, this method comes with an added layer of complexity. In order to securely manage the velocity and scale of agile software delivery, DevOps teams must adopt a “security-first” mentality. This is done by shifting security considerations to the left and building security into their development pipeline. In this webinar, we’ll show you how you can: Build software quickly, using custom job orchestration See all of your builds, including failed builds and slow tests Identify vulnerabilities and malware in your images as you build your application

Deploy Secure Cloud-Native Apps Fast

Codefresh

Watch the video here: https://www.youtube.com/watch?v=z3u5cCV7glo CI/CD isn't some dark art, it's a critical component in increasing developer velocity! At Codefresh, we've been hard at work building a new UI, streamlining pipeline creation, and making it easier than ever to use Codefresh. Watch this webinar to see all of the new enhancements including live demos! See the full blog post here: https://codefresh.io/codefresh-news/p... New to Codefresh? Sign up for a FREE account today at Codefresh.io

Codefresh CICD New Features Launch! May 2019

Codefresh

**Watch the full webinar at https://codefresh.io/events/terraform-gitops-codefresh/ Today we write "Infrastructure as Code" and even "Pipelines as Code", so let's start treating our "code as code" and practice CI/CD with GitOps! In this talk, we'll show you how we build and deploy applications with Terraform using GitOps and Codefresh. Cloud Posse is a Terraform power user that has developed over 130 Terraform modules which are free and open source. We'll share how we handle automation with security while making the process easy for engineers.

Terraform GitOps on Codefresh

Codefresh

>>Read the blog post & watch the webinar here: https://codefresh.io/security-testing/adding-anchore-container-image-scanning-to-codefresh-pipeline/ In a few short years, containers have drastically changed the IT landscape and have helped drive a DevOps revolution. While containers bring a number of advantages, they also present a number of security challenges. These include the lack of support for containers by traditional IT tools and the lack of container experience in operations and security personnel. With container images becoming an ever more regular part of the development lifecycle, security should be woven into an automated pipeline with standardized best practices for testing. In this webinar, we will discuss how you can integrate tools like the open-source Anchore Engine into your Codefresh pipeline to automate image scanning and policy management & achieve a better security and compliance stance when building containers.

Adding Container Image Scanning to Your Codefresh Pipelines with Anchore

Codefresh

>> See the blog post and watch the webinar here: codefresh.io/clair-vulnerability-scans/ In this edition of Codefresh Live, see how the open-source project Clair can be used to scan your Docker images for security vulnerabilities in your CI/CD pipeline. Join Codefresh's Senior Solution Architect Dustin Van Buskirk for a LIVE tutorial including: - Installing Clair into your Kubernetes cluster using Codefresh Helm Repository/Chart Installation - Adding a Clair step to a Codefresh pipeline - Storing your Clair security results into a storage system - Enriching your Docker image’s metadata with Clair results - Enforcing vulnerability level thresholds in your pipelines Gain a 'clear' view of the security for your container-based infrastructure with our simple & easy plugin. Try Codefresh for FREE (120 builds/month!) at Codefresh.io/codefresh-signup

Image scanning using Clair

Codefresh

>>Watch the Codefresh Live webinar at Codefresh.io/events Creating and using custom Helm charts for internal consumption is incredibly powerful. In this edition of Codefresh Live we'll go through best practices for creating, testing, and deploying Helm Charts. We'll mix custom and community charts into our application and show best practices for promoting charts into ACR Chart Repositories. You'll be surprised just how effective Helm Charts can be in a streamlined DevOps workflow.

Updating Kubernetes With Helm Charts: Build, Test, Deploy with Codefresh and...

Codefresh

>>WATCH THE WEBINAR HERE: https://codefresh.io/docker-based-pipelines-with-codefresh/ Most people think that Docker adoption means deploying Docker images. In this webinar, we will see the alternative way of adopting Docker in a Continuous Integration Pipeline, by packaging all build tools inside Docker containers. This makes it very easy to use different tool versions on the same build and puts an end to version conflicts in build machines. We will use Codefresh as a CI/CD solution as it fully supports pipelines where each build step is running on its own container image. Sign up for FREE Codefresh account (120 builds/month) at Codefresh.io/codefresh-signup

Docker based-Pipelines with Codefresh

Codefresh

**Watch the full presentation here: https://codefresh.io/automated-serverless-pipelines-with-gitops-on-codefresh/ Dan Van Brunt introduces you to Serverless, talks about common misconceptions and challenges, and then demos how he uses the Serverless Framework effectively alongside containers. He shares some of the advanced pipelines he's developed so you can replicate his workflow without building a pipeline from scratch! Try Codefresh for FREE (120 builds/month) and get a free custom demo at Codefresh.io

Automated Serverless Pipelines with #GitOps on Codefresh

Codefresh

Watch the full webinar here: https://bit.ly/2NsNSVb 83% of organizations use vulnerable dependencies. By bringing Snyk into the CI/CD process we can scan Docker images as they are created and reveal vulnerabilities and fixes to developers as they work. Don’t wait for production to catch vulnerabilities! In this presentation, we use Codefresh to integrate with Snyk for a secure continuous delivery pipeline. Ready to try Codefresh? Sign up for FREE (120 builds/month) at Codefresh.io/codefresh-signup

Discovering and Fixing Dependency Vulnerabilities for Kubernetes apps with Sn...

Codefresh

WATCH THE WEBINAR HERE: https://codefresh.io/net-pipeline-windows-kubernetes/ For a long time, .Net applications have waited on the sidelines of the container revolution because of poor windows support. Not anymore! In this webinar we'll show a robust CI/CD workflow for building, testing and deploying .Net applications with Windows nodes. To help us out, we've invited Azure expert Jessica Deen. Get practical guidance on how to handle your .Net dev and release flows. Sign up for a FREE Codefresh account and get 120 builds/month at codefresh.io/codefresh-signup

Net Pipeline on Windows Kubernetes

Codefresh

**View the full webinar here: https://codefresh.io/multi-cloud-cicd-kubernetes-failover-across-clouds/ Multi-cloud Kubernetes is all about mitigating risk between hosting providers. In this webinar, we'll leverage Kubernetes as our universal cloud API, standup clusters in Google, Amazon, and Azure, setup multi-deploy so our application is in several locations, and demonstrate failover should one cloud fail. We'll stand up and manage our clusters, then use Istio, Helm, and Codefresh to do a multi-cloud Canary rollout to each cloud. Come ready to see: - Continuous Delivery to multiple Kubernetes providers - Cluster creation on multiple clouds from a single interface - How to create failover rules - A practical guide on how to set it up for yourself Get a free Codefresh account today (that's 120 build/month!) at https://codefresh.io/codefresh-signup/

Multi-cloud CI/CD with failover powered by K8s, Istio, Helm, and Codefresh

Codefresh

>>Watch the full presentation, demo, and live Q&A here: https://codefresh.io/saucelabs-lp The traditional "staging" model is getting in the way of adequately testing our applications. With the power of Kubernetes, Helm, Saucelabs, and Codefresh we can dramatically speed up and improve the quality of integration testing using short-lived environments. Join Kubernaut Dan Garfield as he shows you how to: - Create Automated short-lived environments for every commit/branch, - run integration tests in a scalable way, and - use Helm, Kubernetes, Codefresh, and Saucelabs integrations Sign up for a FREE Codefresh account today: https://codefresh.io/codefresh-signup/

Skip Staging! Test Docker, Helm, and Kubernetes Apps like a Pro

Codefresh

**WATCH THE WEBINAR HERE: https://codefresh.io/canary_helm_lp/ Sign up for a FREE Codefresh account today: https://codefresh.io/codefresh-signup/ Istio is unlocking some amazing ways to handle deployment networking for Kubernetes, but what about when you add in the wrinkle of Helm? In this webinar, Kubernaut Dan Garfield will show how to bring all these technologies together. Who knew Helm would pair so well with Istio?

Istio + Helm + Canary Webinar

Codefresh

**WATCH THE FULL WEBINAR HERE: https://codefresh.io/testim-lp/ Sign up for a FREE Codefresh account today: https://codefresh.io/codefresh-signup/ There are two big hurdles to solve when adding UI testing to your software delivery pipeline: 1) How to stand up an environment and 2) How to create tests that scale In this webinar, testing expert Oren Rubin will join Kubernaut Dan Garfield to present how machine learning, Kubernetes pipelines, and Testim can make test creation painless and easy to accomplish. We'll make continuous delivery a reality.

Selenium Testing your Kubernetes Apps with Machine Learning and Testim

Codefresh

**View the full webinar here: https://codefresh.io/cd-helm-chartmuseum-lp/ Sign up for a FREE Codefresh account today: https://codefresh.io/codefresh-signup/ In this webinar, Stef Arnold from SUSE CaaS Platform & Josh Dolitsky from Codefresh talked about streamlining the delivery of Kubernetes-based applications using the open-source tools Helm and ChartMuseum. They showed you how to use Helm to package your application as a chart, which is a deployable collection of Kubernetes files. Then, how to release your chart to ChartMuseum, which serves as an artifact repository for Helm charts.

Continuous Delivery for Kubernetes Apps with Helm and ChartMuseum

Codefresh

New Features Webinar-April

Codefresh

Mais de Codefresh (20)

Detect, debug, deploy with Codefresh and Lightstep

Simplify Your Code with Helmfile

5 Simple Tips for Troubleshooting Your Kubernetes Pods

VM vs Docker-Based Pipelines

Deploy Secure Cloud-Native Apps Fast

Codefresh CICD New Features Launch! May 2019

Terraform GitOps on Codefresh

Adding Container Image Scanning to Your Codefresh Pipelines with Anchore

Image scanning using Clair

Updating Kubernetes With Helm Charts: Build, Test, Deploy with Codefresh and...

Docker based-Pipelines with Codefresh

Automated Serverless Pipelines with #GitOps on Codefresh

Discovering and Fixing Dependency Vulnerabilities for Kubernetes apps with Sn...

Net Pipeline on Windows Kubernetes

Multi-cloud CI/CD with failover powered by K8s, Istio, Helm, and Codefresh

Skip Staging! Test Docker, Helm, and Kubernetes Apps like a Pro

Istio + Helm + Canary Webinar

Selenium Testing your Kubernetes Apps with Machine Learning and Testim

Continuous Delivery for Kubernetes Apps with Helm and ChartMuseum

New Features Webinar-April

Último

DBX First Quarter 2024 Investor Presentation

Dropbox

When you’re building (micro)services, you have lots of framework options. Spring Boot is no doubt a popular choice. But there’s more! Take Quarkus, a framework that’s considered the rising star for Kubernetes-native Java. It always depends on what's best for your situation, but how to choose the best solution if you're comparing 2 frameworks? Both Spring Boot and Quarkus have their positives and negatives. Let us compare the two by live coding a couple of common use cases in Spring Boot and Quarkus. After this talk, you’ll be ready to get started with Quarkus yourself, and know when to select Quarkus or Spring Boot.

Spring Boot vs Quarkus the ultimate battle - DevoxxUK

Jago de Vreede

Strategies for Landing an Oracle DBA Job as a Fresher

Remote DBA Services

Dubai, often portrayed as a shimmering oasis in the desert, faces its own set of challenges, including the occasional threat of flooding. Despite its reputation for opulence and modernity, the emirate is not immune to the forces of nature. In recent years, Dubai has experienced sporadic but significant floods, testing the resilience of its infrastructure and communities. Among the critical lifelines in this bustling metropolis is the Dubai International Airport, a bustling hub that connects the city to the world. This article explores the intersection of Dubai flood events and the resilience demonstrated by the Dubai International Airport in the face of such challenges.

Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...

Orbitshub

Manulife - Insurer Transformation Award 2024

The Digital Insurer

Architecting Cloud Native Applications

WSO2

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The value of a flexible API Management solution for O...

apidays

Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood

Juan lago vázquez

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

Keynote 2: APIs in 2030: The Risk of Technological Sleepwalk Paolo Malinverno, Growth Advisor - The Business of Technology Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...

apidays

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

Exploring Multimodal Embeddings with Milvus

Zilliz

Tracing the root cause of a performance issue requires a lot of patience, experience, and focus. It’s so hard that we sometimes attempt to guess by trying out tentative fixes, but that usually results in frustration, messy code, and a considerable waste of time and money. This talk explains how to correctly zoom in on a performance bottleneck using three levels of profiling: distributed tracing, metrics, and method profiling. After we learn to read the JVM profiler output as a flame graph, we explore a series of bottlenecks typical for backend systems, like connection/thread pool starvation, invisible aspects, blocking code, hot CPU methods, lock contention, and Virtual Thread pinning, and we learn to trace them even if they occur in library code you are not familiar with. Attend this talk and prepare for the performance issues that will eventually hit any successful system. About authorWith two decades of experience, Victor is a Java Champion working as a trainer for top companies in Europe. Five thousands developers in 120 companies attended his workshops, so he gets to debate every week the challenges that various projects struggle with. In return, Victor summarizes key points from these workshops in conference talks and online meetups for the European Software Crafters, the world’s largest developer community around architecture, refactoring, and testing. Discover how Victor can help you on victorrentea.ro : company training catalog, consultancy and YouTube playlists.

Finding Java's Hidden Performance Traps @ DevoxxUK 2024

Victor Rentea

Artificial Intelligence Chap.5 : Uncertainty

Khushali Kathiriya

Following the popularity of “Cloud Revolution: Exploring the New Wave of Serverless Spatial Data,” we’re thrilled to announce this much-anticipated encore webinar. In this sequel, we’ll dive deeper into the Cloud-Native realm by uncovering practical applications and FME support for these new formats, including COGs, COPC, FlatGeoBuf, GeoParquet, STAC, and ZARR. Building on the foundation laid by industry leaders Michelle Roby of Radiant Earth and Chris Holmes of Planet in the first webinar, this second part offers an in-depth look at the real-world application and behind-the-scenes dynamics of these cutting-edge formats. We will spotlight specific use-cases and workflows, showcasing their efficiency and relevance in practical scenarios. Discover the vast possibilities each format holds, highlighted through detailed discussions and demonstrations. Our expert speakers will dissect the key aspects and provide critical takeaways for effective use, ensuring attendees leave with a thorough understanding of how to apply these formats in their own projects. Elevate your understanding of how FME supports these cutting-edge technologies, enhancing your ability to manage, share, and analyze spatial data. Whether you’re building on knowledge from our initial session or are new to the serverless spatial data landscape, this webinar is your gateway to mastering cloud-native formats in your workflows.

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME

Safe Software

Three things you will take away from the session: • How to run an effective tenant-to-tenant migration • Best practices for before, during, and after migration • Tips for using migration as a springboard to prepare for Copilot in Microsoft 365 Main ideas: Migration Overview: The presentation covers the current reality of cross-tenant migrations, the triggers, phases, best practices, and benefits of a successful tenant migration Considerations: When considering a migration, it is important to consider the migration scope, performance, customization, flexibility, user-friendly interface, automation, monitoring, support, training, scalability, data integrity, data security, cost, and licensing structure Next Wave: The next wave of change includes the launch of Copilot, which requires businesses to be prepared for upcoming changes related to Copilot and the cloud, and to consolidate data and tighten governance ShareGate: ShareGate can help with pre-migration analysis, configurable migration tool, and automated, end-user driven collaborative governance

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff

sammart93

CICD Pipelines for Microservices: Lessons from the Trenches

1. CI/CD Pipelines for Microservices Best Practices DAN GARFIELD

2. Dan Garﬁeld Chief Technology Evangelist @todaywasawesome

3. Agenda 1. Why microservices 2. How Expedia approached microservice CI/CD 3. How we do it at Codefresh @todaywasawesome @codefresh

4. Why Microservices? @todaywasawesome @codefresh

5. @todaywasawesome @codefresh

6. Monolith 1 Monolith 2@todaywasawesome @codefresh

7. Why Microservices? @todaywasawesome @codefresh

8. Multiple services with single UI&UXMultiple monoliths with diff. UI&UX Expedia corporate travel re-architecture Uniﬁed cars UI Car sortCar search Cars (Egencia) Cars (ECT) Cars (Travelforce) Booking service Phone apps.

9. Moving to Microservices at Expedia Approach ● Consolidate code bases and ● Build shared libraries for global platform. Ex: ○ Logging service, monitoring service ● Rely on manual integration testing ● Standardize CI/CD pipelines ● Use Maven for modularity ● Migrate to cloud from on-prem Issues faced ● Geographically distributed ● Tools consolidation was hard ● Too many pipelines as microservices grew. ( 100 pipelines → 1000+ ) ● Pipelines not modular or re-usable ● Jenkins master-slave issues ● Copypasta causing bad patterns ● Central team could not keep up ● Plugin upgrade was a nightmare

10. Moving to Microservices at Expedia Lessons learned and Recommendations ● CI/CD templates should have been prioritized higher than “business needs” ● Bootstrapping new projects should have been externalized from the microservice and adding a new microservice should have full pipeline setup once a repo is created ● A modular pipeline approach would ease the pain caused by different versions ● Reusability in CI/CD platform is critical @todaywasawesome @codefresh

11. Single pipeline per project Can be complex/difﬁcult to be maintain Usually led by a single team (anti-devops) Organizing pipelines for monolithic applications

12. Scalability issues with microservice pipelines Does this look like a plan??

13. Shared libraries are not the solution.

14. Requires everyone to use same version of library Libraries often rely on each other in complex ways Changes have to go to admins Leads to big stability problems Relies on proprietary API Organizing pipelines for monolithic applications

15. How Codefresh does CI/CD for Microservices 1. Container-based pipelines 2. Shared pipelines 3. Deployment testing

16. Codefresh Architecture Diagram MULTI CLOUD

17. Each task is built into a Docker image. Users can self-serve these images. Images DO NOT rely on each other. Containers can be anything: go/node/c++ Container-based pipelines

18. Huge open source library at codefresh.io/steps

19. Maintain a single pipeline Make microservices uniform Change behavior based on context Use a single pipeline that operates with context

20. Triggers carry their context Codebase Tests Docker-compose Helm chart ... Dependencies ... Git checkout

21. Demo time! @todaywasawesome @codefresh

22. Why Canary? Usefulness of early testing Complexity of Infrastructure Testing early becomes less useful as infrastructure complexity rises https://codefresh.io/events/canary-deployment-helm-istio-codefresh/

23. Summary Shared pipelines > libraries Reusable Docker images > Copypasta Deployment validation with canary Read the blog post at https://codefresh.io/continuous-deployment/ ci-cd-pipelines-microservices/

24. Questions? Dan Garﬁeld @todaywasawesome Want to try it yourself? Open a FREE account today at Codefresh.io Kostis Kapelonis @codepipes

CICD Pipelines for Microservices: Lessons from the Trenches

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a CICD Pipelines for Microservices: Lessons from the Trenches

Semelhante a CICD Pipelines for Microservices: Lessons from the Trenches (20)

Mais de Codefresh

Mais de Codefresh (20)

Último

Último (20)

CICD Pipelines for Microservices: Lessons from the Trenches