GitOps: Git come unica fonte di verità per applicazioni e infrastruttura

1. GITOPS 25.06.2020 Git as a single source of truth for declarative infrastructure and applications

2. IMAGE GOES OVER HERE ANDREA PANISSON ● Cloud engineer & Kubernetes expert @ Sparkfabrik https://www.sparkfabrik.com/en/ ● CKAD Certiﬁed @andypanix

3. The session: 1 - Some fundamental concepts for newbies 2 - GitOps - What, Why and Beneﬁts 3 - GitOps in practice with GitHub Actions and ArgoCD

4. THE BASICS OF CICD - DEVOPS - KUBERNETES - DECLARATIVE MODEL

5. WHAT IS GIT? THE BASICS: GIT The most widely used modern version control system in the world today. More one GIT: https://www.atlassian.com/git/tutorials/what-is-git

6. CI: Continuous Integration ● Automates the build and unit test process of new code changes ● A software development practice where all developers merge code changes in a central repository (Git). ● Each change in code (commit) triggers an automated build-and-test stage for the given repo and provides feedback to the developer(s) who made the change. THE BASICS: CICD

7. CD: Continuous Deployments THE BASICS: CICD More one CONTINUOUS DEPLOYMENTS: https://en.wikipedia.org/wiki/Continuous_deployment A software engineering approach in which software functionalities are delivered frequently through automated deployments.

8. CD: Continuous Delivery THE BASICS: CICD More one CONTINUOUS DELIVERY: https://en.wikipedia.org/wiki/Continuous_delivery A software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time and, when releasing the software, doing so manually.

9. KUBERNETES THE BASICS: KUBERNETES Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative conﬁguration and automation. More one KUBERNETES: https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/

10. KUBERNETES Controllers THE BASICS: KUBERNETES “Controllers are control loops that watch the state of your cluster, then make or request changes where needed.” More one K8S CONTROLLERS: https://kubernetes.io/docs/concepts/architecture/controller/

11. The interesting part: if you want, you can write a new controller yourself. THE BASICS: KUBERNETES

12. KUBERNETES Operator THE BASICS: KUBERNETES An application-speciﬁc controller is called Operator. Operators allow automations (monitoring, alerts, actions, deployment) More one DEVOPS: https://azure.microsoft.com/en-us/overview/what-is-devops/

13. B E W A R E S P O I L E R S ! THE BASICS: KUBERNETES A Kubernetes Operator is a main actor in GitOps pull-based CICD pipeline

14. DevOps THE BASICS: DEVOPS Patrick Debois coined the term “DevOps” as part of the ﬁrst DevOpsDays conference in late 2009 More one DEVOPS: https://devopsdays.org

15. DevOps THE BASICS: DEVOPS He was trying to address issues with the “traditional” operations model. More one DEVOPS: https://devopsdays.org

16. DevOps model ● Interdisciplinary teams ● Better collaboration and communication between development and operations ● More frequent releases (improved product quality) ● Reduced time-to-market for new features THE BASICS: DEVOPS Traditional model: ● Different teams for Development, Quality, and Operations ● Strict boundaries between teams ● Lacks of interaction, collaboration and intercommunication between teams.

17. https://www.gkmit.co/blog/devops/devops https://opensenselabs.com/blog/articles/devops-drupal-development THE BASICS: DEVOPS

18. “DevOps isn’t about tools, but about how teams work together. DevOps is about people ﬁrst, then process, then tools.” THE BASICS: DEVOPS

19. CAMS: the pillars of DevOps DevOps is structured around four pillars of attention and focus. Those pillars are: ● CULTURE ● AUTOMATION ● METRICS ● SHARING (CAMS) as it's called for short. THE BASICS: DEVOPS

20. The model of declarative systems and conﬁguration DECLARATIVE MODEL One of the more prominent paradigms emerged from the DevOps movement.

21. Imperative model You describe a sequence of instructions to manipulate the state of the system to reach your desired state DECLARATIVE MODEL Declarative model You describe what you want to be achieved, as opposed to how to get there

22. Kubernetes APIs are declarative. That is why using GitOps with a declarative conﬁguration is such a good ﬁt for Kubernetes. KUBERNETES AND DECLARATIVE MODEL B E W A R E S P O I L E R S !

23. G I T O P S

24. GITOPS “What DevOps is to the Cloud, GitOps is to Cloud Native” https://www.weave.works/blog/gitops-is-cloud-native

25. The ﬁrst appearance of GitOps ● GitOps: Operations by Pull Request https://www.weave.works/blog/gitops-operations-by-pull-request ● The GitOps Pipeline https://www.weave.works/blog/the-gitops-pipeline ● Observability https://www.weave.works/blog/gitops-part-3-observability ● Application Delivery Compliance and Secure CICD https://www.weave.works/blog/gitops-compliance-and-secure-cicd GITOPS The term “GitOps” was coined/popularized in August 2017 by a series of blogs by Alexis Richardson, the co-founder, and CEO of Weaveworks

26. GITOPS TRENDS

27. GITOPS “GitOps is a paradigm, not a speciﬁc tool or technology.” https://www.weave.works/blog/gitops-is-cloud-native

28. What is GitOps? GITOPS More one GITOPS: https://www.gitops.tech/ Is a way of implementing Continuous Deployment / Delivery for cloud native applications

29. What is GitOps? GITOPS More one GITOPS: https://www.gitops.tech/ Is a way to manage the delivery and operations of applications that are deployed to Kubernetes

30. GITOPS https://youtu.be/_pdZnVzQIwc?t=522

31. GITOPS 1

32. GITOPS 1 2

33. GITOPS ● Git as the single source of truth of the system

34. GITOPS ● Git as the single source of truth of the system ● Git as the single place where we operate (create, change and destroy) all environments

35. GITOPS 1 2 3

36. GITOPS 1 2 3 4

37. GITOPS System is described declaratively Kubernetes and Terraform, given their declarative nature, are perfect tools to do GitOps. They allow to describe the entire system (infrastructure and applications) as conﬁguration code.

38. GITOPS Delivery automation and monitoring Kubernetes operators are the perfect tools for delivery automation and monitoring (diff and actions). Flux CD Argo CD Jenkins-x

39. GitOps follows the logic of DevOps GITOPS More one GITOPS: https://www.gitops.tech/ ● Config is code ● Code must be versioned controlled ● Config must be versioned controlled too ● What can be described can be automated ● Describe everything: code, config, monitoring & policy; and then keep it versioned

40. The role of the infrastructure and operations engineers in a GitOps model shifts from performing the infrastructure changes and application deployments to developing and maintaining the GitOps automation and helping teams review and approve changes using Git. GITOSP AND OPERATIONS

41. Let’s talk about CICD pipeline and GitOps

42. CICD PIPELINE AND GITOPS https://www.gitops.tech/#push-based-deployments The GitOps push-based model for CD

46. CICD PIPELINE AND GITOPS https://www.gitops.tech/#push-based-deployments The GitOps push-based model for CD *.tf

47. CICD PIPELINE AND GITOPS https://www.gitops.tech/#push-based-deployments The GitOps push-based model for CD terraform apply *.tf

48. CICD PIPELINE AND GITOPS https://www.gitops.tech/#push-based-deployments The GitOps push-based model for CD Jenkins CircleCI Travis CI GitLab GitHub Actions CICD

49. CICD PIPELINE AND GITOPS https://www.gitops.tech/#pull-based-deployments The GitOps pull-based model for CD

50. CICD PIPELINE AND GITOPS https://www.gitops.tech/#pull-based-deployments The GitOps pull-based model for CD

51. CICD PIPELINE AND GITOPS https://www.gitops.tech/#pull-based-deployments The GitOps pull-based model for CD Jenkins CircleCI Travis CI GitLab GitHub Actions

52. CICD PIPELINE AND GITOPS https://www.gitops.tech/#pull-based-deployments The GitOps pull-based model for CD Jenkins CircleCI Travis CI GitLab GitHub Actions

53. CICD PIPELINE AND GITOPS https://www.gitops.tech/#pull-based-deployments The GitOps pull-based model for CD Jenkins CircleCI Travis CI GitLab GitHub Actions CI CD

54. CICD PIPELINE AND GITOPS Push-based Deployments ● You need to provide the cluster credential to the deployment environment (potential security challenge) ● This kind of pipeline is probably inevitable when provisioning the infrastructure (Terraform) ● The deployment pipeline can’t automatically notice any deviation from the desired state ● We need to install a monitoring tool

55. Pull-based Deployments ● We have a new actor: the operator, living in the same cluster as the application to deploy ● CI do not need the credential of the k8s cluster ● The CD is in charge of the operator ● The operator can monitor the convergence status and take actions in case of divergence(CD) ● We must monitor the operator health (no operator no CD) CICD PIPELINE AND GITOPS

56. GitOps in practice: a GitOps CICD with GitHub Actions and ArgoCD

57. ● GitHub Actions allow to run a workﬂow on any GitHub event (push, issue creation or new release). ● From august 2019, GitHub Actions supports CI/CD, free for public repositories or up to 2000 minutes / month for private repos. ● It’s a great place to test and implements CICD even for learning purposes or of course for open source projects. GitHub Actions GITOPS IN PRACTICE https://github.com/features/actions

58. ● Creating an action is easy as adding YAML a ﬁle in the .github/workﬂows/ folder in your repo. ● You can start with an empty template or choose from a library of existing templates (for example to build a Docker image to deploy, run, or push to a registry). ● Hosted runners supports Linux, macOS, Windows, ARM, and containers GitHub Actions GITOPS IN PRACTICE https://github.com/features/actions

59. ● Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. ● Is an open source project donated to the community by Intuit a big american business and ﬁnancial software company ● Is a CNCF incubator project since april 2019 Argo CD GITOPS IN PRACTICE https://argoproj.github.io/projects/argo-cd

60. ● In november 2019 Argo teamed up with Weave Works and AWS to unify CNCF Flux, a Cloud Native Computing Foundation sandbox project, and Argo CD to announce Argo Flux, an open source GitOps continuous delivery tool that will bring the best of the two projects under one umbrella ● The ﬁrst product of the new team is GitOps Engine (https://github.com/argoproj/gitops-engine), a reusable library that implements core GitOps features. Argo CD GITOPS IN PRACTICE https://argoproj.github.io/projects/argo-cd

61. GITOPS IN PRACTICE GitHub Actions + Argo CD pull-based model for CD GitHub Actions CI CD

62. DEMO

63. RESOURCES WEB SITES https://www.gitops.tech/ https://gitops-community.github.io/kit/#what-is-gitops https://www.weave.works/blog/category/gitops/ https://www.weave.works/technologies/gitops-frequently-asked-questions/ https://www.weave.works/technologies/gitops/ SLIDES https://www.slideshare.net/weaveworks/continuous-lifecycle-london-2018-event-keynote-97418556 BOOKS: https://www.gitops.tech/ https://www.manning.com/books/gitops-and-kubernetes

64. THANK YOU!

65. GitOps And KUBERNETES GITOPS DEFINITION Kubernetes deployments have the following excellent properties: a. AUTOMATION: Kubernetes updates provide a mechanism for automating the process of applying a set of changes correctly and in a timely manner. b. CONVERGENCE: Kubernetes will keep trying to update until success. c. IDEMPOTENCE: multiple applications of convergence have the same outcome. d. DETERMINISM: assuming adequate resources, the updated cluster state depends only on the desired state. So what is so special about Kubernetes, and how and why did it lead to the rise of GitOps?

66. GitOps in practice GITOPS DEFINITION ● Describe the desired state of the whole system using a declarative specification. a. A git repo is the single source of truth b. All changes to the desired state are Git commits c. All specified properties of the cluster are observable in the cluster ● When the desired and observed states are not the same then: a. There is a convergence mechanism to bring the desired and observed states in sync both eventually, and autonomically. Within the cluster, this is Kubernetes. b. This is triggered immediately with a “change committed” alert. c. After a configurable interval, an alert “diff” may also be sent if the states are divergent. ● Hence all Git commits cause verifiable and idempotent updates in the cluster. a. Rollback is: “convergence to an earlier desired state”. ● Convergence is eventual and indicated by: a. No more “diff” alerts during a defined time interval. b. A “converged” alert (eg. webhook, Git writeback event).

67. https://twitter.com/vitorsilva/status/999978906903080961 from Luis Faceira https://twitter.com/luisfaceira The “1 slide” deﬁnition GITOPS

68. What is GitOps? It focuses on a developer-centric experience when operating infrastructure, by using tools developers are already familiar with, including Git and Continuous Deployment tools. The core idea of GitOps is having a Git repository that always contains declarative descriptions of the infrastructure currently desired in the production environment and an automated process to make the production environment match the described state in the repository. If you want to deploy a new application or update an existing one, you only need to update the repository - the automated process handles everything else. It’s like having cruise control for managing your applications in production. GITOPS More one GITOPS: https://www.gitops.tech/ “GitOps is a way of implementing Continuous Deployment for cloud native applications. “

69. What is GitOps? GITOPS ● GitOps is a paradigm, not a speciﬁc tool or technology. ● It’s a new operational model designed for modern systems. ● Is Cloud-Native operations and automation. More one GITOPS [17/10/2017]: https://www.weave.works/blog/gitops-part-3-observability

70. GitOps: Operations by pull requests GITOPS ● “90% best practices and 10% cool new stuff” ● “Git as the source of Truth... using the practices outlined in this post, our developers operate Kubernetes via Git.” ● provisioning of resources and deployment of k8s is declarative (Declarative tools love using Git as source of truth) ● entire system state is under version control and described in a single Git repository ● operational changes are made by pull request ● diff tools detect any divergence and notify us; and sync tools enable convergence (observability) ● rollback and audit logs are also provided via Git The author, Alexis the co-founder and CEO of Weaveworks, was describing their workﬂow, telling how their developers operate K8s via git. In this article he coined the term “GitOps”. More one GITOPS [07/08/2017]: https://www.weave.works/blog/gitops-operations-by-pull-request

71. GitOps: The GitOps Pipeline GITOPS More one GITOPS [30/08/2017]: https://www.weave.works/blog/the-gitops-pipeline Mainstream continuous delivery puts CI at the design centre of the pipeline and treats Git and CD as servant components. a. Git is a service, used by CI to obtain an input for the build. b. CD is an implementation extension powered by CI to promote build artifacts to production c. CI owns delivery orchestration. In the GitOps pipeline model, Git is the design centre. a. Git plays the central role of “source of truth for everything in the system” - code, conﬁg and the full stack. b. CI, build and test services are necessary for constructing deployable artefacts. c. the overall orchestration of delivery is coordinated by the deployment and release automation system - triggered by updates to repos. d. CD owns delivery orchestration, not CI.

72. “This is all about GitOps on Kubernetes” KUBERNETES

73. GITOPS And in the beginning there was...

74. GitOps: The GitOps Pipeline GITOPS More one GITOPS [30/08/2017]: https://www.weave.works/blog/the-gitops-pipeline The traditional pipeline model The GitOps pipeline model (the pull model)

75. GitOps: The GitOps Pipeline GITOPS More one GITOPS [30/08/2017]: https://www.weave.works/blog/the-gitops-pipeline The traditional pipeline model The GitOps pipeline model (the pull model)

76. GitOps: Observability GITOPS Fundamental Theorem of GitOps: “Only what can be described and observed can be automated and operated.” More one GITOPS [17/10/2017]: https://www.weave.works/blog/gitops-part-3-observability

77. GitOps: Observability GITOPS ● A system is Observable if developers can understand its current state from the outside ● Observability is a property of systems - like Availability and Scalability. ● Developers should care about making their applications observable so they can be in charge of monitoring their app’s behavior and impact on their app’s users. ● Monitoring, Tracing & Logging are techniques for baseline observations: measurements like error rate, request latency, queries/sec, i.e. symptoms of operational wellness. More one GITOPS [17/10/2017]: https://www.weave.works/blog/gitops-part-3-observability

78. GitOps: Observability GITOPS ● Git provides a source of truth for the desired state of the system, and observability provides a source of truth for the actual production state of the running system. ● Delivery of software should not be considered “done” until that software is Observable. So make monitoring and management part of your app dev process and not an afterthought. Developers should bake in application monitoring at the start of the design (Site Reliability Engineering). More one GITOPS [17/10/2017]: https://www.weave.works/blog/gitops-part-3-observability

79. GitOps: Observability GITOPS More one GITOPS [17/10/2017]: https://www.weave.works/blog/gitops-part-3-observability Since the entire system conﬁguration is stored on git (the source of truth), an operator can at any point in time compare the actual running state to the desired state held in source control, and verify that they are matching. Kubernetes orchestrator will apply changes to the cluster until its state has converged to the updated conﬁguration

80. GitOps: Application Delivery Compliance and Secure CICD GITOPS ● In GitOps, the CI system does not have direct access to the cluster ● Use pull based deployment - do not let CI push updates into the Kubernetes cluster or use kubectl by hand ● Record everything in Git to have audit and compliance More one GITOPS [17/10/2017]: https://www.weave.works/blog/gitops-part-3-observability

81. Go back to GitOps “deﬁnition”...

82. More GitOps definitions GITOPS DEFINITION https://www.slideshare.net/weaveworks/continuous-lifecycle-london-2018-event-keynote-97418556/9 ● GitOps is: a. An operating model for Kubernetes and other cloud native technologies. It provides a set of best practices that unify deployment, management and monitoring for containerized clusters and applications. b. A path towards a developer centric experience for managing applications, where end-to-end CICD pipelines and git workflows are applied to both operations, and development. ● GitOps is a fast, and secure method for developers and cluster operators to maintain and update complex applications running in Kubernetes ● GitOps is developer workflows for operations ● GitOps is automation for Cloud Native

83. CICD PIPELINE AND GITOPS “Kubernetes anti-patterns: Let's do GitOps, not CIOps!” https://www.weave.works/blog/kubernetes-anti-patterns-let-s-do-gitops-not-ciops

84. CICD PIPELINE AND GITOPS Kubernetes anti-patterns [17/07/2018]: https://www.weave.works/blog/kubernetes-anti-patterns-let-s-do-gitops-not-ciops The CIOps: the push model for CD The Kubernetes antipattern

85. CIOps: The push model for CICD GITOPS Kubernetes anti-patterns [17/07/2018]: https://www.weave.works/blog/kubernetes-anti-patterns-let-s-do-gitops-not-ciops The anti-pattern is when the CI system runs build and tests, followed by a deployment directly to Kubernetes. a. insecure by design: both developer and the CI system have full-access to the cluster as well as the container registry – there are no clear boundaries and this poses potential security risks. b. the CI system is a pretty interesting attack target. c. each CI job must be conﬁgure to deploy to the cluster (inefficient) d. The CI and the Kubernetes cluster both compete to be the source of truth A Kubernetes anti-pattern

86. Should I abandon my CI tool? NO. But you shouldn’t use CI servers to do continuous delivery CICD PIPELINE AND GITOPS

87. CICD PIPELINE AND GITOPS Kubernetes anti-patterns [17/07/2018]: https://www.weave.works/blog/kubernetes-anti-patterns-let-s-do-gitops-not-ciops The GitOps CICD - The pull model for CD The Kubernetes way

88. The GitOps CICD The pull model for CD (the Kubernetes way) GITOPS ● In GitOps, the CI system does not have direct access to the cluster ● Use pull based deployment - do not let CI push updates into the Kubernetes cluster or use kubectl by hand ● Record everything in Git to have audit and compliance More one GITOPS [17/10/2017]: https://www.weave.works/blog/gitops-part-3-observability

89. GITOPS

90. KUBERNETES ● An open-source platform that orchestrates and automates container operations ● Eliminates many of the manual processes involved in deploying and scaling containerized applications ● Helps to automate the deployment, scaling, and management of applications using containers THE BASICS: KUBERNETES More one KUBERNETES: https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/ Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative conﬁguration and automation.

91. KUBERNETES Controller architecture Through listening to the API server events related to the resource type being managed, the controller continuously watches for changes to the resource’s conﬁguration and performs the necessary work to move the current state towards the desired state. THE BASICS: KUBERNETES More one K8S CONTROLLERS: https://kubernetes.io/docs/concepts/architecture/controller/ “Controllers are control (inﬁnite) loops that watch the state of your cluster, then make or request changes where needed.”

92. KUBERNETES Controller architecture THE BASICS: KUBERNETES More one K8S CONTROLLERS: https://kubernetes.io/docs/concepts/architecture/controller/ Kubernetes comes with a set of built-in controllers that run inside the kube-controller-manager. These built-in controllers running in the Controller Manager provide important core behaviors. ● Replication Manager (a controller for ReplicationController resources) ● ReplicaSet, DaemonSet, and Job controllers ● Deployment controller ● StatefulSet controller ● Node controller ● Service controller ● Endpoints controller ● Namespace controller ● PersistentVolume controller ● Others

93. THE BASICS: KUBERNETES Introducing Replication Controllers in Kubernetes: https://freecontent.manning.com/kubernetes-in-action-introducing-replication-controllers/ Replication controller’s reconciliation loop. A replication controller, in essence, constantly monitors the list of running pods and makes sure the actual number of pods of some type always matches the desired number: KUBERNETES Controller architecture

94. KUBERNETES Operator People who run workloads on Kubernetes often like to use automation to take care of repeatable tasks. The Operator pattern captures how you can write code to automate a task beyond what Kubernetes itself provides. THE BASICS: KUBERNETES More one DEVOPS: https://azure.microsoft.com/en-us/overview/what-is-devops/ Operators are software extensions of the Kubernetes API that act as controllers for a Custom Resource. Operators are application-speciﬁc controllers. ● All operators use the controller pattern, but not all controllers are operators. ● Operators follow Kubernetes principles, notably the control loop

95. The model of declarative systems and configuration THE BASICS: DEVOPS ● One of the more prominent paradigms emerged from the DevOps movement. ● With declarative models, you describe what you want to be achieved, as opposed to how to get there. ● This is in contrast to an imperative model, in which you describe a sequence of instructions to manipulate the state of the system to reach your desired state. ● Declarative systems are idempotent; imperative systems are not (idempotency is a property of an operation, whereby the operation can be performed any number of times and still produce the same result) Kubernetes APIs are declarative. That is why using GitOps with a declarative configuration is such a good fit for Kubernetes.

96. DevOps He was trying to address issues with the “traditional” operations model where the organization is split between functional boundaries, with different teams for Development, Quality, and Operations and where these teams lacks of interaction, collaboration and intercommunication. THE BASICS: DEVOPS More one DEVOPS: https://devopsdays.org Patrick Debois coined the term “DevOps” as part of the ﬁrst DevOpsDays conference in late 2009

97. DevOps In the DevOps model, teams are divided between products or components and are interdisciplinary, containing team members with skill sets across all functions. THE BASICS: DEVOPS More one DEVOPS: https://devopsdays.org DevOps is, at ﬁrst, a shift in the Organizational model.

98. DevOps: A deﬁnition DevOps puts a heavy emphasis on shared responsibilities across all teams throughout the software development lifecycle. DevOps isn’t about tools, but about how teams work together. DevOps is about people ﬁrst, then process, then tools. THE BASICS: DEVOPS More one DEVOPS: https://azure.microsoft.com/en-us/overview/what-is-devops/ “DevOps is the union of people, process, and technology to continually provide value to customers.”

99. THE BASICS: DEVOPS

100. DevOps: The beneﬁts ● Better collaboration between development and operations ● Improved product quality ● More frequent releases ● Reduced time-to-market for new features ● Decreasing costs of design, development, and operations THE BASICS: DEVOPS

GitOps: Git come unica fonte di verità per applicazioni e infrastruttura

Esté a ler uma amostra.

Confira estes a seguir

GitOps: Git come unica fonte di verità per applicazioni e infrastruttura

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a GitOps: Git come unica fonte di verità per applicazioni e infrastruttura (20)

Mais de sparkfabrik (20)

Mais recentes (20)