Agile development and operation of complex systems in multitechnology and multicompany environments: Following a DevOps approach
•Transferir como PPTX, PDF•
1 gostou•148 visualizações
Big innovation and research projects usually require merging contributions form organizations with expertise in different domains. Managing and participating in multi-company projects that use multiple state-of-the-art technologies constitute a challenging activity due to many factors such as integration inexperience, evolv-ing components, tentative requirements, independent teams or independent man-agement centers. In the late nineties and in the early years of 2000, several meth-odologies arose with the focus on fast releases of working software, commonly known as agile, that aimed to address many of the challenges that this kind of projects face. However, in most cases, these methodologies were not fully adoptable as the automation investment was too high and it was not recoverable during the duration of the project. The global servitization trend and the appear-ance of approaches, such as DevOps, to support the continuous and fast adjust-ment of those services to stay in business has also impacted innovation and re-search projects. On one hand, matured technologies that reduce the automation investment have arisen. On the other hand, whenever it makes sense, services which benefit from the application of DevOps approaches are required to be im-plemented. This paper explains the implementation of DevOps approaches to support the agile development in the context of innovation and research projects. It also describes two practical implementation cases where such approaches were implemented and how they evolved in the course of the time. This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 653704 (OPERANDO), 726755 (CITADEL), and 731533 (DECIDE)
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (7)
Semelhante a Agile development and operation of complex systems in multitechnology and multicompany environments: Following a DevOps approach
Semelhante a Agile development and operation of complex systems in multitechnology and multicompany environments: Following a DevOps approach (20)
Mais de Citadelh2020
Mais de Citadelh2020 (20)
Último
Último (20)
Agile development and operation of complex systems in multitechnology and multicompany environments: Following a DevOps approach
- 1. WE CAN DO SO MUCH TOGETHER WE CAN DO SO MUCH TOGETHER Gorka Benguria, Juncal Alonso, Iñaki Etxaniz, Leire Orue- Echevarria, Marisa Escalante ICT Division – IT Competitiveness Agile development and operation of complex systems in multitechnology and multicompany environments - following a DevOps approach EuroAsiaSPI 2018 - Zamudio, 07.09.2018
- 2. Agenda 2 ▌ Challenges and motivation What is DevOps? DevOps Framework Approach Practical implementation in multi-technology and multi-company Future work and Extended DevOps concept
- 3. Agenda 3 ▌ Challenges and motivation What is DevOps? DevOps Framework Approach Practical implementation in multi-technology and multi-company Future work and Extended DevOps concept
- 4. Challenges and motivation 4 ▌ Uncertain requirements in R&D projects Non-collocated teams Frequent releases Different development methodologies Repeatable after the Project ends
- 5. Challenges and motivation • Agile methodologies have been in the game for long now … • But a current has risen up in the last years: DevOps 5 ▌
- 6. Agenda 6 ▌ Current challenges and motivation What is DevOps? DevOps Framework Approach Practical implementation in multi-technology and multi-company Future work and Extended DevOps concept
- 7. What is DevOps? A philosophy that aims to improve the interaction between the of development and operations departments while supporting the business objectives 7▌ There are many definitions but one could be:
- 8. What is DevOps? Other definitions of DevOps DevOps is a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality. [Bass, Len; Weber, Ingo; Zhu, Liming. DevOps: A Software Architect's Perspective. ISBN 978-0134049847.] DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology — especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective. [gartner, https://www.gartner.com/it- glossary/devops] 8▌
- 9. What is DevOps? 9 ▌
- 10. What is DevOps? • DevQAOps • SecDevOps • DevSecOps • BizDevOps • … 10▌
- 11. What is DevOps? • DevOps emerge to solve several needs: – Transition from product to service – Demand of QoS – Development environments are becoming more and more complex – Multi-disciplinarity – Business Agility – Higher quality 11▌
- 12. DevOps: a trend https://www.delphix.com/resources/ana lyst-report/gleanster-state-of-devops 12 ▌
- 14. Agenda 14 ▌ Challenges and motivation What is DevOps? DevOps Framework Approach Practical implementation in multi-technology and multi-company Future work and Extended DevOps concept
- 15. DevOps Framework approach • DevOps framework in our context: set of tools that facilitate the transition and interactions between the application development and its exploitation • Result of the practical implementation of DevOps in three research projects: 15 ▌
- 16. • Online Privacy Enforcement, Rights Assurance and Optimization project, the subject of this proposal • Duration: 10.2016 – 09.2019 • Total cost: 4.4 M€ • Programme: H2020-DS-2014-1 Project No.:653704 (IA) • Topics: DevOps, Privacy • Partners: Occ, Arteevo, Pdi, Stelar, Romsoft, Tecnalia, Iti, Univ. Of Piraeus, Ospedale San Rafaele. http://www.operando.eu/ @OperandoH2020 16▌
- 17. • Empowering Citizens to Transform European public administrations • Duration: 10.2016 – 09.2019 • Total cost: 3.6 M€ • Programme: H2020-SC6-CULT-COOP-2016-1 Project No.:726755 (RIA) • Topics: DevOps • Partners: Imec, Leuven Univ, Antwerp City, Timelex, Ictu, Latvian Univ, Varam, Ip, Regione Puglia, Fincons, Cantabria Univ, Tecnalia. http://www.citadel-h2020.eu/ @Citadelh2020 17▌
- 18. • DevOps for trusted, portable and interoperable multi- Cloud applications towards the Digital single market. • Duration: 12.2016 – 11.2019 • Total cost: 3.6 M€ • Programme: H2020-ICT-2016-1 Project No.:731533 (RIA) • Topics: Multi-cloud/DevOps/Brokers • Partners: Aimes, Arsys, Cloudbroker, Fraunhofer, Innovati, Hewlett Packard Time.Lex, Tecnalia http://www.decide-h2020.eu/ @Decideh2020 18▌
- 19. DevOps Framework approach Web Frontend Backend Persistency External Service Development Operations SaaS 19▌
- 20. DevOps Framework approach Development Operations SaaS Web Frontend Android Client Backend Android Extra Backend Persistency Static Content Browser External Service 20▌
- 21. DevOps Framework approach Development Operations Web Frontend Android Client Backend Android Extra Backend Persistency Static Content Browser External Service 21▌
- 22. DevOps Framework approach: open to multiple technologies 22▌ Initially …
- 23. 23▌ DevOps Framework approach: open to multiple technologies
- 24. Establish Versión Control Identify Environments Provision Environments Development Environment Integration Environment Piloting Environment Establish Knowledge Base Establish Monitoring and Feedback Configure Platforms Configure Transitions DevOps framework approach 24 ▌
- 25. Agenda 25 ▌ Challenges and motivation What is DevOps? DevOps Framework Approach Practical implementation in multi-technology and multi-company Future work and Extended DevOps concept
- 26. DevOps framework approach: knowledge management • Description of the deployment options and how they work 26 ▌ +
- 27. DevOps framework approach: Environments Development Operation 27▌
- 28. DevOps framework approach: Environments 28 ▌
- 29. DevOps framework approach: Environments 29 ▌
- 30. Pull source code Package code (compile) DevOps framework approach: Environments Start / Set up Test Monitor Development Operation 30▌
- 31. Pull source code Package code (compile) DevOps framework approach: Environments Start / Set up Test Monitor 24h Development Operation + 31▌
- 32. DevOps Framework approach: microservices 32▌
- 33. DevOps framework approach: Environments – Continuous integration • Build the system in a continuous and automatically manner: – Get dependencies – Compile – Publish – Start – Test – Report 33 ▌
- 34. DevOps framework approach: Environments – Continuous integration configuration 34▌
- 35. DevOps framework approach: Environments – Continuous integration configuration 35▌
- 36. DevOps framework approach: continuous deployment 36 ▌ • Containers-based technology • A Container is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings. https://www.docker.com/resources/what-container
- 37. DevOps framework approach: continuous deployment 37 ▌ • workflow Docker ImageDocker File https://hub.docker.com Docker Container Users
- 38. DevOps framework approach: monitoring • Monitor the environment 39 ▌
- 39. DevOps framework approach: monitoring 40 ▌
- 40. Agenda 41 ▌ Challenges and motivation What is DevOps? DevOps Framework Approach Practical implementation in multi-technology and multi-company Future work and Extended DevOps concept
- 41. Future work: Extended DevOps concept 42 ▌ Continuous architecting Continuous development Continuous pre-deployment Continuous adaptation Continuous delivery
- 42. 43 ▌ This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements No 653704, 726755, and 731533
- 43. Visita nuestro blog: http://blogs.tecnalia.com/inspiring-blog/ www.tecnalia.com Leire Orue-Echevarria Arrieta, PhD, PMP División ICT / ICT Division IT Competitiveness Leire.Orue-Echevarria@tecnalia.com C/ Geldo. Parque Tecnológico de Bizkaia, Edificio 700 E-48160 Derio - Bizkaia (Spain) Tel: 902 760 000 *. Tel: +34 946 430 850 (International Calls). Mob: +34 664 103 005
Notas do Editor
- Barrera: esto lo vemos muy claramente en tecnalia, los desarrolladores somos nosotros, pero los servidores y las comunicaciones están gestionadas por sistemas. Entre nosotros existe un sistema de incidencias que introduce unas esperas no siempre asumibles frente a nuestros clientes Complejos: ahora todo tiene que tener parte web: ya no solo tenemos que dominar el algoritmo, tenemos que dominar su presentación, usabilidad, etc. Además, dependemos de entornos que evolucionan independientemente de nuestro control y pueden hacer que nuestros desarrollos dejen de ser validos en un tiempo mas o menos largo. Pasos: Al tener cosas en un servidor al contrario que una aplicación de escritorio, para poner algo en producción tenemos que compilar, preparar un paquete instalable en servidor, ponerlo en el servidor, hacer un snapshot o backup del estado del servidor, instalarlo, probarlo, desechar el snapshot o backup si todo va bien, etc. Negocio: todo lo que hemos visto antes
- Esto no es algo que estemos nosotros haciendo porque somos un poco raros, Es una tendencia global. Como podemos ver en diferentes informes, mas o menos interesados. 2015 Annual State of DevOps de delphix y gleanster 2,381 profesionales de Norteamérica y europa
- 2017-State of DevOps report, de Puppetlabs y de Dora -> 3200 personas 57% de norteamerica
- Desde el punto de vista del desarrollador la vista es algo diferente, lo que le preocupa principalmente la parte LOGICA: los componentes y sus interfaces. No saben donde se ejecutaran No les preocupa su infraestructura de momento No saben si estarán replicados o no Esta es la vista de “Desarrollador”
- Vamos a añadir algunas partes mas para aproximarnos “un poco” (digo un poco porque el numero de componentes en proyectos como OPERANDO supera los 25, aquí tenemos 8) a los escenarios que estamos manejando en los proyectos en los que estamos aplicando este enfoque. El browser: Al desarrollador le preocupara el navegador por motivos de compatibilidad Ademas añadiremos Un componente a cargo de contenido estático de las paginas Un cliente android Un backend para las funcionalidades especificas de android
- Cada uno de estos desarrolladores influenciara mas o menos en las decisiones tecnológicas, Pueden estar impuestos por el contrato, en el caso de Proyectos bajo contrato En los casos de proyectos de colaboración lo normal es que cada compañía desarrolle en aquellos lenguajes que le son familiares y cómodos. Así pues nos podríamos encontrar en una situación como la siguiente: El browser podria ser chrome que es el mas extendido actualmente El web frontend podria estar desarrollado en .NET, y tener los elementos estaticos en servidores de objetos El backend en java (jaxrs) y desplegado sobre tomcat El cliente android podria ser una app hybrida desarrollada con cordoba y su back-end podria estar desarrollada en node js La capa de persistencia podria estar desarrollada en mysql o maria db
- Empezamos con: Python Flask (el pimiento) Java Jaxrs (el pescado) Tomcat Swagger (los corchetes) Sping-boot JSF .net c# mono PHP Mongo DB Mysql HTML5/CSS3/JS Node js (el hexagono con el simbolo de encender) Hadoop
- Python Flask (el pimiento) Java Jaxrs (el pescado) Tomcat Swagger (los corchetes) Sping-boot (el hexagono con el simbolo de encender) JSF .net c# mono PHP Mongo DB Mysql HTML5/CSS3/JS Node js Hadoop
- El enfoque se resumen en: Acelerar la transición entre el desarrollo y la operacion Establecer mecanismos de comunicación Mirar la pela
- Vimos esto en la parte de entorno
- Que herramientas usamos Para recoger el código del desarrollador usamos git estandar, que esta soportado tanto por GitLab, como Github, como BitBucket Para gestionar todo el ciclo de recoger codigo hasta empaquetar usamos Jenkins y maven Tanto jenkins como maven requieren de configuración explicaremos como lo configuramos, esa configuración la almacenamos en Git Los componentes los empaquetamos en contenedores docker, los arrancamos como docker y las pruebas igualmente se empaquetan y se ejecutan como contenedores docker La monitorización en operación la tenemos planteada con nagios, aunque esto esta algo verde Finalmente para el feedback: Para problemas de empaquetado o pruebas de integracion usamos los correos que envía en jenkins para la monitorización todavía no lo tenemos configurado Para pruebas manuales en operación no tenemos nada establecido, en Operando usan jira Y si también tenemos una excel, para organizar la informacion de cada componente, algún día puede que sea un portal y automaticemos mas cosas … Y también tenemos un wordpress para almacenar los procedimientos de operación de la plataforma de integración
- De los microservicios lo que nos interesa sobre todo es que los componentes se comuniquen via servicios, nada de ficheros o bases de datos compartidas. La comunicación en base a fichero o bases de datos compartidas complica mucho el despliegue posterior en produccion. Ademas, de estas los microservicios tienen otras muchas caracteristicas según el sitio donde busquemos, a nosotros la que nos importa es la que hemos comentado.