O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

DevOps and Build Automation

Carregando em…3

Confira estes a seguir

1 de 51 Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a DevOps and Build Automation (20)


Mais recentes (20)

DevOps and Build Automation

  1. 1. Introduction to DevOps & Build Automation Update WAYI 27 February 2017
  2. 2. Introduction to DevOps
  3. 3. How can we release on-time? How can we have a stable release? How can we address the problem before end users get impacted? Top Concerns
  4. 4. Embrace it.
  5. 5. DevOps = Dev + Ops Bring Dev & Ops closer together - Ideally, unified Culture and Mindset Set of practices centered on automation - Reliability - Repeatability - Traceability - Minimal or no manual intervention
  6. 6. What is happening in software delivery industry right now?
  7. 7. DevOps adoption increased from 66 percent in 2015 to 74 percent in 2016.
  8. 8. DevOps adoption is strongest in the enterprise (81 percent of enterprises adopting DevOps compared to 70 percent in SMBs).
  9. 9. Enterprises are adopting DevOps from the bottom up: Adoption is by projects or teams (29 percent) and business units or divisions (31 percent) vs. company-wide (21 percent).
  10. 10. Based on a 2016 Puppet Labs study Highly effective DevOps organizations have the following impact on the business. Source: 2016 State of DevOps Report, Puppet Labs 200X Frequent Deployments 24X Return to Normal Operations 3X Change Failure 22X Unplanned Work and Rework 50% Security Remediation Time High performing teams offer secure and reliable products, services and capabilities quicker, faster.
  11. 11. 2017? This year, DevOps adoption is expected to increase up to 85%.
  12. 12. Automation is a part of DevOps patterns “Infrastructure as Code”
  13. 13. Standardize development, test and production environments. Effectively deploy and manage cloud resources. Eliminate error-prone, time-consuming manual tasks. Improve cooperation between development and operations. Implement automated release pipelines.
  14. 14. Benefits of automation Automation enables velocity, scale, consistency and feedback. All of these qualities are of a piece. Any one depends on the other three – for example;-  you can’t scale unless you are able to quickly add servers with consistent configurations.  you can’t get feedback automatically without being able to support different real-time testing and monitoring techniques.  you can’t respond to feedback effectively unless you have a high-velocity way to deliver incremental changes safely.
  15. 15. Why Should I Automate?
  16. 16. Repeatability Scripts can be repeated and unless your computer is having a particularly bad day, you can be reasonably certain that the same instructions will be executed in the same order each time the same script is run.
  17. 17. Reliability Scripts reduce changes for human error.
  18. 18. Efficiency Automated tasks will often be faster than the same task performed manually.
  19. 19. Testing Scripted processes undergo testing throughout the development cycle, in much the same way the system code does. This greatly improves changes for successful process execution as the project progresses. Automated scripts eventually represent a mature, proven set of repeatable processes.
  20. 20. Versioning Scripts are artifacts that can be placed under version control. With manual processes, the only artifacts that can be versioned and tracked are procedure documents. Versioning of human beings – the other factor in the manual process equation – is unfortunately not supported by typical source control system.
  21. 21. Leverage Another big benefit to automating is that developers and testers can focus on the areas where the add real value to a project – developing and testing new code and features – instead of worrying about the underlying development infrastructure issues.
  22. 22. What Should I Automate?
  23. 23.  Build and deployment of the system under design.  Unit test execution and report generation.  Code coverage report generation  Functional test execution and report generation.  Load test execution and report generation.  Code quality metrics report generation.  Coding conventions report generation.
  24. 24. Importance of Automated Testing
  25. 25. With automated testing, we are able to catch problem earlier, before they impact our release cycle. The earlier we catch a problem, the easier and less expensive it is to fix, and this is why testing is such an important part of DevOps practice.
  26. 26. Architecture design Component design Coding Unit testing Integration testing Acceptance testing The V-diagram, common in software engineering, illustrate this: As you can see, each kind of testing activity (on the right side of the V) checks a particular phase of development (shown on the left side). The cost of rework rises as defects are discovered later in the project. It’s better to begin testing at the vertex of the V, with unit tests. Catching a defect during a unit test is much easier than trying to fix it when it’s being tested with other components that might make it difficult to discover where the actual problem lies.
  27. 27. DevOps Toolchains
  28. 28. DevOps Toolchain
  29. 29. 1. Code – Code development and review, version control tools, code merging 2. Build – Continuous integration tool, build status 3. Test – Test and results determine performance 4. Package – Artifact repository, application pre-deployment staging 5. Release – Change management, release approvals, release automation 6. Configure – Infrastructure configuration and management, Infrastructure as Code tools 7. Monitor – Application performance monitoring, end-user experience
  30. 30. Examples of DevOps Tools Continuous Integration (CI) - Jenkins, Travis, TeamCity Configuration Management (CM) - Puppet, Chef, Ansible, CFEngine Continuous Inspection - SonarCube, HP Fortify, Coverity Containerization - Vagrant, Docker Virtualization - Amazon EC2, VMWare, Microsoft Hyper-V
  31. 31. Build Automation Update
  32. 32. Most modern development teams have set up some form of build automation. At minimum, build automation involves check out from source control, compilation and linking, and packaging the resulting binaries and other necessary files. These packages can be then deployed and tested in downstream processes. If done correctly, build automation reduces manual labor by developers, ensures builds are consistent and complete, and improves product quality.
  33. 33. Critical Issues with Build Automation 1. Long Builds 2. Large Volume of Builds 3. Complex Builds
  34. 34. 1. The Problem: Long Builds Why it matters?  Developer wait time Most developers spend between 2-10 hours per week waiting for builds to complete – lost time  Context switch developers switch task, lose context related to the problem they were working on – reduces productivity  Build bottleneck Developers sync changes into the main code line and frequent builds are needed  Product quality The less frequent the build, the less issues can be fixed and verified befre the release – hurting product quality
  35. 35. 2. The Problem: Large Volume of Builds Why it matters?  Limited build resources Limited access to build servers during specific time windows or the servers are often overloaded and builds will take much longer – hurting productivity (especially practicing Agile)
  36. 36. 3. The Problem: Complex Builds Why it matters?  Complex builds are brittle Interactions between many different components often lead to manual error, broken builds and worse  Extensive manual efforts Executing a complex build and delivering its results requires a substantial manual effort  Incremental builds are difficult An incremental run could break the build due to partially specified dependencies  Legacy components and fear of change Complex builds tend to have legacy components written years ago by staff members who are no longer at the organization  Complex builds are long As the complexity of the build increases, so the time it takes to run
  37. 37. What Can Be Done?
  38. 38. Solution: Improve build speed: 1. Upgrade our build server 2. Running builds in parallel on one machine 3. Build avoidance / incremental builds – running only those parts of the build that have changed 4. Using a RAM disk – moving the entire build operation to RAM to eliminate disk I/O 5. Precompiled headers – precompiling shared dependencies that changes less frequently Solution: Address long and complex build: 1. Distributed builds – parallelizing build across a cluster of build servers 2. Manually partitioning Makefiles – breaking the build into smaller components 3. Optimizing Makefiles – rewriting Makefiles to make them run more efficiently 4. Unity builds in C/C++ - combining source files into one file to reduce file access Solution: Installing free build acceleration tool – e.g. Huddle from Electric Cloud
  39. 39. Version Control - Team Foundation Version Control (TFVC) - Git - Branching & Merging Project Management - Scrum, Agile - Defect Tracking, Bugs QA Testing / Test Case Management Automated Builds Automated Release Pipelines Team Foundation Server
  40. 40. Source Code Build automation Executable Code Deployment automation Target environment Target environment Target environment
  41. 41. Source Code Build automation Executable Code Deployment automation Target environment Target environment Target environment Team Foundation Server  Maximize automation with scripts and custom tools.  Include automated testing.  Provide solid process and documentation in our software delivery practices.
  42. 42. Advantages of build automation to software development projects:-  A necessary pre-condition for continuous integration and continuous testing  Improve product quality  Accelerate the compile and link processing  Eliminate redundant tasks  Minimize “bad builds”  Eliminate dependencies on key personnel  Have history of builds and releases in order to investigate issues  Save time and money – because of the reasons listed above.
  43. 43. Conclusion
  44. 44. Future GPMT DevOps Ecosystem Workstations - On-Premises DEV TEST QA Environments - On-Premises Build&Test Build/CI Test Develop Developer Workstation Team Collaboration Deploy Release Monitor&Learn Monitor ALMServices - On-Premises TFS Monitoring- On-Premises
  45. 45. DevOps Orchestration Reduce costs and improve productivity and predictability by standardizing Dev and Ops toolchains DevOps teams like to move fast, and ship frequently. Individual teams often depend on (and are sometimes fiercely loyal to) a unique, curated toolchain to help them get things done. But many organizations want to consolidate tools to gain economies of scale and allow cross-team sharing. How can they orchestrate all of these tools in an efficient, auditable way.
  46. 46. Deployment Automation Automate deployments so App Support and Ops teams can keep up with the page of Agile delivery Consistently deploying applications into standardized environments is one of the best ways to achieve application rollout success, but most teams still rely on hard-coded scripts and manual instructions. App Support and Ops teams want to eliminate manual and error- prone deployment processes to achieve the velocity needed to deliver any software into any environment – especially into production, where value is delivered.
  47. 47. Release Management Control and track applications as they are promoted along the path to production Shipping code is the killer feature. Until code is deployed and accessible by users, no value has been created. But coordinating release activities and approvals for both traditional and CD pipelines across multiple teams and shared infrastructure is a logistical challenge. Organizations want a way to manage their enterprise traditional releases and CD pipelines in a scalable, auditable and predictable way.
  48. 48. Agile Automate deployments so Ops teams can keep up with the pace of Agile delivery Iterate and integrate often. Time is money. Requirements are always shifting. Agility is hard to achieve if your builds and tests take a long time or are not automated. Nimble, iterative, and incremental sprints get bogged down without a solid process automation framework in place. Organizations want to automate provisioning, build and release activities and handoffs to eliminate manual errors and wait times, shrinks cycle times and accelerate the feedback loop.
  49. 49. print(“Thank You!”)

Notas do Editor

  • Infrastructure as code
    - Allows to automate deployments in different environments (dev, test, staging, prod)
  • DevOps practitioners rely on automation. An automation platform gives us the ability to describe our entire technology stack as executable code.
  • Some people might question whether gains in efficiency are typical, noting that they have worked on projects where, in their view, trying to automate tasks actually cost the project more time than it saved. Depending on the situation, this may be a real concern. In addition, automation might have been implemented poorly or carried too far on some projects – but keep reading for on what to automate and when.
  • Because DevOps is a cultural shift and collaboration (between dev, ops and testing), there is no single “DevOps tool” – it is rather a set, consisting of multiple tools (DevOps toolchain).

    Generally, DevOps tools fit into one or more of these categories, which is reflective of key aspects of the software dev and delivery process.
  • The more components you add to your software, the more lines of code you maintain, and the more tests and routines you run as part of your build process- the longer the build will take to run. Also, some development environments and technologies, like C/C++, tend to be correlated with longer builds.

    In Agile work environments, builds are expected to be run frequently and the organization depends on build output to guide development work. Long builds can be a big problem, and one that can “creep up” on a development team.

    Why it matters?

    Some builds are so long they can only be ran nightly. But even if your “normal” build takes just 20 or 30 minutes more than it should – there are significant costs to your organization:

    Developer wait time – as we shown in a survey we conducted last year, most developers spend between 2-10 hours per week waiting for builds to complete – more often than not, this is lost time.

    Context switch – while the build runs, developers switch away from the current task to other ones. If the build is long, developers lose context related to the problem they were working on, which reduces productivity. In other words, if builds are supremely fast, developers never have to stop thinking about the problem they are working on right now.

    Build bottleneck – during intensive development phases such as “integration storms”, developers sync changes into the main code line and frequent builds are needed to get everything working. The less frequent the build, the less problems dev can solve every workday.

    Product quality – when a developer commits changes that break something, it will only be discovered after the build runs. When the bug is fixed, again there is a lag waiting for the next build before QA can verify the fix. The less frequent the build, the less issues can be fixed and verified before the release – hurting product quality.
  • Sometimes, individual builds run relatively quickly. But in some organizations, there could be dozens or hundreds of builds run in each dev/test cycle. This could be because numerous teams (sometimes thousands of developers) developing different software components, and each running its own build. Or, the dev team might need to deliver numerous versions of the software for different platforms, customized builds for different customers, etc. Some organizations have relatively short builds– but find themselves needing to support dozens, if not hundreds, of these builds at any given time.

    Why it matters:

    If you’re a build engineer responsible for running 500 builds, you’ll feel the pain even if each of them is 10 minutes long (=~83 hours build time without parallelization). While builds may be short, cumulatively they take very long to run.

    But if you’re a developer or QA engineer using such as build system, and your build takes only 10 minutes, why should you care?

    Limited build resources – because the organization is running large numbers of builds, you’ll find you have limited access to build servers during specific time windows, or the servers are often overloaded and builds will take much longer.When you rely on running builds often to get fast feedback and fix bugs, you’ll notice that having to “wait in line” for your builds to run hurts your productivity (particularly if you’re practicing Agile), and that you can’t move development fast enough because your waiting for a build server.
  • Software projects use a large number of modular components: different frameworks, components developed by different teams or by 3rd-party partners, open source libraries, and so on. As your product evolves, there are multiple versions of your own code, and also multiple versions of these many components, creating a many-dimensional matrix of dependencies and supported behaviors. That’s where things get complex to build.

    Why it matters:

    Complex builds reduce the flexibility of the build process and makes it much more difficult to manage and run the build:

    Complex builds are brittle – interactions between many different components often lead to manual error, broken builds and worse – builds that run correctly but introduce bugs due to partial or incorrect sources.

    Extensive manual efforts – executing a complex build and delivering its results requires a substantial manual effort. Even if the build is automated, it is typically automated in small pieces/components, and there is no orchestration of the entire process.

    Incremental builds are difficult– often you’ll want to run a partial build and re-purpose items that haven’t changed and were previously compiled. With complex builds – due to partially specified dependencies – an incremental run could break the build and teams are forced to run the entire build in all scenarios.

    Legacy components and fear of change – complex builds tend to have legacy components written years ago by staff members who are no longer at the organization. This impedes changes or optimizations to the build, for fear of possibly breaking legacy components that are not well understood.

    Complex builds are long – there is a correlation between the complexity of the build and the time it takes to run, which introduces additional issues as described above.