Mais conteúdo relacionado Semelhante a Distributed Docker Pipeline Architecture with CloudBees Jenkins Enterprise (20) Distributed Docker Pipeline Architecture with CloudBees Jenkins Enterprise1. © 2017 CloudBees, Inc. All Rights Reserved. 1
Distributed Pipeline Architecture
with CloudBees Jenkins Enterprise
and Docker
WORKSHOP
2. © 2017 CloudBees, Inc. All Rights Reserved. 2
Prerequisites
● A GitHub Account - github.com
○ with a personal access token -
https://github.com/settings/tokens
● A Docker Hub Account - hub.docker.com
● General understanding of Jenkins Pipeline
4. © 2017 CloudBees, Inc. All Rights Reserved. 4
CD enables speed
A single CD pipeline provides your
team speed with little instability...
5. © 2017 CloudBees, Inc. All Rights Reserved. 5
But as CD scales business risk increases
As other teams adopt CD instability
increases and speed decreases
6. © 2017 CloudBees, Inc. All Rights Reserved. 6
With CloudBees, continuous scaling just works
DPA enables scale with isolated,
on-demand containerized CD servers
for each of your team
...in fact you can move faster
7. © 2017 CloudBees, Inc. All Rights Reserved. 7
CloudBees Distributed Pipeline Architecture
CloudBees Distributed Pipeline Architecture reduces business
risk through:
• Project Isolation
• DevOps project teams get their own Jenkins Master
• Distributes workload across masters
• Cross project contamination of workspaces and data is
eliminated
• Scalable Architecture
• Scaling and elasticity achieved through use of cluster
managed containers
• Data Isolation
• Data contamination from previous executions are easily
eliminated
8. © 2017 CloudBees, Inc. All Rights Reserved. 8
CloudBees Jenkins Enterprise Cluster
Automated Cluster Install
• Robust CLI for managing cluster
• Automated infrastructure
provisioning for AWS and
OpenStack
Elastic/Ephemeral Jenkins Agents
• Dynamically spins up Docker
images as agents, stops and
removes on job completion
Built-in Fault Tolerance
• JENKINS_HOME is externally
managed to provide HA for
CJOC & Jenkins Masters
• Unhealthy CJOC & Jenkins
Masters are automatically
restarted on a healthy node
Straightforward, Fast, Dynamic
Provisioning of Jenkins Masters
• Easy & quick to create Masters
of any size
• Entire Jenkins Master lifecycle is
managed by CJOC
9. © 2017 CloudBees, Inc. All Rights Reserved. 9
CloudBees Jenkins Enterprise Installation
Standing up a new CloudBees Jenkins Enterprise Cluster is straightforward
and fast:
• Download CJE install/CLI binary to a dedicated admin workstation
• Create a CJE project:
• $ cje init-project cje-project aws
• From the CJE project directory, initialize the new cluster:
• $ cje prepare cluster-init
• Configure common settings in cluster-init.config
• Verify and Install
• $ cje verify
• $ cje apply
• In 30 minutes or so, you will have a new CJE cluster
10. © 2017 CloudBees, Inc. All Rights Reserved. 10
HANDS-ON TIME
CREATE A CJE USER ACCOUNT
● Open https://goo.gl/3tRlSQ
● Click on the Create an account link
● Enter information for all fields for the Sign up form
○ Username - needs to match your GitHub
username
○ Password - no restrictions, keep it simple
● Click the Sign Up button
● Success ...
11. © 2017 CloudBees, Inc. All Rights Reserved. 11
Fast Provisioning of Jenkins Masters
DEMO
13. © 2017 CloudBees, Inc. All Rights Reserved. 13
FROM cloudbees/cje-mm:2.46.3.2
#skip setup wizard and disable CLI
ENV JVM_OPTS -Djenkins.install.runSetupWizard=false -Djenkins.CLI.disabled=true -server
#jenkins master configuration (groovy scripts)
COPY ./init.groovy.d/* /usr/share/jenkins/ref/init.groovy.d/
COPY ./license-activated/*
/usr/share/jenkins/ref/license-activated-or-renewed-after-expiration.groovy.d/
COPY ./quickstart/* /usr/share/jenkins/ref/quickstart.groovy.d/
#install additional plugins
ENV JENKINS_UC http://jenkins-updates.cloudbees.com
COPY plugins.txt plugins.txt
COPY jenkins-support /usr/local/bin/jenkins-support
COPY install-plugins.sh /usr/local/bin/install-plugins.sh
RUN /usr/local/bin/install-plugins.sh $(cat plugins.txt)
1
2
3
4
5
6
14. © 2017 CloudBees, Inc. All Rights Reserved. 14
Configure CJOC to Use Custom Master Image
beedemo/cje-mm:2.46.3.2
DEMO
15. © 2017 CloudBees, Inc. All Rights Reserved. 15
HANDS-ON TIME
PROVISION A MANAGED MASTER
● Open {CJE CJOC URL}
● Select New Item from the left menu
● Enter your first initial and last name as the item
name -> kmadel
● Select Managed Master as the item type
● Click OK
● On the Managed Master configuration page, under
Provisioning, click the Advanced button and
update the Disk value to 10
● Click the Save button at the bottom
16. © 2017 CloudBees, Inc. All Rights Reserved. 16
HANDS-ON TIME
FORK GITHUB REPOS
● Login to github.com
● Open github.com/beedemo/go-demo
● Click on the Fork button on the top-right of the page
● Select the GitHub account/Organization your are using for
this workshop
● Repeat for:
○ github.com/beedemo/maven-project
○ github.com/beedemo/jenkins-dind-agent
17. © 2017 CloudBees, Inc. All Rights Reserved. 17
Explore the Config-as-Code Managed Master
DEMO
19. © 2017 CloudBees, Inc. All Rights Reserved. 19
HANDS-ON TIME
CONFIGURE DOCKER HUB CREDENTIALS
● Open {YOUR MASTER URL} and click the Credentials link
in the left menu
● Under Stores scoped to Jenkins, click on the (global) link
● Click the Add Credentials link in the left menu
● The Kind should default to Username with password - if
not select that from the dropdown
● Enter your Docker Hub Username and Password
● Enter an easy to remember ID such as docker-hub-kmadel
● Click the OK button at the bottom
20. © 2017 CloudBees, Inc. All Rights Reserved. 20
Blue Ocean and Declarative Syntax for Pipeline
Blue Ocean is the new user interface that rethinks the user experience of
Jenkins with a focus on Pipeline.
Declarative syntax for Jenkins Pipelines designed to…
• Have smarter default behavior for most use cases
• Make Pipelines more structured
• Provide better error reporting and handling
21. © 2017 CloudBees, Inc. All Rights Reserved. 21
Explore the go-demo Jenkinsfile
DEMO
22. © 2017 CloudBees, Inc. All Rights Reserved. 22
HANDS-ON TIME
UPDATE THE GO-DEMO JENKINSFILE
● Login to github.com and open github.com/{user/org}/go-demo
● Switch to the build-cache-image branch
● Open the Jenkinsfile in edit mode
● Update the DOCKER_HUB_USER to match your Docker Hub
username
● Update the DOCKER_CREDENTIAL_ID to match the ID of the
Docker Hub credential you created earlier
23. © 2017 CloudBees, Inc. All Rights Reserved. 23
HANDS-ON TIME
UPDATE THE GO-DEMO JENKINSFILE
● Go to the bottom to the go-demo Jenkinsfile - to the post section
● Add a success condition block, and within that block add the following step
to send a Slack notification:
○ slackSend(color: "good", message: "${env.JOB_NAME} completed
successfully, details at ${env.RUN_DISPLAY_URL}")
● Now we will add a failure block with a slight modified Slack step:
○ slackSend(color: "danger", message: "${env.JOB_NAME} failed,
details at ${env.RUN_DISPLAY_URL}")
● Add whatever Commit changes description you like and then click the
Commit changes button to commit to the build-cache-image branch
24. © 2017 CloudBees, Inc. All Rights Reserved. 24
Palace and Docker Agent Templates
• Palace is a built-in Shared Agent Cloud for instant access to dynamically
provisioned Jenkins Agents
• It provides Dynamic, Ephemeral and Elastic Docker Image based agents
• Easy to configure Docker Agent Templates - individual items, not part of
one massive Jenkins Cloud configuration file
• Allows defining agent templates as separate items - no other Jenkins Cloud
plugin has this level of self-service capability
• The Palace service has built-in fault tolerance identical to what is provided
for CJOC and Managed Masters
• If the cluster node running the Palace service goes down, Marathon will
automatically restart it on another healthy node
25. © 2017 CloudBees, Inc. All Rights Reserved. 25
HANDS-ON TIME
CREATE A DOCKER AGENT TEMPLATE
● Open {YOUR MASTER URL} and click the New Item link in the left menu
● Enter dind-compose for the item name
● Next, scroll down and select Docker Agent Template as the item type and click
the OK button
● Update the CPU shares to 0.2 and the Memory (MB) to 512
● For the Definition Image enter localhost:31095/beedemo/dind-compose-agent:go-demo
● Next, under Options, click the Add button and select Launch in privileged
mode - this is required to run Docker in a Docker container
● Click Add again, select Use a custom docker shell command and enter
wrapper.sh as the Custom shell command
● Finally, click the Save button
26. © 2017 CloudBees, Inc. All Rights Reserved. 26
HANDS-ON TIME
CREATE A GITHUB ORG PROJECT WITH BLUE OCEAN
● Open {YOUR MASTER URL} and click on the Blue Ocean link at the top of the page
● Next, click on the New Pipeline button on the left side of the screen
● Click on the Github button
● Enter your GitHub personal access token and click Connect
● Next select your GitHub user or a GitHub Organization you are using for this
workshop
● Select Auto-discover Jenkinsfiles to scan all of the repos in your select
account/Organization
● Click on the Create Pipelines button and watch Blue Ocean discover the repos that
have branches containing a Jenkinsfile and then click on the build-cache-image
branch
27. © 2017 CloudBees, Inc. All Rights Reserved. 27
HANDS-ON TIME
CREATE A PULL REQUEST TO THE GO-DEMO MASTER BRANCH
● Exit to the Jenkins classic UI by clicking on the Go to classic button
● Next, click on the Organization Folder project, click the Configure link, then click the
Save button and then switch back to the Blue Ocean UI
● Now login to github.com and open github.com/{user/org}/settings/hooks
● Next go to github.com/{user/org}/go-demo and switch to the build-cache-image
branch and click the New pull request button
● GitHub will switch you to the beedemo Org, select the base fork drop-down and
select your account/org, then select the base drop-down and select master
● Next click the Create pull request button and you should see that All checks have
passed already, so go ahead and click on Merge pull request and then Confirm
merge
● Switch back over to your Managed Master and the Blue Ocean UI
28. © 2017 CloudBees, Inc. All Rights Reserved. 28
Pipeline Project Recognizers and Custom Marker Files
• By default, the Pipeline Organization and Multi-branch scans will only
include repositories that have at least one branch with a file named
Jenkinsfile
• CloudBees has a feature that allows you to specify arbitrary file names,
such as pom.xml, to use instead of Jenkinsfile when scanning branches
for Pipeline Organization Folder and Multi-branch projects
• This feature allows you to automatically provide standard automation for
teams that aren’t ready to start using Jenkins Pipeline and Jenkinsfiles
quite yet
29. © 2017 CloudBees, Inc. All Rights Reserved. 29
HANDS-ON TIME
SET UP CUSTOM MARKER FILES
● Exit to the Jenkins classic UI by clicking on the Go to classic button
● Next, click on the Organization Folder project, click the Configure link
● Scroll down to the Project Recognizers section and click the Add button
underneath the default Pipeline Jenkinsfile project recognizer and select
Custom script
● Enter pom.xml for the Marker file, then under the Pipeline section, select
Pipeline script from SCM as the Definition and Git as the SCM
● Enter https://github.com/beedemo/custom-marker-pipelines for the
Repository URL and then enter pom-Jenkinsfile as the Script Path
● Now repeat, only specifying Dockerfile as the Marker file and
Dockerfile-Jenkinsfile as the Script Path, the click the Save button
30. © 2017 CloudBees, Inc. All Rights Reserved. 30
Explore the beedemo/custom-marker-pipelines Repo
DEMO
32. © 2017 CloudBees, Inc. All Rights Reserved. 32
HANDS-ON TIME
CONFIGURE GITHUB CREDENTIALS
● Click the Global credentials (unrestricted) link at the top of the
page
● Click the Add Credentials link in the left menu
● The Kind should default to Username with password - if not
select that from the dropdown
● Enter your GitHub account id as the Username value
● Enter your GitHub access token as the Password value
● Enter an easy to remember ID such as github-token-kmadel
● Click the OK button at the bottom