Harness your Jenkins using jobDSL.

1. Take control of your Jenkins jobs via jobDSL
#1 SysOps / DevOps Krak´ow Meetup
Lukasz Proszek
@lukaszproszek
2017-11-29, Krak´ow
Senior DevOps Engineer @Lumesse
1

2. Introduction

3. About Me
• Senior Development Operation Engineer @ Lumesse
• If something doesn’t have a CLI, it is as if it never existed
at all
• Automates almost everything
• My goal: ”Everything works. What do we pay you for?”
• My reality: ”Nothing works. What do we pay you for?”
2

4. Why am I even here?
• A medium sized jenkins installation
• 1 master, 10 slaves
• 1032 Jenkins Jobs
• 29 jobs marked ’to-delete’
• 84 jobs marked ’usunsed’
• dozens of jobs that nobody remembers
• everybody has configure rights
• post-mortem job changes analysis
3

5. Jenkins: GUI

6. Working with Jenkins: GUI
4

7. Working with Jenkins: GUI. cont.
My opinion about the GUI:
• manual
• error prone
• wrong kind of boring
• workable for a dozen of jobs
• unmaintanable for hundreds
5

8. Working with Jenkins: CLI

9. Working with Jenkins: CLI (jar)
• Jenkins CLI via Jar application
• jenkins.io/doc/book/managing/cli
1 wget http://jenkins.local/jnlpJars/jenkins-cli.jar
2 export JENKINS_URL="http://jenkins.local"
3 echo "user:pass" > AUTHFILE
4 java -jar jenkins-cli.jar -auth @AUTHFILE who-am-i
6

10. Working with Jenkins: CLI (ssh)
• Jenkins CLI via SSH client
• paste your ssh pub key in
http://jenkins.local/user/MYUSERNAME/configure
1 curl -LI http://jenkins.local/login | grep ’X-SSH-Endpoint’
2 ssh -l MYUSERNAME jenkins.local -p ${PORT} who-am-i
7

11. Working with Jenkins: CLI (commands)
1 ssh -l MYUSERNAME jenkins.local -p ${PORT} help build
1 ssh -l MYUSERNAME jenkins.local -p ${PORT}
2 build release-my-software
3 -c # build only if there are changes in scm
4 -f # follow progress
5 -v # print console output
List of commands: jenkins.local/cli
8

12. Working with Jenkins: Script
Console

13. Working with Jenkins: Script Console
• Script Console: http://jenkins.local/script
9

14. Working with Jenkins: Scrip Console (example)
Cancelling running builds
1 for (job in this.hudson.instance.items) {
2 for (build in job.builds) {
3 if (build == this.build) { continue; } // don’t cancel ourself!
4 if (!build.hasProperty(’causes’)) { continue; }
5 if (!build.isBuilding()) { continue; }
6 for (cause in build.causes) {
7 if (!cause.hasProperty(’upstreamProject’)) { continue; }
8 if (cause.upstreamProject == this.upstreamProject &&
9 cause.upstreamBuild == this.upstreamBuild) {
10 this.printer.println(’Stopping ’ + build.toString());
11 build.doStop();
12 this.printer.println(build.toString() + ’ stopped.’);
13 }}}}
10

15. Working with Jenkins: Script Console
• Allows to run Groovy scripts on master and/or slave nodes
• Web based shell into the Jenkins runtime
• Can do anything. ANYTHING
• "cat /etc/passwd".execute().text
• No administrative controls
• ⇒ everyone with access is basically a jenkins admin
• Originally was intended as a debugging interface for Jenkins
developers.
11

16. Jenkins: Creating jobs

17. Job configuration: gui example - a simple job
12

18. Job configuration: xml example - a simple job
1 <?xml version=’1.0’ encoding=’UTF-8’?>
2 <project>
3 <keepDependencies>false</keepDependencies>
4 <scm class="hudson.scm.NullSCM"/>
5 <canRoam>true</canRoam>
6 <disabled>false</disabled>
7 <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
8 <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
9 <triggers/>
10 <concurrentBuild>false</concurrentBuild>
11 <builders/>
12 <publishers/>
13 <buildWrappers/>
14 </project>
13

19. Job configuration: xml example - a simple job
• Not so scary.
• But really. It does nothing at the moment
• Lets add a build step
14

20. Job configuration: xml example - a build step
15

21. Job configuration: xml example - a build step
1 <?xml version=’1.0’ encoding=’UTF-8’?>
2 <project>
3 <keepDependencies>false</keepDependencies>
4 <scm class="hudson.scm.NullSCM"/>
5 <canRoam>true</canRoam>
6 <disabled>false</disabled>
7 <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
8 <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
9 <jdk>(System)</jdk>
10 <triggers/>
11 <concurrentBuild>false</concurrentBuild>
12 <builders/>][<2><builders>
13 <hudson.tasks.Shell>
14 <command>cat /etc/passwd</command>
15 </hudson.tasks.Shell>
16 </builders>
17 <publishers/>
18 <buildWrappers/>
19 </project>
16

22. Job configuration: xml example - plugins
But the real power of jenkins is in plugins. Lets add some plugin
usage. Configure a code repository
17

23. Job configuration: xml example - plugins
1 <scm class="hudson.plugins.git.GitSCM" plugin="git@3.6.4">
2 <configVersion>2</configVersion>
3 <userRemoteConfigs>
4 <hudson.plugins.git.UserRemoteConfig>
5 <url>git://github.com/frogu/presentations.git</url>
6 </hudson.plugins.git.UserRemoteConfig>
7 </userRemoteConfigs>
8 <branches>
9 <hudson.plugins.git.BranchSpec>
10 <name>*/master</name>
11 </hudson.plugins.git.BranchSpec>
12 </branches>
13 <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
14 <browser class="hudson.plugins.git.browser.GithubWeb">
15 <url>git://github.com/frogu/presentations</url>
16 </browser>
17 <submoduleCfg class="list"/>
18 <extensions/>
19 </scm>
18

24. Configuration via GUI/XML: conclusion
• Done in web browser
• How to control changes?
• Imagine ediding this xml for more complicted jobs
• XML editing is not human friendly
19

25. No mouse chalenge
Try to write a job without the GUI1
1
https://mn.gov/mnit/blog/?id=38-232094
20

26. Jenkins Job DSL

27. Configuration as Code
• Treat your job configuration as every other piece of
config/code.
• Version your configuration.
• Do not Repeat Yourself
• Validate and review your changes
21

28. Jenkins Job DSL
• Simple (YMMV).
• Intuitve (YMMV again).
• Human and machine friendly.
• Groovy DSL.
• Generate all your jobs from a seed job.
• Abstracts your utility functions.
• Supports plugins through extensions.
• Easily (YMMV :)) template your jobs.
• Originally developed at Netflix
22

29. Job configuration: DSL
Previous example as DSL:
1 folder(’jobsandbox’)
2 folder(’jobsandbox/sodo’)
3 job("jobsandbox/sodo/example") {
4 scm {
5 git { github(’frogu/presentations’)
6 remote{ url("git://github.com/frogu/presentations.git")}
7 branch("master")
8 browser{ githubweb("https://github.com/frogu/presentations") }
9 }
10 }
11 steps {
12 shell("cat /etc/passwd")
13 }
14 }
23

30. DSL: multiple jobs
A several jobs with different names
1 def stages = [’ci’, ’preprod’, ’prod’]
2 stages.each {
3 job("job-for-${it}") {
4 steps{
5 shell("echo ${it}")
6 }
7 }
24

31. DSL: abstract your common functions - definition
A daily job trigger.
1 package devops
2 import javaposse.jobdsl.dsl.Job
3 public class Common {
4 static def addSCMTrigger(Job job){
5 job.with {
6 triggers {
7 scm(’H 12 * * *’)
8 }
9 }
10 }
11 }
25

32. DSL: abstract your common functions - usage
1 import devops.Common
2 def myJob = job {}
3
4 devops.Common.addSCMTrigger(myJob)
26

33. DSL: unsupported plugins - issue
• Currently not all jenkins plugins are covered by the DSL.
• How to deal with that challenge?
• Create a sample job in GUI and inspect it.
• At the time of DSL introduction at Lumesse, it did not
support Maven Metadata Parameter.
27

34. DSL: unsupported plugins - identify
1 <propeties>
2 <parameterDefinitions>
3 <eu.markov.jenkins.plugin.mvnmeta.MavenMetadataParameterDefinition
4 plugin="maven-metadata-plugin@1.5.0">
5 <name>MY_VERSION</name>
6 <description>Package version to be installed</description>
7 <repoBaseUrl>http://nexus.local/releases</repoBaseUrl>
8 <groupId>maven.group.id</groupId>
9 <artifactId>maven_artifact_name</artifactId>
10 <packaging>zip</packaging>
11 <classifier></classifier>
12 <sortOrder>DESC</sortOrder>
13 <maxVersions>10</maxVersions>
14 </eu.markov.jenkins.plugin.mvnmeta.MavenMetadataParameterDefinition>
15 </parameterDefinitions>
16 </properties>
28

35. DSL: unsupported plugins - dsl conversion
jobDSL provides a ”configure” closure.
1 <propeties>
2 <parameterDefinitions>
3 <eu.markov.jenkins.plugin.mvnmeta.MavenMetadataParameterDefinition
4 plugin="maven-metadata-plugin@1.5.0">configure {
5 it / ’properties’ / ’hudson.model.ParametersDefinitionProperty’ /
6 ’parameterDefinitions’ <<
7 ’eu.markov.jenkins.plugin.mvnmeta.MavenMetadataParameterDefinition’ {
8 <name>MY_VERSION</name>name("MY_VERSION")
9 <description>Package version to be installed</description>description ’Pa
10 <repoBaseUrl>http://nexus.local/releases</repoBaseUrl>repoBaseUrl ’http:/
11 <groupId>maven.group.id</groupId>groupId ("maven.group.id")
12 <artifactId>maven_artifact_name</artifactId>artifactId ("maven_artifact_n
13 <packaging>zip</packaging>packaging ("zip" )
14 <classifier></classifier>classifier ("")
15 <sortOrder>DESC</sortOrder>sortOrder ’DESC’
16 <maxVersions>10</maxVersions>maxVersions (10)
17 </eu.markov.jenkins.plugin.mvnmeta.MavenMetadataParameterDefinition>
18 </parameterDefinitions>
29

36. DSL: unsupported plugins - dsl final version
1 configure {
2 it / ’properties’ / ’hudson.model.ParametersDefinitionProperty’ /
3 ’parameterDefinitions’ <<
4 ’eu.markov.jenkins.plugin.mvnmeta.MavenMetadataParameterDefinition’ {
5 name("MY_VERSION")
6 description ’Package version to be installed’
7 repoBaseUrl ’http://nexus.local/releases’
8 groupId ("maven.group.id")
9 artifactId ("maven_artifact_name")
10 packaging ("zip" )
11 classifier ("")
12 sortOrder ’DESC’
13 maxVersions (10)
14 }
15 }
30

37. DSL: unsupported plugins - conclusion
• not so pretty
• but, it works!
• everything possible in the xml can also be done in the DSL
31

38. Jenkins Job DSL: factories

39. Job factory: use case
• Each job should have the same block configured, e.g.
• colorized output
• timestamped output
• e-mail notification on failure
• e-mail notification on success
• execution permissions dependant on the environment stage
• ...
32

40. Job factory: example wrappers
1 package com.lumesse.devops.builders
2 import javaposse.jobdsl.dsl.DslFactory
3 import javaposse.jobdsl.dsl.Job
4 import javaposse.jobdsl.dsl.*
5 class ExampleJobBuilder {
6 List emailRecepients
7 String jobName
8
9 Job build(DslFactory factory){
10 factory.job ("${jobName}"){]
11 wrappers {
12 colorizeOutput()
13 timestamps()
14 }
15 publishers {} // next slide
16 }
17 }
18 }
33

41. Job factory: example publishers
1 publishers {
2 extendedEmail {
3 emailRecepients.each{
4 recipientList(it)
5 }
6 triggers {
7 failure {
8 subject("fail")
9 content("F A I L")
10 }
11 success {
12 subject("success")
13 content ("OK")
14 }
15 }
16 }
17 }
34

42. Job factory: usage
Job Factory Usage
1 import com.lumesse.devops.builders.ExampleJobBuilder
2
3 new ExampleJobBuilder (
4 jobName: ’example-job-from-builder’,
5 emailRecepients: [’devops@company’, ’stakeholder1@company’]
6 ).build(this).with{
7 steps{
8 shell("env")
9 }
10 }
35

43. Jenkins job DSL: deployment
Use this repository as starting point
1 git clone https://github.com/sheehan/job-dsl-rest-example
36

44. Jenkins job DSL: master SeedJob
1 job(’master-seed’) {
2 scm {
3 git{
4 remote { url("ssh://git@git.local/devops/jenkinsdsl") }
5 branch("master")
6 }
7 }
8 triggers {
9 scm ’H/5 * * * *’
10 }
11 steps {
12 gradle ’clean test’
13 dsl {
14 external ’jobs/**/*SeedJob.groovy’
15 additionalClasspath ’src/main/groovy’
16 }
17 }
18 }
37

45. Jenkins job DSL: master SeedJob enablement
• The first job can be uploaded using a simple api call.
1 ./gradlew rest
2 -Dpattern="jobs/MasterSeedJob.groovy"
3 -DbaseUrl=http://jenkins.local/
4 -Dusername=myusername
5 -Dpassword=secret
38

46. Jenkins job DSL: master SeedJob
• Will poll the repository every 5 minutes
• On changed source, it will execute:
• gradle clean test
• a DSL build step that will recursively include all
”*SeedJob.groovy” files from the ”jobs” directory
• archive test results
39

47. Jenkins job DSL: jenkins harness
• You can test jobs before commiting them to the repository
• You can run it as a ”verfy pull request job”
• ./gradlew test
40

48. Conclusion

49. Conclusion
• Jobs are storred in git
• Builders for common jobs
• AdHoc extensible with .build(this).with{}
• Pull requests with approvals
• Pull request verification job
• Automagic job updates
41

50. Thank You
• jenkins.io/doc/book/managing/cli
• jenkinsci.github.io/job-dsl-plugin
• github.com/jenkinsci/job-dsl-plugin/wiki/Frequently-Asked-
Questions
• github.com/sheehan/job-dsl-gradle-example
• github.com/sheehan/job-dsl-rest-example
• wiki2beamer - latex-beamer made easy
• impressive - pdf presentations with sugar added
• github.com/frogu/presentations
42