1.
Analyze This!
CloudBees Jenkins Cluster Operations and Analytics
Ryan Campbell
Cyrille Le Clerc

2.
Today’s Presenters
Cyrille Le Clerc
Director, Product Management
cleclerc@cloudbees.com
@cyrilleleclerc
Ryan Campbell
Engineer
rcampbell@cloudbees.com
@recampbell

3.
Using WebEx
Questions?
Submit via the Q&A window at any time during
the presentation.
We will cover Q&A at the end of the presentation.
Session is being recorded.
Q&A Window

4.
Agenda
● Problem statement
● Jenkins Reference Architecture
● CloudBees Analytics
● Cluster Operations
● Conclusion
● Q&As

5.
Problem Statement

6.
Problem Statement
How to onboard more and more teams on a Jenkins
Continuous Delivery Platform
How to scale Jenkins in an organization

7.
The top down “structured” and “lets grow
vertically” approach
5 projects
0 slaves
25 projects
5 slaves
100 projects
25 slaves
300 projects
50 slaves
600 projects
200 slaves

8.
Vertical Scaling Issue
When is the next failure ?
One size fits all?
Single point of failure

9.
The “organic” agile growth –
“let each group do what they want to” approach
Horizontally … Virally

10.
Horizontal Issues: I Would Like to…
master-a
master-b master-c
Share slaves between masters

11.
Horizontal Issues: I Would Like to…
master-a
master-b master-c
Share security

12.
Horizontal Issues: I Would Like to…
master-a
master-b master-c
Manage the Jenkins
masters “globally”
plugin
core
upgrade / restart
backup
script

13.
Horizontal Issues: I Would Like to…
master-a
master-b master-c
Monitor the Jenkins
masters “globally”

14.
Jenkins Operations Center by CloudBees
master-a
master-b master-c
Jenkins Operations Center
plugin
core
upgrade / restart
backup
script

15.
Jenkins Reference
Architecture

16.
Reference Architecture
LB master-a
LB master-n
***
slaves
LB
Jenkins
Operations
Center
shared slaves
Role Based
Access Control
Update Center
SSO
LB master-test
Update Center
Jenkins Test
Environment

17.
master-a
master-b master-c
Jenkins Operations Center
JOC features
plugin
core
upgrade / restart
backup
CloudBees Analytics
Cluster Operations

18.
CloudBees Analytics

19.
CloudBees Analytics
master-a
master-b master-c
Monitor the Jenkins
masters “globally”

20.
Drivers for Analytics
● Easily understand the performance and behavior
of hundreds of Jenkins nodes
● Consistent interface allowing for easy
comparisons over time and between nodes
● Key questions:
○ What is the user experience?
○ Why is Jenkins slow?
○ How is Jenkins being used?
○ How are resources being used? → capacity

21.
CloudBees Analytics
master
Jenkins Operations Center
Data Collection
Scalable compute and
storage backend
Rich visualization interface

22.
Embedded Architecture
Jenkins Operations Center
Embedded
Elasticsearch
Embedded Kibana

23.
Horizontally Scaled Architecture
Jenkins Operations Center
Embedded Kibana
Multiple Remote
Elasticsearch Instances

24.
Analytics Demo
JOC Dashboard

25.
Analytics Demo
Build Analytics

26.
Analytics Demo
Performance Analytics

27.
Analytics Demo
Dashboard Creator

28.
Cluster Operations

29.
Cluster Operations
master-a
master-b master-c
Manage the Jenkins
masters “globally”
plugin
core
upgrade / restart
backup
script

30.
Cluster Operations Agenda
● Update all plugins on all masters and restart
● Deploy new plugin on masters
● Cluster Operations Ref Card

31.
Update Plugins on All Masters and
Safely Restart

32.
Demo time!

33.
Cluster Operations
Standard Jenkins Style: New Item “Clustered Operations”

34.
Cluster Ops > Master Operation
Standard Jenkins Style: Add Operation “Masters”

35.
Master Operation > Select Targets
Select the Client Masters on which you perform the operation

36.
Master Operation > Define Operations
Define the sequence of operations to apply on the Client Masters

37.
Deploy Plugin on Masters
via a Custom Update Center

38.
Deploy New Plugin
Update Center master-test
Upload Plugin
Refresh Update Center Metadata
Install Plugin

39.
Demo time!

40.
Deploy Plugin on Masters
Parameterized build: File Parameter

41.
Deploy Plugin on Masters
Target Update Center

42.
Deploy Plugin on Masters
Upload Plugin on Custom Update Center

43.
Deploy Plugin on Masters
Target Masters

44.
Deploy Plugin on Masters

45.
Quick Cluster Operations

46.
Demo time!

47.
Quick Cluster Operations

48.
Quick Cluster Operations

49.
Cluster Operations Ref Card

50.
Master Operation > Ref Card
● Backup master
● Disable plugin
● Execute Groovy Script on Master
● Install plugin
● Prepare Master for shutdown
● Refresh update center metadata

51.
Master Operation > Ref Card
● Reindex for Analytics
● Restart Now
● Safe restart
● Upgrade Jenkins
● Upgrade all plugins

52.
Update Center Operations > Ref Card
● Delete core
● Delete plugin
● Promote core
● Pull Everything
● Pull new versions
● Refresh upstream sources

53.
Update Center Operations > Ref Card
● Track latest core
● Track latest plugin
● Upload core
● Upload plugin

54.
Conclusion

55.
master-a
master-b master-c
Jenkins Operations Center
Jenkins Operations Center
plugin
core
upgrade / restart
backup
CloudBees Analytics
Cluster Operations

56.
CloudBees Analytics
master-a
master-b master-c
Monitor the Jenkins
masters “globally”

57.
Cluster Operations
master-a
master-b master-c
Manage the Jenkins
masters “globally”
plugin
core
upgrade / restart
backup
script

58.
Resources
http://www.slideshare.net/cloudbees/analyze-this-jenkins-cluster-operations-and-analytics

59.
Questions

60.
Appendix

61.
Webinar Invitation
More and more organizations are jumping on the Continuous Delivery bandwagon to remain competitive.
As they do so, they use Jenkins to on-board teams and to orchestrate their continuous delivery pipelines.
Jenkins Operations Center by CloudBees is the tool that helps organizations run their CI infrastructure at
scale.
In this webinar, you will learn about:
● Reference architecture to build resilient Jenkins that onboard teams quickly
● Cluster Operations - helps to manage multiple Jenkins instances simultaneously.
● Want to install a new plugin on a 4 Jenkins masters ? We got that covered!
● CloudBees Analytics - offers insight into build and performance analytics.
● Want to know the number of jobs failing across 4 masters - we've got that covered too!