Continuous deployment speeds up the process of existing agile methods, such as Scrum, and Extreme Programming (XP) through the automatic deployment of software changes to end-users upon passing of automated tests. Continuous deployment has become an emerging software engineering process amongst numerous software companies, such as Facebook, Github, Netflix, and Rally Software. A systematic analysis of software practices used in continuous deployment can facilitate a better understanding of continuous deployment as a software engineering process. Such analysis can also help software practitioners in having a shared vocabulary of practices and in choosing the software practices that they can use to implement continuous deployment. The goal of this paper is to aid software practitioners in implementing continuous deployment through a systematic analysis of software practices that are used by software companies. We studied the continuous deployment practices of 19 software companies by performing a qualitative analysis of Internet artifacts and by conducting follow-up inquiries. In total, we found 11 software practices that are used by 19 software companies. We also found that in terms of use, eight of the 11 software practices are common across 14 software companies. We observe that continuous deployment necessitates the consistent use of sound software engineering practices such as automated testing, automated deployment, and code review.

1. 1
Synthesizing Continuous
Deployment Practices Used in
Software Development
Akond Ashfaque Ur Rahman, Eric Helms, Laurie
Williams, and Chris Parnin
Department of Computer Science,
North Carolina State University

2. 2
Why Continuous Deployment ?
• “one of the top ten strategic technology
trends for 2015 ” – Gartner
• emerging software development process

3. 3
Motivation
Software companies learn from the experience
of other similar software companies

4. 4
Research Objective
The goal of this paper is to aid software
practitioners in implementing continuous
deployment through systematic analysis of
software practices that are used by software
companies

5. 5
How Continuous Deployment is
Different ?

6. 6
Definition of Continuous Deployment
“as a software process that releases software
changes automatically to end-users after they
pass the required automated tests. ”
Martin FowlerImage Reference: http://martinfowler.com/

7. 7
Definition of Continuous Delivery
“the software process that actually releases
software to production as soon as they are
ready, resulting in many deployments to
production every day. ”
Image Reference: http://continuousdelivery.com/ Humble & Farley

8. 8
Our Definition
“a software engineering process where
incremental software changes are automatically
tested, and frequently deployed to production
environments. ”

9. 9
Realization of Continuous Deployment
Varies
One Example
vs.

10. 10
Adoptees

11. 11
Our Contributions
• A summary and concise definition of 11
continuous deployment practices
• A mapping of the 11 continuous deployment
practices and the software companies who are
using these practices
• A list of techniques adoptees have used to
realize each of the identified continuous
deployment practices

12. 12
Research Question
How frequently are different software
practices used by software companies that
perform continuous deployment?

13. 13
Methodology
Yes
Other 18 adoptees :
internet artifacts
Facebook :
Feitelson et al.
No

14. 14
1. Automated Deployment (19)
- making software available to end-users automatically
- conducted in between software acquisition and software
execution without manual effort
Technique
Used
Total Count Examples of Adoptees
Other Tools 13 CoolBlue, Etsy
Scripting 4 IMVU, Rally Software
BitTorrent 2 Facebook, Pinterest
Deployinator, Ansible, Codeship,
Octodeploy etc.

15. 15
2. Automated Testing (19)
- practice of automated techniques to perform test case
management, test data generation etc.
Type of Testing Total Count Examples of Adoptees
Unit 15 Etsy, Facebook
Integration 13 Etsy, IMVU
A/B 9 Etsy, Facebook
Functional 8 CoolBlue, Etsy
Acceptance 5 Atlassian, Netflix
Regression 5 CoolBlue, Etsy
Perceptual 1 Google Consumer Surveys

16. 16
2. Automated Testing [Contd.]
Technique Used
Total
Count
Examples of
Adoptees
Automated Test Suites 19 Etsy, Facebook
Separate Testing Team 2 Atlassian, Netflix

17. 17
Technique Used Total Count Example of Adoptees
Other Tools 9
CoolBlue, Google
Consumer Surveys
Github Tools 7 Atlassian, Etsy
Manual
Inspection
6 Atlassian, Facebook
Phabricator 2 Facebook, Quora
3. Code Review (16)
- practice of presenting software changes to different
stakeholders for comments and approval
Lint checkers,
Static analysis
tools, coding
convention tools
etc.

18. 18
4. Dark Launching (9)
- practice of deploying software changes by keeping the functional aspects
hidden to end-users
Technique
Used
Total Count Example of Adoptees
Gatekeeper 1 Facebook
Note: Other adoptees who use this practice did not discuss their
techniques

19. 19
5. End-User Communication (10)
- practice of communicating with end-users to receive feedback and gather
requirements
Technique Used Total Count Example of Adoptees
Official Forums 6 Etsy, Facebook
Social Networking
Websites
2 Github, Spreaker
Phone Calls 1 Wealthfront
Web Seminars 1 Rally Software

20. 20
6. Feature Flags (13)
- the practice of triggering a specific software branch amongst
many software branches
Technique Used Total Count Example of Adoptees
Conditional
Framework
1 Rally Software
Configuration Flags 1 Etsy
Note: Other adoptees who use this practice did not discuss their
techniques

21. 21
7. Intercommunication (14)
- practice of sharing all necessary development and deployment
information amongst team members
Technique
Used
Total Count Example of Adoptees
Conversation
Bots
2 Facebook, Flickr
ChatOps 1 Github
Gerrit 1 URLinkedUp
Note: Other adoptees who use this practice did not discuss their
techniques

22. 22
8. Monitoring (16)
- practice of collecting deployment related information, producing
appropriate performance metrics
Technique
Used
Total Count Example of Adoptees
Graphite 2 Etsy, Github
Nagios 2 IMVU, Rally Software
Splunk 1 Rally Software
Note: Other adoptees who use this practice did not discuss their
techniques

23. 23
- practice of deploying the master or a different branch of the
software
Technique Used
Total
Count
Example of Adoptees
Trunk Shipment 14 Coolblue, Etsy
Branch Shipment 3 Atlassian, Netflix
9. Repository Use (19)

24. 24
10. Shepherding Changes (14)
- practice of making developers responsible for their software
changes
Technique
Used
Total Count Example of Adoptees
On-call policy 2 Etsy, Github
Boot camps 1 Facebook
Note: Other adoptees who use this practice did not discuss their
techniques

25. 25
11. Staging (17)
- practice that controls the deployment of software changes to internal and
production servers
Technique Used
Total
Count
Example of Adoptees
Gradual roll out 17 Etsy, Facebook
Dogfooding 12 Etsy, Facebook

26. 26
Empirical Findings
Type of Product
Deployed
Adoptee Total Count
Other Type of Websites
Atlassian (AT), Google Consumer
Surveys (GCS), Kitchensurfing (K),
Netflix (N), Outbrain (O), Rally
Software (RS), Spreaker (S)
7
Social Networking
Websites
Facebook (FB), Flickr (FR), Github (G),
IMVU (I), Quora (Q), Pinterest (P)
6
E-Commerce Websites
Coolblue (CB), Etsy (E), Flipkart (FK),
URLinkedUp (U), Wealthfront (WF)
5
Desktop Software Atlassian (AT) 1
Review Websites Yelp (Y) 1
Survey, Cooking,
Audio etc.

27. 27
Empirical Findings
0
3
6
9
12
15
18
CountofPractices
Commonality of Practices
Count of Adoptees (Yes) Count of Adoptees (No) Count of Adoptees (Unknown)

28. 28
Empirical Findings
- Academic articles
- Speakerdeck presentations
0
5
10
15
20
25
Blog Posts Slideshare
Presentations
InfoQ Presentations Videos Other
Count
Type of Internet Artifacts
Count of Different Internet Artifacts

29. 29
Empirical Findings
0
3
6
9
12
15
18
CountofPractices
Use of Internet Artifacts and Follow-up Inquiries
Internet Artifacts Follow-up Inquiries

30. 30
Observations
• Commonality of Continuous Deployment
Practices
• Different Types of Testing
• Shepherding Changes

31. 31
Limitations
• Keyword-based search mechanism
• Evaluation of the 11 software engineering
practices are missing

32. 32
Future Directions
• Understanding how these 11 practices help
adoptees to achieve continuous deployment
successfully
• Providing guidelines on how software
companies can use the identified 11 software
practices to implement continuous
deployment effectively

33. 33
Conclusion
• Summarizing 11 specific software practices used in
industry to implement continuous deployment, and
the adoptees that are using these practices
• Mapping of the 11 continuous deployment practices
and the software companies who are using these
practices
• Observing the commonality of practices used to
implement continuous deployment

34. 34
Related Work
• G. G. Claps, R. B. Svensson, and A. Aurum, “On the
journey to continuous deployment: Technical and social
challenges along the way,” in Information and Software
Technology, January, 2015
• N. F. Velasquez, G. Kim, N. Kersten, and J. Humble, “2014
State of DevOps Report,”
https://puppetlabs.com/sites/default/files/2014-state-
of-devops-report.pdf, 2014.
• M. Leppanen, S. Makinen, M. Pagels, V. Eloranta, J.
Itkonen, M. Mantyla, V. Mika, and T. Mannisto, “The
Highways and Country Roads to Continuous
Deployment,” in IEEE Software, March-April, 2015