17. Artifacts in the pipeline
• WAR
• Trace file
[BUILD]
Build: 221
Jenkins URL: http://localhost:2001/job/build/221/
Hg revision: f78504a525a617ad319e75bb288c24bdcb325794
Hg log:
changeset: 40:f78504a525a6
tag: tip
user: Jevgeni Kabanov <jevgeni@zeroturnaround.com>
date: Tue Oct 09 13:16:26 2012 +0000
summary: commented out check for HOTPATCH mode - we can now make health checks from localhost
[TEST]
Jenkins URL: http://localhost:2001/job/automatic-tests/161/
Automated Tests Passed!!!
[QA]
Manual tests passed!!!
[RC]
Marked as RC
24. Pipeline in
action
http://cddemo.zeroturnaround.com/lr-demo/
25. Questions revisited
•How do you package the
application?
•Where did it come from?
•Where does it go?
•How does it get deployed?
•What exactly is in prod now?
29. No way my boss let’s me do this!
•Changing process is hard
30. No way my boss let’s me do this!
•Changing process is hard
•SOLUTION: Sneak it in :)
31. No way my boss let’s me do this!
•Changing process is hard
•SOLUTION: Sneak it in :)
•Create a workflow that
captures current process
32. No way my boss let’s me do this!
•Changing process is hard
•SOLUTION: Sneak it in :)
•Create a workflow that
captures current process
•Then Automate!
33. Conclusions
•Jenkins jobs represent the
workflow
•Nexus is a sync-point for long-
running workflows
•LiveRebel does updates
•Manual flows with email/REST
•Tracking with scripts & text files
34. Pragmatic Continuous Delivery
Neeme Praks
@nemecec
LiveRebel Product Lead
ZeroTurnaround
Want more?
http://zeroturnaround.com
Google: “pragmatic continuous delivery”
Continuous Delivery - a way to automate your development and production environment so you can deliver (small) batches of functionality to production in a safe and predictable way.\n\n\n
Foremost: problem solver\nStarted as a developer and have been a Java developer for 10+ years and occasionally also handling the operations side of software delivery - I have seen and felt the pain of manual updates. After all that, I wanted to help out operations and joined ZeroTurnaround to work on LiveRebel product - the deployment tool for Java applications.\n\nThis talk is based on a talk given by ZT CEO, Jevgeni Kabanov\n
Shameless plug!\n\nJRebel - no more waiting for redeploys while developing!\nLiveRebel - predictable application updates without users noticing!\n\nApproach me during or after the conference:\nI can give a quick demo\nfree trial licenses available! Free licenses for Scala devs!\n\nBTW, we&#x2019;re hiring! If you like hacking the JVM - get in touch!\n
Why we built the pipeline in this particular way\n
Every step is trackable, every failure is recoverable\n
On a high level, Java EE seems similar. And easy.\n\nYet everyone complains, why?\n\nUsually, new applications are rushed into production\n\nAfter a while, you start to get questions like...(next slide)\n
We did a survey, asking ~700 people, if they knew answers to those questions.\n\nConclusion:\nThere is no standard solution - each environment is different.\nWorse yet, typically these processes are not automated!\n\n
Doing all these steps manually: each segment is automated, but human error creeps in between segments\n
Automated pipeline - humans are gatekeepers, but no manual processes.\n\nYou push code in from one end and functionality is pushed to users from the other end. \n\nHow do we achieve this &#x201C;pipe dream&#x201D;?\n
Automate - eliminate human error (human is still in control).\nRecord - to be able to learn from failures, to figure out where something went wrong.\nTests - not just unit, integration or smoke tests. Also monitoring! Little difference between good tests and good monitoring?\nRecover - always have a plan B. If you cannot roll back automatically, you need to know how to roll back manually.\n\n
All tools are available now.\nYou can use your own tools, same concept still applies.\n\nIt took us 3 weeks to build, but a lot of that was wrestling with Jenkins quirks.\n\nTo help you in building your own pipeline, we have prepared a step-by-step tutorial, available on our homepage.\n
Orchestration - something to run the pipeline steps\nRepository - something to hold the artifacts\nDelivery Manager - something to deploy the artifacts\n\nTesting and monitoring is out of scope.\n\n
Jenkins - freely available, open source. Most popular.\nTeamcity/Bamboo are just as good\n\nPlugins: build pipeline plugin, email-ext plugin\nBuild pipeline is the best there is (currently). There is also &#x201C;flow&#x201D; plugin, but it lacks documentation.\n\n
Artifactory is an alternative\n\nWe have set up different repositories for BUILD, QA, RC and TEST - this is used to keep different pipeline stages as independent as possible.\n
You can also use Cargo or just custom scripting.\nLiveRebel can do on-the-fly updates or rolling restarts.\n\n
Small arrows signify artifact movement.\n\nEach time we upload an artifact, we upload it to a different repository: build, test, QA, release candidate\n
Artifacts flowing through the pipeline\n\nNotice the different stages in trace file.\n
Pipeline phases in detail - often they consist of more than one job in Jenkins.\n
This is the beginning of a trail of how artifact progresses through the pipeline.\n\ndemo.chat.version is defined in pom.xml as artifact version\nBUILD_NUMBER is built-in Jenkins environment variable.\n\nUpload WAR and trail.\n\nUpload artifacts to BUILD repository - separate repository to catch any errors early (later steps fail to download artifact).\nYou can also use Jenkins to propagate the artifact, but: no outside access and no permanent storage.\n\nTrigger parameterized build - so we can use same build number throughout the pipeline.\n\n
two jobs: deploy-test and automatic-tests\n\nthis and all subsequent jobs have no source - they just operate with Nexus\n\ndownload artifacts from BUILD repository\nWAR and build.txt\n\nLiveRebel Jenkins plugin\npreviously downloaded WAR\noverrides application name and adds build number - to keep different versions separate\nuploads also build.txt\ncustom context path\ntest server\n\ntrigger next job: passes on current parameters (original build number)\n\nautomatic-tests:\nexecute very simple test - typically you would do Selenium tests instead.\nrecord the result in trace\n\nget WAR from BUILD repository\nupload it to TEST repository (with trace file, notice that classifier is different! otherwise local repo caches it forever)\n\n
build pipeline plugin is buggy - environment is not always propagated!\nwe still need the connection, so that the pipeline plugin shows it as one single pipeline\n\nsend e-mail with URL to trigger next build through API to make it easier for casual users (no need to go to Jenkins UI)\ngotcha: e-mail plugin uses a different syntax for referring to environment variables\n\nthe job triggered via API needs to be parameterized\n\n
\n
\n
\n
\n
Information formalized in the pipeline.\n\nAnd, for each build, you know where it has been!\n
\n
Sneak it in by not changing the current process - automate it instead!\n
Sneak it in by not changing the current process - automate it instead!\n
Sneak it in by not changing the current process - automate it instead!\n
Sneak it in by not changing the current process - automate it instead!\n
