XP2015 - DevOps and Continuous Value Delivery with Chocolate and Lego.
1. DevOps and
Continuous Value Delivery
with Chocolate and LEGO
Dana PylayevaThe 16th International
Conference on Agile
Software Development
2. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
A Little Bit About Me…
3. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
@DanaPylayeva
dpylayeva@gmail.com
Agile Coach
Scrum Master
DBA Manager
Systems Architect
Java Developer
Conference Co-Chair
Speaker, Reviewer
…and a Bit More
4. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
13:30 Part 1
Why DevOps and what is it?
Cyclical value delivery (with Scrum) .
Chocolate, LEGO, Scrum simulation game.
Sprint 1: What is the problem with cyclical value delivery?
Sprint 2: Move security to the right!
15:00 Break
15:30 Part 2:
Chocolate, LEGO, Scrum simulation game:
Sprint 3: DevOps transformation
Sprint 4: From Scrum to Continuous Delivery.
Fishbowl retrospective
About This Workshop
5. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
It Will Get Noisy –
Remember the Sign!
http://commons.wikimedia.org/wiki/File:Katy_Perry_with_arm_raised,_by_medigirol.jpg
6. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
DevOps – Popularity On
The Rise
USA
Finland
Google Trends, Search Term “DevOps”
https://www.google.com/trends/explore#q=DevOps&geo=FI%2C%20US&cmpt=geo&tz=
Velocity
Conference
“10 + Deploys
per Day”
DevOps Finland Meetup
Founded January 2013
Worldwide
7. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Expect innovation and speed.
Judge digital service quality on
their overall experience.
Expect service providers to
deliver value continuously.
What’s Driving This Interest?
Customers in 2015…
9. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
DevOps Definitions
“A movement of people who care about developing
and operating reliable, secure, high performance
systems at scale.” -Jez Humble
“A mix of patterns intended to improve collaboration
between development and operations. DevOps
addresses shared goals and incentives as well as
shared processes and tools.” - Michael Hüttermann
10. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Culture - embrace change and innovation
Automation - CI/CD, “infrastructure as code”
Lean - delivering value, minimizing waste, small batch sizes
Measurement - measure everything, show improvements
Sharing - information sharing and collaboration between dev and
operation
CALMS Model
11. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
From Wall of Confusion to
Collaboration Through Empathy
MUST READ MUST READ
12. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Different Sources, Same Idea
Amplify
learning
Accelerate
feedback
loop
13. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Feedback in Scrum…
14. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Delayed Feedback.
What is the cost of this delay?
15. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Zoom Out. What Do You See?
16. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Conflict of Interests
Development
Team:
Focus on
delivery
Operations
Team:
Focus on
Stability
Customers:
Ever-
changing
demands
17. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Chocolate, LEGO, Scrum
Simulation Game
18. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Operations
Team
Scrum
Team
Scrum
Team
Scrum
Team
Business
Business /
Customers
Feedback loop,
Market demand
User
Story
ChocolateLEGOScrum.com
Enterprise
User
Story
User
Story
19. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Role Cards.
20. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Game Characters:
Scrum
Team
Samuel Scrum
Danny
Developer (4)
Tim Tester (2)
Operations
Team
Sara Security
Adam Admin
Robert Release
Benjamin
Business
Harry
Hacker
Patricia
Product
Observer
21. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Animal Stock Exchange Board –
Current Market Demand.
$1000
22. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Development Environment - Sample
23. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Our Software:
LEGO animal – software features
Chocolate – user documentation
User Story
Deployment
package
Five small
individual
packages
with a
LEGO dog
and a
chocolate
24. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
• Type of an animal as per the
user story.
• Each animal has a label with a
number.
• Each package contains:
• One Lego animal (as a
functionality)
• one chocolate candy (as a
support documentation)
• Content doesn’t fall out from a
package.
Sample animals: A lion
and a horse
Definition of Done:
25. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
What Is The Goal of The Game?
26. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Sprint 1: Cyclical value delivery with
Scrum
Sprint 2: Optimizing Scrum team
Sprint 3: DevOps transformation
Sprint 4: Continuous Delivery
DevOps Transformation
in Four Sprints
27. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Limited communication
between Dev and Ops.
Everyone can perform only
his/her role.
Aiming to deliver a “potentially
shippable product”
Number of releases is limited
Z
Sprint 1: Cyclical value delivery with
Scrum
28. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Debrief
29. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Address the bottlenecks
Developers and Testers are
cross-trained.
Security Testing – moved to
beginning of the process.
First release into
production. Can only be
done by release engineer.
Sprint 2: Optimizing the team
30. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Debrief
31. Part 2: DevOps and
Continuous Value Delivery
with Chocolate and LEGO
Dana PylayevaThe 16th International
Conference on Agile
Software Development
32. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
13:30 Part 1
Why DevOps and what it is?
Cyclical value delivery (with Scrum) .
Chocolate, LEGO, Scrum simulation game.
Sprint 1: What is the problem with cyclical value delivery?
Sprint 2: Move security to the right!
15:00 Break
15:30 Part 2:
Chocolate, LEGO, Scrum simulation game:
Sprint 3: DevOps transformation
Sprint 4: From Scrum to Continuous Delivery.
Fishbowl retrospective
About This Workshop
33. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
“The velocity of change in business
requirements is undeniably increasing
at a frightening rate for those
organizations unable to keep pace”
The Seven Habits Of Highly Effective DevOps
by Glenn O’Donnell and Kurt Bittner, Forrester Research, Inc, September 3, 2013
34. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
11.6 sec
Any idea?
2011. Amazon Deployment Stats.
Mean time between deployments
(on weekdays)
Source:
http://assets.en.oreilly.com/1/event/60/Velocity%20Culture%20Presentation.pdf
35. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
2015. No Longer For
Unicorns Only…
Macy’s, Nordstrom,
GE Capital, Target, Disney,
US Department of
Homeland Security,
IBM, Microsoft, Rally
Software,
Barclays Capital,
Capital One,
Fidelity Investments, ADP,
Walmart…
36. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
What Does This Mean?
A Change Again?
37. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
DevOps: Where Do We Start?
38. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Where Do We Start?
39. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
“…ensure fast, predictable and uninterrupted flow of
planned work that delivers value to the business while
minimizing the impact and disruption of unplanned
work, so you can provide stable, predictable and secure
IT service.”
~Gene Kim, Kevin Behr, George Spafford
“The Phoenix Project“
Step 1: Optimize Your Flow
40. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Examine Your Process.
Does It Look Like This?
41. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Improve Flow – Eliminate Waste
(Value Stream Mapping)
42. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Improve Flow - Address Your
Bottlenecks
43. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
1. Identify the system's constraint(s).
2. Decide how to exploit the system's
constraint(s).
3. Subordinate everything else to the above
decision (align the whole system or
organization to support the decision made
above).
4. Elevate the system's constraint(s) (make
other major changes needed to increase the
constraint's capacity).
5. Rinse and Repeat:
Warning! If in the previous steps a constraint
has been broken, go back to step 1, but do not
allow inertia to cause a system's constraint.
Theory of Constraints.
Systems Thinking.
44. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Different Types of Bottlenecks:
Tools People Policy
45. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Continuously Expand Your Skills!
46. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Scrum Team
Dev Ops
Step2: Fast-Track the Feedback Loop
47. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Dev Ops
Scrum Team
Bring Operations In!
48. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Automate Manual Steps
ANYONE CAN DEPLOY
ANYONE CAN BUILD
and PROVISION
ENVIRONMENTS
DevOps
49. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
First steps towards DevOps
Improving the flow:
Building T-shaped skills
Accelerate Feedback –
Automating deployments
Sprint 3: Optimizing the System
50. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Debrief
51. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
What’s Next?
…Our highest priority is to
satisfy the customer
through early and
continuous delivery of
valuable software…
Agile Manifesto
52. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Continuous delivery is a software
development strategy that optimizes
your delivery process to get high-quality,
valuable software delivered as quickly as
possible.
~Jez Humble
Step 3 - Continuous Delivery
53. "Continuous Delivery
process diagram“
by Jez Humble -
http://continuousdeliver
y.com/2010/02/continuo
us-delivery/. Licensed
under CC BY-SA 1.0 via
Wikimedia Commons -
54. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Accelerated time to market
Ability to get fast feedback and validate assumptions
Increased quality
Minimized customer impact (with blue-green
deployments, canary ( incremental) releases, roll
forward approach)
Advantages of CD
55. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Reducing batch sizes.
Simplifying deployment
process
Implementing continuous
delivery of value.
Sprint 4: Continuous Delivery
56. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Fishbowl Retrospective
58. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
1.DevOps is about creating a fast flow
of work through organization.
2.DevOps is about amplified feedback
loop.
3.DevOps is about experimentation,
repetitions and practice.
4.DevOps is about changing the
culture.
If You Only Remember Four Things:
59. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Recommended Reading.
60. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
1. https://www.getchef.com/blog/2010/07/16/what-devops-means-to-me/
https://www.gartner.com/doc/2847717/seven-steps-start-devops-
initiative
http://www.gartner.com/technology/reprints.do?id=1-
2CBV2MS&ct=150326&st=sb#f-d2e168
https://blog.newrelic.com/2014/05/16/devops-name/
http://continuousdelivery.com/
Web Resources:
61. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
http://pixabay.com/en/clock-analog-time-watch-
147257/
http://pixabay.com/en/brick-lego-yellow-159435/
http://pixabay.com/en/blue-brick-lego-159433/
http://pixabay.com/en/horse-unicorn-run-black-
shadow-194999/
http://vectorcharacters.net/
http://pixabay.com/en/magnifying-glass-
magnification-450690/
Photo Credits
62. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
You’ve earned a badge!
Thank you for playing with me today!
63. @DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License@DanaPylayeva This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License
Notas do Editor
Hello everyone and welcome to DevOps and Continuous Value delivery with Chocolate and Lego!
My name is Dana Pylayeva, I am an Agile Coach at Rakuten Marking.
I am glad to be here today and glad to see so many people interested in DevOps!
Let’s get to know you a little bit.
Show of hands. Who is using DevOps practices today?
And who has heard of DevOps, but don’t know where to start?
Anyone here just to play some Lego and eat some chocolate?
All right, looks like we’ve got just a perfect mix of people here!
I am sure everyone of you here will find something to your taste in this workshop!
I little bit about me:
I grew up in Ukraine, went to college in Moscow. I now live in New York and work at Rakuten Marketing and I am thrilled to be here today at XP2015!
I’ve been in IT for the past 16 years, in various role and through my career I never said no to an opportunity.
This is how one day I found myself managing a group of Database Administrations. Just as a typical developer, I had a very little idea about what operations and DBAs in particular actually do. I knew how to write Oracle trigger, create table statement and thought that it shouldn’t be too hard of a job!
Well, two most stressful years of my life. Imagine living on a volcano and being available 24X7X365.
As stressful as it was, that experience helped me develop an understanding and a deep appreciation for folk in operation, triggered my interest in DevOps and inspired this workshop.
This will not be your typical DevOps talk. This is an Agile Simulations Game which uses chocolate, Lego and elements of gamification to visualize some of the typical issues with cyclical value delivery methods (like Scrum), allows you to see the walls of confusion between organizational silos and helps to discover some of the ways to bring DevOps culture and practices into organizations.
If this sounds like your type of workshop, I am glad you are here!
This will be very energetic workshop, and it will get noisy! I am going to teach you the universal sign: If I raise my hand and stop talking, I am asking you do the same. The first person who sees me, will raise the hand and stop talking. and the next one does the same thing.
It is truly amazing to see this in action in a room of over 100 people. You can almost see the silence traveling from one corner of the room to another. He is your first learning for today
DevOps is truly a grass roots movement, initiated and spearheaded by practitioners and enthusiasts. Lets travel the DevOps timeline with Google trends.
Back in 2007 Paul Dubois an IT consultant from Belgium noticed how vastly different the goals and the processes were in development and IT operations.
Frustrated, he started a quest for finding a better way.
Agile 2008 conference in Toronto, he met Andrew Shafer and they started “Agile Systems Administration Group.”
2009 – Velocity Conference. John Allspaw and Paul Hammond from Flickr deliver “10 + deploys per day” presentation sharing the first successful initiative in this space.
Inspired by this – Paul Debois organized a new conference - DevOpsDay2009 and that’s how the term DevOps is born.
If you look at this graph – interest in DevOps in Finland can be attributed to two events – establishment of the “devops Finland” meetup and DevOps Days Helsinki.
I was speaking at Agile Days in Moscow earlier this year, and notice a very similar trend there – little to no interest until 2013, then spike of popularity after 3 IT enthusiasts established “DevOps Moscow in Russian” meetup and podcast “DevOps Deflope”.
#DevOps
Why is there so much interest in DevOps?
If it was only for “making live of Ops better” it wouldn’t gain momentum so fast!
There is so much interest because customers today demand faster speed to market.
Today’s customers expect innovation and speed.
To stay competitive, organizations must respond quickly to changing customers’ needs.
They need to adopt practices and develop a culture that accelerates speed to market.
In spite of this fast growing popularity of DevOps, there isn’t one standard definition. Is it a role, a product, a job title?
There are a lot of different interpretations of what it is. To the extent that Gartner analysts in their 2014 report “Seven steps to start your devops initiative” recommend as a step #1 “Define DevOps for you.”
Wikepedia defines DevOps as a software development methodology emphasizing communication, collaboration and integration between developers and IT operations professionals.
Jez Humble – “Continuous Delivery”
Michael Hüttermann – “DevOps for Developers”
Culture – embrace change and innovation
Automate everything, CI/CD, “infrastructure as code”
Lean – focus on delivering value, minimizing waste, small batch sizes
Measurement – rely on facts not opinions, measure everything, show improvements
Sharing – Open information sharing and collaboration between dev and operation
here are many tools available today that can help automate deployment, server provisioning, environment configuration and monitoring.
However the tools by themselves will not guarantee success of your DevOps initiative. The fundamental ingredient for your success –cultivating DevOps culture.
It is hard to collaborate with someone, when you don’t share the goals and don’t understand where the person is coming from.
The best way to understand IT operations without actually becoming one is to read The Phoenix Project.
When I came across this book, I couldn’t put it down! It was literally describing MY pain and My experience! Great introduction to DevOps and lots of interesting ideas.
One of them is particularly noticeable as I’d like to call out because it is the same idea used in Principles of Product Development Flow and the same idea that’s used by gamification.
Anyone knows what it might be?
idea of feedback, specifically the idea of accelerating feedback loop to amplify learning.
The Feedback idea isn’t new – we have the feedback in Scrum.
I realize that this is XP conference, but based on the Version One survey, Scrum is one of the most popular agile frameworks.
Who is here not familiar with the basic Scrum Framework?
This framework should look familiar..
Working off of a prioritized product backlog, scrum team builds potentially shippable product increments in 2- 4 week sprints and gets a feedback from the customers on the new features they released.
The key here is POTENTIALLY shippable ( and potentially NOT..)
el.
What do you think happens when the code is not getting deployed into production at the end of the Sprint?
Are we getting any feedback from the customers?
What if our competitors start offering the features that we are waiting to deploy?
Potentially outdated features?
All these factors are contributing to the increasing cost of delay in scrum.
The fundamental problem with this picture is that Scrum is only used as a framework for a development process. Any process improvements that Scrum team identifies during their retrospectives are all focused on increasing development team effectiveness and velocity. They don’t really help with the overall flow of work through organization.
This narrow-focused approach creates an illusion of optimization.
Zoom out and you will see that in reality, While Scrum team metrics are getting better, with high team velocity and optimistic burndown chart,
Deployment queue piles up in front of the IT Operations
Every production deployment becomes a heroic effort.
It may cause outages in production and significantly reduce customer satisfaction.
These painful and manual deployments cause the team to avoid them and do them even less frequently.
It becomes very expensive to deploy.
What do we have here? Ever-changing market demand from our customers.
Development team, excited about new technologies, interested in building cool feature and getting them to production quickly. Their slogan is “Think it, build it, ship it, tweak it”
And the Operations team – focused on stability and “Keeping the lights on”
When deployments are rare and manual it is close to impossible to reconcile these goals.
What is the solution and how can we move forward?
Let’s experiment and find out!
For this simulation, we all work in ChocolateLegoScrum.com company. Her is the big picture:
We have multiple scrum team, operations team and the business team.
Each development team tries to deliver maximum value to the business. Our product – Lego Animals with chocolate.
The type of lego animal and the quantity we know from the Animal Stock Exchange. This is where business people express their needs and how much they are willing to pay.
**Product Owner** of each team works with a **Business** to understand the current demand (LEGO animals to build and chocolates to pack).
**Scrum team** works with the **Product Owner** to prioritize the sprint backlog and build and ship them to the **Market**.
Scrum Master helps the teams to stay on track and removes impediments.
Everyone will have a Role card, describing what the role is capable and responsible of doing. Additionally, listing all the dependencies that this role has on other roles in organization.
In the game we have the following characters.
Harry Hacker and observer are two special characters.
Harry is a fate – nobody knows when the Hacker strikes.
Observer is a human monitor – he will help the team measure the process and identify improvement opportunities.
Market demand is regulated at the Animal Stock Exchange.
This is where the animal prices are established and the orders are placed in Sprint 1.
Orders can be placed in the increments of the batch sizes. Based on a quality and a quantity of products delivered in Sprint 1, the market demand will be adjusted for Sprint 2.
In the first sprint, each order from the market (for example, “5 dogs” or “10 giraffes”) is considered as one story.
To generate business value, the story must be delivered to the market in its entirety. Scrum Teams need to be careful to not over produce as business will not purchase the items above desired quantity indicated.
Team will work with the Product Owner to determine which story has highest value for the market and prioritize what to work on for the next sprint.
-------------------
In Sprint 2, 3 the animal prices will fluctuate during the sprints, hence providing an incentive to the teams to deliver products to the market faster.
Any quantity delivered by a team over the desired quantity number is counted as negative ( company will not be able to sell it = waste)
Team will work with the Product Owner to determine which story has highest value for the market and prioritize what to work on for the next sprint.
In spite of the eagerness to start building, Development team can’t move forward until it has a development environment.
Players are asked to stay within the boundaries of their roles (based on the individual roles they got at the beginning of the game).
This approach creates some constraints in the process as **Scrum team** relies heavily on Operations; only **System Administrator** can build or patch the environments, only **Release Engineer** can deploy etc.
Every Lego animal must have a security label attached to it.
Every package needs to include a chocolate. An animal represents functionality, whereas chocolate represents user documentation.
The LEGO and chocolates reflect knowledge work and not just a physical task: LEGO animal represents a feature and chocolates represent end-user documentation.
Business will have a set of play money to be used for “payment” for accepted animals.
###Sprint 1
In the first sprint, each order from the market (for example, “5 dogs” or “10 giraffes”) is considered as one story.
To generate business value, the story must be delivered to the market in its entirety.
Team will work with the Product Owner to determine which story has highest value for the market and prioritize what to work on for the next sprint.
**Operations team** is a functional silo team in Sprint 1.
Ask your **Security Engineer** to pick three random numbers between 1 – 20.
These will be security bugs. When Scrum team is ready to deliver products into production, **Security Engineer** will “run the security scan”. If a product has a label with one of the three selected numbers, it will need to be returned back to a development team.
**Release Engineer** will need to package features delivered into a deployment package and deploy them into production (place the small packages into a ZipLock back, write team name with a sprint number and deliver it to market analyst's table)
At the end of the Sprint 1, participants will discover that deployments are not allowed until the next deployment window. Everything that scrum teams built is now stuck in the queue in Operations department and no value gets delivered to the market.
In the Sprint 1 retrospective, **Scrum teams** will look at optimizations they can apply to their current process and the constraints that impede their productivity.
###Potential improvements
1. Invite **Sara Security** into the Scrum team.
2. Solicit an early feedback from the **Market**
How was your experience?
Scrum Teams, how many user stories you delivered?
This is great! Why are the customers unhappy?
Did you get any product delivered to you?
Observers, would you like to share you observations?
It took a long time to start – getting the stories, building environments…
Then security testing at the end caused delays and rework.
Then what happen – team built potentially shippable products, but wasn’t able to deploy due to the code freeze.
They are getting no feedback.
Why do you think there IS a code freeze? This is a function of deployment process being manual, brittle and potentially causing service disruption.
With the Water-Scrum-fall it gets expensive to deliver changes and making them live in production.
One of the advantages of embracing DevOps , is that you are able to re-define that!
Ok, good stuff! Let’s continue, this time around
How was your experience?
What was different in Sprint 1 and Sprint 2?
Observers, what were your observations?
What about the first deployment? Robert release was working really hard to get all the stuff into production!
Welcome to part 2 of this workshop,
Firstly, let’s make sure we have complete teams.
Anyone is missing team members? Do we need to hire more people?
This will not be your typical DevOps talk. This is an Agile Simulations Game which uses chocolate, Lego and elements of gamification to visualize some of the typical issues with cyclical value delivery methods (like Scrum), allows you to see the walls of confusion between organizational silos and helps to discover some of the ways to bring DevOps culture and practices into organizations.
If this sounds like your type of workshop, I am glad you are here!
Think about it for a moment.
2009 – 10 deploys per day at Flickr
2011 – 11.6 sec over 7 K deployments per day
2015 – Information from DevOps Enterprise summit. – all these companies are implementing Devops practices now and accelerating
Accelerate delivery of services.
Fast-track the feedback loop.
Minimize customer impact during deployments.
All right, where do we start?
How do you know what to change first?
To answer that question, I’d like to quote “The Phoenix project”
The good place to start would be from optimizing the flow of work.
Start with visualizing your flow. Think about all the places were you may have non-value added work.
****
Wastes: TIM WOODS
T – Transport – Moving people, products & informationI – Inventory – Storing parts, pieces, documentation ahead of requirementsM – Motion – Bending, turning, reaching, lifting
W – Waiting – For parts, information, instructions, equipmentO – Over production – Making more than is IMMEDIATELY requiredO – Over processing – Tighter tolerances or higher grade materials than are necessaryD – Defects – Rework, scrap, incorrect documentationS – Skills – Under utilizing capabilities, delegating tasks with inadequate training
Good way to visualize your process is by creating a value stream map. This map outlines flow and activity in your process, broken down by each step and whether the step is value added or non-value added activities (waits, defects).
Another approach can be focused on identifying and addressing your bottlenecks.
Start with stepping back and taking a look at your overall organization to find your bottlenecks.
Maximize Throughput while Minimizing Inventory and Operating Expense
Once you found your biggest bottleneck, focus on making major changes to improve it’s capacity.
Any improvement done before or after constrain will not have any impact on the overall system performance.
We’ve seen it in the sprint 2. When we added Sara security to the Scrum team, the team was able to build more products.
But it didn’t help to deliver more products to the customers as our major bottleneck was in the deployment process.
Be aware of different types of constraints:
Tool: The way existing tools are used and/or lack of appropriate tools may limit the ability of the system to produce more.
People: Lack of skilled people limits the system. Mental models held by people can cause behavior that becomes a constraint.
Policy: A written or unwritten policy prevents the system from making more.
What this means is that everyone of us can become a bottleneck.
Ken Rubin in his book “Essential Scrum” speaks about I person and T person.
I – someone with deep but very narrow knowledge in one area.
T – a person with deep skills in one area and wide range of exposure to other areas.
To make DevOps practices successful in your company, these T-shaped people need to be in majority.
Did you notice in the second sprint, that bringing Sara Security into the team reduced the need for re-work? By providing receiving timely feedback on security issues, team was able to accelerate development.
Take it a step further and start involving Operations in your Scrum Team. Operations know how you application performs in production. They have access to various monitoring tools. They can also share with you how easy ( or difficult) it is to deploy your application.
This is exactly the type of feedback you need to design and build you application for scalability and performance!
This is also a great opportunity for cross-training and experimenting together on automation.
Ultimate feedback that we are looking for is the feedback from our end-users. Manual and infrequent deployments significantly reduce our opportunity to receive this feedback.
Important step in your devops initiative will be environments standardization and automated provisioning.
The idea is making things self-service so that everyone in the Scrum Team can build an environment, provision and deploy.
Let’s go back to our game and this time start taking first steps towards DevOps:1. Improve the flow and accelerate the feedback loop.
Start learning complimentary skills: on your tables you will find little labels – these are your current skills.
If you are in Scrum team, find someone from Ops, if you are in Ops, find someone from development.
Show them your card, tell them what you do in the game and exchange the labels.
In this sprint, you are no longer required to work with your role only. You can help out where needed to improve the flow of work.
Let’s get started!Get the new user stories from the business, plan on what you are going to deliver.
Remember, this sprint everyone can build a deployment package and everyone can deploy at the end of the sprint as we automated our deployments.
Surprise element: new security vulnerability has been discovered and all the environment must be rebuilt! Everyone has to stop working until that is completed. Good news – this process is now automated and self-service – anyone can build a new environment!
##Retrospectives
How was your experience?
What was different from the previous Sprints? What were some of the process changes that you’ve made in your team?
Observers, what were your observations?
How was the deployment now?
Rather than coming up with a bunch of features and planning a multi-month release, come up with new ideas continually and try them out individually on users.
With enough thought, even big features or large-scale changes can be implemented as a series of smaller steps to get faster feedback, with the ability to change course or stop at any point if your idea is found wanting. With a cross-functional team working to deliver these small increments in hours or days, you can be more innovative than your competition and maximize your return on investment
Key concept in Continuous Delivery is the delivery pipeline. You will notice continuous feedback loop through various stages of the pipeline.
The ideas is that delivery team will get automated feedback on the production readiness of their code every time there is a change ( to the code, infrastructure or configuration).
CD relies heavily on extensive automated testing, continuous integration and configuration management.
Requires a change in your architecture, your process.
Smaller-size user stories, “one piece flow”, feature toggling. Feature toggling – is a an architecture and design approach that allows you to switch a feature on/of when it is already in production. One of the common use of this is to make it available to a subset of target customers for A/B testing.
ability to change course or stop at any point if your idea didn’t pass the a/b testing.
. With a cross-functional team working to deliver these small increments in hours or days, you can be more innovative than your competition and maximize your return on investment
Now will get to the most interesting part, where our company implemented continuous delivery.
We will start from breaking these large user stories into an individual stories, enabling one piece flow
We will also simplify the deployment process and decouple sprint as a planning unit from a release as a deployment unit.
Now, once you are done with the feature, tests are good – don’t wait – deploy it to production!
Everyone is familiar with the this format?
5 chair, 4 people
Only people in the chairs can talk. Even if I want to add something, I need to get on the chair.
One chair is always available as an invitation to join the conversation.
One someone new joins the conversation, whoever was talking the most leave the fishbowl.
DevOps is about creating fast flow of work.
DevOps is about amplified feedback loop.
DevOps is about experimentation, repetitions and practice.
DevOps is about changing the culture.
There are many tools available today that can help automate deployment, server provisioning, environment configuration and monitoring.
However the tools by themselves will not guarantee success of your DevOps initiative. The fundamental ingredient for your success –cultivating DevOps culture.