Every DevOps implementation is unique, but certain challenges are the same. Establishing fast feedback loops, formulating a better definition of done, and anticipating technical gaps to delivery are all milestones to transformation.
In this session, our panel of experts will share their experiences in DevOps transformation. Learn how to:
-Improve your team’s definition of done to cut down rework
-Identify automation gaps that lead to later holding patterns
-Inject fast feedback to catch bugs as early as possible
-Establish autonomy and ownership over winning customer-focused experiences
08448380779 Call Girls In Friends Colony Women Seeking Men
Four Keys to Efficient DevOps
1. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
4 Keys to Efficient DevOps
Patterns to Enable Transformation in Enterprise Development
Web: perfectomobile.com Twitter: @perfectomobile
2. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
● Session is being recorded
● Presentation will be shared
● Questions and conversation
○ Twitter: #DevOpsMilestones
○ WebEx chat panel
○ Q&A at the end of the webinar
Housekeeping
3. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Today’s Panel
Carlo Cadet
Director, Product Marketing &
Evangelism
Perfecto
Tzvika Shahaf
Director, Product Management
Perfecto
Sean Williams
Principal Software Engineer
Prism Tech Studios
James Brown
Director, Enterprise Solutions
VisionIT
Raj Subramanian
Senior UI Automation Engineer
CSG International
4. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
What We’ll Cover
1. Improving your team’s Definition of Done
2. Addressing automation gaps early
3. Fast feedback in your pipelines
4. Autonomy for Customer-focused Experiences
5. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Poll: Where are you on your DevOps Journey?
● We fully embrace DevOps across our organization
● Many teams have re-organized and built efficient tool chains
● There are a few pilots underway to implement DevOps
● We talk about DevOps a lot, but haven’t quite started
● Not going there
6. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Improving Your Team’s Definition of Done
4 Milestones on the Path to Efficient DevOps
Patterns to Enable Transformation in Enterprise Development
7. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Predictable flow
● Clock speed mismatch
○ Business needs to innovate
○ Development needs to deliver
● Delivering predictably
○ Establish cadence that business can rely on
● The “real” flow of our teams
○ Leftover work impacts next sprint’s bandwidth
○ Confidence in go / no-go
Waterfall
(months)
Agile
(weeks)
DevOps
(days)
8. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
What goes into a Definition of Done?
Constructing:
● Common criteria
● Previous RCA on re-work
● Go/no-go blockers
Improving:
● Retros: rejected work
● Minimize tech debt
● Better DoD fit per story
https://www.scrumalliance.org/community/articles/2008/september/definition-of-done-a-reference
9. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Examples of tailoring Definition of Done
● Code produced (all ‘to do’ items in code
completed)
● Built without errors
● Unit tests written and passed
● Peer reviewed and corrected
● Deployed to system test environment and
passed system tests
● Passed UAT (User Acceptance Testing)
● Any build/deployment/configuration
changes documented and communicated
Perfecto: ‘Done is DONE:
(partial list)
● Builds without warnings/lint?
● Automated tests
● Feature flags
● Peer code review
● Acceptance / UI review
● Feature documentation
● Code samples
DoD
~ AC
=Story
Design & code
Code pushed / JIRA updated
10. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
11. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Addressing Automation Gaps Early
4 Milestones on the Path to Efficient DevOps
Patterns to Enable Transformation in Enterprise Development
12. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Areas of Automation “gaps”
● CI
● Device/Simulators
● Maintenance
● Test Coverage
● Performance
From the
Digital Quality Handbook
(pg. 70):
● Android 6: Fingerprint
● Android 7: Split screen
● iOS 9: 3D Touch
● iOS 10: Voice commands
bit.ly/digqual
13. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Why to find them early?
The Key is...
● Quick Feedback
● Better Visibility
● Building “Trust” for the automated tests
14. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
How to find them early?
● Plan automation as a team
○ What areas need to be automated?
○ What areas are more stable?
○ Cost vs Value
● Treat automation as separate stories/requirements
● Do estimation with “buffer time” in mind
15. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Fast Feedback in Your Pipelines
4 Milestones on the Path to Efficient DevOps
Patterns to Enable Transformation in Enterprise Development
16. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Enhanced Quality Visibility Throughout Your Pipeline
17. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Meet Velocity Release Goals
Increase efficiency & Optimize UX
Focus more on feature development
and validation, less on analysis
Empower continuous quality while
eliminating flaky tests for faster
cycles
18. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Tools & processes must deliver value
19. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Shortening the Feedback Loop cycle in CI
20. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
My Build failed – but why?
21. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
My Build failed – but why?
22. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Autonomy for Customer-focused Experiences
4 Milestones on the Path to Efficient DevOps
Patterns to Enable Transformation in Enterprise Development
23. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Autonomy
Process
Autonomy
Cultural
Autonomy
24. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Process Autonomy
Automated Builds
and Deployments
Dev,QA and Staging
Environments
Documentation
… and more
25. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Cultural Autonomy
Empower your team to do what’s best for the end user
● Share end user data
● Share business goals and priorities
● Nurture an owner mindset
26. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Takeaway:
The definition of done is a fundamental communication of success, both within
the DevOps team and out to the larger enterprise.
Build “Trust” and “Respect” the automated tests
Necessity of quality visibility across the entire pipeline to support timely
decision making
Teams are more motivated and make better decisions when development
teams stay close to the real customer experience
27. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
What We’ve Covered
1. Improving your team’s Definition of Done
2. Addressing automation gaps early
3. Fast feedback in your pipelines
4. Autonomy for Customer-focused Experiences
28. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Q&A
(open ended question about “what have you seen works?”)
29. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Additional Resources
● List of links from the group (1 per)
● The Path to Releasing Confidently in DevOps:
http://info.perfectomobile.com/release-confidently-in-devops.html
● Peopleware: Productive Projects and Teams:
https://www.amazon.com/Peopleware-Productive-Projects-Teams-
3rd/dp/0321934113/
30. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Appendix
31. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Abstract
Title: 4 Milestones on the Path to Efficient DevOps
How many new initiatives does it take to transform an organization? Answer: as many as it takes to drive change.
Every DevOps implementation is unique but certain challenges are the same. In our experience with enterprises, establishing fast feedback loops
over your code commits, formulating a better definition of done on features, anticipating technical gaps to delivery, and fostering a culture of
autonomy are milestones to transformation.
In this session, Perfecto Developer Advocate Paul Bruce will host a dialog with a panel of development experts, sharing their experiences in DevOps
transformation. We will discuss:
· Improving your team’s definition of done to cut down rework
· Identify automation gaps in planning that lead to later holding patterns
· Inject fast feedback throughout your pipelines to catch bugs as early as possible
· Establish autonomy and ownership over winning customer-focused experiences
32. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
What you don’t know...hurts.
Examples of automation “gaps”...
● Existing functionality
○ Appium on iOS 10
○ Low platform coverage in CI
● New features
○ Hardware dependencies
(camera, location, BLE)
○ Test coverage in DoD
[Panelist attests, offer
example]
From the
Digital Quality Handbook
(pg. 70):
● Android 6: Fingerprint
● Android 7: Split screen
● iOS 9: 3D Touch
● iOS 10: Voice commands
bit.ly/digqual
33. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
Who’s in the planning room?
● Idea in backlog (PM / PO)
● Prioritization / grooming (PO)
● Defining (PO / dev)
● Scoping (dev / test / ops)
● Estimation (dev / test / ops)
● Signoff (all)
[panelist example of who’s in planning sessions]
34. 4 Milestones on the Path to Efficient DevOps Web: perfectomobile.com Twitter: @perfectomobile
A tale of two features [panelist]
● Feature 1: didn’t have the right people in the room
○ Subsequently, later automation was delayed due to technical gap
○ Impact on business was x
● Feature 2: all members in planning (DevOps)
○ Planning included domain knowledge of all delivery aspects
○ Potential gaps in automation were identified early
○ Estimation included this work
○ Delivery dates were met
Notas do Editor
Sean: other “steak”-holders unrelated to the end user, DoD -> tailoring to teams ⇐ no longer a fan of his own question :-)James: what’s the most challenging aspects of improving developmentRaj: what does “automation” mean to you? Gaps come from diff. mindsets
Why is this a thing? Perspective Developers focus on the code QA focuses on the testing Product Manager focuses on product Communication is Fundamental to Success History of software development Requirements System Requirement Specification Use Cases User Stories DevOps Complexity grows exponentially with team size Mutual Expectations Change is Expected Teams Stakeholders Vision Product Roadmap Politics
Points Raj can contribute to this slide
-Often Testers are often put in a situation to give go/no-go which is not right. Testers provide relevant information about the product and then the business needs to make that decision based on the information. Testers are “Information brokers” not “Gate Keepers”
Definition of Definition of Done A clear, consistent yardstick yielding a binary answer Simple checklist Between 3 and 10 items Lightweight and FocusedHow Context Specific Team Driven Tailor based on context Dimensions of Done SDLC Development Testing Unit Functional Enterprise Project Management Quality Gates Architectural Review System Integration Deployment Product Shippable Customer/Client Dependencies
Raj’s points
-Have a checklist for Definition of Done and make it visible to the team
-The story is complete only when all the points in the checklist are completed
Sean’s example of DoD:
At one company I was at, we had a very light product department relative to developers so getting full requirements was always a challenge. We ended up settling on Acceptance Criteria as the main definition of done. This was a blend of user stories and testing behavior. Here’s an example for adding a Forgot Password link to login:
Go to the Log In page. There should be a “Forgot Password” link.
Click on the “Forgot Password” link. You should go to a “Forgot Password” page, and it should match the attached mockup.
Enter an invalid email. The form should show an error message (specified elsewhere in the requirements)
Enter an email of a user that doesn’t exist in the system. The form should show an error message.
And so on...
Challenges Agreement on the definition Who Decides Buy in
CI
· Frequency - Want the automated tests to run each time a check in happens
· Visibility - No one knows what is going on and when test fails/pass
· Reporting
· Identifying/eliminating flaky tests
Device/Simulator/Emulator
· Where do you want to run this on device or simulators?
· On Simulators difficult to tests
o Wi-Fi/4g/Airplane mode switching (Simulator and emulators cannot tap into Airplane mode)
Maintenance
· Who maintains the automated tests?
· Who maintains the server machines running the automated tests? What if you run out of space or want to run multiple instances of your tests
o People are going to cloud testing for this exact same reason
Test coverage
· Have 2 sets of tests
o Smoke tests – Most Stable and only high level functionalities and runs quickly
o Regression tests/suites – Run during regression testing phase or periodically based on requirement
Performance
· How fast does your tests run? Does it take 5 mins, 30 mins, 1 day?
o People want quick feedback on this
Raj talks about
Need quick feedback if something fails/broken. Find defects “Early”
Need to keep everyone involved in automation effort as it is a “Whole Team” approach not just the automation team . Make it visible to everyone through e-mails, TV boards, stand ups etc
Make people respect the automated tests. If it fails, whoever broke if fixes it. It is not only the automation team members who keep fixing it. Have stable tests and remove flaky tests. Increase your and the automated tests credibility
Sean can comment on:
As a developer, I can attest to trust for automated tests being key. If you have a test that is failing sporadically for a reason other than broken functionality, disable the test until it can be fixed. You don’t want get developers used to seeing tests fail because if they see too many failing and they are not responsible for the failures, developers get the habit of being skeptical towards test failures - or even flat-out ignoring them.
James:
Automation empowers the team to find more complicated or subtle defects and perform more robust edge case testing
The early identification of defects significantly reduces cost to remediate
Raj talks about
-How automation gaps are addressed during planning meetings
-Who is there in the planning meetings - Whole Team includes PO/Solution architect/Tech Doc writer/Dev’s/QA/s
-Examples on this slide
--Through discussion how we identified not to automate “Guide Story” as there was another story which included complete redesign of the guide - Devs pointed it out and thus, it helps to plan automation as with the whole team. Saved a lot of time for automation testers
--Automation needs considerable work/coding. Treat it a separate stories. Have “kick off” meetings on expectations/acceptance criteria, then have development, Code review, Testing, Demo and then Complete just like a normal development story
--”Buffer Time” - Under-stimated different data setup needed for different types of products - Movies with different captions, Movies with different audio lang, Single Episode, mulit episode, rental products for one of the stories. Learnt a valuable lesson of under estimating.
We use a lot of industry standard tools and cloud services here at Perfecto. Everyone has their tools,
The goal of autonomy is to let people and processes fly with as little interference as possible
Autonomy = end-to-end ownership of how to accomplish the goals that leadership sets
Ever worked as the sole developer on a project or at a brand new company? Development goes very fast compared to larger companies. Part of that is because there are no existing systems, but part of it is that a lot of other stuff becomes involved outside of code.
Two types of autonomy:
Process autonomy (automation): construction of systems that minimize the time and error associated with repeated delivery; enables faster work
Cultural autonomy: The business identifies goals, the team figures out how to accomplish them. Aligns team effort to delivering value through a great customer/end-user experience.
Construction of systems that minimize the error associated with repeatable work.
Minimize error, enable faster work.
There are many things that fall into this, and you can fall anywhere on the range:
Automated Builds and Deployments - Maybe you don’t have anything. Maybe you have automated builds but they need to be manually deployed, etc.
Environments: maybe there’s no instructions for developers on how to create a local development environment. Maybe you have a staging environment that simulates production.
Documentation: if you were new, how much information can you find out without having to ask someone who already knows it? And is it easy enough to get to that information?
There are other examples: Automated UI tests, Unit tests
What you need is very custom to your team and project:
If you have a big team, just a small change in some of these can save significant time
If you’re a small team, you may not need these
Prioritize based on your team and tackle one thing at a time. To give an example, I have a project right now that is a small application that only needs to be up for three hours for a live event. Everything is manual - I upload files to the server when I change them, I manually SSH into the server and restart it. Doing automated builds/deployments there would be a waste of my time.
Developers probably don’t use the product, or if they do, they don’t with the same mindset as an end user.
It’s a culture thing, so must be slowly implemented over time
Share end-user data with developers:
email campaign results, A/B testing results, interesting GA data
At one startup I was at in the entertainment industry, the analytics team sent out a bi-weekly report with the stats relevant to each dev team and an explanation of any changes. A/B testing results were available on the internal wiki.
Let developers sit in on end user interaction: feedback sessions, focus groups, demos
Product can share their notes from these sessions
Share business objectives and priorities with developers:
High-level business priorities (ex, quarterly goals)
KPIs of features after rollout (ex, is conversion up after changing our contact form design?)
Be aware of non-end user stakeholders:
Investors
External people w/purchasing power for your product
At another startup in the energy sector, I’d sit in on demos to hear what people commented on. At one SaaS company, we had a feature that was never used but was in the product because people with purchasing power liked to see that it was there on demo.