Big Visible Testing (Full Length)
•
2 likes•4,114 views
This is the story of how I let go of being the lone tester and became the testing teacher and coach for my team. I have always thought of myself as an agile tester. However, over the past couple of years, I have transformed from a pseudo-agile tester to a true agile tester. To improve collaboration with the rest of my product delivery team, I have been gradually increasing the visibility of testing activities through exploratory test charter management, defect backlog organization, and paired exploratory testing with both testers and non-testers. The feedback loops have shortened and the abbreviated time between activities necessitated adjusting how I provide information. Now that the audience for my testing comprises a mix of disciplines and the work environment has shifted from a heavier process to transparent, quick information access, I have been experimenting with different ways to execute testing and to represent the outcomes of that testing activity so that the information consumers understand it in ways that best suit each of their perspectives. We will examine distinct agile team member personas and their implications for presenting and maintaining testing information as well as the inherent tensions between their distinct and various needs. I will trace my learning curve of adjusting to their needs through the various experiments I have completed in this context, though these lessons extend beyond a purely cross-functional agile product development team. Attendees will come away with a fresh perspective on viewing their product team members and focus on the value testing artifacts provide to a software development team as well as answers to these questions - What value can testing bring to an agile software product team? - How can we focus on outcomes over output, both as an agile software product team and as tester contributors? - How do you keep the work sustainable? - How do we maintain effective communication? - What does it mean to test in agile environments?
Recommended
More Related Content
More from Claire Moss
Recently uploaded
Recently uploaded (20)
Big Visible Testing (Full Length)
- 2. @aclairefication #BVT #Agile2013 Tester Developer Product Manager User Experience Designer Lead Developer
- 3. @aclairefication #BVT #Agile2013 Tester Developer Product Manager User Experience Designer Lead Developer
- 4. @aclairefication #BVT #Agile2013 Tester Developer Product Manager User Experience Designer Lead Developer Agile Manifesto!
- 5. @aclairefication #BVT #Agile2013 User Experience Designer Lead Developer Tester Developer Product Manager
- 6. @aclairefication #BVT #Agile2013 User Experience Designer Lead Developer Tester Developer Product Manager
- 7. @aclairefication #BVT #Agile2013 Tester DeveloperProduct Owner Team User Experience Designer Lead Developer Product Manager
- 8. @aclairefication #BVT User Story Grooming, Planning TDD Coding Unit Testing Demo Executing Testing Writing Tests Pairing w/Dev Grooming, Planning Demo Pairing w/Tester #Agile2013
- 9. @aclairefication #BVT #Agile2013 End-User Personas
- 10. @aclairefication #BVT #Agile2013 Product Team Personas
- 11. Who are my customers? @aclairefication #BVT #Agile2013 Product Manager Programmer/ Developer User Experience (UX) Designer
- 12. What do they want? @aclairefication #BVT #Agile2013 Definition of Done
- 13. Bug Board @aclairefication #BVT #Agile2013
- 14. Bug Board after Product Owner Team @aclairefication #BVT #Agile2013
- 17. Ben the Business Advocate @aclairefication #BVT #Agile2013 How quickly can we provide value? ROI Of Testing Can We Ship? Subject Matter Expert Impact On The Bottom Line? Average Computer User Ease Of Learning Severity
- 18. Bug Board - Severity @aclairefication #BVT #Agile2013
- 19. Bug Board - Severity @aclairefication #BVT #Agile2013
- 20. @aclairefication #BVT #Agile2013 Show Me Clusters Of Bugs I Want To Try It Call To Action Visual Moderate Computer User Ease of Learning Accuracy Ursula the User Advocate
- 21. Bug Board – Site Map @aclairefication #BVT #Agile2013
- 22. Bug Board – Site Map @aclairefication #BVT #Agile2013
- 23. @aclairefication #BVT #Agile2013 Todd the Technology Advocate Does It Work? Principle Doer How Bad Is It? Translate To Action What’s Next? Expert Computer User Can We Sustain It? Accuracy
- 24. Sprint Board – Bug Lane @aclairefication #BVT #Agile2013
- 25. Sprint Board – Bugs in Stories @aclairefication #BVT #Agile2013
- 26. Sprint Board – Bugs & ET Charters @aclairefication #BVT #Agile2013
- 27. Special thanks to: - my UX folks Will & Astrid & Emily who taught me all about personas - my employer Oracle for sending me to Agile 2013 - my friends whose photos I used for personas (“whatever those are” ) Find Claire Moss online: Twitter: @aclairefication claire@aclairefication.com @aclairefication #BVT #Agile2013
Editor's Notes
- Hi, I'm Claire Moss and I'm here to talk to you about Big Visible Testing. For those of you who like to tweet, you can see my Twitter handle @aclairefication here in the header and my talk's hash tag #BVT along with the conference hash tag #Agile2013.I have always thought of myself as an agile tester - hence my talk's riff on the agile Big Visible Charts concept. However, over the course of a year, I have been transforming into a truly agile tester working as part of a Scrum team to build a nearly green field product.
- The other members of my product delivery team are from a mix of disciplines but do not have a background in software testing as I know it. This audience for my testing needs transparent, quick information access in a form that best suits their distinct perspectives. It has been a learning curve for all of us and I hope that you benefit from our experience. I hope you will leave this presentation with a fresh perspective on your product team members so that you can focus on the value of the testing artifacts you produce.
- One of the Scrum artifacts that we produced for our product team is the Definition of Done. As you can see, we define the results we want for each story with an emphasis on quality, so the testing information needs to support this value. You may notice that the product team isn't interested in the overall picture of the testing process in the way my testing manager is. They want more short-term feedback from testing so that they can do something about it.
- My first attempt to educate my teammates about testing was to create a low-tech defect dashboard, or as I like to call it the bug board. I started with defects because my teammates already had some understanding of the relationship between software development and that form of testing feedback. I wanted to meet them where they were and to collaborate on the testing reporting they needed to schedule, design, and implement stories.
- Imagine my delight the morning I arrived at work to find my bug board rearranged: it had been the focus of fruitful discussion among the members of my Product Owner team, that is the folks making the decisions about what to do.
- As our pilot release progressed, we had deeper discussions about how testing serves the product team and I gained insight into their various disciplines, ways of thinking, and preferred ways to communicate. After we reflected on what we learned, I resolved to apply this to future experiments.
- We iterated and I came to see my teammates as quite distinct customers of my work. I began to think of them in the way they think of their customers, applying the lessons they taught me to them(selves?). And so I developed user personas corresponding to what value each of them emphasized. Since these personas are quite detailed, I will only be able to hit the highlights in this presentation but will be glad to discuss more afterward.I began by focusing on the goals and greatest concerns for each of these roles. Since user personas can be proposed ad hoc and evaluated through research, I began with a guess based on my past experience and tested that against my current context.
- [READ ABOUT BEN THE BUSINESS ADVOCATE]
- For Ben, I chunked bugs by Severity and discussed that with him, aiming for ease of reference.
- We revised the representation so that he could efficiently gather information and at a glance know when to follow up with questions to elicit well-rounded information for making business decisions. He wanted to know whether to address defects and when we needed to complete them. When our critical and highest severity buckets were empty, he thinks, "Let's get a return on our investment. Ship it!“Although Ben is the CEO of the product, his is not the only perspective to consider. There is an inherent tension between this business motivation and what drives my second persona Ursula.
- [SHOW URSULA]She balances the team by bringing a sharp focus on the users, whom she knows deeply through broad customer contact, whether through usability testing or contextual inquiry. She sees the system through their eyes - and frequently prefers a visual representation of the impact on the user.[READ ABOUT URSULA THE USER ADVOCATE]
- For Ursula, I organized bugs around a site map I built for our web application. Ursula is a "doer" since she designs the user experience. In Scrum terms, she is a pig. In UX terms, she is a primary persona: I want to optimize my quality information for her use in problem resolution.Since Ursula is also a judicious perfectionist with a focus on consistency, she and I have a great relationship and collaborate closely. I use her designs and user research to feed my test design and she uses my granular test output in the larger context of the product.
- Ursula consumes defects more in clusters than singly, so we revised the bug board accordingly. I can enhance the bug board even more by focusing on efficient translation to her action. Thus describing problems as a scenario emphasizes when and how bugs impact a user's day-to-day life with the software so she can iterate on the user experience, which encompasses more than just graphic design. In turn, I can provide feedback and critique mockups or prototypes, asking helpful questions.
- While everyone on the team is conversant with software from our own usage, no one has as deep an understanding as Todd the Technologist - sometimes he has to explain his recursion jokes for the rest of us.[READ ABOUT TODD THE TECHNOLOGIST]As the principle "doer" on the team, he is my highest priority customer. Although Ursula has the chops to get into the user interface code, Todd is the one to keep things running smoothly behind the scenes. Test-driven development and automated checks are his first line of defense again bugs, so he may respond with puzzlement to the presence of a bug backlog. Let's translate it for him.
- First, we tried scheduling a time box for bug fixing, our bug bucket, to attack our most urgent concerns since Todd cares about what is the highest priority work so he can attack it. However, Todd is not interested only in an expedient solution. He wants to keep the product sustainable, so he may report issues for the backlog that incorporate technical debt, which may relate to clustering of defects.
- Ursula isn't the only one who likes a good user scenario, so we shifted to relating bugs to stories scheduled for sprints. After all, Todd needs to know the whole picture so he can incorporate refactoring and bug fixing in the estimate along with the feature enhancement that together produce value for the user. He also likes preventative measures such as a scripted smoke test that covers a story's acceptance criteria - just another way for him to know that a story is done.
- That brings us full circle to Ben who makes the ultimate judgment on whether the story is done when he runs the application against the acceptance criteria. Although Ben may be a secondary persona for me - one I want to accommodate but not as directly as my primaries - he benefits from all the testing information along the way since he has confidence when he accepts a story.Our current sprint board informs Ben about the status of our sprint's story progress, including defect discovery.Now that the whole team understands defects better, we also have begun to incorporate exploratory testing charters in our stories so that everyone can see the progress of testing over time and what charters uncover the defects. I have been able to involve everyone on the team in exploratory testing to some degree and our product's users will reap the benefit of the common understanding. After all, while I may be in the information business as a tester, it is all ultimately in service of our users.
- I want to thank Will & Astrid for teaching me all about user personas and the role that UX plays in software development, but I especially want to thank my employer Oracle for sending me to Agile2013 so we can confer today. And thank you my audience for your time.Any questions from those present or anyone following along at home?