Informed & Agile: Test Driven Design w/ Jon Innes
•Transferir como PPTX, PDF•
7 gostaram•1,345 visualizações
The webinar discussed adapting user experience (UX) testing methods for agile development processes. It covered lean UX and minimum viable products (MVP), comparing agile and lean approaches. The webinar also provided techniques for incorporating user research and testing into sprints, including remote automated usability testing, 3x3 testing of concepts, and A/B testing of design variations. Attendees learned tips for defining hypotheses, metrics, and conducting iterative user tests to guide agile development using an MVP approach.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (15)
Semelhante a Informed & Agile: Test Driven Design w/ Jon Innes
Semelhante a Informed & Agile: Test Driven Design w/ Jon Innes (20)
Mais de UserZoom
Mais de UserZoom (20)
Último
Último (20)
Informed & Agile: Test Driven Design w/ Jon Innes
- 1. Informed & Agile Test Driven Design with Jon Innes
- 2. Quick Housekeeping • Control panel on the side of your screen if you have any comments during the presentation • Time at the end for Q&A • Today’s webinar will be recorded for future viewing • All attendees will receive a copy of the slides/recording • Continue the discussion using #uzwebinar
- 3. Meet the Speakers Jon Innes Founding Principal UX Innovation LLC Robin Richardson VP of Marketing UserZoom
- 4. 4 UX INNOVATION LLC Overview • The often misunderstood MVP • Agile vs. Lean UX—what’s the difference? • User data, the missing ingredient in Agile • Test-driven design—data beats opinions • Practical tactics and lessons learned • Q&A #UZWEBINAR
- 5. 5 UX INNOVATION LLC Crappy Product vs. Minimum Viable Product What’s the difference? #UZWEBINAR
- 6. 6 UX INNOVATION LLC The often misunderstood MVP #UZWEBINAR
- 7. 7 UX INNOVATION LLC Well executed MVP—Tesla Roadster Tests key hypothesis and delights some customers #UZWEBINAR
- 8. 8 UX INNOVATION LLC The Minimum Viable Prototype Tests key hypothesis and doesn’t require building a product #UZWEBINAR
- 9. 9 UX INNOVATION LLC MVP—It’s also contextual… 1984 2013 Yesterday’s MVP may not be viable in a changing market #UZWEBINAR
- 10. 10 UX INNOVATION LLC Find a big unmet need we can solve well… Scott Cook Founder Intuit Focus on what you are trying to learn #UZWEBINAR
- 11. 11 UX INNOVATION LLC Agile vs. Lean UX What’s the difference? #UZWEBINAR
- 12. 12 UX INNOVATION LLC What is Agile? Best practices for developing software including: • Holding regular short status meetings in a standup format • Requirements in the form of user stories or use cases • Defining tests for code before writing the code itself • Working in short iterations to create incremental improvements Common variations include Scrum & XP and all have a mindset that is best described as “lean” #UZWEBINAR
- 13. 13 UX INNOVATION LLC What is Scrum? Product Backlog Sprint Backlog Potentially Shippable Product Increment Daily Scrum Meeting 24 hrs 2-4 week Sprint Image from: http://www.mountaingoatsoftware.com/ ? #UZWEBINAR
- 14. 14 UX INNOVATION LLC What is Lean? Taiichi Ohno’s set of values and practices for removing three types of inefficiencies: • 無駄 = muda wasted outputs • 斑 = mura unnecessary inconsistencies • 無理 = muri wasted efforts It worked for Toyota…and it applies to other fields… #UZWEBINAR
- 15. 15 UX INNOVATION LLC The Waste? Building what nobody wants… #UZWEBINAR
- 16. 16 UX INNOVATION LLC The Lean Startup way http://theleanstartup.com/ Do you have a product? Build & measure it! #UZWEBINAR
- 17. 17 UX INNOVATION LLC What is Lean UX? As response to the overall trend of using agile and lean methods, Lean UX applies the lean principles to UX: • Sizing UX deliverables to fit agile timelines or “just in time” • Eliminating unnecessary internal deliverables • Reducing variability in UX deliverables that create waste • Collaborating more closely with non-UX team members • Getting user feedback earlier and on a more regular basis For more on Lean UX see: http://www.slideshare.net/balancedteam/02-sat-janice-lean-ux-landscape http://joshuaseiden.com/blog/2011/09/what-makes-it-lean/ http://uxdesign.smashingmagazine.com/2011/03/07/lean-ux-getting-out-of-the-deliverables-business/ #UZWEBINAR
- 18. 18 UX INNOVATION LLC The Lean UX way http://www.slideshare.net/clevergirl / Do you need to build a product? Prototype & measure it! #UZWEBINAR
- 19. 19 UX INNOVATION LLC Is the MVP Done? Data beats opinions #UZWEBINAR
- 20. 20 UX INNOVATION LLC Not all ideas are bright… http://en.wikipedia.org/wiki/File:Carbonfilament.jpg #UZWEBINAR
- 21. 21 UX INNOVATION LLC The key? Iterate and measure as you go... Take your best shot, assess & try again #UZWEBINAR
- 22. 22 UX INNOVATION LLC Good metrics aligned to clear goals Define objective metrics as team goals #UZWEBINAR
- 23. 23 UX INNOVATION LLC • Ship a desirable car with a 200 mile range • Get users to add 7 friends in 10 days https://www.teslamotors.com/blog/miles-progress http://genius.com/Chamath-palihapitiya-how-we-put-facebook-on-the-path-to-1-billion-users-annotated Clear Goals Create Great Products #UZWEBINAR
- 24. 24 UX INNOVATION LLC Dave McClure’s Categories of Metrics • Marketing oriented, get user to know offering exists • Get user to engage with product for initial 1st use • Get user to integrate offering into their lives and use long term • Get user to recommend to a friend AARR! These cover the entire product lifecycle or UX Categories based on Dave McClure’s AARRR Startup Metrics for Pirates see: http://500hats.typepad.com/500blogs/2007/06/internet-market.html Acquire Activate Retain Refer #UZWEBINAR
- 25. 25 UX INNOVATION LLC Collaboration Via Shared Goals http://flickr.com/photos/15489034@N00/420003710 #UZWEBINAR
- 26. 26 UX INNOVATION LLC Making Progress? Candidates for design metrics to guide teams: • # of interactions with target user segments (personas) • Sign-up rates • Retention rates • Task completion rates • Time on task • Error rates • Satisfaction scores • Net Promoter Scores Focus on testing the right things Be thoughtful about what you measure! #UZWEBINAR
- 27. 27 UX INNOVATION LLC Drive hypothesis and goal setting • Questions to ask on your team • Have we validated these stories and personas? • Did the iteration incorporate measurable user feedback? • Did you increase usage? • Do users they like it? • Can they use it? • Would they recommend it to a friend? • Did we make it measurably better? • UX principles remain the same in Agile • Iterations without user tests are a lost learning opportunity • Waiting until the end test with users is the waterfall way • Focus on measurable user behaviors not team’s opinions • “Good enough” or “done” should be determined by users #UZWEBINAR
- 28. 28 UX INNOVATION LLC Do customer validation interviews http://steveblank.com/ Do you have a market/user? Get out of the building! #UZWEBINAR
- 29. 29 UX INNOVATION LLC Brief Description Married with 2 young children, Katie is a working mom who wants to cook more at home. She enjoys sharing recipes with friends and considers herself a “foodie”. Computer Gear •Apple MacBook Air (2013) •iPhone 6s •iPad (3rd gen) Pain Points Wants recipes that are nutritious and healthy but quick to make, not recipes that are incomplete or hard to follow. No ingredients she can’t find easily. Hates spending lots of money on exotic ingredients only to find her kids hate the taste. Avoid any extra trips to the store during her busy week for items she forgot, or can’t find easily. Dealing with other children’s food allergies. Values Being considered a good cook. Likes organic and healthy foods but willing to compromise for dishes that are really tasty or that her kids or spouse really love. Simplicity over complexity, but willing to put a little extra effort in to make things that taste great or are really healthy. Saving money by cooking food at home. Key User Stories US1 Find recipes for ingredients I have on hand to avoid extra shopping trips US2 Create a weekly shopping list with ingredients for recipes before shopping US3 See recipes my friends recommend for ideas to plan meals US4 See recipes famous chefs recommend for ideas to plan meals US5 Save recipes with cooking notes so I can reuse them when cooking meals US6 Share my recipes and cooking notes with my friends and family US7 Find recipes for things on sale when planning meals to save money None yet Related User Research Average Task Completion Rate = ?% System Usability Score = ? Net Promoter Score = ? Experience Rating = ? Add stories and metrics to personas #UZWEBINAR
- 30. 30 UX INNOVATION LLC Use story maps to identify MVP stories See Jeff Patten’s article www.agileproductdesign.com/writing/how_you_slice_it.pdf #UZWEBINAR
- 31. 31 UX INNOVATION LLC Iterating without user feedback is fast but futile… Sprint towards clear goals Measure progress #UZWEBINAR
- 32. 32 UX INNOVATION LLC Who?1 What? 2 Groom 3 Estimate 4 Assign5 Market Metric 7 Story Metric 6 Bonus: Add hyperlinks to deliverables #UZWEBINAR
- 33. 33 UX INNOVATION LLC The Challenge: Testing in an Agile Way • Traditional testing does not fit well into agile: • Recruiting can take up to two weeks • Reports and test plans take too long • You often don’t have a stable UI until too late • What can we do? Adapt existing methods? • Include design research tasks as part of sprint planning • 3x3 solves the “waiting till the end” for working UI problem • RITE is a step in the right direction, it speeds up iterations • A/B split tests are effective for tweaking final details • Automate with RUT when at all feasible #UZWEBINAR
- 34. 34 UX INNOVATION LLC Design Refine Combine The 3x3 way: Test multiple concepts early http://www.carolrighi.com/documents/Righi%203x3.pdf • 3 concepts, 3 pages deep • 3 iterations of design & test • 3 or more users per iteration • Test low fidelity first, then high • Improve concept each iteration • Combine best elements into 1 concept & iterate Refine Design #UZWEBINAR
- 35. 35 UX INNOVATION LLC The RITE way: Refine one concept faster Schedule image from Brian Keith Sullivan http://bigdesignevents.com/2011/08/5-ways-to-improve-your-next-agile-project-with-the-rite-method/ RITE: Method http://en.wikipedia.org/wiki/RITE_Method REFINE DESIGN REFINE DESIGN REFINE DESIGN #UZWEBINAR
- 36. 36 UX INNOVATION LLC The A/B way: Build and compare Design A Build & Launch A Test A Design B Build & Launch B Test B • Design & build multiple designs and compare details • By nature less subject to test condition biases • Easy to get large sample sizes & statistically reliable metrics • Not so great at providing deeper insights on “why” Update Repeat #UZWEBINAR
- 37. 37 UX INNOVATION LLC Remote Automated Usability Testing (RUT) • Not just site analytics or surveys • Task oriented vs. page views and time on site • Not limited to existing users of your site • Can be used on prototypes or working code • No moderator required—scales with low effort • Combines the benefits of lab testing with the scalability of A/B testing #UZWEBINAR
- 38. 38 UX INNOVATION LLC Remote Automated Testing supports • Task completion rates vs. click counts • Time on tasks vs. time on site • Click path data by user task • Heat maps showing where users click on a page • Task specific and overall comments • Survey data (e.g., SUS or NPS ) • Video highlights • Larger sample sizes #UZWEBINAR
- 39. UserZoom at a Glance • All-in-one solution to rapidly test Usability & Measure UX • Remote Unmoderated Usability Test (Qualitative or Think-out-loud) • Usability Benchmarking • Card Sorting • Tree Testing • Screenshot Click Testing • Screenshot Timeout Testing • Online Surveys • Test any digital UI at anytime during the product life cycle The All-in-One UX Research Platform for the Data-Driven Enterprise
- 40. 40 UX INNOVATION LLC Tips for Remote Automated Testing • Always run small pilots before large scale tests, in many cases this can provide valid data anyway • Allow at least a week for tests with panel participants • Combine intercept recruiting with panels to minimize costs while reaching both new and existing users • Keep the participant tasks to a small number to minimize drop out rates • Use content analysis techniques to analyze open ended survey data #UZWEBINAR
- 41. 41 UX INNOVATION LLC More Agile tips • Have personas agreed to by team before sprinting • Develop an appropriate recruiting strategy early • Define hypotheses & target metrics for stories in the backlog working with your product owner • Track usability metrics in a public place • Test early design concepts not just final details • Automate UX testing for stories as you go & retest • Waiting until the end is the waterfall way #UZWEBINAR
- 42. 42 UX INNOVATION LLC It is not the strongest of the species that survives, nor the most intelligent. It is the one most adaptable to change. Charles Darwin Final thoughts #UZWEBINAR
- 44. 44 Need more info? Get answers about UserZoom. We are here to help. Talk with one of our experts. Contact us UserZoom 10 Almaden Blvd. #250 San Jose, CA 95113 Toll free number: (866) 599 1550 www.linkedin.com/company/userzoom@UserZoomFollow us
Notas do Editor
- Before we get started, I want to review a few housekeeping items and let you know how you can participate in today´s session. You have the ability to ask questions using your Questions Pane. Simply type in your question and click Send. At the end of the presentation we will do a Q & A session and take as many questions as we have time for.
- Scott Cook founder of Intuit has a great way of putting this…good guidance but it leaves some questions, how do you define big? And what does well mean? In agile we talk about minimum viable products, so this is key. What’s well, or good enough?
- These techniques were identified in the 90’s by various people who all met at a ski resort in Utah 11 years ago to coin the term Agile, and wisely to define a set of values and principles to guide future like minded individuals…
- Note the timelines. You start with basically a to-do list and in the end you’re trying to create something you could ship. Running a traditional test in a month is possible, but 2 weeks is much harder. It can take almost 2 weeks just to recruit users for a traditional study. It means you have to front load some stuff, or run a sprint behind.
- Ever write a usability report nobody read? That’s an example of muda. Did the report note UI inconsistencies due to a lack of communication on the team. That’s mura. Did you stay up all night to write that report. That’s muri.
- Unless you’ve been hiding under a rock, you know that the Lean Startup guys are making a lot of progress here, and their ideas consider UX concepts. I’m excited that Eric and the Lean UX folks have incorporated UX concepts. I’m even more excited that they’ve refocused the agile community on hypothesis testing, which is core to both UX and Agile.
- Janice Fraser coined the term, others like Jeff Gothelf are popularizing it. Today, we’re going to focus on how user testing changes as you become more lean.
- Unless you’ve been hiding under a rock, you know that the Lean Startup guys are making a lot of progress here, and their ideas consider UX concepts. I’m excited that Eric and the Lean UX folks have incorporated UX concepts. I’m even more excited that they’ve refocused the agile community on hypothesis testing, which is core to both UX and Agile.
- Don’t rely on random opinion, or the HiPPO, get facts.
- But we all know that not all ideas are equal.
- Add more attractive images
- Dave McClure, Silicon Valley internet entrepreneur and investor has advised startups to think about 5 key types of metrics using his AARRR acronym. I’ve listed the first 4 here which are UX centric to give you a sense of why UX is not just the product or service design.
- Change only starts when people embrace new values. Tracking objective progress towards the desired outcome by making it visible is key. I’ll assume you care about UX and design, or you wouldn’t be here. The secret sauce in UX is user involvement. The more user involvement throughout the process the more effective your iterations will be.
- Steve Blank author/entrepreneur and business professor
- Here’s an example of a story map from Jeff Patten. Note how this is really just a way of collaboratively organizing the backlog of stories. It focuses on the ideation phase. A great place to start because it focuses the team. It’s great at analyzing user story dependencies and helping teams define the minimum viable functionality
- The UXI Matrix is a tool for tracking UX metrics related to user stories. This is a simplified version showing sample data. It can be extended to cover different metrics or simplified, but the key concept here is to start keeping score, or tracking, how well you are solving an individual story, as well as a set of stories both in a sprint and across sprints. And yes, you could estimate per story for each persona…but that was omitted here to simplify the example.
- The challenge with most traditional usability methods is they are very resource intensive and often take too long for agile timelines. We need to adapt…
- 3x3 is a way of coming up with a conceptual design based on testing early low fidelity prototypes of the alternatives to select the best one. Like Mendel, breeding a better pea. It can save you a lot of rework later doing what Alan Cooper used to call “trying to sand a table into a chair” by ensuring your starting down the right path. It’s ideally suited for new projects or site redesign work. It can be extended to greater numbers of alternatives and more iterations as needed. Key to this type of formative study is recruiting the right participants. Biased results can send you off a cliff.
- RITE or rapid iterative test and evaluation where you run many small sample tests and instead of waiting until the end to fix issues, you fix issues as you go and continue testing. While compatible with agile, scaling over the long term can be challenging as it’s done traditionally. Keep in mind it’s purpose is to rapidly refine an existing design. While it can be adapted it’s best suited for refining a single design.
- A/B tests are very efficient at refining final details such as button size, color or placement for an existing site. But back to what Alan Cooper said, you can’t sand a table into a chair. A/B testing is well not suited to fixing major design problems. It’s a power sander. If you rely solely on intercept testing, you may never expand your market beyond the early adopters who might be giving you the wrong feedback for your ideal market.
- Note the differences between this an A/B testing.
- With large scale automation comes the potential for large scale error. Always pilot, or you’ll end up paying participants for nothing. Remember that you could be introducing a bias if you only test with users who respond to your requests in a very short time. Take advantage of the low cost of intercept recruiting when refining for existing users, and save the cost of panels for what they are best suited for. Avoid trying to run 2 hour studies, you’ll end up with lots of drop outs. Beware of non-automated methods that could create lots of work, plan for open ended questionnaires or other analyses that might be bigger than you bargained for.
- There are many things that you can front load to make your life easier. It’s like having gas in the car before driving to work. There’s nothing non-agile about that. Take personas. If you work with the product owner or marketing staff to define these, it makes recruiting much easier. The budgets for recruiting from panels should come from marketing budgets. Get the whole team involved in defining tests and metrics, including setting goals for releases to improve metrics. Stop thinking about user testing is something you do at the end. That mindset is an artifact of waterfall development.