• 3. We’ll cover...1 The wrong way to A/B test2 5 places to get ideas for testing3 Get the most from your A/B testing tools • 4. HOW MOST PEOPLEA/B Test • 5. A/B testing is totally rad! We should do it! • 6. Everyone loves button colors VS • 7. Next up? Layouts! • 8. What about photos? VS • 9. And conversions went up by 4%! Party time! • 10. There’s a better way... Testing random elements is very inefficient. You will not find the big wins this way. • 11. You’re also not learning Red vs green buttons doesn’t help you understand your customer. • 12. GETQualitative • 13. Qualitative data definition Data that isn’t a number. • 14. Start with qualitative insights Use other sources of data to get a basic understanding of what your customers want. • 15. Understand first, then do the A/B test You’ll focus on the right tests and increase your conversions a LOT faster. • 16. The A/B Testing Process 1 Get qualitative insights 2 Predict how to improve 3 Confirm prediction with an A/B test • 17. Why is this process better? Helps you find the 40% increase in conversions instead of only the 4%. • 18. WHERE IS THE QUALITATIVE Data? • 19. Location #1 Surveys • 20. Use one-question surveys on your site • 21. Also try long surveys • 22. Survey best practices1 Keep it short2 Only ask for answers that you’ll use3 Start with open-ended questions • 23. Location #2 Feedback • 24. Do you have a feedback email? • 25. Talk to your support team They know exactly what problems keep coming up. • 26. Also talk to your sales team They talk to customers every day. • 27. Location #3 Reach Out • 28. Nothing beats talking to customers personally Get them on the phone and deeply understand how they use your product. • 29. If local, take them out to lunch or coffee. • 30. This is not a pitch Focus on asking good questions and listening. • 31. Location #4 User Activity • 32. Look at activity of individual people You’ll need customer analytics for this. • 33. KISSmetrics People report • 34. When you have an insight... Confirm by looking at other people. • 35. Location #5Usability Tests • 36. What’s a usability test? Have someone use your site or product while you watch. • 37. Grab someone off the street • 38. Amazon’s Mechanical Turk - mturk.com • 39. UserTesting.com • 40. Places to Find Qualitative Insights • 41. The A/B Testing Process • 42. When should you test the small stuff? After you’ve found the big wins and you have a full-time growth team. • 43. A/B TESTINGTools • 44. A/B testing tools only measure the next step • 45. Tie A/B tests back to the customer life-cycle • 46. How do we actually do this? • 47. First, make sure you’re using customer analytics • 48. Then use Visual Website Optimizer or Optimizely OR • 49. Now all your A/B testing data is in KISSmetrics