This was a lecture delivered to computer science students on the importance of user-testing. I use the example of Pitch Perfect, our SaaS startup, and how our delay in performing user-testing cost us more time and money than we saved skipping the testing.

1. Practical UX Methods
@kyleracki
kyleracki.com
Oct 8th, 2013

2. Who am I...
Designer/Developer
Co-founder of
headspacedesign.ca
Co-founder of pitchperfectapp.
com
Love SaaS startups

3. What is Pitch Perfect...

4. Not this

5. Proposal software

7. Where the idea came from

8. Who it’s for

9. What we’re up against
• Similar SaaS apps first to market
• Difficult to raise capital

10. The road to get here.
• Prototype in 2012
• Funded in Jan 2013
• In beta since April 2013
• Over 200 signups to-date
• 3 paying customers (as of yesterday)

11. A passable alternative

12. The problem
• Usability testing should have begun earlier
• System was not QA tested
• Remote meant I couldn’t adapt test

13. Where we went wrong
• Limited customer research
• Not knowing what users really want
We thought they were:
• Advertising and marketing agencies
We thought they wanted
• Complete design customization

15. Survey post-beta

16. “More kick-butt templates that make it easy for
me to pop in my content and look like a pro.
Simply put, make me look like a rockstar with
the least amount of effort on my end.”

17. What they really want
1. Quick set up
2. Pre-made design templates
3. Ability to import their content
3. Re-brandable
All of the above while at the same time being
so easy to use...

18. These guys
could use it

19. Tools
+
+

20. Back to the graph paper

21. High-fidelity version

22. What am I missing?

24. Step 1. Who are my user personas?
1. Account manager/Sales rep
2. Entrepreneur/Small business owner

25. Step 2. Go where the people are.

26. Screening
Questions

27. Analyze
Results

28. Step 3. Preparation
• What is the goal of the testing?
• Scenarios
• Line up user tasks with testing goals

29. Do a practice run with family

30. Step 4. Conducting the sessions
• Make participant comfortable
• Record the screen (Screenflow)
• They aren't being tested
• Confirm recording
• Give incentive up-front

31. Clarissa
• Good tester
• Graphic designer by profession

32. Aaron
• Less web savvy

33. Anuj
• Not an ideal tester
• Not verbally expressive
• Computer science student

34. Compiling the results

35. Sharing with the team

36. Key usability issues
● Understanding sections vs pages, and how
to save to library
● Understanding advanced settings vs themes
● Knowing how to drag in/import sections from
library
● Understanding how to edit page template,
add footer, cover page etc.

37. Design solutions

38. Product Evolution

40. Next steps
User test revised prototype
Test working version after QA but before
release

41. Become this guy

42. Key Lessons
Test-early, test-often is not simply best practice
or “ideal in a perfect world”. It’s critical to
designing a successful product and can make
or break a company.
User-testing is not beta-testing, QA-testing or
sending a link with survey questions. Only user-
testing will uncover key insights related to
specific usability problems.

43. Key Lessons
Focus on the onboarding process first, and
deliver the ‘wow’ moment the first time
someone uses the software
You can’t test everything. Align the following:
• Pain
• Value Prop
• Features <- test these first

44. Suggested reading.

45. Thank you.
@kyleracki
kyleracki.com
pitchperfectapp.com
headspacedesign.ca