Main takeaways:
- Importance of informing your decisions with data
- Know the difference between qualitative and quantitative data, when to use it and how to adjust it
- The importance of knowing how to perform proper usability testing and why to expose this to your entire team so they see first - hand how users interact with your product
14. Data beats opinion
The
Boss
If you have two or more opinions in
the room, the one backed by data
is likely to win
HIPPO
Highest Paid Person’s Opinion
15. Data gives you a goal
● You need an indication of progress if
not success
● Forces you to think about how you
measure
● Helps to keep beating back the
aforementioned HIPPO
16. Without data you are
flying blind
There is no point in releasing
software if you have no way of
knowing whether it is working
18. Qualitative data
comes from usability testing
● Needs a relatively small data sample
(5 - 10 participants)
● Less about numerical data more focused
on verbal feedback
● Answers questions you cannot answer
through numbers
19. Quantitative data
comes from analytics
● Gives you answers to questions that need a
large data sample
● Useful for continuous measurement
● Useful for segmenting users
Daily Average Users Return Rate - Weekly
Visitors by Region
% Customers that
clicked on the Thing
63%
5%
16%
Over 1 week
21. Keep it simple
● New designs should start simple -
wireframes or even paper prototypes
● Colours, button styles, graphics are all
distracting in early testing
● You don’t need a lab for usability
testing
22. Basic rules of usability testing
● Always prepare a script
● Identify yourself a researcher
● Ask open-ended questions
● Always record the interview if you can
● Always get permission to record
23. ● You don’t need a usability lab
● Can use Quicktime to record the
screen and audio *
● Observers can join using something
like Google Meet or Skype *
No budget? No problem.
* Don’t forget that whole permission thing...
24. A question for you and your stakeholders
… and what will you do
with the answers?
“What questions do you
want to answer...
25. How to choose the right measure
Consider
● ...whether your question is measurable
with a number.
● ... what is a reasonable timeframe over
which to measure.
● … whether your measures are related to
your quarterly or yearly goals.
27. Picking the right tools
Analytics libraries
Used for tracking page
views and events
● IMHO, Google Analytics is
the best free analytics library.
● Facebook also offers a free
toolkit, but some users find it
intrusive.
● Mixpanel is great, but not
free.
Heat Maps
Used to know when
people click, scroll and
linger
● Hotjar works well and has a
free tier or paid options
● Heatmaps look pretty, but
can be misleading...
Coaching & feedback
Used to help the user along,
or request feedback
● There are free options, but they
often require a lot of work
● WalkMe forces the user to
perform each action in a flow
● OpinionLab can be used to prompt
users for feedback
28. Planning for analytics
Consider
● ...how often you will need to change
libraries and what you are tracking
● ... who will need to make those changes
● … adding an analytics section to new
features and functionality
29. Continually informed
with Product Dashboards
● Put your metrics somewhere the
whole team can see them
● Helps everyone see the progress, or
spot problems
● Keeps everyone thinking about data
Google Data Studio
Geckoboard
31. Further reading...
UX for Lean Startups
This is a great book for learning
how to do user testing properly.
Lean Analytics
The core principle from this book
is to find the One True Metric you
need to measure. Great book
about being targeted in analytics.
32. Thank you! Feedback and questions are very welcome. You
can always reach me at pete@digicratic.com or
@pgoodalluk.
33. www.productschool.com
Part-time Product Management, Coding, Data Analytics, Digital
Marketing, UX Design and Product Leadership courses in San
Francisco, Silicon Valley, New York, Santa Monica, Los Angeles,
Austin, Boston, Boulder, Chicago, Denver, Orange County,
Seattle, Bellevue, Washington DC, Toronto, London and Online
Notas do Editor
Example 1: Engagement is something people often want to measure, but finding an accurate measure is very difficult
Example 2: “We had 19 new sign-ups this week, as opposed to 10 last week.”
* When measuring something like Return Rate – you may want people to return every day, but it may not be reasonable given the use case of your app
•Some measures may be better to measure on a quarterly basis, as you need a series of sprints to make a significant difference. – i.e. retention rate
Example 3: Avoid ”vanity metrics” – a number that sounds impressive, but really has no use – e.g. # of Visitors this month, time on page
* If your goal this quarter is to increase retention, then measure the rate of new sign-ups vs the ratio of lapsed users to active users (over some time period)