We've all heard it. The best UX research method is the mixed-method. By combining both qualitative and quantitative data the better you can understand your users. Is there such thing as too much data? In this session, Alina will talk through how to manage your user insights to tangible actions and plan for your team. She will talk through: - How in Allegro user insights is collated through research, big data and behavioural sciences but what happens next; - How to prioritise your data/insights; - What challenges can you encounter and how to solve them; and - What best practices she uses to ensure the team is aligned in understanding these insights.

1. big data small data
+ = insight
A PRACTICAL GUIDE TO MIXED METHODOLOGIES FOR UX RESEARCH
Alina Magowska
Head of UX Research and UX Analytics

2. 1999
DO WE NEED MORE DATA?
2001
Allegro.pl starts on Dec 13th.
The software of the first
version of the website was so
small that it could fit on one
floppy disk
The millionth auction
2003
The millionth customer
registered the account an Allegro
2021
20 million customers each month
(Gemius 2020)
#1 commerce platform in Poland
#10 e-commerce websites
worldwide

3. DO WE NEED MORE DATA?

4. YES!
DO WE NEED MORE DATA?

5. FRAMEWORK
SETUP
PLAN
MANAGE
STEP CHALLENGE ACTION TO TAKE
#1
#2
#3
What answer do you need and
which methods to mix?
Which scenario to choose?
How to deal with data
diversity?
● Specify challenges you face in the product dev process
(strategy → delivery)
● Determine what knowledge do you need (explicite → latent)
● Mix the right method (small + big)
● Exploratory research
● Exploratory
● Dynamic research
● Collate insights from different sources
● Prioritise data
● Distribute knowledge

6. SETUP

7. PRODUCT
DEVELOPMENT
Strategize
Explore
D
e
s
i
g
n
D
e
v
e
l
o
p
D
e
l
i
v
e
r
Strategic research
Exploratory
research
Define product brief +
success metrics Generate ideas
Prototype &
evaluate ideas
Plan
Build & test
Get
feedback
Product strategy
Release
WHERE ARE YOU IN THE PRODUCT DEVELOPMENT PROCESS?
WHY SHOULD WE DO THIS?
SMALL DATA: Field studies, Cultural probes
BIG DATA: Prescriptive analytics
Allegro - Product Development Process | EXAMPLES OF METHODS
ARE WE DOING THE RIGHT THING?
SMALL DATA: Interview, Customer workshop
BIG DATA: Behavioral analytics
WHAT SHOULD WE DO?
SMALL DATA: Concept Testing
BIG DATA: A/B testing, Descriptive statistics
ARE WE DOING THE PRODUCT RIGHT?
SMALL DATA: Usability Testing
BIG DATA: Experiments
IS IT STILL THE RIGHT THING?
SMALL DATA: UX Metrics, NPS
BIG DATA: Tracking & Evaluation

8. Inspiration: Context mapping: Experiences from practices (source)
SURFACE
DEEP
KNOWLEDGE METHODS WHAT PEOPLE
WHAT KNOWLEDGE DO YOU NEED?
SAY
THINK
EXPLICIT
OBSERVABLE
TACIT
LATENT
DO
USE
KNOW
FEEL
DREAM
SMALL DATA > BIG DATA
SMALL DATA < BIG DATA
SMALL DATA
?
BIG DATA

9. WHICH METHOD TO CHOOSE?
SURFACE
(WHAT)
DEEP
(WHY)
SMALL
DATA
BIG
DATA
Survey A/B Testing
Tracking
Statistical
modeling
Usability testing
Diary studies
Card sorting
Personalization
Interviews
Generative
sessions
Simple algorithms
NLP Neural
Networks
Reinforcement
Learning

10. PLAN

11. Adaptation: The Three Research Designs - Mixed model by OpinionX.co
EXPLANATORY
RESEARCH
insight
big data small data
insight
small data big data
small big
insight
EXPLORATORY
RESEARCH
DYNAMIC
RESEARCH
SCENARIOS

12. WHEN DATA SAYS ONE THING BUT USERS SAY ANOTHER
Less than 1%
use it
EXPLANATORY RESEARCH

13. WHEN YOU WANT TO VERIFY USER INSIGHTS
Results
Selection from
product catalogue -
design lifting
New search
EXPLORATORY RESEARCH

14. WHEN YOU WANT TO GO BEYOND
ML helps understand the reasons for contact, categorizes them and
thanks to that we have quick feedback from customers
Contact rate
1.7M reduction
this year
Buyers can find on Allegro multiple offers of the
same product. We are working on making all the
necessary information readily available, but
sometimes they are not, and customers contact
merchants.
DYNAMIC RESEARCH

15. MANAGE

16. COLLATE USER INSIGHTS FROM DIFFERENT SOURCES
UX/Design
Research
Market
Research
Customer
Support
Product
Data
Commerce
Intelligence
Data
Science
ML
Research
Market
data
Market
data
Market
data
in-house
DATA TRANSPARENCY
INSIGHT LEAD
COLLABORATION & CONNECTION
RESEARCH ONE-PAGER

17. PRIORITISE INSIGHTS
IMPACT INSIGHT A B C
EXPERIENCE How will this affect:
- improving the experience?
- making the experience
outstanding?
BUSINESS To what extent will this affect
business metrics, eg GMV, active
buyers?
TOTAL IMPACT 4 4 5
TECH How much will it cost?
(Who we need to engage, what
technology do we need?)
TOTAL SCORE (impact/tech cost) 4 2 1,6
TOTAL PRIORITY 1 3 2
Inspiration: The Umami Strategy by Aga Szóstek

18. PRIORITISE INSIGHTS
IMPACT INSIGHT A B C
EXPERIENCE How will this affect:
- improving the experience?
- making the experience
outstanding?
BUSINESS To what extent will this affect
business metrics, eg GMV, active
buyers?
TOTAL IMPACT 4 4 5
TECH How much will it cost?
(Who we need to engage, what
technology do we need?)
TOTAL SCORE (impact/tech cost) 4 2 1,6
TOTAL PRIORITY 1 3 2
Inspiration: The Umami Strategy by Aga Szóstek
Impact
Effort by organization
high
high
low
low
Let’s go!
Nope!
Maybe?
Maybe?

19. DEMOCRATIZE OR DISTRIBUTE?

20. FRAMEWORK
SETUP
PLAN
MANAGE
STEP CHALLENGE ACTION TO TAKE
#1
#2
#3
What answer do you need and
which methods to mix?
Which scenario to choose?
How to deal with data
diversity?
● Specify challenges you face in the product dev process
(strategy → delivery)
● Determine what knowledge do you need (explicite → latent)
● Mix the right method (small + big)
● Exploratory research
● Exploratory
● Dynamic research
● Collate insights from different sources
● Prioritise data
● Distribute knowledge

21. Images from: Pexels, Allegro.eu
FRAMEWORK:
1. SETUP
2. PLAN
3. MANAGE
big data small data
+ = insight
ALINA MAGOWSKA
Let's stay in touch!
Special thanks to Paweł Adamczak