Presentation from WIAMIS 2012 in Dublin, titled "Who are the users of a video search system? Classifying a heterogeneous group with a Profile Matrix"

1. Who are the users of a video
search system?
Classifying a heterogeneous group
with a Profile Matrix
Max Kemman, Martijn Kleppe, Henri Beunders
Erasmus University Rotterdam
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

2. User research
• Understanding the user [3, 4]
– Acceptance
– Performance
– Capabilities
– Weaknesses
• Goal
– Creating systems that are intuitive and helpful to
the users
[3] Y. Liu, A. Osvalder, and M. Karlsson, “Considering the importance of user profiles in interface design,” no. May, 2010
[4] J. Preece, Y. Rogers, and H. Sharp, “Interaction Design: Beyond Human-Computer Interaction,” Design, vol. 18, no. 1, pp. 68-68, 2002
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

3. Problem statement
• Investigating a heterogeneous user group
• Extreme case: potential user group consists of
everyone
• Distinguishing groups, using classifications of:
[1]
– Roles
E.g. by occupations
– Demographic characteristics
E.g. by age, gender or socioeconomic status
[1] D. O. Case, Looking for Information - A Survey of Research on Information Seeking, Needs, and Behavior, vol. 29, no. 3. San Diego Academic
Press, 2002
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

4. Research question
Can we classify (sub)groups of users in a
heterogeneous group of people and describe
their characteristics using the Profile Matrix?
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

5. Profile Matrix
• Two axes [5]
• Experience [3]
Experienced
– Amount of skills a
user may have or
lack
Goal-directed Non-goal-directed
• Goal-directedness [7]
– Navigators
– Explorers
[3] Y. Liu, A. Osvalder, and M. Karlsson, “Considering the importance of user profiles in interface design,” no.
May, 2010
[5] K. G. Rasmussen, R. Iversen, and G. Petersen, “M3.2.3 Personas Catalogue,” EuropeanaConnect, 2010.
[7] R. W. R. White and S. M. Drucker, “Investigating behavioral variability in web search,” Proceedings of the Inexperienced
16th international conference on World Wide Web - WWW ’07, pp. 21-30, 2007.
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

6. Goal-directedness axis
• Going directly for a specific goal, or wandering around
• Navigators versus Explorers Experienced
• Self-evaluated 7-point semantic
differential
– Content: what
Goal-directed Non-goal-directed
– Location: where
-2 1 0 1 2
– Structure: how
– Resource: adjust
• Cronbach’s α=0.753
• Aggregated to linear scale from -2 to 2
Inexperienced
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

7. Experience axis
• Amount of skills a user may have or lack
• Self-evaluated 5-point Likert scale 2 Experienced
Features Search
• Share • Success 1
• Subtitles • Ease
• Reuse Goal-directed Non-goal-directed
• Full-screen 0
• Quality
-1
• Cronbach’s α=0.711
• Aggregated to linear scale from -2 to 2
-2 Inexperienced
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

8. How Profile Matrix works
Suppose we have a crowd of people...
http://www.flickr.
com/photos/caitli
nator/400619870
3/
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

9. How Profile Matrix works
...we can distinguish groups based on roles or
demographics...
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

10. How Profile Matrix works
...and then place these groups on a Profile
Matrix
Experienced
Goal-directed Non-goal-directed
Inexperienced
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

11. Survey
• Survey (N=970) for personal use of online video
search systems
– International
• The UK (318)
• The Netherlands (311)
• Germany (341)
– Representative samples
• Age
• Gender
• Education
– 850 people watched video online
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

12. Survey
• A general overview of
– Who the ‘home users’ are
– What they use for online video
• Sections
– Demographics
– Video consumption
– Profile Matrix
– AXES interest
• Current data published Open Access
https://easy.dans.knaw.nl/ui/datasets/id/easy-dataset:49793
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

13. Results: Overall Profile Matrix
Majority appears to be inexperienced and goal-
directed 2
Experienced
1.5
1
0.5
Goal-directed 0
Non-goal-directed
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
-0.5
-1
-1.5
-2
Inexperienced
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

14. Results: Consumption Profile Matrix
• All averages within goal-directed half
• Hours per week
• Significant differences in experience between
all groups Goal-directed
0.4
0.2
0
-1 -0.8 -0.6 -0.4 -0.2 0
-0.2
-0.4
-0.6
-0.8
-1
-0.57692 -0.8268 -0.31007 -0.79092 Inexperienced
>8h 4-8h 1-4h <1h
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

15. Results: Age Profile Matrix
• All averages within inexperienced
goal-directed quadrant
• Significant differences in experience between
– 18-24 and 45-54 Goal-directed
0
– 18-24 and 55+ -1 -0.8 -0.6 -0.4 -0.2
-0.2
0
– 25-34 and 45-54 <45 -0.4
– 25-34 and 55+ ≥45
-0.6
– 35-44 and 45-54 -0.8
– 35-44 and 55+ 18-24
-0.771666667 25-34
-0.758333333 35-44
-0.7850000000
-1
Inexperienced
45-54
-0.718333333 55+
-0.658333333
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

16. Results: Education Profile Matrix
• All averages within inexperienced
goal-directed quadrant
• No significant differences
Goal-directed
0
-1.0 -0.8 -0.6 -0.4 -0.2 0.0
-0.2
-0.4
-0.6
-0.8
-1
Inexperienced
Higher
-0.8 Secundary
-0.7 Primary
-0.6
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

17. Conclusion
• Profile Matrix helps to create (sub)groups of
people
• Better understanding of the heterogeneous
group
• Helps choosing groups for usability research
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

18. Future directions
• Investigating what these differences mean with
regard to users’ performance or acceptance of
video search system
• Groups based on roles instead of demographics
– Broadcast Professionals
– Academics
– Journalists
– Home Users
• Application for other search systems
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu

19. Questions or comments?
Do you think this is useful?
Contact:
Max Kemman
kemman@eshcc.eur.nl
Who are the users of a video search system?
WIAMIS'12 Dublin 23-25 May 2012
www.axes-project.eu