1. User Engagement – A Scientific Challenge Mounia Lalmas Yahoo! Research Barcelona [email_address] Ioannis Arapakis Ricardo Baeza-Yates Georges Dupret Janette Lehmann Lori McCay-Peet Vidhya Navalpakkam Elad Yom-Tov Collaborators
2.
3.
4. Would a user engage with this web site? http://www.nhm.ac.uk/
5. Would a user engage with this web site? http://www.amazingthings.org/ (art event calendar)
6. Would a user engage with this web site? http://www.lowpriceskates.com/ (e-commerce – skating)
7. Would a user engage with this web site? http://chiptune.com/ (music repository)
8. Would a user engage with this web site? http://www.theosbrinkagency.com/ (photographer)
12. Forrester Research – The four I’s Measuring Engagement, Forrester Research, June 2008
13.
14. Subjective measures Record a user’s perception (generally self-reported) of the technology Two-part processes: Increased use of crowd-sourcing based studies
15.
16.
17. Characteristics and measures S. Attfield, G. Kazai, M. Lalmas and B. Piwowarski. Towards a science of user engagement (Position Paper), WSDM Workshop on User Modelling for Web Applications, 2011.
18.
19. Page stylistics + layout +links + saliency Measurements and methodologies + online analytics metrics (dwell time, …) + questionnaires + crowd-sourcing - biometrics (eye tracking, mouse tracking, …) Goals + Models of user engagement - Metrics of user engagement Engagement types + attention + emotion + activity + popularity + loyalty - functionality +/- intent & interest user engagement within and across site
20. Focus – Two “opposite” studies models USER ENGAGEMENT 1. Models of user engagement 2. On saliency, attention and positive effect activity metrics loyalty metrics popularity metrics hobbies navigation social media e-commerce magazine sport news search weather mail weekly news … focused attention affect (emotion) saliency
21.
22.
23.
24. Methodology General models User-based models Time-based models Dimensions 8 metrics 5 user groups 8 metrics per user group weekdays, weekend 8 metrics per time span #Dimensions 8 40 16 Kernel k-means with Kendall tau rank correlation kernel Nb of clusters based on eigenvalue distribution of kernel matrix Significant metric values with Kruskal-Wallis/Bonferonni #Clusters (Models) 6 7 5 Analysing cluster centroids = models
25.
26.
27.
28.
29. Focus – Two “opposite” studies models USER ENGAGEMENT 1. Models of user engagement 2. On saliency, attention and positive effect activity metrics loyalty metrics popularity metrics hobbies navigation social media e-commerce magazine sport news search weather mail weekly news … focused attention affect (emotion) saliency
39. The big picture ……….… my vision Page stylistics + layout +links + saliency Measurements and methodologies + online analytics metrics (dwell time, …) + questionnaires + crowd-sourcing - biometrics (eye tracking, mouse tracking, …) Goals + Models of user engagement - Metrics of user engagement Engagement types + attention + emotion + activity + popularity + loyalty - functionality +/- intent & interest user engagement within and across site
40.
Notas do Editor
As said here it is important to differentiate between user experience and user engagement, the latter is an aspect of the former. User engagement related to the positive aspect of the interaction including that of being captivated. Providing technologies with which user engaged is very important, and this is true of any web service. And those are two nice quotes about the importance of engagement.
Characteristics elaborate the notion of engagement over 3 broad dimensions: emotional, cognitive and behavioural. We identified 8 of them based on previous works / suggested by us. Attention -> exclusivity Affect -> lack of fun can lead as a barrier to shopping (O’Brian) Aesthetics -> promotes attention, stimulate curiosity Endurability -> does the experience meet the expectation?
Novelty -> freshness of content / has to be carefully balanced Richness = complexity of thoughts and actions, increased complexity/ Affect = relates to the emotion experienced during the interaction (fun and enjoyment)
The way it often work A two-part process. One is very exploratory, it is important to “probe” carefully, so that indentify all dimension, usually conducted with a small number of users. Highly qualitative, and really the aim is to identify the important characteristics (work of Tom et al did this in 4 areas, online shopping, web searching, educational web casting, and video games). People in HCI knows how to do this well. Then from there, construct more focused studies, but larger scale, and actually measure them in a particular interactive experience, and again going back to the Dalhousie work, they did so for the online shopping scenario. Constructed an online questionnaire, and got 440 responses. Cleaning of data, and factor analysis to reduce the set of characteristics (100) to few key ones (down to 6). Again what I want to make it clear is the methodology, and here the fact we have qualitative data. Would be important to see how the questionnaire developed by them can be used in other area, other domain (IM different to news portal). Health trust Film aesthetics
In the paper we propose a mapping between objective measures and engagement characteristics, e.g. focused attention can be measured by distorted perception of time, follow-on task performance, and eye tracking Online behaviour = can be large scale IR = special status (metrics) -> Simulated search, user models, linking to user satisfaction