Julia Kiseleva's slides for PhD defense on June 13 2016. The thesis is available by the following link -- https://www.researchgate.net/publication/303285745_Using_Contextual_Information_to_Understand_Searching_and_Browsing_Behavior

1. Using Contextual Information
to Understand
Searching and Browsing Behavior
Julia Kiseleva
Eindhoven University of Technology
Eindhoven, The Netherlands, June 2016

2. Using Contextual Information
to Understand
Searching and Browsing Behavior

3. Searching Behavior
Want to go to
CIKM conference
QUERY SERP

4. Browsing Behavior
User Preferences

5. Using Contextual Information
to Understand
Searching and Browsing Behavior

6. Contextual Information
Explicit Context Implicit Context

7. Contextual Information
Explicit Context Implicit Context

8. Contextual Information
Explicit Context Implicit Context
Contextual Situations
(Android Tablet, Weekend)
Photo credit: Delwin Steven Campbell
via Visualhunt.com / CC BY

9. Using Contextual Information
to Understand
Searching and Browsing Behavior

10. Our Main Research Goal
How to
use
contextual information
in order to
understand
users’ searching and
browsing
behavior on the web?
Improve Online
User Experience

11. Applied Studies
Browsing Behavior

12. Destination Finder
Chapter 3 ‘Contextual Profiles’.
L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015
J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

13. Destination Finder
Chapter 3 ‘Contextual Profiles’.
L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015
J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

14. Destination Finder
Chapter 3 ‘Contextual Profiles’.
L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015
J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

15. Destination Finder
Optimized Ranking of Destinations
Using Contextual Situations
Increased User Engagement
(Click Trough Rate +3.7%)
Chapter 3 ‘Contextual Profiles’.
L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015
J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

16. Applied Studies
Browsing Behavior

17. Applied Studies
Browsing Behavior Searching Behavior
&

18. Changes in User Satisfaction
Want to go to
CIKM conference
QUERY SERP
Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’
J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014
J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

19. Changes in User Satisfaction
QUERY SERP
,
Dynamic over Time
Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’
J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014
J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

20. Changes in User Satisfaction
Time
Satisfaction
Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’
J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014
J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015
QUERY
, SERP

21. Changes in User Satisfaction
Time
#Reformulations
~
Satisfaction
Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’
J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014
J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015
2013
Oct NovSepAugJul
QUERY
, SERP

22. Changes in User Satisfaction
Before November 2013 After November 2013
Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’
J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014
J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015
QUERY= ‘flawless’

23. Changes in User Satisfaction
Before November 2013 After November 2013
Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’
J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014
J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015
QUERY= ‘flawless’

24. Applied Studies
Browsing Behavior Searching Behavior
&
Cortana:
“What can
I help you
do now?”

25. Q1: how is the weather in Chicago
Q2: how is it this weekend
Q3: find me hotels
Q4: which one of these is the cheapest
Q5: which one of these has at least 4 stars
Q6: find me directions from the Chicago airport to
number one
User’s dialogue
with Cortana:
Task is “Finding
a hotel in
Chicago”
Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’
J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016
J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

26. Q1: find me a pharmacy nearby
Q2: which of these is highly rated
Q3: show more information about number 2
Q4: how long will it take me to get there
Q5: Thanks
User’s dialogue
with Cortana:
Task is “Finding
a pharmacy”
Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’
J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016
J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

27. Cortana:
“Here are ten
restaurants
near you”
Cortana:
“Here are ten
restaurants near
you that have
good reviews”
Cortana:
“Getting you
direction to the
Mayuri Indian
Cuisine”
User:
“show
restauran
ts near
me”
User:
“show the
best ones”
User:
“show
directions
to the
second
one”

28. Cortana:
“Here are ten
restaurants
near you”
Cortana:
“Here are ten
restaurants near
you that have
good reviews”
Cortana:
“Getting you
direction to the
Mayuri Indian
Cuisine”
User:
“show
restauran
ts near
me”
User:
“show the
best ones”
User:
“show
directions
to the
second
one”
No Clicks
???

29. Cortana:
“Here are ten
restaurants
near you”
Cortana:
“Here are ten
restaurants near
you that have
good reviews”
Cortana:
“Getting you
direction to the
Mayuri Indian
Cuisine”
User:
“show
restauran
ts near
me”
User:
“show the
best ones”
User:
“show
directions
to the
second
one”
SAT? SAT?
SAT
?
Overall
SAT?
? SAT? SAT?
SAT
?

30. Acoustic Similarity
Phonetic Similarity
Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’
J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016
J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

31. Tracking User Interaction
Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’
J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016
J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

32. 3 seconds 6 seconds
33% of
ViewPort
66% of
ViewPort
ViewPortHeight
2 seconds
20% of
ViewPort
1s 4s 0.4s 5.4s+ + =
Tracking User Interaction
Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’
J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016
J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

33. Quality of Interaction Model
Method Accuracy (%) Average F1 (%)
Baseline 70.62 61.38
Interaction Model 80.81*
(14.43)
79.08*
(28.83)
* Statistically significant improvement (p < 0,05 )
Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’
J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016
J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

34. • Contextual information should be taken into account
to understand web and mobile users’ behavior
• Analyzing behavioral signals over time is needed to
detect changes in user satisfaction with web search
• Touch signals are crucial for inferring user
satisfaction with intelligent assistants on mobile
devices
Conclusion