1. RECOMMENDER SYSTEMS AND
SEARCH ENGINES – TWO SIDES OF
THE SAME COIN!?
Bracha Shapira
Lior Rokach
Department of Information Systems Engineering
Ben-Gurion University
3. ARE YOU BEING SERVED?
What are you looking for?
Demographic – Age, Gender, etc.
Context-
Casual/Event
Season
Gift
Purchase History
Loyal Customer
What is the customer currently wearing?
Style
Color
Social
Friends and Family
Companion
4. RECOMMENDER SYSTEMS
A recommender system (RS) helps people that
have not sufficient personal experience or
competence to evaluate the, potentially
overwhelming, number of alternatives offered by
a Web site.
In their simplest form RSs recommend to their users
personalized and ranked lists of items
Provide consumers with information to help them
decide which items to purchase
7. WHAT MOVIE SHOULD I WATCH?
• The Internet Movie Database (IMDb)
provides information about actors,
films, television shows, television
stars, video games and production
crew personnel.
• Owned by Amazon.com since 1998
• 796,328 titles and 2,127,371 people
• More than 50M users per month.
8. abcd
The Nextflix prize story
In October 2006, Netflix announced it would give a $1 million to whoever created a movie-
recommending algorithm 10% better than its own.
Within two weeks, the DVD rental company had received 169 submissions, including three
that were slightly superior to Cinematch, Netflix's recommendation software
After a month, more than a thousand programs had been entered, and the top scorers were
almost halfway to the goal
But what started out looking simple suddenly got hard. The rate of improvement began to
slow. The same three or four teams clogged the top of the leader-board.
Progress was almost imperceptible, and people began to say a 10 percent improvement
might not be possible.
Three years later, on 21st of September 2009, Netflix announced the winner.
13.10.2012
10. WHERE SHOULD I SPEND MY VACATION?
Tripadvisor.com
I would like to escape from this ugly an tedious work life and
relax for two weeks in a sunny place. I am fed up with
these crowded and noisy places … just the sand and the
sea … and some “adventure”.
I would like to bring my wife and my children on a
holiday … it should not be to expensive. I prefer
mountainous places… not too far from home.
Children parks, easy paths and good cuisine are a
must.
I want to experience the contact with a completely different
culture. I would like to be fascinated by the people and
learn to look at my life in a totally different way.
11. Usage in the market/products Recommendation
State-of-the-art solutions
Examined Solutions
Method Commonness
Jinni Taste Kid Nanocrowd Clerkdogs Criticker IMDb Flixster Movielens Netflix Shazam Pandora LastFM YooChoose Think Analytics Itunes Amazon
Collaborative Filtering v v v v v v v v v v v v
Content-Based Techniques v v v v v v v v v v v
Knowledge-Based Techniques v v v v v v v
Stereotype-Based Recommender Systems v v v v v v v
Ontologies and Semantic Web Technologies for
v v v
Recommender Systems
Hybrid Techniques v v v v v v v
Ensemble Techniques for Improving
v future
Recommendation
Context Dependent Recommender Systems v v v v v v
Conversational/Critiquing Recommender
v v
Systems
Community Based Recommender Systems and
v v v v v
Recommender Systems 2.0
13.10.2012
13. Collaborative Filtering
Description
The method of making automatic predictions
(filtering) about the interests of a user by collecting
taste information from many users (collaborating).
The underlying assumption of CF approach is that 1 Collaborative Filtering
those who agreed in the past tend to agree again in
the future.
Selected Techniques
kNN - Nearest Neighbor
SVD – Matrix Factorization
Similarity Weights Optimization (SWO)
13.10.2012
14. COLLABORATIVE FILTERING
abcd
The Idea
Trying to predict the opinion the user will have on the different items and be able
to recommend the “best” items to each user based on: the user’s previous likings
and the opinions of other like minded users
Negative
Rating
?
Positive
Rating
13.10.2012
15. How collaborative filtering works?
“People who liked this also liked…”
abcd abcd
How it works User-to-User
Recommendations are made by finding users with
similar tastes. Jane and Tim both liked Item 2 and
disliked Item 3; it seems they might have similar taste,
which suggests that in general Jane agrees with Tim.
This makes Item 1 a good recommendation for Tim.
This approach does not scale well for millions of
Item users.
to
Item Item-to-Item
Recommendations are made by finding items that
have similar appeal to many users.
Tom and Sandra are two users who liked both Item 1
and Item 4. That suggests that, in general, people who
User to liked Item 4 will also like item 1, so Item 1 will be
recommended to Tim. This approach is scalable to
User millions of users and millions of items.
13.10.2012
16. KNN - NEAREST NEIGHBOR
Current User Users
1 1st item rate
0 Dislike
?
1
0
1 Like
abcd Prediction
abcd
Unknown Rating
abcd
Other Users
1
This user did not There are other
Items
? 1 The prediction
Unknown rate the item. We
was made based users who rated
0 will try to predict a
on the nearest the same item.
rating according We are interested
1 neighbor.
to his neighbors. in the Nearest
abcdHamming Distance
1 The Hamming distance Neighbors.
is named
after Richard Hamming.
0
User Model = 1
In information theory, the Hamming
distance between two strings of
interaction
abcd
Nearest Neighbors equal length is the number of
We are looking
1 positions at which the corresponding abcd
history the Nearest 1
symbols are different.
for
Neighbor. The Nearest
one with the 1 Neighbor
lowest Hamming 0 14th item rate
distance.
Hamming 5 6 6 5 4 8
distance
13.10.2012
17. IMPORTANT ISSUES
Cold Start
Implicit/Explicit Rating
Sparsity
Long Tail problem - many items in the Long Tail have only
few ratings
Portfolio Effect: Non Diversity Problem
It is not useful to recommend all movies by Antonio
Banderas to a user who liked one of them in the past
Beyond Popularity
Gray sheep problem
Iformation Security
Misuse
Privacy
19. CONTENT-BASED RECOMMENDATION
In content-based recommendations the system tries to
recommend items that matches the User Profile.
The Profile is based on items user has liked in the past or explicit
interests that he defines.
A content-based recommender system matches the profile of the
item to the user profile to decide on its relevancy to the user.
20. SIMPLE EXAMPLE
Read update
User Profile
New books Match User Profile
Recommender
Systems recommendation
22. Context-Based Recommender Systems
abcd
Overview
The recommender system uses additional data about the context of an item
consumption.
For example, in the case of a restaurant the time or the location may be used to
improve the recommendation compared to what could be performed without this
additional source of information.
A restaurant recommendation for a Saturday evening when you go with your
spouse should be different than a restaurant recommendation on a workday
afternoon when you go with co-workers
13.10.2012
23. Context-Based Recommender Systems
Motivating Examples
Recommend a vacation
Winter vs. summer
Recommend a purchase (e-retailer)
Gift vs. for yourself
Recommend a movie
To a student who wants to watch it on Saturday
night with his girlfriend in a movie theater.
13.10.2012
24. Context-Based Recommender Systems
Motivating Examples
Recommend music
The music that we like to hear is greatly affected by a context, such
that can be thought of a mixture of our feelings (mood) and the
situation or location (the theme) we associate it with.
Listen to Bruce Springteen "Born in USA" while driving along the 101.
Listening to Mozart's Magic Flute while walking in Salzburg.
13.10.2012
25. Information Discovery: Example
“Tell me the music that I want to listen NOW"
abcd abcd
Musicovery.com Details
An Interactive personalized
WebRadio
A mood matrix propose a
relationship between music and
mood.
Ethnographic studies have
shown that people choose
music peaces according to their
mood or mood change
expectation.
13.10.2012
26. Context-Based Recommender Systems
What simple recommendation techniques ignore?
What is the user when asking for a recommendation?
Where (and when) the user is ?
What does the user (e.g., improve his knowledge or really buy
a product)?
Is the user or with other ?
Are there products to choose or only ?
13.10.2012
27. Context-Based Recommender Systems
What simple recommendation techniques ignore?
What is the user when asking for a recommendation?
Where (and when) the user is ?
What does the user (e.g., improve his knowledge or really buy
a product)?
Is the user or with other ?
Are there products to choose or only ?
Plain recommendation technologies forget to take
into account the user context.
13.10.2012
28. Context-Based Recommender Systems
abcd
Major obstacle for contextual computing
Obtain sufficient and reliable data describing the user context
Selecting the right information, i.e., relevant in a particular personalization task
Understand the impact of contextual dimensions on the personalization process
Computational model the contextual dimension in a more classical
recommendation technology
For instance: how to extend Collaborative Filtering to include contextual
dimensions?
13.10.2012
29. Context-Based Recommender Systems
abcd
Item Split - Intuition and Approach
Each item in the data base ( ) is a candidate for splitting
Context defines ( ) all possible splits of an item ratings vector
We test all the possible splits – we do not have many contextual
features
We choose one split (using a single contextual feature) that maximizes
an impurity measure and whose impurity is higher than a threshold
13.10.2012
31. Social Based (Trust based) Recommender Systems
abcd
Overview
Intuition – Users tend to receive advice from people they trust, i.e., from their
friends.
Trusted friends can be defined explicitly by the users or inferred from social
networks they are registered to.
.
13.10.2012
33. TRUST METRICS
Global metrics: computes a single global trust value for
every single user (reputation on the network)
b
Pros: d
a
Based on the whole community opinion
c
Cons:
Trust is subjective (controversial users)
34. TRUST METRICS (CONT.)
Local metrics: predicts (different) trust scores that are
personalized from the point of view of every single user
Pros:
More accurate
Attack resistance
Cons:
Ignoring the “wisdom of the crowd” b
a d
c
36. SEARCH ENGINES VS. RECOMMENDER SYSTEMS –
Search Engines Recommender Systems
Goal – answer users ad Goal – recommend services
hoc queries or items to user
Input – user ad-hoc need Input - user preferences
defined as a query defined as a profile
Output- ranked items
relevant to user need Output - ranked items based
(based on her on her preferences
preferences???)
Methods - Mainly IR
Methods – variety of
based methods methods, IR, ML, UM
37. NEW TRENDS …
“Understand” the user actual needs from her context
Personalize results according to the user preferences
Search engines may use some recommender systems
methods to achieve these goals
38. SEARCH ENGINES PERSONALIZATION METHODS
ADOPTED FROM RECOMMENDER SYSTEMS
Collaborative filtering
User-based - Cross domain collaborative filtering is
required???
Content-based
Search history
Collaborative content-based
Collaborate on similar queries
Context-based
Little research – difficult to evaluate
Locality, language, calendar
Social-based
Friends I trust relating to the query domain
Notion of trust, expertise