Linked Open Data-enabled Strategies for Top-N Recommendations

CBRecSys 2014
Workshop on New Trends in
Content-based Recommender Systems
Foster City (CA, United States)
October 6, 2014
Linked Open Data-enabled
Strategies for Top-N
Recommendations
Cataldo Musto, Pierpaolo Basile, Giovanni Semeraro, Pasquale Lops, Marco de Gemmis
(Università degli Studi di Bari ‘Aldo Moro’, Italy - SWAP Research Group)

Outline
• Background
• Content-based RecSys (CBRS)
• Limitations
• Linked Open Data
• What?
• Introducing LOD in CBRS
• Experiments
• Conclusions
Cataldo Musto, Pierpaolo Basile, Giovanni Semeraro, Pasquale Lops, Marco de Gemmis.
2 Linked Open Data-enabled Strategies for Top-N Recommendation. CBRecSys 2014 Workshop, Silicon Valley (US), 6.10.2014

Suggest items similar to those the user liked in the past
(I bought Converse shoes, I’ll continue buying similar sport shoes)

Limitations
Limited content
4
(in several domains)
Linked Open Data-enabled Strategies for Top-N Recommendation. CBRecSys 2014 Workshop, Silicon Valley (US), 6.10.2014

Limitations
Poor Semantics

How can we boost
Content-based
Recommender Systems
with Semantics?
(and with more content)
6
Problem

7
Semantics in CBRS State of the art
Ontologies X
Folksonomies Distributional Semantics
Encyclopedic Knowledge Linked Open Data

8
Top-down approaches
What is the difference?
X
Formal Semantics Large-scale
Folksonomies X X
Ontologies V X
Encyclopedic Knowledge X V
Distributional Semantics X V
Linked Open Data V V

9
Top-down approaches
X
Folksonomies X X
Ontologies V X
Linked Open Data merge the vastness of encyclopedic knowledge
with the formal semantics typical of ontologies

10
Top-down approaches
X
We focus on the introduction of
Folksonomies X X
Linked Open Data in
Ontologies V X
Content-based Recommender
Systems
Linked Open Data merge the vastness of encyclopedic knowledge
with the formal semantics typical of ontologies

11
Linked Open Data
What are we talking about?

12
Linked Open Data
Definition
Methodology to publish, share and link
structured data on the Web

13
Linked Open Data (cloud)
What is it?
A (large) set of interconnected semantic datasets

14
What kind of datasets?

15
DBpedia
http://dbpedia.org

16
http://dbpedia.org
DBpedia
DBpedia is the structured mapping of Wikipedia
It is the core of the LOD cloud.

17
Example: unstructured content from Wikipedia
example
“Foster City is a town in United States located in California”
(from Wikipedia page)

18
How are these data represented?
Semantic Web cake
Information from the
LOD cloud is
represented in RDF

19
Foster City United States
http://dbpedia.org/resource/United_States
California
http://dbpedia.org/resource/Foster_City,_California
http://dbpedia.org/resource/California
dbpedia-owl:country
dbpedia-owl:isPartOf
example

20
Data coming from the LOD cloud have a
formal semantics represented in RDF
Foster City United States
http://dbpedia.org/resource/United_States
California
http://dbpedia.org/resource/Foster_City,_California
http://dbpedia.org/resource/California
dbpedia-owl:country
dbpedia-owl:isPartOf
example

21
Our checklist
Can Linked Open Data boost
content-based recommender systems?
More Semantics More Content
V ?

22
How many data?

23
How many data?
1048 datasets and 58 billions triples
source: http://stats.lod2.eu

24
Our checklist
V V

25
Our checklist
V V
…but

26
Research Question

27
Approach
We propose two methodologies to
introduce LOD-based features into CBRS
Direct Access to DBpedia Entity Linking algorithms

Introducing LOD-based features in CBRS
28
Methodology :: Direct Access to DBpedia
(We assume that each item to be recommender is already in the LOD cloud)
The simplest way to introduce LOD-based features
Domain-dependent features
are manually defined
1.
2.
(e.g. book recommendation —> genre, author, publisher, subject, etc.)
SPARQL queries extract features’ values

Example: The Great and Secret Show (Clive Barker’s book)
29

30
e.g. Book Recommendation: author, genre, publisher, subject

31
Each item is represented through the set of the (manually defined)
features extracted from the LOD cloud.

32
9 LOD-based features: author (Clive Barker), genre (Fantasy Literature), publisher (William
Collins), series (Books of the Art), subject (1980s fantasy novels, William Collins books,
Novels by Clive Barker, British Fantasy Novels)

33
Direct Access to DBpedia
Analysis
- Very Straightforward approach
- SPARQL queries can be easily built
- Properties are manually defined
- Approach is strongly domain-dependent
- Does not exploit unstructured information
Pros:
Cons:

Methodology :: Entity Linking algorithms
• Entity Linking Algorithms!
• Input: free text.
• items description, in our setting
• Output: identification of the most
relevant entities mentioned in the text.
• State of the art
• tag.me(1),
• DBpedia Spotlight(2),
• Wikipedia Miner(3)
(1) http://tagme.di.unipi.it
(2) http://spotlight.dbpedia.org
(3) http://wikipedia-miner.cms.waikato.ac.nz
Cataldo Musto, Pierpaolo Basile, Giovanni Semeraro, Pasquale Lops, Marco de Gemmis. 34

• items description, in our setting
• Output: identification of the most
relevant entities mentioned in the text.
• State of the art
• tag.me(1),
• DBpedia Spotlight(2),
• Wikipedia Miner(3)
(1) http://tagme.di.unipi.it
(2) http://spotlight.dbpedia.org
(3) http://wikipedia-miner.cms.waikato.ac.nz

36
• in this setting: textual
description of the items (e.g.
Wikipedia abstract)
• Output: identification of
the most relevant entities
mentioned in the text.
from Tagme

Entity Linking - output
37
Very human-readable representation!
Free n-grams and entity recognition, free sense disambiguation

Entity Linking - output
not a simple textual feature!
Each entity is a reference to a DBpedia node
http://dbpedia.org/resource/Harry_D'Amour
38

LOD-based representation can be enriched!
through broader categories by exploiting SPARQL queries
39
encoded in the dcterms:subject property

The final
representation of
each item is
obtained by
merging the
DBpedia nodes
identified in the
text with those the
dcterms:subjects
property refers to
(broader categories)
dbpedia nodes+
broader categories
Features =
40

41
Entity Linking Algorithms
Analysis
Pros:
Cons:
- Exploit unstructured information
- Very general approach
- May introduce unexpected
(but relevant) features
- Strong features engineering (which
ones are the best?)
- Threshold score of Entity Linking
algorithms is difficult to be set

42
LOD-based features in CBRS

Experimental Evaluation
Research Hypothesis
43
1. Which is the contribution of
the Linked Open Data features
to the accuracy of
recommendation algorithms?
2. Does the representation based
on Linked Open Data outperform
existing state-of-the-art
recommendation algorithms?

Description of the dataset
44
• Book recommendation • ESWC 2014 Challenge
Dataset (*) • 6,733 books • 6,181 users • 72,372 binary ratings
• 11.71 ratings/user • Very sparse dataset!
• Only 5.37 positive
ratings/user! (*) http://challenges.2014.eswc-conferences.org/index.php/RecSys

Feature combinations
45
• Content (crawled from Wikipedia + NLP processing)
• LOD (direct access to DBpedia)
• Entity Linking (Tagme)
• Content + LOD
• Content + Entity Linking
• LOD + Entity Linking
• All
7 combinations
for each run

Setup
46
• Evaluation of the effectiveness of LOD-based
features on varying six different
recommendation algorithms
• Vector Space Models
• VSM • BM25 • eVSM (*) • Classifiers
• Random Forests • Linear Regression • Graph-based Approaches
• PageRank with Priors
(*) C. Musto: Enhanced vector space
models for content-based recommender
systems. RecSys 2010: 361-364

Design of the Experiment :: Vector Space Models
47
User profile (built upon the
features describing the items the
user liked) used as query
Cosine Similarity to
get the most similar items

Design of the Experiment :: Classifiers
48
Random Forests learn a
classification model which is used to
predict the class (positive/negative)
of unlabeled item.!
Model is based! on the features
coming from labeled items.
Linear Regression also uses
“basic” features (e.g. positive and
negative ratings, average rating of the
user, ratio between positive and
negative ratings, etc.) to learn the
model.

Design of the Experiment :: PageRank with Priors (PRP)
graph-based representation
users, items = nodes positive feedback = edges
PageRank calculates the ‘importance’ of a node according to the
quality and the number of its connections
Equal probability is assigned to all the nodes, by default

graph-based representation
users, items = nodes positive feedback = edges
PageRank calculates the ‘importance’ of a node according to the
quality and the number of its connections
PageRank with Priors introduces a bias towards some nodes !
(in our setting, the items the user liked)

51
Several strategies to build the graph are compared
1. no-LOD.
Graph only models
users and items
2. small-LOD. Graph
expanded with new nodes
by adding basic
properties (subject,
genre, publisher, author,
etc.), of the items as well
as their relationships
3. big-LOD. Graph is
further expanded by
introducing more nodes (e.g.
other resources of the same
genre, other resources
written by the authors, etc.),
as well as their relationships
Rationale: the introduction of new nodes and
connections coming from the LOD cloud can
improve the effectiveness of the PageRank.

52
Several strategies to build the graph are compared
1. no-LOD.
Graph only models
users and items
2. small-LOD. Graph
expanded with new nodes
by adding basic
properties (subject,
genre, publisher, author,
etc.), of the items as well
as their relationships
3. big-LOD. Graph is
further expanded by
introducing more nodes (e.g.
other resources of the same
genre, other resources
written by the authors, etc.),
as well as their relationships
PRP is run and items in the test set are ranked
according to their PageRank

Recap
6 algorithms 7 set of features
• Content
• LOD
• Entity Linking
• Content + LOD
• Content + Entity Linking
• LOD + Entity Linking
• All
• VSM
• BM25
• eVSM
• Linear Regression
• Random Forests
• Page Rank With Priors

Experiment 1
54
Impact of LOD-based features.

Impact of LOD-based features :: VECTOR SPACE MODEL
CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
54,62
54,42
54,59
54,47
54,36
54,69
53,79
+0,17
+0,05
53 53,5 54 54,5 55
55
LOD-based features improve F1-measure

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
54,62
54,42
54,59
54,47
54,36
paired t-test (p<0.01)
54,69
53,79
+0,17
+0,05
53 53,5 54 54,5 55
56
Statistically significant improvement

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
54,62
54,42
54,59
54,47
+0,27
54,36
54,69
53,79
53 53,5 54 54,5 55
57
Best: LOD+Entity Linking (No Content!)

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
54,43
54,56
54,51
54,6
-1,00%
53,9
53,91
53,43
53 53,5 54 54,5 55
58
Impact of LOD-based features :: BM25
Worst (again): LOD alone

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
54,43
54,56
54,51
54,6
53,9
53,91
53,43
+0,17
53 53,5 54 54,5 55
59
Impact of LOD-based features :: BM25
Best (again): LOD+Entity Linking (With Content!)

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
52,9
53,07
52,8
53,04
53,02
53,37
52,06
+0,47
+0,17
+0,14
+0,12
51 51,75 52,5 53,25 54
60
Impact of LOD-based features :: EVSM
Introduction of LOD-based features leads to an improvement again

Experiment 1
Impact of LOD-based features :: LESSONS LEARNED FOR VSMS
61
VSM BM25 eVSM
1.
2.
LOD features alone are always the worst
configuration.
(At least) a LOD-based representation
based on Entity Linking always
improve the content alone

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
53,86
Experiment 1
53,68
53,75
53,76
53,77
53,34
53,52
+0,36
53 53,25 53,5 53,75 54
62
Impact of LOD-based features :: RANDOM FORESTS
Similar outcomes: all but LOD alone lead to improvement

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
53,86
Experiment 1
53,68
53,75
53,76
53,77
53,34
53,52
+0,36
53 53,25 53,5 53,75 54
63
Impact of LOD-based features :: RANDOM FORESTS
Content does matter: LOD+entity+content is the best

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
55,59
55,59
55,67
55,64
55,61
+0,08
55,5
55,57
55 55,25 55,5 55,75 56
64
Impact of LOD-based features :: LINEAR REGRESSION
Entity-based representation is the best one

CONTENT
LOD
ENTITY
CONTENT+LOD
CONTENT+ENTITY
LOD+ENTITY
ALL
Experiment 1
55,59
55,59
55,67
55,64
55,61
+0,08
55,5
55,57
55 55,25 55,5 55,75 56
65
Impact of LOD-based features :: LINEAR REGRESSION
BTW, smaller improvements (due to basic features?)

Experiment 1
Impact of LOD-based features :: LESSONS LEARNED FOR CLASSIFIERS
66
RF LR
1.
2.
LOD features alone never overcome the
content

Experiment 1
67
Same LR outcomes
RF
(algorithm-independent behaviour)
1.
2.
content

Experiment 1
68
Same LR outcomes
RF
(algorithm-independent behaviour)
1.
2.
content

Experiment 1
Impact of LOD-based features :: PAGERANK WITH PRIORS
+0,45
55,44
54,73
54,28
+1,16
53 54 55 56 57
69
NO-LOD
SMALL-LOD
BIG-LOD
The more LOD-based data, the best the accuracy

Impact of LOD-based features :: PAGERANK WITH PRIORS
NO-LOD
SMALL-LOD
BIG-LOD
Experiment 1
55,44
54,73
54,28
53 54 55 56 57
Drawback: more nodes produce an exponential growth of
computational costs (from 3 hours to 120 hours to run the experiment!)
70
+0,45
+1,16

[*] V. Ostuni, T. Di Noia, E. Di Sciascio, R. Mirizzi: Top-N recommendations
from implicit feedback leveraging Linked Open Data. RECSYS 2013
[+] S. Rendle, C.Freudenthaler, Z. Gantner, L. Schmidt-Thieme: BPR:
Bayesian Personalized Ranking from Implicit Feedback. UAI 2009.
Experiment 2
71
Comparison to State of the art
SPRANK (Semantic Path Ranking)[*]
BPRMF (Bayesian Personalized Ranking) [+]
U2U_CF (User to User CF)
I2I_CF (Item to Item CF)

VSM
LR
PRP
SPRANK
BPRMF
U2U_CF
I2I_CF
Experiment 2
52,27
52,28
52,24
54,12
55,67
55,44
54,69
baselines
51 52,25 53,5 54,75 56
Our best-performing configurations are considered as baseline
72
Comparison to state of the art

VSM
LR
PRP
SPRANK
BPRMF
U2U_CF
I2I_CF
Experiment 2
52,27
52,28
52,24
54,12
55,67
55,44
54,69
51 52,25 53,5 54,75 56
Classical CF techniques poorly performs (sparsity?)
73

VSM
LR
PRP
SPRANK
BPRMF
U2U_CF
I2I_CF
Experiment 2
52,27
52,28
52,24
54,12
55,67
55,44
54,69
!
-3,4%
51 52,25 53,5 54,75 56
74
+3,4% over LOD-based state of the art algorithm

VSM
LR
PRP
SPRANK
BPRMF
U2U_CF
I2I_CF
Experiment 2
52,27
52,28
52,24
54,12
+0,57
55,67
55,44
54,69
+1,55
51 52,25 53,5 54,75 56
75
Our approaches overcome Matrix Factorization
+0,32

Conclusions

Lessons Learned
INVESTIGATION ABOUT THE EFFECTIVENESS OF LINKED OPEN DATA IN
Two Solutions have been proposed.!
Direct Access to DBpedia and Entity Linking Algorithms!
!
Evaluation.!
Research Question: What is the impact of LOD-based features on
VSM, Classifiers and Graph-based Algorithms?!
All recommendation approaches significantly benefit of the
introduction of LOD-based features!
Our best-performing configurations overcomes both collaborative
and LOD-based state of the art algorithms
77
CONTENT-BASED RECOMMENDATION TASKS

Future Research
78
Evaluation against different datasets and
stronger baselines;
Better (automatic) tuning of parameters and
integration of more LOD-based datasources
Evaluation of Novelty, Diversity and
Serendipity on LOD-based
Recommendations;

questions?
Cataldo Musto, Ph.D
cataldo.musto@uniba.it

Linked Open Data-enabled Strategies for Top-N Recommendations

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (20)

Semelhante a Linked Open Data-enabled Strategies for Top-N Recommendations

Semelhante a Linked Open Data-enabled Strategies for Top-N Recommendations (20)

Mais de Cataldo Musto

Mais de Cataldo Musto (20)

Linked Open Data-enabled Strategies for Top-N Recommendations