Extracting Semantics from Crowds

Knowledge Management Institute

Extracting Semantics from Crowds
Stanford Center for Biomedical Informatics Research, Colloquium
Stanford University, Palo Alto, CA
Jan 6th, 2010

Markus Strohmaier

Assistant Professor
Graz University of Technology, Austria

Visiting Scientist
(XEROX) PARC, USA
Markus Strohmaier 2011
1


with …
D. H li C Kö
D Helic, C. Körner, J. Pöschko, C. W
J Pö hk C Wagner
(Graz University of Technology, Austria)
D. Benz, G
D Benz G. Stumme
(U. of Kassel, Germany)
A. Hotho
(U. of Würzburg, Germany)
S. Hellmann, J. Lehmann, C. Stadler, J. Unbehauen
(U. Leipzig,
(U of Leipzig Germany)

2


Classification Systems
in Information and Library Sciences

Usually produced and maintained by few
(e.g.
(e g dozens of) domain experts
experts.

but: used by many (potentially millions).

Can a very large group (a crowd) of users
contribute to ontology engineering efforts?

3


Crowdsourcing Semantics:
An Experiment (2008)

LoC
L C posted 3000 photos
t d h t
to flickr:

24 hours after launch:
• over 4,000 unique tags
• about 19,000 tags added

Output? noisy,
How can this data contribute to the
unstructured, unqualified,
induction of semantic structures? weak semantics

4


Research Agenda
Vision:
Vi i
Utilizing online behavior of crowds for
the construction, maintenance and enrichment
construction
of large-scale semantic structures.

Mission:
• to model behavior of large numbers (millions) of users online
• to develop techniques and algorithms that acquire semantic structures
from users‘ interaction with data
• to influence user behavior and emerging semantics
• to evaluate results

5


Extracting Semantics from …

Motivation

Social Labeling
• Hashtag Semantics
Social Tagging
• Tag Relatedness
• Tag Generality
• Tag Hierarchies
Social Navigation
• Navigational Knowledge
Engineering
g g

6


Activities of Users Online
Users engage in…

Labeling Tagging Navigation

Can we tap into the outcome of these
activities to extract semantic structures?
7



Motivation

Social Labeling
Social Tagging
• Tag Relatedness
• Tag Generality
• Tag Hierarchies
Social Navigation
Engineering
g g

8


Social Labeling
Example: Twitter
users label short messages
with concepts (hashtags)

Which hashtags behave as strong identifiers (if any), and could
they be mapped to concept identifiers in the Semantic Web
(URIs)?
[Laniado and Mika 2010]

9

Markus Strohmaier

with
and C.Wagner
2011 J. Pöschko
ConceptNet
developed by

C. Wagner, M. Strohmaie The Wisdom in Tw
W er, weetonomies: Acquiri ing Latent Conceptua Structures from So
al ocial Awareness Stre
eams,
10

Semmantic Search 2010 Workshop (SemSearch2
W 2010), in conjunction with the 19th Internatio
w onal World Wide Web Conference (WWW20
C 010),
Rale
eigh, NC, USA, April 26-30, ACM, 2010.
2

Markus Strohmaier

such a network?

p
with
X is a subconcept of Y
e.g. X is more general than Y,

and C.Wagner
2011 J. Pöschko
ConceptNet

Can we qualify semantic associations in
ent
Ev-
ent
Ev-
developed by

C. Wagner, M. Strohmaie The Wisdom in Tw
W er, weetonomies: Acquiri ing Latent Conceptua Structures from So
al ocial Awareness Stre
eams,
11

Semmantic Search 2010 Workshop (SemSearch2
W 2010), in conjunction with the 19th Internatio
w onal World Wide Web Conference (WWW20
C 010),
Rale
eigh, NC, USA, April 26-30, ACM, 2010.
2



Motivation

Social Labeling
Social Tagging
• Tag Relatedness
• Tag Generality
• Tag Hierarchies
Social Navigation
Engineering
g g

12


Social Tagging
Example: Delicious
Users label and categorize
Resources resources with concepts (tags)

Tags

Users
U

A folksonomy is a tuple F:= (U, T, R, Y) where
• th th
the three di j i t fi it sets U T R correspond t
disjoint, finite t U, T, d to user 1

– a set of persons or users u ∈ U
– a set of tags t ∈ T and
– a set of resources or objects r ∈ R tag 1 res. 1
• Y ⊆ U ×T ×R, called set of tag assignments

13


Tag Relatedness

Different
tag similiarity
measures
applied to del.icio.us

C. Cattuto, D. Benz, A. Hotho, G. Stumme, Semantic Grounding of Tag Relatedness in Social Bookmarking Systems, 7th International Semantic Web Conference
ISWC2008, LNCS 5318, 615-631 (2008).
14


Tag Generality

frequency

Del.icio.us

with D. Benz
et al.
D. Benz, C. Körner, A. Hotho, G. Stumme, M. Strohmaier, One Tag to bind them all: Measuring Term Abstractness in Social Metadata, Submitted to ESWC 2011.

15


Tag Hierarchy

TP..Taxonomic TR..Taxonomic TF..Taxonomic TO…Taxonomic
precision recall F measure overlap

Different folksonomy algorithms
with D. Helic
et al.
M. Strohmaier, D. Helic, D. Benz. Evaluation of Folksonomy Induction Algorithms. Submitted to the ACM Transactions on Intelligent Systems and Technology, (2011).

16



Motivation

Social Labeling
Social Tagging
• Tag Relatedness
• Tag Generality
• Tag Hierarchies
Social Navigation
Engineering
g g

17


Social Navigation
Users search for and navigate
to certain sets of resources

Can we turn ordinary users into contributors for
ontology enrichment?
t l i h t?
Navigational Knowledge Engineering:
A light-weight methodology for low-cost
knowledge engineering by a massive user base.

18


Prototype: HANNE
HANNE: Holistic Application for Navigational
HANNE H li ti A li ti f N i ti l
Knowledge Engineering
http://aksw.org/Projects/NKE
http://aksw org/Projects/NKE

with Sebastian Hellmann et al., University of Leipzig
19


Navigational Knowledge Engineering

Example: Extending DBPedia with NKE

S. Hellmann, J. Lehmann, C. Stadler, J. Unbehauen, M. Strohmaier, Navigational Knowledge Engineering, Submitted to WWW 2011. 20


Navigational Knowledge Engineering

Choose initial positive and
negative examples from the
search result.
Here we are looking for
Football Clubs in Saxony, a
region in German
Germany.

21


Inductive Concept Learning

To test the coverage, the function

Returns the value true if e is covered by H, and false otherwise.

Nada Lavrac and Saso Dzeroski, Inductive Logic Programming: Techniques and Applications, 1994 22


HANNE &
The Problem of Inductive Concept Learning

given:
• Background knowledge (OWL/DL knowledge base)
• positive and negative examples of a concept
(instances)
(i t )

find:
fi d
• A hypothesis (expressed as OWL class descriptions)
that covers all positive and no negative examples

23


Based on the
extension, ICL
searches for
suitable
hypotheses.



Concepts Learned

• The Learned C
Concept is shown in Manchester O OWL SSyntax
• The user can retain the concept for later retrieval.
• Saved concepts are displayed as social navigation
suggestions. Can be used to enrich existing knowledge
base.
base


Result
Useful properties:
• Biased towards high recall
• Scales well: Number of training
examples is more important than the
size of the background knowledge

With only 2 positives and 4 negatives,
it is possible to find 13 more
instances, which are f tb ll clubs
i t hi h football l b
situated close to Saxony, Germany



Mock up (1)

27


Summary
We
W can observe that:
b th t
• semantic structures can be obtained as a byproduct
of online crowd behavior (t i categorizing, navigation)
(tagging, t i i i ti )
• these structures can approximate structures in
reference knowledge bases (DBPedia WordNet etc)
(DBPedia, WordNet,
but:
• different levels of scale mean different degrees of
formality and quality of semantic structures
• pragmatics influences resulting semantics

28


Outlook
Current and planned collaborations:
C t d l d ll b ti
(XEROX) PARC, USA and U. Würzburg, Germany:
• PoSTS: Interactions between pragmatics and semantics in social tagging systems
• DFG/FWF coop. project proposal 2011 2014 (~400.000 EUR, under review)
2011-2014 ( 400.000
• Visiting Scholar with (XEROX) PARC in 2010/2011

Media-X, Stanford U., USA:
• CALHWIN: Implementation of a social health information network for California
Semantic analysis of social media
• NSF project proposal 2011-2014 (under review)

U. Of Leipzig, Germany:
• Navigational Knowledge Engineering

Related events I am involved in:
• ACM SIGWEB Working Group on Social Media (Co-chair)
• ACM Hypertext 2011 track „Social Media“ (Co-chair)
• WWW‘2011 Workshop on Usage Analysis and the Web of Data (PC member)
W k h U A l i d th W b f D t b )

29


A N What‘s the fknowledge of SAS? h
New A s d for S
What
Agenda Semantic R
ti Research

7 January, 2011
Web Resources
Natural Language Constructs
Real-World Happenings
Utilizing online behavior of crowds for

30
the construction, maintenance and enrichment of
large-scale semantic structures
http://www.flickr.com/photos/waldoj/722508166/
http://www.flickr.com/photos/matthewfield/2306001896/ 30


Thank You.

Acknowledgements

collaborators and co-authors:

D. Helic, C. Körner, J. Pöschko, C. Wagner (Graz U. of Technology, Austria)
D. Benz, G. Stumme (U. of Kassel, Germany)
A. Hotho (U. of Würzburg, Germany)
S. Hellmann, J. Lehmann, C. Stadler, J. Unbehauen (U. of Leipzig, Germany)

31

Extracting Semantics from Crowds

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Extracting Semantics from Crowds

Semelhante a Extracting Semantics from Crowds (20)

Último

Último (20)

Extracting Semantics from Crowds