1. Ten ways to make your semantic
app addictive
Elena Simperl
ISWC 2010
www.insemtives.eu 1

2. Executive summary
• Many aspects of semantic content authoring naturally rely
on human contribution.
• Motivating users to contribute is essential for semantic
technologies to reach critical mass and ensure sustainable
growth.
• This tutorial is about
– Methods and techniques to study incentives and motivators
applicable to semantic content authoring scenarios.
– How to implement the results of such studies through
technology design, usability engineering, and game mechanics.
www.insemtives.eu 2

3. Our approach
• Typology of semantic content authoring tasks and
the ways they could motivate users to contribute.
• Methodology for analyzing and designing
incentivized semantic applications.
• Pilots, showcases and technology.
www.insemtives.eu 3

4. Incentives and motivation
• Incentives are ‘rewards’ assigned by an external ‘judge’
to a performer for undertaking a specific task.
• Common belief (among economists): incentives can be
translated into a sum of money for all practical purposes.
• Incentives can be related to both extrinsic and intrinsic
motivations.
– Extrinsic motivation if task is considered
• Boring, dangerous, useless, socially undesirable, dislikable by
the performer.
– Intrinsic motivation if
• The performer likes what he/she is doing
• The act is satisfying in itself (for various reasons).
www.insemtives.eu 4

5. Examples (i)
www.insemtives.eu 5

6. Examples (ii)
www.insemtives.eu 6

7. Examples (iii)
www.insemtives.eu 7

8. Harnessing human intelligence
Facebook reports 4,000,000,000 minutes (> 7500
person years) are spent on the site every day
More than 18 000 titles
preserved in 10 years
Recaptcha users solve 60 million CAPTCHAs
a day, which accounts for around 160,000
human hours (19 person years)
www.insemtives.eu 8

9. Harnessing human intelligence
www.insemtives.eu 9

10. Which tasks can be crowdsourced?
• Modularity/Divisibility: • Combinability
can the task be divided – Additive: pulling a rope
(group performs better than
into smaller chunks (see individuals, but each
casual games, Amazon’s individual pulls less hard)
Mechanical Turk, open – Conjunctive: running in a
pack (performance is that of
source software) the weakest member, group
• Skills and expertise: size reduces group
performance)
does the task address a
– Disjunctive: answering a quiz
broad audience (see (group size increases group
CAPTCHAs, casual performance in term of the
time needed to answer)
games)
www.insemtives.eu 10

11. Example: video annotation
www.insemtives.eu 11

12. Example: ontology alignment
www.insemtives.eu 12

13. Example: ontology evaluation
www.insemtives.eu 13

14. Typology of semantic content
authoring tasks: Table from D1.2.2
www.insemtives.eu 14

15. Challenges
• Task selection, work breakdown and distribution
of labor
• Domain selection and creation of knowledge
corpus
• Deriving formal representations from user inputs
• Technology design
• Intrinsic motivations
www.insemtives.eu 15

16. Factors affecting participation
• prize is higher
• participants are more intrinsically motivated ,
have more free time, are non-experts in the field ,
and are not participating due to career concerns,
social motivations, or to beat others.
(Lakhani et al, 2007)
www.insemtives.eu 16

17. Outline of the tutorial
Time Presentation
09:00 – 09:30 Human contributions in semantic content authoring
09:30 – 10:30 Methods and techniques to analyze and design incentivized semantic
applications
10:30 – 11:00 Coffee break
11:00 – 12:30 Guidelines for incentivized technology design
12:30 - 13:30 Lunch break
13:30 – 14:30 Casual games for semantic content creation
14:30 – 15:00 Hands-on (Part I)
15:00 – 15:30 Coffee break
15:30 – 17:00 Hands-on (Part II)
17:00 – 17:30 Wrap-up and closing
www.insemtives.eu 17

18. Realizing the Semantic Web by
encouraging millions of end-users
8/10/2011 create semantic content.
to www.insemtives.eu 18