In the culture domain, questionnaires are often used to obtain profiles of users for adaptation. Creating questionnaires requires subject matter experts and diverse content, and often does not scale to a variety of cultures and situations. This paper presents a novel approach that is inspired by crowdwisdom and takes advantage of freely available structured linked data. It presents a mechanism for extracting culturally-related facts from DBpedia, utilised as a knowledge source in an interactive user modelling system. A user study, which examines the system usability and the accuracy of the resulting user model, demonstrates the potential of using DBpedia for generating culture-related user modelling questionnaires and points at issues for further investigation.

1. UMAP – 08 July 2014 University of Leeds
Using DBpedia as a Knowledge Source for Culture-
Related User Modelling Questionnaires
Dhaval Thakker (Uni. Of Leeds/UoL)
Lydia Lau(UoL), Ronald Denaux(iSOCO, Spain), Vania Dimitrova(UoL),
Paul Brna(UoL) and Christina Steiner(TU Graz, Austria)

2. UMAP – 08 July 2014 University of Leeds
Outline
• Definitions
• Problem: Modelling of learner's intercultural
awareness
• Approach: interactive dialogue exploiting Linked
data
• Evaluation
• Summary

3. UMAP – 08 July 2014 University of Leeds
t
Kashima, Y. Conceptions of Culture and Person for Psychology. Journal of Cross-Cultural
Psychology. 31, 1 (2000), 14-32.
Gupta, V., Hanges, P.J., Dorfman, P. Cultural clusters: methodology and findings. Journal of
World Business, 37,2 (2002) 11-15.
Culture can be defined as a set of beliefs, values, behaviours
and practices that characterise a given group of people
[Kashima, 2000]
Broad notion of culture
Nationality and countries have been used as fairly reliable
indicators for tackling cultural diversity
[Gupta et al, 2002; Globe clusters; Hofstede dimensions]
Narrowed scope: National Culture
Definitions

4. University of LeedsUMAP – 8 July 2014
Problem
• Cultural Awareness –
key 21st Century skills
• User-adaptive
situational simulators
are effective for training
- first encounter
• BUT suffer from
COLD START
Situational simulators
- Helps users in developing intercultural competencies
- Provide User- Adaptive Learning Experience
- Take into account learner’s knowledge of cultures

5. University of LeedsUMAP – 8 July 2014
Main Challenge
How to obtain a profile of a user’s knowledge of
cultural aspects for different countries?
Questionnaires and situational tests
• Do not scale
• Are not engaging
• Do not handle diversity
Use the crowdwisdom from Wikipedia
DBPedia – semantic version of Wikipedia
• Large, multi-domain knowledge base

6. UMAP – 8 July 2014 University of Leeds
Our Approach
Perico Dialogue Agent
Knowledge
Probing
Quiz Generator
User Profile
GeneratorDialogue Planner
User Model
• Cultural
Exposure Score
Knowledge
Pool
Challenge 2: Extract Knowledge Pool from DBpedia
How to generate relevant knowledge pool from huge knowledge
such as DBpedia?
Challenge 1: Knowledge probing to assess learner's
knowledge of a country

7. UMAP – 8 July 2014 University of Leeds
Extract Knowledge Pool
from DBpedia
Karanasios, S., Thakker, D., Lau, L., Allen, D., Dimitrova, V., & Norman, A. (2013). Making sense of digital traces: An activity theory
driven ontological approach. Journal of the American Society for Information Science and Technology,(JASIST) 64(12), 2452-2467.

8. UMAP – 08 July 2014 University of Leeds
Distilling Intercultural Facts
from DBpedia
Identify DBpedia categories
for seed terms (Clothing)
Find specific categories and
sub-categories (Loden_cape)
Find broader categories
(German_Culture; Germany)
Generate facts
“Loden_cape is a Clothing”
“Loden_cape occursIn Germany”

9. UMAP – 8 July 2014 University of Leeds
Extracted Knowledge Pool
• 40K facts (OWL logical axioms)
• 565 items of clothing
• >4K items of food
• 88 gestures
• 159 currencies
• 288 languages
• 20K annotations (labels and depictions)
Available at : http://imash.leeds.ac.uk/ontologies/amon/

10. University of LeedsUMAP – 8 July 2014
Probing and Modelling
User’s Knowledge
• Goal: Assess learner's (socio-political and
intercultural) knowledge of country
• Ask facts (derived) from DBpedia
• Ask “trick” questions: close world around country
• Mark answers based on expected truth value and add to
Country-awareness Profile
Ronald Denaux, Vania Dimitrova, Lydia Lau, Paul Brna, Dhavalkumar Thakker, Christina
M. Steiner: Employing linked data and dialogue for modelling cultural awareness of a
user. IUI 2014: 241-246

11. University of LeedsUMAP – 8 July 2014
Interaction with Perico
http://imash.leeds.ac.uk:8080/perico/
https://www.youtube.com/watch?v=GTUtKSFjdLo

12. University of LeedsUMAP – 8 July 2014
Evaluation Study
Q1: Usability and interaction
Is Perico usable and intuitive for the intended users; and
what are the possible limitations of the interaction with
Perico?
Q2: UM accuracy
Is the user model produced by Perico accurate against the
user’s perception of his/her knowledge in the selected
cultural aspects?

13. University of LeedsUMAP – 8 July 2014
Evaluation Studies
Participants
- 22 participants (age
mean=28),
- Using GLOBE:
- Group1 – Narrow
Cultural Exposure
- Group 2 – Broad
Cultural Exposure
Method
- Online
- Interaction with Perico –
four countries(none,
low, med, high)
familiarity
- Total 92 questions in
total
User Model
- Aggregated User
model for each country
and each topic
- not-good, need-
improvement, ok, very
good
- Participant rating ->
accurate, under/over
estimated
Post-study
- SUS usability
questionaries'
Q1: Usability and interaction
Q2: UM accuracy

14. University of LeedsUMAP – 8 July 2014
Usability
three questions were tailored to Perico’s interaction:
(SUS11) “The questions asked during the dialogue were easy to understand”;
(SUS12) “The instructions provided during the dialogue were clear”; and
(SUS13) “The assessment made by the dialogue was correct”.
Answering 92 questions
in 30-45 minutes,
the mean dialogue-score
indicates good quality.
System was easy to learn and did not require
Additional support
Integration &
Consistency

15. University of LeedsUMAP – 8 July 2014
Qualitative Feedback
Deficiencies of DBpedia Pool
- ‘People in Cyprus use a garment called Icknield High School’.
- ‘Frank is currency used in Germany
Did not take globalisation into account
- food, clothing, gestures have become common in countries which they
did not originate from.
- Inadequate assertions are hard to detect automatically.
Possible solution
- Allowing users to indicate that something is wrong
- filtering or extending of the extracted DBpedia fact pool.

16. University of LeedsUMAP – 8 July 2014
Interaction Feedback
Limited content:
- Some countries are prominent – e.g. gestures coming largely from USA;
- or that for some countries, e.g. Jordan, the dialogue presented mainly facts related
to other countries.
Possible solution
- Expansion of topics: such as capital, population, climate, religion, festivals,
popular sports, points of interest, were suggested.
Lacking coherence:
- interaction was jumping from question to question and lacked structure
- due to the random selection from the pool of possible axioms
Possible solution
- to follow the GLOBE clusters
- probing the knowledge on countries in the same cultural cluster, broadening, i.e.
exploring countries from different clusters,
- Or comparing, ‘How does Italy’s income inequality compare to the UK –
higher/lower?’).

17. University of LeedsUMAP – 8 July 2014
UM Accuracy
None or Low
Exposure
Medium or High
Exposure
Overestimation 17% 3%
Underestimation 7% 12%
Accurate 76% 86%

18. University of LeedsUMAP – 8 July 2014
UM Feedback
Answer indicated in the question
-‘Indian Rupee’, ‘Japanese Yen’, ‘Bulgarian Lev’, ‘Polish Zloty’
Possible solution
- use rdfs:label (name without the country)
- use dbpprop:nickname (e.g. ‘kint’ instead of ‘Bulgarian lev’).
Answer given via knowledge elimination
Able to answer the questions by knowing the facts about other countries they
knew (canada -> china)
(both cases) Possible solution
(i) Explicitly ask if the user knew or guessed the answer; and
(ii) ask for additional justification or explanation.
Answer given using a clue in the question

19. University of LeedsUMAP – 8 July 2014
Lessons Learnt
Using Linked data for
domain specific tasks
Extensibility
Missing or Dated
Combine: give users
opportunity to enter
knowledge
Gamifying
questionnaires
Engaging (>30 mis)
Variety
Examples
Media
Conversation-like
Structure (GLOBE)
Thanks!
Dr.Dhaval Thakker
Research Fellow
Artificial Intelligence Group
University of Leeds, UK
http://dhavalthakker.wordpress.com/
Twitter: @dr_thakker