Doctoral Consortium at European Conference on Technology Enhanced Learning (EC-TEL 2013 DC), September 17, 2013, Paphos (Cyprus)

1. Adaptive Support
for Educational Question Answering
17th September 2013
EC-TEL 2013 DC
Ivan Srba
Supervisor: Mária Bieliková
Faculty of Informatics and Information Technologies
Slovak University of Technology in Bratislava

2. 2
• How can Internet users satisfy their information needs?
– Web search engines (e.g. Google, Bing)
– Community-based knowledge sharing applications (e.g. forums,
social networking sites, community question answering)
Community Question Answering (CQA)

3. 3
• How can Internet users satisfy their information needs?
– Web search engines (e.g. Google, Bing)
– Community-based knowledge sharing applications (e.g. forums,
social networking sites, community question answering)
• Community Question Answering (CQA) is a web service
where people can:
– seek information by asking a question and
– share knowledge by providing answer on the particular question
Community Question Answering (CQA)

4. 4
1. Question creation
2. Question routing
3. Question answering
4. Question search
Question Answering Process in a Nutshell

5. 5
Knowledge sharing Collaborative learning
Different Perspectives on CQA
Knowledge Management
Knowledge Embedded in
Communities
Communities of Practice
Online Communities of Practice
Technology-Enhanced Learning
Computer-Supported
Collaborative Learning
Peer Learning

6. 6
• Analyses of more than
70 research studies
– Adaptive approaches
which concern with open
problems in CQA
– Categorization of
approaches
State of the Art: CQA Approaches
CQA
Approaches
Domain Entities
Analysis
Question Routing
Question Search
Question
Answering
Question Creation
Question and User Clasification
Question Retrieval
User Expertise Estimation
Topic Hierarchy Maintenance
Automatic Topic Assignment
Answer Ranking
Answer Summarization
Voting Calibration
User Activity Estimation
Content Quality Estimation
Asker Satisfaction Estimation
Bridging the Gap
between Web Search and CQA
User Motivation Estimation

7. 7
• Question answering in intra-organizational context
– Educational, business or research organizations
– We focus on educational organizations (universities)
• Question routing
– Recommendation of the most appropriate answerers
Research Scope of the Dissertation Project
CQA
Approaches
Domain Entities
Analysis
Question Routing
Question Search
Question
Answering
Question Creation
User Expertise Estimation
User Activity Estimation
User Motivation Estimation

8. 8
• Q1: Identification of Specific Organizational Conditions
– Organizational and educational environment is different in
comparison with standard CQA
• The number of users is significantly lower, overloading users
• The great amount of accessible information about users
• The presence of a supervisor
• Students’ knowledge changes significantly in time
• Shift of partial responsibility for learning from a teacher to
students themselves
What are the specific conditions in intra-organizational educational
environments and how these specifics influence collaboration
during question answering process?
Organizational Question Answering

9. 9
Organizational Question Routing
• Q2: Proposal of Organizational Question Routing Methods
– Existing question routing methods focus primarily on knowledge
sharing perspective
• Achieve the best answer in the shortest possible time
• Involve mainly the experts in providing answers
– In organizational context, collaborative learning perspective is
important as well as knowledge sharing perspective
• Achieve the best answer as the result of learning process
• Harness a potential of the whole community
How can we propose new approaches for question routing to match the
specifics of intra-organizational educational environments?

10. 10
• Q3: Designing Organizational CQA Systems
– Educational environment requires additional learning support
• Dashboards with visualizations of students’ profiles
• Multimedia integration into questions and answers (e.g. UML
diagrams)
Which additional/modified functionalities should be provided by intra-
organizational educational CQA system?
Organizational CQA System

11. 11
• Q1: Identification of Specific Organizational Conditions
– Study on influence of students’ personal characteristics on
collaborative learning
• Dataset obtained during a long-term experiment (12 weeks)
• 129 students, 254 dynamic groups, 3,763 collaborative activities
• Students solved short-term questions prepared by a teacher
• Two connected learning systems ALEF and PopCorm
– Implications for CQA:
• Better students can become a driver in collaboration mediated by
students themselves
• Students are able to argument and reach consensus
• …
Preliminary Results

12. 12
Adaptive Learning Framework (ALEF)

13. 13
Popular Collaborative Platform (PopCorm)

14. 14
• Q1: Identification of Specific Organizational Conditions
– Study on influence of students’ personal characteristics on
collaborative learning
• Dataset obtained during a long-term experiment (12 weeks)
• 129 students, 254 dynamic groups, 3,763 collaborative activities
• Students solved short-term questions prepared by a teacher
• Two connected learning systems ALEF and PopCorm
– Implications for CQA:
• Better students can become a driver in collaboration mediated by
students themselves
• Students are able to argument and reach consensus
• …
Preliminary Results

15. 15
• Q2: Proposal of Organizational Question Routing Methods
– User expertise
• Competition-based expertise network: pairwise comparison of users
• An answerer who provided the best answer has higher expertise in
comparison with all other non-best answerers
– User activity
• Overall users’ activity, availability and authority
• Implications from the study on influence of users’ characteristics
– User motivation
• Knowledge bartering: considering the symmetry in knowledge each
student provides and receives
Preliminary Results

16. 16
• The first preliminary phase
– State of the art
– Initial identification of organizational specifics
• The current second phase
– Designing and implementation of educational CQA system
– Long-term experiment with limited number of students
– Verification of specifics (Q1) and enhanced CQA system (Q2)
• The next third phase
– Proposal of the new method to question routing
– Synthetic tests followed by long-term experiment (Q3)
– Conclude results and derive implications
Research Methodology

17. 17
• Case study of employing CQA in educational environment
– Valuable input for further research
• Proposal of the new approach to question routing
– Considers the organizational specifics
– Evaluation in long-term experiments at our faculty
• Open problems
– What is the ideal status in educational question answering?
• Fast answers, symmetry in activities, …
– Which specifics should be considered by the proposed method?
• Organizational and educational specifics
– How should they influence question routing?
• User expertise, activity and motivation
Contributions and Conclusion