O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Gamification of crowdsourcing tasks: What motivates a medical expert?

3.410 visualizações

Publicada em

Gamification of crowdsourcing tasks: What motivates a medical expert?

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Gamification of crowdsourcing tasks: What motivates a medical expert?

  1. 1. Gamification of crowdsourcing tasks: What motivates a medical expert? 18 July 2014 Rens van Honschooten Lora Aroyo (VU) & Robert-Jan Sips (IBM)
  2. 2. What motivates medical experts?
  3. 3. Gamification of crowdsourcing tasks: What motivates a medical expert? Why incentives for a medical expert? ● Train Watson in the medical domain ● To train cognitive systems you need data ○ Lots of it ● To get annotated data:
  4. 4. Gamification of crowdsourcing tasks: What motivates a medical expert? Crowdsourcing games to collect data ● Large amounts of data, fast and cheap [1]
  5. 5. Gamification of crowdsourcing tasks: What motivates a medical expert? Crowdsourcing games to collect data ● One thing they all have in common? ● They motivate their crowd ● The challenge: find out how to motivate busy medical experts
  6. 6. Gamification of crowdsourcing tasks: What motivates a medical expert? Dr. Detective: finding clues for a diagnosis ● Learning features
  7. 7. Gamification of crowdsourcing tasks: What motivates a medical expert? Dr. Detective: finding clues for a diagnosis ● Learning features● Most experts liked playing it ● Downsides: ○ The scoring was not clear ○ No rewards for completing a task ● Incentives ○ Competition ○ Entertainment ○ Learning ● Are these all the incentives? ● Which are the most important?
  8. 8. Gamification of crowdsourcing tasks: What motivates a medical expert? Crowdsourcing incentives (literature) Moon, H., & Baek, Y. (2009). Exploring variables affecting player’s intrinsic motivation in educational games. In Proceedings of the 17th International Conference on Computers in Education (pp. 718-722). Fang, Y., & Neufeld, D. (2009). Understanding sustained participation in open source software projects. Journal of Management Information Systems, 25(4), 9-50. Incentive Example Personal interest Interested in science, medicine, etc. Contributing to the community Helping diabetes patients to eat healthy Status & recognition Being recognized as a good doctor Personal growth Learning new thing in their medical domain Enjoyment Solving challenging medical problems Choice autonomy Choosing what to do and when, at work A monetary reward Receiving money for helping patients
  9. 9. Gamification of crowdsourcing tasks: What motivates a medical expert?
  10. 10. Gamification of crowdsourcing tasks: What motivates a medical expert? Helping patients Improve medical field
  11. 11. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Can we incorporate these incentives to motivate the experts? ● Created a design for next version of Dr. Detective, called Dr. Watson ○ evaluated mockup by interviewing and testing with 6 experts
  12. 12. Gamification of crowdsourcing tasks: What motivates a medical expert? The design of Dr. Watson
  13. 13. Gamification of crowdsourcing tasks: What motivates a medical expert? The design of Dr. Watson ● Personal growth, learning ● Reading most preferred ● Provide overview of literature ● Spark their interest
  14. 14. Gamification of crowdsourcing tasks: What motivates a medical expert? The design of Dr. Watson ● To unlock articles: ○ play games to gather reputation points ● Articles motivate medical experts ○ Except researchers and students ○ Medical books, videos, giveaways and accreditation points should motivate these experts
  15. 15. Gamification of crowdsourcing tasks: What motivates a medical expert?
  16. 16. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Select game mode ● Cannot explore all content ● Make the experts curious ● More difficult tasks, to keep the game challenging
  17. 17. Gamification of crowdsourcing tasks: What motivates a medical expert?
  18. 18. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Make game more challenging ● Motivate the experts to keep playing ○ motivate them to play less popular tasks and annotate articles without annotations
  19. 19. Gamification of crowdsourcing tasks: What motivates a medical expert?
  20. 20. Gamification of crowdsourcing tasks: What motivates a medical expert? ● Goal: beat the score of the crowd ○ average amount of points earned by others ○ matching the score also means you win ● To get points ○ Agreement with other experts ○ Pioneer by suggesting new answer ○ Three or five in a row bonus ● The scoring was clear for the experts ○ streaks motivate them to annotate correctly ● To increase game difficulty use topics the expert is less familiar with
  21. 21. Gamification of crowdsourcing tasks: What motivates a medical expert?
  22. 22. Gamification of crowdsourcing tasks: What motivates a medical expert?
  23. 23. Gamification of crowdsourcing tasks: What motivates a medical expert? ● To support learning goals more topic selection is required ● Medical experts stated to play again ○ To learn more ○ To improve their score ○ Because they enjoyed playing it ● The experts could easily navigate through the UI ○ elements at the bottom of the sidebar ● Experts wanted to see more clearly how Watson is improving
  24. 24. Gamification of crowdsourcing tasks: What motivates a medical expert? Future work - improving Dr. Watson ● Topic selection for tasks ● Personalization of: ○ The difficulty - take specialization in account ○ The homepage - change background image, reorder elements ● Incorporate additional unlockables/rewards ○ Videos ○ Medical books ○ Accreditation points ● Add ‘Watson status’ page ○ track the progress of the crowd and what watson achieved
  25. 25. Gamification of crowdsourcing tasks: What motivates a medical expert? What motivates medical experts? Helping patients Improve medical field
  26. 26. Questions?

×