Presentation slides for a paper titled "A Crowdsourcing Based Mobile Image Translation and Knowledge Sharing Service" by Yefeng Liu, Vili Lehdonvirta, Mieke Kleppe, Todorka Alexandrova, Hiroaki Kimura and Tatsuo Nakajima. Presented by Yefeng Liu at the 9th International Conference on Mobile and Ubiquitous Multimedia (MUM 2010), December 1-3, 2010, Limassol, Cyprus. The full paper is available at http://www.hiit.fi/~vlehdonv/documents/Liu-2010-crowdsourcing-mobile.pdf

1. A Crowd-sourcing Based Mobile
Image Translation and
Knowledge Sharing Service
Yefeng Liu, Vili Lehdonvirta1, Mieke Kleppe2, Todorka
Alexandrova, Hiroaki Kimura, Tatsuo Nakajima
Department of Computer Science
Waseda University, Tokyo, Japan
1Helsinki Institute for Information Technology
2Eindhoven University of Technology
yefeng@dcl.info.waseda.ac.jp

2. Outline
• Introduction
• Human Mobile Image Translation
• Preliminary Study
• Discussion
• Future Directions
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 2

3. Introduction
“...I can’t wear tie
here?? Should I
take off my tie?..”
A menu board outside a restaurant, Tokyo
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 3

4. Real World Problem
• Digitalpocket translators or online translation
services are useless if you don’t know how to
input the characters.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 4

5. (Typical) Mobile Image Translation
Image OCR MT English
Text Optical Character Recognition Machine Translation Text
Poor
Irregular fonts or formats, handwriting, etc. performance
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 5

6. Our Solution: Human Mobile Image Translation
Image Text
Translator English
Outsourcing Text
Question of Community
the image
Crowdsourcing
• Better quality in text recognition and translation
• Human worker can provide richer interpretations and responses
in addition to literal answers.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 6

7. Image Based Translator + Mobile Q&A
• NOT only a translator
• But also a knowledge broker that allows users to share high level
information pertinent to the situation at hand, e.g.
• advices
• instructions
• suggestions
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 7

8. Basic work-flow overview
Kanji English
Open call Scoring
etc.
Requester Best
Translators Requester
answer
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 8

9. Preliminary study
•A preliminary study and design research aims to
• verify the feasibility of the design
• identify real user requirements and design
issues
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 9

10. Preliminary study - Method
Collected around a hundred pictures/questions from potential users
Fifteen characteristic cases were selected from the collected images
Interviewed the requesters what kind of answers they were expecting
Assigned questions to invited translators
Compared the results with the requesters’ expectations
Interviewed translators for their feedbacks
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 10

11. Preliminary Study Cases - Example 1
“...how long do I have to wait?”
Information in the picture is insufficient
to answer this question.
However, most of the repliers can still
suggest an approximate waiting time
according to their life experiences.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 11

12. Preliminary Study Cases - Example 2
“What are the events between 5th
and 8th?”
Poor question text.
Some translators misunderstood the
question, thus provided useless
answers.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 12

13. Preliminary Study - Findings (1)
1. Communication between requester and worker.
Better communication Better understanding Better result
2. Question/Answer style
• Short, but clear (e.g clarify to what level of details is wanted);
• Question with choices is better;
• Asking for links (of image/web page/etc) is a good way to lower the
difficulty and faster the response time.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 13

14. Preliminary Study - Findings (2)
3. “Tweet” and “keywords” style answer is preferred
a). “Pork, spicy, famous chinese food”
b). “Twice cooked pork (huiguo rou)”
- meaningless if don’t know the name
- Many translators use English as 2nd or 3rd language, they often
face the problem of being unable to explain in long sentence.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 14

15. Discussion (1)
1. Quality of outcome
Misunderstanding between requester and
worker strongly affects quality of outcome.
- Workers often are not native English speaker.
- Requesters may use unclear or too complicate English.
- Human always make mistakes.
- Malicious replies.
suggests a single reply can hardly be trusted.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 15

16. Discussion (2)
Kanji English
Scoring
open
call
Best
Requester Translators Proofreaders Requester
answer
An additional proofreading phase.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 16

17. Discussion (3)
2. Different user types (user requirements)
Client Users
Short-term stay Long-term stay
Need immediate Waitable Need immediate Waitable
answer answer
A B C D
may have different preference on the accuracy vs. timeliness trade-off
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 17

18. Future Directions (1)
1. Dynamical task allocation with real time requirement
• Task is better be assigned to worker who is:
i. capable for the task
In this study case, local context of the requester and
background information of the worker is important to
determine the capacity.
ii. available for the task
Not only about if the worker is free, but also involves
other factors like expertise, properties of question, etc.
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 18

19. Future Directions (2)
2. Motivation and Incentive
Social and Intrinsic incentive: game play
A location-based mobile game is designed
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 19

20. Current Status
(some images here..)
Preliminary Prototype Redesign
Study Implementation
Early Usability Test/On
Design Redesign field study
Test
This paper
A Crowdsourcing Based Mobile Image Translation & Knowledge Sharing Service, MUM 2010, Limassol, Cyprus 20

21. Thanks for your attention!
Yefeng Liu, PhD candidate
yefeng@dcl.info.waseda.ac.jp
Distributed & Ubiquitous Computing Lab.
Depart. of Computer Science, Waseda University
http://www.dcl.info.waseda.ac.jp/