By Katie Kuksenok, March 7, 2012, 8 PM, downstairs at Town Hall Seattle. AUDIO: http://www.engage-science.com/helping-computers-find-meaning-they-lost-in-translation/ Thousands of different languages are spoken around the world, limiting our ability to communicate globally. Translation can help to overcome this barrier, but is often prohibitively expensive and time consuming. Automatic translation can allow people to communicate ideas and experiences across languages and cultures, but current technologies still struggle with expressive language. Nuance, idiomatic expressions, and cultural references pose fundamental challenges to how machines translate from one language to another. However, people reading a translation can easily locate problematic wording, suggest possible corrections, and help to improve translation quality. In this talk, we explore how people and computers can work together to overcome language barriers.

1. Helping Computers
Find Meaning They
Lost in Translation
Katie Kuksenok UW Science Now!
University of Washington March 8, 2012

2. Hi! I’m Katie!
Computer Science and Engineering
PhD student
How people
interact with
computers
Richness of
written
language

4. Posterity
Tranquility
Impeachment
liable Indictment
Concurrence

5. Google Searches
By Language

6. Google Searches
By Language

7. Machine Translation
Source language

8. Machine Translation
-----------------
-----------------
-----------------
-----------------
-----------------
---------
translation
Source language Target language

9. Machine Translation
+ Interaction
-----------------
-----------------
-----------------
-----------------
-----------------
---------
translation reading
Source language Target language

10. Machine Translation
+ Interaction
-----------------
-----------------
-----------------
-----------------
-----------------
--------- editing
translation reading
Source language Target language

11. Machine Translation
+ Interaction
-----------------
-----------------
-----------------
-----------------
-----------------
--------- editing
translation reading
Source language Target language

12. The Translating Machine

13. The Translating Machine

14. Artificial Intelligence
…where to begin?
What would a person do?
How can we describe that to a computer?

15. Rules & Dictionaries
?

16. Did it work?

17. Did it work?
The spirit is willing, but the flesh is weak

18. Did it work?
The spirit is willing, but the flesh is weak
(Russian)

19. Did it work?
The spirit is willing, but the flesh is weak
The vodka is good, but the meat is rotten
(Russian)

20. Rules & Dictionaries


21. Rosetta Stone
?

22. Translate this:
Source farok crrrok hihok yorok clok kantok yurp
Target ____ ____ _____ _____ ____ ______ ___

23. Translate this:
Source farok crrrok hihok yorok clok kantok yurp
Target ____ ____ _____ _____ ____ ______ ___
Source wiwok farok izok stok
Target sus clientes estan enfadados
Source lalok farok ororok lalok sprok izok enemok
Target los clientes y los asociados son enemigos
Based on 2 parallel pairs:

24. Source wiwok farok izok stok
Target sus clientes estan enfadados
Source lalok farok ororok lalok sprok izok enemok
Target los clientes y los asociados son enemigos
Translate this:
Source farok crrrok hihok yorok clok kantok yurp
Target ____ ____ _____ _____ ____ ______ ___
Based on 2 parallel pairs:

25. Source wiwok farok izok stok
Target sus clientes estan enfadados
Source lalok farok ororok lalok sprok izok enemok
Target los clientes y los asociados son enemigos
Translate this:
Source farok crrrok hihok yorok clok kantok yurp
Target ____ ____ _____ _____ ____ ______ ___
Based on 2 parallel pairs:

26. Source wiwok farok izok stok
Target sus clientes estan enfadados
Source lalok farok ororok lalok sprok izok enemok
Target los clientes y los asociados son enemigos
Translate this:
Source farok crrrok hihok yorok clok kantok yurp
Target ____ ____ _____ _____ ____ ______ ___
Based on 2 parallel pairs:
clientes

27. Machine Learning
Statistics + Data =
Surprisingly Good Results
More Data + Better Data = Better Results

28. DATA

29. Statistical Machine Translation
state of the art

30. 1954 1966 1991 2012

31. Posterity
Tranquility
Impeachment
liable Indictment
Concurrence

32. How do people
learn different languages?
By Instruction? By Example?

33. By Failing

34. By Failing
Together

35. Interactive Machine Translation
?

36. Interactive Machine Translation
Human
Computation

37. Human Computation
source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.
initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

38. Human Computation
source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.
initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.
reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.
reorder [a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.
retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.
retype A vixen {saw} the hungry crow been in the fig tree, and asked you why did.

39. Human Computation
source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.
initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.
reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.
reorder [a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.
retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.
retype A vixen {saw} the hungry crow been in the fig tree, and asked you why did.
delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.
delete A vixen saw the hungry crow in the fig tree, and asked [you] why did.
replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.

40. Human Computation
source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.
initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.
reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.
reorder [a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.
retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.
retype A vixen {saw} the hungry crow been in the fig tree, and asked you why did.
delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.
delete A vixen saw the hungry crow in the fig tree, and asked [you] why did.
replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.
retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.
reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.
retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.
delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].
final A vixen saw the hungry crow in the fig tree, and asked why he was there.

41. Human Computation
source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.
initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.
reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.
[a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.
retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.
A vixen {saw} the hungry crow been in the fig tree, and asked you why did.
delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.
A vixen saw the hungry crow in the fig tree, and asked [you] why did.
replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.
retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.
reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.
retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.
delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].
final A vixen saw the hungry crow in the fig tree, and asked why he was there.
x 100

42. Human Computation
x 100…
= Human Insight as Data
source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.
initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.
reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.
[a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.
retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.
A vixen {saw} the hungry crow been in the fig tree, and asked you why did.
delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.
A vixen saw the hungry crow in the fig tree, and asked [you] why did.
replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.
retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.
reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.
retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.
delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].
final A vixen saw the hungry crow in the fig tree, and asked why he was there.

43. DATA

44. DATA
 Human Insight as Data

45. Human Computation + Translation
How can we
capture the
editing process ?
source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.
initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.
reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.
[a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.
retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.
A vixen {saw} the hungry crow been in the fig tree, and asked you why did.
delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.
A vixen saw the hungry crow in the fig tree, and asked [you] why did.
replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.
retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.
reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.
retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.
delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].
final A vixen saw the hungry crow in the fig tree, and asked why he was there.

46. Interaction
Drag and drop to reorder
Dropdowns with suggestions

47. Interaction
Drag and drop to reorder
vs
Dropdowns with suggestions
Free type

48. Interaction
Suggested Alternative Translations

49. Many Hypotheses
Interaction
Suggested Alternative Translations

50. Interaction
Suggested Alternative Translations

51. Interaction + Translation
Translator
expertise
NoviceExpert

52. NoviceExpert
Caitra (2010)
Interaction + Translation
Translator
expertise

53. NoviceExpert
Caitra (2010)
TransType (2000)
Interaction + Translation
Translator
expertise

54. NoviceExpert
Google Translate
Caitra (2010)
TransType (2000)
Interaction + Translation
Translator
expertise
Google translate

55. data powers
better systems
NoviceExpert
Google Translate
Caitra (2010)
Active Crowd
Translation (2010)
TransType (2000)
Interaction + Translation
Translator
expertise

56. data powers
better systems
collaboration +
discussion
NoviceExpert
Google Translate
MonoTrans (2011)Caitra (2010)
Active Crowd
Translation (2010)
TransType (2000)
Interaction + Translation
Translator
expertise

57. Beyond Human Computation

58. Beyond Human Computation
Munteanu, Baecker, and Penn, CHI 2008

59. Beyond Human Computation
Duolingo.com

60. Interactive Machine Translation
What if people interact with translation?
How can a computer learn from that?
How powerful is this approach?

61. Production: www.paperhand.org | Photos: mlight.typepad.com

62. A Sea of Information
Production: www.paperhand.org | Photos: mlight.typepad.com

63. A City Obsessed
with More and Newer
Production: www.paperhand.org | Photos: mlight.typepad.com

64. An Insatiable,
Hungry Monster
Production: www.paperhand.org | Photos: mlight.typepad.com

66. An Insatiable,
Hungry Monster
Production: www.paperhand.org | Photos: mlight.typepad.com

67. An Insatiable,
Hungry Monster
Access?
Production: www.paperhand.org | Photos: mlight.typepad.com

68. Effective Access
Production: www.paperhand.org | Photos: mlight.typepad.com

69. Google Searches
By Language

70. Thanks to:
• Mentors
– James Fogarty (UW)
– Srinivas Bangalore (AT&T)
– Luke Zettlemoyer (UW)
– Dan Weld (UW)
– Charlotte P. Lee (UW)
– Jen Mankoff (CMU)
• And many more
– Study Participants
– Colleagues
– You!
• Supporting Organizations
– National Science Foundation
– AT&T Labs
– Microsoft Research
– Google
– University Book Store
– Engage: The Science Speaker
Series
– University of Washington
– Pacific Science Center
– Town Hall Seattle
– Microsoft

71. Thank You!
Questions?
Contact:
kuksenok@cs.uw.edu
Links, Citations, etc:
students.washington.edu/kuksenok/engage