The document outlines an augmented reality translation project developed by Matthias Vandenbussche and Annelies Van der Borght under the supervision of Erik Duval and Sten Govaerts. It describes the development of mobile applications to detect, recognize, and translate text in real-time video streams as well as the technologies and techniques used including OpenCV, JavaCV, and websockets. The document also discusses testing conducted with users and issues encountered with the server performance.

1. Promotor: Erik Duval
Begeleider: Sten Govaerts
http://augmentjapan.wordpress.com/
Matthias Vandenbussche
Annelies Van der Borght

2.  Intro
 Gelevered werk
 Preprocessing
 Client-Server
 Client
 Server
 Technische tegenstribbelingen
 Digitaal prototype 1
 Digitaal prototype 2
 Planning
 Statistieken

3.  Mobiele applicatie
 Video feed
 Tekst zoeken & vertalen
 Getoond op scherm

4. TranslatAR QQ Hui Yan
Pleco WWWJDIC
Word Lens Mezzofanti
OCR Test
 Input  Zoeken van tekst  Vertalen
 Enquête: 227 antwoorden
 Gebruik van video feed
 Zoeken van tekst
TiRG SWT

5. Aspire
NHOCR Tesseract
OCR:
WiseTREND
Google Translate Bing Translator SYSTRAN InterTran
WordLingo myGengo OneHourTranslation Apertium

6.  Technologie:
iPhone
VS
Android
HTML5
VS
Native

7.  Recognition rate 28.85% 54ms
 Greyscaling
 Perfect match rate 26.92% 212ms
 Greyscaling  Binarization
 Connected Components

8. Computationeel intensief

Beperkte rekencapaciteit

~ Real-time

9. 1. Tekstregio’s bepalen
2. Vertalingen opvragen
 HTTP Post
 WebSockets

10. Native Android

11.  Opties
 Pauze
 Gebruikte libraries:
JavaCV/OpenCV en weberknecht

12. Ubuntu 11.10 Apache Tomcat 7 OpenCV Websockets4j
JavaCV
 Websocket server
 Servlet
 Request  aparte Thread

13.  Post request blocks  te veel Threads
 Performantie
 SWT: Connected Components
7s  5s
 Tracking

14.  Performantie
 Android UI Thread
 ... en Tegenslag: voeding server

19.  Pre-test vragenlijst
 Briefing
 4 scenario’s
 Post-test vragenlijst
 System Usability Scale (SUS)
 Stubvertalingen

20.  7♂ 7♀ 20-70j 14

21.  Standaard Android gedrag:
opstartboodschap, opties
 Aanpassingen:
opties
 Slechte vertalingen:
tappen: 2, bewegen: 4, afsluiten: 5
 Focussen: 1 bord  meerdere
1  13
 Niet relevante borden: 3

22.  Opstartboodschap sluiten
 Opties
 Gebruik
 Slechte vertalingen

23.  __Is dit nodig? Just in case__

28. Anything that can go wrong, will – at the
worst possible moment

30. 02/04 – 15/04: 3de gebruikerstest
02/04 – 30/04: herschrijven short paper
16/04 – 22/04: verwerken gebruikerstest
16/04 – 27/04: poster
23/04 – 06/05: vergelijkende gebruikerstest
Draft thesistekst volledig af tegen 18/05

31. Own blog:
 28 posts
 19 comments
Matthias:
 33 comments on other blogs
 296 #thesis11
 445h 30min worked
Annelies:
 18 comments on other blogs
 176 #thesis11
 452h 30min worked

32. TiRG: http://sourceforge.net/projects/tirg/
Detecting Text in Natural Scenes with Stroke Width Transform (B. Epshtein
et al., 2010)
Implementation at:
https://sites.google.com/site/roboticssaurav/strokewidthnokia
OpenCV: http://opencv.willowgarage.com/wiki/
JavaCV: http://code.google.com/p/javacv/
weberknecht: http://code.google.com/p/weberknecht/
Apache Tomcat: http://tomcat.apache.org/
Websockets4j: http://code.google.com/p/websockets4j/
Fast connected component labeling algorithm using a divide and conquer
technique (J. Park et al., 2000)
SUS - A quick and dirty usability scale (J. Brooke, 1996)

33. A comprehensive method for multilingual video text detection, localization, and extraction (M. R. Lyu et al., 2005)
A real-time tracker for markerless augmented reality (A. I. Comport et al., 2003)
A robust text detection algorithm in images and video frames (Qixiang Ye et al., 2003)
Automatic detection and recognition of signs from natural scenes (Xilin Chen et al., 2004)
Automatic detection and translation of text from natural scenes (Jie Yang et al., 2002)
Automatic text location in images and video frames (A. K. Jain and Bin Yu, 1998)
Camera-based Kanji OCR for Mobile-phones: Practical Issues (M. Koga et al., 2005)
Comparative Evaluation of Online Machine Translation Systems with Legal Texts (Chunyu Kit and Tak Ming Wong, 2008)
Correction of perspective text image based on gradient method (Lijing Tong and Yan Zhang, 2010)
Design-based research: what we learn when we engage in design of interactive systems (Željko Obrenović, 2007)
Detecting Text in Natural Scenes with Stroke Width Transform (Boris Epshtein, 2010)
Detection of Text on Road Signs From Video (W Wu et al., 2005)
Evaluation of machine translation and its evaluation (Joseph P. Turian et al., 2003)
Fast and robust text detection in images and video frames (Q. Ye et al., 2005)
Kanji recognition in scene images without detection of text fields - robust against variation of viewpoint, contrast, and background texture (A. Suzuki et al., 2004)
Markerless augmented reality with a real-time affine region tracker (V Ferrari et al., 2001)
Multiple target detection and tracking with guaranteed framerates on mobile phones (D. Wagner et al., 2009)
Performance Evaluation for Text Localization Algorithms: An Empirical Study (Yi-Feng Pan and Cheng-Lin Liu, 2010)
Real-time vision-based camera tracking for augmented reality applications (Dieter Koller et al., 1997)
Robust text detection in natural images with edge-enhanced maximally stable extremal regions (Huizhong Chen et al., 2011)
Sequential correction of perspective warp in camera-based documents (Camille Monnier et al., 2005)
Snoopertext: A multiresolution system for text detection in complex visual scenes (Minetto, R. Et al., 2010)
Text Detection on Nokia N900 Using Stroke Width Transform (Saurav Kumar and Andrew Perrault, 2010)
Text extraction of street level images (J Fabrizio et al., 2009)
Text information extraction in images and video: a survey (K. Jung, 2004)
Text locating from natural scene images using image intensities (Jisoo Kim et al., 2005)
Text/Graphics Separation and Skew Correction of Text Regions of Business Card Images for Mobile Devices (Ayatullah Faruk Mollah et al., 2010)
TranslatAR: A mobile augmented reality translator (Victor Fragoso et al., 2011)
Translation and the Internet: Evaluating the Quality of Free Online Machine Translators (Stephen Hampshire and Carmen Porta Salvia, 2010)
Translation camera on mobile phone (Y. Watanabe et al., 2003)
Video text recognition using feature compensation as category-dependent feature extraction (M. Mori, 2003)

34. A Fast Skew Correction Technique for Camera Captured Business Card Images (A. F. Mollah, 2009)
A new robust algorithm for video text extraction (E. Wong, 2003)
An evaluation tool for machine translation: Fast evaluation for MT research (S. Nieen et al., 2000)
An Overview of the Tesseract OCR Engine (Ray Smith, 2007)
Automatic evaluation of machine translation quality using n-gram co-occurrence statistics (George Doddington, 2002)
Automatic location of text in video frames (Xian-Sheng Hua et al., 2001)
BLEU: a Method for Automatic Evaluation of Machine Translation (Kishore Papineni et al., 2002)
Camera-based analysis of text and documents: a survey (Jian Liang et al., 2005)
Character extraction of license plates from video (Y. T. Cui and Q. Huang, 1997)
Color Edge Detection Using Multiscale Quaternion Convolution (Jiangyan Xu et al., 2010)
Connected components labeling - algorithms in Mathematica, Java, C++ and C# (Mariusz Jankowski and Jens-Peer Kuska , 2004)
End-to-End Scene Text Recognition (Kai Wang et al., 2011)
Error Evaluation and Applicability of OCR Systems (V. Alexandrov, 2003)
Extraction of illusory linear clues in perspectively skewed documents (M. Pilu, 2001)
Fast, cheap, and creative: Evaluating translation quality using Amazon's Mechanical Turk (Chris Callison-Burch, 2009)
From Mirroring to Guiding: A Review of State of the Art Technology for Supporting Collaborative Learning (Amy Soller et al., 2005)
Improvement of video text recognition by character selection (T. Mita and O. Hori, 2001)
JEIDA's Test-Sets for Quality Evaluation of MT Systems: Technical Evaluation from the Developer's Point of View (Hitoshi Isahara, 1995)
Kanji Character Detection from Complex Real Scene Images based on Character Properties (Lianli Xu et al., 2008)
Localizing and segmenting text in images and videos (Rainer Lienhart and Axel Wernicke, 2002)
Locating text in complex color images (Y. Zhong et al., 1995)
Marker-less Vision Based Tracking for Mobile Augmented Reality (D. Beier et al., 2003)
Objective evaluation criteria for machine translation (A. J. Petit, 1977)
Perspective Correction Methods for Camera-Based Document Analysis (L. Jagannathan and C. V. Jawahar, 2005)
Re-evaluating machine translation results with paraphrase support (Liang Zhou et al., 2006)
Re-evaluating the Role of BLEU in Machine Translation Research (Chris Callison-Burch et al., 2006)
Reliable measures for aligning Japanese-English news articles and sentences (Masao Utiyama and Hitoshi Isahara, 2003)
SUS - A quick and dirty usability scale (John Brooke, 1996)
Text detection and segmentation in complex color images (C. Garcia and X. Apostolidis, 2000)
Text scanner with text detection technology on image sequences (T. Kurata and M. Kourogi, 2002)
TextFinder: An Automatic System to Detect and Recognize Text In Images (Victor Wu et al., 1999)
Using multiple edit distances to automatically grade outputs from Machine translation systems (Yasuhiro Akiba et al., 2006)