Presentation given at ICDAR 2007, 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil, September 2007
ABSTRACT: With the emergence of digital pen and paper interfaces, there is a need for gesture recognition tools for digital pen input. While there exists a variety of gesture recognition frameworks, none of them addresses the issues of supporting application developers as well as the designers of new recognition algorithms and, at the same time, can be integrated with new forms of input devices such as digital pens. We introduce iGesture, a Java-based gesture recognition framework focusing on extensibility and cross-application reusability by providing an integrated solution that includes tools for gesture recognition as well as the creation and management of gesture sets for the evaluation and optimisation of new or existing gesture recognition algorithms. In addition to traditional screen-based interaction, iGesture provides a digital pen and paper interface.
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
iGesture: A General Gesture Recognition Framework
1. A General Gesture Recognition Framework
Beat Signer, Ueli Kurmann and Moira C. Norrie
Institute for Information Systems, ETH Zurich
8092 Zurich, Switzerland
http://vub.academia.edu/BeatSigner
ICDAR 2007, September 26
2. Interactive Paper
Digital Pen and Paper
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 2
3. Requirements
Java API
Different types of input devices
digital pen and paper, mouse, etc.
Support application and algorithm developers
Definition and evaluation of new gesture sets
Simple acquisition of gesture data
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 3
4. iGesture Components
Management Evaluation
Console Tools
Recogniser
Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 4
5. Management Evaluation
Console Tools
Recogniser
iGesture Admin Tab Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 5
6. Management Evaluation
Console Tools
Recogniser
Capture of Samples Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 6
7. Management Evaluation
Console Tools
Recogniser
iGesture Test Bench Tab Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 7
8. Management Evaluation
Console Tools
Recogniser
iGesture Test Data Tab Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 8
9. Management Evaluation
Console Tools
Recogniser
Evaluation Tools Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 9
10. Management Evaluation
Console Tools
Recogniser
Recogniser API Common Data Structures
Configuration configuration =
XMLTool.importConfiguration(new File(RUBINE));
GestureSet gestureSet =
XMLTool.importGestureSet(new File(RUBINE_SET));
configuration.addGestureSet(gestureSet);
recogniser = new Recogniser(configuration);
...
ResultSet result = recogniser.recognise(note);
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 10
11. Management Evaluation
Console Tools
Recogniser
Gesture Representation Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 11
12. Management Evaluation
Console Tools
Recogniser
Recognition Algorithm Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 12
13. Management Evaluation
Console Tools
Recogniser
Persistence Common Data Structures
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 13
14. Current Status
Algorithms
Rubine, SiGeR, E-Rubine, SiGrid
Gesture Sets
Graffiti numbers, Graffiti letters, MS application
gestures, customised multi-stroke gestures
Input Devices
Anoto digital pen (e.g. Magicomm G303), mouse
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 14
15. Conclusions
Simple gesture recogniser Java API
Experimental platform for implementing and
testing new gesture recognition algorithms
Definition and evaluation of new gesture sets
Extensibility in terms of
input devices
algorithms
gestures sets
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 15
16. References
iGesture Framework,
Apache License, V2.0
http://www.igesture.org
B. Signer, M. C. Norrie and U. Kurmann. iGesture: A
Java Framework for the Development and Deployment
of Stroke-Based Online Gesture Recognition Algorithms.
Technical Report ETH Zurich TR 561, September 2007
ICDAR 2007, September 26 Beat Signer, signer@inf.ethz.ch 16