From Dr. Anthony: I attended my first Interactive Tabletops and Surfaces (ITS) conference last week and presented an overview of our work to date on the MTAGIC project. I thoroughly enjoyed the conference! There is just so much great and interesting work going on in the areas of touch and gesture interaction for all types of platforms, ranging from smart interactive tabletops, to interactive boards, to Kinect-based mid-air gestures, to mobile touchscreen devices, and more. I presented our MTAGIC findings with respect to how children expect to and do use touchscreen devices differently than adults, focusing on low-level interactions such as touching onscreen targets and making finger gestures.
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture and Input on Mobile Devices
1. mtagic.wordpress.com
Lisa Anthony1 (presenter),
Quincy Brown2, Jaye Nias2, Berthel Tate2, Shreya Mohan1
1University of Maryland Baltimore County (UMBC)
2Bowie State University
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 1
2. mtagic.wordpress.com
Touchscreen shipments to reach 833 million by 2013^
5 million iPhone 5 sales Sept 21-Sept 23, 2012*
Children are using mobile devices (own / parents’)m
^ ISuppli. Touch-Screen Shipments Expected to Reach 833 Million by 2013. 2008.
* http://online.wsj.com/article/BT-CO-20120924-707595.html?mod=WSJ_ComputerHardware_middleHeadlines
mChiong, C. and Shuler, C. Learning: Is there an app for that? Investigations of young children's usage and learning with mobile devices and apps.
The Joan Ganz Cooney Center at Sesame Workshop, New York, 2010.
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 2
3. mtagic.wordpress.com
Manual dexterity / fine motor Little hands activate interactors
control develop with age of child unexpectedly*
Shapes children are expected to Children have difficulty tapping
generate vary by age^ and with drag and dropm
^ Beery, K., Buktenica, N., and Beery, N.A. The Beery Buktenica Developmental Test of Visual-Motor Integration, 5th Edition. 2004.
* McKnight, L. & Cassidy, B. (2010) Children’s Interaction with Mobile Touch-Screen Devices: Experiences and Guidelines for Design.
International Journal of Mobile Human Computer Interaction 2 (2), 18 pp.
mBrown, Q., Bonsignore, E., Hatley, L., Druin, A., Walsh, G., Foss, E., Brewer, R., Hammer, J., and Golub, E. Clear Panels: a technique to design
mobile application interactivity. Proc. DIS 2010, 360–363.
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 3
4. mtagic.wordpress.com
LETTERS (ADULT / CHILD) SHAPES (ADULT / CHILD)
A Δ
E O
Q +
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 4
5. mtagic.wordpress.com
“Mobile Touch and Gesture Interaction for Children”
https://mtagic.wordpress.com/
Funding support: NSF HCC Small
NSF CISE #IIS-1218395/IIS-1218664 Touch cloud
PIs:
Lisa Anthony (UMBC)
Quincy Brown (Bowie State University)
Students:
Robin Brewer, Germaine Irwin, Shreya Mohan,
Jaye Nias, Luis Queral, Berthel Tate
Triangle
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 5
6. mtagic.wordpress.com
Understand differences between kids and adults in
touch / gesture input
e.g., can we reliably identify kids?
Design interaction to help kids have more successful
interaction
e.g., target sizes and active spaces
Develop technology to offer tailored interaction for
kids
e.g., recognizers and widgets
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 6
7. mtagic.wordpress.com
3 studies with kids and adults (S1^, S2*, S3-ongoing)
49 kids (ages 7-17), 36 adults (ages 18+)
Two touchscreen tasks, laboratory setting
Study 2
setting
^ Brown, Q. and Anthony, L. 2012. Toward Comparing the Touchscreen Interaction Patterns of Kids and Adults. Proc. ACM SIGCHI Workshop on
Educational Software, Interfaces and Technology (EIST’2012), Austin, TX, 05-06 May 2012, 4pp.
* Anthony, L., Brown, Q., Nias, J., Tate, B., and Mohan, S. 2012. Interaction and Recognition Challenges in Interpreting Children’s Touch and
Gesture Input on Mobile Devices. Proc. ACM Conf. on Interactive Tabletops and Surfaces (ITS’2012), Cambridge, MA, 14 Nov 2012, 225-234.
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 7
8. mtagic.wordpress.com
Task 1: Touch Interaction
Touch target with finger (4 sizes)
Measure touch time, touch location (x,y), touch pressure, # of
attempts, etc.
S1: 43 total targets, S2&3: 104 targets
Study 2
interfaces
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 8
9. mtagic.wordpress.com
Task 2: Gesture Interaction
Draw gesture with finger
Measure touch properties grouped by strokes and gestures
S1: 9 gestures (x1 sample per user),
S2&3: 20 gestures (x6 samples per user)
Study 2 Study 2 & 3
interfaces gestures
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 9
10. mtagic.wordpress.com
Android platform
Open source and free to develop
Java-based development environment
S1: Google Nexus One (320 x 480 interface)
S2&3: Google Nexus S (480 x 800 interface)
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 10
11. mtagic.wordpress.com
Children miss more targets than adults**
S1: 46% kids vs. 32% adults (of all targets)
S2: 23% kids vs. 17% adults (of all targets)
Smallest targets most challenging**
Study 1 Study 2
(one (must get
attempt target
per correctly)
target)
** significant at the p<0.05 level
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 11
12. mtagic.wordpress.com
Understand: Design:
Children miss more targets than adults** increase area to activate
desired target
S1: 46% kids vs. 32% adults (of all targets) or use probabilities to
S2: 23% kids vs. 17% adults (of all targets) identify most likely target
Smallest targets most challenging** follow recommendations
for target size in paper
Study 1 Study 2
(one (must get
attempt target
per correctly)
target)
** significant at the p<0.05 level
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 12
13. mtagic.wordpress.com
Edge-padded targets more challenging**
S2: miss rate doubles on edge-padded targets
S2: 99% of misses in “gutter”
Study 2
** significant at the p<0.05 level
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 13
14. mtagic.wordpress.com
Understand: Design:
Edge-padded targets more challenging** align targets with edge of
screen (Fitts’ Law holds!)
S2: miss rate doubles on edge-padded targets
S2: 99% of misses in “gutter”
Study 2
** significant at the p<0.05 level
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 14
15. mtagic.wordpress.com
Discovered new phenomenon: holdovers
touches in location of previous target
96% of holdovers were by children
81% of holdovers were smallest targets
Study 2
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 15
16. mtagic.wordpress.com
Understand: Design:
Discovered new phenomenon: holdovers ignore touches in area of
touches in location of previous target previously active target
based on timing
96% of holdovers were by children
81% of holdovers were smallest targets
Study 2
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 16
17. mtagic.wordpress.com
Kids make gestures differently than adults
S1: kids make bigger gestures**
S1: kids make gestures with more strokes**
Child Adult Adult Child
Study 1
** significant at the p<0.01 level
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 17
18. mtagic.wordpress.com
Kids make gestures differently than adults
S1: kids make bigger gestures**
S1: kids make gestures with more strokes**
Kids gestures are recognized^ less accurately than adults
S1: 34% kids vs. 64% adults**
S2: 81% kids vs. 90% adults**, correlated to age**
Study 1
^ Anthony, L. and Wobbrock, J.O. 2010. A Lightweight Multistroke Recognizer for User Interface Prototypes. Proc. Graphics Interface (GI’2010),
Ottawa, Canada, 02 Jun 2010, 245-252.
** significant at the p<0.01 level
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 18
19. mtagic.wordpress.com
Understand: Design:
Kids make gestures differently than adults train on kids’ gestures by
S1: kids make bigger gestures** age group
or develop specialized
S1: kids make gestures with more strokes**
recognizers for kids
Kids gestures are recognized^ less accurately than adults
S1: 34% kids vs. 64% adults**
S2: 81% kids vs. 90% adults**, correlated to age**
Study 1
^ Anthony, L. and Wobbrock, J.O. 2010. A Lightweight Multistroke Recognizer for User Interface Prototypes. Proc. Graphics Interface (GI’2010),
Ottawa, Canada, 02 Jun 2010, 245-252.
** significant at the p<0.01 level
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 19
20. mtagic.wordpress.com
Understand differences between kids and adults in
touch / gesture input
e.g., can we reliably identify kids?
Design interaction to help kids have more successful
interaction
e.g., target sizes and active spaces
Develop technology to offer tailored interaction for
kids
e.g., recognizers and widgets
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 20
21. mtagic.wordpress.com
Short-term:
Looking at younger kids
In-context apps
Mid-term:
Interaction co-design with kids
Long-term:
Tailored recognition for kids
Hokey Pokey Penguin
prototype
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 21
22. mtagic.wordpress.com
Lisa Anthony1 (presenter), lanthony@umbc.edu
Quincy Brown2, qbrown@bowiestate.edu
Jaye Nias2, Berthel Tate2, Shreya Mohan1
1University of Maryland Baltimore County (UMBC)
2Bowie State University
Funding:
NSF CISE IIS #IIS-1218395/IIS-1218664
Dept of Ed HBGI Grant Award #P031B090207-11
Interaction and Recognition Challenges in Interpreting Children's Touch and Gesture Input on Mobile Devices / ITS 2012 22