This is my presentation for the paper with the same name on the HCII 2015 conference. Complete reference:
Pichiliani, Mauro C., Hirata, Celso M. Evaluation of the Recognition Rate of the Android Accessibility API towards a better user experience. Proceedings of the HCI International, Los Angeles, California, EUA, 2015.
Evaluation of the Android Accessibility API Recognition Rate towards a Better User Experience
1. Evaluation of the Android
Accessibility API
Recognition Rate towards a
Better User Experience
Mauro C. Pichiliani (pichilia@ita.br)
Celso M. Hirata (hirata@ita.br)
Instituto Tecnológico de Aeronáutica - Department of Computer Science
2. Evaluate the UI recognition accuracy
rate of mobile applications by
measuring how many UI elements
are correctly identified by
an accessibility API
Goal
3. Introduction
Online stores with thousands of applications
Interaction model: touches and gestures
Special users (visually impaired) and eyes-free
scenarios
Accurate target indentification by accesibility APIs
benefits:
• Accessibility services and applications
• Automatic extraction of task sequences
• UI automation testing
• Collaboration frameworks
What is the recognition accuracy rate of UI
elements provided by Accessibility APIs
on popular applications?
4. Related work
HCI literature: focus on size, location and visual
properties of UI elements
Hurst et al. [5]: 74% of correct target identification on
desktop (location and size)
Content information: used by accessibility services (i.e.
screen readers)
Other approaches to increase the recognition of UI
elements
Recent efforts for Web accessibility
Strategies
Guidelines
Resources
[5] Hurst, A., Hudson, S. E., Mankoff, J. Automatically identifying targets users interact with during real world tasks. In:
Proceedings of the 15th international conference on Intelligent user interfaces, p. 11-20 (2010)
5. Mobile accessibility API
Get low-level information about targets. E.g.: MSAA API
Operating systems have accessibility applications
(screen readers, magnification glass)
Android platform provives the most complete
accessibility API:
Low-level hooks that capture events
Complete identification of the element
Reconstruction of the UI View hierarchy
Default textual description
New accessibility service creation
Integration with external devices (e.g. braile
keyboards)
6. Evaluation methodology (1)
Capture of application’s screenshots
Accessibility service developed to read the contentDescription and
capture events raised
Top 10 most popular applications (February 16th
, 2015)
7. Evaluation methodology (2)
Dynamic content
Dialog messages
Web pages
Common Android activities
Reused elements on distinct activities
9. Discussion (1)
Overall, apps have a high event trigger ratio (99.48%)
Dynamic elements and popup’s unreachable by the
accessibility API. Examples:
10. Discussion (2)
Lack of content description for messages, images and buttons
(94.07%) . Examples:
11. Conclusion & Future work
Mobile apps guided by visual access to on screen targets
Impaired users and eyes-free scenarios
97% recognition rate (99.48% event trigger rate, 94.07%
content rate)
There is room to improve mobile accessibility APIs
Better APIs impact other contexts
Future work:
Evaluation and comparison on other mobile OSs
Validation with users
Comparison of content descripton effectiveness
Test techniques to augment accesibility APIs