The document summarizes usability studies conducted during the development of a mobile game called "Labours of Hercules". It describes how 8 usability tests were performed in 3 phases with team members, users, and an expert to identify and address issues. Tests were done on paper prototypes, digital prototypes, and released versions. Their goal was to improve the user interface and experience. Further testing with actual users is planned after release to the app stores. The studies aimed to prevent usability problems and reduce development costs and time.

1. Baş, T., & Arkün-Kocadere, S. (2016). Usability studies of “Labours of Hercules”: The mythological mobile game. In G. Pop (Ed.).
Proceedings of the Contemporary World Challenges for the European Citizen Conference (pp. 310-313). Cluj-Napoca, Romania.
USABILITY STUDIES OF "LABOURS OF HERCULES":
THE MYTHOLOGICAL MOBILE GAME
Turgay Baş, Hacettepe University, turgaybas@hacettepe.edu.tr
Selay Arkün Kocadere, Hacettepe University, selaya@hacettepe.edu.tr
Abstract: Labours of Hercules (LOH) mobile game is an intellectual output of the “Searching
for the Labours of Hercules” (SFTLOH) Erasmus+ Project which focuses on the integration of
information and communication technologies (ICT) into teaching and learning process. This
study aims to explain the usability studies that were performed in the development process of
the mobile game. Usability is defined as the degree to which a product can be used free of
problems. Usability tests are performed either during the development process and/or at the
end of the process. Usability studies performed at the beginning of the development process
and repeated until completion limit the number of problems and prevent extra development
costs or time. While developing LOH, 8 different usability test in 3 phases were performed with
the participation of 3 team members, 22 users, and 1 expert. In addition to the usability tests
that have already been completed, the 4th phase will be performed with the participation of
real users as an end product test.
Keywords: usability, Labours of Hercules, Erasmus+ project, mobile game, educational game
Introduction
Usability is generally defined as the degree to which a product can be used without encountering
difficulties or errors. Usage without problems result in higher satisfaction, ease of use, and aim
completion. Indicators of usability include not getting an error, solving an error easily,
understanding the usage of product, learning how to use it without help or without having to
think about it, and leaving the usage process satisfied. Usable products usually serve their
design goals (Dumas & Redish, 1999, www.usability.gov, ISO 9241-11).
Usability is generally associated with the field of human-computer interaction and is assumed
to be a user interface feature of computers and computer-like devices such as tablets, phones,
etc. Usability studies are important to help create more user friendly environments by
facilitating interfaces that are errorless, compatible with the target audience, and more
qualitative.
The main aim of usability studies is to determine the degree of usability of a product and to
develop a strategy to improve it. Although various evaluation methods have been described in
the literature, two in particular are more commonly used. The first, user evaluation, is performed
by the representatives of the target group, based on their user experiences. The second method,
expert evaluation, is based on criterion and is performed by experts who have qualifications in
the field of human-computer interaction (Dix, Finlay, Abowd, & Beale, 2004; Barnum, 2010).
Usability studies are also categorized according to their timing, either during the design process
or at the end of it. While usability tests run during the development process generally aim to
improve the usability of product, others aim to determine the usability degree of product.
Ideally, usability studies are performed both within and at the end of the process with the
participation of both users and experts (Rubin & Chisnell, 2008; Barnum, 2010).

2. Baş, T., & Arkün-Kocadere, S. (2016). Usability studies of “Labours of Hercules”: The mythological mobile game. In G. Pop (Ed.).
Proceedings of the Contemporary World Challenges for the European Citizen Conference (pp. 310-313). Cluj-Napoca, Romania.
This study aims to explain the usability studies performed during the design process of the
Labours of Hercules (LOH) mobile game. LOH is an intellectual output of the “Searching for
the Labours of Hercules” (SFTLOH) project supported by the European Union Erasmus+
Program Key Action 2, Mixed Strategic Partnership for School Education. A total of 8
institutions (5 high schools, 1 non-governmental organization, and 2 universities) collaborated
under the leadership of Mehmet Çelikel High School in Zonguldak / Turkey.
Erasmus+ Project: Searching for the Labours of Hercules
Project SFTLOH is based on the mythological story of Hercules and his travels throughout
Europe to complete the Twelve Labours. The Labours of Hercules represents the common
heritage of European countries and aims to promote common heritage amongst cultural
diversity and common heritage of the 6 partner countries of Turkey, Romania, Italy, Spain,
Greece, and Hungary.
The focus of Project SFTLOH is the integration of information and communication
technologies (ICT) into teaching and learning processes. For this purpose, various opportunities
for improving and facilitating the use of technology in instructional processes were provided to
teachers and students. ICT related workshops were given to teachers and students and two
intellectual outputs were designed by two partner universities. These outputs were planned to
produce qualitative examples of the use of ICT for learning.
In this study, we will focus on one of these two examples, the LOH mobile game. Educational
games have been used in teaching since the 1980s and demand for educational games has not
diminished over the years. On one hand, technology, especially mobile technology, has grown
enormously while on the other learners of today have become defined as the game generation.
Such developments have made mobile games an inevitable part of not just for daily life but also
education.
In addition to creating an example of ICT integration into the learning environment in general,
LOH was developed with the aim of producing a learning material for teaching mythology, in
particular the story of the “12 Labours of Hercules”. The game was made up of 12 levels
corresponding to one labour with each level beginning with the runner phase and ending with
a boss fight. Each level included a challenge corresponding to that level’s labour.
Usability studies in design process
Through an integrated approach, the usability studies implemented for the LOH mobile game
were performed in different phases of the design process in different devices by different users
(Table 1). This study only includes the tests not performed by the game developers.
Table 1: Usability studies in design process
1st Phase
Paper Prototype
2nd Phase
Digital Prototype
3rd Phase
Version 1.0
 Design Team Test
 User Test
 Expert Evaluation
 Design Team Test (with test devices)
 User Test (with test devices)
 Design Team (with personal devices)
 User Test (with personal devices)
 User Test
 Expert Evaluation

3. Baş, T., & Arkün-Kocadere, S. (2016). Usability studies of “Labours of Hercules”: The mythological mobile game. In G. Pop (Ed.).
Proceedings of the Contemporary World Challenges for the European Citizen Conference (pp. 310-313). Cluj-Napoca, Romania.
1st
Phase – Paper Prototype
The first usability studies were performed progressively using a paper prototype by 3 members
of the design team, 5 students representing the target group, and 1 expert. The game’s main
screens were presented to all testers. The aim of this test was to determine any complicated or
problematic parts of the user interface and to solve any resultant issues. The majority of the
items reported as problematic were buttons, icons, and direction messages, as well as
complexity of main page. After discussing this feedback with the graphic designers and coders,
the design team made changes considering the usability test reports. At the end of the first
phase, a digital prototype was developed.
2nd
Phase – Digital Prototype
In the 2nd
phase, a draft of the digital version was tested by 3 members of the design team and
10 students representing the target group. First, test devices were used, both on the Android and
IOS platforms, and users were asked to download the game and test it on their own devices.
While few problems were determined in tests on the test devices, there were many technical
problems on the personal devices. The majority of problems were about downloading, loading,
and performance. Results of the phase 2 usability tests were reflected in Version 1.0 of the
game.
3rd
Phase – Version 1.0
Version 1.0 of the game (Appendix) was developed and uploaded to Google Play and the App
Store. Usability tests of this version were completed by 7 students and 1 expert. Thanks to the
previous tests, results were not related with technical issues in this phase but instead provided
feedback for improving the user experience. Considering the tests in this phase, the determined
problems were fixed in Version 1.1.
4th
Phase – Target Group
In the final phase usability studies with real users to improve the game regarding the thoughts
of the target group are planned.
Conclusion
Usability studies performed on the LOH mobile game were completed during the design
process through various methods and are planned for the end stage. In addition to the design
team, 22 volunteer students and an expert participated in the studies. The majority of the
usability problems were prevented and not presented to real users. As the usability studies
started at the beginning of the design process and proceed progressively to the end, they served
to prevent extra cost and time in the revision stages in addition to helping to lower the incidence
of problems.
While running tests without real users might be considered a limit of the study, the participation
of volunteer students served to speed up the testing and fixing process. On the other hand, the
real target group will be included in usability studies performed with the end product available
on the mobile app markets in the 4th phase, also mitigating this limitation.
Acknowledgment
“Labours of Hercules” mobile game is funded by the Erasmus+ Program of the European Union
as a part of project “Searching for the Labours of Hercules” (2014-1- TR01-KA201- 012990)
and Hacettepe University Scientific Research Coordination Unit with the project titled
“Development of an Educational 3D Mobile Game” (SUA-2016- 9609)”. This educational
game is designed and developed in cooperation with Pixega Game Studio.

4. Baş, T., & Arkün-Kocadere, S. (2016). Usability studies of “Labours of Hercules”: The mythological mobile game. In G. Pop (Ed.).
Proceedings of the Contemporary World Challenges for the European Citizen Conference (pp. 310-313). Cluj-Napoca, Romania.
Appendix: Screen shots of the game

5. Baş, T., & Arkün-Kocadere, S. (2016). Usability studies of “Labours of Hercules”: The mythological mobile game. In G. Pop (Ed.).
Proceedings of the Contemporary World Challenges for the European Citizen Conference (pp. 310-313). Cluj-Napoca, Romania.
REFERENCES
Barnum, C. M. (2010). Usability Testing Essentials: Ready, Set, Test. Morgan Kaufmann.
Dix, A., Finlay, J., Abowd, D. G. & Beale, R. (2004). Human-Computer Interaction. (Third
Edition). Prentice Hall.
Dumas, J. S. & Redish, J. (1999). A Practical Guide to Usability Testing. Greenwood
Publishing Group Inc. Westport, CT, USA.
ISO 9241-11 (1998). Ergonomics Requirements for Office Work with Visual Display Terminals
(vdts) part 11: Guidance on Usability.
Rubin, J. & Chisnell, D. (2008). Handbook of Usability testing. (Second Edition). Wiley.
USA Official Web Site of Usability. Retrieved December 19, 2016, from www.usability.gov