The System Usability Scale is a simple, robust, reliable measure and can be considered to be a valuable evaluation tool for any kind of digital system. This poster compares three different software tools used for eportfolios and evaluates them using the SUS tool for 'ease of use'.

1. Acknowledgement: SUS was developed as part of the usability engineering programme in integrated office systems development at Digital Equipment Co Ltd., Reading, United Kingdom.
Introduction
The question of usability of software is often of interest to any new user or to those involved with implementation in an institution. Usability is not a
quality that exists in any real or absolute sense. Perhaps it can be best summed up as being a general quality of the appropriateness to a purpose of
any particular artefact.
The SUS (System Usability Survey) was developed in 1986 by DEC (Digital Equipment Corporation). It is designed for surveying new users after their first
experience of a system. SUS is internationally benchmarked and has proved remarkably robust, with as few as 6-10 users giving a reliable score. Note that
the scale should be administered after the initial experience of the system but before any debrief or focus group discussion. Also note that this scale is not
diagnostic - it will not tell you what needs to be changed to improve the software.
Getting Things SUSsed:
ease of use perceptions and eportfolios
Stephen Bright, Centre for Tertiary Teaching and Learning, University of Waikato
Mahara
34%
PebblePad
25%
Blackboard
Myportfolio
7%
Portfolium
7%
Other
27%
* NOTE: 46 institutions surveyed, 52 software instances reported i.e. some institutions are
using more than one eportfolio software tool
# Other: Chalk & Wire 3%, Brightspace(D2L) 3%, Homegrown 3%, Google [Sites] 2%,
Scorion 1%, Taskstream 1%, Sonia 1%, Wordpress 1%, Wix 1%, Weebly 1%, Student
choice (not specified) 10%
Eportfolio Scan of the Australasian University
Sector July 2019
SUS Blank Form and Scoring
Eportfolio Software Usability Comparison
Eportfolio tool SUS Score* Survey sample Adjective
Mahara v.1.7 54.8 n = 44 Ok/fair
Mahara v.1.8 59.2 n = 13 Ok/fair
Mahara v. 18.04 58 n = 10 Ok/fair
Google Sites 53.9 n = 12 Ok/fair
Chalk & Wire 54 n = 12+ Ok/fair
*NOTE: SUS scores are NOT a percentage
Scoring
Question score Operation Raw score
One ( ) -1
Two 5 – ( )
Three ( ) -1
Four 5 – ( )
Five ( ) -1
Six 5 – ( )
Seven ( ) -1
Eight 5 – ( )
Nine ( ) -1
Ten 5 – ( )
TOTAL RAW SCORE
Total raw score …..... X 2.5 = ……… SUS
score /100
Matching adjective
= ………………………………
Mean of SUS scores/related adjectives
92 = best imaginable
85 = excellent
72 = good
52 = Ok/fair
38 = poor
25 = worst imaginable
Acknowledgements
Original questionnaire © Digital Equipment Corporation, 1986.
Supporting literature
http://cui.unige.ch/isi/icle-wiki/_media/ipm:test-suschapt.pdf (original SUS article by John Brooke)
http://www.usabilityprofessionals.org/upa_publications/jus/2009may/JUS_Bangor_M
ay2009.pdf (article describing research on matching adjectives with SUS scores)
Discussion
Possible factors influencing the similarity of the scores in this survey across
different tools include:
1. Complexity of software function – software used for eportfolios often are powerful
with multiple functions, menus, buttons and pages and therefore they can never be
simple, ‘one click’ designs that would make a higher usability score more common
2. Intertwined with (1) the navigation and user interface design of the software will
have an influence on perception of ‘ease of use’ for first time users
3. Digital fluency and digital confidence of users – this will vary from sample to
sample but it is reasonable to propose that more digitally confident and fluent
users would more easily accommodate a new piece of software into their repertoire
– however further research would be needed to test this
System Usability Scale
Date completed: …………………………
Software being assessed Mahara ePortfolio
1. I think that I would like to use
this eportfolio frequently
2. I found this eportfolio
unnecessarily complex
3. I thought this eportfolio was
easy to use
4. I think that I would need the
support of a technical person to
be able to use this eportfolio
5. I found the various functions
in this eportfolio were well
integrated
6. I thought there was too much
inconsistency in this eportfolio
7. I would imagine that most
people would learn to use this
eportfolio very quickly
8. I found this eportfolio very
cumbersome to use
9. I felt very confident using this
eportfolio
10. I needed to learn a lot of things
before I could get going with
this eportfolio
Strongly
agree
Strongly
disagree
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
References
Bangor, A., Kortum, K., & Miller,J. (2009). Determining what individual SUS scores mean: adding an adjective rating scale. Journal of Usability Studies, (4),(3), 114 - 123.
Brooke, J. (2013). SUS: a retrospective. Journal of Usability Studies, (8),(2), 29 - 40.
Bevan, N, Kirakowski, J and Maissel, J, 1991, What is Usability?, in H.-J. Bullinger, (Ed.).Human Aspects in Computing: Design and use of interactive systems and work with terminals,
Amsterdam: Elsevier.
Kirakowski, J and Corbett, M, 1988, Measuring User Satisfaction, in D M Jones and R Winder (Eds.) People and Computers IV. Cambridge: Cambridge University Press.
Sankey, M. (2019) ACODE scan of Australasian university eportfolio use, personal communication, July 24 2019.