The increasing popularity of the Web and the proliferation of mobile technologies have had a tremendous impact on museums. The deployment of new technology into physical museum spaces has greatly enhanced the in-person museum experience, but efforts to improve the virtual museum experience have been less successful. This lightning talk describes our preliminary efforts to develop and validate a user experience (UX) assessment rubric for online museum collections. Drawing from existing research and current interface design and usability best practices, this rubric provides a set of criteria for assessing the extent to which an online museum collection provides a positive user experience for online visitors. Future research directions will be presented alongside the results from an initial pilot study. Presented at the 2014 Museums and the Web conference in Baltimore, MD.

1. A Rubric for Assessing the
User Experience (UX) of
Online Museum Collections
Preliminary Findings and Future Directions
Craig M. MacDonald, Ph.D.
Seth Persons
Samantha Raddatz
Irene Lopatovska, Ph.D.
Pratt Institute, School of Information and Library Science
Lightning Talk | Museums and the Web 2014

2. The Museum Experience
The proliferation web,
mobile and cloud
computing has had a
tremendous impact on
museums.
Interactive displays,
mobile tour guides, and
a wealth of other digital
technologies have
greatly enhanced the in-
person museum
experience for millions
of people.
2

3. The Virtual Museum Experience
Resources have been
invested in:
digitizing materials
providing access to
these materials via the
web
enhancing websites
with:
• 3-D/virtual reality
• Personalization
• Social communities
3

4. But are virtual users engaged?
Most online users show relatively little interest
in the digital collections and they still use museum
websites primarily for planning their visit to the
museum1.
Facilitating museum visits is a worthy goal of a
museum website -- but can they do more?
Can museums provide a rich and engaging
experience for their virtual users?
An experience that rivals the in-person experience
but also stands on its own in a unique and
memorable way.
41 Haynes & Zambonini, 2007; Fantoni & Stein, 2012

5. More than just usability
Providing a usable website has always been a
goal for museums.
A website should be easy to use and easy to learn, for
both novices and experts.
But, usability alone is no longer sufficient; a
well-designed website must address the entire
User Experience (UX).
Online museums cannot simply provide access
to their digital materials; they must also provide
positive emotional outcomes for their users.
5

6. UX of Online Museum Collections
In this research, we set out to answer a simple
question:
What does it mean for an online museum
collection to provide a “good” user experience?
Why “collections?”
Because people visit a museum to see its
collection; a museum’s collection is what sets it
apart and makes it unique from other museums.
• Physical and architectural features are also important,
but these factors are not part of the virtual experience.
6

7. A Framework for Online Collections
7
Dimensions of
User Enjoyment
1. Engagement
2. Positive Affect
3. Fulfillment
Design Principles
1. Multisensory learning experiences
2. Creating a storyline
3. Mood building
4. Fun in learning
5. Establishing social interaction
SOURCES:
1) Lin, C.H., Fernandez, W., & Gregor, S. (2012). Understanding web
enjoyment experiences and informal learning: A study in a museum
context. Decision Support System, 53, 846-858.
2) Lin, C. H., Gregor, S., & Ewing, M. (2009). Understanding the
Nature of Online Emotional Experiences: A Study of Enjoyment as
a Web Experience. In Proceedings of the 11th International
Conference on Electronic Commerce (ICEC 2009), Taipei, Taiwan.
Design Characteristics
1. Novelty
2. Harmonization
3. No time constraint
4. Appropriate facilitation and
association

8. Testing Lin et al.’s Framework
1. Novelty
2. Harmonization
3. No time constraint
4. Appropriate
facilitations and
associations
5. Multi-sensory
learning experiences
6. Creating a storyline
7. Mood building
8. Fun in learning
9. Establishing social
interaction
8

9. Testing Lin et al.’s Framework
1. Novelty
2. Harmonization
3. No time constraint
4. Appropriate
facilitations and
associations
5. Multi-sensory
learning experiences
6. Creating a storyline
7. Mood building
8. Fun in learning
9. Establishing social
interaction
9
Not quite; we
developed our
own
interpretation of
each dimension
that no longer
matched the
original concept.

10. Toward A UX Framework
UX = cognition + emotion
Don Norman: Three Levels of Design
1. Visceral
How it looks, feels, and/or sounds; beauty
2. Behavioral
How it works; function and understandability
3. Reflective
What it means; self-image, memories, messages
10

11. Strength of Visual Content
Artwork is presented
as the primary focus
of the collection, with
images as the
dominant visual
element. All images
are large and high
quality. Text is used
purposefully but
sparingly to enhance
the visual content.
11
Visceral

12. Visual Aesthetics
Color, graphics,
typography, and other
non-interactive
interface elements are
harmonious and used
consistently. Elicits
affective reactions
that are universally
positive.
12
Visceral

13. Usefulness of Metadata
Metadata structure is
purposefully
designed to enhance
users’ ability to find
and learn about
artworks. Includes
novel metadata
facets that offer
innovative ways to
browse, search, and
filter artworks.
13
Behavioral

14. Interface Usability
Interface is intuitive
and accessible.
Interface elements are
easy to locate and
easy to use, creating
a seamless and
immersive interaction
between the user and
the collection.
14
Behavioral

15. Support for Casual Users
Primarily provides basic
content and
functionality for
casual users.
Advanced features are
visible but
unobtrusive, which
encourages learnability
for casual users. Allows
for a seamless
transition between
casual browsing and
advanced research.
15
Behavioral

16. Uniqueness of Virtual Experience
Virtual museum
experience is entirely
different from the
physical museum
experience. Finding
and viewing virtual
artworks allows for
new and insightful
perspectives that
would not be possible
in the physical
museum.
16
Reflective

17. Openness
Users are given
complete control
over the content, with
clearly marked options
to download, print,
and/or save high-
quality images.
17
Reflective

18. Integration of Social Features
Encourages varying
levels of participation
within a virtual
community, of which the
museum is an active
participant. Social
tools are prominently
integrated into the
collection. Provides
multiple options for
sharing and
communicating with
others, both internally
and externally.
18
Reflective

19. Personalization of Experience
Allows users to craft
dynamic personal
experiences with few, if
any, limitations.
Integrates robust
customization,
gaming, and/or other
innovative
personalization
features. Inspires users
to be active co-
creators of their virtual
museum experience.
19
Reflective

20. UX Assessment Rubric
Visceral
1. Strength of visual content
2. Visual aesthetics
Behavioral
3. Usefulness of metadata
4. Interface usability
5. Support for casual users
Reflective
6. Uniqueness of virtual
experience
7. Openness
8. Integration of social features
9. Personalization of experiences
20
1 Incomplete
2 Beginning
3 Developing
4 Emerged

21. Pilot Results: Museum A
21

22. Pilot Results: Museum B
22

23. Next Steps
Additional development of the rubric, including
soliciting feedback from museum and UX
professionals
Validation studies with both museum and UX
professionals
Case studies: is this rubric actually useful for
museums? How can it be used to drive UX
improvements?
23

24. Thank you
Craig M. MacDonald, Ph.D.
cmacdona@pratt.edu
@CraigMMacDonald
www.craigmacdonald.com