Characterizing the Emotional Impacts of Audio-Haptic Mobile Media Enhancement

Characterizing the
emotional impacts of haptic-emotional impacts of haptic-
enhanced mobile media
Amaya Becvar Weddle, Ph.D. | UX Research Managery , | g
Immersion Corporation, San Jose, CA
© 2013 Immersion Corporation - Confidential
UXPA 2013
12 JULY

What is ‘haptics?’at s apt cs?
The study of how people sense
their environment through touch
Feeling a
Feeling a
real bump
on a
Feeling a
haptic
simulation
of a bump
Vs.
on a
surface
of a bump
on a surface
© 2013 Immersion Corporation - Confidential 2
Haptic interfaces simulate real-world tactile interactions
OVERVIEW | EMOTION| USER STUDY| RESULTS| TAKEAWAYS

Immersion Haptic TechnologyImmersion Haptic Technology
© 2013 Immersion Corporation - Confidential 3OVERVIEW | EMOTION| USER STUDY| RESULTS| TAKEAWAYS

A UX Value of Haptics = Realismp
Multimodal experiences feel
more realistic and immersivemore realistic and immersive
Haptics can reinforce the
illusion of tangible reality
Interactions feel natural andInteractions feel natural and
intuitive
© 2013 Immersion Corporation - Confidential OVERVIEW | EMOTION| USER STUDY| RESULTS| TAKEAWAYS

Realism

Haptic Video Enhancement Feature

Haptic effects generated from
audio signal in real-time
Audio signal sourceud o s g a sou ce
Low frequency,
Haptic outputi
high amplitude
Haptic outputImmersion’s
“Reverb” feature

Mobile video
consumption isconsumption is
on the rise
The overall time spentThe overall time spent
watching video on
mobile devices (tablets
& smartphones) grew
by 19% in Q1 2013*
*Ooyala Global Video Index Report, June 2013

How does haptic-How does haptic-
enhancement impactenhancement impact
the video viewingthe video viewing
experience?p

HypothesisHypothesis
Mobile video enhanced by haptics isy p
more realistic, immersive, and
emotionally involvingemotionally involving.

ApproachApproach
Compare emotional response to
haptic enhanced videos versushaptic-enhanced videos versus
control videos with no haptics.

What do we mean byWhat do we mean by
emotional response?e ot o a espo se?

Arrousal
Valence

Why is emotion important in UX?
In some contexts, usability is less relevant.
Sometimes, two experiences are comparable usability-
i b i ll *wise, but not emotionally*.
Emotion is always part of human experience.
*Agarwal, Anshu, and Andrew Meyer. "Beyond usability: evaluating emotional response as an
g , , y y y g p
integral part of the user experience." CHI'09 Extended Abstracts on Human Factors in Computing
Systems. ACM, 2009.

How can we measure
i lemotional response
i UX t di ?in UX studies?

Two methods:Two methods:
1. Self-report. Se epo t
2. Physiologicaly g
measurements

Self-ReportSe epo t
Affect Grid
Product Reaction Cards
Emo-cards
Wells R ScaleWells R Scale

Self-ReportSe epo t
Affect Grid
Product Reaction Cards
Emo-cards
Wells R ScaleWells R Scale
Russell James A Anna Weiss and Gerald A Mendelsohn "Affect grid: a single-item scale of pleasure andRussell, James A., Anna Weiss, and Gerald A. Mendelsohn. Affect grid: a single item scale of pleasure and
arousal." Journal of Personality and Social psychology 57.3 (1989): 493.
Benedek, J., & Miner, T. (2002). Measuring desirability: New methods for evaluating desirability in a usability lab setting.
Proceedings of UPA 2002 Conference, Orlando, FL, July 8-12, 2002.
Desmet, Pieter, Kees Overbeeke, and Stefan Tax. "Designing Products with Added Emotional Value: Development and
Appllcation of an Approach for Research Through Design." The design journal 4.1 (2001): 32-47.
Wells, W. D. (1986). Three useful ideas. In R. J. Lutz (Ed.), Advances in Consumer
Research (Vol. 13, pp. 9–12). Provo, UT: Association for Consumer
Research.

Self Report – Affect Gridp

x

Participants seem to learn
x
p
to use Affect Grid quickly
& intuitively
Possible to derivePossible to derive
quantitative measures of
two dimensions of
emotion, allowing for
statistical comparisons

Self Report – Wells R Scalep
Emotional Involvement & “Being There”
“I felt like I was right there in the video, experiencing the situation.”f g , p g
“I really got involved in the emotions provoked by the video.”
Typically used to assess audience involvement in
yp y
advertising messages

Physiological Measurementsy g
Electrodermal Response
H t R t
Facial Expression
Heart Rate
TemperatureTemperature

Physiological Measurementsy g
Electrodermal Response
• Measure of physiological arousal
Q Sensor
Affectiva, Inc.
• Measure of physiological arousal
• # of “peaks” increase in heightened
arousal state
ff ,
videos viewed on HD phoneSCR

St d DesignStudy Design

Study Participantsy p
23 ti i t23 participants
Gender balanced
Own Android or iOS device 2 years old or less
All watch media on smartphone or tablet regularly (2 3 times/week)
All watch media on smartphone or tablet regularly (2-3 times/week)
OVERVIEW | EMOTION| USER STUDY| RESULTS| TAKEAWAYS©2013 Immersion Corporation - Confidential

Test ContentTest Content
8 videos
40 – 90s in length
Downloaded from YouTube
Music VideosCommercials
Downloaded from YouTube
Movie Trailers User Created

Test Content
Played on Nexus S smartphonesy p
Reverb feature enabled/disabled in
counterbalanced presentation
Genre /# Content
M i T il 1 “M l’ Th A ”Movie Trailer 1 “Marvel’s The Avengers”
Movie Trailer 2 “House at the End of the Street”
Commercial 1 Budweiser
Commercial 2 Mercedes C63 AMG Black Series
Music Video 1 “Put Your Graffiti on Me” – Kat Graham
Music Video 2 “The Veldt” - Deadmau5
User-Created 1 Skydiving
User Created 2 Motocross race
User-Created 2 Motocross race

Laboratory Set-Upy p
Headphones
GSR SensorPaper survey Smartphone

Video App

Research methodsExperimental Design
Training/BaselineTraining/Baseline
Session 1.Session 1.
Watch 8 videos; doWatch 8 videos; doWatch 8 videos; do
ratings
•Haptic Condition
Watch 8 videos; do
ratings
•Haptic Conditionp
•Control
p
•Control
Feature AssessmentsFeature Assessments

Watch 8 videos; doWatch 8 videos; do
Watch video
Complete questions
Watch 8 videos; do
ratings
•Haptic Condition
Watch 8 videos; do
ratings
•Haptic Condition
Rest period
X 8
p
•Control
p
•Control
Feature AssessmentsFeature Assessments

~7 days
Warm-UpWarm-Up
Watch 8 videos; doWatch 8 videos; do Watch 8 videos; doWatch 8 videos; do
Watch video
Complete questions
Watch 8 videos; do
ratings
•Haptic Condition
Watch 8 videos; do
ratings
•Haptic Condition
Watch 8 videos; do
ratings
•Control
Watch 8 videos; do
ratings
•Control
Rest period
X 8
p
•Control
p
•Control •Haptic Condition•Haptic Condition
ll
Feature AssessmentsFeature Assessments Final AssessmentsFinal Assessments

Research Methods - Data
Haptic Enhancement No Enhancement
• Physiological measure – arousal (GSR)
• Self-Reportsp
• Affect Grid – valence & arousal
• Wells R Scale - involvement with content
• Quality of Experience (QoE)Quality of Experience (QoE)
• Descriptive, qualitative feedback

Res ltsResults

Physiological DataPhysiological Data
EDR

Physiological DataPhysiological Data
Physiological data reflecty g
higher arousal levels in
response to videos with
Reverb appliedReverb applied.
Heightened psychological
aro sal has been linked toarousal has been linked to
ad effectiveness and
persuasion*
Reverb Control
p = 0.006
*(Sanbonmatsu and Kardes, 1988)

Affect Grid DataAffect Grid Data

Avg. Valence = 6.3 Avg. Valence = 5.8
A A l 5 1
Avg. Arousal = 6.1 Avg. Arousal = 5.1

A A l 5 1
p < 0.05
Avg. Arousal = 6.1 Avg. Arousal = 5.1

A A l 5 1
p < 0.05
Avg. Arousal = 6.1 Avg. Arousal = 5.1p < 0.004

Emotional Involvement Data
“I felt like I was right there in the video, experiencing the situation.”
“I really got involved in the emotions provoked by the video.”
Users reported feeling more
immersed and more
emotionally involved with
- Reverb
- Control
emotionally involved with
video content enhanced by
haptic effects.
p < 0.003 p < 0.008

QoE DataQoE Data
Quality of Experience
R b
QoE ratings on
average 15%
higher for
- Reverb
- Control
higher for
Reverb videos
vs. controls.

Qualitative Data
“It’s like the difference of
ki t t
“It was like supplementing
or helped me recall true
experience It brought me
Qualitative Data
making you want to
dance and not dance.”
experience. It brought me
back to the memory of
something, like driving a
cool car or skydiving.”
“It totally enhances the
experience. It kicks it up a
p p
notch. I feel like I’m in a movie
theater.”

Summary
H ti E h d
Summary
Haptic Enhanced
Mobile Video
 More positive and enthusiastic response to video content
 Higher level of emotional involvement
 Higher quality of experience ratings
Higher quality of experience ratings

Impact
Haptic Content Strategic Initiative
F d
Impact
For end-users:
Haptics as a value-added feature to mobile experiences
For content producers:
Haptics as a new medium for creative expression
For advertisers:
Haptics as a novel mechanism to engage customers
For Immersion:
A new business unit!

Measuring Emotiong
in User Experiences
T E l Di d Ad t ( ) Di d t ( )Type Examples Discussed Advantages (+) Disadvantages (-)
Physiological
Measures
Electrodermal
Activity
Powerful &
persuasive data,
“objective” measure
Requires rigorous
experimental design,
very sensitive toobjective measure very sensitive to
exposure, expensive
equipment & software,
takes significant time &
effort to analyze data;effort to analyze data;
arousal, not valence
Self-Report Affect Grid,
Involvement (Wells R
Easy to train
participants to use
Ultimately, a subjective
measure; effect size canInvolvement (Wells R
Scale)
participants to use,
arousal & valence,
relatively easy to
analyze, quantitative
analysis possible
measure; effect size can
be hard for subjects to
estimate
analysis possible.

Thank You!
aweddle@immersion.com

DiscussionDiscussion
Interactive Exercise – Using Emotion Measures in UX
Imagine using emotion measures in your own UX work.
What would represent an ideal outcome? (Your island paradise)
What would represent an ideal outcome? (Your island paradise).
What is holding you back from achieving that goal? (Your anchors).

Appendix.

Self Report – Emocardsp
Image from Agarwal and Meyer, 2009, p. 2923

Self Report – Emocardsp
Participants have a hard
time understandingtime understanding
cartoon faces without
textual labels
Comparisons are
qualitative

QoE Timeline Data
64.6
LFE
71.9
Authored
54.8
No Haptics
54.8
Mean opinion scores of all participants (including “outliers”) is plotted as a bold, colored line.
Grey shading represents the 95% confidence interval for each rating point. (Rating data was gathered at a rate of one rating/s).
Overall grand mean of all ratings is represented as a dotted line.

Characterizing the Emotional Impacts of Audio-Haptic Mobile Media Enhancement

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Characterizing the Emotional Impacts of Audio-Haptic Mobile Media Enhancement

Semelhante a Characterizing the Emotional Impacts of Audio-Haptic Mobile Media Enhancement (20)

Mais de UXPA International

Mais de UXPA International (20)

Último

Último (20)

Characterizing the Emotional Impacts of Audio-Haptic Mobile Media Enhancement