1. INVESTIGATING THE IMPACT OF
SENSORY EFFECTS ON THE QUALITY OF
EXPERIENCE AND EMOTIONAL RESPONSE
IN WEB VIDEOS
Benjamin Rainer, Markus Waltl, Eva Cheng, Muawiyath Shujau, Christian Timmerer,
Stephen Davis, Ian Burnett, Christian Ritz, Hermann Hellwagner
Multimedia Communication (MMC) Research Group, Institute of Information Technology (ITEC), Alpen-Adria-
Universität Klagenfurt, Klagenfurt, Austria, firstname.lastname@itec.uni-klu.ac.at
School of Electrical and Computer Engineering, Royal Melbourne Institute of Technology, Melbourne, VIC,
Australia, 3000,firstname.lastname@rmit.edu.au
School of Electrical Computer and Telecommunications Engineering, University of Wollongong, Wollongong, NSW,
Australia, 2522, {mshujau, stdavis, critz}@uow.edu.au
Acknowledgments. This work was supported in part by Smart Services CRC
(Australia) and the EC in the context of the ALICANTE (FP7-ICT-248652),
SocialSensor (FP7-ICT-287975), and QUALINET (COST IC 1003) projects.

2. OVERVIEW
 Motivation
 Software
 Methodology
 Results
 Conclusion
15 July 2012 2

3. MOTIVATION
 Sensory effects (e.g., wind, vibration, light) are a
fairly new research area
 Only few user studies on sensory effects (e.g.,
scent-based effects, light-based effects)
 Goal: to assess the enhancement of the user
experience and the influence of sensory effects on
the emotions
15 July 2012 3

4. HARDWARE
Test Setup amBX Premium Kit
15 July 2012 4

5. SOFTWARE
Ambientlibrarywith Web
browserplug-in
SensoryEffect Video
Annotation (SEVino)
15 July 2012 5

6. METHODOLOGY
 Subset of our sensory effect dataset
 15 Web video sequences Sequence Name Genre Bit-rate
(Kbit/s)
Length
(sec)
Wind/
Vibration
2012 Action 2186 29.1 6/8
 5 genres Prince of Persia
Tron Legacy
Action
Action
2114
2379
24.89
25.08
7/6
7/4
STS131 Launch News 2812 30.09 7/5
 Action, news, commercial Tornado
Etna erupts
News
News
1299
3165
31.03
40.07
4/12
19/13
documentary, sports GoPro HD Th. Racing
Verizon
Commercial
Commercial
2429
1819
30.09
30.15
8/3
4/4
Audi Commercial 2245 30.19 9/5
AAU Klagenfurt RMIT University UoW
Volcano Britain Documentary 2133 33.1 10/4
amBX Premium Kit (Fan, Vibration Panel, Light, Sound); African Cats Documentary 2562 19.1 6/1
24" Monitor with a resolution of 1400x1050 The Last Lions Documentary 1850 37.04 25/6
Mozilla Firefox 6 & 8 in full-screen mode GoPro HD Berrecloth Sports 3552 32.08 11/23
Ambient Library 1.5 & Web browser plug-in 1.5 Travis Pastranas Rally Sports 2619 32.08 8/8
amBX Software (amBX System 1.1.3.2 and Philips amBX 1.04) GoPro HD Ronnie R. Sports 2245 23.16 7/7
HP Z400: Intel
Dell Optiplex 655:
Xeon Quad Core Intel Quad Core 3
Pentium D 2.8 GHz w/
3.33 GHz w/ 6 GB GHz w/ 3GB RAM &
1 GB RAM & ATI
RAM & Nvidia Intel Q35 384MB
Radeon HD 5450
Quadro FX 1800
Windows XP SP3 Windows 7 64-bit Windows XP SP3
15 July 2012 6

7. METHODOLOGY
 Online test
 Introduction
 Textual introduction
 Provides same introduction to each participant
 Pre-questionnaire
 Gender, age, education, country, occupation
 Actual quality assessment
 Modified DSCQS
 Post-questionnaire
 Duration about 20 minutes for each participant
15 July 2012 7

8. METHODOLOGY
 Web video sequences were selected randomly
 Modified Double Stimulus Continuous Quality Scale (DSCQS)
 T1 … Reference Web video sequence without sensory effects
 T2 … Voting possibility for emotions for the reference sequence
 List of selected emotions
 Taken from: T. Tsoneva, P. Fonseca, J. Skowronek, "Towards a groundtruthforaffectiveclassification
in movies", 11th IEEE International Symposium on Multimedia 2009 (ISM'09), IEEE Computer Society,
Washington, DC, USA, 2009.
 Eachemotion was additionallypresentedwith a scope
 Participant had to choose one or more emotions and rate their intensity from 0 (very
weak) to 100 (very strong)
 T3 … Web Video sequence with sensory effects
 T4 … Voting possibility for emotions for the sequence with sensory effects
 Same as T2 but with sensory effects
 Add’l enhancement of the QoE sequence with sensory effects with respect to the
sequence without sensory effects (continuous scale from 0 to 100)
15 July 2012 8

9. METHODOLOGY
 RMIT University
 21 participants
• 12 female and 9 male
 Aged between 22 to 58  AAU
 26 participants
 University of • 18 female and 8 male
Wollongong  Aged between 20 to 57
 21 participants
• 6 female and 15 male
 Aged between 22 to 53
15 July 2012 9

10. RESULTS
Emotionswithsignificantdifferences
EnhancementofQoEforeachgenrefor all betweenwithandwithoutsensoryeffe
threeuserstudies ctsfor RMIT, Australia
15 July 2012 10

11. EMOTIONS
Emotions with significant Emotions with significant
differences between with and differences between with and
without sensory effects for UoW, without sensory effects for AAU,
Australia Austria
15 July 2012 11

12. CONCLUSIONS
 Sensory effects enhance the QoE
 Some genres are better suited for sensory effects
(e.g., action, sports) than others
 Sensory effects influence the users’ emotions and
their intensity
 Sensory effects positively influence active emotions (e.g.,
worry, fun) active emotions are increased in their
intensity
 General conclusions on the influence on passive emotions
(e.g., boredom, tiredness) cannot be stated
15 July 2012 12

13. THANK YOU FOR YOUR ATTENTION
15 July 2012 13