Omnidirectional video (ODV) streaming applica- tions are becoming increasingly popular. They enable a highly immersive experience as the user can freely choose her/his field of view within the 360-degree environment. Current deployments are fairly simple but viewport-agnostic which inevitably results in high storage/bandwidth requirements and low Quality of Experience (QoE). A promising solution is referred to as tile- based streaming which allows to have higher quality within the user’s viewport while quality outside the user’s viewport could be lower. However, empirical QoE assessment studies in this domain are still rare. Thus, this paper investigates the impact of different tile-based streaming approaches and configurations on the QoE of ODV. We present the results of a lab-based subjective evaluation in which participants evaluated 8K omnidirectional video QoE as influenced by different (i) tile-based streaming approaches (full vs. partial delivery), (ii) content types (static vs. moving camera), and (iii) tile encoding quality levels determined by different quantization parameters. Our experimental setup is character- ized by high reproducibility since relevant media delivery aspects (including the user’s head movements and dynamic tile quality adaptation) are already rendered into the respective processed video sequences. Additionally, we performed a complementary objective evaluation of the different test sequences focusing on bandwidth efficiency and objective quality metrics. The results are presented in this paper and discussed in detail which confirm that tile-based streaming of ODV improves visual quality while reducing bandwidth requirements.

1. Tile-based Streaming of 8K
Omnidirectional Video: Subjective and
Objective QoE Evaluation
Raimund Schatz
AIT Austrian Institute
of Technology
11th International Conference on Quality of Multimedia Experience (QoMEX 2019)
June 5-7, 2019, Berlin
Christian Timmerer
Alpen-Adria-Universität
Klagenfurt / Bitmovin Inc.
Anatoliy Zabrovskiy
Alpen-Adria-Universität
Klagenfurt

2. § QoE of Omnidirectional Video (ODV, aka 360º Movie) Streaming
§ Immersive panorama video surrounding the user
§ User can turn head to change gaze direction
§ ODV streaming = content class with
dramatically increasing popularity
§ Cisco VNI: globally, virtual–reality traffic will
increase 61-fold between 2015 and 2020
Focus of this Talk & Motivation
208.06.19 © AIT
Immersive
Media
VR
Responsive
VR
ODV
MR AR

3. Motivation
§ Current deployments of ODV: technically straightforward …
Drawback: viewport-agnostic!
à High storage & bandwidth requirements
à Impaired QoE (under constrained conditions)

4. Tile-based Streaming to the Rescue!
Mario Graf, Christian Timmerer, and Christopher Mueller. 2017. Towards Bandwidth Efficient
Adaptive Streaming of Omnidirectional Video over HTTP: Design, Implementation, and Evaluation.
In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys'17). ACM, New York,
NY, USA, 261-271. DOI: https://doi.org/10.1145/3083187.3084016
https://multimediacommunication.blogspot.com/2017/04/acm-mmsys17-special-session-paper.html

5. Tile-based Streaming, The Challenge:
Lots of Knobs & Levers …
§ Tiling Patterns?
§ Encoding Quality Levels?
§ Delivery Strategy?
§ …
àWANTED: Valid Ground Truths & Guidelines
based on subjective ODV Quality Studies!
à This Talk: Results (subjective & objective) from
a Joint ODV QoE Study by AIT & AAU
508.06.19

6. The Study
608.06.19

7. Research Questions
RQ1: How does full vs. partial delivery impact ODV
streaming QoE? Is there a clear user preference?
RQ2: What is the QoE impact of different ODV tile quality
encoding levels? (RQ2)
RQ3: Do camera movement (static vs. moving) or head
turning speed exert an influence on quality perception?
708.06.19

8. Subjective Lab Test: SRCs & HRCs
§ SRCs: Two 8k high-quality source clips (duration: 30s)
§ HRCs, Factors:
§ Delivery: Full vs. Partial
§ Head turn: Slow vs. Fast
§ Tile encoding quality:
QP 46, 32, 22
808.06.19

9. SRCs: Source Clips (8k)
908.06.19
URLs:
http://medialab.sjtu.edu.cn/vr8K/index.html
https://www.itu.int/en/ITU-T/studygroups/2017-2020/16/Pages/video/jctvc.aspx

10. HRCs: Full vs. Partial Delivery
1008.06.19
Gray Tiles

11. HRCs: Head-Turn
§ Head turning movement in the middle of the clip
§ 90 deg to the right
§ Fast (1s) vs. Slow (4s)
§ Challenge: slight changes in head movement à huge differences in tile-
based streaming behavior
à Solution: render FoV into PVS!
1108.06.19

12. Processing Pipeline: from 360 SRCs to 2D PVSes
1308.06.19

13. Subjective Lab Test: Setup
§ Tiled ODV QoE Lab Study Sessions in Summer 2018
§ Setup: Large 4k screen (65“, SONY) viewed at 1,5m distance, ratings via
tablet
1408.06.19
Similar setup used in
standardization,
e,g, Versatile Video
Coding (VVC)

14. Subjective Lab Test: Protocol
1508.06.19
Ratings provided on table using the open source software http://www.thefragebogen.de

15. Test Subjects
§ N=35
§ 21 male, 14 female
§ 50% had (some) VR experience
§ Mixed education / backgrounds
§ Age: mean 32, median 33, range: 22-50 yrs
1608.06.19 nafi 4.0

16. The Results
1708.06.19

17. Subjective Ratings: MOS [0-100]
18
§ Delivery: Partial Delivery à visibly low QoE
§ Tile Encoding Quality: QP46 = significant QoE drop, but only very little
difference between QP22 & 32
QP: 46 32 22 | 46 ….

18. Subjective Ratings: Acceptability
19
§ Acceptability Ratings: confirm MOS results
§ In particular: Partial Delivery = really unacceptable
Full Delivery Partial Delivery
QP: 46 32 22 | 46 ….

19. Mixed Model ANOVA Results (Full Delivery only)
2008.06.19
Head
movement
Content /
Cam movement

20. Objective Metrics: wPSNR, SSIM and VMAF for FoV PVS
§ Low sensitivity of SSIM w.r.t. full vs. partial delivery
§ Little impact of content / cam movement speed
§ Consistent impact of head movement speed (but not pronounced, too)
2108.06.19
Results for segments 3-4-5 (i.e. head turn part only):
More objective analysis results à see the paper!

21. Findings & Conclusions
§ RQ1 (Full vs partial delivery):
§ Avoid partial delivery (or find better visual workarounds), since
unacceptable for end users
§ RQ2 (Impact of tile encoding quality):
§ Sufficient to use QP 32 for tile encoding for practical deployments as
saturation kicks in here
§ RQ3 (Impact of camera movement and head turn speed):
§ Camera movement masking low encoding quality (known)
§ Head turn speed: only weak impact, interaction with content/cam
movement (+ for static, - for dynamic scene)
§ Objective Evaluation:
§ Largely in line with subjective results, with some exceptions depending on
metric and impairment
2208.06.19

22. Issues, Challenges & Outlook
§ Several limitations: 2D Display used (not HMD), limited number of SRCs,
head-motion patterns used, etc.
§ Further research needed!
§ Future studies featuring 2D Display vs. HMD viewing impact, larger variety
of content, head-motions, etc.
§ Also: tile-based rendering pipeline = technically challenging, lots of tweaking
& error-checking required à joint efforts required
2308.06.19

23. Thank you for your attention!
Raimund Schatz
AIT Austrian Institute of Technology GmbH
Donau-City-Str. 1, 1220 Vienna
raimund.schatz@ait.ac.at
This work was supported in part by the Austrian Research Promotion Agency
(FFG) under the Next Generation Video Streaming project “PROMETHEUS.
Christian Timmerer
Alpen-Adria-Universität
Klagenfurt / Bitmovin Inc.

24. Objective Metrics: Metrics Calculated for Whole 8k Content

25. Questionnaire
2608.06.19