Video streaming is one of the top traffic contributors in the Internet and a frequent research subject. It is expected that streaming traffic will grow 4-fold for video globally and 9-fold for mobile video between 2017 and 2022. In this paper, we present an automatized measurement framework for evaluating video streaming QoE in operational broadband networks, using headless streaming with a Docker-based client, and a server-side implementation allowing for the use of multiple video players and adaptation algorithms. Our framework allows for integration with the MONROE testbed and Bitmovin Analytics, which bring on the possibility to conduct large-scale measurements in different networks, including mobility scenarios, and monitor different parameters in the application, transport, network, and physical layers in real-time.
2024: Domino Containers - The Next Step. News from the Domino Container commu...
Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
1. Docker-Based Evaluation Framework for
Video Streaming QoE in Broadband Networks
Cise Midoglu1
, Anatoliy Zabrovskiy2
, Özgü Alay3
, Daniel Hölbling-Inzko4
, Carsten
Griwodz5
, Christian Timmerer2,4
1
Simula Research Laboratory, 2
University of Klagenfurt, 3
Simula Metropolitan Center for Digital
Engineering, 4
Bitmovin Inc, 5
University of Oslo
OS-05
2. Motivation
● Video streaming one of the top traffic contributors in the Internet
● Mobile broadband (MBB) networks becoming more and more prominent
● Metadata-rich measurement result
collection is challenging
● Over-the-top (OTT) video analytics
are not used widely in research
OS-05 2
4. Overview
● Automatised measurement and evaluation framework for video streaming
Quality of Experience (QoE)
○ End-to-end with client and server side
○ Client-initiated active measurements
○ Docker virtualization → platform compatibility
○ Headless streaming → focus on objective QoE
○ Allows to monitor and collect data from multiple layers (application, transport,
network, physical)
○ Allows to benchmark video players, ABR algorithms, networks, operators, ...
OS-05 4
5. Client Side
● Designed to emulate real user initiating
a video streaming session
● Selenium instrumentation opens
headless browser and streams from
HTTP server
● Implemented as Docker container
● Compatible with mobile broadband
testbeds (e.g., MONROE)
● Configurable (network, player, ABR
algorithm, run duration, batch
experimentation, ...)
● Additional network monitoring (ping +
traceroute), and metadata if available
OS-05 5
6. Server Side
OS-05 6
● Server hosts landing pages with 3
different video players
○ Bitmovin Player v8
○ Shaka Player v2.5
○ DASH.js Player v2.9
7. Server Side
OS-05 7
● All test pages use the same asset
○ BigBuckBunny video
○ Encoded in 15 qualities (100 Kbps to
15 Mbps)
○ Retrieved from CDN
8. Server Side
OS-05 8
● All test pages are integrated with
Bitmovin Analytics web collector v2.3
○ JavaScript executed on client browser
in runtime
9. Server Side
OS-05 9
● All pages are integrated with Bitmovin
Analytics web collector v2.3
○ More than 70 parameters recorded for
each session (including startup time,
video quality, num. switches, stalls, ...)
10. Server Side
OS-05 10
● All pages are integrated with Bitmovin
Analytics web collector v2.3
○ Possible to view metrics in real-time
from the Bitmovin Dashboard, or export
in raw format using the Bitmovin
Analytics Export Service
16. 16
• Two grand challenges
– Improving open-source HEVC encoding
– Low-latency live streaming
• Focus areas in 2020
– Machine learning and statistical modeling for video streaming
– Volumetric media: from capture to consumption
– Fake media and tools for preventing illegal broadcasts
• A workshop (posters/demos) dedicated to middle and high-school students
• Two confirmed keynotes from Google and MIT
• Expecting reduced registration fees thanks to strong support
Important Dates Submit by
Research Track Jan. 10 (firm)
Demo Track Feb. 29
Open Source/Dataset Feb. 29
Workshops Mar. 27
Conference June 8-11
NEW
Visit http://acmmmsys.org today!
NEW
17. VBIM: Video QoE Dataset Generation
Docker-Based Evaluation Framework for Video Streaming QoE in Broadband Networks
Cise Midoglu, Anatoliy Zabrovskiy, Özgü Alay,
Daniel Hölbling-Inzko, Carsten Griwodz, Christian Timmerer
Simula Research Laboratory, University of Klagenfurt,
University of Oslo, Bitmovin Inc
Open Source Software Competition
Thursday 24 October
10:30 - 12:00
Risso 6