1.
m26906Live Streaming of Video and Subtitles using DASH Cyril Concolato Jean Le Feuvre {concolato, lefeuvre} @ telecom-paristech.fr

2.
Context ■ Need for live subtitles also on the Internet! ■ Multiple specification developments • MPEG DASH Amendment on Timed Text and Timed Overlays • W3C Media Source Extension API • WHATWG/W3C WebVTT • 3GPP Timed Text ■ Development of a Proof-of-Concept1 M26906 – Live Streaming of Video and Subtitles using DASH

3.
Demonstration formats ■ The MP4 file format • To represent live video stream segments ■ The WebVTT subtitle format • To represent live subtitle segments as text files (similar to HLS v9) ■ MPEG DASH MPD • to represent a dynamic adaptive live/long-running streaming service • to enable synchronization of the video and subtitle streams.2 test

4.
Demonstration software components ■ GPAC MP4Box • To produce video segments on-the-fly and delete old ones ■ VTTLiveGenerator • Ad-hoc tool • To produce subtitle segments on-the-fly and delete old ones ■ Google Chrome • Canary version 24 • To display the content3 test

5.
Usage of MP4Box ■ Fragmenting a file repeatedly using -dash-ctx option MP4Box.exe -dash-ctx dash-live.txt -dash 2000 -rap -segment-name counter-live -url-template counter.mp4 ■ Example of MP4Box’s DASH context [DASH] InitializationSegment=counter_liveinit.mp4 BitstreamSwitching=yes InitializationSegmentSize=860 MaxSegmentDuration=1.000000 NextSegmentIndex=11 NextFragmentIndex=11 PeriodDuration=9.960000 [TrackID_1] NextDecodingTime=2490004 test

6.
Example of Video Content5 test

7.
Example of WebVTT Segment ■ Each segment is a complete WebVTT file with multiple cues, • changing header, cue indices, and timing values WEBVTT Segment 6 61 00:00:12.200 --> 00:00:12.400 This is cue 61 (start: 00:00:12.200 -- end: 00:00:12.400) ... 70 00:00:14.000 --> 00:00:14.200 This is cue 70 (start: 00:00:14.000 -- end: 00:00:14.200)6 test

8.
MPD Example ■ Dynamic MPD • 1 single period • 2 adaptation sets (1 video, 1 subtitle) − ContentComponent@type=« text » for subtitles • No minUpdate attribute • Same video and subtitle segments duration (easy) ■ live-vtt-mp4.mpd7 test

9.
Rendering using Google Chrome Canary ■ HTML page with JavaScript code to: • Fetch and parse the MPD, • Create a HTML 5 <video> element to render the video and an HTML 5 <track> element to render the subtitles, • Fetch the video and subtitles segments using the XMLHttpRequest API (as demonstrated in DASH-JS), • Feed the video segments to the media decoder engine using the Media Source extension API (as demonstrated by YouTube MPEG DASH player), • Load each subtitle segment separately (using a work- around current HTML 5 limitations) ■ Synchronization is handled by the video element • Possible improvments: HTML5 timestampOffset and DASH @presentationTimeOffset and @suggestedPresentationDelay8 test

10.
Results9 test

11.
Problems/Limitations ■ Synchronization problems • Too many cues per seconds (Chrome performance?) • Some cues are not loaded correctly (Chrome bug?) • Need a work-around to load cues (HTML5 limitation) − No MSE equivalent for cues − No easy method to parse cues ■ Limitations • Really only a proof of concept (ugly code) • No real MPD parsing yet • Very simple segment index algorithm • No GUI for selecting subtitles10 test

12.
Questions ?