This presentation was made at the MPEG meeting in Shanghai, China, in October 2012, related to the input contribution M26906. It gives the details about the demonstration made during the meeting. This demonstration showed the use of the Google Chrome browser to display synchronized video and subtitles, using the Media Source Extension draft specification and the WebVTT subtitle format. The video and DASH content was prepared using GPAC MP4Box tool.

1. m26906
Live Streaming of Video and Subtitles
using DASH
Cyril Concolato
Jean Le Feuvre
{concolato, lefeuvre} @ telecom-paristech.fr

2. Context
■ Need for live subtitles also on the Internet!
■ Multiple specification developments
• MPEG DASH Amendment on Timed Text and Timed
Overlays
• W3C Media Source Extension API
• WHATWG/W3C WebVTT
• 3GPP Timed Text
■ Development of a Proof-of-Concept
1 M26906 – Live Streaming of Video and Subtitles using DASH

3. Demonstration formats
■ The MP4 file format
• To represent live video stream segments
■ The WebVTT subtitle format
• To represent live subtitle segments as text files (similar
to HLS v9)
■ MPEG DASH MPD
• to represent a dynamic adaptive live/long-running
streaming service
• to enable synchronization of the video and subtitle
streams.
2 test

4. Demonstration software components
■ GPAC MP4Box
• To produce video segments on-the-fly and delete old
ones
■ VTTLiveGenerator
• Ad-hoc tool
• To produce subtitle segments on-the-fly and delete old
ones
■ Google Chrome
• Canary version 24
• To display the content
3 test

5. Usage of MP4Box
■ Fragmenting a file repeatedly using -dash-ctx option
MP4Box.exe -dash-ctx dash-live.txt
-dash 2000 -rap
-segment-name counter-live
-url-template
counter.mp4
■ Example of MP4Box’s DASH context
[DASH]
InitializationSegment=counter_liveinit.mp4
BitstreamSwitching=yes
InitializationSegmentSize=860
MaxSegmentDuration=1.000000
NextSegmentIndex=11
NextFragmentIndex=11
PeriodDuration=9.960000
[TrackID_1]
NextDecodingTime=249000
4 test

6. Example of Video Content
5 test

7. Example of WebVTT Segment
■ Each segment is a complete WebVTT file with multiple
cues,
• changing header, cue indices, and timing values
WEBVTT Segment 6
61
00:00:12.200 --> 00:00:12.400
This is cue 61 (start: 00:00:12.200 -- end:
00:00:12.400)
...
70
00:00:14.000 --> 00:00:14.200
This is cue 70 (start: 00:00:14.000 -- end:
00:00:14.200)
6 test

8. MPD Example
■ Dynamic MPD
• 1 single period
• 2 adaptation sets (1 video, 1 subtitle)
− ContentComponent@type=« text » for subtitles
• No minUpdate attribute
• Same video and subtitle segments duration (easy)
■ live-vtt-mp4.mpd
7 test

9. Rendering using Google Chrome Canary
■ HTML page with JavaScript code to:
• Fetch and parse the MPD,
• Create a HTML 5 <video> element to render the video
and an HTML 5 <track> element to render the subtitles,
• Fetch the video and subtitles segments using the
XMLHttpRequest API (as demonstrated in DASH-JS),
• Feed the video segments to the media decoder engine
using the Media Source extension API (as
demonstrated by YouTube MPEG DASH player),
• Load each subtitle segment separately (using a work-
around current HTML 5 limitations)
■ Synchronization is handled by the video element
• Possible improvments: HTML5 timestampOffset and
DASH @presentationTimeOffset and
@suggestedPresentationDelay
8 test

10. Results
9 test

11. Problems/Limitations
■ Synchronization problems
• Too many cues per seconds (Chrome performance?)
• Some cues are not loaded correctly (Chrome bug?)
• Need a work-around to load cues (HTML5 limitation)
− No MSE equivalent for cues
− No easy method to parse cues
■ Limitations
• Really only a proof of concept (ugly code)
• No real MPD parsing yet
• Very simple segment index algorithm
• No GUI for selecting subtitles
10 test

12. Questions ?