Presentation given during the HTML5 Dev Conf and The Graphical Web, showing how to create and play SVG content in a streaming manner.

1. Streaming
SVG Animations
on the Web
Cyril Concolato
HTML5 Dev Conf / The Graphical Web
22-23 Oct 2013
Cyril Concolato

2. Context and Motivations
 Animating using web technologies is getting popular
• Lots of content being produced
 « SVG 2 » and « Web Animations 1.0 » being drafted
• Opportunities to improve the technologies
 My goal: Enable treating web animations like video
• Why?
• What kind of animations?
• How?
2
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

3. Web Animation Use Cases
 Animations in User Interfaces
 Animations for Data Visualization
 Timeline-based Animations
•
•
•
Graphics animations (cartoons, …)
Synchronized audio & graphics (karaoke, ads …)
Synchronized video & graphics (annotations, subtitles)
Popcorn.js
3
23/10/2013
SVG Wow!
Cyril Concolato
Cartoon
Streaming SVG animations on the Web

4. Timeline-based Animations Possibilities
 Playback control using a clock
• Play(0)/Play(T)
• Pause/Resume
• Speed-up/slow down
 Accurate synchronization
• Between animations or
between animation and media
• Dependent clocks and timestamps
Similar to video
but not allowed in HTML5 <video>
4
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

5. An SVG video-like JavaScript Player
Principles
 Using HTML 5 <video> syntax but for SVG
content
Demo 1
1
2
3
<video controls width="640" height="480">
<source src="myAnimatedFile.svg" type="image/svg+xml" >
</video>
Demo 2
1
2
3
4
<video controls width="640" height="480">
<source src="file.mp4" type="video/mp4" >
<track src="annotations.svg" kind="graphics" type="image/svg+xml">
</video>
5
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

6. An SVG video-like JavaScript Player
Status
 Simple Implementation
•
•
•
•
GUI based on Video.js
Replacing <video> by <svg> at run-time
Using XHR to fetch SVG data and insert in the page
Uses get/setCurrentTime() and un/pauseAnimations()
 Limitations
•
•
•
•
6
23/10/2013
Bugs!
Same origin restrictions
Incremental/progressive loading not working
Cannot reproduce HTML network-related events (can
play, can play through)
Cyril Concolato
Streaming SVG animations on the Web

7. An SVG video-like JavaScript player:
Next Step
 Network-aware Controlled Playback
•
Load SVG data just before it’s needed (and let garbage collection
do the rest)
 How to achieve controlled playback of SVG ?
•
7
23/10/2013
Apply streaming concepts
Cyril Concolato
Streaming SVG animations on the Web

8. Applying streaming concepts to SVG
 Streaming
“Controlled continuous delivery and consumption of data units”
 What’s needed to stream SVG content?
•
Fragmentation of SVG documents into streaming units
─ Needs guidelines for SVG authoring
•
Assign timing to each unit
─ Mapping between byte-offset/time-position of the SVG document
─ Needs a streaming instruction format
•
Seek information
─ Needs explicit processing behavior
 SVG Streaming draft
•
•
http://dev.w3.org/SVG/modules/streaming/
Feedback welcome
 Example: Streaming SVG Cartoons
8
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

9. Cartoons on the Web – Today
Source: www.cartoonnetwork.com
 Largely based on Vector Graphics
•
•
Bezier curves filled with solid color or simple gradients and/or stroke
Sometimes using raster images (e.g. background or texture)
 Mostly frame-based content
•
•
•
Sometimes with additional interpolation
Typically synchronized with a sound track
Good candidate for streaming
 Currently delivered as Flash or video streams
•
9
23/10/2013
Plugins, coding artefacts or bandwidth problems …
Cyril Concolato
Streaming SVG animations on the Web

10. Cartoons on the Web – Next Steps
 Could be better integrated in the web platform
•
Using of vector graphics primitives for scalability and new
primitives for advanced rendering
─ E.g. coons patches, hatches (SVG 2), diffusion curves, …
•
Using other web technologies
─ Styling, language translation, …
 Attempts to convert Flash to HTML5/CSS/SVG/Canvas
•
Many JS-based approaches
─
─
─
─
•
Google Swiffy
Mozilla Shumway
Adobe CreateJS
PixelPlant
Some declarative approaches
─ MP4Box (100% SVG)
10
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

11. Frame-based SVG Cartoons using MP4Box
<svg viewBox="0 0 768 576" ...>
<rect width="100%" height="100%" fill="lightblue"/>
<g id="frame_1" display="none">
<animate attributeName="display" to="inline" begin="0" end="0.16666667"/>
<defs>
<g id="Shape1"><path fill="rgb(111,90,41)" stroke="none" d="M 524.0 -109.0 ..."/>...</g>
<g id="Shape2"><path fill="rgb(111,90,41)" stroke="none" d="M 524.0 -109.0 ..."/>...</g>
<g id="Shape3">...</g>
</defs>
<use xlink:href="#Shape1" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape2" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape3" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
</g>
<g id="frame_2" display="none">
<animate attributeName="display" to="inline" begin="0.16666667" end="0.25"/>
<defs>
<g id="Shape4"><path .../></g>
</defs>
<use xlink:href="#Shape1" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape4" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
<use xlink:href="#Shape2" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/>
</g>...
</svg>
11
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

12. Fragmentation of SVG document
into streaming units
 Identify stream header
•
•
Static throughout the streaming session
Required to initialize the parser
 Make sure SVG data is properly ordered
•
•
Use only backward references
Document order == time order
 Identify units
•
•
Assign timestamps
No overlap in position / possible overlap in time (like WebVTT cues)
 Make self-discardable units
•
<discard>, JavaScript, scoped stylesheets
 Mark self-contained units
•
12
23/10/2013
Random Access Point
Cyril Concolato
Streaming SVG animations on the Web

13. Streaming Instructions Example
<stream timescale="24" file="cuisine.swf.svg">
<header range-end="204"/>
<unit time="0" range-end="353" rap="1" />
<unit time="2" range-end="38403" rap="0" />
<unit time="3" range-end="39591" rap="0" />
<unit time="5" range-end="41923" rap="0" />
<unit time="6" range-end="45825" rap="0" />
<unit time="7" range-end="51276" rap="0" />
<unit time="10" range-end="52596" rap="1" />
<unit time="12" range-end="55261" rap="0" />
…
</stream>
13
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

14. How to generate Streaming Instructions ?
 Post-authoring
• Requires byte positions to be final
 Hard to determine without author’s instructions
• Where does the frame start/end?
• What’s the timestamp of the unit?
• Where are the random access points?
 Possible automatic generation from Flash
 Possible with additional elements/attributes
14
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

15. Flash-to-SVG streaming
SWF
file
Converter
SVG
file
Converter &
Packager
Packager
MP4
file
Extractor
SVG
file
15
23/10/2013
Cyril Concolato
Streaming
Instructions
file
Streaming SVG animations on the Web

16. Generating Streaming Instructions
for SVG content converted from Flash
MP4Box -add flash.swf:fmt=svg file_with_svg_track.mp4
MP4Box -info file_with_svg_track.mp4
MP4Box -six 1 file_with_svg_track.mp4
16
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

17. SVG Streaming Demo
SVG
+
Streaming
Instructions
Demo 3
•
•
•
•
•
Web Server
(byte-range support)
Browser
+
JS Player
Playing an SVG stream
Viewing Random Access Points and others
Seeking into an SVG unit not yet downloaded
Concatenating SVG content (splicing)
Live and pseudo-live streaming of SVG content
─ Web-based Vector Graphics TV channel
17
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

18. SVG streams and Media Containers
MP4
Flash
MP4
SVG
MP4Box
Video
SVG
Audio
 Why? What for?
•
Packaging audio/video/graphics together
─ To download only one resource
─ To facilitate synchronization of tracks
•
Possible use with HTML5 native <video> elements
─ SVG in-band graphical track
•
Possible use with HTML 5 Media Source extensions
─ Adaptive streaming of SVG
 Formats
•
•
18
23/10/2013
ISO/IEC-14496:12 (royalty-free) defines how to store
timed text in MP4
Could be done for other formats (WebM, Ogg, …)
Cyril Concolato
Streaming SVG animations on the Web

19. SVG streams and WebVTT
 WebVTT allows metadata tracks and raw cues
1
2
3
4
<video controls width="640" height="480">
<source src="file.mp4" type="video/mp4" >
<track src="annotations.vtt" kind="metadata">
</video>
SVG
WebVTT
Flash
WebVTT
SVG
MP4Box
MP4Box –webvtt-raw 1 file_with_svg.mp4
MP4Box –webvtt-raw 1:embedded file_with_svg.mp4
19
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

20. SVG embedded in WebVTT
WEBVTT Metadata track generated by GPAC MP4Box
kind: metadata
inBandMetadataTrackDispatchType: image/svg+xml
metadata-header:
<svg viewBox="0 0 384 288" ...>
<rect width="100%" height="100%" fill="lightblue"/>
00:00:00.000 --> 00:00:00.083 isRAP:true
<g display="none">
<animate ... />
<defs>...</defs>
<use .../>
</g>
00:00:00.083 --> 00:00:00.025 isRAP:false
<g>...</g>
20
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

21. SVG referenced in WebVTT
WEBVTT Metadata track generated by GPAC MP4Box
kind: metadata
inBandMetadataTrackDispatchType: image/svg+xml
baseMediaFile: flash7.svg
text-header-length: 207
00:00:00.000 --> 00:00:00.083 mediaOffset:207 dataLength:311 isRAP:true
00:00:00.083 --> 00:00:00.166 mediaOffset:518 dataLength:320 isRAP:false
00:00:00.166 --> 00:00:00.250 mediaOffset:838 dataLength:207 isRAP:false
...
21
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

22. Summary & Conclusion
 Presented
•
•
Guidelines for authoring streamable SVG content
Tools for generating streaming instructions
─ from Flash, will be extended to support SVG natively
•
JavaScript-based players for streamable SVG content
 Outcome
•
Provided that your SVG content is well structured and annotated,
you can stream SVG content
─ In different modes: Controlled Progressive Download, Live, On Demand,
Playlists, Adaptive, …
─ Out of band graphics tracks (currently with JS shim)
─ In band tracks in media containers (in browsers in the future)
 Future work
•
•
•
22
23/10/2013
Implement adaptive streaming (MP4 based)
Implement media synchronization (WebVTT)
Work with CSS-based or JS-based animations
Cyril Concolato
Streaming SVG animations on the Web

23. Thank you
Questions?
23
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web

24. Further reading
 SVG Related
• HTML 5 Video and SVG Graphics Synchronization
• Carriage of SVG and WebVTT in MP4 (Blog, Text
Track CG)
 Streaming related
• Live streaming over HTTP of video and subtitles (Part
1 & Part 2)
• WebVTT streaming
24
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web