2. Context and Motivations
Animating using web technologies is getting popular
• Lots of content being produced
« SVG 2 » and « Web Animations 1.0 » being drafted
• Opportunities to improve the technologies
My goal: Enable treating web animations like video
• Why?
• What kind of animations?
• How?
2
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
3. Web Animation Use Cases
Animations in User Interfaces
Animations for Data Visualization
Timeline-based Animations
•
•
•
Graphics animations (cartoons, …)
Synchronized audio & graphics (karaoke, ads …)
Synchronized video & graphics (annotations, subtitles)
Popcorn.js
3
23/10/2013
SVG Wow!
Cyril Concolato
Cartoon
Streaming SVG animations on the Web
4. Timeline-based Animations Possibilities
Playback control using a clock
• Play(0)/Play(T)
• Pause/Resume
• Speed-up/slow down
Accurate synchronization
• Between animations or
between animation and media
• Dependent clocks and timestamps
Similar to video
but not allowed in HTML5 <video>
4
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
5. An SVG video-like JavaScript Player
Principles
Using HTML 5 <video> syntax but for SVG
content
Demo 1
1
2
3
<video controls width="640" height="480">
<source src="myAnimatedFile.svg" type="image/svg+xml" >
</video>
Demo 2
1
2
3
4
<video controls width="640" height="480">
<source src="file.mp4" type="video/mp4" >
<track src="annotations.svg" kind="graphics" type="image/svg+xml">
</video>
5
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
6. An SVG video-like JavaScript Player
Status
Simple Implementation
•
•
•
•
GUI based on Video.js
Replacing <video> by <svg> at run-time
Using XHR to fetch SVG data and insert in the page
Uses get/setCurrentTime() and un/pauseAnimations()
Limitations
•
•
•
•
6
23/10/2013
Bugs!
Same origin restrictions
Incremental/progressive loading not working
Cannot reproduce HTML network-related events (can
play, can play through)
Cyril Concolato
Streaming SVG animations on the Web
7. An SVG video-like JavaScript player:
Next Step
Network-aware Controlled Playback
•
Load SVG data just before it’s needed (and let garbage collection
do the rest)
How to achieve controlled playback of SVG ?
•
7
23/10/2013
Apply streaming concepts
Cyril Concolato
Streaming SVG animations on the Web
8. Applying streaming concepts to SVG
Streaming
“Controlled continuous delivery and consumption of data units”
What’s needed to stream SVG content?
•
Fragmentation of SVG documents into streaming units
─ Needs guidelines for SVG authoring
•
Assign timing to each unit
─ Mapping between byte-offset/time-position of the SVG document
─ Needs a streaming instruction format
•
Seek information
─ Needs explicit processing behavior
SVG Streaming draft
•
•
http://dev.w3.org/SVG/modules/streaming/
Feedback welcome
Example: Streaming SVG Cartoons
8
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
9. Cartoons on the Web – Today
Source: www.cartoonnetwork.com
Largely based on Vector Graphics
•
•
Bezier curves filled with solid color or simple gradients and/or stroke
Sometimes using raster images (e.g. background or texture)
Mostly frame-based content
•
•
•
Sometimes with additional interpolation
Typically synchronized with a sound track
Good candidate for streaming
Currently delivered as Flash or video streams
•
9
23/10/2013
Plugins, coding artefacts or bandwidth problems …
Cyril Concolato
Streaming SVG animations on the Web
10. Cartoons on the Web – Next Steps
Could be better integrated in the web platform
•
Using of vector graphics primitives for scalability and new
primitives for advanced rendering
─ E.g. coons patches, hatches (SVG 2), diffusion curves, …
•
Using other web technologies
─ Styling, language translation, …
Attempts to convert Flash to HTML5/CSS/SVG/Canvas
•
Many JS-based approaches
─
─
─
─
•
Google Swiffy
Mozilla Shumway
Adobe CreateJS
PixelPlant
Some declarative approaches
─ MP4Box (100% SVG)
10
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
12. Fragmentation of SVG document
into streaming units
Identify stream header
•
•
Static throughout the streaming session
Required to initialize the parser
Make sure SVG data is properly ordered
•
•
Use only backward references
Document order == time order
Identify units
•
•
Assign timestamps
No overlap in position / possible overlap in time (like WebVTT cues)
Make self-discardable units
•
<discard>, JavaScript, scoped stylesheets
Mark self-contained units
•
12
23/10/2013
Random Access Point
Cyril Concolato
Streaming SVG animations on the Web
14. How to generate Streaming Instructions ?
Post-authoring
• Requires byte positions to be final
Hard to determine without author’s instructions
• Where does the frame start/end?
• What’s the timestamp of the unit?
• Where are the random access points?
Possible automatic generation from Flash
Possible with additional elements/attributes
14
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
16. Generating Streaming Instructions
for SVG content converted from Flash
MP4Box -add flash.swf:fmt=svg file_with_svg_track.mp4
MP4Box -info file_with_svg_track.mp4
MP4Box -six 1 file_with_svg_track.mp4
16
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
17. SVG Streaming Demo
SVG
+
Streaming
Instructions
Demo 3
•
•
•
•
•
Web Server
(byte-range support)
Browser
+
JS Player
Playing an SVG stream
Viewing Random Access Points and others
Seeking into an SVG unit not yet downloaded
Concatenating SVG content (splicing)
Live and pseudo-live streaming of SVG content
─ Web-based Vector Graphics TV channel
17
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
18. SVG streams and Media Containers
MP4
Flash
MP4
SVG
MP4Box
Video
SVG
Audio
Why? What for?
•
Packaging audio/video/graphics together
─ To download only one resource
─ To facilitate synchronization of tracks
•
Possible use with HTML5 native <video> elements
─ SVG in-band graphical track
•
Possible use with HTML 5 Media Source extensions
─ Adaptive streaming of SVG
Formats
•
•
18
23/10/2013
ISO/IEC-14496:12 (royalty-free) defines how to store
timed text in MP4
Could be done for other formats (WebM, Ogg, …)
Cyril Concolato
Streaming SVG animations on the Web
19. SVG streams and WebVTT
WebVTT allows metadata tracks and raw cues
1
2
3
4
<video controls width="640" height="480">
<source src="file.mp4" type="video/mp4" >
<track src="annotations.vtt" kind="metadata">
</video>
SVG
WebVTT
Flash
WebVTT
SVG
MP4Box
MP4Box –webvtt-raw 1 file_with_svg.mp4
MP4Box –webvtt-raw 1:embedded file_with_svg.mp4
19
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
21. SVG referenced in WebVTT
WEBVTT Metadata track generated by GPAC MP4Box
kind: metadata
inBandMetadataTrackDispatchType: image/svg+xml
baseMediaFile: flash7.svg
text-header-length: 207
00:00:00.000 --> 00:00:00.083 mediaOffset:207 dataLength:311 isRAP:true
00:00:00.083 --> 00:00:00.166 mediaOffset:518 dataLength:320 isRAP:false
00:00:00.166 --> 00:00:00.250 mediaOffset:838 dataLength:207 isRAP:false
...
21
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web
22. Summary & Conclusion
Presented
•
•
Guidelines for authoring streamable SVG content
Tools for generating streaming instructions
─ from Flash, will be extended to support SVG natively
•
JavaScript-based players for streamable SVG content
Outcome
•
Provided that your SVG content is well structured and annotated,
you can stream SVG content
─ In different modes: Controlled Progressive Download, Live, On Demand,
Playlists, Adaptive, …
─ Out of band graphics tracks (currently with JS shim)
─ In band tracks in media containers (in browsers in the future)
Future work
•
•
•
22
23/10/2013
Implement adaptive streaming (MP4 based)
Implement media synchronization (WebVTT)
Work with CSS-based or JS-based animations
Cyril Concolato
Streaming SVG animations on the Web
24. Further reading
SVG Related
• HTML 5 Video and SVG Graphics Synchronization
• Carriage of SVG and WebVTT in MP4 (Blog, Text
Track CG)
Streaming related
• Live streaming over HTTP of video and subtitles (Part
1 & Part 2)
• WebVTT streaming
24
23/10/2013
Cyril Concolato
Streaming SVG animations on the Web