What is HTML5 Video and how is it different from what we are used to? What problems does it solve, and what issues does it have? This is the web, so it's open source right? In this presentation, we will address these questions and provide some basic terminology for understanding how video works. We'll show how to embed HTML5 Video API and explore the API. Next we'll discuss what browser it doesn't work on, and the capabilities of the ones that it does.
HTML5 Video unfortunately does not mean one video codec. After showing what a codec is, we'll show which browsers and devices support which codecs. Finally, we'll go over video encoding possibilities.
4. Online Video Beginnings
In the 90s it was just cool if video played on the
computer, much less online
5. Online Video Beginnings
In the 90s it was just cool if video played on the
computer, much less online
As of 2000, online video was a reality, but it was a
mess
6. Online Video Beginnings
In the 90s it was just cool if video played on the
computer, much less online
As of 2000, online video was a reality, but it was a
mess
RealPlayer was the best at online video,
but was the quintessential nag-ware and spy-ware
7. Online Video Beginnings
In the 90s it was just cool if video played on the
computer, much less online
As of 2000, online video was a reality, but it was a
mess
RealPlayer was the best at online video,
but was the quintessential nag-ware and spy-ware
Quicktime was better, was crippled by Windows
versioning and backwards compatibility
8. Online Video Beginnings
In the 90s it was just cool if video played on the
computer, much less online
As of 2000, online video was a reality, but it was a
mess
RealPlayer was the best at online video,
but was the quintessential nag-ware and spy-ware
Quicktime was better, was crippled by Windows
versioning and backwards compatibility
Windows Media was still playing catch up
10. Flash Takes Over
In 2002, Macromedia releases Flash video using
Sorenson Spark
11. Flash Takes Over
In 2002, Macromedia releases Flash video using
Sorenson Spark
In 2003 Macromedia releases the FLV format using
VP6
12. Flash Takes Over
In 2002, Macromedia releases Flash video using
Sorenson Spark
In 2003 Macromedia releases the FLV format using
VP6
YouTube, making exclusive use of the Flash FLV format,
is launched in 2005
13. Flash Takes Over
In 2002, Macromedia releases Flash video using
Sorenson Spark
In 2003 Macromedia releases the FLV format using
VP6
YouTube, making exclusive use of the Flash FLV format,
is launched in 2005
In 2006, Adobe added the
option to use the H.264 codec
17. Problems with Flash
Plugins not always an elegant solution
As the Apple vs Adobe battle has exposed
18. Problems with Flash
Plugins not always an elegant solution
As the Apple vs Adobe battle has exposed
Reliance upon proprietary technology
Video is too important a component of the web to
not standardize
Need for at least some knowledge of ActionScript
20. Non-Flash Video Problems
OS and browser combinations create different video
problems
Example: WMV on Firefox on a Mac
21. Non-Flash Video Problems
OS and browser combinations create different video
problems
Example: WMV on Firefox on a Mac
Problems often solved with JavaScript
Uses very inaccurate browser sniffing
22. Non-Flash Video Problems
OS and browser combinations create different video
problems
Example: WMV on Firefox on a Mac
Problems often solved with JavaScript
Uses very inaccurate browser sniffing
Different video formats often needed
WMV, MOV, maybe REAL, etc.
29. HTML5 Video Benefits
First Class citizen
Native to browser / no plugins
One tag handles multiple video types
30. HTML5 Video Benefits
First Class citizen
Native to browser / no plugins
One tag handles multiple video types
No JavaScript nor ActionScript required
33. HTML5 Video Problems
Not supported in IE8 (will be in IE9)
Native controls not consistent amongst browsers
34. HTML5 Video Problems
Not supported in IE8 (will be in IE9)
Native controls not consistent amongst browsers
Fullscreen capability not standard and still under
consideration
35. HTML5 Video Problems
Not supported in IE8 (will be in IE9)
Native controls not consistent amongst browsers
Fullscreen capability not standard and still under
consideration
Need full HTML fullscreen ability, not just video
36. HTML5 Video Problems
Not supported in IE8 (will be in IE9)
Native controls not consistent amongst browsers
Fullscreen capability not standard and still under
consideration
Need full HTML fullscreen ability, not just video
YouTube devs weighing in on this importance
37. HTML5 Video Problems
Not supported in IE8 (will be in IE9)
Native controls not consistent amongst browsers
Fullscreen capability not standard and still under
consideration
Need full HTML fullscreen ability, not just video
YouTube devs weighing in on this importance
41. HTML5 Video Problems (2)
Poor meta info
No DRM, No streaming
No SMIL nor playlist capability
42. HTML5 Video Problems (2)
Poor meta info
No DRM, No streaming
No SMIL nor playlist capability
Early implementation has many cross browser/device
bugs
43. HTML5 Video Problems (2)
Poor meta info
No DRM, No streaming
No SMIL nor playlist capability
Early implementation has many cross browser/device
bugs
Lack of agreement for a common file type
51. API Bugs
iPad bug when dynamically created
iPad poster bug
52. API Bugs
iPad bug when dynamically created
iPad poster bug
iPad source order bug
53. API Bugs
iPad bug when dynamically created
iPad poster bug
iPad source order bug
WebKit error event must be on source objects
54. API Bugs
iPad bug when dynamically created
iPad poster bug
iPad source order bug
WebKit error event must be on source objects
progress event not standardized
55. API Bugs
iPad bug when dynamically created
iPad poster bug
iPad source order bug
WebKit error event must be on source objects
progress event not standardized
Firefox very buggy when changing media
56. API Bugs
iPad bug when dynamically created
iPad poster bug
iPad source order bug
WebKit error event must be on source objects
progress event not standardized
Firefox very buggy when changing media
loadedmetadata fires when meta is is partially
loaded
57. API Bugs
iPad bug when dynamically created
iPad poster bug
iPad source order bug
WebKit error event must be on source objects
progress event not standardized
Firefox very buggy when changing media
loadedmetadata fires when meta is is partially
loaded
Even more bugs are listed here:
http://camendesign.com/code/video_for_everybody
61. File Format
File Format = Container = Wrapper
MP4, WebM and OGV are wrappers
62. File Format
File Format = Container = Wrapper
MP4, WebM and OGV are wrappers
The wrapper is the essentially the meta data file
format that describes how the data is stored and
gives information to your computer so that it can load
the necessary libraries in which to display it.
63. File Format
File Format = Container = Wrapper
MP4, WebM and OGV are wrappers
The wrapper is the essentially the meta data file
format that describes how the data is stored and
gives information to your computer so that it can load
the necessary libraries in which to display it.
A wrapper generally contains a video and an audio
codec
64. File Format
File Format = Container = Wrapper
MP4, WebM and OGV are wrappers
The wrapper is the essentially the meta data file
format that describes how the data is stored and
gives information to your computer so that it can load
the necessary libraries in which to display it.
A wrapper generally contains a video and an audio
codec
Has instructions on how they should play and sync
67. Codec
MP4 uses H264 video AAC audio
WebM uses VP8 video Vorbis audio
68. Codec
MP4 uses H264 video AAC audio
WebM uses VP8 video Vorbis audio
OGG uses Theora video and Vorbis audio
69. Codec
MP4 uses H264 video AAC audio
WebM uses VP8 video Vorbis audio
OGG uses Theora video and Vorbis audio
A codec is the code that handles the decoding of the
images, audio or other data.
70. Codec
MP4 uses H264 video AAC audio
WebM uses VP8 video Vorbis audio
OGG uses Theora video and Vorbis audio
A codec is the code that handles the decoding of the
images, audio or other data.
It usually but not always includes the process for
compressing the encoded data.
75. Implemented by Browser
MP4 OGG WebM
IE9
FireFox 4.0
Chrome 6
Safari 5
Opera 10.6
Safari on the Mac or IE9 on Windows will be able to supported any type if that codec
is installed on the OS. Other browsers (Firefox, Opera, Chrome) will need to specifically
implement the video codec, so their support will be limited to few formats.
78. Implemented by Device*
iPhone
iPad
Android
All handle H264 video codecs
* Not a complete list of course.
79. Implemented by Device*
iPhone
iPad
Android
All handle H264 video codecs
Android handles only MP4 file formats
* Not a complete list of course.
80. Implemented by Device*
iPhone
iPad
Android
All handle H264 video codecs
Android handles only MP4 file formats
iPhone handles MP4, but M4V is a
native Quicktime extension and allows
for more variance in the encodings
* Not a complete list of course.
81. Implemented by Device*
iPhone
iPad
Android
All handle H264 video codecs
Android handles only MP4 file formats
iPhone handles MP4, but M4V is a
native Quicktime extension and allows
for more variance in the encodings
Most other mobile phones use 3GP
* Not a complete list of course.
82. Implemented by Device*
iPhone
iPad
Android
All handle H264 video codecs
Android handles only MP4 file formats
iPhone handles MP4, but M4V is a
native Quicktime extension and allows
for more variance in the encodings
Most other mobile phones use 3GP
* Not a complete list of course.
83. Implemented by Device*
iPhone
iPad
Android
Mobile Phones usually have the
video codec implemented in the
hardware. So iPhone, iPad, Android
phones all will only ever play the
codec that they came with. Namely
H264.
* Not a complete list of course.
85. WebM
For full list, go to the WebM Project Tools Page:
http://www.webmproject.org/tools/
86. WebM
FFMPEG (command line)
http://www.ffmpeg.org/
DirectShow Filters (plugin)
http://www.webmproject.org/tools/
FireFogg (browser based)
http://www.firefogg.org/ Most awesome logo
ever!
For full list, go to the WebM Project Tools Page:
http://www.webmproject.org/tools/
94. VBR/CBR
Constant Bitrate (CBR)
The rate at which a codec's output data should be
consumed is constant
95. VBR/CBR
Constant Bitrate (CBR)
The rate at which a codec's output data should be
consumed is constant
Useful for streaming
96. VBR/CBR
Constant Bitrate (CBR)
The rate at which a codec's output data should be
consumed is constant
Useful for streaming
Variable Bitrate (VBR)
97. VBR/CBR
Constant Bitrate (CBR)
The rate at which a codec's output data should be
consumed is constant
Useful for streaming
Variable Bitrate (VBR)
Allows a higher bitrate to be allocated to the more
complex segments of media files while less space is
allocated to less complex segments.
98. VBR/CBR
Constant Bitrate (CBR)
The rate at which a codec's output data should be
consumed is constant
Useful for streaming
Variable Bitrate (VBR)
Allows a higher bitrate to be allocated to the more
complex segments of media files while less space is
allocated to less complex segments.
Useful for progressively downloaded videos
105. Macroblock
An image compression component and technique similar
to JPEG
The size of a block depends on the codec and is
usually a multiple of 4.
106. Macroblock
An image compression component and technique similar
to JPEG
The size of a block depends on the codec and is
usually a multiple of 4.
In MPEG2 and other early codecs the size is fixed at blocks of 8x8 pixels,
which is where the convention comes from of ecoding with sizes of a multiple
of 8: 320, 640, 720, etc.
107. Macroblock
An image compression component and technique similar
to JPEG
The size of a block depends on the codec and is
usually a multiple of 4.
In MPEG2 and other early codecs the size is fixed at blocks of 8x8 pixels,
which is where the convention comes from of ecoding with sizes of a multiple
of 8: 320, 640, 720, etc.
108. Macroblock
An image compression component and technique similar
to JPEG
The size of a block depends on the codec and is
usually a multiple of 4.
In MPEG2 and other early codecs the size is fixed at blocks of 8x8 pixels,
which is where the convention comes from of ecoding with sizes of a multiple
of 8: 320, 640, 720, etc.
111. Square Pixels
Square Pixels / Pixel Aspect Ratio
An unfortunate artifact with continued confusion from
the early video conversion software days. Essentially it
explains 720x480 using non-square pixels and
640x480 using square pixels.
112. Square Pixels
Square Pixels / Pixel Aspect Ratio
An unfortunate artifact with continued confusion from
the early video conversion software days. Essentially it
explains 720x480 using non-square pixels and
640x480 using square pixels.
If the incorrect conversion is used, the image will be
stretched.
114. Frame Types
Intra-coded: I‑frames are the least compressible but
don't require other video frames to decode.
115. Frame Types
Intra-coded: I‑frames are the least compressible but
don't require other video frames to decode.
Predicted: P‑frames can use data from previous frames
to decompress and are more compressible than
I‑frames.
116. Frame Types
Intra-coded: I‑frames are the least compressible but
don't require other video frames to decode.
Predicted: P‑frames can use data from previous frames
to decompress and are more compressible than
I‑frames.
Bi-Directional Predicted: B‑frames can use both
previous and forward frames for data reference to get
the highest amount of data compression.
117. Frame Types
Intra-coded: I‑frames are the least compressible but
don't require other video frames to decode.
Predicted: P‑frames can use data from previous frames
to decompress and are more compressible than
I‑frames.
Bi-Directional Predicted: B‑frames can use both
previous and forward frames for data reference to get
the highest amount of data compression.
120. Level
Used for MP4 encoding (16 levels, 1 through 5.1)
An indicator for a decompressor's capability
121. Level
Used for MP4 encoding (16 levels, 1 through 5.1)
An indicator for a decompressor's capability
Essentially a shortcut to constrain different video
components when compressing
Macroblocks
Bitrate
Resolution
Frame rate
Key frames
125. Profile
Used for MP4 encoding
Defined sets of capabilities
Larger profiles often contain the lower profiles
126. Profile
Used for MP4 encoding
Defined sets of capabilities
Larger profiles often contain the lower profiles
Common Profiles:
127. Profile
Used for MP4 encoding
Defined sets of capabilities
Larger profiles often contain the lower profiles
Common Profiles:
Baseline: Used for web, videoconferencing and mobile applications.
128. Profile
Used for MP4 encoding
Defined sets of capabilities
Larger profiles often contain the lower profiles
Common Profiles:
Baseline: Used for web, videoconferencing and mobile applications.
Main: Used for standard-definition digital TV broadcasts, or high resolution
web broadcast
129. Profile
Used for MP4 encoding
Defined sets of capabilities
Larger profiles often contain the lower profiles
Common Profiles:
Baseline: Used for web, videoconferencing and mobile applications.
Main: Used for standard-definition digital TV broadcasts, or high resolution
web broadcast
High: Used for broadcast and disc storage applications, particularly Blu-ray
132. OGG / Theora
Open source, patent free, royalty free, under the BSD
License
133. OGG / Theora
Open source, patent free, royalty free, under the BSD
License
Free, as in beer.
134. OGG / Theora
Open source, patent free, royalty free, under the BSD
License
Free, as in beer.
In 2002, On2 TrueMotion donated their VP3 codec to
the Xiph.Org Foundation
135. OGG / Theora
Open source, patent free, royalty free, under the BSD
License
Free, as in beer.
In 2002, On2 TrueMotion donated their VP3 codec to
the Xiph.Org Foundation
Theora was forked off from the released codebase of
VP3.2
136. OGG / Theora
Open source, patent free, royalty free, under the BSD
License
Free, as in beer.
In 2002, On2 TrueMotion donated their VP3 codec to
the Xiph.Org Foundation
Theora was forked off from the released codebase of
VP3.2
Theora 1.0 was released in November 2008
137. OGG / Theora
Open source, patent free, royalty free, under the BSD
License
Free, as in beer.
In 2002, On2 TrueMotion donated their VP3 codec to
the Xiph.Org Foundation
Theora was forked off from the released codebase of
VP3.2
Theora 1.0 was released in November 2008
No hardware support
139. WebM / VP8
Open source, royalty free, under the BSD License
140. WebM / VP8
Open source, royalty free, under the BSD License
Free, as in beer (maybe).
141. WebM / VP8
Open source, royalty free, under the BSD License
Free, as in beer (maybe).
Some hardware support (AMD, ARM), with players
such as Intel and NVIDIA considering it
142. WebM / VP8
Open source, royalty free, under the BSD License
Free, as in beer (maybe).
Some hardware support (AMD, ARM), with players
such as Intel and NVIDIA considering it
Patent-free legitimacy is widely disputed
143. WebM / VP8
Open source, royalty free, under the BSD License
Free, as in beer (maybe).
Some hardware support (AMD, ARM), with players
such as Intel and NVIDIA considering it
Patent-free legitimacy is widely disputed
MPEG LA threatening a patent pool to collect royalties
144. FLV / Flash Video
FLV is not an HTML5 Video codec. It is listed here for comparison.
145. FLV / Flash Video
All are patent-restricted (ergo, not free as in beer)
FLV is not an HTML5 Video codec. It is listed here for comparison.
146. FLV / Flash Video
All are patent-restricted (ergo, not free as in beer)
Sorenson H263 (FLV)
Originally a Quicktime codec
FLV is not an HTML5 Video codec. It is listed here for comparison.
147. FLV / Flash Video
All are patent-restricted (ergo, not free as in beer)
Sorenson H263 (FLV)
Originally a Quicktime codec
VP6 (FLV)
Also used in JavaFX
FLV is not an HTML5 Video codec. It is listed here for comparison.
148. FLV / Flash Video
All are patent-restricted (ergo, not free as in beer)
Sorenson H263 (FLV)
Originally a Quicktime codec
VP6 (FLV)
Also used in JavaFX
H264 (F4V)
Same as MP4 and M4V
FLV is not an HTML5 Video codec. It is listed here for comparison.
149. FLV / Flash Video
All are patent-restricted (ergo, not free as in beer)
Sorenson H263 (FLV)
Originally a Quicktime codec
VP6 (FLV)
Also used in JavaFX
H264 (F4V)
Same as MP4 and M4V
Adobe favoring support for F4V over FLV
FLV is not an HTML5 Video codec. It is listed here for comparison.
150. FLV / Flash Video
All are patent-restricted (ergo, not free as in beer)
Sorenson H263 (FLV)
Originally a Quicktime codec
VP6 (FLV)
Also used in JavaFX
H264 (F4V)
Same as MP4 and M4V
Adobe favoring support for F4V over FLV
WebM
Adobe announced they will soon support WebM
FLV is not an HTML5 Video codec. It is listed here for comparison.
152. MP4 / H264
Free as in... you’ll pay later.
Actually, free for non-paid web use until 2015
153. MP4 / H264
Free as in... you’ll pay later.
Actually, free for non-paid web use until 2015
Enjoys broad adoption of hardware support
154. MP4 / H264
Free as in... you’ll pay later.
Actually, free for non-paid web use until 2015
Enjoys broad adoption of hardware support
Heavily patent encumbered, mercilessly enforced by
the MPEG LA Group
Not affiliated with the MPEG standardization organization
155. MP4 / H264
Free as in... you’ll pay later.
Actually, free for non-paid web use until 2015
Enjoys broad adoption of hardware support
Heavily patent encumbered, mercilessly enforced by
the MPEG LA Group
Not affiliated with the MPEG standardization organization
The last patents for H.264 may not expire until 2028
156. MP4 / H264
Free as in... you’ll pay later.
Actually, free for non-paid web use until 2015
Enjoys broad adoption of hardware support
Heavily patent encumbered, mercilessly enforced by
the MPEG LA Group
Not affiliated with the MPEG standardization organization
The last patents for H.264 may not expire until 2028
MP4 directly based on Apple’s Quicktime MOV
158. MPEG LA Group
group
noun [treated as sing. or pl. ]
Something people do together: We shared a group hug
Something organizations do together: The G8 makes unilateral
decisions for the rest of the world.
160. H264 Patent Holders
Apple Inc., DAEWOO Electronics Corporation, Dolby Laboratories Licensing
Corporation, Electronics and Telecommunications Research Institute, France
Télécom, société anonyme, Fraunhofer-Gesellschaft zur Foerderung der
angewandten Forschung e.V., Fujitsu Limited, Hitachi, Ltd., Koninklijke Philips
Electronics N.V., LG Electronics Inc., Microsoft Corporation, Mitsubishi Electric
Corporation, NTT DOCOMO, INC., Nippon Telegraph and Telephone Corporation,
Panasonic Corporation, Robert Bosch GmbH, Samsung Electronics Co., Ltd.,
Scientific-Atlanta Vancouver Company, Sedna Patent Services, LLC, Sharp
Corporation, Siemens AG, Sony Corporation, Telefonaktiebolaget LM Ericsson, The
Trustees of Columbia University in the City of New York, Toshiba Corporation,
Victor Company of Japan, Limited
162. Apple
Good Guy:
Thanks to their power wielded over Adobe and
their influence afforded to them by the iPhone and
iPad, Apple was able to push the HTML5 Video
standard and its adoption forward
163. Apple
Good Guy:
Thanks to their power wielded over Adobe and
their influence afforded to them by the iPhone and
iPad, Apple was able to push the HTML5 Video
standard and its adoption forward
Bad Guy:
Apple, holder of many H264 patents, is baiting the
entire web into accepting H264 as the standard. The
5 year hiatus on fees could mean larger fees and
back payments in 2015
165. Codec Recap
OGG Theora
Not widely accepted, no hardware support, free
WebM
Free, acceptance growing very quickly including
Flash, hardware support building, patent lawsuits
looming, not supported on iPhone nor iPad, free
and back by Google/YouTube
MP4
Very well developed industry standard, smallest
files and clearest images, broad hardware
support, fees looming, don’t count out Google
167. HTML5 Video Recap
OGG
Good for a few files, small sites, compatibility not
a problem, open source maniacs
WebM
Good for sites with many files, compatibility not
tied financial bottom line, viewable with Flash
soon, not worried about iApple
MP4
Best for overall compatibility and viewer
adoption, playable on all browsers and the newer
mobile platforms if you include a Flash player,
best solution if only one version of the video is
desired