An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance for Open Media.

An Introduction to AV1
The Next-Gen Royalty-Free Codec
From the Alliance for Open Media

Who are we? Martin Smole
Encoding Engineering Director, Bitmovin
@smole3
Nathan Egge
Senior Research Engineer, Mozilla
@nathanegge
Kieran Farr
VP Marketing
@kfarr

1100+ Employees Worldwide
Firefox Quantum
Open Standards
● W3C, IETF, WHATWG, EMCA, Kronos
Policy Advocacy
● Net Neutrality, Let’s Encrypt
Mozilla Research
● Rust, WASM, WebVR, Deep Speech, Codecs
Our mission is to ensure the
Internet is a global public
resource, open and accessible to
all. An Internet that truly puts
people first, where individuals
can shape their own experience
and are empowered, safe and
independent.

Webinar Agenda
● Introductions
● Who are you?
● What is the history of codecs that leads us to AV1?
● What’s the big deal with royalties?
● What is the state of AV1? Is it done yet?
● How does AV1 perform? Are there improvements?
● What does this mean for me? What are the next steps?
● Questions?

North America Internet Traffic

● Opus Audio Codec (started in 2007)
○ Originally designed for web use cases
■ Low latency, high quality music
○ Collaboration with Skype added speech
○ Published at IETF as RFC 6716
● Daala Video Codec (started in 2012)
○ RF by design
○ Revisit fundamental video coding choices
● NETVC Working Group (formed Spring 2015)
○ “Replicate the success of the CODEC WG in producing the Opus audio codec.“
○ Daala (Mozilla) and Thor (Cisco) candidate codecs
● Alliance for Open Media (formed Fall 2015)
○ Contribute Daala technologies to AV1
Mozilla and Royalty Free Codecs

Goals of the Alliance:
● Produce a video codec for a broad set of industry use cases
○ Video on Demand / Streaming
○ Video Conferencing
○ Screen sharing
○ Video game streaming
○ Broadcast
● Open Source and Royalty Free
● Widely supported and adopted
● At least 30% better than current generation video codecs
Alliance for Open Media (AOM)

AOM Members / Content Producers

AOM Members / Real-Time Conferencing

● New high-level syntax
○ Easily parsed sequence header, frame header, tile header, etc
● New adaptive multi-symbol entropy coding
○ Up to 16 possible values per symbol
● New coefficient coder
○ LV-MAP exploits multi-symbol arithmetic coder
● More block sizes
○ Prediction blocks from 128x128 down to 4x4
■ Rectangular blocks
● 1:2 and 2:1 ratios (4x8, 8x4, etc)
● 1:4 and 4:1 ratios (4x16, 16x4, etc)
○ Transform sizes from 64x64 down to 4x4
■ Includes rectangular transforms 1:2, 2:1 and 1:4, 4:1 ratios
● More transform types
○ 16 possible transform types
■ Row and column chosen from: IDTX, DCT, DST, ADST
● More references
○ Up to 7 per frame (out of a store of 8)
● Spatial and temporal scalability
● Lossless mode
● Chroma subsampling
○ 4:4:4, 4:2:2, 4:2:0, monochrome
AV1 Coding Tools Overview
● More prediction modes
○ Intra
■ 8 main directions plus delta for up to 56 directions
■ Smooth HV modes interpolate across block
■ Palette mode with index map up to 8 colors
■ Chroma from luma intra predictor
■ Intra Block Copy
○ Inter
■ Expanded reference list (up to 7 per frame)
■ Allow ZEROMV predictor, which isn’t always (0,0)
■ Compound mode
● Inter-Intra prediction
○ Depends on difference between pixel prediction
○ Smooth blending limited to certain intra modes
● Wedge codebook (Inter-Inter, or Inter-Intra)
■ Warped motion local affine model with neighbors
■ Global motion affine model across entire frame
● Loop filtering
○ Deblocking filter
○ Constrained Directional Enhancement Filter
○ Loop restoration
● Film grain synthesis
Full AV1 Specification: https://aomediacodec.github.io/av1-spec/

● Arithmetic Range Coder
● Code both binary symbols and multi-symbols
○ Alphabet sizes up to 16
● Improve EC throughput with high rate streams
○ Instead of 1 bit per cycle, decode up to 4
Multi-Symbol Entropy Coder
A DB
10 N = 0.4
C
A 0/1
0/1
B 0/1
C D
0 1
0 1
0 1

● Predict chroma channel based on decoded luma
○ Encoder signals best correlation constants: αcb and αcr
● Good for screen content or scenes with fast motion
Chroma from Luma Intra Prediction
αCb-αCb
αcr
-αcr

● Merge of Daala’s directional deringing filter (DERING) and Thor’s constrained lowpass filter (CLPF)
○ Both encoder and decoder search for the direction that best matches
○ Primary filter run along direction, and secondary conditional replacement filter run orthogonally
○ Strength is signaled in the bitstream
● Results exceed both DERING and CLPF alone, as well as applying DERING + CLPF sequentially
Constrained Directional Enhancement Filter (CDEF)

http://www.compression.ru/video/codec_comparison/hevc_2017/MSU_HEVC_comparison_2017_P5_HQ_encoders.pdf
Moscow State University (SSIM - June 2017)

HEVC Licensing Environment
http://blog.chiariglione.org/2018/01/ - Leonardo Chiariglione, Chairman and Co-Founder of Motion Picture Experts Group (MPEG)
● The MPEG business model is broken:
○ Create technology first, figure out
the licensing later
● Guarantees a patent-encumbered mess
● Subject to gaming
○ 30 years of MPEG to figure out how
○ Non-Practicing Entities (NPEs) have
become more aggressive over the years
● Vulnerable to patent holdup
○ Individuals want more than the technology is worth, because they know you have to pay
○ Many companies cannot afford to use the technology they helped create

Video Codec License Terms
https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#Patent_license_terms
Video
Format
Licensor Codec Royalties Codec Royalty
Exemptions
Codec Royalty
Annual Cap
Content
Distribution Fee
HEVC
MPEG LA US $0.20 / unit First 100k units / year US $25 million US $0
HEVC Advance Region 1:
- US $0.40 (mobile)
- US $0.80 (PC)
- US $1.20 (TV)
Region 2:
- US $0.20 (mobile)
- US $0.40 (PC)
- US $0.60 (TV)
US $25,000 / year US $40 million Content free to users:
- US $0
Paid by title model:
- US $0.025 / paid title
Paid subscriber model:
- US $0.005 / month / subscriber
- US $25M cap / year / business model
Technicolor Tailor-made agreements US $0
Velos Media ??? ??? ??? ???
others AT&T, Microsoft, Motorola,
Nokia, Cisco, … ??? ??? ??? ???
AVC
MPEG LA End Users + OEM (not PC OS):
- US $0.20: 100k+ units / year
- US $0.10: 5M+ units /year
Branded OEM for PC OS:
- US $0.20: 100k+ units / year
- US $0.10: 5M+ units / year
End Users + OEM (not PC OS):
- First 100k units / year
- First 100k units / year
End Users + OEM (not PC OS):
- US $9.75M
- US $9.75M
Free television:
- One time US $2,500 / encode
- Or US $2,500 - $10,000 annual fee
Internet broadcast: US $0
Paid subscriber model:
- US $0 / yr : 0k to 100k users
- US $25k / yr : 100k to 250k users
- US $50k / yr : 250k to 500k users
- US $75k / yr : 500k to 1M users
- US $100k / yr : 1M+ users
Maximum Annual Content Royalty
- US $8.125 million
others Nokia, Qualcomm, Broadcom,
Blackberry, TI, MIT ??? ??? ??? ???
AV1 Alliance for Open Media US $0 N/A N/A US $0

The Alliance for Open Media (AOM)
Goals of the Alliance
● Video codec for the web with
state of the art compression efficiency
● Open source & royalty-fee
● Widely supported and adopted
● Develop and deploy faster than industry standard

The State of AV1
● Currently 94 Experiments/Tools adopted
● Bitstream will freeze “soon”
● Software decoding workflows first
● YouTube will adopt AV1 soon after bitstream freeze
● Netflix will be an early adopter
● First Hardware will follow in 1 - 2 years

Bitmovin AV1 Milestones
● NAB 2017
○ Bitmovin presented world’s first AV1 live stream on a 200-cores cluster
○ Bitmovin receives best of NAB Award for the AV1 live stream
● IBC 2017
○ Bitmovin optimized AV1 codec to stream 1.5 Mbps 1080p@30fps live on 32-cores
● November 2017
○ Proof of concept - AV1 Bitmovin encoding + Firefox Nightly decoding
● NAB 2018
○ Release first commercial AV1 encoder

AV1 Relative Quality Vs VP9
Source: Multi-Codec DASH Dataset, An Evaluation of AV1, AVC, HEVC and VP9 (https://bitmovin.com/multi-codec-dash-dataset)

AV1 Relative Quality Vs HEVC
Source: Multi-Codec DASH Dataset, An Evaluation of AV1, AVC, HEVC and VP9 (https://bitmovin.com/multi-codec-dash-dataset)

Encoding Performance
● Reference encoder currently 2500 - 3000 times slower
than VP9
● After bitstream freeze optimizing the encoder will
become a priority
● Bitmovin Encoding Service can reach realtime and faster
encoding speeds today

Why is AV1 encoding so slow?
● Early days.
● Current target it to get the bitstream done not
performance.
● CPU is expensive, but bandwidth is even moreso!

Why is AV1 a net improvement for users?
● Increase video quality, especially in regions with
bandwidth constraints
● Enabling new technologies like 4K, 8K, HFR, HDR, etc.
● Reduces rebuffering events
● Decrease time to first frame

How do I get started?
● Meet us at NAB to learn more: bitmovin.com/nab
○ April 9 - April 10 in Las Vegas Booth SU9901 CM
○ Live demos all week
○ AOMedia Happy Hour Monday April 9
○ Mozilla “pop-up” AV1 booth Tuesday April 10
● Learn more on your own and try encoding & decoding for yourself:
○ AOMedia.org
○ hacks.mozilla.org/2017/11/d
○ aomedia.googlesource.com/aom/

● Multi-symbol Arithmetic Coder
○ Multi-symbol CDF adaptation
○ HW performance improvements
● Chroma-from-Luma Intra Prediction
● Constrained Directional Enhancement Filter (CDEF)
○ Merge of Daala’s directional deringing filter and Thor’s constrained low-pass filter
○ CDEF single-pass algorithm
● Implicit Frame Context Signalling
● Spatial Segmentation
● Daala Distortion Metric (encoder only)
AV1 Tools Contributed by Mozilla

objective-1-fast [1]:
[1] https://arewecompressedyet.com/?job=debargha-vp9-11152017-2%402018-01-13T20%3A09%3A28.783Z&job=av1_8bit_s1_0306
PSNR PSNR HVS SSIM CIEDE PSNR Cb PSNR Cr MS SSIM
Average -27.08 -26.10 -26.19 -28.73 -29.79 -31.06 -25.37
1080p -27.12 -26.73 -25.14 -28.82 -31.49 -31.35 -25.35
1080p-screen -31.17 -25.06 -31.57 -38.95 -42.95 -38.39 -27.96
720p -25.99 -25.22 -26.27 -25.63 -24.42 -30.92 -24.77
360p -25.99 -26.52 -24.81 -25.86 -24.71 -26.50 -24.51
AV1 vs VP9 (as of March 3rd, 2018)

An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance for Open Media.

Recommended

Recommended

More Related Content

Similar to An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance for Open Media.

Similar to An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance for Open Media. (20)

Recently uploaded

Recently uploaded (20)

An Introduction to AV1 - The Next-Gen Royalty-Free Codec From the Alliance for Open Media.

Editor's Notes