1.
Real Time SHVC Decoder: Implementation and
Complexity Analysis
Wassim Hamidouche
Research Engineer
E-mail: Wassim.Hamidouche@insa-rennes.fr
Phone : +33 6 19 14 16 38
October 2014

2.
Outline
Introduction
Real time SHVC decoder
Results and analysis
Conclusion

3.
Outline
Introduction
Real time SHVC decoder
Results and analysis
Conclusion

4.
Introduction (1/3)
Scalable High eciency Video Coding (SHVC) standard:
Enable higher temporal, spatial, quality, bit
depth resolutions and wider color gamut.
Use all the powerful of the HEVC standard:
quadtree-based block partitioning, large
transform and prediction blocks, accurate
intra/inter predictions and the in-loop

5.
lters.

6.
Introduction (2/3)
Block diagram of the SHVC encoder with two spatial scalability
layers:
4K
Downsampling
Upsampling
scaling MV
Entropy coding
EL bitstream
BL bitstream
Original picture SHVC EL encoder
HEVC BL encoder
Loop filter T /Q
T/Q
MC/Inter-layer
prediction
Picture
buffer
Intra
prediction
Loop filter T /Q
T/Q Entropy coding
MC/Inter-layer
prediction
Picture
buffer
Intra
prediction
-1 -1
-1 -1
SHVC bitstream
HD

7.
Introduction (3/3)
Scalable High eciency Video Coding (SHVC) standard:
I Uses Inter-layer predictions to improve the coding eciency
(15%-30% gain vs independent coding) [1]
I Legacy compliant: supports a base layer compliant with the
AVC standard
I Multi-loop based coding structure: all intermediate layers
need to be decoded
Access the complexity of a real time SHVC decoder in respect to a
simulcast HEVC decoder
[1] V. Seregin, T. D. Chuang, Y. He, D. K. Kwon, and F. Le Leannec, AHG
Report: SHVC software, in document JCTVC-L0011. Geneva, Switzerland,
January 2013.

8.
Outline
Introduction
Real time SHVC decoder
Results and analysis
Conclusion

9.
Real time SHVC decoder (1/6)
SHVC decoder:
I Based on the OpenHEVC decoder [2]
I Open source implementation of the HEVC decoder developed
in C language on the top of FFmpeg library
I Supports the decoding of all HEVC pro

10.
les and the HEVC
conformance bitstreams
I The main decoding operations are heavily optimized in SSE
instructions for x86 architecture
I Enables all parallel processing solutions adopted in the HEVC
standard: tile, slice, wavefront and frame-based
[2] Open source HEVC decoder (OpenHEVC) in
https:://github.com/OpenHEVC

11.
Real time SHVC decoder (2/6)
Architecture of the OpenHEVC software:
hls_coding_tree
Decode a slice
hls_sao_filter_ctb
Yes
hls_deblocking_filter_ctb
Is a CTU ?
hls_coding_unit
hls_prediction_unit
hls_transform_tree
Yes
is a TTU ?
hls_transform_unit
No
Is SAO filter
enabled ?
No
Yes
No
No
Slice decoded
Is the CTU
decoded ?
Yes
hls_decode_slice
Yes
No
More CTU
in the slice?
Yes
Is deblocking
filter enabled ?

12.
Real time SHVC decoder (3/6)
Extend the OpenHEVC decoder to decode SHVC enhancement
layers (EL):
I Parse multi-layers syntax elements at the enhancement layers
I New functions to perform the upsampling of the inter-layer
reference picture and the upscaling its MVs (8-tap

13.
lter for
luma and 4-tap

14.
lter for chroma)
I Manage reference lists by including new inter-layer reference
frames
Low level optimizations in the SHVC decoder:
I Optimize the up-sampling

15.
lters in Single Instruction Multiple
Data (SIMD) operations (SSE instructions for x86 processor)

16.
Real time SHVC decoder (4/6)
Multiple instances of the modi

17.
ed OpenHEVC decoder, one
instance to decode each layer:
Demultiplexer
SHVC
bitstream
BL bitstream
OpenHEVC
decoder
OpenHEVC
decoder
EL bitstream
Access to the decoded BL
its MVs
SHVC decoder

18.
Real time SHVC decoder (5/6)
Parallelism in the SHVC decoder :
I SHVC decoder supports three levels of parallelism:
I Wavefront parallelism: the CTB rows of each layer are
decoded in parallel for low latency applications
I Temporal frame-based parallelism: successive temporal frame
are decoded in parallel
I Layers parallelism: frames of the SHVC layers are decoded in
parallel
I Communication control mechanism is implemented to manage
wavefront dependencies as well as inter and inter-layer
prediction dependencies

19.
Real time SHVC decoder (6/6)
Hybrid parallel architecture for the SHVC decoder with two layers:
B frame
Inter prediction
B frame
P frame P frame
No decoded CTBs
B frame B frame
Inter prediction
B frame
Decoded CTBs
CTB in decoding
Wait for the
decoding of the
referenced PU
{
{
Thread 2
Thread 3
Inter layer
prediction
I frame I frame
Base layer Enhancement layer
B frame
{{
Thread 0
Thread 1
Thread 5
Thread 6
{ {
{ {
Thread 4
Thread 5

20.
Outline
Introduction
Real time SHVC decoder
Results and analysis
Conclusion

21.
Results and analysis (1/5)
Experimental con

22.
guration:
I 4 Cores Intel i7 processor running at 2.8 GHz
I Common SHVC test conditions: video sequences from class A
and B and all QP, 3 scalability con

23.
gurations 2x, 1.5x and
SNR
I Reference Scalable Software Model (SHM) 4.1 is used to
encode the test video sequences
I Three con

24.
gurations of parallelism
(n;m) = f(4; 1); (1; 4); (2; 2)g with n: number of threads for
wavefront parallelism and m the number of frames decoded in
parallel

25.
Results and analysis (2/5)
Decoding time performance of the SHVC decoder (mono core):
Sequences
Decoding time (second)
HEVC SHVC
SSE no SSE SNR 2x 1.5x
Kimono 3.53 12.35 5.78 5.06 5.86
ParkScene 4.03 12.08 7.24 5.86 6.92
Cactus 6.52 18.04 12.29 9.51 12.41
BasketBallDrive 8.02 26.56 14.74 11.26 14.27
BQTerrace 9.60 29.53 18.02 12.92 16.56
Trac 4.14 11.27 7.83 6.22 -
PeopleonStreet 6.70 15.90 10.42 9.73 -
Complexity (%) 0 195 80 43 77

26.
Results and analysis (2/5)
Decoding time performance of the SHVC decoder (mono core):
Sequences
Decoding time (second)
HEVC SHVC
SSE no SSE SNR 2x 1.5x
Kimono 3.53 12.35 5.78 5.06 5.86
ParkScene 4.03 12.08 7.24 5.86 6.92
Cactus 6.52 18.04 12.29 9.51 12.41
BasketBallDrive 8.02 26.56 14.74 11.26 14.27
BQTerrace 9.60 29.53 18.02 12.92 16.56
Trac 4.14 11.27 7.83 6.22 -
PeopleonStreet 6.70 15.90 10.42 9.73 -
Complexity (%) 0 195 80 43 77

27.
Results and analysis (3/5)
Time repartition in the SHVC decoder (mono core):
Transfor
m, 2.98
Motion
compensa
tion, 33.75
Inter-layer
prediction
, 28.75
Entropy
decoding,
10.8
In-loop
filters,
16.6
Intra
prediction
, 0.98
Rest, 6.15
(a) 2x (15.1 seconds)
Transfor
m, 3.35
Motion
compensa
tion, 32.45
Inter-layer
prediction
, 29.55
Entropy
decoding,
11.65
In-loop
filters,
15.75
Intra
prediction
, 1.08
Rest, 6.18
(b) 1.5x (16.5 seconds)
Transfor
m, 4
Motion
compensat
ion, 45.55
Inter-layer
prediction
, 7.95
Entropy
decoding,
11.05
In-loop
filters,
22.2
Intra
prediction
, 1.45
Rest, 7.8
(c) SNR (16.3 seconds)
Transfor
m, 4.98
Motion
compensat
ion, 45.38
, 0
In-loop
filters,
22.58
Entropy
decoding,
15.93
Intra
prediction
, 3.45
Rest, 7.7
(d) HEVC (8.5 seconds)

28.
Results and analysis (4/5)
Parallelism performance in the SHVC decoder:
Con

29.
gurations Decoding con

30.
gurations
(1, 1) (4, 1) (1, 4) (2, 2)
Speedup
2 1 2.99 2.02 3.24
1:5 1 3.13 2.21 3.36
SNR 1 2.90 2.50 3.34
HEVC 1 3.10 2.64 3.05
Decoding 2 52 156 107 169
Class B (1080p)
frame rate 1:5 44 138 99 149
(fps) SNR 41 124 108 138
HEVC 72 226 195 224
Decoding 2 20.54 6.88 24.04 16.08
time per 1:5 24.11 7.76 27.82 19.27
frame (ms) SNR 29.47 10.28 33.36 24.81
HEVC 16.07 5.20 24.05 10.66

31.
Results and analysis (5/5)
Parallelism performance in the SHVC decoder:
Con

32.
gurations Decoding con

33.
gurations
(1, 1) (4, 1) (1, 4) (2, 2)
Speedup
2 1 3.6 2.07 3.81
SNR 1 3.49 2.47 3.67
Class A (1600p)
HEVC 1 3.49 2.83 3.22
Decoding 2 23 84 47 88
frame rate SNR 17 62 43 65
(fps) HEVC 33 120 96 110
Decoding 2 47.85 13.65 55 32.65
time per SNR 63.92 18.76 73.67 50.95
frame (ms) HEVC 34.56 10.25 48.65 21.90

34.
Outline
Introduction
Real time SHVC decoder
Results and analysis
Conclusion

35.
Conclusion and Perspectives (1/2)
Conclusion:
I First real time and parallel software (open source)
implementation of the SHVC decoder
I The SHVC decoder decoding two layers introduces 43% and
77% additional complexity in 2x and 1.5x spatial scalability
con

36.
gurations
I Low level optimizations and parallelism are required to reach a
real time decoding of 4Kp60 enhancement layer

37.
Conclusion and Perspectives (2/2)
Perspectives:
I Support the decoding of more than two layers
I Support the decoding of the base layer in legacy AVC
standard
I Uni

38.
ed software for all HEVC extensions: Design a software
decoder that support the decoding of all HEVC extensions
including MV-HEVC, RExt, screen content coding, and
3D-HEVC extensions

39.
Thank you for your attention