高立人資料壓縮

1. 1
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Lecture 09
Nov. 17, 2022
Instructor：高立人
電子工程研究所
國立臺北科技大學
1/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Scalar Quantization
November 17, 2022 2/43

2. 2
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Int. to Scalar Quantization
Q Q(x)
x
 Example:
1

x
x: a r.v. (a random variable),
 Case 1: uniform quantizer
 2-point quantizer
 









0
,
2
1
0
,
2
1
x
x
x
Q
-1 1
x
2
1
2
1

 
x
Q
-1 1
0
2
1
2
1

x
November 17, 2022 3/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Int. to Scalar Quantization
 Case 1: uniform quantizer (cont.)
 4-point quantizer
 

























2
1
1
,
4
3
0
2
1
,
4
1
2
1
0
,
4
1
1
2
1
,
4
3
x
x
x
x
x
Q
-1 1
0 2
1
2
1

x
-1 1
x
4
1
4
1

 
x
Q
4
3
4
3

November 17, 2022 4/43

3. 3
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
 Definition
 Quantization Error
 Defined as
 Q(x) can be regarded as the mean of x so that a smaller
quantization error can be obtained.
 Mean Square Error
 Defined as
 Also called MSE distortion.
 Can be regarded as the variance of quantization values.
 Since Q(x) is the mean of x.

Int. to Scalar Quantization
 
x
Q
x
e 

 
2
e
E
D 
November 17, 2022
 
 
2
x
x
E
D 

5/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
 Case 2: nonuniform quantizer (Nonuniform distribution)
 4-point quantizer
Int. to Scalar Quantization
-1 1
x
8
1
8
1

 
x
Q
2
1
4
1

2
1

4
1
 Question
 Given distribution, how to design Q(x)?
 Ans. Centroid condition and Nearest Neighbor condition.
-1 1
0
2
1
2
1

x
4
1

4
1
Smaller quantization bins here.
November 17, 2022 6/43

4. 4
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
 Case 3: Adaptive quantizer
Int. to Scalar Quantization
Q Q(x)
x(n)
n
Smaller step size Larger step size
:step size varies with time.
 
n

 
n
x
November 17, 2022 7/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Int. to Scalar Quantization
 N-Point Quantizer
Q
x   N
k
R
x
y
x
Q k
k 
,
1
for
if 


5-point quantizer
5
x
4
x
3
x
2
x
1
x
0
x
1
y 2
y 3
y 4
y 5
y
 Rk: quantization cells
 yk: reproduction value, i.e., output value, output points
 Codebook: C={y1, y2,…yN}
 Code rate R: R=Log2N, i.e., 每個index要用R=Log2N個bit
來編碼
November 17, 2022 8/43

5. 5
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Lossless
Decompression
Q
 
x
Q
yk 
Huffman Decoder Decoder
k
y
x
  k
R
x
k
x 
 if

 k
N
k 
,
1

Lossless
Compression
Huffman Encoder
Encoder
Int. to Scalar Quantization
 Index transmission
 We transmit the index (i.e., k) to the receiver and then
the receiver can reconstruct the quantized value from
the codebook using the index.
 Regular quantizer
 Q is regular if
 Rk: is an interval of the form (xk-1, xk)
 yk Rk
November 17, 2022
Table Look Up
9/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Quantizer Function
 Quantizer function
 We transmit the index (i.e., k) to the receiver and then
the receiver can reconstruct the quantized value from
the codebook using the index.
 傳送 k, 不是傳yk .
November 17, 2022 10/43

6. 6
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Quantizer Function
 Quantizer function:
 Mid-tread v.s. Mid-rise quantizer
 
x
Q
x
Mid-tread Q
 
x
Q
x
Mid-rise Q
Generally speaking, the performance of a mid-tread
quantizer is better than that of a mid-rise quantizer.
November 17, 2022 11/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Quantizer Function
 Quantizer function:
 Consider a Gaussian distributed source with zero mean.
Obviously, the use of a mid-rise quantizer would result in
large quantization errors.
 
x
P
x
0
 註:If the source distribution is Gaussian with zero mean→很多值是
0 →此時若使用mid‐rise quantizer, 則Quantization error 會很大。
November 17, 2022 12/43

7. 7
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Symmetric Quantizer
 Symmetric Quantizer:
 Granular / Overload Cells:
   
x
Q
x
Q 


unbounded
x :






 N
x
x ,
0
   




  ,
,
,
: 1
1
1 N
N x
R
x
R
cells
Overload
  1
,...,
2
,
,
: 1 

  N
K
x
x
R
cells
Granular K
K
K
November 17, 2022 13/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Uniform Quantizer
 Uniform Quantizer:
 An N-point quantizer is uniform if it is regular with
N
K
K
K
y
y ,...,
2
,
1





1
,...,
2
,
2
1



 
N
K
x
x
K
K
K
y


3
y
5
x
4
x
3
x
2
x
1
x
0
x
1
y 2
y 4
y 5
y
   
May be unbounded May be unbounded
1
R N
R


1
3
2 ......
: 
N
R
R
R
region
Granular U
U
U
N
R
R
region
Overland U
1
:
November 17, 2022 14/43

8. 8
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Distortion and Code rate R:
 Bounded uniform input
X: uniform distribution over (-v, v)
N: number of quantization cells.
R
V
N
V 



 2
2
2
Step size
 where is the code rate.
N
Log
R 2

R
R
N
N 



 2
1
2
 
x
Px
x
0
V
 V
2V
1
November 17, 2022 15/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:

size.
step
the
is
Δ
where
,
2
2
and
),
(






 e
x
Q
x
e





 


2
,
2
over
d
distribute
uniform
also
is
error
November 17, 2022 16/43

9. 9
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
 Mean square error
  12
2
2 

 e
E
D
  12
3
1
1 2
2
2
3
2
2
2
2 





 





 e
de
e
e
E
 
2
,
2
over
d
distribute
uniformly 



on
distributi
uniform
of
error
square
mean
size.
step
the
is
2
2
2
where
,
12
1 2 R
V
N
V
D 






R
R
V
V
D 2
2
2
2
3
1
)
2
2
(
12
1 




∆
November 17, 2022 17/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
 Take dB value:
R
V
Log
Log
D
Log 02
.
6
10
3
1
10
10 2
10
10
10 





bit.
additional
each
for
6db
by
decreases
D
2
V
D 
smooth.
ly
sufficient
pdf
input
as
long
so
input
nonuniform
for
valid
is
analysis
above
The
November 17, 2022 18/43

10. 10
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
 Unbounded input:
The MSE distortion can be calculated as follows:
       
overload
gran
x
D
D
dx
x
p
e
dx
x
p
e
dx
x
p
e
e
E
D





 


2
2
2
2
Granular region Overload region
 The error in overload region may be infinite,
Fortunately, the probability may also be very
small.
會有問題的就是這塊
November 17, 2022 19/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
 Definition:
 
D
x
E
Log
SNR
2
10
10

2
2
2
2
2
2 x
x
E
x
x
E x
x 















 

Variance=平方的平均值 –
平均值的平方
有些 application 不要
i.e., 取 variance only.
→觀察 small signal 之比
2
x
November 17, 2022 20/43

11. 11
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
 Definition:
D
x
Log
e
E
x
E
Log
D
x
E
Log
SNR
x
2
2
10
2
2
10
2
10 10
10
10























 Some of the applications calculate the
irrespective of the , i.e., use variance only,
to observe the SNR of small signal.
 
2
x
E
x
may be large
x
Mean Square Error Distortion
November 17, 2022 21/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Loading Factor:
 Loading factor:
X: unbounded
Q: Symmetric N-point quantizer
N
x
1

N
x
1
x
0
x
V
 V


r
V
V
r 
 ,
 Loading factor:
where σ is the standard deviation of the source.
to
leads
increasing





gran
gran
overload
D
v
r
D
D
r


November 17, 2022 22/43

12. 12
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
 SNR:
 
D
x
E
Log
SNR
2
10
10

SNR
r
r
optimal
dominate
overload
D
dominate
gran
D
 
   
overload
gran
x
D
D
dx
x
p
e
e
E
D
D
x
E
Log
SNR






2
2
2
10
where
10
November 17, 2022 23/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
SNR
r
r
optimal
dominate
overload
D
dominate
gran
D
 
   
overload
gran
x
D
D
dx
x
p
e
e
E
D
D
x
E
Log
SNR






2
2
2
10
where
10
4
~
2
:
of
choice
typical r
.
&
between
off
trade
a
provide gran
overload D
D
r
v
v

4

r
Gaussion
November 17, 2022 24/43

13. 13
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
 Unbounded input:

r
V 
 input pdf sufficiently smooth, so that e
uniform over 




 


2
,
2
 Overload distortion negligible.(This is an assumption)
R
R
gran r
V
D
D 2
2
2
2
2
2
3
1
2
3
1 



 
November 17, 2022 25/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Distortion and Code rate R
 Quantization error:
SNR
r
r
optimal


V V

V
 V
 decreasing by 6dB for each additional bit
Recall: , i.e., standard deviation,
, is determined by input statistics.
D  
fixed
,
r


 increases
D
  
fixed
,
2
R
r


 increases
D
  
fixed
,
2
R
r 







as
V
fixed
, r
r
V

2
as
increased 
V
變大 
 noise
granular
R
R
gran r
V
D
D 2
2
2
2
2
2
3
1
2
3
1 



 
November 17, 2022 26/43

14. 14
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Design of quantizer function to minimize distortion(MSE)
 Distortion:
 
   
   
value.
quantizer
the
is
where
1 1
2
2
i
N
i
i
i
y
dx
x
p
x
x i
y
x
dx
x
P
x
Q
x
D


 




N
x
1

N
x
1
x
0
x
V
 V
1
y N
y
Quantizer)
(Regular
November 17, 2022 27/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Answer:先決定其中一個!不要兩個同時做.
Find the value of a parameter, and then the
second one. Find one of which, and then other.
 Question:
(兩類參數都要做最佳化無法做)
    
minimized.
is
such that
,
,
,
,
,
.,
i.e
,
,
find
to
How
1
1
2
1
1
0
1
D
y
y
x
x
x
x
x
x
R
R
N
N
N
N


 
ε
x i
i
i
R
R
x
by
determined
if 
D
i i
y
i
y
by
determined
index of quantization bin
November 17, 2022 28/43

15. 15
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Q1:  
.
minimize
that
,
find
to
how
,
,
,
and
x
on
distributi
input
given
a
For
1
N
1
D
R
R
y
y
N
P
N


（N決定分為幾個Quantization bins）
 Q2:
.
minimize
that
,
,
,
,
find
to
how
,
,
,
Given N
2
1
2
1
D
y
y
y
R
R
R N 

November 17, 2022 29/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
1
y 2
y 3
y 4
y N
y
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Ans of Q1:
  2
3
2
2 y
y
x 

Nearest Neighbor Condition :NNC
  
 
2
x
on
quantizati
neighbor
nearest
x.
minimized
x
Q
-
x
if
minimized
1
i
2





i
i y
y
D
November 17, 2022 30/43

16. 16
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Ans of Q2:
minimized.
is
such that
y
find
to
How D
  
       
 
2
2
2
]
[V
E
V
E
dv
V
p
V-y
y
V
E 


 
N
x
1
x
0
x 2
x
重心(mean)
 
V
p
V,
November 17, 2022 31/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Ans of Q2: (CONT)
  
       
 
bin.
on
quantizati
each
on
condition
that
ies
probabilit
l
conditiona
use
we
is
That
Bin.
each
on
condition
i.e.,
y,
probabilit
l
conditiona
a
s
it'
problem,
our
Analogy to
]
[
2
2
2





  V
E
V
E
dV
V
p
V-y
y
V
E
重心(mean)
 





























 R
v
R
v
v
E
v
E
R
v
y
v
E
2
]
[
2
November 17, 2022 32/43

17. 17
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Ans of Q2: (CONT)
  
       dx
x
p
y
x
dx
x
p
x
Q
x
E
N
i
x
x
i
i
i

  




1
2
2
1
0
x 1
x 2
x N
x
Centroid Condition :CC
i
D
 
i
i
i R
x
x
E
y
D 

if
minimized
Centroid
 
 



i
i
R
R
i
dx
x
p
dx
x
xp
y
November 17, 2022 33/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Ans of Q2: (CONT)
Centroid Condition :CC
N
y
y
N ,
,
,
Given 1  NNC
Find Ri
D0
CC
Find yi
D1
NNC D2 CC D3
iteratively
給定yi要決定Ri
給定Ri要決定yi
November 17, 2022 34/43

18. 18
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Ans of Q2: (CONT)
Centroid Condition :CC
 Possibilities:
 Convergence is not promised.
 Can converge to local minima.
 Can converge to global minima.
November 17, 2022 35/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Lloyd-Max Algorithm
 Lloyd iteration
 Nearest Neighbor condition
 Centroid Condition
   
i
N
m R
y
y
C find
,
,
Given 1 

   
i
m
i y
C
R 
1
centroid
Compute
,
Given
m
C Nearest
Neighbor
condition
Centroid
Condition 1

m
C
 
i
R
November 17, 2022 36/43

19. 19
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Lloyd-Max Algorithm
 Algorithm:
 Initialize codebook
 Find improved codebook


threshold
,
1
, 
m
Cm
iteration.
Lloyd
using
1

m
C
to
go
,
1
else
stop,
,
if
distortion
Compute
1
1





m
m
D
D
m
m
 2
November 17, 2022 37/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Lloyd-Max Algorithm
 PS:
 There are many method to make the process stop.
Say, if error 的改進有限時（兩次誤差相減很小）也可停止.
 Say, we can stop the process when the improvement on the
quantization error is quite saturated, i.e.,
or we can stop the iteration process when the number of
iterations has been reached.
,
1 


 m
m D
D
November 17, 2022 38/43

20. 20
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Scalar Quantization of Discrete random Variable.
  level.
on
quantizati
the
is
where
,
,
,
,
1 N
N
m
a
a
x m 
 
Quantization level.
 Nearest Neighbor Condition:
   
i
i R
y find
,
Given
若m<N就不用Quantize 了.
2
a 3
a
1
a
1
y 2
y
4
a
N
y
 
2
1


 i
i
i
y
y
x





i
j
i
j
r
a
j
j
r
a
j
i
a
p
a
p
a
y
)
(
)
(
 Centroid Condition:
  i
i y
R find
,
Given
November 17, 2022 39/43
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
4
x
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 pf:
 
 
 
 
Condition
Neighbor
Nearest
)
2
(
if
uunchanged
suppose
0
,
5
4
4
5
7
4
7
7
7
4
y
y
x
y
a
Q
D
y
a
Q
a
p
a
x







optimal.
not
is
the
of
centroid
of
centroid
5
5
4
4
Q
R
y
R
y





  
y.
probabilit
zero
has
point
boundary
0

i
x
p
x
1
a 2
a 3
a 4
a 5
a 6
a 7
a 8
a 9
a
1
x 2
x 3
x 5
x
4
y 5
y
November 17, 2022 40/43

21. 21
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Problem of Optimal Quantizer:
 Complexity: uniform v.s. Nonuniform
 Uniform coding:
 for a mid-tread Quantizer.










x
round
value
Quantized
1
 0 1
x
2


2

Get the index of the quantization bin.
 Nonuniform coding:
Is there a formula to find the quantized value?
41/43
November 17, 2022
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Companded Quantization:
 Compressor + expander
G(x) Q
Compressor
G-1
(x)
Uniform Expander
 
x
G
x
1
y 2
y 3
y
 
1
y
G
 
2
y
G
 
3
y
G
1
x 2
x 3
x
 
1
x
G
 
2
x
G
 
3
x
G
November 17, 2022 42/43

22. 22
Lih-Jen Kau
Signal Proc. & Intell. Electron. Group
National Taipei Univ. of Technology
Problem of Optimal Quantizer
 Nonuniform distribution input:
 Instead of making the step size small, we could make the
interval in which the input lies with high probability large.
That is, expand the region in which the input lands with high
probability in proportion to the probability with which the input
lands in this region.
 The input is first mapped through a compressor function.
Which stretches the high-probability regions close to the
origin, and correspondingly compress the low-probability
regions away from the origin.
 If the output of the compressor function is quantized using a
uniform quantizer, and the quantized value transformed via
an expander function, the overall effect is the same as using
a nonuniform quantizer.
43/43
November 17, 2022