Introduction to ArtificiaI Intelligence in Higher Education
Image denoising
1. Limitation of Imaging
Technology
Two plagues in image acquisition
Noise interference
Blur (motion, out-of-focus, hazy weather)
Difficult to obtain high-quality images as
imaging goes
Beyond visible spectrum
Micro-scale (microscopic imaging)
Macro-scale (astronomical imaging)
2. What is Noise?
Wiki definition: noise means any
unwanted signal
One person’s signal is another one’s
noise
Noise is not always random and
randomness is an artificial term
Noise is not always bad (see stochastic
resonance example in the next slide)
4. Image Denoising
Where does noise come from?
Sensor (e.g., thermal or electrical
interference)
Environmental conditions (rain, snow etc.)
Why do we want to denoise?
Visually unpleasant
Bad for compression
Bad for analysis
8. Image Denoising
Introduction
Impulse noise removal
Median filtering
Additive white Gaussian noise removal
2D convolution and DFT
Periodic noise removal
Band-rejection and Notch filter
9. Impulse Noise (salt-pepper
Noise)
Definition
Each pixel in an image has the probability of p/2 (0<p<1) being
contaminated by either a white dot (salt) or a black dot (pepper)
with probability of p/2
with probability of p/2
with probability of 1-p
noisy pixels
clean pixels
X: noise-free image, Y: noisy image
Note: in some applications, noisy pixels are not simply black or white,
which makes the impulse noise removal problem more difficult
WjHi
jiX
jiY
≤≤≤≤
=
1,1
),(
0
255
),(
11. MATLAB Command
>Y = IMNOISE(X,'salt & pepper',p)
Notes:
• The intensity of input images is assumed to be normalized to [0,1].
If X is double, you need to do normalization first, i.e., X=X/255;
If X is uint8, MATLAB would do the normalization automatically
• The default value of p is 0.05 (i.e., 5 percent of pixels are contaminated)
• imnoise function can produce other types of noise as well (you need to
change the noise type ‘salt & pepper’)
12. Impulse Noise Removal Problem
Noisy image Y
filtering
algorithm
Can we make the denoised image X as close
to the noise-free image X as possible?
^
X^denoised
image
13. Median Operator
Given a sequence of numbers {y1,
…,yN}
Mean: average of N numbers
Min: minimum of N numbers
Max: maximum of N numbers
Median: half-way of N numbersExample
sorted
]56,55,54,255,52,0,50[=y
54)( =ymedian
]255,56,55,54,52,50,0[=y
14. y(n)
W=2T+1
1D Median Filtering
… …
Note: median operator is nonlinear
MATLAB command: x=median(y(n-T:n+T));
)](),...,(),...,([)(ˆ TnynyTnymediannx +−=
20. Reflections
What is good about median operation?
Since we know impulse noise appears as
black (minimum) or white (maximum) dots,
taking median effectively suppresses the
noise
What is bad about median operation?
It affects clean pixels as well
Noticeable edge blurring after median
filtering
21. Idea of Improving Median Filtering
Can we get rid of impulse noise without
affecting clean pixels?
Yes, if we know where the clean pixels are
or equivalently where the noisy pixels are
How to detect noisy pixels?
They are black or white dots
22. Median Filtering with Noise Detection
Noisy image Y
x=medfilt2(y,[2*T+1,2*T+1]);
Median filtering
Noise detection
C=(y==0)|(y==255);
xx=c.*x+(1-c).*y;
Obtain filtering results
24. Image Denoising
Introduction
Impulse noise removal
Median filtering
Additive white Gaussian noise removal
2D convolution and DFT
Periodic noise removal
Band-rejection and Notch filter
25. Additive White Gaussian Noise
Definition
Each pixel in an image is disturbed by a Gaussian random variable
With zero mean and variance σ2
X: noise-free image, Y: noisy image
Note: unlike impulse noise situation, every pixel in the image contaminated
by AWGN is noisy
WjHiNjiN
jiNjiXjiY
≤≤≤≤
+=
1,1),,0(~),(
),,(),(),(
2
σ
27. MATLAB Command
>Y = IMNOISE(X,’gaussian',m,v)
>Y = X+m+randn(size(X))*v;
or
Note: rand() generates random numbers uniformly distributed over [0,1]
randn() generates random numbers observing Gaussian distribution
N(0,1)
28. Image Denoising
Noisy image Y
filtering
algorithm
Question: Why not use median filtering?
Hint: the noise type has changed.
X^denoised
image
29. f(n) h(n) g(n)
- Linearity
- Time-invariant property
Linear convolution
1D Linear Filtering
See review section
)()()()()()()( nhnfnfnhknfkhng
k
⊗=⊗=−= ∑
∞
−∞=
)()()()( 22112211 nganganfanfa +→+
)()( 00 nngnnf −→−
30. forward
inverse
Note that the input signal is a discrete sequence
while its FT is a continuous function
time-domain convolution frequency-domain multiplication
Fourier Series
∑
∞
∞−
−
= jwn
enfwF )()(
∫−
=
π
ππ
dwewFnf jwn
)(
2
1
)(
)()( nhnf ⊗ )()( wHwF
32. forward transform inverse transform
• Properties
- periodic
- conjugate symmetric
1D Discrete Fourier Transform
Proof:
Proof:
)()( kyNky =+
)()( *
kykNy =−
∑
−
=
=
1
0
)()(
N
n
kn
NWnxky ∑
−
=
−
=
1
0
)()(
N
n
kn
NWkynx
∑∑
−
=
−
=
+
===+
1
0
1
0
)(
)()(
N
n
kn
Nn
N
n
nNk
Nn kyWxWxNky
∑∑
−
=
−
−
=
−
===−
1
0
*
1
0
)(
)()(
N
n
kn
Nn
N
n
nkN
Nn kyWxWxkNy
}
2
exp{
N
j
WN
π
−=
33. Matrix Representation of 1D DFT
Re
Im
DFT:
FA =
=×
NNN
N
NN
aa
aa
......
............
............
......
1
111
1,
,
1
2
==
=
−
N
N
N
j
N
kl
Nkl
WeW
W
N
a
π
NWkl
N
N
l
lk Wx
N
y ∑=
=
1
1
∑=
=
N
l
lklk xay
1
34. Fast Fourier Transform (FFT)*
Invented by Tukey and Cooley in 1965
Basic idea: divide-and-conquer
Reduce the complexity of N-point DFT from
O(N2) to O(Nlog2N)
N/2-point DFT N/2-point DFT
N-point DFT
∑∑
∑∑∑
−=
−
−
=
−=
−
−
=
−
=
+=
+==
12
2/12
2
2/2
12
)12(
12
2
2
2
1
0
mn
km
Nm
k
mn
km
Nm
mn
mk
Nm
mn
mk
Nm
N
n
kn
Nnk
WxWWx
WxWxWxy
N
35. Filtering in the Frequency Domain
convolution in the time domain is equivalent to
multiplication in the frequency domain
f(n) h(n) g(n) F(k) H(k) G(k)
DFT
)()()( nhnfng ⊗= )()()( kHkFkG =
36. 2D Linear Filtering
f(m,n) h(m,n) g(m,n)
2D convolution
MATLAB function: C = CONV2(A, B)
),(),(),(),(),(
,
nmfnmhlnkmflkhnmg
lk
⊗=−−= ∑
∞
−∞=
37. 2D Filtering=Two Sequential
1D Filtering
Just as we have observed with 2D
transform, 2D (separable) filtering can
be viewed as two sequential 1D filtering
operations: one along row direction and
the other along column direction
The order of filtering does not matter
h1 : 1D filter
)()()()(),( 1111
mhnhnhmhnmh ⊗=⊗=
39. Fourier Series (2D case)
Note that the input signal is discrete
while its FT is a continuous function
spatial-domain convolution frequency-domain multiplication
∑ ∑
∞
−∞=
∞
−∞=
+−
=
m n
nwmwj
enmfwwF )(
21
21
),(),(
),(),( nmhnmf ⊗ ),(),( 2121 wwHwwF
42. Image DFT Example (Con’t)
choice 1: Y=fft2(X) choice 2: Y=fftshift(fft2(X))
Low-frequency at the centerLow-frequency at four corners
FFTSHIFT Shift zero-frequency component to center of spectrum.
45. Gaussian Filter=Heat Diffusion
Linear Heat Flow Equation:
scale A Gaussian filter
with zero mean
and variance of t
Isotropic diffusion:
2
2
2
2
),,(),,(
),,(
),,(
y
tyxI
x
tyxI
tyxI
t
tyxI
∂
∂
+
∂
∂
=∆=
∂
∂
)()0,,(),,( tGyxItyxI ⊗=
46. Basic Idea of Nonlinear
Diffusion*
x
y
I(x,y)
image I
image I viewed as a 3D surface (x,y,I(x,y))
Diffusion should be anisotropic
instead of isotropic
54. Advanced Denoising
Techniques*
Basic idea: from linear diffusion (equivalent to Gaussian filtering)
to nonlinear diffusion (with implicit edge-stopping criterion)
IN∇
Is∇
IE∇IW∇
jijiN III ,,1 −=∇ −
jijiS III ,,1 −=∇ +
jijiE III ,1, −=∇ +
jijiW III ,1, −=∇ −
][,
1
, IcIcIcIcII WWEESSNN
t
ji
t
ji ∇+∇+∇+∇+=+
λ
WESNdIgc dd ,,,||),(|| =∇=