Basic intro and hands on exercises im image filtering

1. 1
Yossi CohenIntro to computer Vision

2. 2

3. 3
How to filter
2d Correlation
h=filter2(g,f); or h=imfilter(f,g);
2d Convolution
h=conv2(g,f);
],[],[],[
,
lnkmflkgnmh
lk
−−= ∑
f=image
g=filter
],[],[],[
,
lnkmflkgnmh
lk
++= ∑

4. 4
Correlation filtering
Say the averaging window size is 2k+1 x 2k+1:
Loop over all pixels in neighborhood around
image pixel F[i,j]
Attribute uniform
weight to each pixel
Now generalize to allow different weights depending on
neighboring pixel’s relative position:
Non-uniform weights

5. 5
Correlation filtering
Filtering an image: replace each pixel with a linear
combination of its neighbors.
The filter “kernel” or “mask” H[u,v] is the prescription for the
weights in the linear combination.
This is called cross-correlation, denoted

6. 6
Properties of smoothing filters
Smoothing
 Values positive
 Sum to 1  constant regions same as input
 Amount of smoothing proportional to mask size
 Remove “high-frequency” components; “low-pass” filter

7. 7
Filtering an impulse signal
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 1 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
a b c
d e f
g h i
What is the result of filtering the impulse signal
(image) F with the arbitrary kernel H?
?

8. 8
Filtering an impulse signal
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 1 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
a b c
d e f
g h i
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 i h g 0 0
0 0 f e d 0 0
0 0 c b a 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
What is the result of filtering the impulse signal
(image) F with the arbitrary kernel H?

9. 9
Averaging filter
 What values belong in the kernel H for the moving
average example?
0 10 20 30 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
111
111
111
“box filter”
?

10. 10
Smoothing by averaging
depicts box filter:
white = high value, black = low value
original filtered

11. 11
Gaussian filter
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
1 2 1
2 4 2
1 2 1
This kernel is an
approximation of a 2d
Gaussian function:

12. 12
Smoothing with a Gaussian

13. 13
Smoothing with a Box

14. 14
 Weight contributions of neighboring pixels by nearness
0.003 0.013 0.022 0.013 0.003
0.013 0.059 0.097 0.059 0.013
0.022 0.097 0.159 0.097 0.022
0.013 0.059 0.097 0.059 0.013
0.003 0.013 0.022 0.013 0.003
5 x 5, σ = 1
Slide credit: Christopher Rasmussen
Gaussian

15. 15
Gaussian filters
What parameters matter here?
Size of kernel or mask
Note, Gaussian function has infinite support, but discrete
filters use finite kernels
σ = 5 with
10 x 10
kernel
σ = 5 with
30 x 30
kernel

16. 16
Gaussian filters
What parameters matter here?
Variance of Gaussian: determines extent of
smoothing
σ = 2 with
30 x 30
kernel
σ = 5 with
30 x 30
kernel

17. 17
Matlab
>> hsize = 10;
>> sigma = 5;
>> h = fspecial(‘gaussian’ hsize, sigma);
>> mesh(h);
>> imagesc(h);
>> outim = imfilter(im, h); % correlation
>> imshow(outim);
outim

18. 18
Smoothing with a Gaussian
for sigma=1:3:10
h = fspecial('gaussian‘, fsize, sigma);
out = imfilter(im, h);
imshow(out);
pause;
end
…
Parameter σ is the “scale” / “width” / “spread” of the Gaussian
kernel, and controls the amount of smoothing.

19. 19
Gaussian filters
• Remove “high-frequency” components from the
image (low-pass filter)
Images become more smooth
• Convolution with self is another Gaussian
–So can smooth with small-width kernel, repeat, and
get same result as larger-width kernel would have
–Convolving two times with Gaussian kernel of width σ
is same as convolving once with kernel of width σ√2
• Separable kernel
–Factors into product of two 1D Gaussians
Source: K. Grauman

20. 20
Separability of the Gaussian filter
Source: D. Lowe

21. 21
Separability example
*
*
=
=
2D convolution
(center location only)
Source: K. Grauman
The filter factors
into a product of 1D
filters:
Perform convolution
along rows:
Followed by convolution
along the remaining column:

22. 22
Convolution
 Convolution:
 Flip the filter in both dimensions (bottom to top, right to left)
 Then apply cross-correlation
Notation for
convolution
operator
F
H

23. 23
Convolution vs. correlation
Convolution
Cross-correlation

24. 24
Key properties of linear filters
Linearity:
filter(f1 + f2) = filter(f1) + filter(f2)
Shift invariance: same behavior
regardless of pixel location
filter(shift(f)) = shift(filter(f))
Any linear, shift-invariant operator can be
represented as a convolution
Source: S. Lazebnik

25. 25
More properties
• Commutative: a * b = b * a
 Conceptually no difference between filter and signal
 But particular filtering implementations might break this equality
• Associative: a * (b * c) = (a * b) * c
 Often apply several filters one after another: (((a * b1) * b2) * b3)
 This is equivalent to applying one filter: a * (b1 * b2 * b3)
• Distributes over addition: a * (b + c) = (a * b) + (a * c)
• Scalars factor out: ka * b = a * kb = k (a * b)
• Identity: unit impulse e = [0, 0, 1, 0, 0],
a * e = a Source: S. Lazebnik

26. 26
Matlab
Lab 1 – Smooth/Blur Filters

27. 27
Lets try to blur filter
%%basic image filters
imrgb = imread('peppers.png');
imshow(imrgb);
a = [1 1 1; 1 1 1; 1 1 1]
Corroutimg = filter2(a, imgray);
Convoutimg = conv2(imgray,a);
figure;
imshow(Corroutimg)
figure
imshow(Convoutimg)

28. 28
Fix it
Can we conv2 an RGB image??
Use rgb2gray
Whats wrong now?
Try preserving the image power
Convert image to double

29. 29
Results
%%basic image filters
imrgb = imread('peppers.png');
imgray =
im2double(rgb2gray(imrgb));
imshow(imgray);
a = 1/16*ones(4)
Corroutimg = filter2(a, imgray);
Convoutimg = conv2(a, imgray);
figure;
imshow(Corroutimg)
figure
imshow(Convoutimg)

30. 30
Sharp
%%basic sharp filter
imrgb = imread('peppers.png');
imgray =
im2double(rgb2gray(imrgb));
imshow(imgray);
a = [ 0 0 0; 0 2 0; 0 0 0] -
1/9*ones(3)
Corroutimg = filter2(a,
imgray);
figure;
imshow(Corroutimg)

31. 31
Practical matters
What happens near the edge?
the filter window falls off the edge of the image
need to extrapolate
methods:
clip filter (black)
wrap around
copy edge
reflect across edge
Source: S. Marschner

32. 32
Matlab edge aware filtering
methods (MATLAB):
clip filter (black): imfilter(f, g, 0)
wrap around: imfilter(f, g, ‘circular’)
copy edge: imfilter(f, g, ‘replicate’)
reflect across edge: imfilter(f, g, ‘symmetric’)

33. 33
Practical matters
What is the size of the output?
• MATLAB: filter2(g, f, shape)
shape = ‘full’: output size is sum of sizes of f and g
shape = ‘same’: output size is same as f
shape = ‘valid’: output size is difference of sizes of f and g
f
gg
gg
f
gg
gg
f
gg
gg
full same valid

34. 34
Median filters
A Median Filter operates over a window by
selecting the median intensity in the window.
What advantage does a median filter have
over a mean filter?
Is a median filter a kind of convolution?

35. 35
Matlab
Lab 2 – Median Filter

36. 36
Comparison: salt and pepper noise

37. 37
Median Filter Example
%%Median
imrgb = imread('peppers.png');
imgray =
im2double(rgb2gray(imrgb));
imnoise = imnoise(imgray,'salt &
pepper',0.1);
imshow(imnoise)
figure
imshow(medfilt2(imnoise))

38. 38
Basic Filters summary
Linear filtering is sum of dot
product at each position
Can smooth, sharpen, translate
(among many other uses)
Be aware of details for filter size,
extrapolation, cropping
111
111
111