Image interpolation

1. Image Interpolation

2. Image ]nterpolatiorn
• Introduction
- What is image interpolation?
- Why do we need it?
• Interpolation Techniques
- 1D zero-order, first-order, third-order
- 2D zero-order, first-order, third-order
- Directional interpolation*
• Interpolation Applications
- Digital zooming (resolution enhancement)
- Image inpainting (error concealment)
- Geometric transformations

3. Introductiorn
• What is image interpolation?
-An image f(x,y) tells us the intensity
values at the integral lattice locations,
i.e., when x and y are both integers
-Image interpolation refers to the "guess"
of intensity values at missing locations,
i.e., x and y can be arbitrary
-Note that it is just a guess (Note that all
sensors have finite sampling distance)

4. Introduction (Corn't)
• Why do we need image interpolation?
-We want BIG images
• When we see a video clip on a PC, we like to
see it in the full screen mode
-We want GOOD images
• If some block of an image gets damaged
during the transmission, we want to repair it
-We want COOL images
• Manipulate images digitally can render fancy
artistic effects as we often see in movies

5. Scenario I: Rescolutiorn
Enhancement
-
0
-
0
-
0
0 - 0 -
0 0 0 ()
0 - 0 -
0 0 0 ()
0 - 0
0 0 0 ()
0
0
0 -
0 0
0
0
Low-Res.
High-Res.

6. Scenario ]I: Image
Inpainting
- 0 - - -
0 - -
- - 0 - - -
l - - - 0 - -
- - 0 -
- - - - 0 -
- Non-damaged 0 Damaged

7. Scenario II]: Image
Warping
• • • •
• • • •
• • • •
• • • •
• •'• .
• • • •
• • • •
• • • •

8. Image ]nterpolatiorn
• Introduction
- What is image interpolation?
- Why do we need it?
• Interpolation Techniques
- 1 D zero-order, first-order, third-order
- 2D zero-order, first-order, third-order
- Directional interpolation*
• Interpolation Applications
- Digital zooming (resolution enhancement)
- Image inpainting (error concealment)
- Geometric transformations

9. 1D Zero-order (Replication)
r [ f(n)
f(x)
X

10. 1D First-order Interpolation (Linear)
r r [ f(n)
f(x)
X

11. lirn1ear Interpolation Forrmula
Basic idea: the closer to a pixel, the higher weight is assigned
f(n)
-- __________f_(�-��) f(n+ 1)
---
a 1-a
If(n+a)=(l-a)xf(n)+axf(n+l), O<a<l I
Note: when a=0.5, we simply have the average of two

12. Numerical Examples
f(n)=[O,120,180,120,0]
1 Interpolate at 1/2-pixel
f(x)=[0,60,120,150,180,150,120,60,0], x=n/2
1 Interpolate at 1/3-pixel
f(x)= [0,20,40,60,80,100,120,130,140,150,160,170,180,...], x= n/6

13. 1D Third-order Interpolation (Cubic)
j(n)
r
j Cubic spline fitting I
f(x)

14. From 1D to 2D
•Just like separable 2D transform (filtering) that can be
implemented by two sequential 1D transforms (filters)
along row and column direction respectively, 2D
interpolation can be decomposed into two sequential
1D interpolations.
•The ordering does not matter (row-column= column-row)
•Such separable implementation is not optimal but enjoys low
computational complexity

15. Graphical Interpretation
of In1terpolatior1 at Half-pel
• • • •
• • • •
• • • •
• • • •
f(m,n)
•o •o •o • •••••••
0 000 00 0
• 0 • 0 e 0 • •••••••
row column0 000 00 0
• 0. 0 e O e
• 0. 0 e O e
•••••••
0 000 00 0
•••••••
g(m,n)

16. Numerical Examples
a
a
C
C
zero-ord
1
/
a b b
a b b
C d d
C d d
a b
C d
a
(a+c)/2
C
first-order
(a+b)/2
(a+b+c+d)/4
(c+d)/2
b
(b+d)/2
d

17. Numer·ical Examples (Con't)
row m
Coln
'
'
'
'
Col n+l
--------X(-hir�}-··
r
····-·····-------------X(-mr�+ 1)
' '
' '
' .
:
b
:
' '
! a Y 1-a l
'
'
1-b
row m+1------X(f11+±;-A-)· ------·-·······-·------X(-f-11+1,n+1)
' '
' '
' .
' .
' '
' '
Q: what is the interpolated value· at Y?
Ans.: ( 1-a)(l-b)X(m,n)+( 1-a)bX(m+ 1,n)
+a(l-b)X(m,n+1)+abX(m+ 1,n+ 1)

18. Bicubic Interpolation*
Final fmege
2 2
Foint to estimate (i',j')
F(i',j') • I: L F(i + m,j + n) R(m - dx) R(dy - n)
m•-1 n•-1
1
[
3 3 3 3]
R(x) =
6 P(x + 2) - 4 P(x + 1) + 6 P(x) - 4 P(x - 1)
I
I
.
.
.
.
.
.
. :/' .....
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. .
.
.
.
.
.
. '
.
' . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
Originel lmege
x>O
xiO

19. Limitation with
bilinear/bicubic
• Edge blurring
• Jagged artifacts
Edge blurring
'''''
< z
'
__ ___/
X
Jagged artifacts

20. DirectionaI Interpolat·ion�
Step 1: interpolate the missing pixels along the diagonal
■■■■ a b
■ ■ Since la-cl=lb-dl
X
□ □ x has equal probability
□□□□ of being black or white
C d
black or white?
Step 2: interpolate the other half missing pixels
a
■
d
□ □
C
Since la-cl>lb-dl
x=(b+d)/2=black

21. Image ]nterpolatiorn
• Introduction
- What is image interpolation?
- Why do we need it?
• Interpolation Techniques
- 1D zero-order, first-order, third-order
- 2D zero-order, first-order, third-order
- DirectionaI interpolation*
• Interpolation Applications
- Digital zooming (resolution enhancement)
- Image inpainting (error concealment)
- Geometric transformations

22. Pixel Replication
low-resolution
image (100 X 100)
high-resolution
image (400 X400)

23. Bilinear Interpolation
low-resolution
image (100 X 100)
.--------..,....,
high-resolution
image (400 X 400)

24. Bicubic Interpolation
low-resolution
image (100 X 100)
l1igh-resolutio11
image (400 X 400)

25. Edge-Directed Interpo�atioin
(Li&Orclhard'2000)
low-resolution
image (100 X 100)
high-resolution
image (400 X 400)

26. Image Demosaicing
(Color-FiIter-Array InterpoIatiom)
----
---- ----
---- ----
----
q ----
----
Bayer Pattern
---- ----
---- ----
---- ----
---- ----

27. Image Example
Ad-hoc CFA Interpolation Advanced CFA Interpolation

28. Error Corncealrnent
damaged interpolated

29. Image Irnpainting

30. Geometric Transformation
Widely used in co mputer graphics to generate special effects
MATLAB functions: griddata, interp2, maketform, imtransform

31. Basic Principle
• (x,y) ➔ (x',y') is a geometric
transformation
•We are given pixel values at (x,y)
and want to interpolate the unknown
values at (x',y')
• Usually (x',y') are not integers and
therefore we can use linear
inter olation to uess their values
MATLAB implementation: z'=interp2(x,y,z,x',y',method);

32. Rotation
x'
y'
cos0 sin0 x
-sin0 cos0 y
x'
X

33. -
Rotation Example

34. Scale
a=l/2

35. Affine Trarnsform
square parallelogram

36. Affine Transform Example

37. square
Shear
parallelogram

38. Shear Example

39. Projective Transrorr-m
A B
D C
square
B'
A'
C'
D'
quadrilateral
, alX + a2 y + G3
x =------
a7 x +a8 y+l

40. Projective Transform Example
o-��-�--
o 1
02
0 .
3
U.4
0.5
0.6
0.7
0.8
0 9
0.2 0 4 0.6 0.8
-$ � -4 -2 0 2 4 6
[ 0 O; 1 O; 1 1; O 1] - [-4 2; -8 -3; -3 -5; 6 3]

41. Polar Transform
r
0
120•
135"
-Jx
2
+ y2
165'
180'
tan-1 �
195'·
X 210• •
225'
240'
105'
265'
so· 75•
60'
45•
__1s·
e=o·
345'
no·
315'
300'
270'
285'

42. Iris Image Unwrapping
r
0

43. Use Your Imagination
r-> sqrt(r)
/
i--- --.......
"
/ /
V -----, "'
I I ,V
I I I /
/ '
.. _/ )
",.__ / / I J
I'--- _/ ,j I
' �
"........
I'--- -----/
/
"-...i---... ·----/
http://astronomy.swin.edu.au/~pbourke/projection/imagewarp/

44. Free Form Deformaticorn
Seung-Yong Lee et al., "Image Metamorphosis Using Snakes and
Free-Form Deformations,"S/GGRAPH'1985, Pages 439-448

45. Application irnto Image
Metamorohosis�
JJ

46. Different Image interpolation algorithms used
are:
• Nearest Neighbour Interpolation
• Bilinear Interpolation
• Bicubic Interpolation

47. Nearest Neighbour Interpolation
• Sample method of multivariative interpolation in 1 or more dimensions.
• Simply selects the value of nearest point and neglects other neighbouring
points. Thus, it yields a piecewise-constant interpolant.
• It is one of the easiest method to implement but the quality is worst.
Algorithm
• do scaleheight= height of new image /
height of original image
• do scalewidth= width of new image /
width of original image
• for i=0 to height of the new image
for j=0 to width of the new image
for k=0 to original image colour channels
• do copy channel by channel data of
nearest pixel of original image

48. Bilinear Interpolation
• Extension of linear interpolation for interpolating functions of two variables
on a regular grid.
• Key idea is to perform linear interpolation first in one direction, and then
again in other direction.
• It gives better result than nearest neighbour but is slower than latter.
• As the name suggests, it is linear interpolation in two directions, first in
horizontal direction then by a vertical direction or vice-versa. Bilinear
interpolation uses weighted average of the 4 neighborhood pixels to calculate
its final interpolated pixel.

49. Image Interpolator
Bicubic Interpolation
• It is an extension of cubic interpolation for interpolating data points on a
two dimensional grid.
• It can be accomplished using either Lagrange Polynomials or Cubic
Convolution algorithm.
• It is chosen over Nearest Neighbour and Bilinear interpolation as its
interpolated surface is smoother than both of them and has fewer
interpolation artifacts.

50. Numerical

54. THANK YOU !!!