Slides of IPTA 2010 presentation - How to localize a given binary template in an image?

1. Binary Pattern
Matching from a Local
Dissimilarity Measure
F. Morain-Nicolier - Jérôme Landré - Su Ruan
CReSTIC - URCA - IUT Troyes - FRANCE
http://pixel-shaker.fr
IPTA 2010
1
mercredi 7 juillet 2010 1

2. ?
2
mercredi 7 juillet 2010 2

3. Outline
• Local Dissimilarity Measure
• LDM as Shape Matcher
• Results
3
mercredi 7 juillet 2010 3

4. Local Dissimilarity Map
E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparison with local-dissimilarity quantification”,
Pattern Recognition, vol. 41, n. 5, pp. 1461–1478, jan. 2008
4
mercredi 7 juillet 2010 4

5. Hausdorff Distance
HD(A, B) = max(h(A, B), h(B, A))
with h(A, B) = max(min d(a, b))
a∈A b∈B
5
mercredi 7 juillet 2010 5

6. Hausdorff Distance
HD(A, B) = max(h(A, B), h(B, A))
with h(A, B) = max(min d(a, b))
a∈A b∈B
5
mercredi 7 juillet 2010 5

7. Local Dissimilarity Map
ig. 3. Behavior of the LDMap on simple patterns. A vertical line, a horizontal one, a square and their LDMaps. The dar
measure.
Fig. 4. Letters “co” et “et” and their LDMap. The obtained LDMap (c) shows clearly both locations and quantifi
E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparison with local-dissimilarity quantification”,
Pattern Recognition, vol. 41, n. 5, pp. 1461–1478, jan. 2008
6
mercredi 7 juillet 2010 6

8. Local Dissimilarity Map
ig. 3. Behavior of the LDMap on simple patterns. A vertical line, a horizontal one, a square and their LDMaps. The dar
measure.
Fig. 4. Letters “co” et “et” and their LDMap. The obtained LDMap (c) shows clearly both locations and quantifi
E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparison with local-dissimilarity quantification”,
Pattern Recognition, vol. 41, n. 5, pp. 1461–1478, jan. 2008
6
mercredi 7 juillet 2010 6

9. Local Dissimilarity Map
ig. 3. Behavior of the LDMap on simple patterns. A vertical line, a horizontal one, a square and their LDMaps. The dar
measure.
Fig. 4. Letters “co” et “et” and their LDMap. The obtained LDMap (c) shows clearly both locations and quantifi
E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparison with local-dissimilarity quantification”,
Pattern Recognition, vol. 41, n. 5, pp. 1461–1478, jan. 2008
7
mercredi 7 juillet 2010 7

10. Local Dissimilarity Map
ig. 3. Behavior of the LDMap on simple patterns. A vertical line, a horizontal one, a square and their LDMaps. The dar
measure.
Fig. 4. Letters “co” et “et” and their LDMap. The obtained LDMap (c) shows clearly both locations and quantifi
E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparison with local-dissimilarity quantification”,
Pattern Recognition, vol. 41, n. 5, pp. 1461–1478, jan. 2008
8
mercredi 7 juillet 2010 8

11. Local Dissimilarity Map
Iterative algorithm = slow
LDMA,B (p) = |A(p) − B(p)| max(dtA (p), dtB (p))
dt : distance transform
(distance to nearest foreground pixel)
E. Baudrier, F. Nicolier, G. Millon, S. Ruan, ”Binary-image comparison with local-dissimilarity quantification”,
Pattern Recognition, vol. 41, n. 5, pp. 1461–1478, jan. 2008
9
mercredi 7 juillet 2010 9

12. Local Dissimilarity Map
For binary images :
LDMA,B (p) = |A(p) − B(p)| max(dtA (p), dtB (p))
LDMA,B = BdtA + AdtB
10
mercredi 7 juillet 2010 10

13. LDMap on simple patterns. A vertical line, a horizontal one, a square and their LDMaps. The darker the pixe
o” et “et” and their LDMap. The obtained LDMap (c) shows clearly both locations and quantification of th
Localized and quantified
11
mercredi 7 juillet 2010 11

14. ity scores between images. Figure 5 gives an example of
6
computed local-dissimilarity map (c) from two images.
Figure 5. Local-dissimilarity map of [2]
(a) image 1 (b) image 2 (c) dissimilarity map
The main characteristic and quantified is a comparison
Localized of the method
of images without any feature extraction, a high precision
is obtained about image differences. On the other hand, the
12
method increases computation times and can’t be used on-
mercredi 7 juillet 2010 12

15. sym
asy
F. M
frederic
http://
Plan
Pr´sent
e
Carte d
Locales
Mesure
mesure
Formula
Exempl
Bilan -
Propri´t´s :
ee
localisation, quantification, robustesse textured variations.
Good for non strongly aux petites images
6
13
mercredi 7 juillet 2010 13

16. LDM as template
matcher?
14
mercredi 7 juillet 2010 14

17. P reference shape
I binary image
find the position in I of an instance of P
use LDM as similarity measure
15
mercredi 7 juillet 2010 15

18. x I
y I(x,y) P
P
I(x,y)
to be compared!
LDMA,B = BdtA + AdtB
16
mercredi 7 juillet 2010 16

19. matcher (first version)
I(x,y) P
MI,P (x, y) = LDMI(x,y) ,P (k, l)
k l
MI,P (x, y) = N CSI,P (x, y) + N CSP,I (x, y)
Symetrical Chamfer Matching [Borgefors]
17
mercredi 7 juillet 2010 17

20. use quadratic sums to get
less false positives [Borgefors]
and
LDMA,B = BdtA + AdtB
matcher
2 2
(final QMI,P = dtI P +I dtP ( : cross-correlation)
version)
Quadratic Matcher
(very fast computation via Fourier)
18
mercredi 7 juillet 2010 18

21. Generalization (not used here)
2 2
QMI,P = dtI P +I dtP
2 2
AQMI,P = αdtI P + βI dtP
Links to Tverksy results : human judgement of
similarity is asymmetric
s(A, B) = F (A B) − αF (A − B) − βF (B − A)
19
mercredi 7 juillet 2010 19

22. An example
20
mercredi 7 juillet 2010 20

23. I binary image
P reference shape
Fig. 2. In (c), image response by Borgefors chamfer matcher. – In (d), image
obtained with symmetric LDM-matcher. – A good match with the reference
pattern is reported by low values (with dark gray levels).
Fig. 1. (a) A test image. – (b) a reference pattern, the ideal location in (a)
is labeled with G.
in the area N. The LDM-matcher is more selective. For the
that belong to b but not to a. A comparable idea can be area-N pixels, high values are obtained by the LDM-matcher.
formulated from (10) by weighting the contribution of each The area-N values can be interpreted as false-positives for
term (Generalized Quadratic Matcher - GQM): the Borgefors chamfer matching and true-negatives for the
LDM-matcher. The LDM-matcher can thus provide less false
positives in a pattern matching task than the Borgefors chamfer
AQMI,P = αdt2 P + βI dt2 . (11)
I P matching.
A symmetric matcher is obtained with α = β. Asymmetric Here is an interpretation of these observations. The pixel
ones are obtained with α = β, such as α = 1, β = 0 (the density in area-N is higher than in other locations of the
chamfer matching) or α = 0, β = 1. image. When the pattern template P is somewhere in area-N,
there is a high probability that a subset of pixels matching the
IV. T HE LDM- MATCHER CAN PROVIDE LESS FALSE pattern exists, providing low values by the Borgefors chamfer
POSITIVES matching. High values are given by the LDM-matcher as it is
a symmetric matcher. Low values are only obtained when P
The final LDM-matcher (eq. (11)) (symmetric one, with matches the images and the image matches the pattern. Let’s
α = β = 1) is compared in this section to the Borgefors
chamfer matcher (purely asymmetric, with α = 1 and β = 0).
take a drastic example : figure an area completely filled with Fig. 1. (a) A test image. – (b) a reference pattern,
21
pixels (ie an area filled with foreground pixels) and a pattern to
The behavior of the two matchers is compared in a concrete
example.
be matched. The Borgefors chamfer matching output responses is labeled with G.
would be 0 as distance transform values (dtI in eq. 7) are zero.
mercrediCoast image2010 are given in figure 1. A test pattern to be
7 juillet edges Intuitively each pixel of the pattern is corresponding to a pixel 21

24. low value → match!
Fig. 2. In (c), image response by Borgefors chamfer matcher. – In (d
obtained with symmetric LDM-matcher. – A good match with the r
pattern is reported by low values (with dark gray levels).
Fig. 1. (a) A test image. – (b) a reference pattern, the ideal location in (a)
is labeled with G.
in the area N. The LDM-matcher is more selective. F
that belong to b but not to a. A comparable idea can be area-N pixels, high values are obtained by the LDM-m
Chamfer Matching
formulated from (10) by weighting the contributionimage response by Borgefors chamfer be interpreted as image
Fig. 2. In (c), of each The area-N values can matcher. – In (d), false-positiv Quadratic LDM
term (Generalized Quadratic Matcher obtained with symmetric LDM-matcher. – chamfer match with and reference
- GQM): the Borgefors A good matching the true-negatives f
pattern is reported by low values (with dark gray levels).
LDM-matcher. The LDM-matcher can thus provide les matcher
nce pattern, the ideal location in (a) Fig. 2. In (c), image response by Borgefors chamfer matcher. – In (d), image
AQMI,P = αdt2 P + βI dt2 .
A test image. – (b) a reference pattern, the ideal location in (a)
with G. I
obtained with symmetric LDM-matcher. – A good match with the reference
(11)
pattern is reported by low values (with dark gray levels).
P matching.
(less false positives)
positives in a pattern matching task than the Borgefors c
in the area N. The LDM-matcher is more selective. For the
ng to bA symmetric matcher is obtained with α =the LDM-matcher.
but not to a. A comparable idea can be area-N pixels, high values are obtained by β. Asymmetric Here is an interpretation of these observations. Th
neralized Quadratic Matcher - GQM): the Borgefors in the=area N. The LDM-matcher is more selective. For the
d from (10) by weighting the contribution of each The area-N values can be interpreted as false-positives for
ones are obtained with α = β, chamfer matching and thus provideβforfalse 0 (the density in area-N is higher than in other locations
such as can true-negativesless = 22
α 1, the
A comparable idea = 0, in abe 1. area-NBorgefors chamfer high values are obtained by template P is somewhere in a
chamfer matching) or (11) positives β pattern matching task than the pixels, image. When the pattern the LDM-matcher.
LDM-matcher. The LDM-matcher
AQM 2
= αdt P + βI dt .
I,P I
2
P can
α matching. =
mercredi 7 juillet 2010
metric matcher is obtained with α = β. Asymmetric Here is an interpretation of these observations. The pixel there is a high probability that a subset of pixels match
22

25. Robust?
23
mercredi 7 juillet 2010 23

26. I
a) extract
pattern
P
b) invert some pixels
Pnoisy c) localize
24
mercredi 7 juillet 2010 24

27. [1] A. Andreev, N. Ki
Distances”, Proc. of
and Recognition, pp
[2] E. Baudrier, F. Nico
with local-dissimilar
5, pp. 1461–1478, ja
[3] G. Borgefors, ”Hiera
ing algorithm”, IEE
Intelligence, vol. 10
[4] Calypod - graphiC
http://calypod
[5] A. Ghafoor, R. N. I
Algorithm”, IDEAL
Hong Kong, LLNCS
[6] D.P. Huttenlocher, W
dorff distance”, IEE
Intelligence, vol. 15
[7] U. Montanari, ”A m
Texte distance”, Journal o
pp. 600-624, 1968.
[8] E. Rosch, ”Cognitive
4, pp. 532-547, 1975
[9] S. Santini, R. Jain,
Mach. Intell., vol. 2
[10] A. Tversky, ”Featu
4, pp. 327-352, 1977
[11] B. Zitova, J. Flusse
Fig. 4. Noise robustness of the Borgefors chamfer-matcher and the LDM-
Vision Computing, v
matcher. The mislocation error (a distance in pixel) is given with respect to
the perturbation rate of the pattern (percentage of inverted pixels).
localization error vs pattern perturbation
matchers. The position error is estimated by the euclidean
distance between the real position and the computed pattern
position. For these three images, one thousand patterns have
25
mercredi 7 juillet 2010 been extracted and processed. 25

28. Conclusion
• Template matching via Local Dissimilarity
Measure (LDM)
• LDM = symmetrical Chamfer matching
• Less false positives
• More robust to noise
26
mercredi 7 juillet 2010 26

29. Main future works
• Playing with asymmetry weights
• Formalize and use links with Tversky’s
constrast model
• Going to gray-level images similarity (soon
via extended distance transform)
27
mercredi 7 juillet 2010 27