The document summarizes Michel Alves' 10-minute speech comparing two image quality assessment metrics: normalized cross-correlation (NCC) and structural similarity index (SSIM). It provides background on the metrics, including their mathematical formulas and properties. Various images are used to generate similarity maps using both metrics side-by-side for visual comparison. While NCC is simple and fast, SSIM better captures structural changes but has greater complexity and potential numerical instability. In conclusions, the document advises using an appropriate metric depending on application and reevaluating metrics in some cases.
1. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Ten Minute Speech :: Overview of Activities Developed in Disciplines and Guided Studies :: Laboratory Seminars and Meetings
Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/LCG :: March/2015
Using Similarity Maps in
Visual Comparisons
Michel Alves dos Santos
Ten Minute Speech
(March, 2015)
Advisors: D.Sc. Ricardo Marroquim & Ph.D. Claúdio Esperança
2. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
IntroductionIntroduction
How to measure the visual
similarity between images?
[Lima'15][Nasc'08]
...Using IQA Metrics...
Introduction
-1-1
+1+1
+1+1
-1-1
NCC
SSIM
3. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
MetricsMetrics
What metric should I use?
MSE? PSNR?
<<No!!! Avoid ambiguous metrics!>>
Metrics
Impulsive Noise, MSE = 225
"A Universal Image Quality Index", Wang & Bovik, 2002, IEEE Signal Processing Letters.
Gaussian Bluring, MSE = 225 Contrast Stretching, MSE = 225
MAE?
4. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Today MetricsToday Metrics
Today we have a hundred of
metrics and several surveys!
We will compare the performance of two...
Each of these metrics have specific use...
Metrics
Metrics: NCC and SSIM...
5. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Range:
NCC vs SSIMNCC vs SSIM
Normalized Cross-Correlation:
Structural Similarity Index: [Wang'04]
In our comparison we will assemble similarity maps with these metrics...
NCCxSSIM
Range:
6. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
NCC – Normalized Cross-CorrelationNCC – Normalized Cross-Correlation
covariance
stdev
mean mean
stdev
NCC
7. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
NCC – Normalized Cross-CorrelationNCC – Normalized Cross-Correlation
Homogeneous Patches
...Numerical Instability...
Suggestion: add a small ε
in the denominator...
NCC
8. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
SSIM – Structural Similarity IndexSSIM – Structural Similarity Index
bits per pixel
dynamic range of pixels
suggestion
suggestion
SSIM
9. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
SSIM – Structural Similarity IndexSSIM – Structural Similarity Index
Homogeneous Patches
...Constants Avoid Numerical Instability...
SSIM
10. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Montage of MapMontage of Map
3x3 sub in 5x5 window 7 pixels in 5x5 window 9 pixels in 5x5 window
Centered Window
Directional Window
3x3 sub in 5x5 window
TsujiTomita Nagao7 Nagao9 NagMod
(compute a mean neighbors value around a central pixel)
(target pixel value)(types of centralized neighborhood)
Montage
11. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
ComparisonsComparisons
Cartoon Lighting Color Harmonization
Image-Based Reconstruction
Comparisons
12. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
NCC Map[Lima'15][Nasc'08]
*NCC Map with 3x3 window (centered neighborhood)
Comparisons – Gingerbread - NCCComparisons – Gingerbread - NCC
-1-1 +1+1
*Map obtained using YIQ luminance.
Comparisons
13. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
SSIM Map[Lima'15][Nasc'08]
*SSIM Map with 3x3 window (centered neighborhood)
Comparisons – Gingerbread - SSIMComparisons – Gingerbread - SSIM
-1-1 +1+1
*Map obtained using YIQ luminance.
Comparisons
14. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
NCC Map SSIM Map Overlay
-1-1 +1+1-1-1 +1+1
*NCC and SSIM Maps with 3x3 window (centered neighborhood)
Comparisons – Gingerbread - MapsComparisons – Gingerbread - Maps
-1-1 +1+1
Comparisons
15. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Purple Flower - NCCComparisons – Purple Flower - NCC
NCC Map[Lima'15][Nasc'08]
-1-1 +1+1
Observe the behavior of this index in
homogeneous areas...
*NCC Map with 3x3 window (centered neighborhood) *Map obtained using YIQ luminance.
Comparisons
16. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Purple Flower - SSIMComparisons – Purple Flower - SSIM
SSIM Map[Lima'15][Nasc'08]
-1-1 +1+1
Observe the behavior of this index in
homogeneous areas...
*SSIM Map with 3x3 window (centered neighborhood) *Map obtained using YIQ luminance.
Comparisons
17. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Purple Flower - MapsComparisons – Purple Flower - Maps
Overlay[Lima'15][Nasc'08]NCCMap
SSIMMap
-1-1
+1+1
-1-1
+1+1
Comparisons
18. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Color HarmonizationComparisons – Color Harmonization
-1-1
+1+1
-1-1
+1+1
InputHarmonized
*Maps obtained using average luminance.[Color Harmonization Project'14]
Comparisons
NCCMapSSIMMap
19. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Color HarmonizationComparisons – Color Harmonization
-1-1
+1+1
-1-1
+1+1
InputHarmonized
*Maps obtained using YIQ luminance.[Color Harmonization Project'14]
Comparisons
NCCMapSSIMMap
20. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Image-Based ReconstructionComparisons – Image-Based Reconstruction
-1-1
+1+1
-1-1
+1+1
NCCMapSSIMMap
[Cout'15]
PhotoRender
*Maps obtained using YIQ luminance.
*Dataset: Nana.
Comparisons
21. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Image-Based ReconstructionComparisons – Image-Based Reconstruction
-1-1
+1+1
-1-1
+1+1
NCCMapSSIMMap
[Cout'15]
PhotoRender
*Back vision of the previous model.
*Maps obtained using YIQ luminance.
Comparisons
22. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Image-Based ReconstructionComparisons – Image-Based Reconstruction
-1-1
+1+1
-1-1
+1+1
NCCMapSSIMMap
[Cout'15]
PhotoRender
*Maps obtained using YIQ luminance.
*Dataset: Buddha.
Comparisons
23. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
Comparisons – Image-Based ReconstructionComparisons – Image-Based Reconstruction
-1-1
+1+1
-1-1
+1+1
NCCMapSSIMMap
[Cout'15]
PhotoRender
*Maps obtained using YIQ luminance.
*Dataset: Gertrud.
Comparisons
24. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Introduction ConclusionComparisonsMetrics MontageNCCxSSIM NCC SSIM
ConclusionsConclusions
NCC:
SSIM:
fast and simple...
has numerical instability control...
is not so simple...
Advice: Use appropriate metric ever!
excellent perception of structural change...
not designed for images only...
good perception of local variation...
needs revaluation in pathological cases...
presents numerical instability...
Conclusion
25. Michel Alves – Laboratory of Computer Graphics (LCG) – Graduate Program in Systems Engineering and Computing – UFRJ/COPPE/PESC
Ten Minute Speech :: Overview of Activities Developed in Disciplines and Guided Studies :: Laboratory Seminars and Meetings
Federal University of Rio de Janeiro (UFRJ) :: Campus Cidade Universitária :: Rio de Janeiro :: Fundão Island :: COPPE/PESC/LCG :: March/2015
Thanks for your attention!
References:
[Nasc'08]: Representation and Ilumination of 2D Objects, 2008,
Renata Nascimento et al.
[Lima'15]: Local lighting cartoons method using tree regions,
2015, Ailton Lima.
[Cout'15]: Per Vertex BRDF Acquisition, 2015, Daniel Coutinho.
[CHPr'14]: Color Harmonization Project, 2014, Michel Alves.
[Wang'04]: Image quality assessment: From error visibility to
structural similarity, 2004, Zhou Wang et al.
Michel Alves dos Santos