Exploring the Future Potential of AI-Enabled Smartphone Processors
5 igarss2011_mdm.pdf
1. A General Approach to the Spatial Simplication of Remote Sensing
Images Based on Morphological Connected Filters
Mauro Dalla Mura
†, , Jón Atli Benediktsson , Lorenzo Bruzzone†
† Department of Information Engineering and Computer Science
University of Trento.
Faculty of Electrical and Computer Engineering
University of Iceland.
IGARSS 2011
24-29 July
2. Outline
1 Introduction
2 General Approach for Image Simplication
Connected Operators
Methodology
3 Conclusion and Future Developments
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 2 / 19
3. Introduction
Remote Sensing VHR Images
QuickBird 60cm, Panchromatic image, Bam (Iraq) Geoeye 50cm, Pansharpened images, Vancouver (Canada)
c Google
ROSIS-03 1.3m, Hyperspectral image, Pavia (Italy) TerraSAR-X 1.1m, Spotlight SAR, Dorsten (Germany)
The information extraction in remote sensing images is becoming increasingly
complex due to the progressively higher spatial resolution of the current
sensors.
How to extract the informative components dealing with the huge amount
of details?
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 3 / 19
4. Introduction
Remote Sensing VHR Images
QuickBird 60cm, Panchromatic image, Bam (Iraq) Geoeye 50cm, Pansharpened images, Vancouver (Canada)
c Google
ROSIS-03 1.3m, Hyperspectral image, Pavia (Italy) TerraSAR-X 1.1m, Spotlight SAR, Dorsten (Germany)
The information extraction in remote sensing images is becoming increasingly
complex due to the progressively higher spatial resolution of the current
sensors.
How to extract the informative components dealing with the huge amount
of details?
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 3 / 19
5. Introduction
Image Simplication
Spatial simplication of the image.
Pre-processing operation for many remote sensing applications:
Image segmentation;
Supervised/unsupervised thematic classication;
Land cover change analysis;
Object recognition and extraction;
Denoising SAR images;
Analysis of multiangular images.
Image simplication leads to:
noise reduction;
exploiting the contextual relations;
modeling spatial relations;
removing or attenuating undesired details.
Simplication of the image performed by spatial ltering, a 2-step procedure
composed of:
1 selection of the lters parameters;
2 application of the operator on the image.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 4 / 19
6. Introduction
Image Simplication (examples)
VHR optical image - dierent simplications
Which details should be removed?
Which operator should be applied? And which lter parameters should be
selected?
It depends on the application and on the type of image.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 5 / 19
7. Introduction
Image Simplication (examples)
VHR optical image - dierent simplications
Which details should be removed?
Which operator should be applied? And which lter parameters should be
selected?
It depends on the application and on the type of image.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 5 / 19
8. Introduction
Image Simplication (examples)
VHR optical image - dierent simplications
Which details should be removed?
Which operator should be applied? And which lter parameters should be
selected?
It depends on the application and on the type of image.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 5 / 19
9. Introduction
Motivation
Issues related to image simplication
The selection of the parameters of the lters is application dependent.
Proper operators should be used.
Manual operation.
Aims of the work
Dene a novel general approach to image simplication
based on morphological connected operators;
suitable for the processing of dierent types of images and dierent
applications;
suitable to be performed in an automated way.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 6 / 19
10. General Approach for Image Simplication Connected Operators
Connected Operators - Operators by Reconstruction
Connected operators are morphological lters that process an image by only
merging its at zones.
Either completely remove or entirely preserve a region in the image.
They do not distort the geometrical characteristics (e.g., shape, edges) of the
structures in the image.
Operators by Reconstruction
Closing Closing by rec. Original image Opening by rec. Opening
φB (f ) φB (f ) = Rf [δB (f )]
R
ε f B δ
γR (f ) = Rf [εB (f )] γB (f )
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 7 / 19
11. General Approach for Image Simplication Connected Operators
Connected Operators - Operators by Reconstruction
Connected operators are morphological lters that process an image by only
merging its at zones.
Either completely remove or entirely preserve a region in the image.
They do not distort the geometrical characteristics (e.g., shape, edges) of the
structures in the image.
Operators by Reconstruction
Closing Closing by rec. Original image Opening by rec. Opening
φB (f ) φB (f ) = Rf [δB (f )]
R
ε f B δ
γR (f ) = Rf [εB (f )] γB (f )
Connected operators are suitable for the analysis of VHR images.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 7 / 19
12. General Approach for Image Simplication Connected Operators
Connected Operators - Attribute Filters
Attribute lters are connected operators dened in the mathematical
morphology framework and recently used for the analysis of remote sensing1 .
Based on a measure (attribute) computed on the regions of an image.
Filtering performed by removing the regions that do not fulll a condition (T )
which compares an attribute attr against a reference value λ
(e.g., T = attr ≥ λ).
Main operators:
attribute thinning, γT ;
attribute thickening, φT .
Example: Dierent attributes
Original image Area Standard deviation Moment of inertia Solidity
1 M. Dalla Mura, J. A. Benediktsson, B. Waske, and L. Bruzzone, Morphological attribute proles for the analysis
of very high resolution images, IEEE Transactions on Geoscience and Remote Sensing, vol. 48, no. 10, pp.
IGARSS 20113762, Oct. 2010.
3747 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 8 / 19
13. General Approach for Image Simplication Methodology
Architecture of the Proposed General Approach
X
Parameters Selection
Bank of
Scenario 1 Y
Connected Filters
e. g., attribute thinning,
attribute thickening, ...
Application
Scenario 2
Knowledge
Scene Performance
Knowledge Scenario 3 Assessment
The proposed approach is composed of two modules that perform the operations
of:
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 9 / 19
14. General Approach for Image Simplication Methodology
Architecture of the Proposed General Approach
X
Parameters Selection
Bank of
Scenario 1 Y
Connected Filters
e. g., attribute thinning,
attribute thickening, ...
Application
Scenario 2
Knowledge
Scene Performance
Knowledge Scenario 3 Assessment
The proposed approach is composed of two modules that perform the operations
of:
1 selection of the parameters and operators;
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 9 / 19
15. General Approach for Image Simplication Methodology
Architecture of the Proposed General Approach
X
Parameters Selection
Bank of
Scenario 1 Y
Connected Filters
e. g., attribute thinning,
attribute thickening, ...
Application
Scenario 2
Knowledge
Scene Performance
Knowledge Scenario 3 Assessment
The proposed approach is composed of two modules that perform the operations
of:
1 selection of the parameters and operators;
2 ltering.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 9 / 19
16. General Approach for Image Simplication Methodology
Filter Selection and Operative Scenarios
Operative Scenarios
Information available as prior
knowledge: X
Parameters Selection
1 Scenario 1
%
Scene Knowledge Scenario 1
Bank of
Y
%
Connected Filters
Application Knowledge e. g., attribute thinning,
2 Scenario 2 Application
attribute thickening, ...
%
Scene Knowledge
Knowledge
Scenario 2
Application Knowledge
3 Scenario 3 Scene
Scenario 3
Performance
Knowledge Assessment
Scene Knowledge
Application Knowledge
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 10 / 19
17. General Approach for Image Simplication Methodology
Filter Selection and Operative Scenarios
Operative Scenarios
Information available as prior
knowledge: X
Parameters Selection
1 Scenario 1
%
Scene Knowledge Scenario 1
Bank of
Y
%
Connected Filters
Application Knowledge e. g., attribute thinning,
2 Scenario 2 Application
attribute thickening, ...
%
Scene Knowledge
Knowledge
Scenario 2
Application Knowledge
3 Scenario 3 Scene
Scenario 3
Performance
Knowledge Assessment
Scene Knowledge
Application Knowledge
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 10 / 19
18. General Approach for Image Simplication Methodology
Filter Selection and Operative Scenarios
Operative Scenarios
Information available as prior
knowledge: X
Parameters Selection
1 Scenario 1
%
Scene Knowledge Scenario 1
Bank of
Y
%
Connected Filters
Application Knowledge e. g., attribute thinning,
2 Scenario 2 Application
attribute thickening, ...
%
Scene Knowledge
Knowledge
Scenario 2
Application Knowledge
3 Scenario 3 Scene
Scenario 3
Performance
Knowledge Assessment
Scene Knowledge
Application Knowledge
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 10 / 19
19. General Approach for Image Simplication Methodology
Data set
Quickbird
panchromatic
image of 995×995
pixels, 0.6 m
resolution.
Acquired over a
residential urban
area of Bam, Iran.
Panchromatic image
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 11 / 19
20. General Approach for Image Simplication Methodology
Scenario 1 - Application Knowledge %, Scene Knowledge %
Aim: Generic reduction of the complexity of the image by reducing non
informative components.
Filtering aiming at reducing:
1 Noisy components ⇒ Removing small regions with values signicantly dierent
from their surroundings;
2 Inter-object variability ⇒ Flattening small values dierences in homogeneous
regions.
Suitable to cope with most of the applications.
Eases the interpretation of the scene.
Exploits the contextual relations of the pixels.
Fully automatic suitable for batch processing.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 12 / 19
21. General Approach for Image Simplication Methodology
Scenario 1 - Example
Generic simplication performed by:
γ T φT with area attribute (remove small bright and dark regions);
γ T with T based on relations between the regions1 (merge nested regions).
VHR image
Building rooftop (80×60 pixels). 2789 at regions. Simplied image. 1059 regions.
1 V. Caselles and P. Monasse, Geometric Description of Images as Topographic Maps. Springer, 2010.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 13 / 19
22. General Approach for Image Simplication Methodology
Scenario 2 - Application Knowledge , Scene Knowledge %
Filters parameters selected according to the application.
The translation of the characteristics of the objects of interest from the
concept to the lter parameters.
Example:
Application: building extraction.
Aim of the simplication: enhance rectangular regions
Concept: keep rectangular regions.
Filtering: attribute lter with criterion: {rectangularity 0.5});
Automation
Modeling the range of values of the features that drive the lters with fuzzy
possibilistic functions.
Defuzzify in order to get the values for the lters parameters.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 14 / 19
23. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Buildings (Particulars)
Panchromatic image (f )
Filter by rec.
B
f − γR (f ) (B : disk radius 7 pixels)
Attribute lter γ T (f ) with T = (R 0.3) ∧ (I 0.5) ∧ (50 A 5000)
R: rectangularity; I: moment of inertia; A: area
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 15 / 19
24. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Buildings (Particulars)
Panchromatic image (f )
Filter by rec.
B
f − γR (f ) (B : disk radius 11 pixels)
Attribute lter γ T (f ) with T = (R 0.3) ∧ (I 0.5) ∧ (50 A 5000)
R: rectangularity; I: moment of inertia; A: area
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 15 / 19
25. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Buildings (Particulars)
Panchromatic image (f )
Filter by rec.
B
f − γR (f ) (B : disk radius 15 pixels)
Attribute lter γ T (f ) with T = (R 0.3) ∧ (I 0.5) ∧ (50 A 5000)
R: rectangularity; I: moment of inertia; A: area
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 15 / 19
26. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Buildings (Particulars)
Panchromatic image (f )
Filter by rec.
B
f − γR (f ) (B : disk radius 19 pixels)
Attribute lter γ T (f ) with T = (R 0.3) ∧ (I 0.5) ∧ (50 A 5000)
R: rectangularity; I: moment of inertia; A: area
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 15 / 19
27. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Dark Elongated Structures (Particulars)
Panchromatic image (f )
Filter by rec. C[φB (f ) − f ] (B :
R
disk radius 3 pixels)
Attribute lter φT (f ) with T = (H 10000) ∧ (I 1.0)
H: height; I: moment of inertia
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 16 / 19
28. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Dark Elongated Structures (Particulars)
Panchromatic image (f )
Filter by rec. C[φB (f ) − f ] (B :
R
disk radius 7 pixels)
Attribute lter φT (f ) with T = (H 10000) ∧ (I 1.0)
H: height; I: moment of inertia
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 16 / 19
29. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Dark Elongated Structures (Particulars)
Panchromatic image (f )
Filter by rec. C[φB (f ) − f ] (B :
R
disk radius 11 pixels)
Attribute lter φT (f ) with T = (H 10000) ∧ (I 1.0)
H: height; I: moment of inertia
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 16 / 19
30. General Approach for Image Simplication Methodology
Scenario 2 - Enhancement of Dark Elongated Structures (Particulars)
Panchromatic image (f )
Filter by rec. C[φB (f ) − f ] (B :
R
disk radius 15 pixels)
Attribute lter φT (f ) with T = (H 10000) ∧ (I 1.0)
H: height; I: moment of inertia
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 16 / 19
31. General Approach for Image Simplication Methodology
Scenario 3 - Application Knowledge , Scene Knowledge
Ad hoc parameters selection. If the available information is a set of labeled
samples (i.e., a training set), the reduction of the image complexity
generated by the ltering aims at increasing the separability of the classes.
Performance assessment. The quality of the simplication obtained can be
evaluated on the known samples according to a given criterion.
Automation
Dene a cost function to minimize, representing the tness of the generated
result with the input requirements;
Dene a optimization procedure and a stopping condition.
See on Thurstday: S. Peeters, P. R. Marpu, J. A. Benediktsson, M. Dalla Mura Classication using extended
morphological attribute proles based on dierent feature extraction techniques.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 17 / 19
32. Conclusion and Future Developments
Conclusion and Future Developments
Conclusion
Denition of a novel general approach for image simplication based on
connected lters (in particular attribute lters).
Suitable for
processing many dierent image types;
dierent applications involving image analysis.
Contributions:
identication of three scenarios modeling common dierent operative
conditions;
giving guidelines for the automation of the process;
qualitative evaluation of the proposed approach on a real data set in dierent
scenarios.
Future Developments
Extensively test the approach on dierent type of images and applications.
Improve the automation of the process.
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 18 / 19
33. Conclusion and Future Developments
Thanks for your attention!
IGARSS 2011 (24-29 July) Mauro Dalla Mura dallamura@disi.unitn.it 19 / 19