The document proposes a new approach for hyperspectral image classification that incorporates spectral unmixing. It aims to improve classification map spatial resolution by (1) using unsupervised clustering to identify classes from hyperspectral data, (2) computing abundance maps through spectral unmixing, (3) creating a finer classification map, and (4) applying spatial regularization. Experiments on real data show the approach increases classification accuracy and spatial resolution compared to traditional techniques.

1. Unsupervised classification and spectral unmixing for sub-pixel labelling
A.Villa , ,† , J.Chanussot , J.A. Benediktsson , C.Jutten
GIPSA-lab, Signal & Image Dept., Grenoble Institute of Technology, France.
Faculty of Electrical and Computer Engineering, University of Iceland, Iceland.
†
Aresys, Politecnico di Milano, Italy.
IEEE IGARSS 2011
Vancouver, Canada - 2011

2. A new approach to classification Experiments Conclusions
Hyperspectral Images
Widely used in remote sensing:
λ
√
Wide spectral range and large
number of wavelengths
- Trees
- Grass
√
Very high spectral resolution
VIS NIR
0.4 μm 2.4 μm × Tradeoff between spectral and
spatial resolution
Jocelyn Chanussot Gipsa-Lab 2 / 21

3. A new approach to classification Experiments Conclusions
Challenges
Low spatial resolution → appearance of mixed pixels
• Common in hyperspectral images
Pure pixel: • Traditional classifiers inadequate,
100% grass
partially addressed by mixed pixel
techniques
Mixed pixel: • Critical for land cover mapping
70% metal sheet
30% grass
Joint use (full + mixed techniques) desirable, but little investigated
[Wang and Jia, 2010].
Jocelyn Chanussot Gipsa-Lab 3 / 21

4. A new approach to classification Experiments Conclusions
Challenges
Low spatial resolution → appearance of mixed pixels
• Common in hyperspectral images
Pure pixel: • Traditional classifiers inadequate,
100% grass
partially addressed by mixed pixel
techniques
Mixed pixel: • Critical for land cover mapping
70% metal sheet
30% grass
Incorporation of spectral unmixing in the classification process:
• Does it provide accuracy improvement?
• Is it possible to improve the classification map spatial resolution?
Jocelyn Chanussot Gipsa-Lab 3 / 21

5. A new approach to classification Experiments Conclusions
1 A new approach to classification
2 Experiments
3 Conclusions
Jocelyn Chanussot Gipsa-Lab 4 / 21

6. A new approach to classification Experiments Conclusions
Context
Traditional techniques neglect sub-pixel and spatial information
Additional information provided by unmixing not fully exploited
0.6
0.9
Pure pixel:
100% grass 1 0.9 0.8
0.6 1
Mixed pixel: 1 1 0.8
70% metal sheet
30% grass
0.9 0.6 1
Original image Classification Unmixing Finer resolution?
How to jointly use full and mixed pixel techniques?
Jocelyn Chanussot Gipsa-Lab 5 / 21

7. A new approach to classification Experiments Conclusions
Proposed Approach
Low resolution
hyperpspectral data
Unmixing
Classes Abundances
identification maps
Classification
"Upsampled"
classification map
Spatial regularization
Final map
Jocelyn Chanussot Gipsa-Lab 6 / 21

8. A new approach to classification Experiments Conclusions
Proposed Approach
1. Abundances fractions are computed from a HSI
Step 1:
Low resolution
hyperpspectral data
Pure pixel:
100% grass
Step 1
Step 2 Mixed pixel:
Classes Abundances 70% metal sheet
identification maps 30% grass
Step 2:
"Upsampled" 0.6
classification map
0.9
1 0.9 0.8
Spatial regularization
0.6 1
1 1 0.8
Final map
0.9 0.6 1
Jocelyn Chanussot Gipsa-Lab 7 / 21

9. A new approach to classification Experiments Conclusions
The proposed approach
M = Mixed pixel
M Proposed method
M M The abundances computation is in two
steps, to take the spatial information into
account:
1. Pixels with an abundance over a
certain threshold are considered ’pure’
M M
2. Abundances of ’mixed’ pixels are
M M computed by selecting as endmembers
pixels spatially close
M
Jocelyn Chanussot Gipsa-Lab 8 / 21

10. A new approach to classification Experiments Conclusions
The proposed approach
M = Mixed pixel
Proposed method
The abundances computation is in two
steps, to take the spatial information into
account:
1. Pixels with an abundance over a
certain threshold are considered ’pure’
2. Abundances of ’mixed’ pixels are
computed by selecting as endmembers
pixels spatially close
Jocelyn Chanussot Gipsa-Lab 8 / 21

11. A new approach to classification Experiments Conclusions
Proposed Approach
2. Creation of a finer classification map
Step 2:
0.6
Low resolution
hyperpspectral data
0.9
1 0.9 0.8
0.6 1
Step 2 1 1 0.8
Classes Abundances
identification maps
0.9 0.6 1
Step 3
"Upsampled"
Step 3:
classification map
Spatial regularization
Final map
Jocelyn Chanussot Gipsa-Lab 9 / 21

12. A new approach to classification Experiments Conclusions
Proposed Approach
3. Final spatial regularization
Step 3:
Low resolution
hyperpspectral data
Classes Abundances
identification maps
Step 3
"Upsampled"
Step 4:
classification map
Step 4
Spatial regularization
Final map
Jocelyn Chanussot Gipsa-Lab 10 / 21

13. A new approach to classification Experiments Conclusions
Spatial regularization
Criterion: minimization of the total perimeter of the connected areas (e.g.,
belonging to the same class)
M
0,5 0,3
0,2 M
0,5 0,3
0,2
0,7
M
0,3 M
0,6
0,4
0,7
M
0,3 M
0,6
0,4
0,9
0,8 0,9
0,6 0,9
0,8 0,9
0,6
M 0,4
M
0,1
0,1
0,2 M 0,4
M
0,1
0,1
0,2
0,9 0,9 0,9 0,9
M
0,1 M
0,7
0,1
0,3 M
0,1 M
0,7
0,1
0,3
M
0,9
0,1 M
0,9
0,1
Criterion not satisfied Criterion satisfied
Jocelyn Chanussot Gipsa-Lab 11 / 21

14. A new approach to classification Experiments Conclusions
Spectral unmixing based approach [Villa Novelties introduced:
et al., 2010]
1. Retrieve classes with unsupervised
1. VCA for class retrieval clustering
(→ more robust to outliers)
2. FCLS for abundance determination
2. Include spatial information
(→ use more accurate
3. Simulated Annealing for spatial
endmembers)
regularization
Jocelyn Chanussot Gipsa-Lab 12 / 21

15. A new approach to classification Experiments Conclusions
VCA vs. K-MEANS
6000 6000
5000 5000
4000 4000
3000 3000
2000 2000
1000 1000
0 0
4000 4000
2000 2000
6000 4000 5000 6000
0 2000 3000 4000 5000 0 1000 2000 3000
0 1000 0
VCA K-MEANS
Jocelyn Chanussot Gipsa-Lab 13 / 21

16. A new approach to classification Experiments Conclusions
1 A new approach to classification
2 Experiments
3 Conclusions
Jocelyn Chanussot Gipsa-Lab 14 / 21

17. A new approach to classification Experiments Conclusions
How to verify the results?
Decrease original
resolution
Final map
Proposed
(sub-pixel precision)
approach
Jocelyn Chanussot Gipsa-Lab 15 / 21

18. A new approach to classification Experiments Conclusions
Experiments on real data
ROSIS University data set AISA data set
• Classification of a metal sheet roof • 400×500 pixels area, six classes of
(120×90 pixels) interest
• 1.3 m spatial resolution, 103 • 6 m spatial resolution, 252 spectral
spectral bands. bands
• Spatial resolution of the original • Spatial resolution of the original data
data degraded of a factor 3 degraded of a factor 5
Jocelyn Chanussot Gipsa-Lab 16 / 21

19. A new approach to classification Experiments Conclusions
Real data sets
ROSIS data set:
Original Image K-means (93.75%) VCA+SU (96.95%) KM+SU (95.89%)
Jocelyn Chanussot Gipsa-Lab 17 / 21

20. A new approach to classification Experiments Conclusions
Real data set
AISA data set:
K-means (51.61%) VCA+SU (59.69%) KM+SU (75.72%)
Jocelyn Chanussot Gipsa-Lab 18 / 21

21. A new approach to classification Experiments Conclusions
1 A new approach to classification
2 Experiments
3 Conclusions
Jocelyn Chanussot Gipsa-Lab 19 / 21

22. A new approach to classification Experiments Conclusions
Conclusions and Perspectives
New method to improve spatial resolution of thematic maps:
• Unsupervised clustering to define classes
• Integration of spatial information to locally model abundances
• Simulated Annealing proposed for spatial regularization
Clustering less sensitive to extreme pixels, VCA better in highly mixed scenarios
Next step: Incorporate spectral variability of the classes
Jocelyn Chanussot Gipsa-Lab 20 / 21

23. A new approach to classification Experiments Conclusions
Unsupervised classification and spectral unmixing for sub-pixel labelling
A.Villa , ,† , J.Chanussot , J.A. Benediktsson , C.Jutten
GIPSA-lab, Signal & Image Dept., Grenoble Institute of Technology, France.
Faculty of Electrical and Computer Engineering, University of Iceland, Iceland.
†
Aresys, Politecnico di Milano, Italy.
IEEE IGARSS 2011
Vancouver, Canada - 2011
Jocelyn Chanussot Gipsa-Lab 21 / 21

24. A new approach to classification Experiments Conclusions
Challenges
Hyperspectral images issues:
1 Curse of dimensionality
2 Exploitation of contextual information
3 Presence of mixed pixels
• Common in hyperspectral images
Pure pixel:
100% grass
• Traditional classifiers inadequate
Mixed pixel:
• Usually not considered for
70% metal sheet classification!
30% grass
Jocelyn Chanussot Gipsa-Lab 22 / 21

25. A new approach to classification Experiments Conclusions
Context
Traditional approaches to image analysis are full pixel and mixed pixel techniques
• Full pixel techniques are traditional classification algorithms
• Mixed pixel techniques are spectral unmixing, soft classification, . . .
Joint use is desirable, but little investigated [Wang and Jia, 2010].
Incorporation of spectral unmixing in the classification process:
• Does it provide accuracy improvement?
• Is it possible to improve the classification map spatial resolution?
Jocelyn Chanussot Gipsa-Lab 23 / 21

26. A new approach to classification Experiments Conclusions
Linear Spectral Unmixing
Abundances estimation through spectral unmixing:
• Goal: find extreme pixels (endmembers) that can be used to "unmix" other pixels.
• Each "mixed" pixel is a combination of endmember fractional abundances.
Jocelyn Chanussot Gipsa-Lab 24 / 21

27. A new approach to classification Experiments Conclusions
Linear Spectral Unmixing
Abundances estimation through spectral unmixing:
• Goal: find extreme pixels (endmembers) that can be used to "unmix" other pixels.
• Each "mixed" pixel is a combination of endmember fractional abundances.
Jocelyn Chanussot Gipsa-Lab 24 / 21

28. A new approach to classification Experiments Conclusions
Context
Traditional techniques neglect information
Additional information provided by unmixing not fully exploited
0.6
0.9
Pure pixel:
100% grass 1 0.9 0.8
0.6 1
Mixed pixel: 1 1 0.8
70% metal sheet
30% grass
0.9 0.6 1
Original image Classification Unmixing Finer resolution?
How to jointly use full and mixed pixel techniques?
Jocelyn Chanussot Gipsa-Lab 25 / 21

29. A new approach to classification Experiments Conclusions
The proposed approach
M = Mixed pixel
Proposed method
M We propose a technique in four steps:
M M 1. Preliminary classification with
probabilistic classifier (SVM)
2. Choose suitable endmember
candidates and perform unmixing
M M 3. Split every pixel into n sub-pixels, and
assign them to a class
M M
4. Perform spatial regularization in order
M to correctly locate sub-pixels
A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21

30. A new approach to classification Experiments Conclusions
The proposed approach
M = Mixed pixel
Proposed method
We propose a technique in four steps:
1. Preliminary classification with
probabilistic classifier (SVM)
2. Choose suitable endmember
candidates and perform unmixing
3. Split every pixel into n sub-pixels, and
assign them to a class
4. Perform spatial regularization in order
to correctly locate sub-pixels
A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21

31. A new approach to classification Experiments Conclusions
The proposed approach
Proposed method
0,5 0,3
0,2 M We propose a technique in four steps:
0,7
M
0,6 1. Preliminary classification with
M
0,3 0,4 probabilistic classifier (SVM)
2. Choose suitable endmember
candidates and perform unmixing
0,9
0,8 0,9
0,6
M 0,4
M
0,1
0,1
0,2 3. Split every pixel into n sub-pixels, and
assign them to a class
0,9 0,9
M
0,1 M
0,7
0,1
0,3 4. Perform spatial regularization in order
M
0,9
0,1
to correctly locate sub-pixels
A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21

32. A new approach to classification Experiments Conclusions
The proposed approach
Proposed method
0,5 0,3
0,2 M We propose a technique in four steps:
0,7
M
0,6 1. Preliminary classification with
M
0,3 0,4 probabilistic classifier (SVM)
2. Choose suitable endmember
candidates and perform unmixing
0,9
0,8 0,9
0,6
M 0,4
M
0,1
0,1
0,2 3. Split every pixel into n sub-pixels, and
assign them to a class
0,9 0,9
M
0,1 M
0,7
0,1
0,3 4. Perform spatial regularization in order
M
0,9
0,1
to correctly locate sub-pixels
A. Villa, J. Chanussot, J.A. Benediktsson, C. Jutten, "Spectral Unmixing for the Classification of Hyperspectral Images
at a Finer Spatial Resolution", IEEE J. Sel. Topics Sign. Proc., vol. 5, n. 3, 2011
Jocelyn Chanussot Gipsa-Lab 26 / 21

33. A new approach to classification Experiments Conclusions
Simulated Annealing
Minimize a given Cost Function introducing
random perturbations:
• decreases of the CF are always accepted
• increases of the CF accepted with a probability
inversely proportional to the degradation
• probability of ’bad solutions’ decreases as the
search continues
Simulated Annealing optimization avoids local minima leading to global optimal
solution
Jocelyn Chanussot Gipsa-Lab 27 / 21

34. A new approach to classification Experiments Conclusions
Simulated Annealing
Cost function to be minimized: total perimeter of the connected areas (e.g.,
belonging to the same class)
M
0,5 0,3
0,2 M
0,5 0,3
0,2
0,7
M
0,3 M
0,6
0,4
0,7
M
0,3 M
0,6
0,4
0,9
0,8 0,9
0,6 0,9
0,8 0,9
0,6
M 0,4
M
0,1
0,2
0,1 M 0,4
M
0,1
0,2
0,1
0,9 0,9 0,9 0,9
M
0,1 M
0,7
0,1
0,3 M
0,1 M
0,7
0,1
0,3
M
0,9
0,1 M
0,9
0,1
Cost function not optimized Cost function optimized
Jocelyn Chanussot Gipsa-Lab 28 / 21

35. A new approach to classification Experiments Conclusions
Simulated Annealing
M
0,5 0,3
0,2 M
0,5 0,3
0,2
0,7
M
0,3 M
0,6
0,4
0,7
M
0,3 M
0,6
0,4
0,9
0,8 0,9
0,6 0,9
0,8 0,9
0,6
M 0,4
M
0,1
0,2
0,1 M 0,4
M
0,1
0,2
0,1
0,9 0,9 0,9 0,9
M
0,1 M
0,7
0,1
0,3 M
0,1 M
0,7
0,1
0,3
M
0,9
0,1 M
0,9
0,1
Initial condition Iteration 1
M
0,5 0,3
0,2 M
0,5 0,3
0,2
0,7
M
0,3 M
0,6
0,4
0,7
M
0,3 M
0,6
0,4
0,9
0,8 0,9
0,6 0,9
0,8 0,9
0,6
M 0,4
M
0,1
0,2
0,1 M 0,4
M
0,1
0,2
0,1
0,9 0,9 0,9 0,9
M
0,1 M
0,7
0,1
0,3 M
0,1 M
0,7
0,1
0,3
M
0,9
0,1 M
0,9
0,1
Iteration n Final result
Jocelyn Chanussot Gipsa-Lab 28 / 21

36. A new approach to classification Experiments Conclusions
Simulated Annealing
Cost function to be minimized: total perimeter of the connected areas (e.g.,
belonging to the same class)
M
0,5 0,3
0,2 M
0,5 0,3
0,2
0,7
M
0,3 M
0,6
0,4
0,7
M
0,3 M
0,6
0,4
0,9
0,8 0,9
0,6 0,9
0,8 0,9
0,6
M 0,4
M
0,1
0,1
0,2 M 0,4
M
0,1
0,1
0,2
0,9 0,9 0,9 0,9
M
0,1 M
0,7
0,1
0,3 M
0,1 M
0,7
0,1
0,3
M
0,9
0,1 M
0,9
0,1
Cost function not optimized Minimum cost function
Jocelyn Chanussot Gipsa-Lab 28 / 21

37. A new approach to classification Experiments Conclusions
Experiment on real data
AVIRIS Indian Pine data set
• (145×145 pixels, 220 bands), 16 classes of interest
• Spatial resolution of the original data degraded of a factor 2
• 10% of the labelled samples used as training set
AVIRIS Hekla data set
• (180×180 pixels, 157 bands), 9 classes of interest
• Spatial resolution of the original data degraded of a factor 2
• 15% of the labelled samples used as training set
Comparison with SVM 1vs1, RBF kernel
20 20
40
40
60
60
80
80 100
120
100
140
120
160
140
180
20 40 60 80 100 120 140 20 40 60 80 100 120 140 160 180
Indian Pine GT Hekla GT
Jocelyn Chanussot Gipsa-Lab 29 / 21

38. A new approach to classification Experiments Conclusions
Evaluation of the results
Jocelyn Chanussot Gipsa-Lab 30 / 21

39. A new approach to classification Experiments Conclusions
Evaluation of the results
Jocelyn Chanussot Gipsa-Lab 30 / 21

40. A new approach to classification Experiments Conclusions
Evaluation of the results
Jocelyn Chanussot Gipsa-Lab 30 / 21

41. A new approach to classification Experiments Conclusions
AVIRIS Indian Pine
20 10
40 20
60 30
80 40
100 50
120 60
140 70
20 40 60 80 100 120 140 10 20 30 40 50 60 70
Ground truth SVM map (OA = 72.31%)
20 20
40 40
60 60
80 80
100 100
120 120
140 140
20 40 60 80 100 120 140 20 40 60 80 100 120 140
Proposed, before SA (OA = 89.82%) Proposed, final map (OA = 91.10%)
Jocelyn Chanussot Gipsa-Lab 31 / 21

42. A new approach to classification Experiments Conclusions
AVIRIS Hekla
10 10
20 20
30 30
40 40
50 50
60 60
70 70
80 80
90 90
10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90
Low res. GT SVM map (OA = 69.19%)
20 20
40 40
60 60
80 80
100 100
120 120
140 140
160 160
180 180
20 40 60 80 100 120 140 160 180 20 40 60 80 100 120 140 160 180
Proposed, before SA (OA = 78.90%) Proposed, final map (OA = 81.71%)
Jocelyn Chanussot Gipsa-Lab 32 / 21

43. A new approach to classification Experiments Conclusions
A robust method
AVIRIS Indian Pine (Complete) AVIRIS Indian Pine (full data set)
90 90
85 85 Proposed method
Overall Accuracy (%)
Overall Accuracy (%)
Traditional SVM
Traditional SVM
Proposed Method
80 80
75 75
70 70
0.6 0.65 0.7 0.75 0.8 5 10 15 20
Treshold Pure Pixels Number of ’candidates endmember’
AVIRIS Hekla AVIRIS Hekla
82 82
80 80
78 78
Overall Accuracy (%)
Overall Accuracy (%)
Proposed Method
76 76
Traditional SVM
74 74
72 72
70 70
68 68
0.6 0.65 0.7 0.75 0.8 5 10 15 20
Treshold Pure Pixels Number of ’candidates endmember’
Jocelyn Chanussot Gipsa-Lab 33 / 21

44. A new approach to classification Experiments Conclusions
Conclusions and Perspectives
New method to improve spatial resolution of thematic maps:
• Spectral Unmixing considered to handle mixed pixels and abundances determination
• Simulated Annealing proposed for spatial regularization
Better definition of spatial structures with respect to full pixel classifiers when the
image contains mixed pixels
Large quantitative improvement
Jocelyn Chanussot Gipsa-Lab 34 / 21