TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
ROBUST UNMIXING OF HYPERSPECTRAL IMAGES: APPLICATION TO MARS
1.
2. Motivation
Necessity of combining physical models and
statistical techniques in unmixing
Methods of integration of scientific models &
statistical algorithms not always obvious
– Enable machine learning techniques to produce scientifically
meaningful results in unsupervised settings
Meaningfulinformation not always readily
accessible or easily readable
– Representation = Simplification
3. Outline
Background
– Mineral spectral signatures in VNIR
Meaningful features for mineral identification
Data challenges for planetary data
Data processing pipeline
Validation: expert assessment
Comparison with state-of-the-art
Conclusions and future work
4. Mineral spectral signatures
Each mineral has
a distinct spectral
shape (signature)
Discriminative
information mostly
in absorption
band positions
and shapes
Difference can be
subtle
Can create
parameters for
discrimination
5. Mineral spectral signatures
Each mineral has
a distinct spectral
shape (signature)
Discriminative
information mostly
in absorption
band positions
and shapes
Difference can be
subtle
Can create
parameters for
discrimination
7. Spectral features for minerals
Use splines
Select knots so that
– Reconstruction insensitive to
artifacts
8. Spectral features for minerals
Use splines
Select knots so that
– Reconstruction insensitive to
artifacts
– Reconstruction with higher
sensitivity in diagnostic areas
9. Spectral features for minerals
Use splines
Select knots so that
– Reconstruction insensitive to
artifacts
– Reconstruction with higher
sensitivity in diagnostic areas
– Reconstruction sharper
(green) for vibrational bands
and smoother (red) for
electronic transition bands
B-splinecoefficients as
feature vector
10. hyperspectral data challenges
Cloud has curved
boundaries and is non
convex
Some dimensions
uninformative
No apparent clusters, high
density
Noise creates outliers
Most unique spectra =
extreme points or “corners”
or “image endmembers”
11. Objectives for spectral unmixing
Separation of spectral pixels in families (image
segmentation)
Sensitivity to subtle changes in spectral absorption
positions and shapes (mineral sub-families)
Sensitivity to small (spatial) outcrops
Robustness with respect to noise
Useful visualization
14. Operation modes
100 pixels
50 pixels
Two capabilities:
– Select areas based on parameter maps (user version)
– Divide the image in sections (pipeline version)
Operate on each area independently
16. Dimensionality reduction: issues
Intrinsic dimensionality of data is low: benefit from
dimensionality reduction.
From movie:
– Need nonlinear transform.
– Need to preserve local geometry
– Need to highlight natural clusters
Reduce dimensionality to 2 – 3 for visualization
17. Dimensionality reduction
High-D Feature Space
Low-D Space
x1 , . . . , xn(known) as vertices y1 , . . . , yn (unknown) as
of a graph
vertices of a graph
Edge weights proportional to Edge weights fixed
spectral dissimilarities and spatial
adjacency
x1 x3
y1 y3
x2 y2
yn
xn
23. Graph partitioning as clustering
Cluster points in the transformed
space to take advantage of separated
sections
The geometry is nonlinear: need
clustering on curved structure
Consider the set of vertices yi of the
graph and the edge weights qij (yi , yj )
Clustering is equivalent to partitioning
graph into disjoint subsets.
– can be done by spectral clustering
because CNE creates several
connected components
24. Image segmentation
Original Segmentation
Clustering = mineral image
map
family mapping =
image
segmentation
26. Local endmember detection
1
2
4
3
The clusters in the original space are roughly convex
Locally to a cluster can assume linear mixing
approximate the data cloud with a conic or convex
combination of a small number of “endmembers”
Have a way to extrapolate endmembers if the data
does not support clear detections
27. Robust Nonneg. Matrix Factorization
minimize ϕ(Y − W H) + 2
λ||DW ||F
subject to W,H ≥ 0, 1T H = 1 T
W ∈ Rm×k , H ∈ Rk×n
k is the number of local endmembers
ϕ is a robust estimator
D imposes smoothness and corrects MNF “problems”
Solve with alternating projected gradient
Zymnis 2009, Parente 2009, Parente 2011
29. Spectral pruning
Features for Features for
spectrum 1
spectrum 2
Cross-correlate
Spectrum 1 is any local
feature vectors
endmember candidate
Spectrum 2 is either a
local endmember or an
estimate of the baricenter
of the cloud
If the score is higher than
a threshold Spectrum 1 is
pruned
Score
30. Validation
Martian image analysis lacks ground truth
Simulation of the complete hyperspectral image
formation process (Parente et al. 2010)
– Soil mixing, atmosphere, instrument response, noise
Comparison with manual expert assessment
– the expert can extract the complete spectral variability (3E12)
– the expert can only extract partial spectral variability (94F6)
– The expert cannot extract spectral variability
Self-consistency: comparison with state-of the
art (partial)
31. Validation: 3E12
Different mineral
families evident
from RGB
Low noise
Good spectral
variability
48. Comparison with state of the art
Current unmixing algorithms:
– require convexity
– developed for earth
environmental conditions are known
ground truth is available
– donʼt consider impulsive noise
– some require linear assumptions
Nonlinear unmixing not yet mature
Not able to discriminate subtle spectral differences
49. Comparison with other algorithms
The proposed algorithm is The SMACC algorithm is
insensitive to noise and extremely sensitive to noise
picks up more surface
components
53. Conclusions
Presented a novel method for unmixing
The algorithm effectively captures the image spectral
variability, down to subtle differences, is robust to noise and
outperforms current state-of-the-art algorithms
Can be applied to any hyperspectral dataset
Produces segmentation and endmember maps
We proposed this technique to the CRISM and M3 teams
as the “official” data summarization tool for their processing
pipelines.
54. Future work
Include a physical unmixing layer: use radiative
transfer theory
Provide mechanism to tag “virtual” endmembers
Complete validation process with expert feedback
55. References
L. van deer Maaten and G. Hinton, (2008). Visualizing data using t-
SNE, Journal of Machine Learning, 9, pp. 2579-2605.
A. Ng, M. Jordan and Y. Weiss, (2001). On spectral clustering:
Analysis and an algorithm, NIPS.
M. Parente , J.T. Clark, A. Brown and J.L. Bishop (2010). End-to-
end simulation of the image generation process for CRISM
spectrometer data, IEEE Transactions on Geoscience and Remote
Sensing.
M. Parente, (2011). Summarization of hyperspectral images:
application to Mars, IEEE Transactions on Geoscience and
Remote Sensing, (in review).
M. Parente, J. L. Bishop and J. F. Bell III, (2009), Spectral unmixing
and anomaly detection for mineral identification in Pancam images
of Gusev soils, Icarus, Vol 203, N. 2, p. 421-436.
57. Publications based on project
Parente M. and A. Plaza (2010), Survey of geometric and statistical unmixing algorithms for
hyperspectral images, IEEE 2nd WHISPERS (Workshop on hyperspectral image and signal
processing: evolution of remote sensing) Conf. June 14-16, Reykjavyk, Iceland (invited keynote
presentation for special session on “Geometric vs. statistical unmixing algorithms”).
M. Parente Spectral unmixing using nonnegative basis learning: comparison of geometrical and
statistical endmember extraction algorithms. (invited paper) Space Exploration Technologies,
edited by Wolfgang Fink Proc. of SPIE Vol. 6960, 69600P, (2008). doi: 10.1117/12.777895
M. Parente Exploratory data analysis of planetary datasets – new development, (invited talk) Jet
Propulsion Laboratory, Pasadena CA, December 4 2008.
Parente M., Clark J.T., Brown A.J., and Bishop J.L.. (2009). Simulation of the image generation
process for CRISM spectrometer data. IEEE WHISPERS (Workshop on hyperspectral image
and signal processing: evolution of remote sensing) Conf. Aug 26-28 Grenoble, France. (Best
paper award)
Bishop J. L., Noe Dobrea E. Z., McKeown N. K., Parente M., Ehlmann B. L., Michalski J. R.,
Milliken R. E., Poulet F., Swayze G. A., Mustard J. F., Murchie S. L., and Bibring J.-., P. (2008)
Phyllosilicate diversity and past aqueous activity revealed at Mawrth Vallis, Mars. Science 321,
DOI: 10.1126/science.1159699, pp. 830-833.
Parente, M. and J.L. Bishop, (2010). Extracting endmember spectra from CRISM images:
comparison of new Direx image transform technique with MNF, Lunar Planet Science Conf, XLI
abstr. #2633.
59. MRO-CRISM: VNIR Spectra Can Characterize
Small Deposits on Mars
Examples of surface features at different CRISM spatial
resolutions
• Global Mode: 70 channels
• Targeted Mode: 544 channels
OMEGA CRISM multispectral survey (100-200 CRISM targeted hyperspectral
(300-1000 m/pixel, 13 nm/ch.) m/pix, 70 ch.) discovers small (15-38 m/pixel, 6.55 nm/ch)
discovers large deposits deposits
characterizes deposits
60. CRISM Noise sources
1. Vertical striping due to
miscalibration of pixel sensors
(red arrows).
2. Pixels with elevated bias or
abnormal dark ("bad" pixels)
create stripe segments (cyan)
Both artifacts create spikes
in the spectral domain
60/40
61. Noise removal with CIRRUS
Original
Cleaned
Original
CIRRUS (CRISM Iterative
Recognition and Removal
of Unwanted Spiking)
Cleaned
(Parente 2008)
CIRRUS currently in use in
CRISM processing pipeline
62. Comparison with PCA
Proposed approach (3D)
PCA (first 3 PCs)
Natural clusters well Natural clusters not
separated
evident
Between-clusters, similar points can
different spectra
differ in norm
Within-cluster, similar 1st PC illumination
spectra
gradient
63. Comparison with other techniques
Proposed approach (3D)
PCA (first 3 PCs)
LLE (3D)
Natural clusters well Natural clusters not Natural clusters not
separated
evident
evident
Between-clusters, similar points can Some endmembers
different spectra
differ in norm
evident
Within-cluster, similar 1st PC illumination Clustering particularly
spectra
gradient
hard
69. Clustering performance comparison
Original Proposed K-means in K-means with Hierarchical in Hierarchical in
image
approach
original correlation in original space
3-D space
space
original space
70. K-Eigenvector Clustering
(Ng et al. 2001)
1. Construct matrix of normalized weights Aʼ
2. Decomposition: Find the eigenvectors of Aʼ
corresponding to the k largest eigenvalues.
These form the the columns of the new matrix X.
3. Form the matrix Y
– Renormalize each of Xʼs rows to have unit length
– Y |
– Treat each row of Y as a point in
3. Cluster into k clusters via k-means
4. Final Cluster Assignment
– Assign point to cluster j iff row i of Y was assigned to cluster j
k can be found by maximum spread between eigenvalues
71. Validation
This software is undergoing extensive validation
ID Solicitation
aimed at confirming that the proposed method
can be used pervasively and reliably in the
summarization of the whole CRISM database.
The validation process starts with requesting
Processing
from the community image IDʼs with manually
selected endmembers.
An automated pipeline is in place that sends
back via email the spectra retrieved by the
Feedback
algorithm to each author of manual analysis.
Upon receiving feedback on dissimilarities and
quality of the detections the pipeline will
Validation
calculate validation statistics and will send them
statistics to the team for review.
After validation the production stage will begin.