354 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSIN G, VOL. 5, NO. 2, A
PRIL 2012
Hyperspectral Unmixing Overview: Geometrical,
Statistical, and Sparse Regression-Based Approaches
José M. Bioucas-Dias, Member, IEEE, Antonio Plaza, Senior Member, IEEE, Nicolas Dobigeon, Member, IEEE,
Mario Parente, Member, IEEE,QianDu, Senior Member, IEEE, Paul Gader, Fellow, IEEE,and
Jocelyn Chanussot, Fellow, IEEE
Abstract—Imaging spectrometers measure electromagnetic
energy scattered in their instantaneous field view in hundreds or
thousands of spectral channels with higher spectral resolution than
multispectral cameras. Imaging spectrometers are therefore often
referred to as hyperspectral cameras (HSCs). H igher spectral res-
olution enables material identification via spectroscopic analysis,
which facilitates countless applications that require identifying
materials in scenarios unsuitable for classical spectroscopic anal-
ysis. Due to low spatial resolution of HSCs, microscopic material
mixing, and multiple scattering, spectra measured by HSCs are
mixtures of spectra of materials in a scene. Thus, accurate estima-
tion requires unmixing. Pixels are assumed to be mixtures of a few
materials, called endmembers. Unmixing involves estimating all or
some of: the number of endmembers, their spectral signatures, and
their abundances at each pixel. Unmixing is a challenging, ill-posed
inverse p roblem because of model inaccuracies, observation noise,
environmental conditions, endmember variability, and data set
size. Researchers have devised and investigated many models
searching for robust, stable, tractable, and accurate unmixing
algorithms. This paper presents an overview of unmixing methods
from the time of Keshava and M ustard’s unmixing tutorial [1] to
the present. Mixing models are first discussed. Signal-subspace,
geometrical, statistical, sparsity-based, and spatial-contextual
unmixing algorithms are described. Mathematical problems and
potential solutions are described. Algorithm characteristics are
illustrated experimentally.
Index Terms—Hyperspectral imaging, hyperspectral remote
sensing, image analysis, image processing, imaging spectroscopy,
inverse problems, linear mixture, machine learning algorithms,
nonlinear mixtures, pattern recognition, remote sensing, sparsity,
spectroscopy, unmixing.
Manuscript received February 27, 2012; accepted March 22, 2012. Date of
publication May 15, 2012; date of current version May 23, 2012.
J. M . Bioucas-Dias is with the Instituto de Telecomunicações, Instituto
Superior Técnico, Technical University of Lisbon, Lisbon, Portugal (e-mail:
bioucas@lx.it.pt).
A. Plaza is with the Hyperspectral Computing Laboratory, Department of
Technology of Computers and Communications, University of Extremadura,
10003 Caceres, Spain (e-m ail: aplaza@unex.es).
N. Dobigeon is with the University of Toulouse, IRIT/INP-EN-
SEEIHT/TeSA, Toulouse, F rance (e-mail: Nicolas.Dobigeon@enseeiht.fr).
M. Parente is with the Department of Electrical and Com puter Engineering,
University of Massachus etts Amherst, Amherst, MA 01 0 03 USA (e-mail:
mparente@ecs.umass.edu).
Q. Du is wit h the Department of Electrical and Computer Engineering, Mis-
sissippi State Univ ersity, Mississippi State, MS 39762 USA (e-m ail: du@ece.
msstate.edu).
P. Gader is with the Department of Com puter and Information Science and
Engineering, University of Florida, Gainesville, FL 32611 USA and GIPSA-
Lab, Grenoble In stitute of Technology, Grenoble, France (corresponding author,
e-mail: pgader@cise.ufl.edu).
J. Chanussot is with the GIPSA-Lab, Greno ble Institute of Technology,
Grenoble, France (e-mail: jocelyn.chanussot@gipsa-lab.grenoble-inp.fr).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Dig
ital Object Identifi er 10.1109/JSTARS.2012.2194696
Fig. 1. Hyperspectral imaging concep t.
I. INTRODUCTION
H
YPERSPECTRAL cameras [1]–[11] contribute signifi-
cantly to earth observation and remote sensing [12], [13].
Their potential motivates the development of small, commer-
cial, high spatial and spectral resolution instruments. They have
also been used in food safety [14]–[17], pharmaceutical process
monitoring and quality control [18]–[22], and biomedical, in-
dustrial, and biometric, and forensic applications [23]–[27].
HSCs can be built to function in many regions of th e electro-
magnetic spectrum. The focus here is on those covering the vis-
ible, near-infrared, and shortwave infrared spectral bands (in the
range 0.3
to 2.5 [5]). Disregard ing atmospheric effects,
the signal recorded by an HSC at a pixel is a mixture of light
scattered by substances located in the fieldofview[3].Fig.1
illustrates the measured data. They are organized into planes
forming a data cube. Each plane corresponds to radiance ac-
quired over a spectral band for all pixels. Each spectral vector
corresponds to the radiance acquiredatagivenlocationforall
spectral bands.
A. Linear and Nonlinear Mixing Models
Hyper
spectral unmixing (HU) refers to any process that sep-
arat
es the pixel spectra from a hyperspectral image into a col-
1939-1404/$31.00 © 2012 IEEE
BIOUCAS-DIAS et al.: HYPERSPECTRAL UNMIXING OVERVIEW: GEOMETRICAL, STATISTICAL, AND SPARSE REGR
ESSION-BASED APPROACHES 355
lection of constituent spectra, or spectral signatures, called end-
members and a set of fractional abundances, one set per pixel.
The endmembers are generally assumed to represent the pure
materials present in the image an d the set of abundances, or
simply abundances, at each pixel to represent the percentage of
each endmember that is present in the pixel.
There are a n um ber of subtleties in this definition. Firs t, the
notion of a pure material can be subjective and prob lem de-
pendent. For example, suppose a hyperspectral image contains
spectra measured from bricks laid on the gro und , the mortar
between the bricks, an d two types of plants t hat are growing
through cracks in the brick. One m ay suppose then that there
are four endmembers. However, if the percentage of area that
is covered by the mortar is very small then we may not want
to have an endmember for mortar. We may just want an end-
member for “brick”. It depends on if we have a need to directly
measure t he proportion of mortar present. If we have need to
measure the m ortar, then we may not care to distinguish be-
tween the plants since th ey may have similar sig natu res. On t he
other hand, suppose that one type of plant is desirable and the
other is an invasive plant that needs to be removed. Then we
may want two plant endmembers. Furthermore, one may only
be interested in the chlorophyll present in the entire scene. Ob-
viously, this discussion can be continued ad nauseum butitis
clear that the definition of the endmembers can depend upon the
application.
The second subtlety is with the p roportions. Most researchers
assume that a proportion represents the percentage of material
associated with an endmember present in the part of the scene
imaged by a particular p ixel. Indeed, Hapke [28] states that the
abundances in a linear mixture represent the relative area of
the corresponding en dm emb er in an imaged region. Lab experi-
ments conducted by some of the authors have confirmed this in a
laboratory setting. However, in the nonlinear case, the situatio n
is not as straightfor ward. F or example, calibration objects can
sometimes be used to m ap hyperspectral measurements to re-
flectance, or at least to relative reflectance. Therefore, the coor-
dinates of the endmembers are approximations to the reflectance
of the m aterial, which we may assume for the sake of argu-
ment to be accurate. The reflectance is usually not a linear func-
tion of the mass of the material nor is it a linear function of
the cross-s ectional area of the material. A highly reflective, yet
small object may dominate a much larg er but dark object at a
pixel, which may lead to inaccurate estimates of the amount of
material present in the region imaged by a pixel, but accurate
estimates of the contribution of each material to the reflectivity
measured at the pixel. Regardless of t hese subtleties, the large
number of applications of hyperspectral research in the past ten
years indicates that current models h ave value.
Unmixing algorithms currently rely on the expected type of
mixing. Mixing models can be characterized as either linear or
nonlinear [1], [29]. Linear m ix ing holds when the mixing scale
is macroscopic [30 ] and the incident light interacts w ith just one
material, as is the case in checkerboard type scenes [31], [32]. In
this case, the mixing occurs within the instrument itself. It is due
to the fact that the resolu tion of the instrument is not fine enough.
The light from the materials, although almost completely sepa-
rated, is mixed within the measuring instrument. Fig. 2 depicts
Fig. 2. Linear mixing. The measured radiance at a pixel is a weighted average
of the radiances of the m aterials present at the pixel.
linear mixing: Light scattered by three materials in a scene is
incident on a detector that measures radiance in
bands. The
measured spectrum
is a weighted average of the mate-
rial spectra. The relative amount of each material is represented
by the associated weight.
Conversely, nonlinear mixing is usually due to physical inter-
actions be tween the li ght scattered by multiple mater ials in the
scene. These interactions can be at a classical,ormultilayered,
leveloratamicroscopic,orintimate, level. Mixing at the clas-
sical level occurs when light is scattered from one or more ob-
jects, is reflected off additional objects, and eventually is mea-
sured by hyperspectral imager. A nice illustrative derivation of
a multilayer model is given by Borel and Gerstl [33] who show
that the model results in an i nfinite sequence of p owers of prod-
ucts of reflectances. Generally, however, the first order terms
are sufficient and this leads to t he bi linear model. Microscopic
mixing occurs when two m aterials are homogeneously mixed
[28]. In this case, the interactions consist of photons em itt ed
from molecules of one ma terial are absorbed b y m olecules o f
another material, which may in turn emit more photons. The
mixing is modeled by H ap ke as occurring at the albedo level and
notatthereflectance level. The apparent albedo of the mixture is
a linear average of the albedos of the individu al substances bu t
the reflectance is a nonlinear function of albedo, thus leading to
a d ifferent type of nonlin ear model.
Fig. 3 illustrates two non-linear mixing scenarios: the left-
hand panel represents an intimate mixture, meaning that the ma-
terials are in close pr o x imity; the right- hand panel illustrates a
multilayered scene, where there are multiple interactions among
the scatterers at the different layers.
Most of this overview is devo ted to the linear mixing model.
The r eason is that, despite its simplicity, it is an acceptable
approximation of the light scattering mechanisms in many real
scenarios. Furthermore, in contrast to nonlinear mixing, the
linear mix ing model is the basis of a plethora of unmixing
models and algorithms spanning back at least 25 years. A sam -
pling can be found in [1], [34]–[47]). Othe rs will be discussed
throughout the rest of this paper.
B. Brief Overview of Nonlinear Approaches
Radiative transfer theory (RTT) [48] is a well established
mathematical mod el f or the transfer of energy as photons
interacts with the materials in the scene. A complete physics
based approach to nonlinear unm ixing would r equire inferring
356 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATION S AND REMOTE SENSING, VOL. 5, NO. 2, A
PRIL 2012
Fig. 3. Two nonlinear mixing scenarios. Left hand: intimate mixture; Right
hand: multilay e red scene.
the spectral sign a ture s and material densiti es based on the
RTT. Unfortunately, this is an extremely complex ill-posed
problem, relying on scene parameters very hard or impossible
to ob tain. The Hapke [31], Kulbelka-Munk [49] and Shkurato v
[50] scattering formulations are three approximation s for the
analytical solution to the RTT. The former h as been widely used
to study diffuse reflection spectra in chemistry [51] whereas
the later two have been used, for example, in mineral unmixing
applications [1], [52].
One wide class of strategies is aimed at avoiding the complex
physical models usin g simpler but physics insp ired models, such
kernel methods. In [53] and following works [54]–[57], Broad-
water et al. have proposed several kernel-based unmixing al-
gorithms to specifically account for intimate mixtures. Some of
these kernels are designed to be sufficiently flexible to allow
several nonlinearity degrees (using, e.g., radial basis functions
or polynomials expansions) while others are physics-inspired
kernels [55].
Conversely, bilinear models have been successively proposed
in [58]–[62] to handle scattering effects, e.g., occurring in t he
multilayered scene. These models generalize the standard linear
model by introducing additional i nteraction terms. They mainly
differ from each other by the additivity constraints imposed on
the mixing coefficients [63].
However, limit ations inh e rent to the u nmixing a lgorithms
that explicit ly rely on both models are twofold. F ir stly, they are
not multipurpose in the sense th at those d evelo ped to p rocess
intimate mixtures are in efficient in the multiple interaction
scenario (and vice versa). Secondly, they generally require the
prior knowledge of the endmemb er signatures. If such infor-
mation is not available, these signatures have to be estimated
from the d ata by using an endmember extraction alg orithm .
To achieve flexibility, some have resorted to machine learning
strategies such as neural networks [64]–[70], to nonlinearly re-
duce dim ensionality or learn mo del parameters in a supervised
fashion from a collection of examples (see [35] and references
therein). T he poly nom ial post nonlinear mixing m od e l intro-
duced in [71] seems also to be sufficiently versatile to cover a
wide class of nonlinearities. How ever, again, these algorithms
assumes t he prior knowledg e or extraction of the endmem bers.
Mainly d u e to the difficulty of the issue, very few attempts
have been condu cted to address the problem of fully unsuper-
vised nonlinear unmixing. One must still concede that a sig-
nificant contribu tion has been carried by Heylen et al. in [72]
where a strategy is introduced to extract endmem bers that have
been nonlinearly mixed. The algo rithmic scheme is similar in
many resp ects to the well-kno wn N-FINDR algorithm [73]. The
key idea i s to maximize the simplex volume compu ted with
geodesic m easures on the data manifold. In this work, exact
geodesic distances are approximated by shortest-path distances
in a nearest-neighbor g raph. Even m ore recently, same authors
have shown in [74] that exact geodesic distances can be derived
on any data manifold induced by a nonlinear mixing model, such
as the generalized bilinear model introduced in [62].
Quite recently, Close and Gader have devised two methods
for f ully unsup ervised nonlinear unm ixing in the case of in-
timate mixtures [75], [76] based on Hapke’s average albedo
model cited above. One method assumes that each pixel is either
linearly or nonlinearly mixed. The other assum es that there can
be both nonlinear and linear mixing presen t in a single pixel.
The methods were shown to more accurately estimate physical
mixing parameters using measurements made by Mustard et al.
[56], [57], [64], [77] than existing techniques. There is still a
great deal of work to b e done, including evaluating the useful-
ness of combining bilinear m odels with average albedo models.
In summary, although researchers are beginning to expand
more aggressively into nonlinear mixing, the research is imma-
ture compared with linear mixing. There has been a tremendous
effort in the past decade to solve linear unmixing problems and
that is what will be discussed in the rest of this paper.
C. Hyperspectral U nm ixing Processing Chain
Fig. 4 shows the processing steps usually involved in the
hyperspectral unmixing chain: atmospheric correction, di-
mensionality reduction, and unm ixing, which may be tackled
via the classical endmember determination plus in version,
or via sparse regression or sparse coding approaches. Often,
endmember determination and inversion are implemented
simultaneously. B elow, we provide a brief characterization of
each of these steps:
1) A tmospheric correction. The atmosphere attenuates and
scatterers the light and therefore affects the radiance at the
sensor. The atmosp heric correction compensates fo r these
effects by converting radiance into reflectance, which is
an intrinsic property of the materials. We stress, however,
that linear unmixing can be carried out directly on radiance
data.
2) Data reduction. The dimensionality of the space spanned
by spectra from an image is generally much lower than
available number of bands. Identifying appropriate sub-
spaces facilitates dimensionality reduction, improving
algorithm performance and complexity and data storage.
Furthermore, if the linear mixture model is accurate, the
signal sub sp ace dimension is one less than equal to t he
number of endmembers, a crucial figure in hyperspectral
unmixing.
3) Unmixing. The unmixing step con sists of identifying the
endmembers in the scene and the fractional abundances
at each pixel. Three general approaches w ill be discussed
here. Geometrical approaches exploit the fact that linearly
BIOUCAS-DIAS et al.: HYPERSPECTRAL UNMIXING OVERVIEW: GEOMETRICAL, STATISTICAL, AND SPARSE REGR
ESSION-BASED APPROACHES 357
Fig. 4. Schema tic diagram of the hyperspectral unmixing process.
Fig. 5. Illustration of the simplex set for ( is the convex h ull of the
columns of
, ). Green circles repre s ent spectral vectors. Red
circles represent vertic es of th e simplex and correspo nd to the en dmembers.
mixed vectors are in a simplex set or in a positive cone.
Statistical approaches focus on u sing parameter estim a-
tion techniques to determine endmember and abundance
parameters. Sparse regression approaches, which formu-
lates unmixing as a lin ear sparse regression p rob lem, in a
fashion similar to tha t of compressive sensing [78], [79].
This framework relies on the existence of spectral libraries
usually acquired in laboratory. A step forward, t erm e d
sparse coding [80], consists of learning the dictionary
from the data and, thus, av oid ing not only the need of
libraries but also calibration issues related to different
conditions under which the libraries and the data were
acquired.
4) Inversion. Given the observed spectral v ectors and the
identified endmembers, the inversion step consists of
solving a constrained optimization probl em whi ch mini -
mizes the residual between the o bserved spectral vectors
and the linear space spanned by the inferred spectral
signatures; the implicit fractional abundances are, very
often, constrained to be nonneg ativ e and to sum to one
(i.e., they belong to the probabili ty simplex). There are,
however, many hyperspectral unmixing approaches in
which the endmem ber determination and inversion steps
are implemented simultaneously.
The remainder of the paper is organized as follows. Section II
describes the linear spectral mixture model adopted as the base-
line model in this contribution. Section III describes techniques
for subspace identification.SectionsIV,,,VIIdescribefour
classes of techniques for endmember and fractional abundances
estimation under the linear spectral unmixing. Sections IV and
V are devoted to the longstanding geome trical and statistical
based approaches, respectively. Sections VI and VII are devoted
to the recently introduced sparse regression based unmixing and
to the exploitation of the spatial contextual information, respec-
tively. Each of these s ection s introduce the un derlying mathe-
matical problem and summarizes state-of-the-art algorithms to
address such problem.
Experimental results obtained f ro m simulated and real data
sets illustrating the potential and limitations of each class of al-
gorithms are described. The experiments do not constitute an
exhaustive comparison. Both code and data for all the exper-
iments described her e are available at http://www.lx.it.pt/~bi-
oucas/code/unmixing_o verv iew.zip. The paper concludes with
a s um m ary and discussion of plau sible future developments in
the area of spectral unmi xing.
358 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATION S AND REMOTE SENSING, VOL. 5, NO. 2, A
PRIL 2012
Fig. 6. Illustration of the concept o f simplex o f minimum volume containin g the data for three data sets. The endme mbers in the left hand side and in the middle
are identifiable by fitting a simplex of minimum volume to the data, whereas this is not applicable to the righ t hand side data set. The former data set correspond
to a highly mix ed scenario.
II. LINEAR MIXTURE MODEL
If the mul tip le scattering among distinct endm emb e rs is neg-
ligible and the surface is partitioned according to the fractional
abundances, as illustrated in Fig. 2, then the spectrum of each
pixel is well approximated by a linear mixture o f endmember
spectra weighted by the corresponding fractional abundances
[1], [3], [29], [ 39]. In this case, the spectral measurement
1
at
channel
( is the total number o f channels)
from a given pixel, den oted by
,isgivenbythelinear mixing
model (LMM)
(1)
where
denotes the spectral measurement of endmember
at the spectral band, denotes th e frac-
tional abu ndan ce of endmember
, denotes an additive per-
turbation (e.g., noise and modeling er rors), and
denotes the
number o f endmembers. At a giv en pixel, the fractional abun-
dance
, as the name suggests, represents the fractio nal area
occupied by the
th endmem ber. Therefore, the fractional abun-
dances are subject to the following constraints:
(2)
i.e., the fractional abundance vector
(the notation indicates vector transposed) is in the standard
-simplex (or unit -simplex). In HU jargon, the
nonnegativity and the sum-to-o ne constraints are termed a bun -
dance nonnegativity constraint (ANC) and abunda nce sum con-
straint (A SC), respectively. Researchers may sometimes expect
that the abundance fractions su m to less than one since an algo -
rithm m ay not be able to account for every m aterial in a pixel;
it is not clear whether it is better to relax the constraint or to
simply consider that part of the modeling e rror.
Let
denote a -dimensional column vector, and
denote the spectral signatu re of the
th endmember. Expression (1) can then be written as
(3)
1
Although the typ e of spe ctral quantity (radiance, reflectance, etc.) is impor-
tant when processing data, specification is not necessary to derive the math e-
matical approaches.
where is the mixing matrix containing
the signatures of the endmembers present in the covered area,
and
. Assuming that the columns of are
affinely independent, i.e.,
are linearly independent, then the set
i.e., the convex hull of the columns of ,isa -simplex
in
. Fig. 5 illustrates a 2-simplex for a hypothetical mixing
matrix
containing three endmembers. The points in green de-
note spectral vectors, whereas the points in red are vertices of
the simplex and corresp ond t o the e nd members. Note that the
inference of the mixing matrix
is equivalent to identifying
the vertices of the simplex
. This geometrical point of view,
exploited by many unmixing algorithms, will be fu rt her devel-
oped in Section IV-B.
Since m any algorithms adopt either a geometrical or a sta-
tistical framework [34], [36], they are a focus of this paper. To
motivate these two dir ectio ns, let us consid er the th ree d ata sets
shown in Fig. 6 generated under the linear model given in (3)
where the noise is assumed to be negligible. The spectral v ec-
tors generated according to (3) are in a simplex whose vertices
correspond to the endmembers. The l eft hand side data set con-
tains pure pixels, i.e, for any of the
endmembers th ere is at
least one pixel containing only the correspondent material; the
data set in the middle does not contain pure pixels but contains
at least
spectral vectors on each facet. In both data sets
(left and middle), the endm em bers may by inferred by fitting a
minimum volume (MV) simplex to the data; this rather simple
and yet powerful idea, introduced by Craig in his seminal work
[81], underlies several geometrical based unmixing algorithms.
A sim ilar idea was introduced in 1989 by Perczel in the area of
Chemometrics et al. [82].
The MV simplex show n in the right hand side example of
Fig. 6 is smaller than the true one. T his s ituation corresponds
to a highly mixed data set where there are no spectral vectors
near the facets. For these classes of problems, w e usually re-
sort to the statistical framework in which the estimation of the
mixing matrix and of the fractio nal abundances are formulated
as a statistical inference problem by adop tin g suitable proba-
bility models for the variables and p arameters involved, namely
for the fractional abundances and for the mixing matri x.
![Fig. 16. Projected pixels (black points), actual endmembers (black circles), endmembers estimated by N-FINDR (blue stars), endmembers estimated by VCA (green stars) and endmembers estimated by the algorithm in [163] (red stars.](/figures/fig-16-projected-pixels-black-points-actual-endmembers-black-306rvb4z.webp){kind=tracking}
