Recovering Lost Sensor Data through Compressed Sensing

Recovering Lost Sensor
Data through Compressed
Sensing
Zainul Charbiwala

Collaborators:Younghun Kim, Sadaf Zahedi, Supriyo Chakraborty,
Ting He (IBM), Chatschik Bisdikian (IBM), Mani Srivastava

The Big Picture

Lossy Communication Link

zainul@ee.ucla.edu - CSEC - Jan 2010 2

The Big Picture


How do we recover from this loss?


The Big Picture


• Retransmit the lost packets


The Big Picture

Generate Error
Correction Bits


• Proactively encode the data with some protection bits


The Big Picture

Generate Error
Correction Bits


• Proactively encode the data with some protection bits
• Can we do something better ?


The Big Picture - Using Compressed Sensing


CSEC


Generate Compressed
Measurements


CSEC


Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

How does this work ?



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• Use knowledge of signal model and channel



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• CS uses randomized sampling/projections



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• Random losses look like additional randomness !



Generate Compressed
Measurements


CSEC
Recover from
Received Compressed
Measurements

• Random losses look like additional randomness !
Rest of this talk focuses on describing
“How” and “How Well” this works

Talk Outline

‣ A Quick Intro to Compressed Sensing
‣ CS Erasure Coding for Recovering Lost Sensor Data
‣ Evaluating CSEC’s cost and performance
‣ Concluding Remarks


Why Compressed Sensing ?
Physical Sampling Compression Communication Application
Signal


Why Compressed Sensing ?
Physical Sampling Compression Communication Application
Signal

Physical Compressive
Communication Decoding Application
Signal Sampling

Shifts computation to
a capable server


Transform Domain Analysis


‣ We usually acquire signals in the time or spatial domain


‣ By looking at the signal in another domain, the signal may be
represented more compactly



‣ Eg: a sine wave can be expressed by
3 parameters: frequency, amplitude
and phase.



and phase.
‣ Or, in this case, by the index of the
FFT coefﬁcient and its complex
value



and phase.
‣ Or, in this case, by the index of the
FFT coefﬁcient and its complex
value
‣ Sine wave is sparse in frequency
domain

Lossy Compression


Lossy Compression

‣ This is known as Transform Domain Compression


Lossy Compression

‣ The domain in which the signal can be most compactly represented
depends on the signal


Lossy Compression

‣ The signal processing world has been coming up with domains for many
classes of signals


Lossy Compression

classes of signals

‣ A necessary property for transforms is invertibility


Lossy Compression

classes of signals

‣ It would also be nice if there were efﬁcient algorithms to convert
the signals to transform between domains


Lossy Compression

classes of signals

‣ It would also be nice if there were efﬁcient algorithms to convert
the signals to transform between domains
‣ But why is it called lossy compression?


Lossy Compression
‣ When we transform the signal to the right domain, some
coefﬁcients stand out but lots will be near zero
‣ The top few coeffs describe the signal “well enough”


Lossy Compression


Lossy Compression

• JPEG(100%) :
407462 bytes, ~
2x gain


Lossy Compression

• JPEG(100%) : • JPEG (10%) :
407462 bytes, ~ 7544 bytes,
2x gain ~ 100x gain


Lossy Compression

• JPEG(100%) : • JPEG (10%) : • JPEG
(1%) :
407462 bytes, ~ 7544 bytes, 2942 bytes,
2x gain ~ 100x gain ~ 260x gain


Compressing a Sine Wave



‣ Assume we’re interesting in acquiring a single sine wave x(t) in a
noiseless environment



‣ An inﬁnite duration sine wave can be expressed using three
parameters: frequency f, amplitude a and phase φ.



‣ An inﬁnite duration sine wave can be expressed using three
parameters: frequency f, amplitude a and phase φ.
‣ Question: What’s the best way to ﬁnd the parameters ?



‣ Technically, to estimate three parameters one needs three good
measurements



measurements
‣ Questions:



measurements
‣ Questions:
‣ What are “good” measurements ?



measurements
‣ Questions:
‣ What are “good” measurements ?

‣ How do you estimate f, a, φ from three measurements ?


Compressed Sensing


Compressed Sensing

‣ With three samples: z1, z2, z3 of the sine wave at times t1, t2, t3


Compressed Sensing

‣ We know that any solution of f, a and φ must meet the three
constraints and spans a 3D space:


Compressed Sensing

z i = x(t i ) = a sin(2π ft i + φ )
∀i ∈{1, 2, 3}


Compressed Sensing

∀i ∈{1, 2, 3}
φ
‣ Feasible solution space is much smaller

a


Compressed Sensing

∀i ∈{1, 2, 3}
φ
‣ Feasible solution space is much smaller

‣ As the number of constraints grows (from
more measurements), the feasible solution
a
space shrinks
‣ Exhaustive search over this space reveals the
right answer

Formulating the Problem
‣ We could also represent f, a and φ as a very long, but mostly
empty FFT coefﬁcient vector.


empty FFT coefﬁcient vector. Sine wave.
Amplitude
represented by
x color


Amplitude
represented by
Ψ (Fourier Transform) x color


Amplitude
represented by
y = Ψ (Fourier Transform) x color


Amplitude
represented by
y = Ψ (Fourier Transform) x color

− j2 π ft + φ
ae


Sampling Matrix

‣ We could also write out the sampling process in matrix form


Sampling Matrix


x


Sampling Matrix


Φ x


Sampling Matrix


z = Φ x


Sampling Matrix


z = Φ x

Three non-zero entries at some
“good” locations


Sampling Matrix


Three measurements
z = Φ x



Sampling Matrix


Three measurements
z = Φ x
k

n



Exhaustive Search
‣ Objective of exhaustive search:
‣ Find an estimate of the vector y that meets the constraints and is the most
compact representation of x (also called the sparsest representation)

‣ Our search is now guided by the fact that y is a sparse vector
‣ Rewriting constraints:
z = Φx
y = Ψx
−1
z = ΦΨ y


Exhaustive Search

ˆ %
y = arg min y l 0
%
y
z = Φx
y = Ψx %
s.t. z = ΦΨ y −1

z = ΦΨ y −1 y l0
@ {i : yi ≠ 0}


Exhaustive Search

ˆ %
y = arg min y l 0
%
y
z = Φx
y = Ψx %
s.t. z = ΦΨ y −1

z = ΦΨ y −1 y l0
@ {i : yi ≠ 0}

This optimization problem
is NP-Hard !

l1 Minimization

‣ Approximate the l0 norm to an l1 norm

ˆ %
y = arg min y l 1
%
y y = ∑ yi
l1

% −1 i
s.t. z = ΦΨ y

‣ This problem can now be solved efﬁciently using linear
programming techniques
‣ This approximation was not new
‣ The big leap in Compressed Sensing was a theorem that showed
that under the right conditions, this approximation was exact!


The Restricted Isometry Property
ˆ %
y = arg min y l 1
%
y

%
s.t. z = ΦΨ y −1 Rewrite as:
%
z = Ay
For any positive integer constant s, ﬁnd the smallest δs such that:
2 2 2
(1 − δ s ) y ≤ Ay ≤ (1 + δ s ) y
holds for all s-sparse vectors y
A vector is said to s-sparse if it has at most s non-zero entries


The Restricted Isometry Property
ˆ %
y = arg min y l 1
%
y

%
s.t. z = ΦΨ y −1 Rewrite as:
%
z = Ay
For any positive integer constant s, ﬁnd the smallest δs such that:
2 2 2
(1 − δ s ) y ≤ Ay ≤ (1 + δ s ) y
holds for all s-sparse vectors y
A vector is said to s-sparse if it has at most s non-zero entries

The closer δs(A) is to 0, the better the matrix combination
A is at capturing unique features of the signal


CS Recovery Theorem
Theorem: Assume that δs(A) <√2-1 for some matrix A, then the
solution to the l1 minimization problem obeys:
[Candes-Romberg- ˆ
y− y ˆ
≤ C 0 y − ys
Tao-05] l1 l1

C0
ˆ
y− y l2
≤ ˆ
y − ys l1
s
for some small positive constant C0
ys is an approximation of a non-sparse vector with only its s-largest
entries
If y is s-sparse, the reconstruction is exact


Gaussian Random Projections

‣
1
Gaussian: independent realizations of N (0, )
n

* y



‣
1
n

Ψ-1 (Inverse Fourier Transform) * y



‣
1
n

Φ * Ψ-1 (Inverse Fourier Transform) * y



‣
1
n

z = Φ * Ψ-1 (Inverse Fourier Transform) * y


Bernoulli Random Projections

‣
 +1 −1 
Realizations of equiprobable Bernoulli RV  , 
 n n



Uniform Random Sampling

‣ Select samples uniformly randomly



Per-Module Energy Consumption on Mica
ˆ
y %!
;<= >?) 001 ;@=A3(1B
$!
23456(789:
#!
H
"!

!
etection pro- "!&'( #!&'( $!&'( #+!&' "!#%&' "!#%&'
on )* )* )* )* ,-.* ,*/001
"!!!
ctions that de- ;<= >?) 001 ;@=A3(1B
;D<<A<E(1A85(78F:

C+!
m. Model pa-
nergy accurate +!!
MicaZ [18]. The #+!
zed sampling is
Irwin-Hall dis- !
"!&'( #!&'( $!&'( #+!&' "!#%&' "!#%&'
niform random )* )* )* )* ,-.* ,*/001
rmed using an
Figure 3: Power and Duty Cycle costs for Compres-
-bit operation.
sive Sensing versus Nyquist Sampling w/ local FFT.
he 4KB ‣RAMFFT computation higher than transmission cost
of each block for every one second window. This equates to
‣ Highest the achievable duty cycle ofrandom number generator
consumer in CS is the the node, lower values for which
further improve the overall energy eﬃciency. The ADC la-
on of the com- tency is clearly visible here as - CSEC - Jan 2010 component for
zainul@ee.ucla.edu the dominant 22

Compressive Sampling
Physical Sampling Time domain
Signal samples
n
x ∈° z = In x

Physical Compressive Randomized
Signal Sampling measurements
n
x ∈° z = Φx
kxn
k<n


Compressive Sampling
Physical Sampling Compression Compressed
Signal domain samples
n
x ∈° z = In x y = Ψz
nxn

Physical Compressive Compressed
Decoding
Signal Sampling domain samples
n
x ∈° z = Φx %
y = arg min y l 1
kxn %
y
k<n
%
s.t. z = ΦΨ y −1


Handling Missing Data
Physical Compressed
Sampling Compression
n
nxn


Physical Compressed
n
nxn

Missing
Communication
samples

When communication channel is lossy:


Physical Compressed
n
nxn

Missing
Communication
samples

• Use retransmissions to recover lost data


Physical Compressed
n
nxn

Missing
Communication
samples

• Use retransmissions to recover lost data
• Or, use error (erasure) correcting codes


Physical Compressed
n
nxn

Missing
Communication
samples


Physical Compressed
n
nxn

Recovered
Channel Channel
Missing
Communication compressed
Coding Decoding
samples domain samples


Physical Compressed
n
nxn

Recovered
Channel Channel
Missing
Coding Decoding
+
w = Ωy wl = Cw y = ( CΩ ) wl
ˆ
mxn
m>n


Physical Compressed
n
x ∈° z = In x y = Ψz Done at
nxn application layer

Recovered
Channel Channel
Missing
Coding Decoding
+
ˆ
mxn
m>n


Physical Compressed
n
x ∈° z = In x y = Ψz Done at
nxn application layer

Recovered
Channel Channel
Missing
Coding Decoding
+
ˆ
mxn
m>n Done at physical layer
Can’t exploit signal
characteristics


CS Erasure Coding
Communication Decoding
n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
kxn %
y
k<n
%
s.t. zl = CΦΨ y −1


CS Erasure Coding
n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
kxn %
y
k<n
%

n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
mxn %
y
k<m<n
%


CS Erasure Coding
n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
kxn %
y
k<n
%

Over-sampling in CS is
Erasure Coding !

n
x ∈° z = Φx zl = Cz %
y = arg min y l 1
mxn %
y
k<m<n
%


Features of CS Erasure Coding
‣ No need of additional channel coding block
‣ Redundancy achieved by oversampling
‣ Recovery is resilient to incorrect channel estimates
‣ Traditional channel coding fails if redundancy is inadequate

‣ Decoding is free if CS was used for compression anyway


Features of CS Erasure Coding
‣ No need of additional channel coding block
‣ Redundancy achieved by oversampling
‣ Recovery is resilient to incorrect channel estimates
‣ Traditional channel coding fails if redundancy is inadequate

‣ Decoding is free if CS was used for compression anyway
‣ Intuition:
‣ Channel Coding spreads information out over measurements

‣ Compression (Source Coding) - compact information in few measurements

‣ CSEC - spreads information while compacting !


Effects of Missing Samples on CS

z = Φ x



z = Φ x

Missing
samples at the
receiver



z = Φ x

Missing
samples at the
Same as missing
receiver
rows in the
sampling matrix



z = Φ x

What happens if we over-sample?



z = Φ x

• Can we recover the lost data?



z = Φ x

• Can we recover the lost data?
• How much over-sampling is needed?


Some CS Results

‣ Theorem: If k samples of a length n signal are acquired uniformly
randomly (if each sample is equiprobable) and reconstruction is
performed in the Fourier basis:
k
[Rudelson06] s≤C · 4 ′
w.h.p.
log (n)

‣ Where s is the sparsity of the signal


Extending CS Results

‣ Claim: When m>k samples are acquired uniformly randomly and
communicated through a memoryless binary erasure channel that
drops m-k samples, the received k samples are still equiprobable.

‣ Implies that bound on sparsity condition should hold.

‣ If bound is tight, over-sampling rate (m-k) is same as loss rate

[Charbiwala10]


Evaluating the RIP
Create CS Compute RIP
Simulate
Sampling+Domain constant of
Channel
Matrix received matrix

Φ * Ψ-1 (Inverse Fourier Transform)

103 instances, size
256x1024


Evaluating the RIP
Simulate
Channel

A= Φ* Ψ-1

103 instances, size
256x1024


Evaluating the RIP
Simulate
Channel

A= Φ* Ψ-1 A’= C*Φ* Ψ-1

103 instances, size
256x1024


RIP Veriﬁcation in Memoryless Channels

Fourier Random Sampling
Baseline performance - No Loss
(Shading: Min - Max)




20 % Loss - Increase in
RIP constant




20 % Loss - Increase in
RIP constant
20 % Oversampling -
RIP constant recovers


RIP Veriﬁcation in Bursty Channels





20 % Loss - Increase in RIP
constant and large variation




20 % Oversampling - RIP constant
reduces but doesn’t recover




20 % Oversampling - RIP constant
reduces but doesn’t recover

Oversampling + Interleaving
- RIP constant recovers


Signal Recovery Performance Evaluation

Create CS Interleave Lossy CS Reconstruction
Signal Sampling Samples Channel Recovery Error?


In Memoryless Channels



20 % Loss - Drop in
recovery probability



20 % Loss - Drop in

20 % Oversampling -
complete recovery



20 % Loss - Drop in

20 % Oversampling -
complete recovery

Less than 20 %
Oversampling -
recovery does not fail
completely


In Bursty Channels


In Bursty Channels

20 % Loss - Drop in


In Bursty Channels

20 % Loss - Drop in

20 % Oversampling -
doesn’t recover
completely


In Bursty Channels

20 % Loss - Drop in
- Still incomplete recovery
20 % Oversampling -
doesn’t recover
completely


In Bursty Channels
Worse than baseline Baseline performance - No Loss

20 % Loss - Drop in
- Still incomplete recovery
20 % Oversampling -
doesn’t recover
completely

Better than baseline

‣ Recovery incomplete because of low interleaving depth
‣ Recovery better at high sparsity because bursty channels deliver
bigger packets on average, but with higher variance


In Real 802.15.4 Channel



20 % Loss - Drop in



20 % Loss - Drop in

20 % Oversampling -
complete recovery



20 % Loss - Drop in

20 % Oversampling -
complete recovery

Less than 20 %
Oversampling -
recovery does not fail
completely


Cost of CSEC
5
Rnd ADC FFT Radio TX RS

4
Energy/block (mJ)

3

2

1

0
m=256 S-n-S m=10 C-n-S m=64 CS k=320 S-n-S+RS k=16 C-n-S+RS k=80 CSEC

Sense Sense, CS Sense Sense, CSEC
and Compress and and Compress and
Send (FFT) Send Send and Send
and (1/4th with Send
Send rate) Reed with
Solomon RS


Summary

‣ Oversampling is a valid erasure coding strategy for compressive
reconstruction
‣ For binary erasure channels, an oversampling rate equal to loss
rate is sufﬁcient (empirical)
‣ CS erasure coding can be rate-less like fountain codes
‣ Allows adaptation to varying channel conditions

‣ Can be computationally more efﬁcient than traditional erasure
codes


Closing Remarks

‣ CSEC spreads information out while compacting
‣ No free lunch syndrome: Data rate requirement is higher than if using good
source and channel coding independently

‣ But, then, computation cost is higher too

‣ CSEC requires knowledge of signal model
‣ If signal is non-stationary, model needs to be updated during recovery

‣ This can be done using over-sampling too

‣ CSEC requires knowledge of channel conditions
‣ Can use CS streaming with feedback


Recovering Lost Sensor Data through Compressed Sensing

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (7)

Último

Último (20)

Recovering Lost Sensor Data through Compressed Sensing