Exploring connections between Spectral Estimation for
Graph Signals, Coding Theory and Compressed Sensing
Nagaraj T. Janakiraman
joint work with Abhishek Deb, Krishna R. Narayanan
Department of Electrical and Computer Engineering
Texas A&M University
June 2, 2017
1 / 29
Motivation
• Developed fairly independently – several important connections identified
• Connections between non-binary BCH/RS decoding, spectral estimation for
time series, Prony’s method of curve fitting – Wolf in 1967
2 / 29
Motivation
• Developed fairly independently – several important connections identified
• Connections between non-binary BCH/RS decoding, spectral estimation for
time series, Prony’s method of curve fitting – Wolf in 1967
Similar connections to spectral estimation for graph signals?
2 / 29
Outline
• Introduce Coding Theory, CS, Spectral Estimation
• Explore connections between Coding/CS and Spectral Estimation for time
series
• Establish connections for graph signals
• Applications
3 / 29
ROR-CORRECTION
CODING Introduction
MODELto Coding and
2. THE
CODING
MODEL
CS
on, information
andERROR-CORRECTION
coding
to control errors
that
2.
THEtheory
ERROR-CORRECTION
CODING MODEL
ECTION CODING MODEL
ta over noisy and untrusted communication channels. A Error-correction coding also known as chan
Error-correction coding also k
2.
THE
CODING
MODEL
ng also 2.
known
as channel
coding
is ERROR-CORRECTION
a technique
used especially
in
THE
ERROR-CORRECTION
CODING
MODEL
onknown
channel
coding
is a technique
used
especially
in Fig.as1.4.
In this
technique,
the
sender
encodes
the
inis the
field
of used
telecommunication,
information
-correction
coding
also
known
as
channel
coding
is
a
technique
used
especially
Error-correction
coding
also
known
as
channel
coding
a
technique
especially
in
the
fieldinofintelecommunicatio
channel
coding
is a technique
used especially
cation,
information
and
coding
theory
to
controlerrors
errorsthat
that occur
on,
information
and
coding
theory
to
control
by the
introducing
redundancy withinformation
the help of
error- occur toduring
transmission
of transmission
data over noisy
a
occur
during
of data
of telecommunication,
andancoding
control
errors
occur
of
information
and coding
theorytheory
to control
errors
thatthat
occur
tiontelecommunication,
and field
coding
theory to control
errors that
orrection
coding
ataover
overnoisy
noisy
anduntrusted
untrustedcoding
communication
channels.
A coding
model is a technique used especially in
Error-correction
also
known
as
channel
ata
and
communication
channels.
A
T
correction
coding
coding is
a technique
especially
in
model used
of this
technique
shown
in Fig.
1.4.
coded
message
c =also
[c1coding
,known
c2data
, ...,channels.
casn ]channel
is transmitted
through
Error
correction
modelA
ofis
this
technique
is shown
during
transmission
of
communication
channels.
model
sy
and
untrusted
A and untrusted
ansmission
of communication
data over
noisyover
andnoisy
untrusted
communication
Tthemessagechannels. A model T
wn
in
Fig.
1.4.
In
this
technique,
the
sender
encodes
the
in
Fig.
1.4.
In
this
technique,
the
sender
encodes
mit
a
message
m
=
[m
,
m
,
.
.
.
,
m
]
through
a
channel
that
introdu
the
field
of
telecommunication,
information
and
coding
theory
to
control
errors
that
occur
T
1[m
2m
l ]Tthe
messageerrors
m channel
=that
[m
, m2 , ...,
m
introducin
f.4.telecommunication,
information
and
coding
control
1occur
message
m introduces
=l ][mby
Transmit
mmessage
=Fig.
,is
. . .theory
, mby
es
errors
m.a message
The
received
1 , m2 , ..., ml ]
In
technique,
the
sender
encodes
1the
2 ,this
l tothrough
ofthis
thisin
technique
is noisy
shown
in
1.4.
In
technique,
the sendera encodes
thethat
message
chnique
is shown
in Fig. 1.4.with
In this
technique,
the
sender encodes
the message
TWe need
to
carefully
introduce
redundancy
to
correct
these
errors.
by
introducing
redundancy
the
help
of
an
errorerrors.
We
need
to
carefully
introduce
redundancy
to
correct
these
errors.
by introducing
redundancy
with
theover
helpnoisy
ofcommunication
anand
error-correcting
during
transmission
ofand
data
untrusted
communication
channels.
A model
correcting
code. The
resulting
encoded
message
nsmission
of ,data
over
model
T
correcting
code.
The
resulting
enc
with
the noisy
help
an untrusted
errormredundancy
= [m
by ofintroducing
redundancy
help of an A
error-correcting
ducing
decoded
using
algorithms
to find with
the the channels.
1 Tmefficient
2 , ..., ml ] decoding
T
mncoded
m
]
by
introducing
redundancy
with
the
help
of
an
error-correcting
1 , m2 , ...,
l
T
message
cc==
[c11transmitted
...,
c ]through
istransmitted
transmitted
through
T [c
2is
aa noisy
channel
introduces
errors in m
oded
message
ctechnique
,,cc21.4.
,,...c
isin
noisy
of1shown
this
shown
Fig.c =
1.4. Inthrough
this
technique,
thethe
sender
encodes
the message
awhich
noisy
n ]nthis
sage
ccode.
=
[c
,The
c2 , ...,
hnique
is
in
In
sender
message
n ] Fig.
resulting
c22, ,...,
...c
transmitted
through
achannel
noisy which introduce
original
message.
m1is
, mencoded
c1 , c2technique,
, ..., cn e[c1 ,1e,the
enn]T isencodes
2 , ..., ml message
T
he
resulting
encoded
message
[c
]byisthe
transmitted
a ...,
noisy
Tc =
errors
in
The
noisy
message
is
1 , creceived
2 , ..., credundancy
receiver
asthe
rthrough
=
[rreceiver
rnr] =and
Tm.
aserror-correcting
[r decoded
, r , ..., rn ]usin
and
1 , r2 ,an
nces
m.
noisy
message
the
=
[m
,m
...,received
merrors
byby
help
es
errors
in
The
noisy
message
received
bynthe
the
receiver
aserror-correcting
1introduces
2 , is
l ] redundancy
, The
...,m
m
]which
by
introducing
help
ofiswith
an
1, m
2channel
lm.
inisintroducing
m.
Thewith
noisy
message
received
by theofreceiver
as 1 2
emessage
. . [c,isthe
enreceived
nd
decoded
using
efficient
decoding
algorithms
1 ,toTc. find
errors
and
get
back
the corrected
original
messag
which
introduces
errors
in
m.
The
noisy
the
receiver
as back
T by
errors
andthrough
get
the corrected
o
using
efficient
decoding
algorithms
find[c
the
code.
The
resulting
encoded
is transmitted
eded
resulting
message
cto=
cto
iserrors
transmitted
through
noisy
1 , cand
2 , ...c
n ]to
decoding
algorithms
find
get
rusing
=information
r1 ,efficient
rencoded
decoded
efficient
decoding
algorithms
find
theaerrors
and get a noisy
1 ,message
2 , ...c
n ]the=
2 , ...rn and
ation,
theory,
and using
coding
theory,
error corm1m,rn1.channel
.mand
.2,,...,m
.in,cncm.en1,The
, find
rrnnism̂the
. ,m̂as
m̂
...,
m̂rl m̂
r1errors
,,m̂.2 ,....,
original
,,c.2e,n....,
e2 ,algorithms
...,re1nr,1 ,.r.2 ,.to
,]message.
mll m
c1 ,1c, 2m
, ...,
cn em
, lec2c1
,1...,
1 , m̂
2, ,r...,
2 , ...,
n m̂1
l l
2 , ...,
1efficient
rssage.
decoded
using
decoding
2 , ...,
which
introduces
errors
message
received
byand
the1receiver
which
introduces
errors
in
m.
The
noisy
message
isnoisy
received
by
the receiver
as
back
the
corrected
original
message.
Message
Encoder
Channel
Receiver
Decoder
nal
Message
Encoder
Receiver
Encoder
Decoder
ed message.
for controlling
errors
in dataChannel
transmission
overDecoder
noisy
In the field of telecommunica
the
field of telecommunication,
informa
cdecoding
2 information
Fefficient
rtoChannel
2Inalgorithms
F
iMessage
palgorithms
i find
pReceiver
the
original
message.
Encoder
Decoder
r=
r1field
,information
r2 , of
...r
decoded
using
decoding
to
find
the corerrors and get
In
the
telecommunication,
theory,
and
coding
theory,
error
,mmunication,
...rncorrected
and
decoded
using
efficient
the
errors
and
get
n and
theory,
and
coding
theory,
error
corannels. The idea
the sender
encodes
the message in a rection coding is rection
rmation
and is
coding
theory,
cor- theory,
codingused
is a technique
use
a technique
for controlli
cation, theory,
information
theory,
anderror
coding
error cor.,orrected
en ]rection
[used
m̂back
, m̂
, controlling
...,corrected
m̂
Sender
coding
is la] technique
usedmessage.
for
controlling
errors
in dataChannel
transmission
over Decoder
noisy
message.
Message
Encoder
Channel
Receiver
DecoderReceiver
original
Message
Encoder
1original
2the
nique
for
errors
in
data
transmission
over
noisy
-correcting
code.
redundancy
allows the receiver
to and untrusted communication
rolling
in dataThe
transmission
overtransmission
noisy
and untrusted communication
channels. The cha
id
sed
forerrors
controlling
errors
in data
over noisy
and
untrusted
channels. The
idea
is the
sender
encodes
the
message
in a error corfield
ofof
telecommunication,
information
theory,
and
coding
theory,
cor-theory,
e
field
telecommunication,
information
theory,
and
coding
theory,
error
corIn thecommunication
field of telecommunication,
information
theory,
anderror
coding
cation
channels.
The
idea
is
the
sender
encodes
the
message
in
a
e
idea
is
the
sender
encodes
the
message
in
a
redundant
way
by
using
an
errors that may
occur
in the
message,
and oftenintoaredundant way by using an error-correcting cod
hannels.
The
ideaanywhere
is the sender
encodes
the message
redundant
way
byused
using
an
error-correcting
code.
redundancy
the
receiver
toover noisy
ding
isThe
arection
technique
controlling
in The
data
transmission
overtransmission
noisy
coding
is afor
technique
usederrors
for
controlling
errors
inallows
data
oding
is
a
technique
used
for
controlling
errors
in receiver
data
transmission
over
code.
redundancy
allows
the
receiver
to
detect
anoisy
limited
number
of errors
an error-correcting
code.
The
redundancy
allows
the
a limited number
of
errors
that may
occ
ansmission
The
American
mathematician
Hamr-correcting
code.
The
redundancy
allows Richard
the
receiver
todetect to
a
limited
number
of
errors
that
may
occur
anywhere
in
the
message,
and
often
to
sted detect
communication
channels.
The
idea
is
the
sender
encodes
the
message
in
a
and untrusted communication channels. The idea is the sender encodes the message in a4 / 29
Non-binary error correction
occur anywhere
in the message,
and often
usted
communication
channels.
Thetoidea is the sender encodes the message
in a errors without retra
correct these
Introduction to Coding and CS
Syndromes and decoding
h1
h2 hi
..
.
· · · hn
···
1
ei
..
.
hn
ej
⇥
..
.
+
..
.
=
0 +
et
Hm⇥n
ym⇥1
(P arity checks)
cn⇥1
(code vector)
Syndromes
e
(error vector)
t sparse
• Syndrome : Linear combination of hi , i.e., y = ei hi
• Decoding : Find min weight e : y = ei hi
..
.
ej hj
e j hj
ek hk
ek hk
5 / 29
Introduction to Coding and CS
Syndromes and decoding
h1
h2 hi
..
.
· · · hn
···
1
ei
..
.
hn
ej
⇥
..
.
+
..
.
=
0 +
et
Hm⇥n
ym⇥1
(P arity checks)
cn⇥1
(code vector)
Syndromes
e
(error vector)
t sparse
• Syndrome : Linear combination of hi , i.e., y = ei hi
• Decoding : Find min weight e : y = ei hi
..
.
ej hj
e j hj
ek hk
ek hk
Coding theory is about the construction of H and efficient decoding algorithms,
i.e., given a linear combination of the columns of H, it develops tools to
determine a sparse e
5 / 29
Introduction to Coding and CS
Compressed Sensing
a1
..
.
=
a2
..
.
⇥
..
.
am
ym⇥1
(observations)
Am⇥n
(m⌧n)
xn⇥1
(K sparse)
Classical compressed sensing
• x is a k-sparse vector over R or C
• We ‘compress’ x by storing only y = A x
• Reconstruction - Solve x̂ = arg min ||z||0 : y = Az
• CS - Solve x̂ = arg min ||z||1 : y = Az
6 / 29
Introduction to Coding and CS
Compressed Sensing
a1
..
.
=
a2
..
.
⇥
..
.
am
ym⇥1
(observations)
Am⇥n
(m⌧n)
xn⇥1
(K sparse)
Classical compressed sensing
• x is a k-sparse vector over R or C
• We ‘compress’ x by storing only y = A x
• Reconstruction - Solve x̂ = arg min ||z||0 : y = Az
• CS - Solve x̂ = arg min ||z||1 : y = Az
Coding-theoretic approach - syndrome source coding over complex numbers
• Sensing matrix A , Parity check matrix H
6 / 29
Introduction to Coding and CS
Connection between Coding Theory/Compressed Sensing
a1
..
.
=
a2
..
.
⇥
..
.
am
ym⇥1
Am⇥n
(m⌧n)
xn⇥1
(K sparse)
Coding
Parity check matrix
Errors
k-error correcting code
Syndromes
Symbols from Fq
Decoding
,
,
,
,
,
,
,
Compressed sensing
Sensing matrix
Non-zero coefficients
k-sparse recovery
Measurements/Sketch
Symbols from R or C
Sparse recovery
7 / 29
Spectral estimation for time series – connections to Coding/CS
t-error correcting RS code over GF(p)
np
1 over GF(p)
2
1
61
y1
6
6 .. 7 61
4 . 5=6
6 ..
4.
ym
1
2
3
1
W
W2
..
.
W 2k
1
W2
W4
..
.
1
W 4t
...
...
...
..
.
2
...
1
Wn
W 2n
..
.
W (2k
1
2
1)(n
2 3
0
3 6 .. 7
6.7
6 7
7 6 e1 7
76 7
7 6 .. 7
76 . 7
76 7
5 6ek 7
6 7
1) 6 . 7
4 .. 5
0
where, W is a (primitive) element such that 1, W, W 2 , . . . , W p
2
are distinct
8 / 29
Spectral estimation for time series – connections to Coding/CS
t-error correcting RS code over GF(p)
np
1 over GF(p)
2
1
61
y1
6
6 .. 7 61
4 . 5=6
6 ..
4.
ym
1
2
3
1
W
W2
..
.
W 2k
1
W2
W4
..
.
1
W 4t
...
...
...
..
.
2
...
1
Wn
W 2n
..
.
W (2k
1
2
1)(n
2 3
0
3 6 .. 7
6.7
6 7
7 6 e1 7
76 7
7 6 .. 7
76 . 7
76 7
5 6ek 7
6 7
1) 6 . 7
4 .. 5
0
where, W is a (primitive) element such that 1, W, W 2 , . . . , W p
2
are distinct
Decoding
• H - Vandermonde structure
• Berlekamp-Massey decoder
• Complexity is O(n + k 2 )
8 / 29
Spectral estimation for time series – connections to Coding/CS
t-error correcting RS code over C
Compressed Sensing: For any n over C, W = e
2
1
61
y1
6
6 .. 7 61
4 . 5=6
6 ..
4.
ym
1
2
3
1
W
W2
..
.
W 2k
1
W2
W4
..
.
1
W 4t
...
...
...
..
.
2
...
j2⇡
n
1
Wn
W 2n
..
.
W (2k
1
2
1)(n
2 3
0
3 6 .. 7
6.7
6 7
7 6 e1 7
76 7
7 6 .. 7
76 . 7
76 7
5 6ek 7
6 7
1) 6 . 7
4 .. 5
0
9 / 29
Spectral estimation for time series – connections to Coding/CS
t-error correcting RS code over C
Compressed Sensing: For any n over C, W = e
2
1
61
y1
6
6 .. 7 61
4 . 5=6
6 ..
4.
ym
1
2
3
1
W
W2
..
.
W 2k
1
W2
W4
..
.
1
W 4t
...
...
...
..
.
2
...
j2⇡
n
1
Wn
W 2n
..
.
W (2k
1
2
1)(n
2 3
0
3 6 .. 7
6.7
6 7
7 6 e1 7
76 7
7 6 .. 7
76 . 7
76 7
5 6ek 7
6 7
1) 6 . 7
4 .. 5
0
Decoding
• 2k-consecutive (or periodically spaced) rows of the IDFT matrix
• H - Vandermonde structure
• Berlekamp-Massey decoder modified for the complex field
• Complexity is O(n + k 2 )
9 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for Time Series
4
Amplitude
Real part
4
2
0
2
4
0
0.1
0.2
0.3
time (in secs)
0.4
0.5
2
0
2
0
20
40
60
80
Frequency (in Hz)
10 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for Time Series
4
Amplitude
Real part
4
2
0
2
4
0
0.1
0.2
0.3
0.4
2
0
0.5
2
time (in secs)
0
20
40
60
80
Frequency (in Hz)
2
1
61
y1
6
6 .. 7 61
4 . 5=6
6 ..
4.
ym
1
2
3
1
W
W2
..
.
W 2k
1
W2
W4
..
.
1
W 4t
...
...
...
..
.
2
...
1
Wn
W 2n
..
.
W (2k
1
2
1)(n
2 3
0
3 6 .. 7
6.7
6 7
7 6 e1 7
76 7
7 6 .. 7
76 . 7
76 7
5 6ek 7
6 7
1) 6 . 7
4 .. 5
0
W =e
j2⇡
n
10 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for time series - Prony’s method
• Vandermonde structure – converts the set of non-linear equations to linear
equations
11 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for time series - Prony’s method
• Vandermonde structure – converts the set of non-linear equations to linear
equations
• Decoder - two steps
11 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for time series - Prony’s method
• Vandermonde structure – converts the set of non-linear equations to linear
equations
• Decoder - two steps
Berlekamp-Massey – error positions
Input: time domain samples (syndromes) - y
Output: error locator polynomial
⇤(x) = (x
i1 )(x
il
=W
i2 ) · · · (x
ik )
il
11 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for time series - Prony’s method
• Vandermonde structure – converts the set of non-linear equations to linear
equations
• Decoder - two steps
Berlekamp-Massey – error positions
Input: time domain samples (syndromes) - y
Output: error locator polynomial
⇤(x) = (x
i1 )(x
il
=W
i2 ) · · · (x
ik )
il
Forney’s algorithm – error values
Input: syndromes, error locator polynomial
Output: error values ei1 , ei2 , · · · eik
11 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for time series - Prony’s method
• Vandermonde structure – converts the set of non-linear equations to linear
equations
• Decoder - two steps
Berlekamp-Massey – error positions
Input: time domain samples (syndromes) - y
Output: error locator polynomial
⇤(x) = (x
i1 )(x
il
=W
i2 ) · · · (x
ik )
il
Forney’s algorithm – error values
Input: syndromes, error locator polynomial
Output: error values ei1 , ei2 , · · · eik
• Sample complexity – 2k samples
• Time complexity:
Syndrome computation - O(k) (take 2k samples from time)
Decoding complexity - O(k2 )
11 / 29
Spectral estimation for time series – connections to Coding/CS
Spectral Estimation for time series - Prony’s method
• Vandermonde structure – converts the set of non-linear equations to linear
equations
• Decoder - two steps
Berlekamp-Massey – error positions
Input: time domain samples (syndromes) - y
Output: error locator polynomial
⇤(x) = (x
i1 )(x
il
=W
i2 ) · · · (x
ik )
il
Forney’s algorithm – error values
Input: syndromes, error locator polynomial
Output: error values ei1 , ei2 , · · · eik
• Sample complexity – 2k samples
• Time complexity:
Syndrome computation - O(k) (take 2k samples from time)
Decoding complexity - O(k2 )
If, k = O(n ), < 1/2, then has sublinear time complexity
11 / 29
Spectral estimation for time series – connections to Coding/CS
Is there an equivalent Prony’s method for
Graph Signals?
12 / 29
Spectral estimation for time series – connections to Coding/CS
Graph Spectral Estimation
Problem Statement
• GFT of ~
x is k-sparse
• Sampling strategy - less number of samples/observations
• Recovery algorithm - low time complexity
13 / 29
Spectral estimation for Graph signals – connections to Coding/CS —insert
figure here—
real orcorrelation
complex
example
of a
wijvalues.Fig
(i, A
j) graph
2 shows
E any
0annon-negative
signal
is
an attribute
about certain
properties
the
signals
define
Forus
a general
weighted
graph,
w
r
i,j can beof
Notations
• G(V, E) : Graph
• x - Graph domain signal x :
j) = associated with the node.Mathemati
A graph signal is A(i,
an attribute
and a graph signal.It
cans=[s
also1 , be
noted
the n
signal
s2 , ...,
sN ]T that
is a scalar
can be a special
case
ofotherwise
GSP
in the sense, issignals
d
0 The
on aDSP
given
laplacian
signal
s=[s
, s similarity
, ..., s ]T is function.
a scalar function
defined onmatrix
vertices ofdefine
a grap
1
2
N
real or complex
values.Fig
shows
about certain correlation
properties
the siga
and a ring us
graph
are nothing but aperiodic
time of
signals
a
real or complex values.Fig shows an example
of a graph
withcan
unity
weig
and
a
graph
signal.It
also
be n
DSP
can
be
a
special
case
of
GSP
in
the
For a general respectively.
weighted graph, wi,j can be any non-negative real number sense
calcu
Lcertain
= D correlation
and a graph signal.It can also be notedusthat
the
notion
ofA
adjacency on
and The
a ring
graph matrix
are about
nothing
but as,
aperiodicprope
tim
on a given similarity
function.
laplacian
is defined
—insert
figure
here—
us about certain correlation propertiesDSP
of the
definedcase
on the
gr
cansignals
be a special
of GS
respectively.
DSP
can be
case matrix
of GSPwith
inand
the
signals
defined
on
a sense,
ring
graph
are
nothing
but
where,
D ais special
a diagonal
the
diagonal
entries
are
the
—insert
figure
V!R
L
= Dhere—
A
Illustration
of time
Dynamic
Samplin
2 3 and
respectively.
and a ring graph are nothing but aperiodic
signals
periodic
nodes and all(Successive
other entries are
zero.Aggregations
A graph signal
isat
an sing
attrib
x1
Local
• N - Number of nodes in G ,N = |V|
respectively.
• k - Sparsity
• A - Adjacency matrix of G
6 7here—
—insert figure
6 7T
6xx2degree
7
3co
graph
x=[x
, ...,
a2the
scalar
where, D Mathematically,
is a diagonal matrixawith
thesignal
diagonal
entries
are
the
1 , x2
6 N7] is of
—insert figure here—
x
6 7
6 17
6x 7
x
=
6 1.1
7
nodes andof
allaother
entries
are zero.
graph
is an values.
attribute
associated
6 3 7 Figure
graph.
It can
haveAreal
orsignal
complex
6x2 7wis
6 7
6 7
6 7
6
7
T
x
6
7
4
Mathematically, a graph signal x=[x1 , x2 , ..., xN ]21is3a scalar
4 function
5 x = 6defined
x 7
6 37
6 7
x
• V - Eigen vector matrix of Aof a graph. It can have real or complex values.1 Figure
x5
7
6 1 7 1 1.1 2shows
an 6
example
x4 7
6
6 7
• x̂ - Frequency domain signal of x , x̂ = V
• C - Selection Matrix i.e C 2 (0, 1)
1
x
k⇥N
• ei - N ⇥ 1 vector with all entries zero except the
i-th one
• Ek - Tall matrix, [e1 , e2 , ..., ek ]
2
6 x2 7
61 7
6 7
x=6
x3 7
7
2 6
6 7
1
6 7
6 x4 7
4 5
3x5
Consider the above undirected graph of N
adjacency matrix which is given by:
2
0
61
4 60
S=A=6
6
41
0
4
1
1
5
1
4
4 5
x5
= 5 nodes. The4 Shift op
1
0
1
1
0
0
1
0
1
1
1
1
1
0
1
3
0
07
7
17
7
15
0
14 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Aggregation Sampling Approach - Marques, Segarra et al
Shifted Signal: y (l) = Al x
Y = [y (0) , y (1) , .....y (N
1) T
]
Assumptions
• Unique and non-zero eigen values
• For some node i, all elements of vi
Aggregated Signal at node i:
yi = Y T e i
are non-zero
vi = V T ei
Sampled Signal: y˜i = Cyi
15 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Illustration of Aggregation Sampling
16 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Illustration of Aggregation Sampling
2
3
x3
5
x2 + x4 + x5
yi = 4
2x1 + x2 + 3x3 + 2x4 + x5
16 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Illustration of Aggregation Sampling
2
3
x3
5
x2 + x4 + x5
yi = 4
2x1 + x2 + 3x3 + 2x4 + x5
ỹi = Cyi =

x3
2x1 + x2 + 3x3 + 2x4 + x5
16 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Vandermonde Structure
Presence of Vandermonde structure
y˜i = Cyi = C(V
where,
1
Y )T vi = [diag(x̂)
T T
] vi = C diag(x̂)vi = C diag(vi )x̂
: Vandermonde Matrix
17 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Vandermonde Structure
Presence of Vandermonde structure
y˜i = Cyi = C(V
where,
1
Y )T vi = [diag(x̂)
T T
] vi = C diag(x̂)vi = C diag(vi )x̂
: Vandermonde Matrix
Alternative form
ỹi = He
• H=C
– Vandermonde matrix
• e = diag(vi )x̂ – k-sparse vector
17 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Vandermonde Structure
Presence of Vandermonde structure
y˜i = Cyi = C(V
where,
1
Y )T vi = [diag(x̂)
T T
] vi = C diag(x̂)vi = C diag(vi )x̂
: Vandermonde Matrix
Alternative form
ỹi = He
• H=C
– Vandermonde matrix
• e = diag(vi )x̂ – k-sparse vector
Equivalent Prony’s method for GSP!
17 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Example- Prony’s method for GSP
Assume sparsity k = 2,
2
3
+0.203
6 0.0787
6
7
7
x=6
6+0.1617
4 0.2315
+0.054
2
3
0.2
60 7
6 7
7
x̂ = 6
60.37
40 5
0
18 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Example- Prony’s method for GSP
2
6
6
y3 = 6
6
4
3
x3
7
x2 + x4 + x5
7
7
2x1 + x2 + 3x3 + 2x4 + x5
7
5
3x1 + 7x2 + 4x3 + 5x4 + 4x5
14x1 + 14x2 + 19x3 + 19x4 + 10x5
2
3
x3
6
7
x2 + x4 + x5
7
ỹ3 = Cy3 = 6
4 2x1 + x2 + 3x3 + 2x4 + x5 5
3x1 + 7x2 + 4x3 + 5x4 + 4x5
19 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Example- Prony’s method for GSP
2
6
y˜3 = Cy3 = 6
4
1
1
2
1
3
1
1
2
2
2
3
2
1
3
2
3
3
3
1
4
2
4
3
4
2
3
3 v11 xˆ1
2
7
1 6
6 0 7
76
7 6
5 7 6v33 xˆ3 7
6
25 6
7=4
5 6
7
3 4
0 5
5
0
3
0.161
0.2557
7
0.404 5
0.641
Boils down to a RS decoding problem
• Syndromes: ỹ3
• Errors: diag(v3 )x̂
20 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Example- Prony’s method for GSP
• Berlekamp-Massey algorithm to find support
Input: Time domain measurements (syndromes)
⇥
⇤T
ỹ = 0.161, 0.255, 0.404, 0.641
Output: Error locator polynomial
⇤(x) = (x
1
)(x
1
1
)
3
Mapping to positions- P = {1, 3}
21 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Example- Prony’s method for GSP
• Berlekamp-Massey algorithm to find support
Input: Time domain measurements (syndromes)
⇥
⇤T
ỹ = 0.161, 0.255, 0.404, 0.641
Output: Error locator polynomial
⇤(x) = (x
1
)(x
1
1
)
3
Mapping to positions- P = {1, 3}
• Forney’s algorithm to find non-zero values
Input: syndromes, error locator polynomial
Output: Error values - ê1 =
Mapping to coefficients xˆi =
1.0165, ê3 =
êi
vi
0.202
=) x̂1 = 0.2, x̂3 = 0.3
21 / 29
Spectral estimation for Graph signals – connections to Coding/CS
Example- Prony’s method for GSP
• Berlekamp-Massey algorithm to find support
Input: Time domain measurements (syndromes)
⇥
⇤T
ỹ = 0.161, 0.255, 0.404, 0.641
Output: Error locator polynomial
⇤(x) = (x
1
)(x
1
1
)
3
Mapping to positions- P = {1, 3}
• Forney’s algorithm to find non-zero values
Input: syndromes, error locator polynomial
Output: Error values - ê1 =
Mapping to coefficients xˆi =
1.0165, ê3 =
êi
vi
0.202
=) x̂1 = 0.2, x̂3 = 0.3
• Sample complexity – 2k samples
• Time complexity:
Syndrome computation - O(kN )
Decoding complexity - O(k2 )
21 / 29
Applications
Application I: Multiple Access Communication Channel
Problem statement
• Sensor network of N nodes in a
geographic region
• Multiple Access Communication -
central base station collects all
measurements
• Objective: maximize spectral
efficiency of the network
22 / 29
Applications
Application I: Multiple Access Communication Channel
Naive solution
• Ignore dependence in the signals
between the nodes
• Time division Multiple Access
(TDMA)
N time slots – one per
sensor/node
Latency: N units
23 / 29
22]. So,
[15]
and
[21,
22]
inspired
this
thesis
tovalues
establish
a0recovery
algorithm
[17,
consisting of N nodes where V represents
a set
of
nodes
[v
,....,v
]shift
and
Eset
represents
shows
a nodes
simplified
way
to
determine
the
error
too.
The
relation
between
1real
0ondefined
0consisting
0 1like
0these
1 2 0 is
1
2where
N
operators.
The
adjacency
matrix
For aus
general
weighted
w,v
can
be
any
non-negative
number
calculated
about
certain
correlation
properties
of
the
defined
the
1.2
Existing
GSP
3physical
2nodes
30] graph.Classical
2as,
i,j
wmethod
(i,
j)
2
E
0matrix
shift
operators.
The
adjacency
issignals
N
⇥
Nfound
matrix
consisting
ofFramework
N
represents
aan
of
[v
and
E
represents
ofor
Nbased
nodes
1 ,v2 ,....,v
N
w
(i,
j)
2V
E
0graph
ijgraph,
6
7form
6[21,
7w
For
a general
weighted
graph,
w
can
be
any
non-negative
real
number
calculated
based
decoding
algorithms
and
Prony’s
of
curve
fitting
[20]
was
by
Wolf
in
ij
erties
like
similarity
Applications
i,j
ason
an
adjacency
erties
like
similarity
or
physical
proximity.
Consider
adevelop
represented
as
G=(V,
E),
T to
71
and
aaj)
graph
signal.It
can
also
be
noted
that
notion
of
adjacency
Aweighted
graph
iss=[s
abased
versatile
tool
represent
the
high
dimensional
and
data
the For
edges
which
are
relation.
E
can
be
represented
in
matrix
0graph.It
0complicated
0 method
0can
07
16
0Wolf
13aingraph
02 2tel
signal
sw
...,j)
ssome
]A(i,
is
scalar
function
on
vertices
of1the
a6
have
18,
19]
forE
graph
signals
and
an equivalent
Prony’s
for
them.
2number
3[20]
2
=
(1.1)
1 , A(i,
2 ,on
N
wmethod
(i,
j)
2
0defined
=
decoding
algorithms
andgraph
Prony’s
method
of6curve
fitting
was
found
by
6
7path
6
7
ij
a
general
weighted
graph,
can
be
any
non-negative
real
calculated
based
decoding
algorithms
and
Prony’s
of
curve
fitting
[20]
was
found
by
Wolf
in
[21,
0defined
1G=(V,
0complicated
0(1.1)
0E),
1matrix
0which
1[21,
7
6
71are
6 0wei
i,j
erties
like
similarity
or
physical
proximity.
Consider
a
represented
as
the
edges
which
are
weighted
based
on
some
relation.
E
can
be
represented
the
in
edges
DSP
can
be
a
special
case
of
GSP
in
the
sense,
signals
on
a
graph
A
graph
is
a
versatile
tool
to
represent
the
high
dimensional
and
data
6
7
6
consisting
of
N
nodes
where
V
on
a
given
similarity
function.
The
laplacian
matrix
is
defined
as,
A
graph
is
a
versatile
tool
to
represent
the
high
dimensional
and
complicated
data
the edges which are weighted based
onj)some
relation. E0 canotherwise
be represented
72The
6
1 in
06 0matrix
0 0 (1.1)07 6
1 7shift
0 61operators.
0
17
ad2
A(i,
=
A(i,
6
6
7
6
7
6
us
about
certain
correlation
properties
of
the
signals
defined
on
the
graph.Classica
6w
70a06
7 0[17,
6 17
22].
So,
[15]
and
[21,
22]
inspired
this
thesis
to
establish
a
recovery
algorithm
like
[17,
0toL.
1weights,five
0 and
0E
1form
0prop1 an1like
3,
22].
So,
[15]
and
[21,
22]
inspired
this
thesis
establish
recovery
algorithm
on
a
given
similarity
function.
The
laplacian
matrix
is
defined
as,
real
or
complex
values.Fig
shows
an
example
of
a
graph
with
unity
nodes
consisting
of
N
nodes
where
V
represents
a
set
of
nodes
[v
,v
,....,v
]
represents
1.2
Existing
GSP
Framework
6
7
6
7ma
0
otherwise
1
2
N
(i,
j)
2
E
,
6
7
6
7
6
form
as
an
adjacency
matrix
A
or
a
laplacian
matrix
These
are
commonly
called
as
as
adjacency
6
7
6
7
6
ij
where
the
data
elements
form
a
node
and
are
connected
to
each
other
based
on
some
0
0
1
0
0
0
1
0
1
1
form as an adjacency
matrix
A or
ainspired
laplacian
L. establish
These
commonly
called
as
where
the
datamatrix
elements
form
abut
nodeaperiodic
and are
are connected
to6each6
other
based
on
some
prop7
6 signals
7g
For
a
general
weighted
the
edges
which
are
weighted
b
and
a
ring
graph
are
nothing
time
signals
and
periodic
time
7
6
7
6
6
7
6
7
6
22].
So,
[15]
and
[21,
22]
this
thesis
to
a
recovery
algorithm
like
[17,
A(i,
j)
=
(1.1)
a given
similarity
function.
TheaVnode
laplacian
matrix
defined
as,
0 equivalent
1called
0and
0some
17prop0for7them.
1 61 07 A(i,
1 63j)
60on
71
consisting
of N
nodes
where
represents
aconnected
set
ofisnodes
[vdevelop
,vother
] sense,
Eas
represents
0matrix
otherwise
1GSP
2 ,....,v
N
6
6
7
6
7
6
6
where
the data
elements
form
and
are
to
each
based
18,
19]
for
graph
signals
and
an
Prony’s
method
form as on
an adjacency
matrix
A
or
a
laplacian
L.
These
are
commonly
DSP
can
be
a
special
case
of
in
the
signals
defined
on
a
path
grap
,
,
6
7
6
7
For
aerror
general
0 the
0represented
1dimensional
071as 6
0weighted
16byoperators.
0Forney
17 11[19]
217 1
Berlekamp-Massey
[17,
18]
correctly
identifies
locations.
Work
and
a graph
signal.It
can
also
bean
noted
thematrix
notion
of
adjacency
on
a0form
graph
tells
shift
operators.
The
adjacency
istoanrepresent
N
⇥
Ngraph
matrix
defined
as,
shift
The
6
7graph
6adjac
Aor
graph
isthat
a proximity.
versatile
tool
the
high
and
complicated
data
6
7
6
6
erties
like
similarity
physical
Consider
a
G=(V,
E),
0
0
0
0
1
1
0
6
7
18,
19]
for
graph
signals
and
develop
equivalent
Prony’s
method
for
them.
as
an
adjacency
matrix
A
0
otherwise
the adjacency
edges
which
are
weighted
on
some
relation.
EA
be
represented
in matrix
6 calculated
7 6
6 5
A(i,
=
For
a general
weighted
graph,
wbased
canmatrix
be any
non-negative
number
6can
6based
67
respectively.
i,j
40 07
41E),
erties
like
similarity
or
physical
proximity.
Consider
a real
graph
represented
,as
, j)
shift operators.
The
matrix
isdevelop
an N
⇥
N
defined
as,
0thefor
0error
1 values
07G=(V,
1 50similarity
17
26
1 3
L form
= method
D
(1.2)
6them.
6
6and
71each
6
7time
67
on
a]in
given
func
18,
19]
forwhich
graph
signals
and
an
equivalent
Prony’s
shows
a simplified
way
to determine
too.
The
between
these
1.2
Existing
GSP
Framework
andofbe
aN ring
graph
nothing
aperiodic
time
and
where
the
dataare
elements
a set
node
are
to
based
on1The
some
propthe
edges
are or
weighted
based
on
some
relation.
E
can
be
represented
matrix
0calculated
0signals
0other
1 periodic
1 weighted
10 m2
6connected
7
6E),
7 signal
6
6nodes
7
6
consisting
nodes
where
V
represents
abut
of
[v
,v02For
,....,v
Erelation
us
about
certain
properties
of
the
defined
on
graph.Classical
1the
Na
shift
adjacency
a general
weighted
graph,
wcorrelation
can
any
non-negative
real
number
general
based
erties
like
similarity
physical
Consider
aissignals
graph
represented
0as
0G=(V,
0and
0operators.
0 007
116
shift operators.For
The
adjacency
matrix
is
an
N
NThe
matrix
defined
as,
i,j⇥proximity.
4
5similarity
41represents
6
7
6
7
651 4grap
on
aacommonly
given
function
onasa given
similarity
function.
laplacian
matrix
defined
as,
—insert
figure
here—
L
=
D
A
(1.2)
w
(i,
j)
2
E
0
form
an
adjacency
matrix
A
or
a
laplacian
matrix
L.
These
are
called
as
0
0
0
1
0
1
1
1
0
1
1
2
6
7
6
7
6
ij
For
a
general
weighted
graph,
w
can
be
any
non-negative
real
number
calculated
based
For a general 1.2
weighted
graph,
w
can
be
any
non-negative
real
number
calculated
For
general
based
weighted
graph,
decoding
algorithms
and
Prony’s
method
of
curve
fitting
[20]
was
found
by
Wolf
in
[21,
i,j toproximity.
erties
like
similarity
oron
physical
Consider
aand
graph
represented
as1 G=(V,
E), 2
i,jFramework
Existing
GSP
consisting
of
N
nodes
where
V
represents
a
set
of
nodes
[v
,v
,....,v
]
E
represents
respectively.
A
graph
is
a
versatile
tool
represent
the
high
dimensional
and
complicated
data
the
edges
which
are
weighted
based
some
relation.
E
can
be
represented
in
matrix
1
2
N
0
0
0
0
1
0
0
1
0
1 w2
4
5
4
5
4
A(i,
j)
=
(1.1)
canmatrix
be a special
case
in the sense,
signals
on a path
graph
L
=ofDGSP
A
Z
=
form
as similarity
anofadjacency
A
a laplacian
matrix
L. These
aredefined
commonly
called
as(1.2) functio
Existing
GSP
Framework
consisting
NDSP
nodes
where
V or
represents
aSo,set[15]
nodes
[v
,vlaplacian
]set
and
E
on 1.2
a given
function.
The
laplacian
matrix
is
defined
as,
on
arepresents
similarity
1V
2 ,....,v
N
22].
and
[21,
22]
inspired
this
thesis
to
establish
agiven
recovery
algorithm
like
[17,
on
a given
similarity
function.
The
matrix
is
defined
as,
consisting
ofof
N
nodes
where
represents
a
of
nodes
[v
,v
,....,v
]
and
E
represents
0matrix
otherwise
Now,
considering
the
node
3
is
chosen
0
0
0
0
1
0
0
1
1
0
1
2
1
1
2
N
where
the
data
elements
form
a
node
and
are
connected
to
each
other
based
on
some
propwhere,
D
is
a
diagonal
matrix
with
the
diagonal
entries
are
the
degree
of
the
corresponding
form
as
an
adjacency
matrix
A
or
a
laplacian
L.
These
are
commonly
called
as
For aongeneral
weighted
wA(i,
For a graph,
general
weighted
—insert
i,jT
shiftedges
operators.
The
adjacency
matrix
is2figure
an
Nhere—
⇥of
Ntime
matrix
as,
on a given similarity
function.
The
laplacian
matrix
is
defined
as,
a given
similarity
function.
wijnothing
(i,
j)but
E
0relation.
and
a ring
graph
are
aperiodic
signals
andbeperiodic
time
signals
the
which
are
weighted
on
some
Edefined
represented
in
matrix
Illustration
Dynamic
Sampling
L
=⇥
D
A
(1.2)
For
a can
general
weighted
graph,
wmatrix
b
Awhich
graph
is
aweighted
versatile
toolj)based
to
represent
dimensional
and
data
18,
19]
for
graph
signals
and
develop
an
equivalent
Prony’s
method
for
them.
Now,
considering
the
node
3as isG=(V,
the
2
3
i,jE),can
theN
edges
which
arehigh
weighted
based
on
some
relation.
E(1.1)
canrepresented
be
represented
inchosen,
matrix
shiftedges
operators.
The
adjacency
matrix
is
an
Nthe
matrix
defined
as,
w
(i,
2The
E
0relation.
erties
like
similarity
or
Consider
acomplicated
graph
is
picked
and
the
scaled
factor
is fu
fo3
operators.
adjacency
matrix
isphysical
an
N
⇥proximity.
N
matrix
defined
as,
A(i,
j)
=
ijshift
the
are
based
on
some
E
can
be
represented
in
matrix
on
a
given
similarity
nodes
and
all(Successive
other
entries
are
zero.
Adimensional
graph
signal
is
attribute
associated
with
theweighted
node.
graph
isrespectively.
a j)
versatile
tool
to
represent
the
high
and
complicated
data
For
a general
weighted
graph,
wi,j can
any
non-negative
real
number
calculated
a chosen,
general
based
xbe
Local
Aggregations
at
single
node)
on
aanagiven
similarity
function.
The
1 the
where,AD
is aasdiagonal
with
the
diagonal
entries
are
degree
of
the
corresponding
Now,
considering
the
node
3Foris
the
3rdg
A(i,
= matrix
(1.1)
L
=
D
A
(1.2)
form
as anofadjacency
matrix
A
or
ais
laplacian
matrix
L.
These
arefactor
commonly
called
as
6
7
picked
and
the
scaled
matrix
is
formed
form
an
adjacency
matrix
A
or
a
laplacian
matrix
L.
These
are
commonly
called
as
1.2
Existing
GSP
Framework
consisting
N
nodes
where
V
represents
set
of
nodes
[v
,v
,....,v
]
and
E
represents
1
2
N
0 diagonal
otherwise
6todegree
7given
on
aeach
similarity
function.
lapla
where, D
is aas
diagonal
matrix
withA
the
entries
are
the
corresponding
where
the data
elements
form
node
and
are
connected
other
based
on
some
L
=D
Ax=[x
(1.2)
i!
Slot
a given
similarity
function.
The
laplacian
defined
as,
on
given The
similarity
func
Tmatrix
2isof
3the
6
7j)
form
anwhere,
adjacency
matrix
or aaon
a graph
laplacian
L.
These
commonly
called
asapropi1!
,adjacency
xSlot
,each
xbased
]are
aN
function
defined
on
vertices
x
For
apropgeneral
weighted
graph,as
w
0 matrix
otherwise
2N
2are
shift
operators.
The
matrix
is2is
an
⇥
N matrix
zmatrix
D—insert
isMathematically,
a diagonal
the
diagonal
entries
the
degree
ofscalar
the
corresponding
figure
here—
is
picked
and
the
scaled
factor
matrix
is
formed
w...,
(i,
E
0Channel
where
theall
data
elements
form
node
and
are
connected
to
other
based
on
some
6
ijE
the
edges
which
are
weighted
some
relation.
Edefined
can
beas,
represented
in matrix
w
j)
2
0on
L
= with
D
Asignal
(1.2)
iis!
Slot
ij
xassociated
A
graph
a(i,
tool
to 7
represent
the
and
complicated
data2
1 high dimensional
nodes
and
other
entries
area zero.
A graph
an
with
the
isignal
!
Slot
6attribute
7
A(i,
j)
=is
(1.1) node.
jversatile
!
Node
z
shift
operators.
The
adjacency
matrix
is
an
N
⇥
N
matrix
defined
as,
Channel
6
7
j
!
Node
w
(i,
j)
2
E
0
6
7
GSP basednodes
scheme
where,
D
is
a
diagonal
mT0
A(i,
j)
=
(1.1)
ij
on
a
given
similarity
function.
where,
D
is aasdiagonal
matrix
with
the
diagonal
are
the
degree
of
theE),
corresponding
xvalues.
=
erties
like
similarity
or physical
proximity.
Consider
a
graph
represented
as
G=(V,
6 entries
7 with
x
and
alloperators.
other
entries
are
zero.
Aare
graph
signal
is
an
attribute
associated
with
the
node.
0
form
an
adjacency
matrix
A
or
a
laplacian
matrix
L.
These
are
commonly
called
as
3
6
7
nodes
and
all
other
entries
zero.
A
graph
signal
is
an
attribute
associated
the
node.
j
!
Node
of
a graph.
It
can
have
real
or
complex
Figure
1.1
shows
an
example
of
a
graph
0
otherwise
shift
The
adjacency
matrix
is
an
N
⇥
N
matrix
defined
as,
wherejthe
elements form6a L
node
and are
each
other
based
onsignal
some
!data
Node
2 prop6
So is
the
scaled
is given
b
=
A6connected
(1.2)
i ! Slot to D
a(1.1)
diagonal
matrix
x2 7
j)
= non-negative
7D
erties
like similarity
ori,jphysical
proximity.
Consider
represented
asE G=(V,
E),
z ij6where,
6
Channel
For a general
weighted
graph,
w
canA(i,
be x=[x
any
number
calculated
based
T aaregraph
w
(i,7j)
2
0
6
7graph
0 similarity
otherwise
nodes
other
zero.
signal
an attribute
associated
with0the0node.
601 L
!
Node
Mathematically,
graph
signal
,x
, ...,
xentries
]real
isthe
a A(i,
scalar
function
defined
vertices
6Consider
SLOT-1
shift
adjacency
matrix
N
⇥j 7
Nis
matrix
defined
as, where,
X
T
1 and
2all
NThe
1be
xA4j)
6
7[v
D6
is a(1.1)
diagonal
=+is an
• For
or
proximity.
graph
represented
as2
G=(V,
E),
a general
weighted
graph,
waof
can
any
non-negative
real
number
calculated
Exploit
the
dependence
the
where,
Dconsisting
is aMathematically,
diagonal
matrix
with
the
of
the
corresponding
D
ison
a diagonal
matr10
6
3aphysical
T xaN
a graph
signal
x=[x
xoperators.
,entries
...,
]2
is
scalar
function
defined
on
vertices
7based
i,jin
1, x
2like
nodes
and
all
other
entrie
N
nodes
where
Vdiagonal
represents
set
of
nodes
,....,v
] aand
E
represents
1are
=
0erties
otherwise
26
Nwhere,
4degree
5 1x,v
x3real
Mathematically,
a
graph
signal
x=[x
,
x
,
...,
]
is
a
scalar
function
defined
on
vertices
Z
=
6
6
SLOT-1
6
7
X
+
1
2
N
1
For
a
general
weighted
graph,
w
can
be
any
non-negative
number
calculated
based
T
i,j
0 corresponding
otherwise
Dconsisting
is a diagonal
with
theVdiagonal
entries
the
of
where,
D
isfunction
athe
diagonal
matrix
wi
Mathematically,
aare
graph
xthe
, ...,
x
]and
aE
scalar
defined
on
vertices
0entries
0a0diagonal
116200are
0
SLOT-1
and
all
other
of Nmatrix
nodes
where
represents
set
ofmatrix
nodes
,v
]set
represents
X
+
6
7ofisthe
12,,....,v
N
1xsignal
1degree
N
where,
Dconsisting
is aais
diagonal
the
diagonal
entries
are
degree
of
corresponding
D
is6
m
zdefined
on awhere,
given
similarity
function.
The
laplacian
matrix
as,
nodes
other
nodes
where
anodes
nodes
] and 6
represents
11
2 ,....,v
Nwhere,
6 ent
w
j)
2[vx=[x
E12V1.1
02shows
6all
xshows
6
7with
6
7an
5represents
ijof 1N (i,
of
acan
graph.
It can
real
or similarity
complex
values.
Figure
of[va1 ,vgraph
between
nodes
2ofother
3
2
ofnodes
a graph.
Itother
have
real
ormatrix
complex
values.
Figure
aEand
Z6
=graph
6entries
4 a
6
71 1.1
x(i,
6example
zdefined
on
a given
function.
The
laplacian
matrix
is an
as,
zdefined
onsignals
a given similarity
function.
The
laplacian
is
and
alledges
entries
arehave
zero.
A
graph
isas,
an
attribute
with
nodes
the
and
node.
all
47
w
j)
2Mathematically,
Eexample
0shows
ijbe
the
which
are
weighted
based
some
relation.
Eassociated
can
represented
in
matrix
w
(i,
j)
2
E
0Ashows
6
6a
of
avalues.
graph.
Itother
can
have
real
or complex
values.
Figure
1.1
an
example
a0graph
graph
A(i,
j)and
=ijsignal
(1.1)
6
71.1
4
5
0inall
1of
13zgrap
1
nodes
allon
entries
are
zero.
graph
signal
is
an
attribute
with
nodes
the
and
node.
other
entrie
x
of
a graph.
It
can
have
real
or
complex
Figure
an
example
of
a
graph
the
edges
which
are
based
onbesome
relation.
Eassociated
can
be
represented
matrix
For
a
general
weighted
graph,
w
can
be
any
non-negative
real
number
calculated
based
2weighted
Mathematically,
a
2
1
2
A(i,
j)
=
(1.1)
6
where,
D
is
a
diagonal
matrix
with
s
z
6
i,j
3
7
nodes
and
all
other
entries
are
zero.
A
graph
signal
is
an
attribute
associated
with
nodes
the
and
node.
all
other
entries
are
ze
6
7
For
a
general
weighted
graph,
w
can
any
non-negative
real
number
calculated
based
1
11
12
1
where,
D
is
a
diagonal
matrix
wi
i,j be represented
the
which
are weighted
relation.
can
in matrix
6
6
Mathematically,
signa
Z =a7
A(i,
j) = onbesome
(1.1)
6graph
4
6 7 E where,
For edges
a general
weighted
graph,
wbased
any
real
calculated
based
2as0non-negative
4 anumber
xotherwise
a diagonal
matrix
d
6
6
i,j can
• Signal is kMathematically,
0]T isis
62with
sparse
for
chosen
6
7matrix
Mathematically,
aTgraph
x=[x1function
, x2or, ...,
xND
a5matrix
scalar
function
defined
Mathematically,
ona 7
vertices
athe
graph
SLOT-2
X
+vertices
2xscalar
form
adjacency
A
L.
commonly
= otherwise
6
7
6graph
13zIt 3as
24zhav
awell
graph
signal
x=[x1A
, x2or, ...,
is
asignal
defined
Mathematically,
onaThese
acalled
sig
SLOT-2
of
graph.
can
6as
+laplacian
on
axgiven
similarity
TheThese
laplacian
matrix
isof
defined
as,are
2 matrix
form
as
an
adjacency
a
laplacian
matrix
L.
are
commonly
called
7 Base
5r1
NX]anx
3function.
nodes
and
all
other
entries
are
zer
6
7
2
s
L
=
D
A
(1.2)
2
21
22
T
6
7
6
graph.
It
can
have
0
otherwise
on an
given
similarity
function.
matrix
is1 +defined
as,
2commonly
Base
Station Co
6 L.
7complex
nodes
and
all
other
entries
are
zero.
aa graph
signal
x=[x
, x=2or, D
...,
xgraph.
] iscanamatrix
scalar
function
defined
Mathematically,
onmatrix
vertices
aaofAccess
graph
x
SLOT-2
Xlaplacian
2 matrix
Application
II:
Multiple
Access
form
as
adjacency
A
aThe
are
called
as
7a4have
6
(1.2)
1L
NA
Application
II:
the
scaled
graph
signal
formed
of
alaplacian
real
or
values.
Figure
shows
an Multiple
example
of
graph.
Itissignal
can
have o
r
shiftItoperators.
The
adjacency
matrix
is 1anFinally,
Nof
⇥ N1.1
defined
as, 6
6
7 These
graph.
It
can
real
graph Mathematically,
Application
II:
Multiple
Access
Communicatio
on a given
similarity
function.
TheA
laplacian
isxdefined
as,
21haveweighted
6
7 graph
6Communicat
Base
Station
nodes
and
alla(1.2)
other
entries
A
graph
3azero.
7based
5 .gr
4x
For a matrix
real
number
calculated
6
Decod
L adjacency
=
Multiple
Acce
4 7graph, wi,j can be any non-negative
6II:
7
6have
.are
. 4signal
of a graph.
can have The
real
or D
complex
values.
1.1
shows
an
example
of aofMathematically,
graph.
a graph
can
real
2isgeneral
shiftItoperators.
matrix
anFigure
N⇥
defined
as,A Application
4N
5matrix
Station
6 It
7
6 (1.2)as:
Finally,
the
scaled
graph
signal
is
formed
1
s
=
=
L
=
D
4
3
5
2
3
2
3
Mathematically,
a can
graph
x=[
of a slots
graph.
Itoperators.
can
realweighted
or
complex
values.
Figure
shows
an
example
of
graph.
graph
or
D iswais
diagonal
matrix
with
the diagonal
thea
degree
ofof
theas,
corresponding
6 based
7cansignal
6real
shiftas
The
adjacency
matrix
an
N
⇥any
N1.1
matrix
defined
as,areof
aIt
graph.
Ithave
or c
c
For
ahave
general
graph,
can
be
non-negative
real
number
calculated
on
a given
similarity
Theentries
laplacian
matrix
isadefined
i,j
• Only 2k
x5function.
7 have
6 . real
opposed
tograph,
Nwhere,
in
2 w
2 6signal
3 1x.,1 x
z ij (i,calculated
. 7
Mathematically,
j) 2 E as0graph
1
For a general
weighted
w andcan
be any non-negative
real
number
based
6
6x=[x
Finally,
scaled
graph
is 0node.
formed
7signal
7
nodesi,j
all other entries
are
zero. A
A graph
signal
an attribute
the
2associated
36
36
A(i,
j) =is the
(1.1)
71 6
L=
Dgraph
(1.2)
06
02as:
07
6 here
7 with
6 7
Consider
the above
undirected
of N = 5 nodes.
The
Shift operator
is6have
the
4
where,
D
is
a
diagonal
matrix
with
the
diagonal
entries
are
the
degree
of
the
corresponding
6
7
6
2
3
2
of
a
graph.
It
can
real
or
com
6
7
6
7
TDMA
on
a given
similarity
function.
The
laplacian
matrix
is defined
as, entries
where,
D isA
a diagonal
matrix
with the
diagonal
of
the
L which
=
(1.2)
. corresponding
.17 6x.2 7
s2 7degree
s1are6the
6SLOT-2k
6vertices
7 66x(1.2)
T L 0= D otherwise
adjacency
matrix
iszsignal
by:
A
where,
D is a diagonal
matrix
with
the
diagonal
the
degree
of
the
2Dmatrix
Mathematically,
aare
graph
x=[x
xas,
function
onreal
2 37
41 52 defined
36
41 5can
3+have
45
6
of
a]corresponding
graph.
It
1, x
2 , ...,
N
2 1 23
2
3 17
on a given
similarity
function.
Theentries
laplacian
isN defined
5or
6
70 0!
00 4
1complex
7
wgiven
(i,
j)
2
E
0is3aXscalar
7Time
76
ij
6041 103
Time
Time 6
Slots
! !
Slots
2 w
67
6
7=
and
allj)
other
are
zero. A graph
signal
is
attribute
associated
withx7
the6node.
znodes
27Slots
26 2k2
sScheme:
=
6
6of as2k
2 entries are zero. A graph
s36
(i,attribute
2 entries
E20 associated
sNaive
san
Naive
Scheme:
zx2k1
37
6
716
6
6node.
7
1shows
2an
27 6zx
ij
based
scheme
10 0 1Figure
0
Naive
Scheme:
nodes andO(k
all other
signal
is
an
with
the
A(i,
(1.1)
of a graph.
Itj)
can=
have
real
orGSP
complex
values.
1.1
example
graph
6
6
7
• nodes
6
Latency:
)
units
6
7
6
7
6
726based
(TDMA)
6 real
70 number
6 67
a general weighted
wi,j1with
can07be any
non-negative
calculated
(TDMA)
2defined
11 6
3on
17
Tnode.
61graph,
(TDMA)
6
7
1
0
1
0
0
1
0
0
6
7
and all other entries are zero. A graphA(i,
signal
is anFor
attribute
associated
the
0
1
6
7
7
j)
=
(1.1)
6
7
6
Mathematically,
a
graph
signal
s=[s
,
s
,
...,
s
]
is
a
scalar
function
vertices
6
7
6
7
1 diagonal
6matrixare
72degree
1 26
4scorresponding
57
where,
D is a diagonal
with the
are
the
degree4of the 7
corresponding
736=
766
where, D is a diagonal matrix with the
diagonal
entries
of
the
6sN2entries
76
7x35
=
s36
60 1 0 1the
x726
46
47
76
6=x6
0=entries
6
74Slots
6
17
S = otherwise
=function.
Ton a L
6matrix
6
7
7 47
6
6
6 theThe
7 of
given
similarity
laplacian
is vertices
defined
5as,
5
Time
!
4A have
D
A
where,
D is a diagonal
matrix
the
diagonal
are
degree
the
corresponding
671.1
6
764first
SLOT-2k
3example
7(1.2)
41of7
5a6
Mathematically,
a graph
signal x=[x
]
a
scalar
function
defined
on
X
+1graph
N 1 , xwith
2Inassociated
1this
3 example,
27
of is
a graph.
Itother
can
or2zero.
complex
Figure
shows
an
graph
41real
5 values.
6
7
0
1
0
1
1
6
7
0
otherwise
6
7
6
2 , ...,
the
1
1
0
nodes
and
all
entries
are
A
signal
is
an
attribute
with
the
node.
T xN
6
7
6
6
7
6
7
=scalar
D References
A
(1.2) 7 6x5 67x=4 76 x45
Naive
6svertices
7=
SLOT-2k
Mathematically, a graph
signal
defined
on
+ 1 0s =
s467 sScheme:
1 , x2 , ..., xN ] isLaX
54
37 6
0 an
0 1attribute
6associated
6 34
7 56
nodes
and x=[x
allNother
entries are zero. Mathematically,
A graphfunction
signal
is
with
the
node.
6
7
5
T 4(TDMA)
6
7
6
7
3 3node.
7 2defined
4 157 on
4 vertices6 2x31
graph
signal s=[sassociated
scalar
2graph
1function
1 , s2 , ..., sN ] is with
all real
otherorentries
are zero.
A graph
signal
an
attribute
the
7 4
of a graph. nodes
It canand
have
complex
values.
Figure
1.1isashows
an example
ofsa5a6
L1= 6
D
A
4calculated
56
2 s4 7
7
6x4 7x(1.2)
5
After eigenvalue
decomposition,
we getexample
A = U Ureal
,of6
a general
weighted
graph,
wi,j can
beshows
any
based
of a graph. It can haveFor
real
or complex
values.
Figure
asignal
graph
Tanreal
of 1.1
It cannon-negative
complexfunction
values.
Figure
1.1
antoexample
graph
4number
5
4of a5station.
The
s isshows
the
Mathematically,
agraph,
graph
signal
s=[s
,a diagonal
sgraph.
...,
]have
is a orreal
scalar
defined
7sent
4on
5vertices
4 base
3x1
For a where,
generalDweighted
w
bethe
number
calculated
based
2 ,non-negative
21any
33 corresponding
i,j can
T sN entries
is
a
diagonal
matrix
with
are
the
degree
of
the
Mathematically,
a graph
signal
s=[s
s2 , ...,
sN entries
] 0.4294
is a are
scalar
function
on vertices
0.3717
0.4390
0.6015ofdefined
0.3505
1 , diagonal
sthe
x5
where,
D
is
a
diagonal
matrix
with
the
the
degree
corresponding
5
6
7 the to
where, D is a diagonal
with the
diagonal
entries
degree
the corresponding
on a given similarity function. The laplacian
matrixmatrix
is0.5100
defined
as,
0.3717
0.4700
The
signal
s is are
sent
theofbase
station.
6 0.6015 0.1378
7
2
of
a
graph.
It
can
have
real
or
complex
values.
Figure
1.1
shows
an
example
of
a
graph
on a given
similarity
function.
The
laplacian
matrix
is
defined
as,
6
7
2
0.1378 is 0.5100
0.3717associated
0.47007 with the node.
U =6
nodes and
allhave
otherreal
entries
are 2zero.
A0.6015
graph
signal
an
attribute
nodes
and all
other
entries
areshows
zero.
A graph
signal iswith
an attribute
with the node.
of a graph.
can
complex
values.
Figure
1.1
an example
a associated
graph
4
5 of
nodes
and allItother
entries
areor2zero.
A graph
signal
is
an
attribute
associated
the
node.
0.0000
0.7702
0.3069
0.0000
0.5590to
The signal
s is sent
the
base
station.
0.3717 0.4294
0.4390
22
a graph
signal s=[s1 ,0.6015
s2 , ..., sN ]T0.3505
is a scalar function defined on vertices
T
Mathematically, a graph signal s=[sMathematically,
,
s
,
...,
s
]
is
a
scalar
function
defined
on
vertices
1 2 T N
Mathematically, a graph signal s=[s1 , s2 ,of...,
sN=] D
scalar
defined
onshows
vertices
23 / 29
A
(1.2)
7 of a graph
aL
graph.
It2is
canahave
real orfunction
complex values.
Figure 1.1
an example
Application I: Multiple Access Communication Channel
i ! Slot
j ! Node
Outline
Outline
7
Outline
Outline
. whereGSP
.elements
.way
.19]7
decoding
algorithms
Prony’s
method
of curve
fitting
[20]
found
by
in [19]
[21,
shows
assimplified
to
determine
the
error
values
too.
The
relation
between
these
1.2
Framework
for
graph
signals
and
develop
an18]
equivalent
Prony’s
method
them.
and
a 18,
graph
signal.It
can
also
be
noted
that
notion
of
adjacency
on
acommonly
graph
tells
6i.e, .zijExisting
where
the
form
node
and
are
connected
each
other
based
onwas
some
propBerlekamp-Massey
[17,
correctly
identifies
the
error
locations.
Work
by Wolf
Forney
⇥
matrix
Z
contains
numbers
for
each
slot
i and
node
j,in
j ,data
as of
an
adjacency
matrix
A
ortoathe
laplacian
matrix
L.for
These
are
called
asA
consisting of itsNdata
nodes
where
V
represents
aasuitable
set
nodes
[v
,vand
,....,v
]shift
and
E
represents
operators.
The
adjacency
matr
4
5form
shows
a simplified
way
to determine
theshows
error
values
too.
The
between
these
1.2
GSP
Framework
whereExisting
the
dataV
elements
form
node
are
connected
to
each
other
based
on
some
22].
So,
[15]
and
[21,
22]
inspired
this
thesis
tovalues
establish
a0recovery
[17,
consisting of N nodes
where
represents
aa set
ofand
nodes
[v
]shift
and
Erelation
represents
a simplified
waybe
to0
determine
the
error
too.
The
relation
between
1real
0on
0prop0 algorithm
0 1like
0these
1 0 is
1
2 ,....,v
N
operators.
The
adjacency
matrix
For
aus
general
weighted
graph,
w,v
can
any
number
calculated
based
about
certain
correlation
properties
of
thenon-negative
signals
defined
the
graph.Classical
1.2
Existing
GSP
Framework
z
z
.
.
z
w
(i,
j)
2
E
For
a
general
weighted
graph,
w
can
be
any
non-negative
real
number
calculated
based
decoding
algorithms
and
Prony’s
method
of
curve
fitting
[20]
was
found
by
Wolf
in
[21,
like
oron
physical
Consider
arepresented
graph represented
as G=(V,data
E),
Aweighted
graphsimilarity
is abased
versatile
tool
toproximity.
represent
the
dimensional
and
complicated
the For
edges
which erties
are
some
relation.
can
be
incalculated
matrix
wmethod
(i,
j) curve
2 EE
0 high
A(i,
j)
= be
a general
weighted
graph,
wtool
can
any
non-negative
number
based
algorithms
and
Prony’s
of
fitting
[20]real
was
by
Wolf
in(1.1)
[21,
erties
like
similarity
oraon
physical
Consider
arepresented
graph
represented
G=(V,
E),
Atoproximity.
graph
isfunction.
a versatile
tool
to
represent
the
high1found
dimensional
complicated
data
60as
consisting
nodes
V
onA(i,
given
similarity
The
laplacian
matrix
isin
defined
as,
graph
is abased
versatile
represent
the
high
dimensional
and
complicated
the edges which decoding
areAweighted
relation.
E
can
be
6
72The
6
06
0matrix
0and
07 data
17
0of6
1operators.
0 where
17
shift
ad2
j)some
=
(1.1)
A(i,
6
7algorithm
6N
0thesis
otherwise
6
7
6
usVabout
certain
correlation
properties
of
the
signals
defined
on
the
6w
7
6
7 0[17,
67 6
the error values, 2k syndromes collected
after
sampling
aresimilarity
passed
on
toof
the
Berlekamp22].
So,
[15]
and
[21,
22]
inspired
this
to
establish
a
recovery
algorithm
like
[17,
0
1
0
0
0
1
0
1 graph.Classica
1like
17
3
22].
So,
[15]
and
[21,
22]
inspired
this
thesis
to
establish
a
recovery
on
a
given
function.
The
laplacian
matrix
is
defined
as,
real
or
complex
values.Fig
shows
an
example
of
a
graph
with
unity
weights,five
nodes
consisting
N
nodes
where
represents
a
set
of
nodes
[v
,v
,....,v
]
and
E
represents
1.2
Existing
GSP
Framework
6
7
6
7ma
0
otherwise
2to6
N
(i,
j)6
2
E0based
,prop7on
6
So the scaled
signal
is given
by,
as
anand
adjacency
matrix
A orare
awith
laplacian
matrix
L.
These
are
an
asadjacency
61help
7periodic
6commonly
ij
where
theBase
data
form
aform
node
are
connected
to
each
other
based
on
0For
0some
0form
0assignals
17
067
16
form
as an
adjacency
matrix
Aelements
or
matrix
L.
These
commonly
called
the signals.
The
Station
hasathe
priorthe
knowledge
of
the
graph
structure
the
where
data
elements
form
abut
node
and
are
connected
each
other
some
prop7
6called
a01like
general
weighted
gb1,
the
which
are
and
alaplacian
ring
graph
are
nothing
aperiodic
time
and
time
6
7edges
6
6317
60 signals
7represents
61as
7 weighted
671 (1.1)
22].
So,
[15]
and
[21,
22]
inspired
this
thesis
to
establish
a
recovery
algorithm
[17,
A(i,
j)
=as,
on
a
given
similarity
function.
The
laplacian
matrix
is
defined
1
0
0
0
0
1
1
0
6
7
6
7
consisting
of
N
nodes
where
V
represents
a
set
of
nodes
[v
,v
,....,v
]
and
E
0
otherwise
1
2
N
A(i,
j)
6
7prop6for
7graph
6grap
6an
7
607
7path
67
where
the dataA
elements
form
a node
and
are
connected
tomatrix
each
other
based
on
some
18,
19]
graph signals
and
develop
equivalent
Prony’s
method
them.
form as an adjacency
matrix
ora graph
a laplacian
matrix
L.
These
are
commonly
called
as
DSP
can
be
afor
special
case
of
GSP
in
the
sense,
signals
defined
on
a
,
,
6
7
6
7
4
For
a
general
weighted
0
0
1
0
0
1
0
1
1
2
Berlekamp-Massey
[17,
18]
correctly
identifies
the
error
locations.
Work
by
Forney
[19]
and
signal.It
can
also
be
noted
that
the
notion
of
adjacency
on
a
graph
tells
shift
operators.
The
adjacency
is
an
N
⇥
N
matrix
defined
as,
shift
operators.
The
adjac
6
6
6
A
graph
is
a
versatile
tool
to
represent
the
high
dimensional
and
complicated
data
6
670 =
erties
like
similarity
or physical
proximity.
Consider
G=(V,
0 them.
07 form
06
0an
17 1j)
6represented
7matrix
61E),
71
18,
19]
for
graph
signals
and
develop
an
equivalent
method
for
of which
individual
signal
values
are
decoded
ascan
shown
in Figure
2.2.Prony’s
adjacency
matrix
0 a graph
otherwise
thethe
edges
which
are
weighted
on
some
relation.
EA
can
be
represented
in
6
71asas6
6 1A
A(i,
For
a general
weighted
graph,
wbased
be
any
non-negative
number
6
6based
respectively.
i,j
40 07
41E),
erties
like
similarity
or
physical
proximity.
Consider
a real
graph
represented
,as
,6
shift operators.
The
adjacency
matrix
isdevelop
an N
⇥
N
matrix
defined
as,
0thefor
0error
1calculated
07G=(V,
1 50similarity
17
26
153
L form
= method
D
(1.2)
6them.
6
6and
71each
6
7time
67
on
aa]in
given
func
18,
19]
for
graph
signals
and
an
equivalent
Prony’s
shows
a simplified
way
to determine
values
too.
The
relation
between
these
1.2
Existing
GSP
Framework
andofbe
aN ring
graph
nothing
but
aperiodic
time
signals
and
periodic
signal
where
the
dataare
elements
a set
node
are
connected
to
other
based
on1The
some
propthe
edges
which
are
weighted
based
on
some
relation.
E
can
be
represented
matrix
0
0
0
0
1
1
0
1
1
2
6
7
6
7
6
6
7
6
7
6
consisting
nodes
where
V
represents
a
of
nodes
[v
,v
,....,v
and
E
represents
us
about
certain
correlation
properties
of
the
signals
defined
on
the
graph.Classical
1
2
N
shift
operators.
adjacency
For
a
general
weighted
graph,
w
can
any
non-negative
real
number
calculated
For
general
based
weighted
grap
erties
like
similarity
or
physical
proximity.
Consider
a
graph
represented
as
G=(V,
E),
0
0
0
0
1
0
0
1
1
0
shift
matrix
is an
Ni,j⇥
NThe
matrix
defined
as,
SLOT-1
4
4
6
7 6 5similarity
65 4 m
2 operators.
3 2 The adjacency
32
3matrix
2 figure
3
on
a1 0given
function
similarity
function.
laplacian
matrix
is defined
as,
—insert
here—
L
=7like
D
A
(1.2)
wmethod
(i,
j)
2 0E acommonly
formonExisting
asa given
angraph,
adjacency
A
aarelaplacian
matrix
L.can
These
are
as117
0For
1represented
1called
1 byas
0Wolf
1based
2 w2
60calculated
7[20]real
6E
7
6E),
ijbe
For
aor
general
weighted
graph,
w
any
non-negative
number
calculated
For a general 1.2
weighted
wi,jFramework
can
berespectively.
any
non-negative
real
number
general
based
weighted
graph,
decoding
algorithms
and
Prony’s
curve
fitting
found
[21,
i,j
erties
similarity
oron
physical
Consider
a0
graph
GSP
consisting
of
N
nodes
where
V
represents
ais....
set
of
nodes
[v
]0dimensional
and
represents
Aweighted
graph
abased
versatile
tool
toproximity.
represent
the
high
and
the
edges
which
some
relation.
E
be
represented
matrix
1 ,v
2of,....,v
0can
0N
0in complicated
0(1.2)
1 G=(V,
1 2
4
50path
41was
5in0data
4 Z
A(i,
j)These
=signals
(1.1)
DSP
can
be
a
special
case
of
GSP
in
the
sense,
defined
on
a
graph
s
z
z
z
z
z
x
z
x
+
z
x
+
L
=
D
A
=
form
as
an
adjacency
matrix
A
or
a
laplacian
matrix
L.
are
commonly
called
as
1
11
12
13
14
15
1
11
1
12
2
6 7 6 on 1.2
7 6where
7 The
722]
Existing
Framework
consisting
of NGSP
nodes
V6
represents
aSo,set[15]
nodes
[v
,vlaplacian
]set
E
represents
a given
similarity
function.
laplacian
matrix
is
defined
as,
similarity
functio
1V
2 ,....,v
N
22].
and
[21,
inspired
this
thesis
to
establish
agiven
algorithm
on
a given
similarity
function.
The
matrix
defined
consisting
ofof
N
nodes
where
represents
of
nodes
[v
,vother
]1and
represents
0matrix
Now,
considering
the
node
3 propis[17,
0otherwise
0and
0ison
1toacommonly
0as,
0like
1chosen
2i,j 1
1recovery
2 ,....,v
N
6 7 6
6 adjacency
6diagonal
7The
where
the
data
elements
form
a node
and
are
each
based
on
some
where,
D7laplacian
is aw
matrix
with
the
diagonal
entries
are
degree
of0the
corresponding
anmatrix
adjacency
matrix
A
a7
laplacian
L.0
These
are
called
as
For
aaconnected
general
weighted
wA(i,
For
a1Egraph,
general
weighted
33
—insert
shiftedges
operators.
is2figure
an
Nhere—
⇥
N
matrix
as,
function.
The
matrix
is
defined
as,
athe
given
similarity
function.
T
(i,
j)
E
0orrelation.
6 s 7 on6azgivenz similarity
7 6are
7form
6asijznothing
7time
and
a ring
are
aperiodic
signals
andon
time
signals
the
which
weighted
based
on
some
Edefined
can
beperiodic
represented
in
matrix
Illustration
of
Dynamic
Sampling
z23A zgraph
z25is
xgraph
zbut
x
+
...
L
=⇥
A
(1.2)
22 operators.
24 The
2 7 matrix
21 x
1+
22
2D
For
a
general
weighted
graph,
w
can
b
6 2 7 6 21 shift
7
6
6
7
a
versatile
tool
to
represent
the
high
dimensional
and
complicated
data
18,
19]
for
graph
signals
and
develop
an
equivalent
Prony’s
method
for
them.
Now,
considering
the
node
3
is
chosen,
the
2
3
i,j
the
edges
which
are
weighted
based
on
some
relation.
E
can
be
represented
in
matrix
adjacency
is
an
N
N
matrix
defined
as,
w
(i,
j)
2
E
0
erties
like
similarity
or
physical
proximity.
Consider
a
graph
represented
as
G=(V,
E),
is
picked
and
the
scaled
factor
matrix
is fu
fo3
shift
operators.
The
adjacency
matrix
is
an
N
⇥
N
matrix
defined
as,
A(i,
=tool
(1.1)
the A
edges
which
are
on
some
relation.
can
be isnon-negative
represented
in matrix
6 7 6
7weighted
6j)and
7ij allbased
6
7wE can
on awith
given
similarity
nodes
other
entries
are
zero.
A
graph
signal
an
attribute
associated
the
node.
graph
is
a
versatile
to
represent
the
high
dimensional
and
complicated
data
For
a
general
weighted
graph,
be
any
real
number
calculated
For
a
general
based
weighted
g
i,j are
xNow,
6 7 6 where, D is a diagonal
76
7 (Successive
6 the diagonal
7matrix
Local
Aggregations
at
single
node)
on
a
given
similarity
function.
The
1 the
matrix
with
entries
degree
of
the
corresponding
respectively.
considering
the
node
3
is
chosen,
the
3rd
A(i,
j)
=
(1.1)
L
=
D
A
(1.2)
form
as
an
adjacency
A
or
a
laplacian
matrix
L.
These
are
commonly
called
as
6V represents
7
6 . 7where,
6 . D is
7
6
7
6
7
. aform
.
.
.
.
.
is picked
the scaled
factor
is formed
as
an
adjacency
matrix
A
or
a
laplacian
matrix
These
called
1.2
Existing
Framework
consisting
of NGSP
nodes
whereL.
aare
setand
ofcommonly
nodes
[v1 ,v2 ,....,v
Eas
represents
N ] andmatrix
0
otherwise
6
7
on
a
given
similarity
function.
The
lapla
diagonal
matrix
with
the
diagonal
entries
are
the
degree
of
the
corresponding
where
the data elements
form
node
and
are
connected
to each
other
some
7 6
7 matrix
6 7=
6or aon
7
L
=D
Ax=[x
(1.2)
i!
Slot
a given
similarity
function.
The
laplacian
as, on
on
given similarity func
Tmatrix
2is defined
3based
s=6
6
7j)
form
as
anwhere,
adjacency
A
laplacian
L.
These
commonly
called
asapropi1!
aa graph
,adjacency
xSlot
,each
xbased
]are
aN
function
defined
on
vertices
x
For
apropgeneral
weighted
graph,as
w
6 7=6
6 form
6
7
07
otherwise
2N
2are
shift
operators.
The
matrix
is2is
an
⇥
N matrix
zmatrix
D—insert
isMathematically,
a7diagonal
the
diagonal
entries
the
degree
ofscalar
the
corresponding
here—
Channel
is
picked
and
the
scaled
factor
matrix
is
formed
w...,
(i,
E
where
the
data
elements
a zero.
node
and
are
connected
to
other
based
on
some
6
ijE
the
edges
which
are
weighted
some
Edefined
can
beas,
represented
in matrix
w
j)
2
0on
L
D
Asignal
(1.2)
iis!
Slot
ij
x01relation.
6 . 7 6 nodes
7The
6 .figure
7 matrix
6 = with
7
A
graph
a(i,
tool
to 7
represent
the
high dimensional
and
complicated
data2
and
other
are
A graph
is
an
with
the
isignal
!
Slot
.alloperators.
.
.entries
6attribute
7
A(i,
j)
=
(1.1) node.
jversatile
!
Node
z=ijis.is
shift
adjacency
matrix
an
N
N
matrix
defined
as,
Channel
6 associated
7 where,
j⇥
!
Node
6 7nodes
6 . and .all
7are
6 zero.
7 physical
6
7
w
(i,
j)
2
E
0
6x
7
D
is
a
diagonal
m0
A(i,
j)
(1.1)
on
a
given
similarity
function.
where,
D
a
diagonal
matrix
with
the
diagonal
entries
are
the
degree
of
the
corresponding
x
=
erties
like
similarity
or
proximity.
Consider
a
graph
represented
as
G=(V,
E),
6
7
other
entries
A
graph
signal
is
an
attribute
associated
with
the
node.
6 7 6
7other
6graph.
7 matrix
6are
form
as
an
adjacency
matrix
or
laplacian
matrix
L.
These
are
commonly
called
as60 T
3 7aFigure
6A
nodes and
all
entries
zero.
A
graph
signal
is7an
associated
with
the
node.
jdata
!
Node
of
a
It
can
have
real
or
complex
values.
1.1
shows
an
example
of
a
graph
0attribute
otherwise
shift
operators.
The
adjacency
is
an
N
⇥
N
matrix
defined
as,
where
the
elements
form
a
node
and
are
connected
to
each
other
based
on
some
propj
!
Node
2
6
7
So
the
scaled
signal
is
given
b
L
=
D
A
(1.2)
i
!
Slot
where,
D
is
a
diagonal
matrix
x
= non-negative
(1.1)
6 represented
7 calculated
6 For
7 6 . erties
7
7A(i,
6j)
2 7 as G=(V,
or6
physical
proximity.
Consider
graph
E),
z ij6 (i,
6
Channel
weighted
any
number
based
T a7
w
j) 2
0
. like
. similarity
.graph,
. w
.can
. ,operators.
6
7graph
6 . 7 a general
6 Mathematically,
7 i,j
61signal
7 be6x=[x
7
0 similarity
otherwise
nodes
other
are
zero.
signal
anEattribute
associated
with0the0node.
601 L
!
Node
graph
,x
...,
xentries
]real
isthe
a A(i,
scalar
function
defined
on
vertices
6Consider
SLOT-1
shift
adjacency
matrix
an
N
⇥j 7
Nis
matrix
defined
as, where,
X
T
1 and
2all
NThe
10
xA4j)
6
7[v
D6
is a(1.1)
diagonal
=+is,v
erties
or
proximity.
aand
graph
represented
as2
G=(V,
E),
5Mathematically,
4 weighted
5nodes
4 be
5
4non-negative
5
For4a general
graph,
waof
can
any
real
number
calculated
where, Dconsisting
is aMathematically,
diagonal
matrix
with
the
diagonal
are
degree
of
the
corresponding
where,
D
is
a
diagonal
matr
6
3aphysical
T xaN
a graph
signal
x=[x
x2like
,entries
...,
]2
is
scalar
function
defined
on
vertices
6
7based
i,j N
1, x
nodes
and
all
other
entrie
where
V
represents
set
of
nodes
,....,v
]
E
represents
1
x
=
0
otherwise
1
2
N
4
5
x
a
graph
signal
x=[x
,
x
,
...,
]
is
a
scalar
function
defined
on
vertices
Z
=
6
6
3real
6
X
+ non-negative
1 Mathematically,
2 weighted
Ngraph,
1 with
Fordiagonal
general
can be x=[x
any
based0 matrix
T7 number
i,j signal
0SLOT-1
otherwise
where,
Dconsisting
a diagonal
the
entries
are
the
of
corresponding
where,
D
iscalculated
athe
diagonal
wi
graph
xthe
, ...,
x
]and
aE
scalar
function
defined
on
vertices
0a0diagonal
116200are
0
SLOT-1
and
all
other
of. Nmatrix
where
Va represents
set
ofwmatrix
nodes
,v
]set
represents
X
+
6
7ofisthe
12,,....,v
N
1x1degree
son
z2k1similarity
zis
. function.
znodes
xN
z2k1
x1 +
zis2k2
x
+
...
N
2k a given
2k2
2kN The
2aof
where,
Dconsisting
aais
diagonal
diagonal
entries
are
degree
of
corresponding
Dentries
is6
m
zdefined
laplacian
matrix
as,
nodes
other
nodes
where
anodes
nodes
] and 6
represents
11
2 ,....,v
Nwhere,
6 ent
w
j)
2[vthe
E12V1.1
02shows
6all
xshows
6
7with
6
7an
5represents
ij 1N (i,
of
acan
graph.
It can
real
or similarity
complex
values.
Figure
of[va1 ,vgraph
2ofother
3
2
ofnodes
a graph.
Itother
have
real
ormatrix
complex
values.
Figure
aEand
Z6
=graph
6entries
4 a
6
71 1.1
x(i,
6example
zdefined
on
a given
function.
The
laplacian
matrix
is an
as,
zdefined
on a given similarity
function.
The
laplacian
is
and
alledges
entries
arehave
zero.
A
graph
signal
isas,
an
attribute
with
nodes
the
and
node.
all
47
w
j)
2Mathematically,
Eexample
0shows
ijbe
the
which
are
weighted
based
on
some
relation.
Eassociated
can
represented
in
matrix
w
(i,
j)
2
E
0Ashows
6
6
of
avalues.
graph.
Itother
can
have
real
or complex
values.
Figure
1.1
an
example
of
a0graph
a
graph
ij
A(i,
j)
=
(1.1)
6
71.1
4
5
0
1
1
nodes
and
all
entries
are
zero.
graph
signal
is
an
attribute
associated
with
nodes
the
and
node.
all
other
entrie
x
of
a
graph.
It
can
have
real
or
complex
Figure
an
example
of
a
graph
the
edges
which
are
weighted
based
on
some
relation.
E
can
be
represented
in
matrix
For
a
general
weighted
graph,
w
can
be
any
non-negative
real
number
calculated
based
2
Mathematically,
a33zgrap
1with
A(i,
j) be
=
6 6
where,
Dtheis
ain
diagonal
matrix
sa1aentries
z(1.1)
62based
i,j For
721
nodes and
alledges
otherwhich
entriesare
areweighted
zero. A
graph
is
anrelation.
attribute
associated
with
nodes
and
node.
allmatrix
other
ze
6 1 7 graph,
a general
weighted
wi,j
can
be
any
non-negative
real
number
11are
12
where,
D6
wi
the
on
some
can
represented
7
Mathematically,
signa
Zis calculated
=diagonal
A(i,
j) can
=signal
(1.1)
6graph
4 matrix
6 7 E where,
For
general
weighted
graph,
wbased
beform
any
real
calculated
based
2as0non-negative
4 anumber
xotherwise
a diagonal
matrix
d
6
6
0]T isis
62with
In this example, the
firsta four
powers
(0 tosignal
2k-1)of
Ai,jare
by:
6
7matrix
Mathematically,
aTx
graph
signal
x=[x1function
, x2or, ...,
xND
a5matrix
scalar
function
defined
Mathematically,
ona 7
vertices
athe
graph
SLOT-2
X
+vertices
2xscalar
an
adjacency
A
laplacian
L.
These
arecalled
commonly
called
as
= otherwise
6
7
6
1
3
2
Mathematically,
a graph
x=[x
, xgiven
, ...,
x
]
is
a
defined
Mathematically,
on
a
graph
sig
SLOT-2
of
graph.
It
can
hav
6
X
+
on
a
given
similarity
function.
The
laplacian
matrix
is
defined
as,
2 matrix
form
as
an
adjacency
a
laplacian
matrix
L.
These
are
commonly
as
3
7
4
1A
2or
N
3
Base
nodes
and
all
other
entries
are
zer
6
7
2
z21
z22 5r1
= defined
D A as,
(1.2)
6as2graph
74can
6
of
a(1.2)
graph.
It
have
0laplacian
otherwise
on an
given
similarity
function.
matrix
isThese
2commonly
Base
Station Co
6 L.
7L
nodes
and
all
other
entries
are
zero.
aa graph
signal
x=[x
, x=2or, D
...,
xgraph.
]T iscan
amatrix
scalar
function
defined
Mathematically,
onmatrix
vertices
x
SLOT-2
X
1 + matrix
2 matrix
Application
II: aof
Multiple
Access
form
as
adjacency
A
aThe
are
called
as
7a have
6
1L
NA
Application
II:
Access
the
scaled
graph
signal
formed
of
alaplacian
real
or
values.
Figure
shows
an Multiple
example
of
graph.
Itissignal
can
have o
r
shiftItoperators.
The
adjacency
is 1anFinally,
Nof
⇥ N1.1
defined
as, 6
6
7complex
graph.
It
can
Application
II:
Multiple
Access
Communicatio
on a given
similarity
function.
TheA
laplacian
isxdefined
as,
21haveweighted
6
7 graph
6Communicat
GSP basedMathematically,
scheme
Base
Station
nodes
and
alla(1.2)
other
entries
are
A
Mathematically,
graph
signal
3azero.
7based
4real
5 .gr
4x
For a matrix
general
real
number
calculated
6
Decod
L
=
D
Application
II:
Multiple
Acce
4 7graph, wi,j can be any non-negative
6
7
6
.
.
of a graph.
can have The
real adjacency
or complex
values.
1.1
shows
an
example
of
a
of
graph.
a
graph
It
can
have
real
2is anFigure
shiftItoperators.
matrix
N⇥
defined
as,
4N
5matrix
Station
6
7
6
Finally,
the
scaled
graph
signal
is=signal
formed
1entries
sgraph.
=can
L=
D5the
A
(1.2)as:
4an
3
2
3as,
2or
3
Mathematically,
a
graph
x=[
of a graph.
Itoperators.
canahave
realweighted
or complex
values.
Figure
1.1
shows
example
of
a
of
graph.
a
graph
It
have
real
c
where,
D iswais
diagonal
matrix
with
the
diagonal
are
degree
of
the
corresponding
6
7
6
shift
The
adjacency
matrix
an
N
⇥
N
matrix
defined
as,
of
a
It
can
have
real
or
c
For
general
graph,
can
be
any
non-negative
real
number
calculated
based
on
a
given
similarity
function.
The
laplacian
matrix
is
defined
i,j
x5
6 . 3 x.,1 x
2 w
2 6signal
• Graph known
z ij (i,calculated
. 7
all sensors
andnodes
base
j) 2 E as0graph
1
For to
a general
weighted graph,
w
real
number
based
1 7
6
7 6x=[x
i,j can be any non-negativeMathematically,
Finally,
scaled
graph
is 0node.
formed
7signal
and all other entries
are
zero. A
A graph
signal
an attribute
the
2associated
36
A(i,
j) =is the
(1.1)
71 6
L=
Dgraph
(1.2)
06
02as:
03 6
6 here
7 with
6 7
Consider
the above
undirected
of N = 5 nodes.
The
Shift operator
is6have
the
4graph.
where, D is a diagonal
matrix
with the
diagonal
entries
the degree
of
the
corresponding
6the
7 real
6 or
2
3
27
of
a
It
can
com
6
7
on
a
given
similarity
function.
The
laplacian
matrix
is
defined
as,
where,
Dare
isA
a diagonal
matrix
with
the
diagonal
entries
are
the
degree
of
corresponding
L
=
D
(1.2)
.
.17
s
s
station
6
76
0
otherwise
2
1
6vertices
7 66x(1.2)
T L=D
adjacency
matrix which
iszsignal
given
by:
6x.2 7
A
where, D is a diagonal
matrix
with the
diagonal
the
degree
of
the
SLOT-2k
2 matrix
Mathematically,
aare
graph
x=[x
xas,
]corresponding
is aXscalar
function
on
2 37
416
52 defined
1 23
36
41 5can
3+have
475
N defined
6
7
of
a
graph.
It
real
or
complex
1, x
2 , ...,
N
2
2
3
on a given
similarity
function.
Theentries
laplacian
is
4
5
4
6
70 0!
00 16 17 6 7
wijall other
(i,entries
j)2 2areEzero. A0graph
7Time
601 103
Time
Time 6
Slots
! !
Slots
2 w
6
67
3 signal
6
7=
nodes and
is
an
attribute
associated
withx7
the6node.
z
27Slots
26
sScheme:
=
6
6
7
s
x
(i,
j)
2
E
0
s
s
x
Naive
Scheme:
s
z
z
3
37
6
7
6
7
6
Naive
1
1
2
2
ij
GSP
based
scheme
0
1
0
1
0
2k
2k1
2k2
Naive
Scheme:
nodes
and
all
other
entries
are
zero.
A
graph
signal
is
an
attribute
associated
with
the
node.
A(i,Itj)
(1.1)
ofby
a graph.
can=
have real or complex values. Figure 1.1
shows
an
graph
6(TDMA)
6
76example
6of a calculated
76 6
6 (TDMA)
7
7based
• nodes
70 number
7
Node
a general weighted
can07be any
3on
i,j1with
Tnode.
61graph,
6 a scalar
7(TDMA)
1602defined
0 011 6
17
6
76 6
06 real
06
1(1.1)
andjalltransmits
other entries signal
are zero.scaled
A graphA(i,
signal
is anFor
attribute
associated
0 with
1ws=[s
6sNnon-negative
6
7 15
j)
=
6
72vertices
67
Mathematically,
a graph
signal
s72 , ...,the
]7
is
function
7
1 , diagonal
6
6
7
6
1
2
3
4
5
where,
D
is
a
diagonal
matrix
the
entries
are
the
degree
of
the
corresponding
6
7
6
7
4
where, D is a diagonal matrix with the
diagonal
entries
the17degree
of
the
6s 2=
76
6x726
7x3 766
=x6
6=scorresponding
7
s
60 1 are
46
47
6
7
3
0
otherwise
7
0
1
S
=
A
=
T
6matrix
6
7
7 4764 6
6
756
6 theThe
7 of
onis
aL
given
similarity
function.
laplacian
is vertices
defined
4Slots
5as,
5
Time
!
4 have
=
D
A
(1.2)
D is a diagonal
matrix
the
diagonal
entries
are
degree
the
corresponding
6
7
6
7
SLOT-2k
zij Mathematically,
in slot i where,a graph
3
7
4
signal x=[x
x
]
a
scalar
function
defined
on
X
+
N 1 , xwith
2
1
3
2
1
of
a
graph.
It
can
real
or
complex
values.
Figure
1.1
shows
an
example
of
a
graph
4
5
6
7
0
1
0
1
1
6
7
0
otherwise
6
7
6
2 , ...,
N
Inassociated
this example,
the
1 are
1 2zero.
1 0 A1graph signal 6
nodes
an6
with the6
T
7
7attribute
76first
=
Dand all
Aother entries
(1.2)
Naive
6svertices
7 is=
7node.
References
4
SLOT-2k
Mathematically, a graph
signal
scalar
defined
on
+ 1 0s =
76
s467 sScheme:
1 , x2 , ..., xN ] isLaX
54
x536
4 76 x5
37 6
0 1attribute
6associated
7x=
nodes
and x=[x
allNother
entries are zero. Mathematically,
A
graphfunction
signal
is0 signal
an
the
node.
6
76
52 with
4
56 2x3
T 4(TDMA)
6
7
6
7
3
7
4
5
4
a
graph
s=[s
,
s
,
...,
s
]
is
a
scalar
function
defined
on
vertices
1
3
2
1
1
2
N
6
7
6
nodes
all real
other
are zero.
A graph
signal
an attribute
associated
the node. 5 6 7(1.2)4 1
• zij ofchosen
that
receiver
a graph. such
It canand
have
orentries
complex
values.
Figure
1.1is shows
an example
ofwith
graph
L1= 6
D
A
2 s4 7
s5a4calculated
7
6x4 7x5
After eigenvalue
decomposition,
we getexample
A = U Ureal
,of6
a general
graph,
wi,j can
beshows
any
based
of a graph. It can haveFor
real
or complex
values.
Figure
asignal
graph
Tanreal
21weighted
of 1.1
as
graph.
It s
cannon-negative
have
or
complexfunction
values.
Figure
1.1
antoexample
graph
4number
5
4of a5station.
The
s3 isshows
the
Mathematically,
a
graph
signal
s=[s
,
,
...,
]
is
a
scalar
defined
7sent
4on
5vertices
4 base
3x1
For
a
general
weighted
graph,
w
can
be
any
non-negative
real
number
calculated
based
1
2
N
2
3
i,j
where, D isaashifts
diagonal
with
diagonal
are
the
degree0.3505
of the
obtains 2k consecutive
ofmatrix
the
Mathematically,
graph
signal
s=[s
s2the
, ...,
sN entries
]T 0.4294
is aentries
scalar
function
oncorresponding
vertices
0.3717
0.4390
0.6015ofdefined
1 , diagonal
sthe
x5
where,
D
is
a
diagonal
matrix
with
the
are
the
degree
corresponding
5
6
7
where,
D
is
a
diagonal
matrix
with
the
diagonal
entries
are
the
degree
of
the
corresponding
on a given similarity function. The laplacian
matrix is0.5100
defined
as,
0.3717
The
signal0.4700
s is sent
to the base station.
6 0.6015 0.1378
7
2 an
of a node
graph.
can
have
real
complex
values.
Figure
1.1
example
a graph
on a given
similarity
function.
The
matrix
is defined
as, shows
7 withof
signal sampled
at
2
0.1378
0.3717associated
0.4700
U =6
2zero.
nodes
andcan
allItihave
other
entries
areorlaplacian
A0.6015
graph
signal
is 0.5100
an
attribute
the
node.
6
7
nodes
and all
other
entries
areshows
zero.
A graph
signal iswith
an attribute
with the node.
of a graph.
real
complex
values.
Figure
1.1
an example
a associated
graph
4
5 of
nodes
and allItother
entries
areor2zero.
A graph
signal
is
an
attribute
associated
the
node.
0.0000
0.7702
0.3069
0.0000
0.5590to
The signal
s is sent
the
base
station.
ertiescollected
like similarity
or physical
proximity.
Consider a graph represented
as
erties
G=(V,
like
E),
similarity
or
p
1 passed
2shows
N Berlekamp7 sampling
3the
2edges
Applications
the
error values,
2k syndromes
after
are
on to the
the
data
elements
form
a3n
which
are
w
gives rise to an equation which is similar to the syndrome equation in coding
theory.We
real
or complex
values.Fig
example of2a graphwhere
with
unity
weights,five
node
A graph
signal
is an
attribute
associated
with
thean
node.Mathematically,
a graph
2 7w
2nodes
3] and
2as,consisting
the2k1
2k time
slots.of When
signals
due
to
the
shift
operators.
Thei,j
adjacency
matrix
an
N6
matrix
defined
consisting
N
nodes
where
setNof⇥
[v1 ,v2 ,....,v
Esimilarity
represents
of N3physical
nodes
2k2
2kN i,j
N
show that, one of the terms in this equationeach
has a of
Vandermonde
structure
eigenval-the transmitted
w
(i,
j) superposition
2VErepresents
0 isanature
ij ofadd
7form
6 as an
erties
like
adjacency
4 ij
T
71
and
also
be
that
notion
of
adjacency
on13ainor
graph
0graph.It
0 30 method
0can
07
16
0Wolf
02 2tel
signal s=[s
is aaj)graph
scalar
function
defined
onnoted
vertices
of1the
a6
have
18, 19]signal.It
foralgorithms
graph can
signals
and
develop
Prony’s
for
them.
2an equivalent
2
=
(1.1)
1 , s2 , ..., sN ]A(i,
decoding
andbased
Prony’s
method
of6curve
fitting
was
found
by
6
7path
6
7
ues, which paves way for various coding theory
to be employed.
find
out can
0defined
0 6
1matrix
0which
1[21,
7
71are
6 0wei
i,j naturally
edges
which
areofweighted
some relation.
E
can1 [20]
be0represented
in
edges
of techniques
the wireless
medium,Tothe
receiver
obtains
a7weighted
linear
of
DSP
be ij
athespecial
case
GSP
in combination
theonsense,
signals
on
a0the
graph
Application I: Multiple Access Communication Channel
i ! Slot
j ! Node
Outline
Outline
Outline
Outline
2322
/ 29
0.3717 0.4294
0.4390
7
Mathematically,
a graph
signal s=[s1 ,0.6015
s2 , ..., sN ]T0.3505
is a scalar function defined
on vertices
T
erties
like similarity
a graph represented
G=(V,
E),
The
adjacency
matrixorisphysical
an N ⇥proximity.
N matrixConsider
defined as,
otherwise
on aasgiven
similarity
function. 0The laplacian
matrix
Applications
eneral weighted graph, wi,j can be any non-negative real
number calculated
For a general
based weighted graph, wi,j can be any non-negat
L
= Da setA
(1.2)
form Existing
as anofadjacency
matrix
a laplacian
matrix
L. These
are commonly
called
as
1.2
Framework
consisting
NGSP
nodes
where A
V or
represents
of nodes
[v1 ,v2 ,....,v
]
and
E
represents
N
en similarity function. The laplacian matrix is defined as,
on a given similarity function. The laplacian matrix is de
For a general
weighted graph, wi,j can be any non-negative rea
shiftedges
operators.
The
adjacency
is2an
N ⇥0relation.
N matrixEdefined
as,
wij matrix
(i, j) on
E
the
which
are
weighted
based
some
can
be
represented
in matrix
L=D A
A graph A(i,
is a j)
versatile
tool
to
represent
the
high
dimensional
and
complicated
data
=
(1.1)
on
a
given
similarity
function.
The laplacian matrix is defined a
D
is
a
diagonal
matrix
with
the
diagonal
entries
are
the
degree
of
the
corresponding
form
as
an
adjacency
matrix
A
or
a
laplacian
matrix
L.
These
are
commonly
called
as
0
otherwise
where the data elements form a L
node
and are
=D
A connected
(1.2) propL=D A
i ! Slot to each other based on some
z ij (i,i !j)Slot2 E 0
Channel
w
nd
all other entries
are zero.matrix
A graph
signal
isNode
an attribute
associated
with the node.
shift
The adjacency
N
matrix
defined
as, where,
j⇥
!j !
Node
D is a(1.1)
diagonal matrix with the diagonal entri
A(i, j)
= is an NConsider
ertiesoperators.
like similarity
or physical
proximity.
a graph
represented
as G=(V,
E),
L=D A
ighted graph,
wi,j signal
can be x=[x
any non-negative
calculated
based on vertices
T
0 xNotherwise
matically,
a graph
]real
is number
a scalar
function
defined
X
1 , x2 , ...,
1 with
Dconsisting
is a diagonal
theVdiagonal
entries
degree
theSLOT-1
corresponding
is all
a diagonal
matrixare
with
the diagonal
nodes
and
other entries
zero.
A graphentries
signalaris
of Nmatrix
nodes
where
represents
a setare
of the
nodes
[v+1 ,vof
] and ED
represents
2 ,....,v
Nwhere,
zdefined
arity function. The laplacian matrix is w
as,2 E 0
(i,
j)
ij
ph.
It
can
have
real
or
complex
values.
Figure
1.1
shows
an
example
of
a
graph
T a
nd
all
other
entries
are
zero.
A
graph
signal
is
an
attribute
associated
with
nodes
the
and
node.
all
other
entries
are
zero.
A
graph
signal
the
which
are weighted
represented
in matrix
a graph
signal
x=[x1entries
, x2 , ...,are
xis
]an
id
A(i,
j) = on some relation. E can
(1.1)
Nthe
For edges
a general
weighted
graph,
wbased
real be
number
D Mathematically,
is calculated
a diagonalbased
matrix
with the
diagonal
i,j can be any non-negativewhere,
T
T
matically,
a graph
signal
x=[x1A
, x2or, ...,
xNX0] is otherwise
amatrix
scalar L.
function
defined
Mathematically,
on
vertices
a
graph
signal
x=[x
,
x
,
...,
x
]
is
a
sc
SLOT-2
+
2 matrix
form
as
an
adjacency
a
laplacian
These
are
commonly
called
as
1
2
N
graph.
It can
or complex
Fig
on a given similarity function.
laplacian matrix is defined
as, andof
Base
nodes
alla(1.2)
other
entries
arehave
zero.real
A graph
signal isvalues.
an attribut
Decoder
L = DTheA
2
Station
ph.
can have The
real adjacency
or complex
values.
showsdefined
an example
a graphIt can have real or complex values.
Figure 1
shiftItoperators.
matrix
is anFigure
N ⇥ N1.1matrix
as, of aofgraph.
T
a graph
signal x=[x1 , x2 , ..., xN ] is a scalar fu
For a general weighted graph, wi,j can be any non-negativeMathematically,
real number calculated
based
2
L
=
D
A
(1.2)
gonal
matrix
with the
diagonal
are matrix
the degree
ofof
theas,
acorresponding
graph. It can have real or complex values. Figure 1.1 sho
on a given
similarity
function.
Theentries
laplacian
is defined
2 w
2
z ij (i, j) 2 E 0
er entries are zero. A graphA(i,
signal
j) =is an attribute associated with the node.
(1.1)
where, D is a diagonal matrix with the
diagonal entries are the degree of the corresponding
2
T L 0= D otherwise
(1.2)
SLOT-2k
a graph signal x=[x , x , ..., x ] is aXscalar A
function defined
on
vertices
+
Application I: Multiple Access Communication Channel
N
1
2
N
nodes and all other entries are zero. A graph signal is an attribute associated with the node.
n have real or complex values. Figure 1.1 shows an example of a graph
For a general weighted graph, wi,j can be any non-negative
real number calculated based
Mathematically,
a graph
signal
s=[s
s2 , ..., sN entries
]T is a are
scalar
on vertices
1 , diagonal
where,
D is a diagonal
matrix
with
the
thefunction
degree ofdefined
the corresponding
on a given similarity function. The laplacian matrix is defined as,
of aKey
graph.
can have
real
complex
values.
1.1 shows
an example
a graph
nodes
and points
allItother
entries
areor2zero.
A graph
signalFigure
is an attribute
associated
withofthe
node.
• Syndromes/observations
done
naturally
Mathematically,
a graph signal s=[s1 , s2 , ..., sN ]Tcomputation
is a scalar functionare
defined
on vertices
L=D
(1.2)
2 A
(no
cost)
of a graph.model
It can have
realextra
or complex
values. Figure 1.1 shows an example of a graph
by the channel
• RS decoder O(k 2 ) – independent of N
where, D is a diagonal matrix with the diagonal entries are the degree of the corresponding
2
nodes and all other entries are zero. A graph signal is an attribute associated with the node.
24 / 29
Applications
Application I: MAC Analysis
Es : Energy used by each user for each channel use in GSP model
GSP Scheme
Naive Scheme (TDMA)
• Time: N secs (N slots)
• Time: N secs (2k slots)
• Energy: N Es /ch. use
• Energy: N Es /ch. use
• Capacity
• Capacity (lattices - decode sum)
2
• Total Energy: N Es
N
2
log(1 +
N Es
2
) > N log(2⇡e↵)
Es >
• Total Energy: 3N
3 2
N
2
(linear)
• Total Energy : N 2 Es
N
4k
log( E2s ) > 1
Es > 2
• Total Energy:
N2
2
N
4k
N
4k
2
2
(exponential)
25 / 29
Applications
Application II: Anomaly Detection
26 / 29
Applications
Application II: Anomaly Detection
Problem statement
• k out of N nodes are malfunctioning x0 = x + e
• Identify the malfunctioning and correct the values
26 / 29
Applications
Application II: Anomaly Detection
+
Noise
Graph Signal
(k-sparse)
+
GFT
CS
Algorithms
Dynamic
Sampling
2k
Syndromes
Error Locations/Values
BM/PGZ
Decoder
Roots
Mapper
Error
Locations
Error Locator
Polynomial
Errors at random
locations
Forney's
Decoder
Error Values
• Errors in graph domain - sampling in frequency domain (dual problem)
• Noise robustness - Compressed Sensing
27 / 29
Applications
Conclusions/ Ongoing Work
Conclusions
• Explored connections between Coding Theory, Compressed Sensing and
Spectral Estimation for graph signals
• Shown an equivalent Prony’s method for graph signals that has sub-linear
time complexity
• Proposed two applications:
E↵ective Multiple Access Communication (MAC) strategy
Anomaly detection scheme
28 / 29
Applications
Conclusions/ Ongoing Work
Conclusions
• Explored connections between Coding Theory, Compressed Sensing and
Spectral Estimation for graph signals
• Shown an equivalent Prony’s method for graph signals that has sub-linear
time complexity
• Proposed two applications:
E↵ective Multiple Access Communication (MAC) strategy
Anomaly detection scheme
Ongoing work
• Design new sampling strategies to induce good codes
• Design less complex decoders like LDPC decoder for the spectral estimation
problem
• Analyze the case for noisy measurements/samples
28 / 29
Applications
Questions?
Thank you!
29 / 29
Joint Inference of Multiple Networks from
Stationary Graph Signals
Antonio G. Marques
King Juan Carlos University
[email protected]
http://tsc.urjc.es/~amarques/
Co-authors: Santiago Segarra and Samuel Rey-Escudero
ACK: Spanish MINECO Grant No TEC2013-41604-R
GSP Workshop – Pittsburgh, USA – June 2, 2017
Antonio G. Marques
Graph SP Workshop 2017
1 / 13
Graph SP for Network and Data Science
I
Desiderata: Process, analyze and learn
from network data [Kolaczyk’09]
Antonio G. Marques
Graph SP Workshop 2017
2 / 13
Graph SP for Network and Data Science
I
Desiderata: Process, analyze and learn
from network data [Kolaczyk’09]
I
Network as graph G = (V, E): encode pairwise relationships
I
GSP: interest not in G itself, but in data associated with nodes in V
I
Associated with G is the graph-shift operator S = V⇤VH 2 MN
) Sij = 0 for i 6= j and (i, j) 62 E (local structure in G)
) Ex: A, degree D and Laplacian L = D
Antonio G. Marques
Graph SP Workshop 2017
A matrices
2 / 13
Graph SP for Network and Data Science
I
Desiderata: Process, analyze and learn
from network data [Kolaczyk’09]
I
Network as graph G = (V, E): encode pairwise relationships
I
GSP: interest not in G itself, but in data associated with nodes in V
I
Associated with G is the graph-shift operator S = V⇤VH 2 MN
) Sij = 0 for i 6= j and (i, j) 62 E (local structure in G)
) Ex: A, degree D and Laplacian L = D
I
A matrices
Properties of signal x related to topology of G (e.g., smoothness)
) Sometimes an underlying G exists
) Sometimes G used to explain parsimonious relation among data
Antonio G. Marques
Graph SP Workshop 2017
2 / 13
Network topology inference and GSP
I
Network topology inference from nodal observations [Kolaczyk’09]
) Approaches use Pearson correlations to construct graphs [Brovelli04]
) Partial correlations and conditional dependence [Friedman08, Karanikolas16]
I
Key in neuroscience [Sporns’10]
) Functional net inferred from activity
Antonio G. Marques
Graph SP Workshop 2017
3 / 13
Network topology inference and GSP
I
Network topology inference from nodal observations [Kolaczyk’09]
) Approaches use Pearson correlations to construct graphs [Brovelli04]
) Partial correlations and conditional dependence [Friedman08, Karanikolas16]
I
Key in neuroscience [Sporns’10]
) Functional net inferred from activity
I
Early GSP works: How known graph S a↵ects signals and filters
I
Reverse path: How to use GSP to infer the graph topology?
) Gaussian graphical models [Egilmez16]
) Smooth signals [Dong15], [Kalofolias16]
) Stationary signals [Segarra16], [Pasdeloup15]
) Directed graphs [Mei-Moura15], [Shen16]
Antonio G. Marques
Graph SP Workshop 2017
3 / 13
Joint network topology inference
I
Most works (CS, NetSci, GSP) have looked at identifying a single network
I
Many contemporary setups involve multiple networks
) Same nodes, di↵erent links, each with its own observations
Ex1. Comms. nets. in dynamic environm. ) changes with time
Ex2. Brain networks of di↵erent patients
Ex3. Gene-to-gene networks of di↵erent tissues
I
Joint topology inference has received some attention
) Gauss-Markov Random Fields [Guo11, Danaher14, Ryali12, Honorio10]
) Time-varying graphs [Zhou10, Baingana17, Kalafolias17]
I
Today’s talk: how to use GSP to infer multiple networks
) Key assumption: observations stationarity on the sought nets
Antonio G. Marques
Graph SP Workshop 2017
4 / 13
Problem statement
Setup
I
(k) K
K di↵erent graphs {G (k) }K
k=1 and GSOs {S }k=1
) Same set of nodes, possibly di↵erent edges (support) and weights
I
(k)
(k)
X(k) := [x1 , ..., xRk ] 2 RN⇥Rk signals observed for G (k)
) Sample covariances ⌃(k) :=
Antonio G. Marques
1
Rk
X(k) (X(k) )T .
Graph SP Workshop 2017
5 / 13
Problem statement
Setup
I
(k) K
K di↵erent graphs {G (k) }K
k=1 and GSOs {S }k=1
) Same set of nodes, possibly di↵erent edges (support) and weights
I
(k)
(k)
X(k) := [x1 , ..., xRk ] 2 RN⇥Rk signals observed for G (k)
) Sample covariances ⌃(k) :=
1
Rk
X(k) (X(k) )T .
Problem statement
(k) K
Given observations {X(k) }K
k=1 , find topologies in {S }k=1 using:
(k)
(k)
(AS1) X are stationary in S ; and
0
0
(AS2) Graphs G (k) and G (k ) are “close” ) d(S(k) , S(k ) ) is small
Antonio G. Marques
Graph SP Workshop 2017
5 / 13
(AS1): Graph stationarity
I
x(k) is a stationary process on the unknown graph S(k)
(k)
) Observed {xi } are random realizations of x(k)
Antonio G. Marques
Graph SP Workshop 2017
6 / 13
(AS1): Graph stationarity
I
x(k) is a stationary process on the unknown graph S(k)
(k)
) Observed {xi } are random realizations of x(k)
I
Definition of graph stationarity? [Girault15, Perraudin16, Marques16]
⇥
⇤
) S(k) and cov. C(k) = E x(k) x(k)T share eigenvecs. (GFT)
) x(k) is the output of a linear di↵usion of a white input
x
(k)
= ↵0
1
Y
l=1
Antonio G. Marques
(I
↵l S
(k)
)w =
✓ NX1
hl S
l=0
Graph SP Workshop 2017
(k) l
◆
w := H(k) w
6 / 13
(AS1): Graph stationarity
I
x(k) is a stationary process on the unknown graph S(k)
(k)
) Observed {xi } are random realizations of x(k)
I
Definition of graph stationarity? [Girault15, Perraudin16, Marques16]
⇥
⇤
) S(k) and cov. C(k) = E x(k) x(k)T share eigenvecs. (GFT)
) x(k) is the output of a linear di↵usion of a white input
x
(k)
= ↵0
1
Y
(I
↵l S
(k)
)w =
l=1
I
✓ NX1
l=0
(k)
Graph stationarity ⌘ mapping S(k) ! Cx
) Correlation methods )
hl S
(k) l
(k)
Cx
=S
◆
w := H(k) w
is polynomial
(k)
(k)
) Precision methods (graphical Lasso) ! Cx = (S(k) )
(k)
) Structural EM methods ) Cx = (I
Antonio G. Marques
Graph SP Workshop 2017
S(k) )
1
(I
1
S(k) )
1
6 / 13
(AS2): Similarity among graphs
I
Graphs G (k) and G (k
) d(S
I
(k)
,S
(k 0 )
0
)
are “close”
) is small
Q1: Form of the distance function d(·, ·)
) kvec(S(k)
) kvec(S(k)
0
S(k ) )kp with p = 0/1 same support and weights
0
S(k ) )k22 similar weights
) Same of them (and kS(k)
Antonio G. Marques
0
S(k ) k1,1 ) used in graphical lasso
Graph SP Workshop 2017
7 / 13
(AS2): Similarity among graphs
I
Graphs G (k) and G (k
) d(S
I
(k)
,S
(k 0 )
0
)
are “close”
) is small
Q1: Form of the distance function d(·, ·)
) kvec(S(k)
) kvec(S(k)
0
S(k ) )kp with p = 0/1 same support and weights
0
S(k ) )k22 similar weights
) Same of them (and kS(k)
I
0
S(k ) k1,1 ) used in graphical lasso
Q2: Determining the proximity degree among the di↵erent graphs
) Weighted and directed graph GK (graph of graphs)
) Node set is the K GSOs and Wk,k 0 represents GSO similarity
Ex1. k indexes time (dyn. environm.): GK directed path
Ex2. k indexes patients with a particular disease: GK full graph
Antonio G. Marques
Graph SP Workshop 2017
7 / 13
Problem formulation
I
We can use extra knowledge/assumptions to choose the graphs
) Of all graphs, select one that is optimal in some sense
min
{S(k) }K
k=1
s. t.
Antonio G. Marques
X
↵k kS(k) k0 +
⌃
S
k
(k) (k)
=S
(k)
⌃
X
k,k 0
(k)
Wk,k 0 kS(k)
0
S(k ) k0
, S(k) 2 S (k) , for all k.
Graph SP Workshop 2017
(1)
8 / 13
Problem formulation
I
We can use extra knowledge/assumptions to choose the graphs
) Of all graphs, select one that is optimal in some sense
min
{S(k) }K
k=1
s. t.
I
X
↵k kS(k) k0 +
⌃
S
k
(k) (k)
=S
(k)
⌃
X
k,k 0
(k)
0
Wk,k 0 kS(k)
S(k ) k0
, S(k) 2 S (k) , for all k.
(1)
Set S (k) contains admissible matrices (e.g. adjacency or Laplacian)
S (k) := {S | Sij
0, S 2 MN, Sii = 0,
P
j
S1j = 1}
I
Properties of the sough graphs: promote vs. enforce
I
Stationarity in single-net topo-id [Segarra16]: eigenv(S(k) ) = eigenv(⌃(k) )
Antonio G. Marques
Graph SP Workshop 2017
8 / 13
Solving the optimization
I
`0 -norm renders problem (1) non-convex
) Size of the feasibility set small ) Easier optimization
) Approach relax to `1 -norm minimization, e.g., [Tropp’06]
) Key constraint ) observations are stationary on the GSO
min
{S(k) }K
k=1
s. t.
I
X
k
↵k kS(k) k1 +
X
k,k 0
Wk,k 0 kS(k)
0
S(k ) k1
⌃(k) S(k) = S(k) ⌃(k) , S(k) 2 S (k) , for all k.
(2)
Guarantees for solution to (2) to coincide with solution to (1)?
Antonio G. Marques
Graph SP Workshop 2017
9 / 13
Recovery guarantees
B and Z incidence matrices 2 {0, 1, 1}
Definitions
⌃ := diag([ ⌃(1)
T
⌃(1) , . . . , ⌃(K )
⌃(K ) ])
T
T
:= [IK ⌦ BT , IK ⌦ [IN 2 ]T
D , ⌃ , (e1 ⌦ 1N )]
T
Theorem
:= [diag(↵), ZT diag(W)]T ⌦ IN 2
The solutions to (1) and (2) coincide if:
1) [ T ]J is full row rank; and
2) There exists a constant > 0 such that
:= k
I
I
Lc (
2
T
+
T
Lc
Lc )
1
T
L kM(1) <
1.
Cond. 1) ensures solution is unique
Cond. 2) guarantees existence of a dual certificate for `0 optimality
Antonio G. Marques
Graph SP Workshop 2017
10 / 13
Recovery guarantees
B and Z incidence matrices 2 {0, 1, 1}
Definitions
⌃ := diag([ ⌃(1)
T
⌃(1) , . . . , ⌃(K )
⌃(K ) ])
T
T
:= [IK ⌦ BT , IK ⌦ [IN 2 ]T
D , ⌃ , (e1 ⌦ 1N )]
T
Theorem
:= [diag(↵), ZT diag(W)]T ⌦ IN 2
The solutions to (1) and (2) coincide if:
1) [ T ]J is full row rank; and
2) There exists a constant > 0 such that
:= k
I
I
I
I
Lc (
2
T
+
T
Lc
Lc )
1
T
L kM(1) <
1.
Cond. 1) ensures solution is unique
Cond. 2) guarantees existence of a dual certificate for `0 optimality
Q1: Robust recovery for noisy observations?
Q2: Statistical interpretation, consistent estimator?
Antonio G. Marques
Graph SP Workshop 2017
10 / 13
Inference from sample covariances
I
White signal di↵used by a di↵erent types of filters
) Separate id vs joint (K = 2)
) N = 50, on average graphs di↵er on 4 links
) Mean (left) vs. median (right) error
I
Joint topology better performance and more robust
I
Error decreases with increasing number of observed signals
) When very high, error performance comparable
Antonio G. Marques
Graph SP Workshop 2017
11 / 13
Inference from noisy graphs
I
White signal di↵used by a di↵erent types of filters
) E-R graphs with Gaussian noise
) Tested schemes: Separate id, K = 2 joint, K = 3 joint
) Graph similarity: 2 (left) vs. 10 (right) edges
I
Error decreases with increasing SNR
I
If graphs are indeed similar joint recovery helps
) Gains smaller as SNR increases
I
If graphs are not very similar joint recovery only helps for low SNR
Antonio G. Marques
Graph SP Workshop 2017
12 / 13
Closing remarks
I
Network topology inference cornerstone problem in network science
I
I
I
Early GSP works analyze how S a↵ect signals and filters
More recent how to use GSP to infer the graph topology?
Our goal here to use GSP for joint inference of multiple networks
(AS1) Signals are stationary
(AS2) Similarity/distance between pairs of networks is known
Antonio G. Marques
Graph SP Workshop 2017
13 / 13
Closing remarks
I
Network topology inference cornerstone problem in network science
I
I
Early GSP works analyze how S a↵ect signals and filters
More recent how to use GSP to infer the graph topology?
I
Our goal here to use GSP for joint inference of multiple networks
(AS1) Signals are stationary
(AS2) Similarity/distance between pairs of networks is known
I
Approach formulate a sparse recovery problem
I
I
Antonio G. Marques
Keys: stationary constraints and topological priors
`1 relaxation with recovery guarantees
Graph SP Workshop 2017
13 / 13
Closing remarks
I
Network topology inference cornerstone problem in network science
I
I
Early GSP works analyze how S a↵ect signals and filters
More recent how to use GSP to infer the graph topology?
I
Our goal here to use GSP for joint inference of multiple networks
(AS1) Signals are stationary
(AS2) Similarity/distance between pairs of networks is known
I
Approach formulate a sparse recovery problem
I
I
I
Keys: stationary constraints and topological priors
`1 relaxation with recovery guarantees
Synthetic simulations confirm recovery gains
I
I
I
Antonio G. Marques
Real-data simulations going on
Additional theoretical results (consistency, robustness)
Low complexity algorithms for large N or K
Graph SP Workshop 2017
13 / 13
Network Topology Inference
from Non-stationary Graph Signals
Gonzalo Mateos
Dept. of ECE and Goergen Institute for Data Science
University of Rochester
[email protected]
http://www.ece.rochester.edu/~gmateosb/
Co-authors: Rasoul Shafipour, Santiago Segarra, and Antonio G. Marques
GSP Workshop, CMU, June 2, 2017
Blind Identification of Graph Filters
GSP Workshop 2017
1
Network Science analytics
Onlinesocialmedia
Internet
Cleanenergyandgridanaly,cs
I
Network as graph G = (V, E): encode pairwise relationships
I
Desiderata: Process, analyze and learn from network data [Kolaczyk’09]
Blind Identification of Graph Filters
GSP Workshop 2017
2
Network Science analytics
Onlinesocialmedia
Cleanenergyandgridanaly,cs
Internet
I
Network as graph G = (V, E): encode pairwise relationships
I
Desiderata: Process, analyze and learn from network data [Kolaczyk’09]
I
Interest here not in G itself, but in data associated with nodes in V
I
) The object of study is a graph signal
Ex: Opinion profile, bu↵er congestion levels, neural activity, epidemic
Blind Identification of Graph Filters
GSP Workshop 2017
3
Graph signal processing (GSP)
I
) Aij = Proximity between i and j
I
x2
Undirected G with adjacency matrix A
Define a signal x on top of the graph
4
3
5
x1
1
) xi = Signal value at node i
Blind Identification of Graph Filters
x4
2
x3
GSP Workshop 2017
x5
4
Graph signal processing (GSP)
I
) Aij = Proximity between i and j
I
x2
Undirected G with adjacency matrix A
Define a signal x on top of the graph
4
3
5
x1
1
) xi = Signal value at node i
I
x4
2
x3
x5
Associated with G is the graph-shift operator S = V⇤VT 2 MN
) Sij = 0 for i 6= j and (i, j) 62 E (local structure in G )
) Ex: A, degree D and Laplacian L = D
Blind Identification of Graph Filters
A matrices
GSP Workshop 2017
5
Graph signal processing (GSP)
I
) Aij = Proximity between i and j
I
x2
Undirected G with adjacency matrix A
Define a signal x on top of the graph
3
5
1
x3
x5
Associated with G is the graph-shift operator S = V⇤VT 2 MN
) Sij = 0 for i 6= j and (i, j) 62 E (local structure in G )
) Ex: A, degree D and Laplacian L = D
I
4
x1
) xi = Signal value at node i
I
x4
2
A matrices
Graph Signal Processing ! Exploit structure encoded in S to process x
) Our view: GSP well suited to study (network) di↵usion processes
I
Take the reverse path. How to use GSP to infer the graph topology?
Blind Identification of Graph Filters
GSP Workshop 2017
6
Topology inference: Motivation and context
I
Network topology inference from nodal observations [Kolaczyk’09]
I
I
I
Partial correlations and conditional dependence [Dempster’74]
Sparsity [Friedman et al’07] and consistency [Meinshausen-Buhlmann’06]
Key in neuroscience [Sporns’10]
) Functional net inferred from activity
Blind Identification of Graph Filters
GSP Workshop 2017
7
Topology inference: Motivation and context
I
Network topology inference from nodal observations [Kolaczyk’09]
I
I
I
Partial correlations and conditional dependence [Dempster’74]
Sparsity [Friedman et al’07] and consistency [Meinshausen-Buhlmann’06]
Key in neuroscience [Sporns’10]
) Functional net inferred from activity
I
Noteworthy GSP-based approaches
I
I
I
I
I
Gaussian graphical models [Egilmez et al’16]
Smooth signals [Dong et al’15], [Kalofolias’16]
Stationary signals [Pasdeloup et al’15], [Segarra et al’16]
Directed graphs [Mei-Moura’15], [Shen et al’16]
Our contribution: topology inference from non-stationary graph signals
Blind Identification of Graph Filters
GSP Workshop 2017
8
Generating structure of a di↵usion process
I
Signal y is the response of a linear di↵usion process to an input x
y = ↵0
1
Y
l=1
(I
↵l S)x =
1
X
lS
l
x
l=0
) Common generative model. Heat di↵usion if ↵k constant
I
We say the graph shift S explains the structure of signal y
Blind Identification of Graph Filters
GSP Workshop 2017
9
Generating structure of a di↵usion process
I
Signal y is the response of a linear di↵usion process to an input x
y = ↵0
1
Y
(I
↵l S)x =
l=1
1
X
lS
l
x
l=0
) Common generative model. Heat di↵usion if ↵k constant
I
We say the graph shift S explains the structure of signal y
I
Cayley-Hamilton asserts we can write di↵usion as
y=
✓ NX1
l=0
h l Sl
◆
x := Hx
) Graph filter H is shift invariant [Sandryhaila-Moura’13]
) H diagonalized by the eigenvectors V of the shift operator
Blind Identification of Graph Filters
GSP Workshop 2017
10
Our approach for topology inference
I
Two-step approach for graph topology identification
Signal realizations
or their statistics
Inferred network S
A priori info, desired
topological features
Step 1:
Identify the eigenvectors
of the shift
I
Inferred eigenvectors V
Step 2:
Identify eigenvalues to
obtain a suitable shift
Beyond di↵usion ! Alternative sources for spectral templates V
I
I
Design of graph filters [Segarra et al’15]
Graph sparsification and network deconvolution [Feizi et al’13]
Blind Identification of Graph Filters
GSP Workshop 2017
11
Step 2: Obtaining the eigenvalues
I
We can use extra knowledge/assumptions to choose one graph
) Of all graphs, select one that is optimal in some sense
S⇤ := argmin f (S, )
S,
I
s. to S =
N
X
T
k vk vk ,
k=1
S2S
Set S contains all admissible scaled adjacency matrices
P
S := {S | Sij 0, S 2 MN, Sii = 0,
j S1j = 1}
) Can accommodate Laplacian matrices as well
Blind Identification of Graph Filters
GSP Workshop 2017
12
Step 2: Obtaining the eigenvalues
I
We can use extra knowledge/assumptions to choose one graph
) Of all graphs, select one that is optimal in some sense
S⇤ := argmin f (S, )
S,
I
s. to S =
N
X
T
k vk vk ,
k=1
S2S
Set S contains all admissible scaled adjacency matrices
P
S := {S | Sij 0, S 2 MN, Sii = 0,
j S1j = 1}
) Can accommodate Laplacian matrices as well
I
Problem is convex if we select a convex objective f (S, )
Ex: Sparsity (f (S) = kSk1 ), min. energy (f (S) = kSkF ), mixing (f ( ) =
I
2)
Robust recovery from imperfect or incomplete V̂ [Segarra et al’16]
Blind Identification of Graph Filters
GSP Workshop 2017
13
Step 1: Obtaining the eigenvectors
Stationary graph signal [Marques et al’16]
Def: A graph signal y is stationary P
with respect to the shift S if
L 1
and only if y = Hx, where H = l=0 hl Sl and x is white.
Blind Identification of Graph Filters
GSP Workshop 2017
14
Step 1: Obtaining the eigenvectors
Stationary graph signal [Marques et al’16]
Def: A graph signal y is stationary P
with respect to the shift S if
L 1
and only if y = Hx, where H = l=0 hl Sl and x is white.
I
I
The covariance matrix of the stationary signal y is
h
i
⇥
⇤
T
Cy = E Hx Hx
= HE xxT HT = HHT
Key: Since H is diagonalized by V, so is the covariance Cy
Cy = V
L 1
X
2
h l ⇤l
VT
l=0
) Estimate V from {yi } via Principal Component Analysis
Blind Identification of Graph Filters
GSP Workshop 2017
15
Non-stationary graph signals
I
Q: What if the signal y = Hx is not stationary (i.e., x colored)?
) Matrices S and Cy no longer simultaneously diagonalizable since
Cy = HCx HT
Blind Identification of Graph Filters
GSP Workshop 2017
16
Non-stationary graph signals
I
Q: What if the signal y = Hx is not stationary (i.e., x colored)?
) Matrices S and Cy no longer simultaneously diagonalizable since
Cy = HCx HT
I
Key: still H =
PL
1
l=0
hl Sl diagonalized by the eigenvectors V of S
) Infer V by estimating the unknown di↵usion (graph) filter H
) Step 1 boils down to system identification + eigendecomposition
I
Leverage di↵erent sources of information on the input signal x
(a) Input-output graph signal realization pairs {ym , xm }
(b) Input covariance Cx and positive semidefinite filter H < 0
(c) Input covariance Cx and generic filter H
Blind Identification of Graph Filters
GSP Workshop 2017
17
Input-output graph signal realization pairs
I
Consider M di↵usion processes on G , where ym = Hxm (xm colored)
) Assume that realizations {ym , xm }M
m=1 are available
I
Filter H and, as byproduct, its eigenvectors V can be estimated as
Ĥ = argmin
H
I
M
X
m=1
kym
2
Hxm k
Define X = [x1 , . . . , xM ] and Y = [y1 , . . . , yM ]. Then, Ĥ given by
vec(Ĥ) = (XT )† ⌦ IN vec(Y)
) If M
Blind Identification of Graph Filters
N and X is full rank, the minimizer Ĥ is unique
GSP Workshop 2017
18
Inferring a brain network
I
Consider a structural brain graph with N = 66 neural regions
I
I
I
P
Signals di↵used either by H1 = 2l=0 hl Al or H2 = (I + ↵A)
M
Observe realizations {ym , xm }m=1 and vary M
Also noisy signals ym = Hi xm + wm , with wm ⇠ N (0, 10
10
100
Recovery Error
Recovery Error
100
-2
10-4
10-6
10
Noiseless H1
Noiseless H2
20
30
40
50
60
70
Recovery error kA
I)
Noisy H1
Noisy H2
10-1
10-2
100
M
I
2
1
200
300
400
500
M
ÂkF /kAkF small for M
66, even with noise
) Performance roughly independent of the filter type
Blind Identification of Graph Filters
GSP Workshop 2017
19
Input covariance and positive semidefinite filters
I
Realizations of the input may be challenging to acquire
) Consider instead that Cx,m = E[xm xm T ] are known
(p)
m
) Estimate output covariance Ĉy,m from realizations {ym }Pp=1
I
Goal is to find H such that Ĉy,m and HCx,m HT are close
) Least squares yields a fourth-order cost in H ! Challenging
Blind Identification of Graph Filters
GSP Workshop 2017
20
Input covariance and positive semidefinite filters
I
Realizations of the input may be challenging to acquire
) Consider instead that Cx,m = E[xm xm T ] are known
(p)
m
) Estimate output covariance Ĉy,m from realizations {ym }Pp=1
I
Goal is to find H such that Ĉy,m and HCx,m HT are close
I
) Least squares yields a fourth-order cost in H ! Challenging
P1
Assume H is PSD, e.g, in Laplacian di↵usion y = ( l=0 l Ll )x, l > 0
) Well-defined square roots, hence H can be identified as
Ĥ = argmin
M
X
H2MN
++ m=1
I
k(Cx,m 1/2 Ĉy,m Cx,m 1/2 )1/2
Cx,m 1/2 HCx,m 1/2 k2F
If Cy ,1 known, even with M = 1 PSD assumption renders H identifiable
Blind Identification of Graph Filters
GSP Workshop 2017
21
Inferring Zachary’s karate club network
I
Social network with N = 34 club members
I
I
I
1
Model opinion di↵usion with S = I ↵L, where ↵ = max
(L)
For M = 1, 5, 10 input covariances Cx,m assumed given
(p)
m
Estimate Cy ,m from {ym }Pp=1
via sample averaging, varying Pm
0.4
Recovery Error
0.35
0.3
0.25
0.2
0.15
0.1
0.05
1
10
M=1
M=5
M=10
10
2
10
3
10
4
10
5
Number of Observations
I
With imperfect estimates Ĉy,m , performance improves with M
Blind Identification of Graph Filters
GSP Workshop 2017
22
Input covariance and generic filters
I
Q: What about identifying a generic symmetric filter H?
I
Filter is no longer PSD, square roots not prudent ) Try to solve
Ĥ = argmin
H2MN
I
M
X
m=1
kĈy,m
HCx,m HT k2F
Non-convex problem can be tackled by gradient descent or ADMM
{H⇤L , H⇤R } = argmin
M
X
HL ,HR 2MN m=1
||Cy,m
HL Cx,m HR T ||2F
s. to HL = HR
) In general, identifiability cannot be guaranteed. Larger M helps
Blind Identification of Graph Filters
GSP Workshop 2017
23
Inferring a brain network
I
Consider a structural brain P
graph with N = 66 neural regions
2
l
I
I
Signals di↵used by H =
l=0
hl A , hl ⇠ U [0, 1]
Performance comparison against counterpart in [Segarra et al’16]
I
Assumes ym stationary ) Estimates V directly from Ĉy,m
0.3
Recovery Error
0.25
0.2
0.15
0.1
0.05
0
Proposed Method
Stationary Method
2
4
6
8
10
12
M
I
Error decays with M, almost all edges in S recovered for M = 9
) Outperforms algorithm agnostic to signal non-stationarities
Blind Identification of Graph Filters
GSP Workshop 2017
24
Closing remarks
I
Network topology inference from di↵used non-stationary graph signals
I
I
Graph shift S and covariance Cy are not simultaneously diagonalizable
Di↵usion filter H and graph shift S still share spectral templates V
) Two step approach for topology inference
i) Obtain Ĥ ) V̂; ii) Given V̂, estimate Ŝ via convex optimization
I
Estimate Ĥ under di↵erent settings
I
I
I
Input-output graph signal realization pairs {ym , xm }
Input covariance Cx and positive semidefinite filter H < 0
Input covariance Cx and generic filter H
Blind Identification of Graph Filters
GSP Workshop 2017
25
Closing remarks
I
Network topology inference from di↵used non-stationary graph signals
I
I
Graph shift S and covariance Cy are not simultaneously diagonalizable
Di↵usion filter H and graph shift S still share spectral templates V
) Two step approach for topology inference
i) Obtain Ĥ ) V̂; ii) Given V̂, estimate Ŝ via convex optimization
I
Estimate Ĥ under di↵erent settings
I
I
I
I
Input-output graph signal realization pairs {ym , xm }
Input covariance Cx and positive semidefinite filter H < 0
Input covariance Cx and generic filter H
Ongoing work and future directions
I
I
I
Identifiability and convergence guarantees for generic H
Extensions to directed graphs
Inference of time-varying networks
Blind Identification of Graph Filters
GSP Workshop 2017
26
Optimal Graph Filter for Estimating
the Mean of a WSS Graph Process
Fernando Gama & Alejandro Ribeiro
Dept. of Electrical and Systems Engineering
University of Pennsylvania
[email protected]
GSP Workshop, June 2, 2017
Gama, Ribeiro
Optimal Graph Filter for Estimating the Mean
1/19
Stochastic Processes: Stationarity and Ergodicity
I
Stochastic processes are essential to model random phenomena
) Extract useful information from the available (noisy) data
I
Stationarity ) Conditions on probability distribution of process
) Wide Sense Stationarity (WSS) ) First, second order moments
) Mean and covariance completely characterize the process
I
Ergodicity ) Infer parameters from a single realization
) Fundamental task in statistical signal processing
) Accurate modeling of random phenomena under study
Gama, Ribeiro
Optimal Graph Filter for Estimating the Mean
2/19
Ergodicity in Time and Law of Large Numbers
I
I
Ergodicity ) Realization averaging converge to ensemble averaging
Law of Large Numbers ) Sample mean µ̂n converges to true mean µ
|µ̂n
I
µ| = O
✓
1
p
n
◆
;
µ̂n =
n
n 1
1X
1X t
xk 1 =
Sx
n
n t=0
k=1
On directed cycle (time) ) Aggregation of information ) Shifting
) Combine information ) Add shifts and adequately rescale
P5
x + Adc x
t=0
x1
x1 + x6
x6
x2
x5
x3
x4
Gama, Ribeiro
x6 + x5
x2 + x1
x5 + x4
x3 + x2
Atdc x
x4 + x3
Optimal Graph Filter for Estimating the Mean
P
P
xk
xk
P
P
xk
xk
P
P
xk
xk
3/19
Ergodicity in Stationary Graph Processes
I
Extend the notion of ergodicity to WSS graph processes
) Unbiased estimator of the mean by di↵usion (shifting)
) Consistency under some conditions on the graph spectra
µ̂n = c ·
n 1
X
t
Sx
|[µ̂n
,
t=0
µ]` | = O
✓
1
p
n
◆
(mostly)
) Reminiscent of Weak Law of Large Numbers (WLLN)
I
Design an optimal graph filter ) Works on every graph
5
5
5
10
10
10
15
15
15
20
20
25
10
15
20
25
Sample average
Gama, Ribeiro
20
25
5
25
5
10
15
20
25
Single realization
Optimal Graph Filter for Estimating the Mean
5
10
15
20
25
Di↵usion estimator
4/19
Graph Fourier Transform
I
I
I
I
I
I
Weighted graph G = (V, E, W) with n nodes ) Irregular support
Graph signal x 2 Rn ) Data value on each node
Graph shift operator S 2 Rn⇥n ) Captures local structure in G
Assume the graph shift operator is normal ) S = V⇤VH
Project graph signal onto eigenbasis ) x̃ = VH x
) Defined as the graph Fourier transform (GFT)
Linear combination of eigenvectors weighted by GFT coefficients
) x = Vx̃ ) Inverse graph Fourier transform (iGFT)
Gama, Ribeiro
Optimal Graph Filter for Estimating the Mean
5/19
Linear Shift-Invariant Graph Filters (LSI-GF)
I
I
I
Graph filter H : Rn ! Rn ) Map between graph signals
Consider filters that are linear ) H is a n ⇥ n matrix
Polynomial in S of degree b
1 with coefficients h = [h0 , . . . , hb
H = h0 I + h1 S + · · · + hb
b
1S
1
I
Linear shift-invariant graph filters (LSI-GF)
I
GFT of filter depends on eigenvalues ) h̃ =
b 1
X
T
h ` S`
`=0
) Distributed implementation ) Only up to b-hop information
) With [ ]k,` =
Gama, Ribeiro
=
1]
` 1
k
2C
n⇥b
h 2 Cn
Vandermonde matrix
Optimal Graph Filter for Estimating the Mean
6/19
WSS Graph Processes
I
I
Graph G = (V, E, W) with n nodes and GSO S = V⇤VH (normal)
Probability space (⌦, F, P) ) Random vector x : ⌦ ! Rn
) [x]k random variable on each node of G
) Mean µ = E[x] and covariance matrix CX = E[(x
I
µ)(x
µ)H ]
WSS impose statistical structure related to underlying support
) E[x] = µvm where vm eigenvector of S
) Cx = Vdiag(p)VH ) p : PSD ) Cx̃ = diag(p)
) Covariance matrix and GSO are simultaneously diagonalizable
Gama, Ribeiro
Optimal Graph Filter for Estimating the Mean
7/19
The Mean of a WSS Graph Process
I
Traditional SP ) Mean is DC (constant) component of signal
I
GSP ) Find the slowest node-varying eigenvector ) vm
I
) Contribution of zero-frequency coefficient (slowest time-varying)
Use concept of total variation (TV) to find vm
TV (x) =
n
X
k=1
I
Ordering )
max
xk
X w`,k
x` = x
| max |
1
|
`2Nk
max |
AT x
1
is real and positive for connected graphs
) vmax is the slowest node-varying eigenvector ) vm = vmax
I
I
Gama, Ribeiro
) TV increases as eigenvalues are located further away from
max
Eigenvector vmax has positive elements ) Number of zero-crossings
Order eigenvalues from slowest to fastest )
Optimal Graph Filter for Estimating the Mean
1
=
max ,
2, . . . ,
n
8/19
Unbiased Di↵usion Estimator
I
Discrete-time estimator ) Directed cycle ) Shifting
n
1X
n
I
k=1
n 1
n 1
1X t
1X t
xk 1 =
Adc x =
Sx
n t=0
n t=0
Extend to GSP ) Estimate µ di↵using single realization
µ̂n = Pn
1
1
t=0
I
t
1 t=0
St x
Since
Gama, Ribeiro
Pn
1
PSD q : qk = pk Pt=0
n 1
| t=0
1
|
k|
t 2
k|
t |2
1
,
x2
x5
x3
x4
x + Adc x
x1 + x 6
x6 + x5
x2 + x1
x5 + x4
x3 + x2
x4 + x 3
Unbiased ) Covariance matrix Cµ̂ = Vdiag(q)VH
|
I
n 1
X
x1
x6
k = 1, . . . , n
) Di↵usion estimator acts as LPF
Optimal Graph Filter for Estimating the Mean
P5
t
t=0 Adc x
P
xk
P
xk
P
xk
P
xk
P
xk
P
xk
9/19
Graph Weak Law of Large Numbers
Theorem: Weak Law of Large Numbers for WSS graph processes
Assume | k |/ 1 = o(n /2n ), > 0 or 1 = 1. Then,
min P (|[µ̂n
`=1,...,n
I
I
Gama, Ribeiro
1 p1
·
+ o(n
n ✏2
)
Bound error of estimating mean at node `
P (|[µ̂n
I
µ]` | > ✏) 
µ]` | > ✏) 
n
1 X
qk |v`,k |2
✏2
k=1
) Depends on estimator PSD q and on rows of V (orthonormal)
Under assumptions ) q1 = p1 and qk = o(n
) for k = 2, . . . , n
Directed cycle and Erdős-Rényi graphs satisfy this condition
Optimal Graph Filter for Estimating the Mean
10/19
Unbiased Graph Filter Estimator
I
Di↵usion estimator is a LSI graph filter with constant taps
ht = Pn
1
1
t=0
I
Consider a general unbiased LSI graph filter estimator
zn = P n
I
Gama, Ribeiro
n 1
X
1
1
t=0
I
t
1
ht t1 t=0
h t St x
Covariance matrix Cz = Vdiag(r)VH ) Recall h̃ =
P
|
ht
PSD r : rk = pk Pt=0
| t=0 ht
t 2
k|
t |2 =
1
pk
|h̃k |2
,
|h̃1 |2
h filter GFT
k = 1, . . . , n
PSD rescaled by normalized GFT coefficients of filter
Optimal Graph Filter for Estimating the Mean
11/19
Optimal Estimator: Minimize the MSE
I
Mean squared error (MSE) of the unbiased estimator
tr[Cz ] =
n
X
k=1
rk =
n
X
k=1
pk
|h̃k |2
|h̃1 |2
Proposition: Optimal filter
The GFT of the filter taps that minimize the MSE are given by
h̃1 6= 0 ,
h̃k = 0 , k = 2, . . . , n
so that the MSE is tr[Cz ] = p1 .
I
Gama, Ribeiro
Attenuates all frequencies except for the DC component
Optimal Graph Filter for Estimating the Mean
12/19
Optimal Estimator: Consistency
Theorem: Consistency of optimal filter
The error probability of the optimal graph filter zn at some node is
min P (|[zn
`=1,...,n
µ]` | > ✏) 
1 p1
·
n ✏2
for any graph a WSS process can be defined on.
I
Bound error of estimating mean at node `
P (|[zn
I
Gama, Ribeiro
µ]` | > ✏) 
n
1 X
rk |v`,k |2
✏2
k=1
For optimal filter ) r1 = p1 and rk = 0 for k = 2, . . . , n
Optimal Graph Filter for Estimating the Mean
13/19
Numerical Example: Erdős-Rényi Graph
I
Erdős-Rényi Graph of size n and probability p = 0.2
I
Set µ = 3, SNR = 10 log10 (µ2 /p1 ) = 10dB, 50 graphs per size
I
Gaussian WSS graph process ) 105 realizations per graph
100
100
Di,usion Estimator
Bound
Optimal Estimator
Bound
10!1
Probability of error
Probability of error
10!1
10!2
10!3
10!4
10
10!2
10!3
13
17
22
28
36
46
60
77
100
10!4
10
13
17
22
n
Di↵usion Estimator
I
Gama, Ribeiro
28
36
46
60
77
100
n
Optimal Estimator
The probability of error decreases as n increases (WLLN)
Optimal Graph Filter for Estimating the Mean
14/19
Numerical Example: Erdős-Rényi Graph
I
Erdős-Rényi Graph of size n and probability p = 0.2
I
Set µ = 3, SNR = 10 log10 (µ2 /p1 ) = 10dB, 50 graphs per size
I
Gaussian WSS graph process ) 105 realizations per graph
100
Probability of error
10
Optimal
Optimal (Bound)
Di,usion
Di,usion (bound)
!1
10!2
10!3
10!4
10
13
17
22
28
36
46
60
77
100
n
Comparison
I
Gama, Ribeiro
For large n the di↵usion and the optimal estimators coincide
Optimal Graph Filter for Estimating the Mean
14/19
Numerical Example: Stochastic Block Model
I
Stochastic Block Model of size n with 4 communities, prob. 0.6, 0.1
I
Set µ = 3, SNR = 10 log10 (µ2 /p1 ) = 10dB, 50 graphs per size
I
Gaussian WSS graph process ) 105 realizations per graph
100
100
Di,usion Estimator
Bound
10
Probability of error
Probability of error
10
Optimal Estimator
Bound
!1
10!2
10!3
10!4
10
!1
10!2
10!3
13
17
22
28
36
46
60
77
100
10!4
10
13
17
22
n
Di↵usion Estimator
I
Gama, Ribeiro
28
36
46
60
77
100
n
Optimal Estimator
Similar behavior to ER graphs ) Error decrease as n increases
Optimal Graph Filter for Estimating the Mean
15/19
Numerical Example: Stochastic Block Model
I
Stochastic Block Model of size n with 4 communities, prob. 0.6, 0.1
I
Set µ = 3, SNR = 10 log10 (µ2 /p1 ) = 10dB, 50 graphs per size
I
Gaussian WSS graph process ) 105 realizations per graph
100
Probability of error
10
Optimal
Optimal (Bound)
Di,usion
Di,usion (bound)
!1
10!2
10!3
10!4
10
13
17
22
28
36
46
60
77
100
n
Comparison
I
Gama, Ribeiro
Similar behavior to ER graphs ) Both estimators coincide
Optimal Graph Filter for Estimating the Mean
15/19
Numerical Example: Covariance Graph
I
Zero-mean Gaussian vectors of size n with covariance matrix ⌃
I
Generate 106 training samples ) Estimate ⌃ ) Adopt as GSO
I
Vary n ) Generate 50 graphs per n ) Generate WSS process
100
100
Di,usion Estimator
Bound
Optimal Estimator
Bound
10!1
Probability of error
Probability of error
10!1
10!2
10!3
10!4
10!5
10
10!2
10!3
10!4
13
17
22
28
36
46
60
77
100
10!5
10
13
17
22
n
Di↵usion Estimator
I
Gama, Ribeiro
28
36
46
60
77
100
n
Optimal Estimator
The probability of error decreases as n increases
Optimal Graph Filter for Estimating the Mean
16/19
Numerical Example: Covariance Graph
I
Zero-mean Gaussian vectors of size n with covariance matrix ⌃
I
Generate 106 training samples ) Estimate ⌃ ) Adopt as GSO
I
Vary n ) Generate 50 graphs per n ) Generate WSS process
100
Optimal
Optimal (Bound)
Di,usion
Di,usion (bound)
Probability of error
10!1
10!2
10!3
10!4
10!5
10
13
17
22
28
36
46
60
77
100
n
Comparison
I
Gama, Ribeiro
Optimal estimator yields a better performance
Optimal Graph Filter for Estimating the Mean
16/19
Numerical Example: Gaussian-Markov Random Field
I
Sensor measurements contaminated by spatially correlated noise
) Estimate mean of Gaussian-Markov Random Field (GMRF)
) GMRF is WSS on the sensor network graph
I
Consider the mean to be µ = µ · v1 with µ = 3
I
2, 000 sensors ) SNR = 10 · log10 (µ2 /p1 ) = 10 dB
I
True mean field from averaging 105 realizations
Di↵usion estimator resembles measurements of true mean field
1
1
0.9
0.9
0.8
0.8
0.8
0.7
0.7
0.7
0.6
0.6
0.6
0.5
0.5
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.2
0.2
0.2
0.1
0.1
0
0
0
Gama, Ribeiro
x2
1
0.9
x2
x2
I
0.2
0.4
0.6
0.8
1
0.1
0
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
x1
x1
x1
Sensor measurements
Di↵usion estimator
True mean field
Optimal Graph Filter for Estimating the Mean
1
17/19
Conclusions
I
Extended a notion of ergodicity to WSS graph processes
I
Consistent unbiased estimator of the mean
) Consistency shown under some conditions on the graph spectra
) Reminiscent of weak law of large numbers
I
Computed by di↵using a single realization ) Ergodicity
I
Designed optimal graph filter that minimizes MSE
) Consistency shown for any underlying graph support
I
Applied to ER graphs, SBM and covariance graphs
I
Applied to estimating the mean of a GMRF
Gama, Ribeiro
Optimal Graph Filter for Estimating the Mean
18/19
Length of the Filter
I
I
Stochastic Block Model and Covariance graphs ) Fixed n = 20
Function of the length of the filter (number of filter taps) b
µ̂b = Pb
b 1
X
1
1
t=0
t
1 t=0
St x , lim
min P (|[µ̂b
µ]` | > ✏) 
b!1 `=1,...,n
100
1 p1
·
n ✏2
100
Probability of error
Di,usion Estimator
Bound
Probability of error
Di,usion Estimator
Bound
10!1
10!1
10!2
10
20
30
40
50
60
70
80
90
100
10
20
30
40
50
b
SBM
I
Gama, Ribeiro
60
70
80
90
100
b
Covariance
After b > 20 the estimator does not get any better
Optimal Graph Filter for Estimating the Mean
19/19
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Congestion Detection and Traffic Prediction in Transportation
Networks Using Graph Signal Processing
†Arman Hasanzadeh, †Xi Liu, †Krishna Narayanan, †Nick Duffield
§Byron Chigoy, §Shawn Turner
†Department of Electrical and Computer Engineering
Texas A&M University
§Texas A&M Transportation Institute
GSP Workshop at CMU
June 2nd 2017
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
1 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Motivation, Problem Statement
Intelligent Transportation Systems (ITS)
Collect and process traffic data in real-time
Car traffic delays costs $45 billion 1
Detecting congestion and its effect on neighboring roads
Updating routing algorithms and traffic management strategies
Problem Statement
Real-time short-term traffic forecasting in transportation networks
1. https ://www.citylab.com/life/2013/10/us-transportation-system-has-100-billion-worthinefficiencies/7076/
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
2 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Importance Of Spatial Relation
Spreading of congestion in transportation network - spatial relation
Credit : Benzi et. al., Principal Patterns on Graphs : Discovering Coherent Structures in Datasets
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
3 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Dataset
More than 10 billion data points from GPS, routing Apps, road cameras,...
Average speed of vehicles (2min timestep) of 4700 road segments in Dallas
Reported crashes to police dataset
Collected by Texas A&M Transportation Institute for a year
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
4 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Previous Works
Principal component analysis based prediction
Projecting data to data driven orthogonal basis
Predicting projected signal in orthogonal basis
Actual spatial relation of adjacent roads is ignored
Vectored ARMA (Pavlyuk, 2017)
Capturing local spatial relation
High complexity - computationally expensive
Learning (Wu et. al., 2016 - Shahsavari et. al., 2015)
Spatio-temporal prediction using deep learning and neural networks
Computationally expensive
Changing with graph structure
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
5 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Using GSP For Prediction
How to solve the problem using GSP ?
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
6 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Network Line Graph
B = incidence matrix
LG = 12 (D + ≠ W + D ≠ ≠ W T ) = B B T = Laplacian of directed graph
U = eigenvector matrix of LG
Xt = graph signal at time t
‚t = U T Xt
GFT (Xt ) = X
Network graph
Intersections = nodes
Roads = directed edges
Signal defined on edges
Arman Hasanzadeh - [email protected]
Network line graph
Roads = nodes
Intersections = directed edges
Signal defined on nodes
Traffic Prediction Using GSP
June 2nd 2017
7 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
System Overview
Xt≠m , . . . , Xt
Joint Time-Vertex Filter
Ât+1 , . . . , X
Â
X
t+k
 denotes predicted signal
X
Prediction filter can be defined using :
ARMA models
ARMA model and semi-discrete partial differential equation on graphs
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
8 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
WSS Process & ARMA Model
Properties of WSS process in time
It can be generated by filtering white noise
Process is uncorrelated in the spectral domain
First two moments are invariant to translation
Auto-regressive moving average
Predicting by filtering previous samples and zero mean white noise
ARMA(m, q) : Â
xt = c + Át +
qm
i=1
ai xt≠i +
qq
c is a constant and Á is zero mean white noise
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
i=1
bi Át≠i
June 2nd 2017
9 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Joint Graph
Definitions
LG = graph Laplacian
LT = time series Laplacian
LJ = LT
UJ = UT
o
o
IN + IT
LG = joint Laplacian
o
JFT (x) =
UG = joint Fourier transform eigenvectors
UJú x
Arman Hasanzadeh - [email protected]
where x = vec(XN◊T ).
Traffic Prediction Using GSP
June 2nd 2017
10 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Joint Time-Vertex Wide-Sense Stationary Process
Joint graph
LG = graph Laplacian
LT = time series Laplacian
LJ = LT
UJ = UT
o
o
IN + IT
LG = joint Laplacian
o
UG = joint Fourier transform eigenvectors
JFT (x) = UJú x where x = vec(XN◊T ).
Joint time-vertex wide-sense stationary (JWSS) (Loukas et. al., 2016)
x = h(LJ )Á
Á ≥ D(c, INT ) and h is joint filter as a function of LJ
Covariance matrix is diagonizable with LJ
LJ x̄ = 0N◊T and (t1 , t2 ) = (1, 1 + t2 ≠ t1 ) = “· (LG )
· = t2 ≠ t1 and “ is a graph filter
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
11 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
ARMA Model For JWSS Process (Loukas et. al., 2016)
Signal in frequency domain uncorrelated in each frequency
GFT of signal at each time step - uncorrelated time series in GF domain
Independent ARMA models for time series at each frequency
Low complexity compared to VARMA in neighborhood or whole graph
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
12 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Numerical Results
Prediction error for one day of traffic data
6 step prediction (k=6), ARMA(10, 10)
Data of previous day used for training ARMA model
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
13 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Numerical Results, continued
Multiple peaks in prediction error
High error in prediction ≈∆ crash/congestion happened
By cross referencing with crash dataset and detected congestions
High error after crash/congestion
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
14 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Numerical Results, continued
Congestion example
Speed (Km/h)
100
80
60
40
40
45
50
55
50
55
50
55
Time Index
Speed (Km/h)
100
80
60
40
40
45
Time Index
Speed (Km/h)
100
80
60
40
40
45
Time Index
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
15 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Post-Congestion Prediction
How to decrease error after crash ?
Use different ARMA model after crash
Use semi-discrete PDE on graph to model traffic
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
16 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Two State ARMA Model
Learning optimal ARMA model based on hidden state of system :
Non-congested
Congested
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
17 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Post-Congestion ARMA Model
Prediction error for one day of traffic data
6 step prediction (k=6) using ARMA(2, 2) for after crash/congestion
Data of previous day used for training ARMA models
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
18 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Post-Congestion ARMA Model, Continued
Prediction errors after congestion decreased significantly
Use only post-congestion data for predicting signal after congestion
Resetting memory !
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
19 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Post-Congestion ARMA Model, Continued
Mean absolute error (MAE) =
Arman Hasanzadeh - [email protected]
1
NT
ÿ
i=0,...,N
j=0,...,T
|
Âj (i)
Xj (i) ≠ X
|
Xj (i)
Traffic Prediction Using GSP
June 2nd 2017
20 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
PDEs On Graphs
Semi-discrete PDE
Discretization of space ∆ difference-differential equations on graphs
Exterior derivative = incidence matrix (B) of graph
Laplacian operator = laplacian matrix (L) of graph
Generally in the form of ˆ m X /ˆt m = C(X ), where C is discrete difference
operator defined by modified incidence matrices
Discrete PDE
Discretization of space and time ∆ difference equations on graphs
Derivative in discrete time Xt ≠ Xt≠1
Laplacian operator in discrete time = Laplacian matrix (LT ) of ring graph
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
21 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
PDEs On Graphs, examples
Heat Diffusion
Continuous : ˆX /ˆt = ≠Ò2 X
Semi-discrete : ˆX /ˆt = ≠LG X
Discrete : Xt = (I ≠ LG )≠1 Xt≠1
Wave Equation
Continuous : ˆ 2 X /ˆt 2 = ≠Ò2 X
Semi-discrete : ˆ 2 X /ˆt 2 = ≠LG X
Discrete : XN◊T LT = LG XN◊T
In discrete equation signal is a matrix with dimension N ◊ T
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
22 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Convection-Diffusion Process
Convection-diffusion PDE in continuous space
ˆX /ˆt = Di Ò2 X ≠ Ò.(˛v .X )
˛v and Di are constant vector field
Describes chemical concentration in flowing fluid with diffusion
Semi-discrete convection-diffusion PDE
ˆX /ˆt = Di LG X ≠ B diag(v )Bv X
Bv = (diag(sign(v ))B T )+
Finding best vector fields that describes traffic after crash/congestion ?
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
23 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Convection-Diffusion Process, Example
t = 0
t = 2
t = 1
t = 3
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
24 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Discrete Convection-Diffusion Process, Example
t = 0
t = 2
t = 1
t = 3
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
25 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
ARMA-PDE Model
ARMA model for non-congested state
Semi-discrete PDE model for traffic prediction given congestion happened
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
26 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Conclusion
Conclusion
Two models have been proposed for short-term traffic forecasting :
ARMA/semi-discrete PDE model
Two state ARMA model
PDE model can be applied to JWSS and non-JWSS signals
ARMA model can be applied to JWSS signals only
Ongoing works
Using adaptive ARMA instead for hidden states
Finding optimal constant in PDE model
Effect of applying filters locally and globally
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
27 / 28
Introduction
System Model
Two State ARMA Model
ARMA-PDE Model
Conclusion
Thanks !
Arman Hasanzadeh - [email protected]
Traffic Prediction Using GSP
June 2nd 2017
28 / 28