结构化低秩矩阵分解方法介绍

Find a polynomial close to p(x) with multiple roots.
• The Sylvester matrix of p(x) and p (x) will be a (2n − 1) × (2n − 1) matrix formed from the coefficients of p(x) and p (x). • For example, if n = 4 and p(x) = a4x4 + a3x3 + a2x2 + a1x + a0, then a4 a3 a2 a1 a0 0 0 0 a a a a a 0 4 3 2 1 0 0 0 a a a a a 4 3 2 1 0 Sp,p = 4a4 3a3 2a2 1a1 0 0 0 0 4a4 3a3 2a2 1a1 0 0 0 0 4a4 3a3 2a2 1a1 0 0 0 0 4a4 3a3 2a2 1a1 • p(x) has a multiple root ⇐⇒ det(Sp,p ) = 0. (resultant) • Find a low rank approximation to Sp,p with the above structure.
• The Sylvester matrix of p(x) and q (x) is an (m + n) × (m + n) matrix formed from the coefficients of p(x) and q (x). • For example, if m = 3, n = 2, p3 0 = q2 0 0 p2 p3 q1 q2 0 p1 p2 q0 q1 q2 p0 p1 0 q0 q1 0 p0 0 0 q0
5
Problem Statement II
• Consider a data matrix D = D + N , where D is a structured low rank matrix and N is a noise matrix. • Consider structures of the form D = – Examples: a b Hankel: c d e b c d e f c d e f g d e f g h e p3 0 f g , Sylvester: q2 h 0 i 0 p2 p3 q1 q2 0 p1 p2 q0 q1 q2 p0 p1 0 q0 q1 0 a f p0 0 , Toeplitz: g 0 h q0 i b a f g h c b a f g d c b a f e d c . b a αk Dk (αk are scalars, Dk are matrices).
Can we recover D given D?
6
Applications I
Let p(x), q (x) be polynomials of degree m and n with inexact coefficients.
Find a pair of polynomials with common roots close to p(x), q (x).
4
Problem Statement I
Let T be the set of all matrices with a certain structure. Adding a structural constraint to the previous problem gives:
min rank(X ) s.t.
• These prediction matrices have the Hankel structure, and in the absence of noise, have rank K , where K is the number of damped sinusoids.
9
Applications III
k
– In the above examples, Di, Dj = 0 if i = j . – Note: A, B = tr(A B ) = vect(A) · vect(B ) where vect(A) is the vector formed by stacking the columns of A on top of each other.
2
Motivation I
• Many recent papers involve solving problems of the form
min rank(X ) s.t.
A (X ) − b 2 Rp is a linear map, b ∈ Rp, and θ is a bound on the noise. • This problem comes up in applications such as – Matrix Completion: ∗ Recover a low-rank matrix given a subset of its entries. ∗ Netflix problem. – Image compression: ∗ Recover an image that can be well-approximated by a low-rank matrix. – Multivariate Regression: ∗ Estimate a prediction function, mapping covariates to output vectors. ∗ In certain settings, the prediction function is a low rank matrix.
Parameter Estimation of Damped Sinusoidal Signals with Noise
• To reduce the noise effect, these algorithms first use low rank matrix approximation on prediction matrix to obtain a less noisy set of observations y (n). • This set of observations is then used in the algorithms in place of y (n). • It has been shown (Razavilar, Li, Liu 1996), that if the Hankel property of the prediction matrix is also preserved when finding the low rank approximation, the performance of the parameter estimation can be improved. • Thus, a low rank Hankel approximation of Ay is found to obtain a less noisy set of observations y (n) before continuing with the algorithm.
Sp,q
• p(x), q (x) have a common root ⇐⇒ det(Sp,q ) = 0. • Find a low rank approximation to Sp,q with a Sylvester structure.
7
Applications II
Let p(x) be a polynomial of degree n with inexact coefficients.
3
Motivation II
• Many algorithms to solve such problems: – SDP approach: ∗ Relax problem to a semidefinite program (SDP). ∗ Many available solvers for SDPs (SeDuMi, SDPA, etc) – Singular Value Thresholding Algorithms ∗ Uses a version of fixed point iteration. ∗ Faster than using the SDP approach. `s, Shen) ∗ FPCA (Ma, Goldfarb, Chen), SVT (Cai, Cande • In some applications, want to recover a structured low rank matrix. • Motivated by the nonstructed low rank approximation problem, hope to develop good algorithms to do this.
An Introduction to Structured Low Rank Matrix Approximation
Cinna Wu Department of Mathematics University of California, Berkeley
1
Contents
• Motivation and Problem Statement • Applications • Theory • Methods of Computation • Future Work
• Several parameter estimation algorithms first involve forming prediction matrices from the observations y (n) (Kumaresan-Tufts (KT) algorithm, total least square (TLS) algorithm). y (0) y (1) ... y (L − 1) y (1) y (2) ... y (L ) . . . . Ay = . . . . . . . . y (L − 1) y (L) ... y (2L − 2)