Simple Optimal Algorithm for Solving Rank-r Complex Equations Ax = b Rendering Sparse Solution xS with Support Surprisingly Confined to the Indices of r Independent Columns of A

In this article we will study optimal sparse solutions for linear equations , where is known, is unknown, is the measurement vector (possibly noisy, i.e., ), and . The proposed algorithm computes an optimal sparse solution, say , denotes transposition), by using to precompute indices satisfying 1 that render , where denotes 's complementary indices. We will show that is a basic solution associated with the columns indexed by of some intermediate underdetermined linear equations whose underlying matrix is ( x )-dimensional. Hence, the cardinality of satisfies for a non-degenerate basic solution and s < r for a degenerate basic solution. The proposed algorithm is based on the compact canonical form (C-CF): , where both and have rank . We will focus on its special case, i.e., the compact singular value decomposition (C-SVD): , where is a diagonal matrix whose diagonal elements are the r positive singular values of in nonincreasing order, is the ( x ) dimensional unit matrix, and * denotes the Hermitian operator. A special case of C-SVD for reducing computations is the thin SVD (T-SVD), where we retain the first singular values of ; we thus obtain an ( x )-dimensional rank- matrix, say . Notice that an optimal sparse solution, say , associated with will satisfy | support ( ) | We wish to minimize - , where denotes any norm. Using the C-CF: the proposed algorithm consists of following two stages (i) find minimizing , possibly with constraints on e; and, (ii) any solution of the underdetermined linear equations will be an optimal solution for . Hence, in stage (i) sparsity is not an issue, whereas in stage (ii) such that Q's submatrix whose columns are indexed by , say , is regular, also called nonsingular. The desired sparse solution can be obtained by letting (i) , (ii) computing the basic solution , (iii) letting , and outputting . sparse solution is guaranteed to exist when < , i.e., for the underdetermined case , the square case , and the overdetermined case . The proposed algorithm is most efficient when we wish to minimize the Euclidean norm without any constraints. We could thus provide a new proof of the Moore-Penrose solution, say , published independently by Moore in 1920, Bjerhammar in 1951, and Penrose in 1955. The outline of this proof is as follows: (i) compute minimizing ; and, then (ii) compute subject to . We thus obtain that . However, as shown above we can also compute a sparse solution satisfying and since also we obtain , i.e., preserves optimality. The sparse solution thus obtained should be preferred over since it preserves optimality, does not require any optimization software, and by the parsimony principle gives a much simpler explanation of the observations. Surprisingly, $\alpha$ can be precomputed from A, therefore, we only have to compute followed by computing the basic solution ; then, letting followed by letting renders the desired . In retrospect it turns out that any independent columns of , say , suffice to compute while the remaining columns of can be discarded. Hence, the optimal solution of possibly with constraints on , say , renders