Generalized inverse of a symmetric matrix

This article is originally published at https://www.alexejgossmann.com

I have always found the common definition of the generalized inverse of a matrix quite unsatisfactory, because it is usually defined by a mere property, , which does not really give intuition on when such a matrix exists or on how it can be constructed, etc… But recently, I came across a much more satisfactory definition for the case of symmetric (or more general, normal) matrices. :smiley:

As is well known, any symmetric matrix is diagonalizable,

where is a diagonal matrix with the eigenvalues of on its diagonal, and is an orthogonal matrix with eigenvectors of as its columns (which magically form an orthogonal set :astonished:, just kidding, absolutely no magic involved).

The Definition :heart:

Assume that is a real symmetric matrix of size and has rank . Denoting the non-zero eigenvalues of by and the corresponding columns of by , we have that

We define the generalized inverse of by

Why this definition makes sense :triumph:

1. The common definition/property of generalized inverse still holds:

   $$% < ![CDATA[ \begin{eqnarray} A A^{-} A &=& \sum_{i=1}^k \lambda_i q_i q_i^T \sum_{m=1}^k \frac{1}{\lambda_m} q_m q_m^T \sum_{j=1}^k \lambda_j q_j q_j^T \nonumber \\ &=& \sum_{i,m=1,\dots,k} \lambda_i \frac{1}{\lambda_m} q_i q_i^T q_m q_m^T \sum_{j=1}^k \lambda_j q_j q_j^T \nonumber \\ &=& \sum_{i=1}^k q_i q_i^T \sum_{j=1}^k \lambda_j q_j q_j^T \nonumber \\ &=& \sum_{i,j=1,\dots,k} \lambda_j q_i q_i^T q_j q_j^T \nonumber \\ &=& \sum_{i=1}^k \lambda_i q_i q_i^T = A, \nonumber \end{eqnarray} %]]&gt;$$

   where we used the fact that unless (i.e., orthogonality of ).

2. By a similar calculation, if is invertible, then and it holds that

   

3. If is invertible, then has eigenvalues and eigenvectors (because for all ).

   Thus, Definition () is simply the diagonalization of if is invertible.

4. Since form an orthonormal basis for the range of A, it follows that the matrix

   

   is the projection operator onto the range of .

But what if A is not symmetric? :fearful:

Well, then is not diagonalizable (in general), but instead we can use the singular value decomposition

and define

Easy. :relieved:

References

Definition is mentioned in passing on page 87 in

• Morris L. Eaton, Multivariate Statistics: A Vector Space Approach. Beachwood, Ohio, USA: Institute of Mathematical Statistics, 2007.

Thanks for visiting r-craft.org
This article is originally published at https://www.alexejgossmann.com
Please visit source website for post related comments.