a while ago (before Englund became an MVC) I posted a proof about another result in factor analysis. I thought it would be nice to resurrect it (briefly) and add it here to our small (but growing) compendium of proofs. the original thread is here

http://www.talkstats.com/threads/factor-analysis-proof.45041/
and the proof goes like this:

Let \(\bf{S}\) be a covariance matrix with eigenvalue-eigenvector pairs (\(\lambda_1, \mathbf{e}_1\)), (\(\lambda_2, \mathbf{e}_2\)), ..., (\(\lambda_p, \mathbf{e}_p\)), where

\(\lambda_1 \ge \lambda_2 \ge ... \ge \lambda_p\). Let \(m<p\) and define:

\(\bf{L} = \{l_{ij}\} = \left[\sqrt{\lambda_1 }\mathbf{e}_1\ |\ \sqrt{\lambda_2} \mathbf{e}_2\ |\ ...\ |\ \sqrt{\lambda_m} \mathbf{e}_m \right] \)

and:

\(\

\mathbf\Psi =

\left(

\begin{array}{cccc}

\psi_1 & 0 & ... & 0 \\

0 & \psi_2 & ... & 0 \\

\vdots & \vdots & \ddots & \vdots \\

0 & 0 & ... & \psi_p \\

\end{array}

\right)

\text{ with } \psi_i = s_{ii} - \sum_{j=1}^{m} l_{ij}^2\)

Then, PROVE:

\(

\text{Sum of squared entries of } (\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi})) \le \lambda_{m+1}^2 + \cdots + \lambda_p^2\)

Spunky's attempt of a proof:

By definition of \(\psi_i\), we know that the diagonal of \((\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi}))\) is all zeroes. Since

\((\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi})))\) and \((\mathbf{S} - \mathbf{LL'})\) have the same elements except on the diagonal, we know that

\(\text{(Sum of squared entries of } (\mathbf{S} - (\mathbf{LL'} + \mathbf{\Psi}))) \leq \text{ Sum of squared entries of } (\mathbf{S} - \mathbf{LL'}) \)

Since \(\mathbf{S} = \lambda_1 \mathbf{e}_1 \mathbf{e}'_1 + \cdots + \lambda_p \mathbf{e}_p \mathbf{e}'_p \)

and \(\mathbf{LL'} = \lambda_1 \mathbf{e}_1 \mathbf{e}'_1 + \cdots + \lambda_m \mathbf{e}_m \mathbf{e}'_m \), then it follows that

\(\mathbf{S} - \mathbf{LL'} = \lambda_{m+1} \mathbf{e}_{m+1} \mathbf{e}'_{m+1} + \cdots + \lambda_p \mathbf{e}_p \mathbf{e}'_p\)

Writing it in matrix form, this is saying \(\mathbf{S} - \mathbf{LL'} = \mathbf{P}_2 \mathbf{\Lambda}_2 \mathbf{P}'_2\) where

\(\mathbf{P}_2 = [ \mathbf{e}_{m+1} | \cdots | \mathbf{e}_p ]\) and \(\mathbf{\Lambda}_2 = Diag(\lambda_{m+1}, \cdots, \lambda_{p})\)

Then, the following is true:

\(\text{Sum of squared entries of }(\mathbf{S}- \mathbf{LL'})= \text{tr}((\mathbf{S} - \mathbf{LL'}) (\mathbf{S} - \mathbf{LL'})')=\)

\(\text{tr} (( \mathbf{P}_2 \mathbf{\Lambda}_2 \mathbf{P}'_2)( \mathbf{P}_2 \mathbf{\Lambda}_2 \mathbf{P}'_2)')=\text{tr}( \mathbf{P}_2 \mathbf{\Lambda}_2\mathbf{\Lambda}_2 \mathbf{P}'_2)\)

\(tr(\mathbf{\Lambda}_2\mathbf{\Lambda}_2)=\lambda_{m+1}^2 + \cdots + \lambda_p^2.\)

All the \(\bf{P}_2\) disappear because by the definition of \(\bf{P}_2\) we know that \(\bf{P}_2 '\bf{P}_2=\bf{I}\)