Why do positive definite symmetric matrices have the same singular values as eigenvalues?
I realize that this is because when the eigenvalues are either 0 or 1 they will have the same square root. But why does this happen?
Let $\lambda, v$ be an eigenpair for a positive definite matrix $P$, i.e. $Pv = \lambda v$. Multiply both sides of this equation by $P^\top$ to get $P^{\top}Pv = P^{\top}\lambda v$. We have $P^\top = P$ and hence $P^{\top}Pv = \lambda^2 v$. Therefore $\lambda^2$ is an eigenvalue for $P^{\top}P$, which is the square of a singular value for the matrix $P$. Since $P$ is positive definite, $\lambda > 0$ and hence $\sqrt{\lambda^2} = \lambda$. Therefore, the singular value is equal to the eigenvalue.
Another approach:
if $A$ is positive definite, then $A$ can be diagonlized by an orthogonal matrix $P$:
$PAP^T=D$ where $D$ is diagonal matrix with the eigenvalues on the diagonal.
Since $A=A^T$:
$AA^T=P^TDPP^TDP=P^TD^2P$
Or in other words:
$PAA^TP^T=D^2$, so the diagonal form of $AA^T$ is $D^2$ and it has $AA^T$'s eigenvalues on the diagonal.
This shows us that if $\alpha$ is an eigenvalue of $A$, then $\alpha^2$ is an eigenvalue of $AA^T$. Since $A$ is positive definite, we know $\alpha >0$, and so $\alpha=\sqrt{\alpha ^2}$