The relationship between the eigenvalues of matrices $XY$ and $YX$

Solution 1:

It is very easy to check that $XY$ and $YX$ have the same nonzero eigenvalues. Just apply $Y$ to the identity $XYv=\lambda v$ and note $Yv\neq 0$ if $\lambda v\neq 0$.

But if you mean $\lambda$ to be the set of eigenvalues counted with multiplicities, then what you are asking is equivalent to $$ \chi_{XY}(t)=t^{m-n}\chi_{YX}(t) $$ where $\chi_A(t)=\det(tI-A)$ is the charateristic polynomial.

Big hint: $$ \left(\matrix{I&X\\Y&tI}\right)\left(\matrix{tI&-X\\0&I}\right)=\left(\matrix{tI&0\\*&tI-YX}\right) $$ and $$ \left(\matrix{I&X\\Y&tI}\right)\left(\matrix{tI&0\\-Y&I}\right)=\left(\matrix{tI-XY&*\\0&tI}\right). $$

Solution 2:

You could modify the proof of Sylvester's determinant theorem to show that $$\text{det} \left( \lambda I_m - XY \right) = \text{det} \left( \lambda I_n - YX \right)$$ for all $\lambda \neq 0$. This shows equivalence for all nonzero eigenvalues. For the zero eigenvalues, an application of the fundamental theorem of algebra is sufficient. Note that the characteristic polynomial of $XY$ (or $YX$) must have $m$ (or $n$) roots. Since we have examined all roots $\lambda\neq0$, the remaining roots must be zero.

Edit: This proof does not work, see the comments.