Eigenvalues of $AB$ and $BA$ where $A$ and $B$ are square matrices

Solution 1:

Here is a proof similar to what the OP has tried:

Let $\lambda$ be any eigenvalue of $AB$ with corresponding eigenvector $x$. Then

$$ABx = \lambda x \Rightarrow \\ BABx = B\lambda x \Rightarrow\\ BA(Bx) = \lambda (Bx) $$

which implies that $\lambda$ is an eigenvalue of $BA$ with a corresponding eigenvector $Bx$, provided $Bx$ is non-zero. If $Bx = 0$, then $ABx = 0$ implies that $\lambda = 0$.

Thus, $AB$ and $BA$ have the same non-zero eigenvalues.

Solution 2:

It suffices to show that $AB$ and $BA$ have the same characteristic polynomial. First assume that $A$ is invertible then

$$\chi_{AB}(x)=\det(AB-xI)=\det A\det(B-xA^{-1})\\=\det(B-xA^{-1})\det A=\det(BA-xI)=\chi_{BA}(x)$$ Now since $\operatorname{GL}_n(K)$ is dense in $\operatorname{M}_n(K)$ then there's a sequence of invertible matrices $(A_n)$ convergent to $A$ and by the continuity of the $\det$ function we have $$\chi_{AB}(x)=\det(AB-xI)=\lim_{n\to\infty}\det(A_nB-xI)=\lim_{n\to\infty}\det(BA_n-xI)\\=\det(BA-xI)=\chi_{BA}(x).$$

Solution 3:

Alternative proof #1:

If $n\times n$ matrices $X$ and $Y$ are such that $\mathrm{tr}(X^k)=\mathrm{tr}(Y^k)$ for $k=1,\ldots,n$, then $X$ and $Y$ have the same eigenvalues.

See, e.g., this question.

Using $\mathrm{tr}(UV)=\mathrm{tr}(VU)$, it is easy to see that $$ \mathrm{tr}[(AB)^k]=\mathrm{tr}(\underbrace{ABAB\cdots AB}_{\text{$k$-times}}) =\mathrm{tr}(\underbrace{BABA\cdots BA}_{\text{$k$-times}})=\mathrm{tr}[(BA)^k]. $$ Now use the above with $X=AB$ and $Y=BA$.

Alternative proof #2:

$$ \begin{bmatrix} I & A \\ 0 & I \end{bmatrix}^{-1} \color{red}{\begin{bmatrix} AB & 0 \\ B & 0 \end{bmatrix}} \begin{bmatrix} I & A \\ 0 & I \end{bmatrix} = \color{blue}{\begin{bmatrix} 0 & 0 \\ B & BA \end{bmatrix}}. $$ Since the $\color{blue}{\text{red matrix}}$ and the $\color{red}{\text{blue matrix}}$ are similar, they have the same eigenvalues. Since both are block triangular, their eigenvalues are the eigenvalues of the diagonal blocks.

Solution 4:

If $A$ is invertible, user63181 already showed that

$$\det(AB-xI)=\det(BA-xI)$$

We now prove this equality in general.

Fix $x$

Let $$P_x(y):=\det[(A-yI)B-xI]-\det[B(A-yI)-xI] \,.$$

Then $P_x(y)$ is a polynomial of degree at most $n$ in $y$. Whenever $y$ is not an eigenvalue of $A$ the matrix $A-yI$ is invertible, thus by the first pat $P_x(y)=0$. Hence $P_x$ has infinitely many roots, and hence $P_x \equiv 0$.

This proves that $P_x(0)=0$ which is exactly what you need to prove.