Example of similar matrices $A$ and $B$ such that products $AB$ and $BA$ are not similar

Solution 1:

$A = \begin{bmatrix}1 & 0\\ 1 & 0\end{bmatrix},\ B = \begin{bmatrix}0 & 0\\1 & 1\end{bmatrix}$

Let $P = \begin{bmatrix}1 & -1\\-3 & 1\end{bmatrix}$. Then
$PAP^{-1} = \begin{bmatrix}1 & -1\\-3 & 1\end{bmatrix}\begin{bmatrix}1 & 0\\ 1 & 0\end{bmatrix}\begin{bmatrix}-1/2 & -1/2\\-3/2 & -1/2\end{bmatrix} = B$.

$AB = \begin{bmatrix}0 & 0\\0 & 0\end{bmatrix},\ BA = \begin{bmatrix}0 & 0\\2 & 0\end{bmatrix}$

$AB$ and $BA$ are not similar as $\text{rank}(AB) = 0$ but $\text{rank}(BA) = 1$.