Let $A$ and $B$ be square matrices of the same size such that $AB = BA$ and $A$ is nilpotent. Show that $AB$ is nilpotent.

A square matrix $A$ is called nilpotent is $A^k=0$ for some positive integer $k$.

Let $A$ and $B$ be square matrices of the same size such that $AB = BA$ and $A$ is nilpotent. Show that $AB$ is nilpotent.

Following that is $AB$ not equal to $BA$ in part (b), must $AB$ be nilpotent?

I don't get how to prove this. Since $A$ can represent any matrix. I have no idea how to show it.

Solution 1:

Hint: if $AB=BA$ then $$(AB)^k=A^kB^k\ .$$

For the second part (with $AB\ne BA$) try $$A=\pmatrix{1&-1\cr1&-1\cr}\ ,\quad B=\pmatrix{4&3\cr2&1\cr}\ .$$

Solution 2:

Let A be nilpotent and let $k$ be the positive integer such that $A^k=0$. Since $AB=BA$, then:

$$(AB)^k=\underbrace{(AB)(AB)\cdots(AB)}_{k\text{ times}}=(AB)(BA)(AB)\cdots(AB)=ABBA(AB)\cdots(AB)= B^kA^k=A^kB^k.$$

We obtained the product above by rearranging the parentheses and using commutativity repeatedly.Since $A^k=0$, $(AB)^k=A^kB^k=0B=0$ But then this means for $k$, $(AB)^k=0$ and that means $AB$ is nilpotent. QED.

The reason it doesn't work if the matrices don't commute is because the product $(AB)^k$ cannot be rearranged into separate products of $A$ and $B$ to yield $A^kB^k$. To see this, expand the product and see if you can rearrange it without the commutativity. You can't.

Of course, you can also disprove it via a specific counterexample like David does, but that doesn't tell you why it fails in general.