Square matrices satisfying certain relations must have dimension divisible by $3$

I saw this tucked away in a MathOverflow comment and am asking this question to preserve (and advertise?) it. It's a nice problem!

Problem: Suppose $A$ and $B$ are real $n\times n$ matrices with $A^2+B^2=AB$. If $AB-BA$ is invertible, prove $n$ is a multiple of $3$.

Let $\omega$ be a primitive third root of unity. Then $(A+\omega B)(A+\omega^2B)=\omega(BA-AB)$. Since $A+\omega^2B=\overline{A+\omega B}$, we get that $\det(A+\omega B)(A+\omega^2B)$ $=$ $\det(A+\omega B)\det(A+\omega^2B)$ $=$ $\det(A+\omega B)\det(\overline{A+\omega B})$ $=$ $\det(A+\omega B)\overline{\det(A+\omega B)}$ is a real number, that is, $\omega^n\det(BA-AB)$ is a real number. Since $\det(BA-AB)\neq 0$ we get that $\omega^n$ is a real number, and this happens iff $3\mid n$.

I. Let $k$ be a field and consider the algebra $\def\A{\mathcal A}\A_k=k\langle a,b:a^2+b^2-ab\rangle$. We have $$ aaa = aab - abb = abb - bbb - abb = - bbb $$ and $$ aaa = aba - bba, $$ so that $$ aba = bba - bbb. $$ Let $x=ab-ba$. Then \begin{gather} ax = aab-aba = abb - bbb - bba + bbb = abb - bba, \\ bx = bab-bba, \\ xa = aba - baa = bba - bbb - bab + bbb = bba - bab, \\ xb = abb - bab, \\ \end{gather} so that in fact \begin{gather} ax = x(-a+b), \\ bx = x(-a). \end{gather} There is an automorphism $\sigma:\A_k\to\A_k$ such that $\sigma(a)=-a+b$ and $\sigma(b)=-a$, and the two equations above immediately imply that $$ \text{$ux = x\sigma(u)$ for all $u\in\A_k$.}$$ Since $\sigma^3=\mathrm{id}_{\A_k}$, it follows that $x^3$ is central in $\A_k$.

If $V$ is a $\A_k$-module, let us write $m_u:v\in V\mapsto uv\in V$.

II. Let $V$ be a simple finite dimensional left $\def\CC{\mathbb C}\A_\CC$-module, and let us suppose that $m_x$ is invertible. The map $m_{x^3}$ is an endomorphism of $V$ as a module because $x^3$ is central in $A$, so Schur's lemma tells us that there exists a $\lambda\in\CC$ such that $m_{x^3}=\lambda\cdot\mathrm{id}_V$. Since $m_x$ is invertible, $\lambda\neq0$, and there exists a $\mu\in\CC\setminus0$ such that $\mu^3=\lambda$. Let $y=x/\mu$, so that $m_y^3=m_{y^3}=\mathrm{id}_V$.

It follows that the eigenvalues of $m_{y}$ are cubic roots on unity. On the other hand, we have $m_y=[m_a,m_b]/\mu$, so that $\operatorname{tr}m_y=0$. Now, the only linear combinations with integer coefficients of the three cubic roots of unity which vanish are those in which the three roots have the same coefficient. Since $\operatorname{tr} y$ is the sum of the eigenvalues of $m_y$ taken with multiplicity, we conclude that thee three roots of unity have the same multiplicity as eigenvalues of $m_y$. This is only posssible if $\dim V$ is divisible by 3.

III. More generally, let now $V$ be a finite dimensional $\A_\CC$-module such that the map $v\in V\mapsto xv\in V$ is invertible. Since $V$ has finite dimension, it has finite length as an $\A$-module, and it is an iterated extension of finitely many simple $\A$-modules. Each of this modules has the property that $x$ acts bijectively on it, so we know their dimension is divisible by $3$. It follows then that the dimension of $V$ itself is divisible by $3$.

IV. Let now $V$ be a finite dimensional $\def\RR{\mathbb R}\A_\RR$-module on which $x$ acts bijectively. Then $\CC\otimes_\RR V$ is a finite dimensional $\CC\otimes_\RR\A_\RR$-module. Since $\CC\otimes_\RR\A_\RR$ is obviously isomorphic to $\A_\CC$, we know $\dim_\CC\CC\otimes_\RR V$ is divisible by three, and then so is $\dim_\RR V$ because this is in fact equal to $\dim_\CC\CC\otimes_\RR V$.

This conclusion is precisely the one we sought.

• The conclusion reached in III is in fact stronger than the one in IV.

• All this looks weird, but it is very, very natural. A pair of matrices satisfying the relation $A^2+B^2=AB$ gives a representation of the algebra $\A_k$. It is natural to try to obtain a basis for $\A_k$, at the very least, and this is usually done using Bergman's diamond lemma. The first part of the computation done in I is the start of that.

• Next, we are not interested in all $\A_k$ modules but only in those in which $X=[A,B]$ is invertible. This is the same as the modules which are actually modules over the localization of $\A_k$ at $x=[a,b]$. Comuting in such a localization is a pain, unless the element at which we are localizing is normal: one thus it motivated to check for normality. It turns out that $x$ is normal; there is then an endomorphism of the algebra attached to it, which is $\sigma$. One immediatelly recognized $\sigma$ to be given by a matrix of order $3$. The rest is standard representation theory.

• In a way, this actually gives a method to solve this sort of problems. Kevin asked on MO: «Are there really books that can teach you how to solve such problems??» and the answer is yes: this is precisely what representation theory is about!