If $AB=I_n$ and $BA=I_m$ then $n=m$.

Let $A, B$ matrices $n\times m$ and $m\times n$ respectively so that $AB=I_n$ and $BA=I_m$. Show that $n=m$.

$\newcommand{\tr}{\text{tr}}$ For a square $k\times k$ matrix $A = (a_{ij})$ we have the trace defined as $$ \tr(A) = a_{11} + a_{22} + \dots + a_{kk}. $$ That is, the trace of a square matrix is the sum of the diagonal elements. It is a fact that the trace of $AB$ is equal to the trace of $BA$: $$ \tr(AB) = \tr(BA) $$ for any matrices where the product makes sense. Note that $AB$ and $BA$ necessarily need to be square matrices. This fact is not too hard to prove using the definition of the matrix product and the definition of the trace.

Now using this fact, you have $$ n = \tr(I_n) = \tr(AB) = \tr(BA) = \tr(I_m) = m. $$

There's nothing wrong with Thomas's argument. It's the one I would have used myself. But, I thought I'd add the other "obvious" argument.

Since $AB = I_n$ we know that $A$ has full rank equal to $n$. This is because, for any $b \in \mathbb{R}^n$, the equation $Ax = b$ has the solution $x = Bb \in \mathbb{R}^m$. Since $\operatorname{rank}(A) = n$, the rank-nullity theorem applied to $A$ implies that $m \geq n$.

Similarly, from $BA = I_m$, one deduces $m \leq n$, so $m=n$ as desired.

Hint:

$tr(AB) = tr(BA)$;

where $tr(A)$ is the trace of $A$ (sum of diagonal elements in a square matrix).