In a matrix ring, no zero divisors may have an inverse

I'm glad to see that Dedekind finite rings have been raised as the right concept to look at, but I think a little more can be done to help you see where the condition lies with respect to other conditions you might know.

Here is a diagram reproduced from a portion of my notes that gives a view (at least a partial one):

There are a lot of strange names up there, but between Lam's two books (First course on noncommutative rings and Lectures on modules and rings) you would be able to reproduce this table. In addition to those books, there is an additional paper by Lam that talks about stable range and its immediate relatives: A crash course on...

Of course, matrix rings over division rings lie in the "semisimple" box. In addition to the Dedekind finite box, I think you should also know about "stably finite". A ring $R$ is said to be stably finite if all of the $n\times n$ matrix rings over $R$ are Dedekind finite.

Since Noetherian and perfect rings have already been mentioned, I'd like to contribute by pointing out two "new" examples given here. Since there are two-sided self-injective rings which aren't Noetherian and aren't semiperfect, and unit regular rings which aren't Noetherian and aren't semiperfect, nor injective on both sides, I think both of those classes of rings are "new" to this post.

Finally, there is one thing to add to this diagram. I don't normally keep a box for "commutative ring," but if it were in this picture, it would have an arrow pointing to "Stably finite."

There is one more condition which might interest you. A ring in which every element which isn't a left zero divisor is a unit is called a right cohopfian ring. This means essentially that if multiplication by an element on the left makes an injective map $R\to R$, then it is an isomorphism. This relates closely to an alternative definition of Dedekind finite: If multiplication on the left by $a$ is surjective then it is an isomorphism. This is saying that $R$ is "hopfian". It turns out that for a ring, this condition is left-right symmetric, but the cohopfian version isn't :)

I've got a bit of info on them in this solution: https://math.stackexchange.com/a/135051/29335 . One-sided cohofian rings are also Dedekind finite. I would have to double check which things on the graph point to "cohopfian" :) They're a bit of an oddball...

It is a peculiarity of “vaguely finite” rings.

A right Artinian ring has an even stronger property shared by matrices: if $a$ is not a left zero-divisor (there is no nonzero $b$ with $ab=0$), then $a$ has both a left and right inverse (and so they are equal). Indeed, it need only have the DCC on principal right ideals, a so-called left perfect ring.

The exact rings you are looking for are the “Dedekind-finite” rings: those rings in which $ab=1$ implies $ba=1$. Every one-sided noetherian ring is Dedekind finite.

One also has in general, that if $ab=1$ then $a(ba-1) = aba-a = 0$ so that if $a$ has a right inverse, then either it is a unit (has a left inverse identical to its right inverse) or it is a left zero divisor.

If $ab=1$, but $ba\neq 1$, then notice that $a \cdot \left((1-ba)a^j +b\right) = 0 + ab = 1$ and that for distinct $i,j$, $(1-ba)a^i \neq (1-ba)a^j$, so that if $a$ has a right inverse, but no left inverse, then it must have infinitely many right inverses.

Proof of equivalence: Suppose no one-sided zero-divisor can have a one-sided inverse. Then every element with a one-sided inverse must have a two-sided inverse, but that is precisely the condition that $ab=1 \implies ba=1$, that is, the ring is Dedekind-finite. Conversely suppose that the ring is Dedekind-finite: then any element with a one-sided inverse has a two-sided inverse, and so cannot be a zero divisor.

One can find this and more in Lam's First Course on Non-commutative Rings, especially chapter 1.

It's quite a bit more general than just matrix rings, though obviously some finiteness condition is required. It's a property of (left) Noetherian rings that anything that has a left inverse also has a right inverse.

Proof: if $x \in R$ has a left inverse $x^{-1}$, then right multiplication by $x$ is surjective since $y=(yx^{-1})x$. It is also injective: let $I_n$ be the (left ideal) kernel of right multiplication by $x^n$, and choose $n$ with $I_n=I_{n+1}$ (Noether used here). Then if $y \in I_1$, we can write $y=z x^n$ for some $z$, so we have $$0=y x=zx^{n+1} \quad \implies \quad 0=zx^n=y$$ by choice of $n$. Thus right multiplication by $x$ is also injective. Hence $$0=(1-x x^{-1})x \implies 1=x x^{-1}$$ so $x^{-1}$ is a right inverse for $x$.

PS: If the ring is right Noetherian, then one applies the opposite argument (interchanging right and left) to the element $x^{-1}$ to show that it also has a left inverse. So being Noetherian on one side is enough.