Why are invertible matrices called 'non-singular'?

Where in the history of linear algebra did we pick up on referring to invertible matrices as 'non-singular'? In fact, since

• the null space of an invertible matrix has a single vector
• an invertible matrix has a single solution for every possible $b$ in $AX=b$

it's easy to imagine that that invertible matrices would be called 'singular'. What gives?

Solution 1:

If you take an $n\times n$ matrix "at random" (you have to make this very precise, but it can be done sensibly), then it will almost certainly be invertible. That is, the generic case is that of an invertible matrix, the special case is that of a matrix that is not invertible.

For example, a $1\times 1$ matrix (with real coefficients) is invertible if and only if it is not the $0$ matrix; for $2\times 2$ matrices, it is invertible if and only if the two rows do not lie in the same line through the origin; for $3\times 3$, if and only if the three rows do not lie in the same plane through the origin; etc.

So here, "singular" is not being taken in the sense of "single", but rather in the sense of "special", "not common". See the dictionary definition: it includes "odd", "exceptional", "unusual", "peculiar".

The noninvertible case is the "special", "uncommon" case for matrices. It is also "singular" in the sense of being the "troublesome" case (you probably know by now that when you are working with matrices, the invertible case is usually the easy one).

Solution 2:

I don't know for sure the source of the term "singular", but one reason to call matrices singular when the determinant vanishes, is that curves and surfaces and manifolds have singularities, that is, non-smooth points, when the Jacobian matrix of its coordinate functions is a non-invertible matrix. Smooth points are generic, and non-smooth points are very special, singular, according to Sard's lemma.

Another closely related reason to call the singular matrices "singular" is that they are themselves the singular locus of the algebraic variety of matrices up to some fixed rank, which is known as a determinantal variety. So calling them "singular" seems quite natural.

In response to comments, let me flesh out this last justification. If $M(m, n)$ is the set of all $m\times n$ matrices, then let $M_{\leq r}\subseteq M(m, n)$ be the matrices of rank at most $r$, for $r<\text{min}(m,n).$ Then $M_{\leq r}$ is an irreducible variety of codimension $(m-r)(n-r)$. It is defined as the space of matrices for which every order $r+1$ minor determinant vanishes.

Hence $M_{\leq r}$ is the zero locus of the cofactor map or the adjugate map $\text{Adj}_{r+1}\colon M(m, n)\to M\left(\binom{m}{r+1},\binom{n}{r+1}\right),$ which sends a matrix $A$ to the $\binom{m}{r+1}\times\binom{n}{r+1}$ matrix whose $IJ^{\text{th}}$ component, for ordered tuples $I=(i_1,\dotsc,i_{r+1}), 1 \leq i_1 < i_2 < \dotsb < i_r < i_{r+1} \leq m$ and $J=(j_1,\dotsc,j_{r+1}), 1 \leq j_1 < j_2 < \dotsb < j_r < j_{r+1} \leq n$, is the determinant of the matrix whose rows are the $i_1,i_2,\dotsc,i_{r+1}$ rows of $A$, and whose columns are the $j_1,\dotsc,j_{r+1}$ columns of $A$. This map is polynomial in the components of the matrix $A$, hence this shows that $M_{\leq r}$ is an algebraic variety.

Recall that the partial derivative of a general $n\times n$ determinant function is

$$ \frac{\partial(\det A)}{\partial A_{ij}}=\text{Adj}_{n-1}(A)_{ij} $$

so the derivative of the $IJ^\text{th}$ component of the map $\text{Adj}_{r+1}\colon M(m, n)\to M\left(\binom{m}{r+1},\binom{n}{r+1}\right)$ is a determinant minor of rank $r$ of $A$. Hence the Jacobian $d(\text{Adj}_{r+1})$ vanishes if and only $\text{Adj}_{r}$ vanishes. In other words, the rank at most $r-1$ matrices are the singular locus of the variety of rank at most $r$ matrices.

For example, the locus of rank at most 1 matrices in $2\times 2$ matrices is those matrices $ \left(\begin{smallmatrix} a & b\\ c & d \end{smallmatrix}\right) $ for which $ad-bc=0$. This is a quadric hypersurface in $\mathbb{A}^4$ of rank 4, which means (as described for example in this answer by Georges Elencwajg) that it is a cone on the Segre embedding $\mathbb{P}^1\times\mathbb{P}^1\to\mathbb{P}^3$, with cone point $A=0.$ That cone point is the sole rank 0 matrix, the singular locus of the quadric hypersurface that is the rank at most 1 matrices.

Note that this doesn't apply when $r=\text{min}(m,n),$ because the set of matrices of rank at most $\text{min}(m,n)$ is all of $M(m, n)=\mathbb{A}^{mn}$ which is nowhere singular. Only the lower than full rank matrices are singular loci. The above argument doesn't apply because there are no $r+1$ order minors when $r=\text{min}(m,n)$. If $\text{min}(m,n)=m$ and $r=m+1$, then $\binom{m}{r+1}=0,$ so $M\left(\binom{m}{r+1},\binom{n}{r+1}\right) = M\left(0,\binom{n}{r+1}\right),$ the empty matrix. This map is constant, hence constant rank (I think the empty matrix has rank zero?), so has no singular points. (Looking for singular points by taking derivatives does not apply because the constant polynomial is not irreducible).

Versions of this fact are also found in the answer here by Georges Elencwaijg and the one here by Qiaochu Yuan for comparison.