Why the SVD is named so...

Solution 1:

From On the Early History of the Singular Value Decomposition by Pete Stewart:

The term "singular value" seems to have come from the literature on integral equations. A little after the appearance of Schmidt's paper, Bateman refers to numbers that are essentially the reciprocals of the eigenvalues of the kernel as singular values. Picard combined Schmidt's results with Riesz's theorem on the strong convergence of generalized Fourier series to establish a necessary and sufficient condition for the existence of solutions of integral equations.

In a later paper on the same subject, he notes that for symmetric kernels Schmidt's eigenvalues are real and in this case (but not in general) he calls them singular values. By 1937, Smithies was referring to singular values of an integral equation in our modern sense of the word. Even at this point, usage had not stabilized. In 1949, Weyl speaks of the "two kinds of eigenvalues of a linear transformation," and in a 1969 translation of a 1965 Russian treatise on nonselfadjoint operators Gohberg and Krein refer to the "s-numbers" of an operator.

See the paper for more fascinating accounts on how SVD came to be, even before the seminal paper of Golub/Kahan.

Solution 2:

From Schwartzman's The Words of Mathematics:

singular (adjective), singularity (noun): from Latin singulus "separate, individual, single," from the Indo-European root sem- "one, as one." If there is just a single example of something, that example becomes special, so singular took on the meaning "out of the ordinary." [...] The meaning "out of the ordinary, troublesome," explains why a singular matrix is a square matrix whose determinant equals $0$ rather than $1$, as the etymology implies.

Also, note that the terms singular matrix and singular value are contemporary. They made their first documented appearance in 1907 and 1908, respectively.

As such, I'd guess that singular values are called like that just because they are indeed out of the ordinary. They provide a nice invariant for any complex valued matrix.