Why is minimizing the nuclear norm of a matrix a good surrogate for minimizing the rank?

Solution 1:

Why does compressed sensing work? Because the $\ell_1$ ball in high dimensions is extremely "pointy" -- the extreme values of a linear function on this ball are very likely to be attained on the faces of low dimensions, those that consist of sparse vectors. When applied to matrices, the sparseness of the set of eigenvalues means low rank, as @mrig wrote before me.

Solution 2:

The nuclear norm can be thought of as a convex relaxation of the number of non-zero eigenvalues (i.e. the rank).

Solution 3:

A nuclear norm of a matrix is equivalent to the L1-norm of the vector of its eigenvalues. Thus, you are injecting sparsity to the vector of eigenvalues. Essentially, this sparsity means you are reducing the rank of the original matrix.

Solution 4:

To be accurate, it has been shown that the $\ell_1$ norm is the convex envelope of the $\| \cdot \|_0$ pseudo-norm while the nuclear norm is the convex envelope of the rank.

As a reminder, the convex envelope is the tightest convex surrogate of a function. An important property is that a function and its convex envelope have the same global minimizer.