How to calculate the degrees of freedom of an $r$-ranked matrix with the size being $n\times n$?

Treat matrices as vectors lying in $\mathbb{R}^{n^2}$. It can be imagined matrices with rank $r (r<n)$ are supposed to lie within a manifold of lower dimension. For example, the singular matrices lie within a $(n^2−1)$-dimensional manifold, because they satisfy $\det(M)=0$, the sole constraint. Typically, the number of independent constraints is equal to the difference between those two dimensions.

Alternatively, we call the dimension of that lower-dimensional manifold as "degrees of freedom".

Now how do we calculate the degrees of freedom of an $r$-ranked matrix with size $n\times n$?

The answer says the degrees of freedom is $n^2-(n-r)^2=(2n-r)r$. I try to interpret it as follows: First, by elementary matrices, every matrix $M$ with the rank of $r$ can be transformed into $$M\sim\begin{pmatrix} I_r&0\\ 0&0 \end{pmatrix}_{n\times n}$$

Now the constraints come from the block at bottom right, where the entries are suppressed to be zero. So there are $(n-r)^2$ constraints, which lead to the answer (why the number of constraints do not agree with the number of zeroes is because they are not all independent, intuitively).

The explanation is not formal at all. Can anyone provide a refined version? Thank you~

EDIT: Provide further explanation of "degrees of freedom"

Generically, the following procedure can be used to construct an $n\times n$ matrix with rank $r$:

1. Choose the first $r$ columns of the matrix, with $n$ degrees of freedom for each column. Generically, the results will be linearly independent.

2. We can now choose the remaining columns to be linear combinations of the first $r$ columns. This gives $r$ degrees of freedom for each column, namely the $r$ coefficients of the linear combinations.

Thus, the dimension of the space of rank-$r$ matrices is $$ rn + (n-r)r\text{,} $$ which is the same as $n^2 - (n-r)^2$.