Why is $A^TA$ invertible if $A$ has independent columns?

How can I understand that $A^TA$ is invertible if $A$ has independent columns? I found a similar question, phrased the other way around, so I tried to use the theorem

$$ rank(A^TA) \le min(rank(A^T),rank(A)) $$

Given $rank(A) = rank(A^T) = n$ and $A^TA$ produces an $n\times n$ matrix, I can't seem to prove that $rank(A^TA)$ is actually $n$.

I also tried to look at the question another way with the matrices

$$ A^TA = \begin{bmatrix}a_1^T \\ a_2^T \\ \ldots \\ a_n^T \end{bmatrix} \begin{bmatrix}a_1 a_2 \ldots a_n \end{bmatrix} = \begin{bmatrix}A^Ta_1 A^Ta^2 \ldots A^Ta_n\end{bmatrix} $$

But I still can't seem to show that $A^TA$ is invertible. So, how should I get a better understanding of why $A^TA$ is invertible if $A$ has independent columns?

Solution 1:

Consider the following: $$A^TAx=\mathbf 0$$ Here, $Ax$, an element in the range of $A$, is in the null space of $A^T$. However, the null space of $A^T$ and the range of $A$ are orthogonal complements, so $Ax=\mathbf 0$.

If $A$ has linearly independent columns, then $Ax=\mathbf 0 \implies x=\mathbf 0$, so the null space of $A^TA=\{\mathbf 0\}$. Since $A^TA$ is a square matrix, this means $A^TA$ is invertible.

Solution 2:

If $A $ is a real $m \times n $ matrix then $A $ and $A^T A $ have the same null space. Proof: $A^TA x =0\implies x^T A^T Ax =0 \implies (Ax)^TAx=0 \implies \|Ax\|^2 = 0 \implies Ax = 0 $.

Solution 3:

Let $A \in \mathbb R^{m \times n}$. Note that

$$f (x) := x^T A^T A x = \|A x\|_2^2$$

is positive semidefinite. Function $f$ vanishes when $A x = 0_m$. If $A$ has full column rank, i.e., if its $n$ columns are linearly independent, then $A x =0_m$ implies that $x = 0_n$, i.e., $f$ is positive definite and, hence, $A^T A$ is positive definite and, thus, invertible.

Solution 4:

I think it need be mentioned that we deal with the real field, as the mentioned both facts may not be true for arbitrary field $F$.

Example 1. Let $F=\mathbb Z_5$ and $A=\left[\begin{smallmatrix} 1 & 3\\ 2 & 1\\ 0 & 1 \end{smallmatrix}\right]$. Then $A$ has two independent columns, but $A^T A=\left[\begin{smallmatrix} 0 & 0\\ 0 & 1 \end{smallmatrix}\right]$ is not invertible.

Example 2. Let $F$ and $A$ be the same as above. Then $A=\left[\begin{smallmatrix} 1 & 3\\ 2 & 1\\ 0 & 1 \end{smallmatrix}\right]\sim \left[\begin{smallmatrix} 1 & 0\\ 0 & 1\\ 0 & 0 \end{smallmatrix}\right]$, i.e., ${\rm null}(A)=\{0\}$. But ${\rm null}(A^TA)={\rm span}(\left[\begin{smallmatrix} 1 \\ 0 \end{smallmatrix}\right])$ not equal to ${\rm null}(A)$.