Range of A and null space of the transpose of A
Solution 1:
a) By definition $Ax\in R(A)$; on the other hand, $A^TAx=0$, by assumption, so $Ax\in N(A^T)$.
b) It is clear that $N(A)\subseteq N(A^TA)$. Suppose $x\in N(A^TA)$; then $x^TA^TAx=0$ as well, so $(Ax)^T(Ax)=0$, which implies $Ax=0$.
c) The rank-nullity theorem says that, if $B$ is an $m\times n$ matrix, then $\dim R(B)+\dim N(B)=n$. Let $A$ be $m\times n$ and apply the rank nullity theorem to $A$ and $A^TA$: \begin{align} n&=\dim R(A)+\dim N(A) \\ n&=\dim R(A^TA)+\dim N(A^TA) \end{align} Since by part b we have $\dim N(A)=\dim N(A^TA)$, we conclude $\dim R(A)=\dim R(A^TA)$.
d) If $A$ has linearly independent columns, its rank is $n$; so also $A^TA$ has rank $n$, hence it is invertible.
Solution 2:
Let us show that (c) and (d) are both consecuences of (b):
Let $A\in R^{nxm}$ and $nul(A) = nul(A^TA)$
$ \dim nul(A) = \dim nul(A^TA) $
$ n-rank(A) = n-rank(A^TA)$
$rank(A)=rank(A^TA)$
and, if $A$ has linearly independent columns, $nul(A)=\{0_{R^m}\} = nul(A^TA)$; and thus is proven that $\det(A^TA) \neq 0$
As for the proof of (b), and a more rigorous proof of $rank(A) = rank(A^TA)$, I'll post it after dinner.
EDIT:
$\dim col(A) + \dim nul(A) = n$
This property is what I had left unproven in my proof that (c) is consequence of (b), so there:
Let $A = \left( \begin{array}{ccc} - & v_1 & -\\ & \ldots &\\ - & v_n & - \end{array} \right)$ then:
$Ax = 0$ iff $v_ix = 0, \forall i$
$x \perp v_i, \forall i$
$x$ is perpendicular to every column of $A^T$, so it follows that
$x \in nul(A)$ iff $x \in (col(A^T))^\perp$
And so:
$\dim nul(A) = \dim col(A^T)^\perp$
$ = n- \dim col(A^T) = n-rank(A^T) = n-rank(A)$
EDIT:
Proof of (b):
Let $A \in R^{nxm}$:
$nul(A^TA) = \{ x \in R^m | A^TAx = 0_{R^m} \}$
$= \{ x \in R^m | Ax = 0_{R^n} \quad$ or $\quad Ax \in nul(A^T) \}$
but we have proven that $x \in nul(A^T) \quad$ iff $\quad x \in (col(A))^\perp$
There can be no $Ax \neq 0_{R^m}$ such that $A^TAx = 0_{R^n} \quad$ !
and thus: $\quad nul(A^TA) = \{ x \in R^m | Ax = 0_{R^n} \} = nul(A)$