Properties of the A-transpose-A matrix

Yes, it has all the properties of a covariance matrix because it is one. You can define a multivariate normal distribution for which $A^T A$ is the covariance matrix.