Why is the expected value of the product of standard normals the identity?
Solution 1:
If $Z \sim \mathcal{N}(0, \sigma^2 I_n)$ is normally distributed on $\mathbb{R}^n$ with iid components $Z_i$ then
$$E[ZZ^\top]_{ij} = E[Z_iZ_j] = \delta_{ij}.$$