Why is the expected value of the product of standard normals the identity?

Solution 1:

If $Z \sim \mathcal{N}(0, \sigma^2 I_n)$ is normally distributed on $\mathbb{R}^n$ with iid components $Z_i$ then

$$E[ZZ^\top]_{ij} = E[Z_iZ_j] = \delta_{ij}.$$