What is the *real* representation theory of $SO(3)$?

Solution 1:

The result that a representation of a compact group is a direct sum of irreducibles holds over the reals too. If you know the proof for complex representations, the same proof works for real ones (if you have a representation on a vector space $V$ and $U$ is an invariant subspace, pick $\pi':V\rightarrow U$ a projection of vector spaces, and then use integration over the group to produce $\pi:V\rightarrow U$ a projection which is also a map of representation spaces).

Solution 2:

Partial answer to the second part of your question. Yes, that is enough. The spherical harmonics form a basis for the $2 \ell + 1$ dimensional representations of SO(3). Here is a nice reference. The author calculates the action of rotation about the z axis applied to the $Y_{l,m}$ and from this the character table for $SO(3)$.

Solution 3:

Since I had a similar question which I think I found the answer to I might as well post it again here.

The universal (double) cover $SU(2, \mathbb{C})$ has irreducible real representations in all dimensions which are odd or divisible by $4$, see e.g. here. As in the complex case, the irreducible real representations in odd dimensions descend to $SO(3, \mathbb{R})$ and are isomorphic to the real vector space of spherical harmonics of degree $\ell$ on the $2$-sphere.

Some more detail regarding $SU(2, \mathbb{C})$, $SO(3, \mathbb{R})$ and real representations is given in chapter II sections 5 and 6 of Representations of Compact Lie Groups by Bröcker and Dieck.