Correspondence of representation theory between $\mathrm{GL}_n(\mathbb C)$ and $\mathrm U_n(\mathbb C)$

The representations theory of $U_n$ is simpler than that of $GL_n(\mathbb{C})$: every continuous representation of $U_n$ is semisimple, and every continuous irreducible representation is finite dimensional and is the restriction of an irreducible algebraic representation of $GL_n(\mathbb{C})$. In fact, restriction from finite dimensional algebraic representations of $GL_n(\mathbb{C})$ to $U_n$ is a $\mathbb{C}$-linear equivalence of tensor categories. The basic idea of the proof is to observe that $U_n$ is Zariski dense in $GL_n$ and therefore irreducible algebraic representations remain irreducible upon restriction (this sometimes goes by the name Weyl's unitary trick and can be used to prove his eponymous character formula). Of course, the representation theory of the non-compact Lie group $GL_n(\mathbb{C})$ is much more complicated than this!

The relation between $SO_n$ and $O_n$ is even simpler: since every matrix in $O_n$ has determinant $\pm 1$, $SO_n$ is an index $2$ normal subgroup of $O_n$. Clifford theory then completely describes the relationship between their irreducible representations: each irreducible $O_n$-module either remains irreducible or splits into two irreducibles upon restriction, and which of these happens is determined by the behavior of representations upon tensoring with the determinant representation.

The "nice" representations of $GL_n(\mathbb{C})$ are equivalent to the "nice" representations of $U_n$; the question is just how you should define nice. Let $\rho: GL_n(\mathbb{C}) \to GL_N(\mathbb{C})$ be a map of groups. Then the following are equivalent:

• The entries of the matrix $\rho(g)$ are complex analytic functions of the entries of the matrix $g$.
• The entries of the matrix $\rho(g)$ are polynomials in the entries of the matrix $g$ and $\det(g)^{-1}$.

Let $\sigma: U_n \to GL_N(\mathbb{C})$ be a map of groups. The following are equivalent:

• The entries of the matrix $\rho(g)$ are continuous functions of the entries of the matrix $g$.
• The entries of the matrix $\rho(g)$ are smooth functions of the entries of the matrix $g$.
• The entries of the matrix $\rho(g)$ are polynomials in the entries of the matrix $g$ and their complex conjugates.

If we use any of the conditions in these lists to define "nice" representations, the restriction from $GL_n(\mathbb{C})$ to $U_n$ is a functor from nice $GL_N$ representations to nice $U_n$ representations.

Theorem This functor is an equivalence of categories. In particular, every $U_n$ representation lifts to a $GL_N$ representation, $\dim Hom_{GL_N}(V,W) = \dim Hom_{U_n}(V|_{U_n}, W|_{U_n})$ and $V$ is irreducible if and only if $V|_{GL_n}$ is irreducible.

The easy parts of this are that $\dim Hom_{GL_N}(V,W) = \dim Hom_{U_n}(V|_{U_n}, W|_{U_n})$ and $V$ is irreducible if and only if $V|_{GL_n}$ is irreducible. I don't know an easy way to see that every nice $U_n$ representation extends to a nice $GL_n$ representation. I covered this material on October 10 of my representation theory course. See Chapter 5 of Joel Kamnitzer's lecture notes for a more general discussion.

I don't know of any good results about non-nice representations of $U_n$. Non-nice representations of $GL_n$, however, are interesting for two reasons. First of all, you might want to study $GL_n(\mathbb{C})$ as a topological group, ignoring its complex structure. For example, $SL_2(\mathbb{C})$ is a double cover of $SO(3,1)$. Physicists are very interested in the representation theory of $SO(3,1)$, but they don't particularly care that the representations be analytic.

Second, there are interesting infinite dimensional representations of $GL_n$. In contrast, by the Peter-Weyl theorem, any continuous Hilbert space representation of $U_n$ is a direct sum of finite dimensional representations, so passing to infinite dimensions doesn't make the $U_n$ theory deeper.