Reference for "A finite group cannot be union of conjugates of a proper subgroup"

You might want to look at the following paper (1872) by Jordan:

C. Jordan, Recherches sur les substitutions, J. Liouville 17 (1872), 351–367. PDF link

In this paper, he proves that if a finite group $G$ acts transitively on a set $X$ of size $\geq 2$, then there exists an element $g \in G$ that does not fix any point in $X$.

The theorem you mention is a corollary of Théorème I. If $H$ is a proper subgroup, then $G$ acts transitively on the left cosets of $H$ by multiplication. Now $[G:H] \geq 2$ since $H$ is a proper subgroup, so there exists a $g \in G$ such that $gaH \neq aH$ for all $a \in G$. Hence $a^{-1}ga \not\in H$ and $g \not\in aHa^{-1}$ for all $a \in G$. In other words, $g$ is an element of $G$ that is not contained in $\bigcup_{a \in G} aHa^{-1}$.

In fact, we can use the theorem to prove Théorème I. If $G$ acts transitively on $X$, then the point stabilizers of $X$ are conjugate. Thus if every point fixes some point of $X$, then $G = \bigcup_{g \in G} aHa^{-1}$ for any $H = \operatorname{Stab}(x)$. If $|X| \geq 2$, then $H$ is a proper subgroup since $G$ is transitive.

I don't know a lot of French, but I don't think I see him actually mentioning in the paper that "a finite group cannot be the union of conjugates of a proper subgroup" (someone might want to check this). But I guess it is still safe to attribute that to Jordan, since it is an easy corollary of Théorème I, and the two results are equivalent anyway.