If $|H|$ and $[G:K]$ are relatively prime, then $H \leq K$

Let $H$ and $K$ be subgroups of a finite group $G$, at least one of which is normal. Show that if $|H|$ and $[G:K]$ are relatively prime, then $H \leq K$.

In the case that $K$ is normal, let $\pi : G \rightarrow G/K$ be the quotient map. Then $\pi(H)$ is a subgroup of $G/K$ and so $|\pi(H)|$ must divide both $|G/K|=[G:K]$ and $|H|$. These are relatively prime, so that $|\pi(H)|=1$, or $H \leq \ker \pi = K$.

If $H$ is normal, I'm not sure what to try. A hint would be perfect. This question is similar, but omits the normality hypothesis.

Hint : $HK$ is a subgroup of $G$ and $[HK:K]$ divides $[G:K]$. Here's a reasonably complete answer, so look away if you don't want that yet:

Also $|HK|/|K| = [H:H \cap K]$ divides $|H|$. Thus $[HK:K]$ divides two relatively prime integers, so divides $1$. Hence $[H:H \cap K] = 1$ and $H \leq K$. You don't really need normality, just that $HK=KH$ is a subgroup ( by the way, whether or not $HK$ is a subgroup, its cardinality is $|H| |K|/|H \cap K|$).

EDIT I see now that you just wanted a hint. Here's a hint: Using your idea in the first case, show that $|H|$ must divide $|K|/|\pi(K)|$ and that $|K|/|\pi(K)| = |H\cap K|\leq |H|$. Show this implies the inequality is actually an equality and conclude what you want from there.

Read further on only if you want everything spoiled. (I typed it all out before I realized you just wanted a hint, and on the off chance someone wants the whole answer, I think it's best to leave it up at this point).

Using you idea from the first case: If $H$ is normal, then we can consider $\pi(K)$ in $G/H$. As in your first case, we learn that $|\pi(K)|$ divides $|G|/|H|$. Rearranging what it means to divide, we get that $|H|$ divides $|G|/|\pi(K)| = |G|/|K| * |K|/|\pi(K)|$. Since $H$ and $|G|/|K|$ are relatively prime, it must be the case that $|H|$ divides $|K|/|\pi(K)|$.

Next, I claim that $|K|/|\pi(K)| = |\ker\pi \cap K|$. To see this, once and for all pick representatives $k_1,...,k_n$ for each element of $\pi(K)$ and define a function $f: (\ker\pi\cap K)\times \pi(K)$ by $f(k, \pi(k_i)) = kk_i$.

This function is 1-1 since if $kk_i = k'k_j$, then $k_ik_j^{-1} = k'k^{-1}\in\ker\pi$ so $\pi(k_i) = \pi(k_j)$ which implies $i=j$ by our choice of representatives. Then, this easily implies $k = k'$.

To see it's onto, let $g\in K$. Let $k_i$ be the representative we chose for $\pi(k)$ and set $k = gk_i^{-1}$. Then $k\in \ker\pi\cap K$ and clearly $kk_i = g$. Thus, $f$ is a bijection between the two sets, establishing the claim. (This must be some form of the orbit-stabilizer theorem, but I just couldn't find the right way to apply it.)

Putting this together, we know that $|H|$ divides $|\ker\pi \cap K|$. But $\ker \pi = H$ so $|\ker\pi\cap K| \leq |H|$. Thus, we must have that $|H|=|\ker\pi\cap K| = |\ker \pi|$ which implies that $H = \ker \pi\subseteq K$.