If $H\unlhd G$ with $(|H|,[G:H])=1$ then $H$ is the unique such subgroup in $G$.

Here is a problem from "An introduction to the Theory of Groups" by J.J.Rotman:

Let $G$ be a finite group, and let $H$ be a normal subgroup with $(|H|,[G:H])=1$. Prove that $H$ is the unique such subgroup in $G$.

I assumed there was another normal subgroup like $H$, say $K$, such that $(|K|,[G:K])=1$. My aim was to show that $[K: K\cap H]=1 $ that was not held if I didn’t suppose $|H|=|K|$ . My question is if my last assumption about two subgroups is right? If it isn’t, please guide me. Thanks.

Alternatively, you might note that if $K$ is any other subgroup of order $|H|$, whether or not $K$ is normal (but assuming $H$ is normal), then $HK$ is a subgroup of $G$, so its order must divide $|G|$.

Let $L$ be any subgroup of $G$ with order $|H|$. Let $p$ be the natural projection $G \to \frac{G}{H}$. Then $p(L)$ is a quotient group of $L$, so $|p(L)|$ divides $|L|=|H|$. But $p(L)$ is also a subgroup of $\frac{G}{H}$, so $|p(L)|$ also divides $|G:H|$. By the coprimality hypothesis, $|p(L)|=1$ so $p$ is trivial on $L$. This means that $L \subseteq H$. Finally $L=H$ because those two subgroups have the same cardinality.

One way to see this is to look at the quotient map $f\colon G \to G/H$. Using Lagrange's theorem a couple of times, what can you say about the order of $f(K)$? [This isn't far from proving a well known formula for $\#(HK)$.]

Consider the more general result proven by user Gastón Burrull here. It states

If $K\triangleleft G$, $|K|$ finite, $H\leq G$, $[G:H]$ finite and $|K|$, $[G:H]$ are relatively prime then $H\leq K$

In other words, if the index of a subgroup is coprime to the cardinality of a normal subgroup, then it is contained in that normal subgroup. So if $K$ is any other subgroup of cardinality $|H|$, then since the group is finite, $H$ and $K$ have the same index, hence the index of $K$ is coprime to $|H|$, and so $K\leq H$. But since $K$ and $H$ are finite subgroups of equal order, $K=H$. So $H$ is the only subgroup of order $|H|$.

To make things explicit: Assume $\lvert H \rvert = \lvert K \rvert$.

As a consequence of the 2nd isomorphism theorem, $HK \le G$ and $$\lvert HK \rvert = \frac{\lvert H \rvert \lvert K \rvert}{\lvert H \cap K \rvert} = \frac{\lvert H \rvert^2}{\lvert H \cap K \rvert}$$

By Lagrange's Theorem: $$\lvert G \rvert = [G:H] \lvert H \rvert = [G:HK] \lvert HK \rvert $$

We derive $$[G:H] = [G:HK] \frac{\lvert H \rvert}{\lvert H \cap K \rvert}$$

If $\lvert H \cap K \rvert < \lvert H \rvert$ then there is some factor $n \ne 1$ of $\lvert H \rvert$ on the RHS (note: $\lvert H \vert /\lvert H \cap K \rvert = n \in \mathbb N$). Thus $n \mid [G:H] $, a contradiction that $\operatorname{gcd}(\lvert H \rvert, [G:H]) = 1$. We conclude $\lvert H \cap K \rvert = \lvert H \rvert$ and since $\lvert H \rvert = \lvert K \rvert$ we can conclude $H = K$.