What are all finite groups such that all isomorphic subgroups are identical?

There is a simple argument that shows that any two subgroups of a cyclic group that are isomorphic must be identical. This is because they can each be represented in terms of the generator of the cyclic group.

This made me wonder, what are all finite groups with the property that any two isomorphic subgroups are identical? My conjecture is that they must all be cyclic. Is this correct?

Let $G$ be a finite group with this property. Let $a\in G$ and let $a$ have order $k$. Then every element of order $k$ in $G$ generates $\left<a\right>$. So there are exactly $\varphi(k)$ elements of order $k$ in $G$. So for each $k$, $G$ has either $0$ or $\varphi(k)$ elements of order $k$.

Let $|G|=n$. Then for each $k$ for which there are elements of order $k$ in $G$, then $k\mid n$. But $n=\sum_{k\mid n}\varphi(k)$. If for some $k\mid n$ there are no elements of order $k$ in $G$, then the total number of elements in $G$ is $<n$, which is impossible.

In particular, $G$ has an element of order $n$. Thus $G$ is cyclic.

From the comments, here is short answer.

As pointed out, any exception to the conjecture must be a group that is non-abelian /and/ has every subgroup normal.

This means the group has the property of being a Dedekind group. In particular, there is a result that the quaternion group $Q_8$ is a subgroup of all such groups. The quaternion group as 6 elements of order four, which cannot belong to a single cyclic group of order four.

We conclude such an exception is not possible.

This depends heavily of the properties of a Dedekind group, which I have no insight to. A better answer would be much appreciated.