All subgroups normal $\implies$ abelian group

This is actually not true. A group for which all subgroups are normal is called a Dedekind group, and non-abelian ones are called "Hamiltonian". The smallest example is the quaternion group $Q_8$. See this MO discussion for more info.


Hint: consider $Q_{8}$, the quaternion group.