All subgroups normal $\implies$ abelian group
This is actually not true. A group for which all subgroups are normal is called a Dedekind group, and non-abelian ones are called "Hamiltonian". The smallest example is the quaternion group $Q_8$. See this MO discussion for more info.
Hint: consider $Q_{8}$, the quaternion group.