What finite groups always have a square root for each element?
If $G$ is a finite group of even order, then it has an element of order $2$, $a$ say. Then $a^2=e=e^2$, and the squaring map is not injective. By finiteness, the squaring map is not surjective: there are elements in $G$ which aren't squares.
If $G$ has odd order, then for each $b\in G$, $b^{(|G|+1)/2}$ is a square root of $b$.