Does an equivalence of $G$-sets and $H$-sets imply an isomorphism of $G$ and $H$?

The category of left $G$-sets is equivalent to the category of left $H$-sets if and only if $G$ is isomorphic to $H$.

Indeed, consider $G$ as a left $G$-set. It has the property that $\mathrm{Hom}_G (G, -) : G \textbf{-Set} \to \mathbf{Set}$ preserves all colimits. Moreover, up to isomorphism, $G$ is the unique such left $G$-set, i.e. if $X$ is a left $G$-set such that $\mathrm{Hom}_G (X, -) : G \textbf{-Set} \to \mathbf{Set}$ preserves all colimits, then $X \cong G$. In addition, $G$ can be recovered as a group: $G \cong \mathrm{Aut} (X)$. Thus $G$ (as an abstract group) can be recovered from $G\textbf{-Set}$ (as an abstract category).

The above arguments are very specific to this situation. For example, if you replace $\mathbf{Set}$ with $\mathbf{Ab}$ then you can have non-trivial equivalences of categories of representations. This is the phenomenon of Morita equivalence.