Let $H\leq G$. Prove $x^{-1}y^{-1}xy\in H\text{ }\forall x,y\in G \iff H\trianglelefteq G \text{ and } G/H \text{ is abelian}$.

For backward direction, you already started really well. Since $G/H$ is abelian, we have $$H = (xH)(x^{-1}H)(yH)(y^{-1}H) =(x^{-1}H)(y^{-1}H)(xH)(yH) = (x^{-1}y^{-1}xy)H$$ since we can swap places of elements in $G/H$.

For forward direction, in order to show $H \unlhd G$, pick an element of the form $g^{-1}hg \in G$. Then, $$h^{-1}g^{-1}hg \in H \implies g^{-1}hg \in hH = H \implies g^{-1}Hg \subseteq H \implies H \unlhd G$$ I leave as an exercise to you to show that $G/H$ is abelian.

For the forward direction, you have that $y^{-1}xy \in xH$ for all $x$ and $y$ in $G$, as you say. Therefore, if $y$ is an arbitrary element of $G$, and $x$ is an arbitrary element of $H$, then $xH = H$, and therefore $y^{-1}xy \in H$. This is exactly what it means for $H$ to be a normal subgroup of $G$.

To see that $G/H$ is abelian, you need only show that $xyH = yxH$ for any $x, y \in G$. This is equivalent to saying that $x^{-1}y^{-1}xyH = H$ for all $x$ and $y$, which is of course true by your hypothesis.

It is quite easy to get the reverse direction from the previous paragraph.