Abelian $p$-group with unique subgroup of index $p$

If $G$ is finite then every proper subgroup is contained in a maximal subgroup, so every proper subgroup is contained in $H$. I claim therefore that $\langle x \rangle = G$ for any $x\in G\setminus H$ - why?

If $\langle x \rangle$ were proper in $G$, then we would have $\langle x \rangle \leqslant H$, a contradiction. So $\langle x \rangle=G$.

Note that this argument does not require that $G$ is abelian. In fact, we could even extend your hypothesis to "$G$ is a finite group with a unique maximal subgroup."

Hint: It suffices to show that $G$ has precisely one subgroup of order $m$ for each $m$ dividing $|G|$. Show by induction that if we get more than one subgroup of order $p^k$ with $p^k\mid |G|$ then we can lift this to two distinct index $p$ subgroups. Thus, $G$ will have precisely one subgroup of order $p^k$ for each $p^k\mid|G|$ and thus $G$ will be cyclic.