Infinite group has infinitely many subgroups, namely cyclic subgroups.

I think the contrapositive is much clearer:

If a group has finitely many subgroups, then the group is finite.

Indeed, let $G$ be a group with finitely many subgroups. Then $G$ has finitely many cyclic subgroups. An infinite cyclic group has infinitely many subgroups. Therefore, all cyclic subgroups of $G$ are finite. Finally, $G$ is finite because it is the union of its cyclic subgroups, which is a finite union of finite sets.

The proof given is correct, and I'm suggesting an alternative only for the sake of style/clarity (which is more subjective than correctness).

The point in the OP's proof where a detailed argument appears is nested inside the case analysis (finitely many vs. infinitely many cyclic subgroups). Pulling that argument out as a Lemma serves both to motivate the result and to simplify the main argument that follows:

Lemma An infinite cyclic group has infinitely many (cyclic) subgroups.

Proof: An infinite cyclic group is isomorphic to additive group $\mathbb Z$. Each prime $p\in \mathbb Z$ generates a cyclic subgroup $p\mathbb Z$, and distinct primes give distinct subgroups. So the infinitude of primes implies $\mathbb Z$ has infinitely many (distinct) cyclic subgroups. QED

Proposition An infinite group has infinitely many (cyclic) subgroups.

Proof: Let $G$ be an infinite group. Every $g\in G$ belongs to at least one cyclic subgroup of $G$, namely $\langle g \rangle$. (1) If there exist infinitely many (distinct) cyclic subgroups of $G$, then we are done.

So assume (2) $G$ has only finitely many cyclic subgroups $H_1,H_2,\ldots,H_k$. Since $G$ is infinite, at least one of these $H_i$ must be infinite (otherwise we have a finite covering of $G$ with finite sets, implying $G$ is finite). Then the Lemma above says such infinite $H_i$ has infinitely many cyclic subgroups, which implies also that $G$ does (since a cyclic subgroup of $H_i$ is a cyclic subgroup of $G$). QED

Assumption (2) actually leads to a contradiction, but we haven't highlighted that. Some authors would prefer to phrase the proof in those terms, but I wanted to emphasize keeping your structure of proof after pulling out the case where $G$ is infinite cyclic as a Lemma.