Finite number of subgroups $\Rightarrow$ finite group

Yes, it's correct, though perhaps somewhat disorganized.

The fact that a group is a union of its cyclic subgroups holds regardless of any hypothesis on the group. So you can state and prove that first.

Then: an infinite cyclic group has infinitely many (cyclic) subgroups. So, a group is infinite if and only if it has infinitely many cyclic subgroups: If the group is finite, then it has only finitely many subsets, so finitely many subgroups, so finitely many cyclic subgroups. If the group is infinite, then being a union of its cyclic subgroups, either one of the cyclic subgroups is infinite (and hence there will be infinitely many cyclic subgroups, since that subgroup has infinitely many cyclic subgroups), or else there are infinitely many finite cyclic subgroups in the union. Either way you are done.

That is your argument, just organized a bit more.