Is there a group which has precisely all finite groups as subgroups?

No. Jeremy already commented that we may assume the center to be trivial.

Now look at the involutions (= elements of order 2). Their centralizers are finite, so there are countably many conjugacy classes of involutions. Two involutions generate a dihedral group. If both involutions are not conjugate (in $G$ and hence within the dihedral group), the dihedral group has a non-trivial center (contained in the centralizer of either involution). As there are infinitely many involutions not conjugated to a fixed involution $\tau$, there has to be a non-trivial element $\sigma$ of the (finite) centralizer of $\tau$ that lies in the centralizers of infinitely many of those involutions. Hence the centralizer of $\sigma$ is a proper infinite subgroup of $G$.