Finite Subgroups of $GL_2(\mathbb Q)$
Solution 1:
One approach is the following: Let $M = M_{2}(\mathbb{Q}).$ Note that if $x$ is an element of order $4$ or $3$ in $G,$ then the subalgebra $C_{M}(x)$ is a division ring by Schur's Lemma, as $x$ acts irreducibly. Hence $C_{G}(x)$ is cyclic. In particular, Sylow $3$-subgroup of $G$ is cyclic, of order $3$ as you have shown already. Let $S$ be a Sylow $2$-subgroup of $G,$ and let $A$ be a maximal Abelian normal subgroup of $S$. If $A$ has exponent $2,$ then $A$ is elementary Abelian, and hence has order at most $4.$ Otherwise, $A$ contains an element $x$ of order $4,$ and then $A$ is cyclic as $C_{G}(x)$ is cyclic. Also, as you have shown, $|A| \leq 4$ in that case. Now as $A$ is maximal Abelian normal in $S,$ we have $C_{S}(A) = A,$ and $S/A$ is isomorphic to a subgroup of ${\rm Aut}(A).$ If $A$ is a Klein $4$-group, then ${\rm Aut}(A) \cong S_{3}$ and if $A$ is cyclic of order $,$ then ${\rm Aut}(A)$ has order $2.$ Hence in either case, $|S| \leq 8.$ Since you have already ruled out a quaternion group of order $8,$ the only possibility if $S$ is non-Abelian of order $8$ is that it is dihedral ( and we already know that if $|S| = 8,$ it is non-Abelian).
By the way, you can't rule out all non-Abelian group of order $12,$ since a dihedral group with $12$ elements does occur.
I don't know how much group theory you know, but I would finish as follows: let $F = F(G),$ the Fitting subgroup of $G.$ If $F$ contains an element of order $3$ or then $C_{G}(F) = F$ is cyclic of order $3$ or $6$ (for $F$ contains a central element of order $3$ in that case). Then $G/F$ has order dividing $2$. I won't give all details, but if $|G/F| =2,$ that leads to $G$ dihedral with $6$ or $12$ elements ($G=F$ leads to $G$ cyclic of order $3$ or $6$ in this case).
There remains the case that $F$ is a $2$-group. If $F$ is Abelian , we have $F$ cyclic of order $4$ or a Klein $4$-group. The first possibility leads to $G = F.$ The second also leads to $G = F,$ eg by Clifford's theorem. there can be no element of order $3$ in $G$ in that case. If $F$ is non-Abelian of order $8,$ then we must also have $G = F$, because a dihedral group of order $8$ has no automorphism of order $3.$