A Nontrivial Subgroup of a Solvable Group

A simple solvable group must be cyclic of prime order (since it must be abelian, and so cannot have proper [normal] subgroups). But a simple solvable group would not contain nontrivial normal subgroups, so the proposition would be true for such a group by vacuity (the hypotheses are never satisfied). Alternatively, if you allow the whole group to be a "nontrivial normal subgroup", your $H$ can only be $G$ itself, which is already abelian, so you can set $A=H=G$; either way, the proposition is true for such a group.

(If $G$ is solvable, then $[G,G]$ is a proper subgroup of $G$; since it is always normal in $G$, if $G$ is also simple, then we must have $[G,G]=\{1\}$, hence $G$ is abelian).

Hint for the question. If $H$ is abelian, you are done. If not, then $[H,H]$ is nontrivial; use the fact that $H^{(n)}\subseteq G^{(n)}$, where $G^{(k)}$ is the $k$th term of the derived series of $G$, to show that $H$ is solvable, and use the fact that $H\triangleleft G$ to show $[H,H]\triangleleft G$. Then replace $H$ with $[H,H]$, lather, rinse, and repeat.

Different hint. Let $1=G_0\triangleleft G_1\triangleleft\cdots\triangleleft G_s = G$, with $G_i/G_{i+1}$ abelian. By Problem 8, you can pick the $G_i$ normal in $G$. Look at $H_i=H\cap G_i$.

Added. Sigh. I didn't notice that Problem 8 assumes $G$ is finite; the result is true, as witnessed by the derived series, but again it's probably not what you want.

Added 2. Okay, this should work; it takes some of the ideas of the hint in Problem 8 of the same page, so it should be "reasonable". Let $i$ be the largest index such that $G_i\cap H$ is a proper subgroup of $H$. Then $H\subseteq G_{i+1}$, hence $G_i\cap H\triangleleft H$. Moreover, $H/(H\cap G_i) \cong (HG_i)/G_i \leq G_{i+1}/G_i$, so $H/(H\cap G_i)$ is abelian. As in problem 8, this means that $x^{-1}y^{-1}xy\in H\cap G_i$ for all $x,y\in H$. Show that this is also true for all $G$-conjugates of $H\cap G_i$, hence their intersection, which is normal in $G$, contains all $x^{-1}y^{-1}xy$ with $x,y\in H$. If this is trivial, then $H$ is abelian and we are done. If it is not trivial, then this intersection is nontrivial, and normal in $G$. Replace $H$ with this intersection, and note that the largest index $j$ such that the intersection with $G_j$ is proper is striclty smaller than $i$; so you can set up a descent. Lather, rinse, and repeat.

Hints (for you to prove):

1) It is true that

$$H\geq H'\geq\ldots\geq H^{(n)}=1\,\, ,\,\, \text{for some}\,\,\,n\in\Bbb N$$

2) Show that $\,H^{(n-1)}\triangleleft G\,\;\;$ (Yes, not only in $\,H\,$ ...!)