Prove that every finite p-group is nilpotent.

Let $G$ be a p-group $|G| = p^n$. Then G is nilpotent in the following sense:

Let $C^1(G) = [G,G] = G'$ And $C^n(G) = [G,C^{n-1}]$.

G is nilpotent iff there is a $n_0$ such that $C^n = \{e\}$.

What I proved in previous exercises is:

If H is a proper subgroup of a nilpotent group then H is a normal proper subgroup of its normalizer. i.e. if $H \lneq G$ then $H \lneq N_G(H)$.

I use induction: If $|G| = p$, then G is isomorphic to $\mathbb{Z}/p\mathbb{Z}$ which is cyclic. therefore $C^1(G) = G' = \{e\}$. Suppose that the following assertion is true:

if $|L| = p^i$, for $i \lt n$ then L is nilpotent.

Let G be a group of order $p^n$. If I can just prove $G' \neq G$, then we'd have $|G'| = p^i, i \lt n$ and in such a case G' would be nilpotent. So that G would be nilpotent.

Is it? If G is a p-group, then $G'\neq G$?

A very easy way to prove that $G'\neq G$ is to note that $G'$ is the smallest normal subgroup of $G$ such that $G/G'$ is abelian. If you can find any proper normal subgroup of $G$, $N$, such that $G/N$ is abelian, then it will follow that $G'\subseteq N$, and so $G'\neq G$.

For example: have you proven that a maximal subgroup of a $p$-group must be normal? This can be done easily by induction, using the fact that $Z(G)$ (the center) is nontrivial. With that in hand, it follows that $G'$ is always contained in a maximal subgroup of $G$, and you'd be done.

(The previous exercise you quote can't help you because it assumes your group is nilpotent, and here you are trying to prove that a group is nilpotent).

Added. Another way of proving this:

Show that if $f\colon G\to K$ is an onto group homomorphism, then $f([G,G])=[K,K]$. In particular, if $[K,K]\neq K$, then $[G,G]\neq G$. Then use the fact that if $G$ is a $p$-group then $Z(G)\neq \{1\}$ and induction to show that either $Z(G)=G$ (so $[G,G]=\{1\}$) or else $[G/Z(G),G/Z(G)]\neq G/Z(G)$ and get the result.

Note: Your notation for the lower central series is uncommon. It is more common to use $G_1=G$ and $G_{n+1}=[G_n,G](=[G,G_n])$.

Added. I wasn't paying enough attention when I answered this, and what Derek Holt points out in comments is quite true: it is not enough to prove that $[G,G]$ is nilpotent in order to prove that $G$ is nilpotent, even if $[G,G]\neq G$: there are groups where $[G,G]$ is nilpotent, but $G$ is not. For example, $S_3$ has $[G,G]\cong A_3$, which is abelian, but $S_3$ is not nilpotent.

Instead, we can proceed by induction on $n$, where $|G|=p^n$.

If $n=1$ or $n=2$, then $G$ is abelian, hence $G$ is nilpotent.

Assume that $G$ has order $p^n$. If $Z(G)=G$, then $G$ is abelian and hence is nilpotent. Otherwise, consider $\mathfrak{G}=G/Z(G)$. This is a $p$-group of order strictly less than $p^n$ (since the center of a finite $p$-group is always nontrivial), so by the induction hypothesis, $\mathfrak{G}$ is nilpotent. Now we use the lemma mentioned above, slightly strengethened (I will use $H_n$ to denote the $n$th term of the lower central series of $H$, that is, $H_n$ is what the original post calls $C^n(H)$):

Lemma. Let $f\colon G\to K$ be a group homomorphism. Then for every $m$ we have $f(G_m) \subseteq K_m$. If $f$ is onto, then $f(G_m)=K_m$ for all $m$.

Proof. The result holds for $m=1$. Assume $f(G_m)\subseteq K_m$. A generator for $G_{m+1}$ is of the form $[g,g']$, where $g'\in G_m$, hence $f([g,g']) = [f(g),f(g')]\in [K,K_m]=K_{m+1}$, so $f(G_{m+1})\subseteq K_{m+1}$, as claimed. Assume now that $f$ is onto and $f(G_m)=K_m$. A generator for $K_{m+1}$ is of the form $[k,k']$ with $k\in K$ and $k'\in K_{m+1}$; since $f$ is onto there exists $g\in G$ with $f(g)=k$; since $f(G_m)=K_m$, there exists $g'\in G_m$ with $f(g')=k'$. Hence $f([g,g'])=[f(g),f(g')] = [k,k']$, and since $[g,g']\in G_{m+1}$, it follows that $K_{m+1}\subseteq f(G_{m+1})$, as desired. $\Box$

Now, since $\mathfrak{G}$ is nilpotent, there exists $r$ such that $\mathfrak{G}_r$ is trivial; therefore, by the lemma, $f(G_r)$ is trivial, so $G_rZ(G) = Z(G)$. Hence $G_r\subseteq Z(G)$, so $G_{r+1}=[G,G_r]\subseteq [G,Z(G)] = \{1\}$, so $G$ is nilpotent, as claimed. $\Box$

Here is a beautiful little proof, but it depends on a theorem you may not have known. Also, this only works for finite $p$-groups.

Proof that every finite $p$-group is nilpotent:

Let $G$ be a finite $p$-group.

We do a proof by induction on the order of $G$.

Base case:

$|G| = 1$. Then $G = \{e\}$. $G$ is clearly nilpotent.

Inductive step:

$|G| > 1$. We assume that all $p$-groups of order less than $|G|$ are nilpotent (our inductive hypothesis).

$G$ is a nontrivial $p$-group, so $Z(G)$ is also nontrivial (a lemma that can be proved using the conjugacy class equation).

$Z(G)$ is abelian, so it is nilpotent.

$|G/Z(G)| < |G|$ since $Z(G)$ is nontrivial. So $G/Z(G)$ is nilpotent by the induction hypothesis.

Since $Z(G)$ is nilpotent and $G/Z(G)$ is nilpotent, it follows that $G$ is nilpotent (a theorem).

By induction, we conclude that every $p$-group of finite order is nilpotent!! :)

The theorem we used is:

$G$ is a group. If $N \leq Z(G)$ and $G/N$ is nilpotent, then $G$ is nilpotent.

$G$ and all its nontrivial quotients are $p$-groups, and therefore have non-trivial centers.

Hence if $G\neq C_i(G)$, then $G/C_i(G)$ is a $p$-group, and $C(G/C_i(G))$ is non-trivial. Thus $C_{i+1}(G)$, the inverse image of $C(G/C_i(G))$ under $\pi:G\to G/C_i(G)$, strictly contains $C_i(G)$.

Since $G$ is finite, $C_n(G)$ must be $G$ for some $n$.

I would like to use a different notation than $C^n(G)$ used by OP as following:

$\gamma_1(G) = G$;

$\gamma_{i+1}(G) = [\gamma_i(G), G]$ for $i >= 1$;

A group $G$ is nilpotent if $\gamma_{c+1}(G) = 1$ for some $c$.

First we need this Corollary 1:

Let $G$ be a finite p-group. Then the center of $G$ is nontrivial.

The proof is here.

Then we need this Lemma 2:

If $\pi:G \rightarrow K$ is a surjective homomorphism, then $\pi(\gamma_i(G)) = \gamma_i(K)$ for all $i$.

( Ask and comment me for the proof. )

We prove by induction on $|G|$. Suppose $|G| > 1$ and assume that the statement "p-group is nilpotent" is true for p-groups with smaller order. It would be eventually as if $|G| = 1$ then $\gamma_1(G) = G = 1$ so $G$ is nilpotent.

By Corollary 1, $Z(G) \ne 1$. Consider the quotient group $G/Z(G)$, and as $Z(G) \ne 1$, $|G/Z(G)| < |G|$, hence $G/Z(G)$ is a p-group with order smaller than $G$. By induction assumption above, $G/Z(G)$ is nilpotent. By definition of nilpotent group we have:

$\gamma_{x+1}(G/Z(G)) = 1$.

Let $\pi:G \rightarrow G/Z(G)$ be the homomorphism. By Lemma 2:

$\pi(\gamma_{x+1}(G)) = \gamma_{x+1}(G/Z(G)) = 1$.

But $\pi(\gamma_{x+1}(G)) = 1$ means that $\gamma_{x+1}(G) \le ker \: \pi = Z(G)$. Thus:

$\gamma_{x+2}(G) = [\gamma_{x+1}(G), G] \le [Z(G), G] = 1$ because $\gamma_{x+1}(G) \le Z(G)$ and $[Z(G), G] = 1$.

As $\gamma_{x+2}(G) = 1 = \gamma_{c+1}(G)$, $G$ is nilpotent as required.