Why is the set of commutators not a subgroup?
I was surprised to see that one talks about the subgroup generated by the commutators, because I thought the commutators would form a subgroup. Some research told me that it's because commutators are not necessarily closed under product (books by Rotman and Mac Lane popped up in a google search telling me). However, I couldn't find an actual example of this. What is one? The books on google books made it seem like an actual example is hard to explain.
Wikipedia did mention that the product $[a,b][c,d]$ on the free group on $a,b,c,d$ is an example. But why? I know this product is $aba^{-1}b^{-1}cdc^{-1}d^{-1}$, but why is that not a commutator in this group?
I. D. MacDonald gives reasonable examples in [I. D. MacDonald, Commutators and Their Products, The American Mathematical Monthly Vol. 93, No. 6 (1986), pp. 440-444]
If you have access to JSTOR, it is at http://www.jstor.org/stable/2323464
In particular, he proves by a simple counting argument the nice theorem that
if $G$ is a finite group and $|G:Z(G)|^2<|G'|$, then $G'$ has elements which are not commutators.
Here $Z(G)$ is the center of $G$ and $G'$ its derived subgroup.
I had minor problems convincing myself of the fact that the group described by Geoff exists (See also Derek Holt's answer to the previous version of this question), most notably that it has the prescribed order. So I spent some time on it, and want to share this more concrete version. Hopefully I didn't fumble this.
Inside the group of upper triangular 3x3 matrices (entries from $F_p$) we have the (often used) matrices $$ A=\left(\begin{array}{ccc}1&1&0\\0&1&0\\0&0&1\end{array}\right),\quad B=\left(\begin{array}{ccc}1&0&0\\0&1&1\\0&0&1\end{array}\right),\quad C=\left(\begin{array}{ccc}1&0&1\\0&1&0\\0&0&1\end{array}\right), $$ satisfying the relations $A^p=B^p=C^p=1, [A,B]=C, [A,C]=[B,C]=1$.
Using these we can realize that group as $m\times m$ upper triangular matrices, where $m=3n(n-1)/2$, using $n(n-1)/2$ blocks (sized 3x3) along the diagonal. Label the blocks with pairs of indices $(i,j), 1\le i<j\le n$. The generator $x_i$ has matrix $A$ in any block with label $(i,y), y>i$, matrix $B$ in any block with label $(x,i),x<i$ and the identity matrix in the other blocks. Consequently the commutator $[x_i,x_j]$ has the matrix $C$ in the block labelled $(i,j)$ and the identity matrix elsewhere.
The entire group $G$ then consists of matrices with blocks $$ g_{i,j}=\left(\begin{array}{ccc}1&u_i&v_{i,j}\\0&1&u_j\\0&0&1\end{array}\right)=A^{u_i}C^{v_{i,j}-u_ju_i}B^{u_j}, $$ where the $n(n+1)/2$ coefficients $u_i,v_{i,j}$ are arbitrary elements of $F_p$.
If there is a simpler concrete description of this group, I'm all ears :-).
Edit: Anyway, we have $x_i^p=1$ for all $i$, $[x_i,x_j]^p=1$ for $i<j$ and all the commutators are central. The commutator subgroup consists of all those matrices with $u_i=0$ for all $i$. A commutator of two elements $[(g_{i,j}),(g'_{i,j})]$ has $v_{i,j}=u_iu'_j-u_ju'_i$, and there are too few of those.