What are the symmetries of a colored rubiks cube?

First consider an unpainted $2\times 2\times 2$ Rubik's cube:

What is the "symmetry group"?

Before we can discuss the symmetry group of this cube (or any Rubik's cube), we must be clear about what operations are allowed. There are three pertinent questions:

1. Do rigid rotations of the cube count as "symmetries"?
   (If no, we must somehow exclude them.)

2. Are we allowed to reflect the cube?

3. Are we allowed to take the cube apart and put it back together?

In most discussions of the $3\times 3\times 3$ Rubik's cube, the answer to all three questions is "no". In particular, it is common to disallow rigid rotations by requiring that each of the six center faces be fixed.

For our discussion, we will use "yes" for the first question, but "no" for the other two.

Of course, it would be possible to answer "no" for the first question by fixing the position of one of the corner pieces. Unfortunately, $\mathsf{GAP}$ claims that the resulting group (which is the same as the group defined by the code at http://cubeman.org/2x2x2.txt) does not contain a copy $Q_8$. If we want to get a $Q_8$, we will need to allow rigid rotations.

Structure of the Group

Let $G$ be the full symmetry group of the $2\times 2\times 2$ Rubik's cube. This group has order 88,179,840, and turns out to be a semidirect product: $$ G \;=\; (\mathbb{Z}_3)^7 \rtimes S_8. $$ The normal subgroup $N = (\mathbb{Z}_3)^7$ consists of all elements of $G$ that rotate each of the eight subcubes in place.

Note that $N$ is not isomorphic to $(\mathbb{Z}_3)^8$, even though there are eight subcubes. The reason is that not all rotations of the eight subcubes are possible. In particular, $N$ turns out to be the subgroup of $(\mathbb{Z}_3)^8$ consistsing of all elements whose coordinates sum to zero modulo three.

The group $S_8$ acts on $N$ by permuting the eight coordinates in this $(\mathbb{Z}_3)^7$. That is, $S_8$ permutes the eight cubes. Since this is not a normal subgroup, there are several possible copies of $S_8$ to choose from, none of which is "canonical". We will see one possible choice below.

Coloring Opposite Faces

If we color two opposite faces of the cube red, we can ask about the subgroup $H$ of symmetries that preserve the coloring.

According to $\mathsf{GAP}$, this group is isomorphic to $S_8$, which isn't too surprising: elements of $H$ permute the eight red faces, so the only question was whether any permutation would be allowed.

Incidentally, it turns out that this group acts on the remaining sixteen gray faces with two orbits. The following picture shows the cube with these two orbits labeled yellow and blue.

This is the "chessboard" coloring mentioned by the OP. Any symmetry of the cube that maps red faces to red faces automatically maps blue faces to blue faces and yellow faces to yellow faces as well.

The Quaternion Subgroup

Since $H$ is isomorphic to $S_8$, it has a subgroup isomorphic to $Q_8$. According to $\mathsf{GAP}$, there is only one conjugacy class of $Q_8$ subgroups in $G$, so the following description will be unique up to conjugacy.

To describe the $Q_8$ subgroup, I will describe the action of the three generators $i$, $j$, and $k$. The generator $i$ simply acts as a $90^\circ$ rotation of the cube:

(action of $\boldsymbol{i}$)

The generator $j$ does something interesting: it switches the top and bottom of the cube, while also rotating them $180^\circ$ degrees relative to one another. The following picture shows this operation.

(action of $\boldsymbol{j}$)

Since $k=ij$, the action of $k$ is simply the action of $i$ followed by the action of $j$. This turns out to be a transformation that's somewhat similar to $j$.

(action of $\boldsymbol{k}$)

More generally, each of the eight elements of $Q_8$ can be obtained as follows:

1. Either flip the cube over (for $j$, $-j$, $k$, and $-k$) or don't (for $1$, $i$, $-1$, and $-i$).

2. If you flipped the cube over, rotate the top layer $180^\circ$.

3. Finally, rotate the entire cube a multiple of $90^\circ$ around the vertical axis.

A few notes about this copy of $Q_8$:

1. According to $\mathsf{GAP}$, there is only one conjugacy class of $Q_8$ subgroups of $G$. Thus, this is essentially the only way for $Q_8$ to act by symmetries on the Rubik's cube (not counting moves that reflect the cube, or take the cube apart and put it back together).

2. This subgroup does not correspond to some coloring of the faces of the cube. In particular, $Q_8$ acts transitively on the faces colored red, yellow, and blue in the pictures above, so this coloring cannot be refined further.

Another Interesting Subgroup

In the comments above, Rschwieb mentioned the possibility of coloring opposite faces the same color:

For this coloring, it turns out that there are two orbits of subcubes: those which are colored red-yellow-blue in counterclockwise order, and those that are colored red-yellow-blue in clockwise order. There are four of each type, and it turns out the subgroup of $G$ that preserves this coloring is isomorphic to $S_4 \times S_4$.

Method

All of these computations were done using $\mathsf{GAP}$, and the animations were made in Mathematica. I used the following $\mathsf{GAP}$ code to define the group $G$:

cube := Group( (2,14,23,9)(3,6,22,17)(7,15,16,8), (3,11,24,16)(4,19,23,8)(9,10,18,17), (20,14,16,18)(19,13,15,17)(21,22,23,24), (1,2,3,4)(5,7,9,11)(6,8,10,12), (1,7,22,20)(2,15,21,12)(5,6,14,13), (1,10,24,13)(4,18,21,5)(11,19,20,12) );

That is, $G$ the a subgroup of $S_{24}$ generated by the given permutations. These generators were obtained by numbering the faces of the cube, and then examining the effect of each of the six possible face-rotation moves.

$\mathsf{GAP}$ is fantastic at dealing with permutation groups. The following code gives the group $H$ defined above (the group that preserves the red coloring of two opposite faces).

H := Stabilizer( G, [1,2,3,4,21,22,23,24], OnSets );

Here 1,2,3,4,21,22,23,24 are the numbers for the eight red faces. The command

StructureDescription(H);

reveals that $H$ is isomorphic to $S_8$. I used the Stabilizer command to define all relevant groups, and the IsomorphicSubgroups command to search for subgroups isomorphic to $Q_8$. A similar method could be used to analyze any coloring.