The relation between axes of 3D rotations

Let's suppose we have two rotations about two different axes represented by vectors $v_1$ and $v_2$: $R_1(v_1, \theta_1)$, $R_2(v_2,\theta_2)$.

It's relatively easy to prove that composition of these two rotations gives rotation about axis $v_3$ distinct from axes $v_1$ and $v_2$ .

Indeed
if for example $v_3=v_1$ then
$R_1(v_1, \theta_1) R_2(v_2,\theta_2)=R_3(v_1,\theta_3)$ leads to $R_2(v_2,\theta_2)=R_1^T(v_1, \theta_1)R_3(v_1,\theta_3)=R(v_1,\theta_3 -\theta_1)$ what gives $v_1=v_2$. ... Contradiction...
We see that composition of two rotations about different axes always generates a new axis of rotation.

The problem can be extended for condition of the plane generated by the axes.

Question:

• Is it true that composition of two rotations generates the axis which doesn't belong to the plane which is constructed by the original axes of rotations ?

• How to prove it ?

• If the statement is not however true what are conditions for not changing a plane during the composition of rotations $ ^{[1]}$ ?

$ ^{[1]}$ It can be observed that even in the case of quite regular rotations the above statement is true

Let's take $Rot(z,\dfrac{\pi}{2})Rot(x,\dfrac{\pi}{2})= \begin{bmatrix} 0 & -1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 0 & -1 \\ 0 & 1 & 0 \\ \end{bmatrix} = \begin{bmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \\ \end{bmatrix} = Rot([1,1,1]^T, \dfrac{2}{3}\pi)$

or

$Rot(x, \pi )Rot(z, \pi )= \begin{bmatrix} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & -1 \\ \end{bmatrix} \begin{bmatrix} -1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} = \begin{bmatrix} -1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & -1 \\ \end{bmatrix} = Rot( y, \pi)$

So I suppose it is generally true but how to prove it ?

This is easily seen if we assume familiarity with the use of unit quaternions in representing rotations. A rotation $R$ about the axis given $\vec{v}=v_1\bf{i}+v_2\bf{j}+v_3\bf{k}$ by the angle $\theta$ is represented by the quaternion $$ q=\cos\frac\theta2+\sin\frac\theta2\vec{v}. $$ Here it is essential that $\vec{v}$ is a unit vector. The connection is that the rotated version $R\vec{u}$ of a vector $\vec{u}$ is then given by the quaternion product $$ R\vec{u}=q\vec{u}\overline{q}, $$ where $\overline{q}=\cos\frac\theta2-\sin\frac\theta2\vec{v}$ is the conjugate quaternion.

The composition of two such rotations is then faithfully reproduced as a product of the representing quaternions. So if another rotation $R'$ is represented by $q'=\cos\frac\alpha2+\sin\frac\alpha2\vec{v}'$, the composition $R'\circ R$ is represented by the product $$ \begin{aligned} qq'&=\left(\cos\frac\alpha2\cos\frac\theta2-\sin\frac\alpha2\sin\frac\theta2\,\vec{v}'\cdot\vec{v}\right)+\\ &+\cos\frac\alpha2\sin\frac\theta2\vec{v}+\cos\frac\theta2\sin\frac\alpha2\vec{v}'+\sin\frac\alpha2\sin\frac\theta2\,\vec{v}'\times\vec{v}. \end{aligned} $$ From the second row we can read the axis of the composition - it is the unit vector parallel to that linear combination of $\vec{v}$, $\vec{v}'$ and their cross product. The first two terms are in the plane $T$ spanned by $\vec{v}$ and $\vec{v}'$, but the cross product is perpendicular to $T$. Therefore the axis of the combined rotation is in the plane $T$ if and only if that cross product term is zero. Either of the sines vanishes only when the rotation is trivial ($\alpha=0$ or $\theta=0$). The cross product vanishes iff $\vec{v}$ and $\vec{v}'$ are parallel.

In other words, your hunch is correct.

Using quaternions, as in Jyrki’s answer, is straightforward and provides some useful insights into the nature of the resultant rotation. As he suspects, the result can also be found another way.

Suppose that the new rotation axis is coplanar with the original axes. Since vectors along the axis of rotation are eigenvectors of $1$, we must have $$R_1R_2(av_1+bv_2)=av_1+bv_2$$ for some $a$ and $b$ not both zero. Rearranging and again using the fact that $v_1$ and $v_2$ are eigenvectors of the respective rotations gives $$a(R_2-I)v_1=b(R_1^{-1}-I)v_2.$$ Wlog we can take $v_1$ and $v_2$ to be unit vectors. Substituting Rodrigues’ formula for the rotations then makes this $$a[(1-\cos{\theta_2})v_2\times(v_2\times v_1)+\sin{\theta_2}(v_2\times v_1)] = b[(1-\cos{\theta_1})v_1\times(v_1\times v_2)-\sin{\theta_1}(v_1\times v_2)]$$ which can be rearranged as $$[a(1-\cos{\theta_2})v_2+b(1-\cos{\theta_1})v_1]\times(v_2\times v_1) + (a\sin{\theta_2}-b\sin{\theta_1})(v_2\times v_1) = 0.\tag{*}$$ This clearly holds if $v_1$ and $v_2$ are colinear. If they aren’t, then the two vectors being added together in (*) are orthogonal, so they must both be zero. Any linear combination of $v_1$ and $v_2$ is orthogonal to $v_1\times v_2$, so the expression in square brackets must be zero, and since by hypothesis $v_1$ and $v_2$ are not colinear, we have $a(1-\cos{\theta_2})=0$ and $b(1-\cos{\theta_1})=0$, therefore at least one of the rotation angles must be zero.