Rotating 3D coordinates to 2D plane

Solution 1:

Let the vectors be denoted by $v_1$ and $v_2$ (my preference :)).

  1. Get the cross product of the two: $c = v_1 \times v_2$
    $c$ is orthogonal to both $v_1$ and $v_2$.

  2. Get the angle between $c$ and $k=(0,0,1)$: $$ \cos\theta = \frac{c\cdot k}{|c|} $$

  3. Rotate both vectors about $a = c \times k$ by $\theta$ using Rodrigues' rotation formula
    You have to normalize $a$ to get a unit vector before applying it to the formula: $a' = a/|a|$.

The pair of resulting vectors is one instance of possible rotations and both are perpendicular to $k$ ($z$-axis).

The rotation matrix can be obtained from Rotation matrix from axis and angle