Rotating 3D coordinates to 2D plane
Solution 1:
Let the vectors be denoted by $v_1$ and $v_2$ (my preference :)).
-
Get the cross product of the two: $c = v_1 \times v_2$
$c$ is orthogonal to both $v_1$ and $v_2$. -
Get the angle between $c$ and $k=(0,0,1)$: $$ \cos\theta = \frac{c\cdot k}{|c|} $$
-
Rotate both vectors about $a = c \times k$ by $\theta$ using Rodrigues' rotation formula
You have to normalize $a$ to get a unit vector before applying it to the formula: $a' = a/|a|$.
The pair of resulting vectors is one instance of possible rotations and both are perpendicular to $k$ ($z$-axis).
The rotation matrix can be obtained from Rotation matrix from axis and angle