Signed angle between two 3D vectors with same origin within the same plane

What I need is a signed angle of rotation between two vectors Va and Vb lying within the same 3D plane and having the same origin knowing that:

1. The plane contatining both vectors is an arbitrary and is not parallel to XY or any other of cardinal planes
2. Vn - is a plane normal
3. Both vectors along with the normal have the same origin O = { 0, 0, 0 }
4. Va - is a reference for measuring the left handed rotation at Vn

The angle should be measured in such a way so if the plane would be XY plane the Va would stand for X axis unit vector of it.

I guess I should perform a kind of coordinate space transformation by using the Va as the X-axis and the cross product of Vb and Vn as the Y-axis and then just using some 2d method like with atan2() or something. Any ideas? Formulas?

Solution 1:

Use cross product of the two vectors to get the normal of the plane formed by the two vectors. Then check the dotproduct between that and the original plane normal to see if they are facing the same direction.

angle = acos(dotProduct(Va.normalize(), Vb.normalize()));
cross = crossProduct(Va, Vb);
if (dotProduct(Vn, cross) < 0) { // Or > 0
  angle = -angle;
}

Solution 2:

The solution I'm currently using seems to be missing here. Assuming that the plane normal is normalized (|Vn| == 1), the signed angle is simply:

For the right-handed rotation from Va to Vb:

atan2((Va x Vb) . Vn, Va . Vb)

For the left-handed rotation from Va to Vb:

atan2((Vb x Va) . Vn, Va . Vb)

which returns an angle in the range [-PI, +PI] (or whatever the available atan2 implementation returns).

. and x are the dot and cross product respectively.

No explicit branching and no division/vector length calculation is necessary.

Explanation for why this works: let alpha be the direct angle between the vectors (0° to 180°) and beta the angle we are looking for (0° to 360°) with beta == alpha or beta == 360° - alpha

Va . Vb == |Va| * |Vb| * cos(alpha)    (by definition) 
        == |Va| * |Vb| * cos(beta)     (cos(alpha) == cos(-alpha) == cos(360° - alpha)


Va x Vb == |Va| * |Vb| * sin(alpha) * n1  
    (by definition; n1 is a unit vector perpendicular to Va and Vb with 
     orientation matching the right-hand rule)

Therefore (again assuming Vn is normalized):
   n1 . Vn == 1 when beta < 180
   n1 . Vn == -1 when beta > 180

==>  (Va x Vb) . Vn == |Va| * |Vb| * sin(beta)

Finally

tan(beta) = sin(beta) / cos(beta) == ((Va x Vb) . Vn) / (Va . Vb)

Solution 3:

You can do this in two steps:

1. Determine the angle between the two vectors

   theta = acos(dot product of Va, Vb). Assuming Va, Vb are normalized. This will give the minimum angle between the two vectors

2. Determine the sign of the angle

   Find vector V3 = cross product of Va, Vb. (the order is important)

   If (dot product of V3, Vn) is negative, theta is negative. Otherwise, theta is positive.