Retrieving the 3D Position of a Rectangle Using a Single 2D Image

The setup is as following:

• a rectangular object whose width and height are known is placed somewhere in 3D space
• an image is taken of the object producing a 2D projection of the rectangle onto an imaging plane

Given the coordinates of the four vertices of the rectangular object in the 2D image is it possible to determine the coordinates of the rectangular object in 3D space relative to the camera position?

I've found a paper claiming to do exactly that (semanticscholar, pdf). However, the authors' claim of retrieving a rectangular object in 3D space is not correct, because the only constraint they take into account is that $P_0P_1 = P_2P_3$ (i.e. that the opposite sides are of equal length) ensure that the object is a parallelogram but not necessarily a rectangle.

Since the system of equations they got is completely determined, is there a way to ensure that the resulting coordinates are indeed of a rectangular object?

If not, how would one begin to add error terms to the original coordinates without getting completely entangled in the expressions? Otherwise, is there another approach to take for the same problem?

The paper mentioned: Prematunga, H.G.L. and Dharmaratne, A.T. (2012). Finding 3D Positions from 2D Images Feasibility Analysis. In In Proc. of The Seventh International Conference on Systems (ICONS 2012).

[EDIT] I haven't properly emphasized that the output must necessarily be a rectangle, if that is not the case, the 2D points are misplaced. In that case, it would be useful to find a meaningful way to transform the coordinates of the 3D parallelogram into a rectangle with the least degree of error.

I've read the problem formulation in the paper, and from what I understand, you basically have four image points of the rectangle vertices, let's call them $q_1, q_2, q_3, q_4 $ ordered in (anti-) clockwise direction. From the camera model, the actual positions of the vertices is given by

$ p_1 = - t_1 q_1 , p_2 = -t_2 q_2 , p_3 = -t_3 q_3, p_4 = -t_4 q_4 $

where $t_1, t_2, t_3, t_4 \gt 0 $

The main equation here is that

$ p_1 p_2 = p_4 p_3 $

(remember the order of the points)

Hence,

$ -t_2 q_2 + t_1 q_1 = -t_3 q_3 + t_4 q_4 $

If we define the $3 \times 4$ matrix $Q$ as follows

$ Q = \begin{bmatrix} q_1 && - q_2 && q_3 && -q_4 \end{bmatrix} $

and the vector $T = \begin{bmatrix} t_1 \\ t_2 \\ t_3 \\ t_4 \end{bmatrix} $

Then

$Q T = 0 $

This is a linear system of $3$ equations in $4$ unknowns, and can be solved to yield

$T = \lambda V $

where $V$ is a constant vector resulting from solving the above linear system, and $\lambda$ is an arbitrary real scalar. From the nature of the problem we want to take the entries of $V$ to be all positive and $\lambda \gt 0$ as well.

Finally, we have

$ |p_1 p_2 |^2 = a^2 $

where $a$ is the known length of the rectangle. Therefore,

$ \| -t_2 q_2 + t_1 q_1 \|^2 = a^2 $

and this simplifies to,

$ \| - \lambda V_2 q_2 + \lambda V_1 q_1 \|^2 = a^2$

which simplifies further to,

$ \lambda^2 \| - V_2 q_2 + V_1 q_1 \|^2 = a^2 $

which directly results in the positive value of $\lambda$. Once $\lambda$ is known, all the $t$'s are known and thus all the vertices coordinates become known.

At the final stage, after having computed all the coordinates we can check that

$ p_1 p_2 \perp p_2 p_3 $

Note that this condition is equivalent to

$ \lambda^2 ( V_2 q_2 - V1 q_1 ) \cdot ( V_3 q_3 - V2 q_2 ) = 0 $

Therefore, and since $\lambda \ne 0$, if the imaged quadrilateral is indeed a rectangle we must have

$ ( V_2 q_2 - V_1 q_1 ) \cdot ( V_3 q_3 - V_2 q_2 ) = 0 $

If this condition is satisfied then our quadrilateral is indeed a rectangle, otherwise, it is a parallelogram.

One could also check that

$ \| p_2 p_3 \| = b $

where $b$ is the other dimension of the rectangle.