Given four points, determine a condition on a fifth point such that the conic containing all of them is an ellipse

The image of the question if you don't see all the symbols

The given points $p_1,p_2,p_3,p_4$ are located at the vertices of a convex quadrilateral on the real affine plane.

I am looking for an explicit condition on the point $p_5$ necessary and sufficient for the conic which is determined by $p_1,p_2,p_3,p_4,p_5$ to be an ellipse.

Could you give me a hint, please?

I tried to "go" to $\mathbb P_2$ and change the coordinates of these points to more convenient (for example, if $a=(1:a_1,a_2)$ change it to $(1:1:0)$) but I am not sure that this transformation preserves conic


One can always construct two parabolas passing through $p_1$, $p_2$, $p_3$, $p_4$ (green and pink in the figure below), each one possibly degenerating into a couple of parallel lines if two opposite sides of quadrilateral $p_1p_2p_3p_4$ are parallel. Point $p_5$ will determine an ellipse if it lies inside either parabola but not in their intersection.

This follows from the fact that five points always determine a conic section, and because the parabola is a limiting case between ellipse and hyperbola: each time $p_5$ crosses the boundary of a parabola, conic section $p_1p_2p_3p_4p_5$ switches from ellipse to hyperbola (or viceversa).

enter image description here


I'll expand-upon my hint, because the raw calculations get pretty ugly, yet there is a reasonably-tame way to express the desired condition.


The equation of a conic through $P$, $Q$, $R$, $S$, $T$ is given by:

$$\left|\begin{array}{cccccc} x^2 & y^2 & x y & x & y & 1 \\ P_x^2 & P_y^2 & P_x P_y & P_x & P_y & 1 \\ Q_x^2 & Q_y^2 & Q_x Q_y & Q_x & Q_y & 1 \\ R_x^2 & R_y^2 & R_x R_y & R_x & R_y & 1 \\ S_x^2 & S_y^2 & S_x S_y & S_x & S_y & 1 \\ T_x^2 & T_y^2 & T_x T_y & T_x & T_y & 1 \\ \end{array} \right| = 0 \tag{1}$$

Expanding the determinant yields an equation of the form $$A x^2 + B x y + C y^2 + D x + E y + F = 0 \tag{2}$$ This represents an ellipse when $$B^2-4AC < 0 \tag{3}$$ (likewise, a hyperbola when $>0$, and a parabola when $=0$).

Condition $(3)$, in $xy$-coordinates, turns out to be an expression in over $14,000$ terms. We can collapse the complexity a bit by using a self-induced coordinate system; specifically, we'll use barycentric coordinates based on $\triangle PQR$ (which we'll assume is non-degenerate).

We can give $S$ and $T$ respective coordinates $(s_P:s_Q:s_R)$ and $(t_P:t_Q:t_R)$. That is, we can write $$S = \frac{s_P P + s_Q Q + s_R R}{s_P+s_Q+s_R} \qquad\qquad T = \frac{t_P P + t_Q Q + t_R R}{t_P + t_Q+t_R} \tag{4}$$ Substitution into $(3)$ collapses the relation into a mere $21$ terms (and a discardable factor corresponding to the area of $\triangle PQR$). This is better, but still a little messy. It cleans up nicely when reciprocating the elements, however; defining $x' := 1/x$, we can write

$$\begin{align} &\phantom{4}\left( s^{\prime}_P t^{\prime}_Q + s^{\prime}_Q t^{\prime}_R + s^{\prime}_R t^{\prime}_P + s^{\prime}_Q t^{\prime}_P + s^{\prime}_R t^{\prime}_Q + s^{\prime}_P t^{\prime}_R \right)^2 \\[4pt] <\; &4 \left(s^{\prime}_P s^{\prime}_Q + s^{\prime}_Q s^{\prime}_R + s^{\prime}_R s^{\prime}_P \right) \left(t^{\prime}_P t^{\prime}_Q + t^{\prime}_Q t^{\prime}_R + t^{\prime}_R t^{\prime}_P \right) \\ \left(\quad =\; \right. & \frac{4}{s_P s_Q s_R\,t_P t_Q t_R}\left.\left(s_P+s_Q+s_R\right)\left(t_P+t_Q+t_R\right)\quad\right) \end{align} \tag{$\star$}$$

If we know $P$, $Q$, $R$, $S$ (and therefore $s_P$, $s_Q$, $s_R$ and $s^\prime_P$, $s^\prime_Q$, $s^\prime_R$), then $(\star)$ gives a condition on parameters defining $T$.

(There's probably a nice projective interpretation of $(\star)$ that could've saved all the trouble of deriving it. :)