Is the least squares solution to an overdetermined system a triangle center?

Solution 1:

As Rahul Narain commented, the precise position of the least-squares solution depends on the scaling of the individual equations: as you scale up an equation, so does the importance of the distance of your solution to the corresponding line, and the least-squares solution will end up closer to that line. You can say that in any case the least-squares solution will lie in the interior of the triangle formed by the three lines, since for a point in any of the other $6$ sectors into which the lines cut up the plane, moving in some direction will decrease the distances to all three lines at the same time, so one can never have a minimum there.

In the spacial case where you scale the equations in such a way that for each of the the sum-of-squares-of-coefficients gives the same value (for instance$~1$), then you can describe the least-square solution as the unique point$~P$ such that the sum of the three vectors from $P$ to its orthogonal projections on the three lines is zero; in other words $P$ is the barycenter of its own pedal triangle (the triangle formed by those projections). That this is so can easily be seen by taking the formula implicit in the comment by @bubba and differentiating it.

Giving a geometric construction of such a point is an interesting problem that probably has been considered by people, but I don't see an easy solution right now, nor do I recall having seen anything directly related. In particular, I am unsure if this point coincides with any one of the many "centers" associated to a triangle; however given that there are so many of them, and the description of the point we are interested is is quite natural, I would bet that it does equal one of those centers.

Indeed, thank you darij grinberg for pointing out that this is the Symmedian Point, the properties listed under the link correspond exactly to what we are after here.