get closest point to a line

Here's Ruby disguised as Pseudo-Code, assuming Point objects each have a x and y field.

def GetClosestPoint(A, B, P)

  a_to_p = [P.x - A.x, P.y - A.y]     # Storing vector A->P
  a_to_b = [B.x - A.x, B.y - A.y]     # Storing vector A->B

  atb2 = a_to_b[0]**2 + a_to_b[1]**2  # **2 means "squared"
                                      #   Basically finding the squared magnitude
                                      #   of a_to_b

  atp_dot_atb = a_to_p[0]*a_to_b[0] + a_to_p[1]*a_to_b[1]
                                      # The dot product of a_to_p and a_to_b

  t = atp_dot_atb / atb2              # The normalized "distance" from a to
                                      #   your closest point

  return Point.new( :x => A.x + a_to_b[0]*t,
                    :y => A.y + a_to_b[1]*t )
                                      # Add the distance to A, moving
                                      #   towards B

end

Alternatively:

From Line-Line Intersection, at Wikipedia. First, find Q, which is a second point that is to be had from taking a step from P in the "right direction". This gives us four points.

def getClosestPointFromLine(A, B, P)

  a_to_b = [B.x - A.x, B.y - A.y]   # Finding the vector from A to B
                                        This step can be combined with the next
  perpendicular = [ -a_to_b[1], a_to_b[0] ]
                                    # The vector perpendicular to a_to_b;
                                        This step can also be combined with the next

  Q = Point.new(:x => P.x + perpendicular[0], :y => P.y + perpendicular[1])
                                    # Finding Q, the point "in the right direction"
                                    # If you want a mess, you can also combine this
                                    # with the next step.

  return Point.new (:x => ((A.x*B.y - A.y*B.x)*(P.x - Q.x) - (A.x-B.x)*(P.x*Q.y - P.y*Q.x)) / ((A.x - B.x)*(P.y-Q.y) - (A.y - B.y)*(P.y-Q.y)),
                    :y => ((A.x*B.y - A.y*B.x)*(P.y - Q.y) - (A.y-B.y)*(P.x*Q.y - P.y*Q.x)) / ((A.x - B.x)*(P.y-Q.y) - (A.y - B.y)*(P.y-Q.y)) )

end

Caching, Skipping steps, etc. is possible, for performance reasons.

if anyone is interested in a C# XNA function based on the above:

    public static Vector2 GetClosestPointOnLineSegment(Vector2 A, Vector2 B, Vector2 P)
    {
        Vector2 AP = P - A;       //Vector from A to P   
        Vector2 AB = B - A;       //Vector from A to B  

        float magnitudeAB = AB.LengthSquared();     //Magnitude of AB vector (it's length squared)     
        float ABAPproduct = Vector2.Dot(AP, AB);    //The DOT product of a_to_p and a_to_b     
        float distance = ABAPproduct / magnitudeAB; //The normalized "distance" from a to your closest point  

        if (distance < 0)     //Check if P projection is over vectorAB     
        {
            return A;

        }
        else if (distance > 1)             {
            return B;
        }
        else
        {
            return A + AB * distance;
        }
    }

Your point (X) will be a linear combination of points A and B:

X = k A + (1-k) B

For X to be actually on the line segment, the parameter k must be between 0 and 1, inclusive. You can compute k as follows:

k_raw = (P-B).(A-B)  /  (A-B).(A-B)

(where the period denotes the dot product)

Then, to make sure the point is actually on the line segment:

if k_raw < 0:
    k= 0
elif k_raw > 1:
    k= 1
else:
    k= k_raw

The answer from Justin L. is almost fine, but it doesn't check if the normalized distance is less than 0, or higher than the AB vector magnitude. Then it won't work well when P vector proyection is out of bounds (from the line segment AB). Here's the corrected pseudocode:

    function GetClosestPoint(A, B, P)
{
  vectorAP = (p.x - a.x, p.y - a.y)     //Vector from A to P
  vectorAB = (b.x - a.x, b.y - a.y)     //Vector from A to B

  magnitudeAB = vectorAB[0]^2 + vectorAB[1]^2  
  //Magnitude of AB vector (it's length)


  ABAPproduct = vectorAB[0]*vectorAP[0] + vectorAB[1]*vectorAP[1] 
  //The product of a_to_p and a_to_b


  distance = ABAPproduct / magnitudeAB       
  //The normalized "distance" from a to your closest point

  if ( distance < 0)     //Check if P projection is over vectorAB
    {
        returnPoint.x = a.x
        returnPoint.y = a.y
    }   
  else if (distance > magnitudeAB)
    {
        returnPoint.x = b.x
        returnPoint.y = b.y
    }
  else
    {
        returnPoint.x = a.x + vectorAB[0]*distance
        returnPoint.y = a.y + vectorAB[1]*distance
    }

}

I wrote this a long time ago, it's not much different to what others have said, but it's a copy/paste solution in C# if you have a class (or struct) named PointF with members X and Y:

private static PointF ClosestPointToSegment(PointF P, PointF A, PointF B)
{
    PointF a_to_p = new PointF(), a_to_b = new PointF();
    a_to_p.X = P.X - A.X;
    a_to_p.Y = P.Y - A.Y; //     # Storing vector A->P  
    a_to_b.X = B.X - A.X;
    a_to_b.Y = B.Y - A.Y; //     # Storing vector A->B

    float atb2 = a_to_b.X * a_to_b.X + a_to_b.Y * a_to_b.Y;
    float atp_dot_atb = a_to_p.X * a_to_b.X + a_to_p.Y * a_to_b.Y; // The dot product of a_to_p and a_to_b
    float t = atp_dot_atb / atb2;  //  # The normalized "distance" from a to the closest point
    return new PointF(A.X + a_to_b.X * t, A.Y + a_to_b.Y * t);
}

Update: Looking at the comments it looks like I adapted it to C# from the same source code mentioned in the accepted answer.