Find shortest distance between lines in 3D

So you have two lines defined by the points $\mathbf{r}_1=(2,6,-9)$ and $\mathbf{r}_2=(-1,-2,3)$ and the (non unit) direction vectors $\mathbf{e}_1=(3,4,-4)$ and $\mathbf{e}_2 =(2,-6,1)$.

The coordinates of all the points along the lines are given by

$$\begin{align} \mathbf{p}_1 & = \mathbf{r}_1 + t_1 \mathbf{e}_1 \\ \mathbf{p}_2 & = \mathbf{r}_2 + t_2 \mathbf{e}_2 \\ \end{align}$$

where $t_1$ and $t_2$ are two scalar values. To find the closest points along the lines you recognize that the line connecting the closest points has direction vector $$\mathbf{n} = \mathbf{e}_1 \times \mathbf{e}_2 = (-20,-11,-26)$$

If the two direction vectors $\mathbf{e}_1$ and $\mathbf{e}_2$ are parallel (not in this specific case), this method cannot be applied because the cross product is zero:

$$\mathbf{e}_1 \times \mathbf{e}_2 = 0$$

If the points along the two lines are projected onto the cross line the distance is found with one fell swoop

$$ d = \frac{ \mathbf{n}\cdot \mathbf{p}_1}{\|\mathbf{n}\|} - \frac{ \mathbf{n}\cdot \mathbf{p}_2}{\|\mathbf{n}\|} = \frac{ \mathbf{n} \cdot ( \mathbf{p}_1-\mathbf{p}_2)}{\| \mathbf{n} \|} = \frac{ \mathbf{n} \cdot ( \mathbf{r}_1-\mathbf{r}_2+t_1 \mathbf{e}_1 -t_2 \mathbf{e}_2)}{\| \mathbf{n} \|} $$

But since $\mathbf{n}\cdot \mathbf{e}_1 = \mathbf{n}\cdot \mathbf{e}_2 = 0$, the above is

$$ \boxed{ d = \frac{ \mathbf{n} \cdot ( \mathbf{r}_1-\mathbf{r}_2)}{\| \mathbf{n} \|} }$$

In this case $$ d = \frac{ (-20,-11,-26) \cdot (3,8,-12) }{3 \sqrt{133}} = 4.74020116673185 $$

Here is an approach that uses dot products instead of cross products.

This works in any number of dimensions, not just 3.

The skew lines are $L = a+bt, M=c+ds$.

The distance between two points on $L$ and $M$ is $D =(a+bt-c-ds)^2 =(e+bt-ds)^2 $ where $e = a-c$.

For this to be a minimum, taking partials, we want $D_s = D_t = 0$.

$D_s = -2d(e+bt-ds) $ and $D_t = 2b(e+bt-ds) $.

Therefore, with multiplication of vectors being dot product, $0 =d(e+bt-ds) =de+dbt-d^2s $ and $0 =b(e+bt-ds) =be+b^2t-bds) $.

These are two equations in the two unknowns $s$ and $t$:

$\begin{array}\\ de &= d^2s-dbt\\ be &= bds-b^2t\\ \end{array} $

The determinant is $A =-b^2d^2+(bd)^2 =-(b^2d^2-(bd)^2) $. By Cauchy-Schwarz, this is non-zero unless $b$ and $d$ are parallel (which is a good thing).

The solutions (by Cramer's rule) are $s =\dfrac{-(b^2)(de)+(be)(db)}{A} $ and $t =\dfrac{(d^2)(be)-(be)(db)}{A} $.

Putting these into $L = a+bt, M = c+ds, D =(e+bt-ds)^2 $ we get the endpoints of the closest line and the distance.