Line and plane intersection in 3D

Your intuition of setting the two equations equal is correct and that is how you solve for the intersection. I'll provide a full explanation, with code examples.

A common way of representing a plane $\mathbf{P}$ is in point-normal form, $\mathbf{\vec{n}} \cdot (X-Y)=0$, where $\mathbf{\vec{n}}$ is the plane normal and both $X$ and $Y$ are points that lie in the plane. This can be rewritten into constant-normal form by distributing the dot product and rearranging terms to obtain: $\mathbf{\vec{n}} \cdot X = d$, where $d = \mathbf{\vec{n}} \cdot Y$ which is equal to the distance from the origin when $\mathbf{\vec{n}}$ is unit-length. Below is a simple data structure that you might use to represent a plane, and the signature of a constructor that will compute the plane from three points in $\mathbb{R^3}$. Implementation is left as an exercise to the reader ;).

struct Plane {
    Vector3 n; // normal
    float d; // distance from origin

    Plane(); // default constructor
    Plane(Vector3 a, Vector3 b, Vector3 c); // plane from 3 points
    Vector3 intersectLine(Vector3 a, Vector3 b); // we'll get to this later
};

Given two points, $A$ and $B$, a line can be represented parametrically by adding to one point the vector formed by the two points, scaled by a parameter $t$. In symbols, $L(t) = A + t(B-A)$. Using your intuition, we insert this equation (whose output is a point), into $X$ in the constant-normal plane representation: $\mathbf{\vec{n}} \cdot [A + t(B-A)] = d$. We want to know how many copies of $(B-A)$ we need to add to $A$ to get to a point that lies within the plane, in other words we want to solve for $t$. Doing some fancy algebra, we obtain: $t = \frac{d-\mathbf{\vec{n}} \cdot A}{\mathbf{\vec{n}} \cdot (B-A)}$. We can (finally) stick this expression for $t$ back into the equation for our line to obtain: $I = A+\frac{d - (\mathbf{\vec{n}} \cdot A)}{\mathbf{\vec{n}} \cdot (B-A)}(B-A).$

Armed with this equation, we can now implement a nice function that will tell what we want to know:

Vector3 Plane::intersectLine(Vector3 a, Vector3 b) {
    Vector3 ba = b-a;
    float nDotA = Vector3::dotProduct(n, a);
    float nDotBA = Vector3::dotProduct(n, ba);

    return a + (((d - nDotA)/nDotBA) * ba);
}

Hopefully this works for you, and hopefully I didn't fudge any of the details! If you plan to be doing a lot of this sort of geometric computing it's worthwhile to pick up Christer Ericson's Real-time Collision Detection, which is an excellent reference source for this sort of thing. Alternatively, you could snag some already-constructed classes from something like OGRE3D, if you're not particularly interested in creating your own.

For these, and many similar requests, go to

http://www.graphicsgems.org/

Also, look up its descendant, The Journal of Graphics Tools at

http://jgt.akpeters.com/