Efficient maths algorithm to calculate intersections

Most of the answers already here seem to follow the general idea that:

1. find the intersection of two straight lines passing the given points.
2. determine if the intersection belong to both line segments.

But when intersection does not occur often, a better way probably is to reverse these steps:

1. express the straight lines in the form of y = ax + b (line passing A,B) and y = cx + d (line passing C,D)
2. see if C and D are on the same side of y = ax+b
3. see if A and B are on the same side of y = cx+d
4. if the answer to the above are both no, then there is an intersection. otherwise there is no intersection.
5. find the intersection if there is one.

Note: to do step 2, just check if (C.y - a(C.x) - b) and (D.y - a(D.x) - b) have the same sign. Step 3 is similar. Step 5 is just standard math from the two equations.

Furthermore, if you need to compare each line segment with (n-1) other line segments, precomputing step 1 for all lines saves you time.

If you get the chance you should really check out the Collision Detection bible, "Real Time Collision Detection" if you plan on doing anything non-trivial. I'm not a professional game programmer and I understood and could apply the concepts in it with little trouble.

Amazon - Real Time Collision Detection

Basically, doing a set of line intersection tests is expensive no matter what. What you do is use things like Bounding Boxes (axis aligned or oriented) over your complex polygons. This will allow you to quickly do a worst case O(N^2) check of collision between each "object". You can then speed things up even further by using spatial trees (Binary Partitioning or QuadTrees) by only checking intersections of objects close to eachother.

This allows you to prune many, many collision tests. The best optimization is not doing something at all. Only once you have a collision between bounding boxes do you do your expensive line intersections to determine if the objects truly do intersect or not. This allows you to scale the number of objects up while still keeping the speed reasonable.