What is the purpose of Stirling's approximation to a factorial?

The purpose (as for all asymptotic expressions) is to replace a "complicated" function (in this case the factorial) with some expression which is "simpler". So you might object that $\sqrt{2\pi n} (n/e)^n$ is simpler than $n!$. But if I ask you the question whether $e^n$ or $n!$ grows faster when $n \to \infty$ you might appreciate Stirling's result. Or try to answer the question, how many digits $n!$ has when $n$ is large. Or ...

One result from Computer Science:

The minimum number of comparisons needed to sort any $n$ items using a comparison-sort is

$$\log_2(n!)$$

(this can be seen by taking every possible ordering, and forming binary tree of necessary comparisons of minimum height).

By Stirling's approximation, we have

$$\log_2(n!)$$ $$> \log_2\left(\sqrt{2\pi n}\left(\frac{n}{e}\right)^n\right)$$ $$= n \log_2\left(\frac{n}{e}\right) + \log_2\left(\sqrt{2\pi n}\right)$$

Or, as you will see written it in every Computer Science book ever,

The number of comparisons necessary for any comparison-sort is lower-bounded by $\Omega(n \log_2n)$

Also, a sidenote: OP stated that both $n!$ and $n^n$ require $O(n)$ multiplications to compute. This is false - $n^n$ can be computed quite easily in $O(log_2(n))$

Abraham de Moivre was the person who first introduced Stirling's formula. His friend James Stirling is the one who found that the constant is $\sqrt{2\pi}$; de Moivre only knew it numerically.

Demoivre used it to approximate the probability that the number of heads you get when you toss a coin 1800 times is $x$, for $x$ not too many standard deviations away from 900. He wrote about this in his book titled The Doctrine of Chances (google the title!). The title of the book is in effect 18th-century English for "the theory of probability". The phrase appears again in Thomas Bayes' famous posthumous paper "An essay towards solving a problem in the doctrine of chances" (google that title too). De Moivre derived the bell-shaped curve $$ x\mapsto \text{constant} \cdot e^{-x^2/2} $$ from this formula.

Stirling's approximation is used extensively in Physics, in Boltzmann's description of entropy:

$$ S = k \log_e W$$

Where W is the number of possible permutations of a group of materials, given by

$$W = \frac{N!}{\prod_i N_i!}$$

Taking the the approximation makes the maths much easier.