What do $\pi$ and $e$ stand for in the normal distribution formula?

I'm a beginner in mathematics and there is one thing that I've been wondering about recently. The formula for the normal distribution is:

$$f(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-\displaystyle{\frac{(x-\mu)^2}{2\sigma^2}}},$$

However, what are $e$ and $\pi$ doing there? $\pi$ is about circles and the ratio to its diameter, for example. $e$ is mostly about exponential functions, specifically about the fact that $\frac{\mathrm{d}}{\mathrm{d}x} e^x = e^x$.

It is my firm conviction that proofs and articles are available, but could someone perhaps shed some light on this and please explain in a more 'informal' language what they stand for here?

I'm very curious to know as those numbers have very different meanings as far as I'm concerned.

So I think you want to know "why" $\pi$ and $e$ appear here based on an explanation that goes back to circles and natural logarithms, which are the usual contexts in which one first sees these.

If you see $\pi$, you think there's a circle hidden somewhere. And in fact there is. As has been pointed out, in order for this expression to give a probability density you need $\int_{-\infty}^\infty f(x) \: dx = 1$. (I'm not sure how much you know about integrals -- this just means that the area between the graph of $f(x)$ and the $x$-axis is 1.) But it turns out that this can be derived from $\int_{-\infty}^\infty e^{-x^2} dx = \sqrt{\pi}$.

And it turns out that this is true because the square of this integral is $\pi$. Now, why should the square of this integral have anything to do with circles? Because it's the total volume between the graph of $e^{-(x^2+y^2)}$ (as a function $g(x,y)$ of two variables) and the $xy$-plane. And of course $x^2+y^2$ is just the square of the distance of $(x,y)$ from the origin -- so the volume I just mentioned is rotationally symmetric. (If you know about multiple integration, see the Wikipedia article "Gaussian integral", under the heading "brief proof" to see this volume worked out.)

As for where $e$ comes from -- perhaps you've seen that the normal probability density can be used to approximate the binomial distribution. In particular, the probability that if we flip $n$ independent coins, each of which has probability $p$ of coming up heads, we'll get $k$ heads is $$ {n \choose k} p^{k} (1-p)^{n-k} $$ where ${n \choose k} = n!/(k! (n-k)!)$. And then there's Stirling's approximation, $$ n! \approx \sqrt{2\pi n} (n/e)^{n}. $$ So if you can see why $e$ appears here, you see why it appears in the normal. Now, we can take logs of both sides of $n! = 1 \cdot 2 \cdot \ldots \cdot n$ to get $$ \log (n!) = \log 1 + \log 2 + \cdots + \log n $$ and we can approximate the sum by an integral, $$ \log (n!) \approx \int_{1}^{n} \log t \: dt. $$ But the indefinite integral here is $t \log t - t$, and so we get the definite integral $$ \log (n!) \approx n \log n - n. $$ Exponentiating both sides gives $n! \approx (n/e)^n$. This is off by a factor of $\sqrt{2\pi n}$ but at least explains the appearance of $e$ -- because there are logarithms in the derivation. This often occurs when we deal with probabilities involving lots of events because we have to find products of many terms; we have a well-developed theory for sums of very large numbers of terms (basically, integration) which we can plug into by taking logs.

One of the important operations in (continuous) probability is the integral. $e$ shows up there just because it's convenient. If you rearrange it a little you get $$ {1 \over \sqrt{2\pi \sigma^2}} (e^{1 \over 2\sigma^2})^{-(x-\mu)^2},$$ which makes it clear that the $e$ is just a convenient number that makes the initial constant relatively straightforward; using some other number in place of $e$ just rescales $\sigma$ in some way.

The $\pi$ is a little tougher to explain; the fact you just have to "know" (because it requires multivariate calculus to prove) is that $\int_{-\infty}^{\infty} e^{-x^2} dx = \sqrt{\pi}$. This is called the Gaussian integral, because Gauss came up with it. It's also why this distribution (with $\mu = 0, \sigma^2 = 1/2$) is called the Gaussian distribution. So that's why $\pi$ shows up in the constant, so that no matter what values you use for $\sigma$ and $\mu$, $\int_{-\infty}^{\infty} f(x) dx = 1$.