Intuition for $N(\mu, \sigma^2)$ in terms of its infinite expansion

Solution 1:

Take an experiment that has two outcomes, success (S) and failure (F), of probability $p$ and $q=1-p$ respectively. The probability of S after one trial is $p$. The probability of two S after two trials is $p^2$, for one S and one F is $2pq$ and for two F is $q^2$. In general, for $N$ trials, the probability of having k S is given by the Binomial distribution $P_S(k,N)= \frac{N!}{k!(N-k)!}p^k q^{N-k}$. These are just the coefficients in front of the terms in $(p+q)^N$ after multiplying them out

What happens if we take the limit of large $N$? The Binomial coefficients at large $N$ approximate a Gaussian, which can be seen visually by looking at a low row Pascal's Triangle. We can derive it from the above formula. To make this simple let's work with the symmetric case $p=q=1/2$, so that we have $$ P_S(k,N)= \frac{N!}{k!(N-k)!}\frac{1}{2^N} $$ An easy way to get the desired result is to exchange the index $k$ for one that starts from the center of the triangle where values are largest $ x\in (-N/2,N/2),\; k=x+N/2$. Then we swap this in and apply Stirling's approximation $n!=\sqrt{2\pi}n^ne^{-n}$:

$$ P_S(x,N)= \frac{N!}{(N/2+x)!(N/2-x)!}\frac{1}{2^N} \approx \frac{2}{\sqrt{2\pi N}}\frac{1}{(1-\frac{4 x^2}{N^2})^{(N+1)/2}}\left(\frac{1-\frac{2x}{N}}{1+\frac{2x}{N}}\right)^x $$

Finally exponentiation and taking the log we get $$ P_S(x,N) \approx \frac{2}{\sqrt{2\pi N }} e^{-\frac{1}{2}(N+1)\log(1-\frac{4 x^2}{N^2}) +x(\log(1-\frac{2x}{N})-\log(1+\frac{2x}{N})) } $$

Using the expansion $\log(1+x)=x-x^2/2+ \cdots$, keep everything to order $1/N$ $$ P_S(x,N) \approx \frac{2}{\sqrt{2\pi N}} e^{-\frac{2 x^2}{N} } $$

Now you can feel free to tack a on $dx$ and use a transformation of variables to scale $x \rightarrow \sqrt N x/2$ -- then $x$ is an "implicit" variable.

$$ P_S(x,N)dx \approx \frac{1}{\sqrt{2\pi}} e^{-\frac{ x^2}{2}} dx $$

For general $p,q$, the derivation is similar, see for example http://scipp.ucsc.edu/~haber/ph116C/NormalApprox.pdf

This isn't exactly an infinite expansion like in your example, but there's a similar vein of thought in the conclusion that $N$ choose $k$ limits to a Gaussian type shape for large $N$.