Proof that the limit of the normal distribution for a standard deviation approximating 0 is the dirac delta function.

You have to be a bit careful about what this notation you are using actually means. Rigorously, the dirac delta is only defined when acting on functions in the Schwartz space. There is no rigorous justification for handling it as a function $x \mapsto \delta(x)$ for real $x$. I'll discuss the case where we are talking about convergence in the distributional sense. Let $\varphi \in C_{c}^{\infty}(\mathbb{R})$. I'll note $ f_{\epsilon} := \frac{1}{\sqrt{2\pi\epsilon}}e^{\frac{-x^2}{2\epsilon}}$. Using the substitution $y = x/\sqrt{\epsilon}$ we obtain $$\langle f_{\epsilon},\varphi \rangle = \int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\epsilon}}e^{\frac{-x^2}{2\epsilon}} \varphi(x) \mathrm{d}x =\int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}}e^{-y^2 /2} \varphi(\sqrt{\epsilon}y) \mathrm{d}y $$ As $\varphi$ is continous and compactly supported, this integrand is dominated by $\frac{1}{\sqrt{2\pi}}e^{-y^2 /2} \|\varphi\|_{\infty}$ which integrates to $\|\varphi\|_{\infty}$. Moreover, because $\varphi$ is continous the integrand converges pointwise to $\frac{1}{\sqrt{2\pi}}e^{-y^2 /2} \varphi(0)$ as $\epsilon \to 0$. Applying the dominated convergence theorem yields

$$\lim_{\epsilon\to 0} \langle f_{\epsilon},\varphi \rangle = \int_{-\infty}^{\infty} \lim_{\epsilon\to 0} \frac{1}{\sqrt{2\pi}}e^{-y^2 /2} \varphi(\sqrt{\epsilon}y) \mathrm{d}y = \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}}e^{-y^2 /2} \varphi(0) \mathrm{d}y = \varphi(0) $$ which finally shows $$\lim_{\epsilon\to 0} \langle f_{\epsilon},\varphi \rangle = \langle \delta_0, \varphi \rangle$$


You know that $e^x\ge 1+x$. Then $$ e^{-\frac{x^2}{2ϵ}}\le\frac1{1+\frac{x^2}{2ϵ}}=\frac{2ϵ}{2ϵ+x^2} $$ gives a sufficiently small upper bound for the considered point-wise limit.


In general you have for any positive function $\phi$ of integral $1$ that $\frac1ϵ\phi(\frac xϵ)$ converges to $δ$. This is part of the idea of approximations of unity ($δ$ is the unit element under convolution).