Proof of the second symmetric derivative
Solution 1:
One way would be to observe that
$f''(a)=\lim_{h\to0}\frac{f'(a+h)-f'(a-h)}{2h}$
and so the result follows by a simple application of L'Hospital'sRule on
$\lim_{h\to0}\frac{f(a+h)+f(a-h)-2f(a)}{h^{2}}$
Remark:
The first item is proved by noting that
$f''(a)=\lim_{h\to0}\frac{f'(a+h)-f'(a)}{h}$
but also
$f''(a)=\lim_{h\to0}\frac{f'(a-h)-f'(a)}{-h}$
so adding these we get the first formula.
Solution 2:
Taylor's theorem shows that there exists some $\eta$ such that $f(a+h) = f(a) + f'(a)h + {1 \over 2} f''(a) h^2 + \eta(h)$, where $\lim_{h \to 0} {\eta(h) \over h^2} = 0$.
Then ${1 \over h^2} (f(a+h)+f(a-h)-2f(a)) = {1 \over 2} (f''(a)+f''(a)+{\eta(h) \over h^2} + {\eta(-h) \over h^2} )$ from which the result follows.
Aside: Note that with $f(x) = x |x|$, we see that the limit $\lim_{h \to 0} {f(h)+f(-h)-2f(0) \over h^2} = 0$ but $f$ is not twice differentiable at $h=0$.