Proving that $F_{\delta}(x)=\frac{1}{2\delta}\int_{-\delta}^{\delta} f(x+t) dt$ is continuously differentiable for continuous $f(x)$
Solution 1:
OP explicitly stated in the comments that they're not allowed to use the fundamental theorem of calculus.
So going back to the definition of the derivative: $$\begin{split} 2\delta \cdot\frac{F_\delta(x+h)-F_\delta(x)}{h} &= \frac 1 h\int_{-\delta}^{\delta}f(x+t+h)dt-\frac 1 h\int_{-\delta}^{\delta}f(x+t)dt\\ &= \frac 1 h\int_{-\delta+x+h}^{\delta+x+h}f(u)du-\frac 1 h\int_{-\delta+x}^{\delta+x}f(v)dv&\,\,&\text{ (1)}\\ &= \frac 1 h\int_{\delta+x}^{\delta+x+h}f(t)dt-\frac 1 h\int_{-\delta+x}^{-\delta+x+h}f(t)dt&\,\,&\text{ (2)}\\ &= f(\delta+x)+\frac 1 h\int_{\delta+x}^{\delta+x+h}(f(t)-f(\delta+x))dt\\ &-f(-\delta+x)-\frac 1 h\int_{-\delta+x}^{-\delta+x+h}(f(t)-f(-\delta+x))dt&\,\,&\text{ (3)}\\ \end{split}$$ where $(1)$ is obtained by changing the variable in the integrals and $(2)$ by rearranging the integral domains. Also, $(3)$ stems from the fact that
$$f(\delta+x)=\frac 1 h \int_{\delta+x}^{\delta+x+h}f(\delta+x)dt\text{ and }f(-\delta+x)=\frac 1 h \int_{-\delta+x}^{-\delta+x+h}f(-\delta+x)dt$$
Now, because $f$ is continuous, for any arbitrarily small $\varepsilon>0$, there exists $H_1>0$ such that for all $|h|<H_1$ and all $t \in[\delta+x, \delta+x+h]$, we have $$|f(\delta+x)-f(t)|<\varepsilon$$ Likewise, there exists $H_2>0$ such that for all $|h|<H_2$ and all $t \in[-\delta+x, -\delta+x+h]$, we have $$|f(-\delta+x)-f(x)|<\varepsilon$$ Thus, for all $|h|<\min(H_1, H_2)$, the two integrals in equation $(3)$ are both less than $\varepsilon$, and you can conclude that $$\lim_{h\rightarrow 0}\frac{F_\delta(x+h)-F_\delta(x)}{h} = \frac{f(\delta+x)-f(-\delta+x)}{2\delta}$$