Discontinuity of Dirac Delta distribution
Solution 1:
The text you quoted messes with different notions of differentiability and I can't make any sense of it.
Since the delta distribution is a linear functional on a space of test functions it is not a function on the real line and hence, it does not make sense to say something like "$\delta$ is differentiable on $(-\infty,\infty)$ except at $x=0$" since $\delta$ is not defined on that set.
When viewing $\delta:\mathcal{S}\to\mathbb{R}$ (linear and continuous with respect to the usual semi-norms on the Schwartz-space – or similar on the space of test functions), it makes sense to say that $\delta$ is continuous. To say that it is differentiable, one has to define the notion of differentiability for such objects (which is done by duality).
Solution 2:
Suppose you have $\delta_0\in D'(\Bbb R)$ - Dirac delta distribution. If you take a test function $\phi\in D(\Bbb R)$, then we write by definition $$\langle\delta_0,\phi\rangle:=\phi(0).$$
You can also use the Heaviside function $H(x)$, $\langle H,\phi\rangle:=\int_0^\infty\phi(x)dx$.
Then again, you define the derivative of any distribution $T\in D'(\Bbb R)$ by setting $$\langle T',\phi\rangle:=-\langle T,\phi'\rangle.$$
Then, it's easy to show that $H' =\delta_0$. Even further $\langle\delta_0',\phi\rangle=-\phi'(0)$.
To repeat, all distributions are differentiable (in the sense of distributions, of course) and not all distributions can be represented by $L^1_{loc}$ functions.
Does this answer your question?