Relation between Heaviside step function to Dirac Delta function
Actually, the non rigorous definition of $\delta$ stipulates that $\int\limits_a^b\delta(x)\mathrm dx$ is $1$ if $a\lt0\lt b$ (what you recalled in your post) but also that it is $0$ if $a\leqslant b\lt0$ or if $0\lt a\leqslant b$.
Hence $H(x)=\int\limits_{-\infty}^x\delta(\xi)\mathrm d\xi$ should be $0$ if $x\lt0$ and $1$ if $x\gt0$.
Two ways to answer:
The distributional derivative of $H(x)$ is $\delta$. So if you want to mimick formally the second fundamental theorem of calculus, that formula is what you get.
$\delta(\xi) d \xi$ can also be viewed as a probability measure with support $\{0\}$. The cumulative distribution function is precisely $H$. Your formula is just the definition of cumulative distribution function.