Why not define infinite derivatives?

You would lose the sum, product, and quotient rules for derivatives. You would lose the chain rule. You would lose the fact that a derivative at a point implies continuity at that point. The intermediate value theorem would no longer apply to differentiable functions. You lose the Darboux property of derivatives. Say goodbye to Taylor. Our freshmen are going to love it!

It is a matter of convention and agreement between mathematicians.

For me, there is no problem if you say that the function is differentiable at some point $c$ even if $\lim_{h \to 0} \frac{f(c+h) - f(c)}{h}$ is equal to $+\infty$ or $- \infty$. This would only extend differentiability of some functions to some larger set, for example your example $x \mapsto x^{\frac 13}$ would with your definition be differentiable on $\mathbb R$ and not only on $\mathbb R \setminus \{0\}$. So I would not say as you say that derivatives of some functions in some points that are equal to $+\infty$ or $- \infty$ are not well-defined, they are well-defined, it is just that we usually take by definition that the derivative of some function at some point is a finite number.

Situation is, viewen in this way, similar to that of series.

You could define that the series $\sum_{i=1}^{\infty}a_i$ of real numbers is convergent if the limit $\lim_{n \to \infty}\sum_{i=1}^{n}a_i$ exists and is equal to some member of the set $\mathbb{R} \cup \{+\infty, -\infty\}$. With this definition, for example, the harmonic series $\sum_{i=1}^{\infty}\frac {1}{n}$ would be a convergent series.

The only "problem" that I see with these extended definitions of the derivative at some point and convergence of the series is that maybe we would have to, when proving some theorems, replace the assumption Suppose that $f$ is differentiable at some point $c$... with the assumption Suppose that $f$ is differentiable at some point $c$ and that the derivative at that point is not equal to $+ \infty$ or $- \infty$... (and similarly for the series(and integrals)).

So, I would say that there is nothing wrong with your extended definition.