Transfinite derivatives

In Ralph P. Boas's A primer of real functions, page 118, this is discussed in the following way: The derivative of infinite order of $f$ is defined on an interval $I$ iff the sequence $(f^{(n)})$ converges uniformly on $I$ (it is enough to require uniform convergence on compact subsets of $I$). Call $L$ the limit of this sequence, so $L$ is continuous and therefore Riemann integrable over bounded intervals. Fix $a\in I$, and for $x\in I$ consider $\lim_n\int_a^x f^{(n)}$. Note that, on the one hand, this is $$\lim_n (f^{(n-1)}(x)-f^{(n-1)}(a))= L(x)-L(a),$$ and on the other it is $\int_a^x L$, that is $L(x)=ce^x$ for some constant $c$, which ends up trivializing the notion. The reference given there is Lee Lorch, Derivatives of infinite order, Pacific Journal of Mathematics, 3, (1953), 773-778, MR0060553 (15,689c).

We used uniform convergence on compact subsets twice. First, to ensure that $L$ is integrable, though we in fact get more, as $L$ ends up being $C^\infty$. And second, to ensure that the limit of the integrals is the integral of the limit. We can avoid referring to integration by instead using the following well-known fact:

If $(f_n)_n$ is a sequence of differentiable functions on an interval $I$, $a\in I$, $(f_n(a))_n$ converges, and $(f_n')_n$ converges uniformly, then $(f_n)$ converges uniformly to a differentiable function $f$ such that $f_n'\to f'$.

From this it follows that if $(f^{(n)})_n$ converges uniformly, then the limit $L$ is $C^\infty$ and satisfies $L=L'$.

Lorch's paper provides additional references discussing various notions of derivatives of infinite order (for $C^\infty$ and specially for analytic functions), and discusses a few variants as well. The paper and some of the references it lists (particularly the Boas-Chandrasekharan papers), discuss some of the issues with the different suggestions, and the kind of results one can expect in each case.

As Qiaochu and David pointed out, the limit $d^n f(x)/dx^n$ doesn't make much sense. However, you may want to consider formal power series $a(f):=\Sigma_{n=0}^\infty a_nt^n$ over indeterminate variable $t$, with coefficients $a_n=\frac{1}{n!}\frac{d^n f(x)}{dx^n}$.

Such formal series, with the formal product rule $(ab)_n=\Sigma_{k=0}^n a_kb_{n-k}$ would encapsulate all the derivatives. You can view it this infinite formal power series as a "transfinite limit" for the finite derivatives.

No doubt you recognized Taylor series, but the point here is not to produce series that converge to the function given a specific value of $t$, the point is to produce an object that would encapsulate, in some way, the limit of $n^{th}$ derivative for $n\to\infty$. And the appropriate object for that is not a function that is the "infinite derivative", the appropriate object is an expression that encapsulates infinitely many derivatives.

In fact, something like that appears in often enough in Algebraic Geometry: if the object itself (such as $n^{th}$ derivative in this case) does not generalize directly into the desired domain (in this case $n=\infty$) then replace the object with a functor from a more general category (in this case the category is the formal power series, the functor is the $n^{th}$ term of the series times $n!$, and the intuition that guides this categorification is Taylor series), and then proceed working within the more general category.