Interchanging the order of differentiation and summation

Can the order of a differentiation and summation be interchanged, and if so, what is the basis of the justification for this?

E.g. is $\frac{\mathrm{d}}{\mathrm{d}x}\sum_{n=1}^{\infty}f_n(x)$ equal to $\sum_{n=1}^{\infty}\frac{\mathrm{d}}{\mathrm{d}x}f_n(x)$ and how can it be proven?

My intuition for this is that it should be the case, since in the limit, the summation becomes an integral and this can be interchanged with the differentiation operator, but I don't know how to justify this? Would it suffice to say that $\frac{\mathrm{d}}{\mathrm{d}x}f_1(x)+\frac{\mathrm{d}}{\mathrm{d}x}f_2(x) = \frac{\mathrm{d}}{\mathrm{d}x}(f_1(x)+f_2(x)$ because it is a linear operator and then somehow extend this to an infinite sum?

Thanks in advance!

If $\sum f_n'$ converges uniformly, then yes. This is a standard theorem proved in texts like Rudin's Principles of mathematical analysis (see 7.17, 3rd Ed for details).

More generally, one of the following 3 things can happen:

1. The series is not differentiable.
2. The series of derivatives does not converge.
3. The series of derivatives converges to something other than the derivative of the series.

Every continuous function on $[0,1]$ is a uniform limit of polynomials, but there are continuous functions whose derivative exist nowhere. If the limits are interpreted as series as in Johannes's post, this gives examples of 1. Johannes gives an example of 2. Example 7 on page 80 of Counterexamples in analysis covers case 3.

This is under the assumption that you are taking a sequence of functions, i.e., your limit here is over a sequence $f_n$ of functions with a single variable $x$.

This is not always possible. While differentiation is linear, this does not extend to infinite sums.

This question is related to the question if limits and differentiation commute, which is not the case: Consider $f_n(x) = (\sin n x)/n$. Then $f_n \to 0$ for $n \to \infty$ (it actually converges uniformly), but $\lim_{n \to \infty} \frac{d f_n(x)}{d x} = \lim_{n \to \infty} \cos nx$ does not converge at all (example taken from Forster)

Now, by setting $g_1(x) = f_1(x)$, $g_n(x) = f_n(x) - f_{n-1}(x)$ for $n > 1$, the same counter-example works for infinite series, summing over the $g_n$.

Interchanging summation and differentiation is possible if the derivatives of the summands uniformly converge to 0, and the original sum converges. This follows from the equivalent criterion for interchanging limits and differentials.