Why can't we interchange differentiation with taking a limit of a series of functions?

While learning a little Fourier analysis, I ran into this interesting phenomenon:

Consider a series of sawtooth waves such that the height and width of the sawteeth shrinks to zero, but the slope of the sawteeth remains the same. To be specific, let

$$f_n(x) = \frac{nx - \lfloor nx\rfloor}{n}$$

Then define

$$F(x) = \lim_{n\to\infty}f_n(x)$$

It seems intuitively clear that $F(x) = 0$ for all $x$ because the global maximum of $f_n$ is $\frac{1}{n}$.

If $F(x) = 0$, then we should have $F'(x) = 0$ as well. However, if we choose an irrational value of $x$, then $f'_n(x) = 1$ for all $n$, so if $F'(x)$ is found instead by taking

$$F'(x) = \lim_{n\to\infty}f'_n(x)$$

we do not get $F'(x) = 0$.

It seems like the derivative of a limit is not the same as the limit of a derivative, which is pretty counterintuitive to me.

What's going on?

You say that you expect to be able to interchange limit and derivative operations. Now, derivative itself includes a limit operation, so I wonder whether you expect to be able to interchange any two different limit operations. If so, this discussion by Tim Gowers might be helpful.