Spivak Chapter 14 Problem 9

Let $f(x)=\cos \frac{1}{x}$ for $x \ne 0$, and $f(x)=0$ for $x=0$. Is the function $F(x)=\int_0^x f$ differentiable at 0?

My progress:

$$\lim_{h\to0^+} \frac{F(h)}{h} = \lim_{h\to0^+} \frac{\int_0^h f}{h}=\lim_{h\to0^+}\frac{\cos(\frac{1}{h})}{1}$$,

which does not exist.

We know that $$F'(0) = \lim_{h \to 0} \frac{F(h)}{h}$$. Therefore, the limit above should be equal to $0.$ However, the answer is that $F'(x)=0.$

What did I mess up?

This is a tricky problem and Spivak has given a hint which you have perhaps overlooked. Consider the function $g(x) =x^2\sin(1/x),g(0)=0$ then it can be proved that $$g'(x) =2x\sin(1/x)-\cos (1/x), g'(0)=0$$ so that $$g'(x) =h(x) - f(x) $$ where $f$ is the function in your question and $$h(x) =2x\sin (1/x),h(0)=0$$ We have now via Fundamental Theorem of Calculus $$g(x) =g(x) - g(0)=\int_{0} ^{x} g'(t) \, dt=\int_{0}^{x}h(t)\,dt-\int_{0}^{x}f(t)\,dt$$ or $$F(x) =\int_{0}^{x}h(t)\,dt-g(x)$$ and using Fundamental Theorem of Calculus again we get $$F'(0)=h(0)-g'(0)=0$$ because $h$ is continuous at $0$.

Edits made: Your wanting to use the L’Hospital’s rule is fair, but your conclusion is not valid

You do find a zero numerator and denominator. So you are right to try progress by differentiating both. Edit: thanks to what is pointed out in the comments, the derivative is indeed $f(h)$. However, the limit does not exist. So you cannot conclude anything from L’H rule (see the section “requirement that the limit exist” here for an example). In brief, “L’H computations lead to a no-limit situation” does not imply the limit does not exist.

As to why the derivative is zero, an outline: try writing the limit as $$\lim_{h \to 0} \frac{\int_0^h f}{h}=\lim_{M\to \infty} M\int_M^{\infty}\frac{\cos y}{y^2}dy $$

Now split the integral into intervals of length $2\pi$. In each interval, bound the positive part above (for example in $[2N\pi,(2N+1)\pi]$, we have $\frac{1}{y^2}\leq \frac{1}{(2N\pi)^2}$ and the negative part from below to show that M times the sum of integrals on these intervals is smaller than a constant times $1/M$. (There is a series sum involved). Reverse the bounds on the positive and negative parts to see that it is bounded below by a constant times $-1/M$. As $M \to \infty$, the limit is zero.

There might be a more elegant way to show the effect of the cancellation, but that’s all I could work out for now.