Solution 1:

Question statement transcribed:

Exercise 11. Suppose that $f:(a, b) \rightarrow \mathbb{R}$ is $k$ times differentiable, and $f^{\prime}\left(x_{0}\right)=\cdots=$ $f^{(k-1)}\left(x_{0}\right)=0, f^{(k)}\left(x_{0}\right) \neq 0 .$ Show that

(i). if $k$ is even then $f$ has a local max at $x_{0}$ if $f^{(k)}\left(x_{0}\right)<0$ and a local min if $f^{(k)}\left(x_{0}\right)>0$, and

(ii). if $k$ is odd then $f$ is strictly increasing in a neighbourhood of $x_{0}$ if $f^{(k)}\left(x_{0}\right)>0$ and strictly decreasing if $f^{(k)}\left(x_{0}\right)<0$

Hint: Use induction on $k$, applying the induction hypothesis to $f^{\prime}$, to prove both statements simultaneously.

Comment from Vityôk (who set a bounty):

@CalvinKhor, i wanted to know how to incorporate the remainder, and i don't understand the case 2. Why in the subcase 1 of case 2 we have: $f(x)-f(x_{0})=f^{(k)}\frac{(x-x_{0})^k}{n!}>0$? I don't understand why it is $>0$. – Vityôk

  1. It is worth noting that the question (ii) is wrong. Differentiable at a point with positive derivative implies increasing in neighborhood of point? gives a counterexample (everywhere differentiable function with $f'(0)>0$) for $k=1$ (odd) which is not monotonic near $0$. What you can prove is what the OP did (modulo incorporating the remainder).

  2. To "incorporate the remainder", one should apply a correct version of Taylor's theorem. There are many variants; since we only have $k$ times differentiability, we have to use the Peano remainder form which is the weakest kind*. It reads: for every $x_0$, and for every $k$, if $f$ is $k$ times differentiable, then there exists a function $r$ with $r(x)\to 0$ as $x\to x_0$ such that $$ f(x)= \sum_{n=0}^k \frac{f^{(n)}(x_0)(x-x_0)^n}{n!} + r(x)(x-x_0)^k$$

  3. Once you have this correctly stated, the rest of the proof is easily modified to be correct (after the question is corrected). Let me write explicitly write the subcase indicated by Vityôk, which is when $k$ is odd and $f^{(k)}(x_0)<0$. Taylor's theorem gives for any $x$, $$ f(x)-f(x_0)= \left[\frac{f^{(k)}(x_0)}{k!} + r(x)\right](x-x_0)^k$$ For $x$ sufficiently close to $x_0$, $\frac{f^{(k)}(x_0)}{k!} + r(x)<0$. When $x>x_0$, then $(x-x_0)^k>0$; product of negative and positive is negative. So $f(x)<f(x_0)$. Similarly for $x<x_0$. But note we cannot get monotonicity on an interval since we know nothing about $f^{(k)}$ at other points than $x_0$.


*$k+1$ times differentiable lets you use Cauchy and Lagrange forms; absolute continuity of $f^{(k)}$---which also follows from $k+1$ times differentiability---allows you the integral form. These imply continuity of the $k$th derivative which then in fact gives a result like (ii).