Could someone give a detailed (yet elementary) proof for Jensen's inequality?
I want to prove that
Suppose there is a function $f:[a,b] \to \mathbb R$, and there are $x_i \in [a,b], w_i \gt 0 $ for $i=1,\dots,n$ such that $\sum_{i=1}^nw_i=1$, then if the function is convex, the following inequality holds $$\sum_{i=1}^nw_if(x_i) \ge f(\sum_{i=1}^n w_ix_i)$$ and if the function is concave, flip the sign of inequality.
Please include (very) detailed explanation, because I do not have much mathematical maturity, and perhaps may not be able to understand what the equation defining convex means if the explanation is too concise.
There is a proof by induction that I read as a note in an open directory of university notes site for a related course, see [1]. I indicate a detailed outline:
I am assuming $\omega_i\geq 0$.
Show that case $n=1$ is trivial. What is the value of $\omega_{1}$ in $f(\omega_1 x_1)\leq \omega_1f(\omega_1)$?
Derive the case $n=2$ directly (trivially) from definition of your convex function. What is the relation between $\omega_1$ and $\omega_2$? Previous answer help you to understand why you can use your definition of convex function (if your definition of a convez function is written in term of $\lambda$ terms, ask you what is the relation between weigths $\omega_1$, $\omega_2$ and these $\lambda$ terms).
Write the statement for $N=k-1$, the so called inductive hypothesis.
Define the following modified weigths $\widehat\omega_i$ satisfying $(1-\omega_k)\cdot\widehat\omega_i=\omega_i$, for each $1\leq i\leq k-1$.
From $\sum_{i=1}^{k-1}\omega_i f(x_i)+\omega_k f(x_k)$ show that it is equal to $(1-\omega_k)\cdot\sum_{i=1}^{k-1}\widehat\omega_i f(x_i)+\omega_k f(x_k)$. Now we can use the inductive hypothesis. But before, recognize the purpose of this manipulation (of terms and weigths versus the condition of lambda terms in definition of a convex function) that we have made?
After the use of inductive hypothesis we obtain
$$\sum_{i=1}^{k-1}\omega_i f(x_i)+\omega_k f(x_k)\geq (1-\omega_k)f\left(\sum_{i=1}^{k-1} \widehat\omega_i x_i\right)+f(x_k)\omega_k$$ and this, by case 2 (basically the definition) is geatest or equal than $$f\left((1-\omega_k)\sum_{i=1}^{k-1} \widehat\omega_i x_i+\omega_k x_k \right)$$
And now go back first two steps, using the definition of new weights and the decomposition of the sum.
Sorry by my english. I hope this helps and there aren't missprints. After your computations, in a days you can read the reference. There are a nice graph too.
References:
[1] Konstantinos Derpanis, Jensen's Inequality (March 12, 2005), page 2.