Frechet derivative of shift operator in $l_2$?

Solution 1:

If you indeed find extremely awkward proofs tempting, then go for it... but a proof of $$\lim_{h\to0} \frac{|\sum_{k = 1}^{+\infty}h_kh_{k+ 1}|}{\|h\|_{l_2}} = 0$$ need not be awkward. Since $2h_{k}h_{k+1}\le h_k^2+h_{k+1}^2 $, the numerator is at most $\|h\|_{l_2}^2$.

Can the first solution be somehow linked with the second?

In the second approach, instead of manipulating with a particular quadratic form, we work with a general one. It's like solving $x^2+5x+6$ with factorization trick versus deriving a formula for solution of $ax^2+bx+c=0$ and then just using it.

Why is it suddenly possible to apply matrix calculus to an operator which isn't working with finite spaces?

Because it's been proved to work. And if it wasn't, it should have been done. When a professor skips a proof, it's not as much magic as laziness or lack of time in lecture.

The proof isn't difficult, though: $$\left< A(x+h),x+h \right>-\left<Ax,x\right> = \left<Ax,h\right>+\left<Ah,x\right>+\left<Ah,h\right>$$ The second term on the right is $\left<A^*x,h\right>$, and the last one is bounded by $\|A\| \|h\|^2$.

Am I right that $D(DJ(u)) = DJ(u)$

No, the second derivative is never equal to the first derivative. They have different domains. The second derivative at $x$ takes two vectors as arguments, the first derivative takes only one. Correct formula: $$D^2J(u)(h,k) = \left<(A+A^*)h,k\right>$$ This is using the idea of second derivative as a bilinear form $Q$ on vectors $h,k$ such that $$J(u+h+k) - J(u) - DJ(u)(h+k) = Q(h,k)+o(\|h\|^2+\|k\|^2) $$ In your case the computation is simple because the $o$ term is actually zero.

Above, "bilinear form" is a linear map from $X$ to the space of linear functionals. So, the second derivative is a map from $X$ to $L(X\to L(X,\mathbb R))$, which happens to be constant in this case (there is no $u$ in the second derivative).