How to find derivative and directional derivative of this map

Solution 1:

Let's write down the definitions.

\begin{equation} \begin{split} \partial_H f(A) & := \lim_{t\to 0}\dfrac{f(A+tH)-f(A)}{t}\\ & = \lim_{t\to 0} \dfrac{[A+tH]_{ij}^2-[A]_{ij}^2}{t}\\ & = \lim_{t\to 0} \dfrac{ 2 t[A]_{ij}[H]_{ij} +t^2[H]_{ij}^2}{t}(*)\\ & = 2[A]_{ij}[H]_{ij}. \end{split} \end{equation} My guess is that your issue was in the ($*$) step. Notice that the squaring is happening entry by entry, so matrix product never enters the picture.

If we identify $M(n,\mathbb{R})$ with $\mathbb{R}^{n\times n}$ as $$ A"="([A]_{11},\ldots, [A]_{1n},[A]_{21},\ldots, [A]_{2n},[A]_{31},\ldots, [ A]_{nn}), $$ then we can write $$f=(f_1,\ldots f_{n^2})= ([f]_{11},\ldots, [f]_{1n}, [f]_{21}, \ldots, [f]_{nn}).$$ The derivative of this map at $A$ is a linear map $Df(A): \mathbb{R}^{n\times n}\to \mathbb{R}^{n\times n}$, which can be written as an $n^2\times n^2$ matrix, whose $k$-th row is given by the gradient of $f_k$. For this notice that if $f_k=[f]_{ij}$ for some $1\leq i,j\leq n$, then $$ \partial_{lm} f_k(A) = \partial_{lm} [A]_{ij}^2 = \delta_{li}\delta_{mj} 2[A]_{ij}, $$ where $\partial_{lm}$ denotes the directional derivative in the direction of the matrix having only a 1 in the $lm$ entry, and zero everywhere else. $\delta$ denotes the Kronecker delta. This gives that the Jacobian matrix (with the above identification of the matrix space) is represented by a diagonal matrix (where you write down 2 times the matrix entries, read row by row, left to right, along the diagonal).