How to find the derivative of the following vectors and matrices
Solution 1:
A slightly different approach is to note that the function you want to differentiate can be written as $$ \phi = \| \mathbf{x} \otimes \mathbf{x} \|_2^2 = \| \mathbf{x} \mathbf{x}^T \|^2_F $$
The differential writes $$ d\phi = 2 \mathbf{x} \mathbf{x}^T:d(\mathbf{x} \mathbf{x}^T) = 4 \mathbf{x} \mathbf{x}^T \mathbf{x}:d\mathbf{x} $$ The gradient is thus $$ \frac{\partial \phi}{\partial \mathbf{x}} = 4 \| \mathbf{x} \|_2^2 \mathbf{x} $$
Solution 2:
$
\def\a{\alpha}\def\b{\beta}\def\l{\lambda}\def\o{{\tt1}}
\def\B{\Big}\def\L{\left}\def\R{\right}
\def\LR#1{\L(#1\R)}
\def\BR#1{\B(#1\B)}
\def\trace#1{\operatorname{Tr}\LR{#1}}
\def\v#1{\operatorname{vec}\LR{#1}}
\def\qiq{\quad\implies\quad}
\def\t{\otimes}
\def\p{\partial}
\def\grad#1#2{\frac{\p #1}{\p #2}}
\def\c#1{\color{red}{#1}}
$The Frobenius product is a concise notation for the trace
$$\eqalign{
A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij} \;=\; \trace{A^TB} \\
A:A &= \big\|A\big\|^2_F \\
}$$
This is also called the double-dot or double contraction product.
When applied to vectors $(n=\o)$ it reduces to the standard dot product.
The distributive rule for a mixed Kronecker-Frobenius product is
$$\eqalign{
(A\t B):(X\t Y) = (A:X)\t(B:Y) \\
}$$
First calculate the gradient of the simple function $$\eqalign{ \a &= \|x\|^2 = x:x \qiq \c{d\a = 2x:dx} \\ }$$ Then calculate the differential (and gradient) for the first function of interest $(\b)$. $$\eqalign{ \b &= \|x\t x\|^2 \\&= (x\t x):(x\t x) \\&= (x:x)\t(x:x) \\&= \a^2 \\ d\b &= 2\a\;\c{d\a} \\&= 2\a\,\LR{2x\,dx} \\&= 4\|x\|^2\,x\,dx \\ \grad{\b}{x} &= 4\|x\|^2\,x \\ }$$ The second function of interest $(y)$ can also make use of the simple $\a$ function. $$\eqalign{ \l &= \|x\| \\ \l^2 &= \a \qiq 2\l\,d\l = d\a = 2x^Tdx \\ d\l &= \l^{-1}x^Tdx \\ \\ y &= \frac{x}{\|x\|} = \l^{-1}x \\ dy &= \l^{-1}dx-x\l^{-2}\,\c{d\l} \\ &= \l^{-1}I\,dx-x\l^{-2}\,\c{\l^{-1}x^Tdx} \\ &= \l^{-1}\LR{I-yy^T}\,dx \\ \grad{y}{x} &= \l^{-1}\LR{I-yy^T} \\\\ }$$
NB: Your expansion of $(x\t x)$ is incorrect. What you have written is actually the matrix $xx^T$ which you must vectorize to recover the desired Kronecker product, i.e. $$(x\t x) = \v{xx^T}$$ However, there is another property of the Frobenius product $$\v{A}:\v{A} = A:A$$ which allows you to use your matrix without vectorizing it $$\eqalign{ \b &= \v{xx^T}:\v{xx^T} \\&= \LR{xx^T}:\LR{xx^T} \\ &= \trace{xx^Txx^T} \\&= \a^2 }$$