Gradient of $\sum_{i,j}^n A_{ij}x_i^TB^iC{B^j}^Tx_j$

Solution 1:

$$ f \left( {\bf x}_1, \dots, {\bf x}_n \right) := \sum_{i,j=1}^n a_{ij} {\bf x}_i^\top {\bf B}_i {\bf C} \, {\bf B}_j^\top {\bf x}_j = \sum_{i,j=1}^n {\bf x}_i^\top \underbrace{\left( a_{ij} {\bf B}_i {\bf C} \, {\bf B}_j^\top \right)}_{=: {\bf Q}_{ij}} {\bf x}_j = {\bf x}^\top {\bf Q} \, {\bf x}$$

where

$$ {\bf Q} := \mbox{diag} \left( {\bf B}_1, \dots, {\bf B}_n \right) \left( {\bf A} \otimes {\bf C} \right) \mbox{diag} \left( {\bf B}_1, \dots, {\bf B}_n \right)^\top $$

where $\otimes$ denotes the Kronecker product. Note that $\bf Q$ is symmetric. Thus,

$$ \nabla_{{\bf x}} f \left( {\bf x} \right) = 2 \, {\bf Q} \, {\bf x} $$