Transformation Matrix representing $D: P_2 \to P_2$ with respect to the basis $B$.
Solution 1:
Let $[D]$ denote the matrix representation of the operator $T$ with respect to the given basis $B$ (in both the domain and range space of $D$). Let $[v]_B$ denote the coordinates of the vector $v$ in the basis $B$. If $[D]$ is the matrix of $D$ (with respect to $B$), then that means $$ D([v]_B)=[D][v]_B. $$ That is, we want to find $[D]_B$ such that $$ D\left(\begin{bmatrix} a\\b\\c\end{bmatrix}_B\right)=[D]\begin{bmatrix} a\\b\\c\end{bmatrix}_B. $$ Expanding the left-hand side based on the definition of $D$, \begin{align} D\left(\begin{bmatrix} a\\b\\c\end{bmatrix}_B\right)&={d\over dt}(t\cdot (a\cdot 1+b\cdot(1+t)+c\cdot(1+t+t^2)))\\ &={d\over dt}((a+b+c)t+(b+c)t^2+ct^3)\\ &=a+b+c+2(b+c)t+3ct^2. \end{align} So we want to rewrite the last line above in terms of a matrix multiple of $\begin{bmatrix} a\\b\\c\end{bmatrix}_B$. That is, find $k_1,k_2,k_3$ such that $$ a+b+c+2(b+c)t+3ct^2=k_1\cdot 1+k_2(1+t)+k_3(1+t+t^2). $$ Equating like terms, we get \begin{align} a+b+c&=k_1+k_2+k_3\\ 2b+2c&=k_2+k_3\\ 3c&=k_3, \end{align} which has solution \begin{align} k_1&=a-b-c\\ k_2&=2b-c\\ k_3&=3c. \end{align} Thus, we now see how the operator $D$ acts on input vectors (in $B$ coordinates): $$ \begin{bmatrix} a\\b\\c\end{bmatrix}_B\overset{D}{\longmapsto}\begin{bmatrix} a-b-c\\2b-c\\3c\end{bmatrix}_B. $$ Therefore, the matrix of the transformation $D$ (with respect to the $B$ basis) is $$ [D]_B=\begin{bmatrix} 1 & -1 & -1\\ 0 & 2 & -1\\ 0 & 0 & 3\end{bmatrix} $$ since $$ D\left(\begin{bmatrix} a\\b\\c\end{bmatrix}_B\right)=\begin{bmatrix} 1 & -1 & -1\\ 0 & 2 & -1\\ 0 & 0 & 3\end{bmatrix}\begin{bmatrix} a\\b\\c\end{bmatrix}_B=\begin{bmatrix} a-b-c\\2b-c\\3c\end{bmatrix}_B. $$