Show that the change of variables transforms a linear system

You're on the right track, but what you've shown is that $$ \left\{ \begin{aligned} x' &= -4u-v \\ y' &= -6u-v \end{aligned} \right. $$ By subbing in expressions in $u$ and $v$ for $x$ and $y$, you can convert the right sides of these equations to the new variables. But you also need to convert the left sides (the derivatives), and for that you need to reverse that change of variables.


Details: using your favorite technique from linear algebra, you can find $$ \left\{ \begin{aligned} x &= 2u + v \\ y &= 3u + v \end{aligned} \right. \qquad\Longleftrightarrow\qquad \left\{ \begin{aligned} u &= -x + \phantom{2}y \\ v &= \hskip2.5pt 3x - 2y \end{aligned} \right. $$

Thus, \begin{align} u' &= (-x + y)' \\ &= -x' + y' \\ &= -(x-2y) + (3x-4y) \\ &= 2x - 2y \\ &= 2(2u+v) - 2(3u+v) \\ &= -2u \end{align}

I leave the analogous calculation that $v' = -v$ up to you.


If you squint really hard, you can see that what you are doing is diagonalizing a square matrix by converting to basis of eigenvalues: $P^{-1}AP = D$, i.e. $$ \begin{bmatrix} -1 & 1 \\ 3 & -2 \end{bmatrix} \begin{bmatrix} 1 & -2 \\ 3 & -4 \end{bmatrix} \begin{bmatrix} 2 & 1 \\ 3 & 1 \end{bmatrix} = \begin{bmatrix} -2 & 0 \\ 0 & -1 \end{bmatrix} $$