Wronskian of vector
Solution 1:
The Wronskian of vector-valued functions $\{\mathbf{v}_1(x), \dots, \mathbf{v}_2(x)\}$ is defined as $$ W\{\mathbf{v}_1, \dots, \mathbf{v}_n\}(x) = \det\begin{bmatrix} \mathbf{v}_1(x) & \dots & \mathbf{v}_n(x) \end{bmatrix}. $$ I assume that $\mathbf{Y}_1$ is a solution to the planar system and you want to solve for the other solution $\mathbf{Y}_2$. Following your notation, let $\mathbf{Y}_2(x) = (ae^{rx}, be^{rx})^T$, where I assume that the coefficient matrix $A$ has distinct real eigenvalues. The Wronskian of $\mathbf{Y}_1, \mathbf{Y}_2$ is given by $$ W\{\mathbf{Y}_1, \mathbf{Y}_2\}(x) = be^{rx}e^{2x} + ae^{rx}e^{2x} = (a + b)e^{(2 + r)x}. $$ With the two given conditions on the Wronskian, we have $(a + b)e^{4 + 2r} = e^{3}$ and $(a + b)e^{2 + r} = e^2$. Equivalently, we have $(a + b)e^{2r} = e^{-1}$ and $(a + b)e^r = 1$. Solving these two equations, we get $r = -1$ and $a + b = e$.