Solution to a system of ODE:s

Using the matrix exponential as integrating factor you get $$ (e^{-At}x(t))'=e^{-At}C $$ which can be integrated to (at first only for regular $A$, but in the end for all $A$) $$ e^{-At}x(t)-x(0)=-A^{-1}(e^{-At}-I)C \implies x(t)=e^{At}x(0)+t\phi_1(At)C $$ where $\phi_1$ is the matrix version of the function $\phi_1(z)=\frac{e^z-1}{z}$, continued with $\phi_1(0)=1$ as per its power series. Note that $\phi_1$, despite its singular definition, is an analytical or entire function, similar to the exponential.

These modified exponentials, matrix phi functions or whatever name they got in-between, $\phi_2(z)=\frac{e^z-1-z}{z^2}$ etc. occur also in exponential Runge-Kutta methods, thus they are also implemented in good numerical linear algebra libraries along the matrix exponential.