Is there a general formula for the derivative of $\exp(A(x))$ when $A(x)$ is a matrix?

It's easy for scalars, $(\exp(a(x)))' = a' e^a$. But can anything be said about matrices? Do $A(x)$ and $A'(x)$ commute such that $(\exp(A(x)))' = A' e^A = e^A A'$ or is this only a special case?

Tom Stephens linked to the exponential map, which states that

$ \frac{d}{dt}e^{X(t)} = \int\limits_0^1 e^{\alpha X(t)} \frac{dX(t)}{dt} e^{(1-\alpha) X(t)} d\alpha $

If $X(t)$ and $\frac{d}{dt}X(t)$ commute, the latter also commutes with $\exp(X(t))$ and then it simplifies into $ \frac{d}{dt}e^{X(t)} = \frac{d X(t)}{dt} e^{X(t)}$.

A counter-example is $$X(t) = \begin{pmatrix} \cos(t) & \sin(t) \\\ \sin(t) & -\cos(t) \end{pmatrix}$$ at $t=0$, i.e. $X(0) = \sigma_3, \dot X(0) = \sigma_1$ ($\sigma_i$ are the non-commuting Pauli Matrices)

The question comes down to computing what's called the "derivative of", the "differential of", or the "tangent map to", the exponential map from $M_n(\mathbb R)$ into itself at a given matrix $A$ (not necessarily the zero matrix). There is a classical formula for this. Here is the first reference I found: pages 1 and 2 of

http://www.math.columbia.edu/~woit/notes4.pdf

by Peter Woit. Here it is (with Peter Woit's notation) $$\exp_*(X)\ Y=\exp(X)\ \frac{1-e^{-ad(X)}}{ad(X)}\ Y.$$ Here is a reference for the Chain Rule:

http://en.wikipedia.org/wiki/Chain_rule#The_fundamental_chain_rule

It reads, in Peter Woit's notation and under appropriate assumptions,

$$(f\circ g)\_*(x)=f_*(g(x))\circ g_*(x).$$

[Thank you to KennyTM for having edited this formula.]

EDIT 1. Here are the two formulas written in another notation:

$$\exp'(X)=\exp(X)\ \frac{1-e^{-ad(X)}}{ad(X)}\quad,$$

$$(f\circ g)'(x)=f'(g(x))\circ g'(x).$$

EDIT 2. Here is another reference. This is a post by Akhil Mathew:

http://deltaepsilons.wordpress.com/2009/11/07/helgasons-formula-ii/