operator exponential

Yes, you can define an exponential of any linear BOUNDED operator by this series. If the operator is unbounded then it is not always possible.

The exponential series has a remarkably "ubiquitiuos" convergence. As soon as you have a $\mathbb Q$-algebra $M$ with a norm such that $||X Y||\le c\cdot ||X||\cdot ||Y||$ for some $c$, then $\exp(A)$ converges for all $A$ with respect to this norm. Hence if $M$ is complete, you indeed obtain an element of $M$. Moreover, if $AB=BA$ then $\exp(A+B)=\exp(A)\exp(B)$ holds.

There are even cases when the exponential series is useful even when division by $k!$ is undefined. One just has to be careful that $A$ must be nilpotent enough (i.e. $A^k=0$ for all $k$ for which divison by $k!$ is undefined)

As you have suggested, if $A$ is a linear operator then:

$$\exp A = I + A + \frac{1}{2}A^2 + \cdots + \frac{1}{k!}A^k + \cdots \, . $$

These are very common in physics. Here is a link to a PDF file.