Yes, it is an upper triangular matrix. If for instance,$$J=\begin{bmatrix}\lambda&1&0\\0&\lambda&1\\0&0&\lambda\end{bmatrix},$$then, since$$J=\lambda\operatorname{Id}+\begin{bmatrix}0&1&0\\0&0&1\\0&0&0\end{bmatrix}$$and these two matrices commute, then\begin{align}e^J&=\exp\left(\lambda\operatorname{Id}\right)\exp\left(\begin{bmatrix}0&1&0\\0&0&1\\0&0&0\end{bmatrix}\right)\\&=\begin{bmatrix}e^\lambda&0&0\\0&e^\lambda&0\\0&0&e^\lambda\end{bmatrix}\begin{bmatrix}1&1&\frac12\\0&1&1\\0&0&1\end{bmatrix}\\&=\begin{bmatrix}e^\lambda&e^\lambda&\frac{e^\lambda}2\\0&e^\lambda&e^\lambda\\0&0&e^\lambda\end{bmatrix}.\end{align}The general case is similar.